SpyCheese
1b4fb42859
End validator session stats
2024-08-19 19:55:11 +03:00
SpyCheese
0ca022cc72
Improve validator session stats
...
* Collator stats: block limits, number of processed external messages
* Collator and validator work time
* Last key block seqno
* Approvers and signers
2024-08-19 18:35:50 +03:00
SpyCheese
7ecc31bc96
Use block data from block candidate in AcceptBlockQuery
2024-08-09 15:49:47 +03:00
SpyCheese
dcb409462a
Cleanup mempool on collators
2024-08-09 09:20:42 +03:00
SpyCheese
4a6479af57
Merge branch 'testnet' into accelerator
2024-08-07 15:28:31 +03:00
EmelyanenkoK
e985ac0358
Add --fast-state-serializer flag and responding to AdnlMessageCreateChannel with Nop ( #1096 )
...
* Add --fast-state-serializer and tools for jemalloc
* Disable fast state serializer by default unless RAM is >= 90GB
* Print jemalloc stats once a minute
* Dump jemalloc profile on request
* Respond to AdnlMessageCreateChannel with Nop
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-08-07 14:25:45 +03:00
SpyCheese
70322f58c1
Use candidates from DB in wait block data
2024-08-01 13:57:32 +03:00
SpyCheese
d4da0ace74
Merge branch 'testnet' into accelerator
2024-07-29 15:28:00 +03:00
EmelyanenkoK
b3828f8eb2
Print oldest celldb snapshot to stats ( #1078 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-07-26 22:11:58 +03:00
SpyCheese
0e7374610d
Improve block broadcasts
2024-07-25 16:33:19 +03:00
SpyCheese
4c8d25ac1b
Save and send block candidates in collator nodes
2024-07-24 20:04:24 +03:00
SpyCheese
4b68c7b165
Merge branch 'testnet' into accelerator
2024-07-23 18:07:58 +03:00
SpyCheese
1b9372804f
Make block overloaded if dispatch queue processing limit is reached ( #1070 )
2024-07-23 16:47:28 +03:00
SpyCheese
58ca7b44ff
Add null check in manager.cpp ( #1069 )
2024-07-23 15:17:36 +03:00
SpyCheese
5743dbf335
Changes in fast sync overlays
...
Join all overlays (not only monitoring shards) in order to be able to send broadcasts to other overlays
2024-07-18 12:43:55 +03:00
SpyCheese
36b98e8aa0
Fix compilation error in shard-client.cpp
2024-07-17 16:52:56 +03:00
SpyCheese
d81b69e38b
Use previous and next validators in fast sync overlays
2024-07-17 15:34:45 +03:00
SpyCheese
3ab7cc3c73
Merge branch 'testnet' into accelerator
2024-07-17 15:33:47 +03:00
SpyCheese
b9e89d4c66
"getcollatoroptionsjson" command in validator console ( #1059 )
...
* "getcollatoroptionsjson" command in validator console
* Improve state serializer
Use previous persistent state to speed up reading
2024-07-17 15:10:55 +03:00
SpyCheese
3a8ef60bbb
Fast-sync overlays
...
* Semiprivate overlays
* Fast sync overlays in full-node
* Adjust shard overlays
2024-07-13 15:40:26 +03:00
SpyCheese
ac3b8d6088
Merge branch 'testnet' into accelerator
2024-07-13 15:38:56 +03:00
EmelyanenkoK
00cd053dbc
Fix processing dispatch queue ( #1057 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-07-13 14:43:04 +03:00
SpyCheese
d74d3f1fc2
Merge branch 'testnet' into accelerator
2024-07-10 15:35:54 +03:00
EmelyanenkoK
57f95cc282
Add collator options ( #1052 )
...
* Set collator options from validator console
* Fix compilation error in manager-disk
* Defer all messages if out msg queue is too big
* Fix checking queue size in collator
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-07-10 14:58:13 +03:00
SpyCheese
c54f095c1b
Don't use user-provided fees for internal messages ( #1050 )
2024-07-10 09:17:37 +03:00
SpyCheese
7f263b8b11
Don't request queue size if not needed in validate-query
2024-06-28 12:46:08 +03:00
SpyCheese
172e404426
Merge branch 'testnet' into accelerator
2024-06-28 11:55:12 +03:00
tom
fae7763ec7
Fix typos ( #998 )
2024-06-27 16:46:37 +03:00
EmelyanenkoK
0daee1d887
Merge message dispatch queue ( #1030 )
...
* Deferred messages and msg metadata
* Store out msg queue size in state
* Add checks for queue processing
1. Collator must process at least one message from AccountDispatchQueue (unless block is full)
2. The first message from a transaction is not counted, it cannot be deferred (unless AccountDispatchQueue is not empty)
* Return msg metadata from LS in listBlockTransactions[Ext]
* Enable new features by capabilities
* Changes in deferred messages
* Process deferred messages via new_msgs in collator
* Rework setting deferred_lt, bring back check_message_processing_order, check order of deferred_lt in validator
* Use have_unprocessed_account_dispatch_queue_ in collator
* Fix setting transaction lt for deferred messages
* Fix lite-client compilation error
* Changes in process_dispatch_queue, rename deferred_lt -> emitted_lt
* Fix compilation error
* Use uint64 for msg queue size
* Add liteServer.getBlockOutMsgQueueSize
* Fix compilation error
* Fix typos in comments
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-06-27 16:12:23 +03:00
SpyCheese
90d2edf535
Improve CollatorNode
...
* Keep track of validator groups
* Pre-generate shard blocks
2024-06-25 14:06:15 +03:00
SpyCheese
3695bf0797
Merge branch 'testnet' into accelerator
2024-06-12 18:53:54 +03:00
EmelyanenkoK
bd23029d0a
Soft send message validation ( #1021 )
...
* check mode on invalid action_send_msg
* Fix random seed generation
* Explicitly skip invalid actions
* Count skipped valid messages, rename cfg option to message_skip_enabled
* Allow unfreeze via external messages
* Detect and handle bounce_on_fail mode for invalid messages
* Fix codestyle
* Adjust doc
2024-06-11 15:08:08 +03:00
SpyCheese
38ab70c037
Changes in validation
...
* Configure collator list in validator-console
* Remove "lite-validator" flags
* Better compatibility in validate-query.cpp
2024-06-11 13:39:14 +03:00
SpyCheese
b8999be2c0
New archive format and importing archive slices
2024-06-06 18:31:28 +03:00
SpyCheese
eb4c876f22
Merge branch 'testnet' into block-generation
2024-06-04 16:49:47 +03:00
EmelyanenkoK
f9b6d21620
Update validator list on each keyblock
2024-06-04 13:09:06 +03:00
EmelyanenkoK
0301e9fbb7
Fix loading key block config in FullNodeImpl ( #1013 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-06-04 13:03:58 +03:00
EmelyanenkoK
3d7e85bafd
Rollback celldb default flag values and decrease default TTLs ( #1012 )
...
* Rollback celldb default flag values and decrease default TTLs
* Fix description
2024-06-02 13:41:02 +03:00
EmelyanenkoK
74801d00b8
Command to disable state serializer ( #1011 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-31 20:50:31 +03:00
SpyCheese
c129a784c6
Prepare transition between old and new private overlays
2024-05-30 18:57:50 +03:00
SpyCheese
1ee9e47007
Merge branch testnet into block-generation
2024-05-30 11:21:39 +03:00
EmelyanenkoK
8a4d44de1e
Compress block candidate broadcasts ( #1007 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-29 15:10:42 +03:00
EmelyanenkoK
ceefac74cf
Limit checked external messages per address ( #1005 )
...
* Limit checked external messages per address
* Change max_ext_msg_per_addr_time_window; cleanup mempool by timer
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-28 13:31:13 +03:00
EmelyanenkoK
539d5dd2de
Add candidates cache ( #1000 )
...
* Broadcast shardchain block candidates in private overlays, generate proof links from candidates
* Disable shardchain block broadcasts in private overlays
* Send block candidate broadcasts to custom overlays, allow non-validators to receive candidates
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-24 09:58:07 +03:00
SpyCheese
eb41e1a10e
Merge branch 'testnet' into block-generation
2024-05-22 11:00:24 +03:00
EmelyanenkoK
561f342639
Add --celldb-direct-io and --celldb-preload-all ( #993 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-14 16:05:29 +03:00
SpyCheese
172c16ca2e
Merge branch 'testnet' into block-generation
2024-05-13 16:38:48 +03:00
EmelyanenkoK
816dd9cf2d
Add option --catchain-max-block-delay ( #990 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-13 14:55:32 +03:00
EmelyanenkoK
c7fd75ce56
Fix creating rocksdb cache ( #989 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-13 14:54:57 +03:00
EmelyanenkoK
1433f23eff
Add option --celldb-cache-size ( #988 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-13 12:48:18 +03:00
EmelyanenkoK
d5c09936cf
Block broadcasts in custom overlays ( #986 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-10 17:04:49 +03:00
EmelyanenkoK
6fb2019a4f
Improve validator session stats ( #982 )
...
* Add list of validators
* Fix producer stats
* Make round and stats timestamps more meaningful
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-08 17:41:15 +03:00
EmelyanenkoK
037053fffe
More verbose state serializer logs ( #976 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-29 16:33:10 +03:00
SpyCheese
a745f6f679
Merge branch 'testnet' into block-generation
2024-04-26 12:08:28 +03:00
EmelyanenkoK
9a543c6b28
Add more info to session stats ( #973 )
...
* Improve validator session stats
* Improve validator session stats
Add got_submit_at, collation_cached, validation_cached.
Fix stats cleanup.
* Fix setting timestamp for cached blocks
* Add serialize/deserialize time, serialized size to validator session stats, fix setting is_accepted
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-25 18:47:15 +03:00
EmelyanenkoK
d2b012c883
Add write time stats to celldb/db_stats.txt ( #972 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-25 18:45:39 +03:00
SpyCheese
1b7f7aed9f
Merge branch 'refs/heads/testnet' into block-generation
2024-04-15 14:09:19 +03:00
EmelyanenkoK
190aa6bd4a
Use shard client timestamp to determine expiration of temp archives ( #965 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-15 13:44:12 +03:00
SpyCheese
8143eb074e
Merge branch 'testnet' into block-generation
2024-04-08 14:44:28 +03:00
EmelyanenkoK
a2bd695b89
Fix prepare_vm_c7 ( #958 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-08 14:29:54 +03:00
EmelyanenkoK
b8111d8b5b
Fix setting due_payment in storage phase ( #957 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-08 11:15:52 +03:00
EmelyanenkoK
cc4244f5a4
Fix setting inited_ in FullNodeCustomOverlay ( #954 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-03 12:11:07 +03:00
EmelyanenkoK
8390d887d3
Set temp packages ttl to 1h ( #953 )
...
This leaves only 3 last temp packages
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-02 11:46:40 +03:00
SpyCheese
1fda6902a9
Merge branch 'testnet' into block-generation
2024-04-01 18:32:44 +03:00
EmelyanenkoK
f7907bdd58
Rename private ext msg overlays to custom overlays, change interface a bit ( #952 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-01 18:24:08 +03:00
SpyCheese
4916e4847a
Merge branch 'testnet' into block-generation
2024-04-01 17:28:23 +03:00
EmelyanenkoK
0434eadc1f
Add custom overlays for external messages ( #949 )
...
* Private overlay for external messages
* Improve ext msg overlays
* Manage from validator console
* Bypass out queue size limit for high-priority messages
* Shuffle messages in get_external_messages
* Cleanup mempool when creating validator group
* Improve private overlays for externals
1. Allow using validator adnl ids in addition to fullnode ids
2. Set priority per sender, not per overlay
3. Require the same overlay name for all nodes
4. Enable lz4 in private block overlay
* Fix typo, add debug logs
* Enable lz4 in private block overlay by config
Change proto_version for lz4 in catchain overlays to 4
* Add logs for broadcasts in fullnode
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-01 16:44:08 +03:00
Marat
46ca0e6014
Check pack_statistics for nullptr ( #950 )
2024-03-29 22:01:53 +03:00
SpyCheese
f5cedc3b6e
Enable compression in private overlays v2 and in collator node
2024-03-29 15:04:07 +03:00
SpyCheese
8385336eab
Merge branch 'testnet' into block-generation
2024-03-28 18:12:21 +03:00
Marat
10487b1c71
Collect statistics for .pack files ( #944 )
...
* Statistics for .pack files
* optimizations
* fix typo
* fix erasing packages
2024-03-27 14:23:11 +03:00
SpyCheese
036ce69786
Use compression in new private overlays
2024-03-26 16:22:52 +03:00
SpyCheese
7999a7e2c1
Merge branch 'testnet' into block-generation
2024-03-26 16:19:34 +03:00
EmelyanenkoK
b07614335c
liteServer.getOutMsgQueueSizes method ( #943 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-03-26 15:55:58 +03:00
EmelyanenkoK
0bcebe8a0e
Exp/compress candidates ( #942 )
...
* Compress block candidates in validator-session
* Compress blocks in full-node (disabled for now)
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-03-26 14:52:46 +03:00
EmelyanenkoK
9452c367e4
LS queries to nonfinal blocks ( #941 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-03-26 14:51:06 +03:00
SpyCheese
3190c67f18
Simplify getCapabilities in fullnode
2024-03-26 11:42:30 +03:00
SpyCheese
2ee7cbc106
Merge branch 'testnet' into block-generation
...
# Conflicts:
# tonlib/tonlib/TonlibClient.cpp
# validator-engine/validator-engine.hpp
# validator/validator-options.hpp
# validator/validator.h
2024-03-25 18:40:47 +03:00
Marat
0feaaf591c
Mode 644 for db_stats.txt ( #938 )
2024-03-20 15:57:20 +03:00
Marat
7a6bfa7e7a
Save rocksdb statistics to file every minute ( #932 )
...
* Save rocksdb statistics to file every minute
* Add flag to disable collecting rocksdb statistics
2024-03-20 14:21:40 +03:00
EmelyanenkoK
0a82a19313
Don't serialize expired persistent states ( #935 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-03-19 15:43:03 +03:00
EmelyanenkoK
f1592641de
Tonlib improvement fixes ( #934 )
...
* 3.6. Fix directory traversal in KeyValueDir
* 3.9. Fix buffer_to_hex reversing nibbles
* 3.5. Fix error handling at blocks.getBlockHeader
* 3.11. Fix query.forget
* 3.12. Fix error handling in RemoteRunSmcMethod
* 4.1. Delete unused files
* 3.10. Use named constants instead hardcoded constants
* 3.4. Fix response block header verification
* 3.1. Check proof of blocks.getShards response
* fix td::buffer_to_hex + test
* 3.2. Add proof check for listBlockTransactions response in RunEmulator actor
* 3.8. Add proof checking for getLibraries method
* fix regression tests
* 3.3 Add proof checking for lookupBlock method
* Add publishers to proof of getLibrariesWithProof response (#25 )
* fix missing return, fix requesting mc block
* Fix requesting lookupBlock with client mc blk == mc ref block
* Fix duplicating lib data in proof and data, add mode 2 for not including the data
* Migration of LastBlockStorage with fixed td::buffer_to_hex
---------
Co-authored-by: ms <dungeon666master@protonmail.com>
Co-authored-by: Marat <98183742+dungeon-master-666@users.noreply.github.com>
2024-03-19 15:31:29 +03:00
Marat
69de1cb621
Fix linker error: undefined reference to LiteServerCacheImpl::MAX_CACHE_SIZE ( #930 )
...
Co-authored-by: EmelyanenkoK <emelyanenko.kirill@gmail.com>
2024-03-19 15:24:15 +03:00
EmelyanenkoK
fc69377f9c
Fix running VM in liteserver and tonlib ( #933 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-03-19 15:23:34 +03:00
SpyCheese
e216651112
Merge branch 'testnet' into block-generation
...
# Conflicts:
# crypto/block/mc-config.cpp
# crypto/block/mc-config.h
2024-03-11 18:08:08 +03:00
EmelyanenkoK
9d05696452
Add infrastructure for precompiled smartcontracts ( #927 )
...
* Utils for writing precompiled contracts
* Precompiled contracts in config, override gas_usage for them
* Add base class for precompiled contracts
* Improve utils for precompiled smc
* Implement GETPRECOMPILEDGAS
* Enable precompiles by flag
* Process null data in PrecompiledSmartContract
* Fix ton_block wasm build
* Fix vm::util::store_(u)long
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-03-05 16:54:49 +03:00
SpyCheese
6c680dbc4e
Cleanup code in validator-group.cpp (after merge)
2024-02-22 15:00:41 +03:00
SpyCheese
0fb781b4c3
Improve queue cleanup (after merge)
2024-02-22 14:55:11 +03:00
SpyCheese
28699654fb
New private block overlays
2024-02-21 18:38:43 +03:00
EmelyanenkoK
310dd6dec1
Fix fetching mc config for runSmcMethod, fix caching in LS ( #915 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-02-21 16:13:55 +03:00
EmelyanenkoK
c7302bc4a3
Fix checking ext message broadcasts ( #912 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-02-19 19:01:50 +03:00
SpyCheese
e575d27302
Remove unused priority_broadcast_receivers
2024-02-19 14:21:15 +03:00
SpyCheese
747b24aa18
Fix QueueSizeCounter and ValidatorGroup after merge
2024-02-19 13:10:47 +03:00
SpyCheese
a2eb3f3631
Merge branch 'testnet' into block-generation
2024-02-16 16:38:47 +03:00
EmelyanenkoK
af71dae31b
Ls run method ( #906 )
...
* Improve LS runSmcMethod
* Full c7 and libraries
* Don't request c7 (mode 8) in lite-client
* Prioritize remote result in lite-client runmethod (but not runmethodfull)
* Return simplified c7 from runSmgMethod, add flag to return full
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-02-16 16:30:43 +03:00
EmelyanenkoK
eb4831d7d6
Add --archive-preload-period ( #904 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-02-15 12:03:57 +03:00
EmelyanenkoK
4d39772e40
Drop only accepted duplicate ext messages, don't cache errors ( #902 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-02-12 10:42:02 +03:00
EmelyanenkoK
f344aa46c3
Drop duplicate ext msg broadcasts ( #894 )
...
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-02-09 15:28:23 +03:00
EmelyanenkoK
79c48ebbba
Add cache for some LS requests ( #893 )
...
* Cache runSmcMethod queries to LS
* Drop duplicate sendMessage
* Drop sendMessage cache once a minute
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-02-07 15:45:51 +03:00
EmelyanenkoK
12c1b1a2e6
Limit file descriptors num by adding archive slice lru ( #892 )
...
* --max-archive-fd option limits open files in archive manager
* Don't close the latest archives + bugfix
* Delete temp packages early
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-02-07 14:56:37 +03:00
