* Persistent state download improvements
1) Don't start over on restart
2) Download shards one at a time to reduce RAM usage
3) More logs
* Remove old peers from adnl stats
* getactorstats query for validator-engine-console
* celldb in-memory mode (--celldb-in-memory option)
* rldp2: bugfix - do not estimate speed while nothing is sent
* add simple ed25519 benchmark
* fix compilation errors of different platforms and move to c++20
* fix some warnings
* turn on TON_USE_ABSEIL for glibc 2.27 nix build
---------
Co-authored-by: birydrad <>
* Improve validator session stats
* Collator stats: block limits, number of processed external messages
* Collator and validator work time
* Last key block seqno
* Approvers and signers
* End validator session stats
This commit contains some parts of https://github.com/ton-blockchain/ton/tree/accelerator
This is auxiliary code that mostly does not change node behavior.
1) Semiprivate overlays and other improvements in overlays code
2) Rename actual_min_split -> monitor_min_split, fix building shard overlays
3) Loading block candidates by block id from DB, fix accept_block after validator restart
4) Cells: ProofStorageStat and changes in CellUsageTree
5) Remove some unused code, other minor changes
* Improve validator session stats
* Collator stats: block limits, number of processed external messages
* Collator and validator work time
* Last key block seqno
* Approvers and signers
* End validator session stats
* Set collator options from validator console
* Fix compilation error in manager-disk
* Defer all messages if out msg queue is too big
* Fix checking queue size in collator
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* Deferred messages and msg metadata
* Store out msg queue size in state
* Add checks for queue processing
1. Collator must process at least one message from AccountDispatchQueue (unless block is full)
2. The first message from a transaction is not counted, it cannot be deferred (unless AccountDispatchQueue is not empty)
* Return msg metadata from LS in listBlockTransactions[Ext]
* Enable new features by capabilities
* Changes in deferred messages
* Process deferred messages via new_msgs in collator
* Rework setting deferred_lt, bring back check_message_processing_order, check order of deferred_lt in validator
* Use have_unprocessed_account_dispatch_queue_ in collator
* Fix setting transaction lt for deferred messages
* Fix lite-client compilation error
* Changes in process_dispatch_queue, rename deferred_lt -> emitted_lt
* Fix compilation error
* Use uint64 for msg queue size
* Add liteServer.getBlockOutMsgQueueSize
* Fix compilation error
* Fix typos in comments
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* Private overlay for external messages
* Improve ext msg overlays
* Manage from validator console
* Bypass out queue size limit for high-priority messages
* Shuffle messages in get_external_messages
* Cleanup mempool when creating validator group
* Improve private overlays for externals
1. Allow using validator adnl ids in addition to fullnode ids
2. Set priority per sender, not per overlay
3. Require the same overlay name for all nodes
4. Enable lz4 in private block overlay
* Fix typo, add debug logs
* Enable lz4 in private block overlay by config
Change proto_version for lz4 in catchain overlays to 4
* Add logs for broadcasts in fullnode
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* Fix "sgn" in fift; fix marking infinite loops as noterurn in func
* TON-P1-1: Remove unused catchain queries
* TON-P1-15: Avoid synchronization with self
* TON-P1-3, TON-P1-17: Disallow more than one candidate per src per round (to prevent flood), add checks to process_broadcast
* TON-P1-10: Fix fast/slow attempts
* TON-P1-14: Add named constants
* TON-P1-18, TON-P1-19: Alloc temporary memory in the same way as persistent memory
* TON-P1-20: Add comment to choose_blocks_to_approve
* TON-P1-16: Avoid creating two catchain blocks on need_new_block
* TON-P1-8: Add some validation to validator-engine parameters
* TON-P1-6: Don't allow sending the same block many times
Many requests for the same block are not unusual (however, there's no need to answer them all)
* TON-P1-2: Enable prohibiting dependencies from blamed nodes (2.7.5 in CatChain doc), fix processing blame proofs
* Best practices
bp-6: Fix incorrect warning
bp-7: Remove unused code
bp-8: Bring back PerfWarningTimer logging (only when no callback)
bp-9: Remove unnecessary condition
bp-11: Remove commented-out code
bp-13: Divide code in validator-session-state
Adherence to Specification: Fix typo
* Fix error handling in Torrent.cpp, improve choosing peers for upload
* Various improvements in storage daemon
"get-pieces-info"
Store "added at"
Improve calculating up/down speed
Improve TL protocol for future compatibility
Remove empty directories on "--remove-files"
Better windows support
Debug logs in PeerActor
More restrictions on TorrentInfo
Bugfixes
* Global speed limits for download and upload
+bugfix
* Reset download/upload speed on changing settings or completion
* Exclude some system files in TorrentCreator
* Option "--disable-ext-msg-broadcast"
* "Get shard out queue size" query
* Move disabling ext msg broadcasts from command-line arguments to config
* Fix compilation error
* Asynchronous store_cell and gc in celldb
* Make GC in celldb work evenly over time
* Increase timeouts for downloading blocks
* Reuse blocks from previous rounds in validator session
* Use Rldp2 in FullNode for downloading persistent states and archives
* Improve logs in download-archive-slice and download-state
* Decrease delay between serializing shards
* Make CellDbIn::load_cell synchronous to avoid interfering with store_cell
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* Rename chunk to piece in MerkleTree for consistency
* Refactor PeerManager
* Make PeerState thread-safe
* Download torrent by hash
* First version of storage daemon
* Download torrents partially
* Improve storing and loading torrent state in DB
* Rewrite MerkleTree
* "Remove torrent" in storage daemon
* Process errors, fix bugs in storage
* Move TonlibClientWrapper from rldp-http-proxy to tonlib
* Initial version of storage provider
* Move interaction with contracts to smc-util
* Improve TonlibClientWrapper interface
* Various improvements in storage provider
* Fix TorrentCreator.cpp
* Improve interface for partial download
* Client mode in storage-daemon
* Improve interface of storage-daemon-cli
* Fix calculating speed, show peers in storage-daemon
* Use permanent adnl id in storage daemon
* Fix sending large "storage.addUpdate" messages
* Improve printing torrents in cli
* Update tlo
* Fix RldpSender::on_ack
* Update storage provider
* Add "address" parameter to get-provider-params
* Allow client to close storage contract
* Limit torrent description
* Add more logs to storage provider
* smc.forget tonlib method
* Use smc.forget in storage daemon
* Optimize sending messages in smc-util.cpp
* Fix verbosity, remove excessive logs
* Json output in storage-daemon-cli
* Update storage provider contracts
* Fix rldp2 acks
* Change verbosity of logs in rldp2
* Update help and output of commands and in storage-daemon-cli
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* Add getperfwarningtimeraverage query for validator-engine-console
* Fix for getperfwarningtimeraverage query
* getperftimerstats
* Add history max size: 1 hour
* PerfWarningTimer: move callback instead of copy
* PerfWarningTimer: fix move constructor bug
* PerfWarningTimer: fix bug: lifetime of the callback was greater than lifetime of the local variable 'manager' captured by reference
* Fix validate-query.cpp: 'manager' used after it was moved
* PerfWarningTimer: remove logs
* getperftimerstats: write to json file
* getperftimerstatsjson
Co-authored-by: legaii <jgates.ardux@gmail.com>
Co-authored-by: Ivan Siomash <106972486+legaii@users.noreply.github.com>
