1
0
Fork 0
mirror of https://github.com/ton-blockchain/ton synced 2025-02-15 04:32:21 +00:00
Commit graph

47 commits

Author SHA1 Message Date
SpyCheese
70322f58c1 Use candidates from DB in wait block data 2024-08-01 13:57:32 +03:00
SpyCheese
0e7374610d Improve block broadcasts 2024-07-25 16:33:19 +03:00
SpyCheese
3ab7cc3c73 Merge branch 'testnet' into accelerator 2024-07-17 15:33:47 +03:00
SpyCheese
b9e89d4c66
"getcollatoroptionsjson" command in validator console (#1059)
* "getcollatoroptionsjson" command in validator console

* Improve state serializer

Use previous persistent state to speed up reading
2024-07-17 15:10:55 +03:00
SpyCheese
3a8ef60bbb Fast-sync overlays
* Semiprivate overlays
* Fast sync overlays in full-node
* Adjust shard overlays
2024-07-13 15:40:26 +03:00
SpyCheese
172e404426 Merge branch 'testnet' into accelerator 2024-06-28 11:55:12 +03:00
EmelyanenkoK
0daee1d887
Merge message dispatch queue (#1030)
* Deferred messages and msg metadata

* Store out msg queue size in state

* Add checks for queue processing

1. Collator must process at least one message from AccountDispatchQueue (unless block is full)
2. The first message from a transaction is not counted, it cannot be deferred (unless AccountDispatchQueue is not empty)

* Return msg metadata from LS in listBlockTransactions[Ext]

* Enable new features by capabilities

* Changes in deferred messages

* Process deferred messages via new_msgs in collator
* Rework setting deferred_lt, bring back check_message_processing_order, check order of deferred_lt in validator

* Use have_unprocessed_account_dispatch_queue_ in collator

* Fix setting transaction lt for deferred messages

* Fix lite-client compilation error

* Changes in process_dispatch_queue, rename deferred_lt -> emitted_lt

* Fix compilation error

* Use uint64 for msg queue size

* Add liteServer.getBlockOutMsgQueueSize

* Fix compilation error

* Fix typos in comments

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-06-27 16:12:23 +03:00
SpyCheese
b8999be2c0 New archive format and importing archive slices 2024-06-06 18:31:28 +03:00
SpyCheese
eb4c876f22 Merge branch 'testnet' into block-generation 2024-06-04 16:49:47 +03:00
EmelyanenkoK
74801d00b8
Command to disable state serializer (#1011)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-31 20:50:31 +03:00
SpyCheese
1ee9e47007 Merge branch testnet into block-generation 2024-05-30 11:21:39 +03:00
EmelyanenkoK
539d5dd2de
Add candidates cache (#1000)
* Broadcast shardchain block candidates in private overlays, generate proof links from candidates

* Disable shardchain block broadcasts in private overlays

* Send block candidate broadcasts to custom overlays, allow non-validators to receive candidates

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-24 09:58:07 +03:00
SpyCheese
172c16ca2e Merge branch 'testnet' into block-generation 2024-05-13 16:38:48 +03:00
EmelyanenkoK
d5c09936cf
Block broadcasts in custom overlays (#986)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-10 17:04:49 +03:00
EmelyanenkoK
6fb2019a4f
Improve validator session stats (#982)
* Add list of validators
* Fix producer stats
* Make round and stats timestamps more meaningful

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-08 17:41:15 +03:00
SpyCheese
4916e4847a Merge branch 'testnet' into block-generation 2024-04-01 17:28:23 +03:00
EmelyanenkoK
0434eadc1f
Add custom overlays for external messages (#949)
* Private overlay for external messages

* Improve ext msg overlays

* Manage from validator console
* Bypass out queue size limit for high-priority messages
* Shuffle messages in get_external_messages

* Cleanup mempool when creating validator group

* Improve private overlays for externals

1. Allow using validator adnl ids in addition to fullnode ids
2. Set priority per sender, not per overlay
3. Require the same overlay name for all nodes
4. Enable lz4 in private block overlay

* Fix typo, add debug logs

* Enable lz4 in private block overlay by config

Change proto_version for lz4 in catchain overlays to 4

* Add logs for broadcasts in fullnode

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-01 16:44:08 +03:00
SpyCheese
7999a7e2c1 Merge branch 'testnet' into block-generation 2024-03-26 16:19:34 +03:00
EmelyanenkoK
9452c367e4
LS queries to nonfinal blocks (#941)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-03-26 14:51:06 +03:00
SpyCheese
747b24aa18 Fix QueueSizeCounter and ValidatorGroup after merge 2024-02-19 13:10:47 +03:00
SpyCheese
f4fd3ff3be Merge branch 'testnet' into block-generation 2024-02-01 19:29:25 +03:00
EmelyanenkoK
b3be4283ff
Better error messages from LS for missing blocks (#837)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2023-12-25 09:30:51 +03:00
EmelyanenkoK
5e6b67ae96
Improve handling outbound message queues (#825)
* Improve handling outbound message queues

* Cleanup queue faster
* Calculate queue sizes in background
* Force or limit split/merge depending on queue size

* Increase validate_ref limit for transaction

* Add all changes of public libraries to block size estimation

* Don't crash on timeout in GC

* Don't import external messages when queue is too big

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2023-12-13 12:57:34 +03:00
SpyCheese
1e3a12259b Optimize masterchain collation
Use only shard blocks with ready msg queues
2023-08-30 20:14:19 +03:00
SpyCheese
5c02459fd8 Optimize importing out queues 2023-07-31 18:12:09 +03:00
SpyCheese
f1e62d0075 Move msg queue limits to config 2023-07-24 15:29:55 +03:00
SpyCheese
d652f7d706 Merge branch 'testnet' into block-generation
# Conflicts:
#	adnl/adnl-query.cpp
#	crypto/block/block.tlb
#	crypto/block/mc-config.h
#	lite-client/lite-client.cpp
#	overlay/overlay-manager.h
#	overlay/overlay-peers.cpp
#	overlay/overlay.cpp
#	overlay/overlay.h
#	overlay/overlay.hpp
#	overlay/overlays.h
#	rldp-http-proxy/DNSResolver.cpp
#	rldp-http-proxy/rldp-http-proxy.cpp
#	tl/generate/scheme/ton_api.tl
#	tl/generate/scheme/ton_api.tlo
#	tl/generate/scheme/tonlib_api.tlo
#	ton/ton-types.h
#	tonlib/tonlib/ExtClient.cpp
#	tonlib/tonlib/ExtClient.h
#	tonlib/tonlib/ExtClientLazy.cpp
#	tonlib/tonlib/ExtClientOutbound.h
#	tonlib/tonlib/ExtClientRaw.h
#	tonlib/tonlib/TonlibClient.cpp
#	tonlib/tonlib/TonlibClient.h
#	tonlib/tonlib/tonlib-cli.cpp
#	validator/impl/collator.cpp
#	validator/impl/validate-query.cpp
#	validator/impl/validate-query.hpp
#	validator/manager.cpp
#	validator/state-serializer.cpp
#	validator/state-serializer.hpp
#	validator/validator-group.cpp
#	validator/validator-group.hpp
#	validator/validator.h
2022-12-22 23:37:35 +03:00
EmelyanenkoK
1b106ef270
Use state with applied shards in liteserver (#553)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2022-12-13 17:46:54 +03:00
SpyCheese
d4339b839c Add "delshard", "delcollator" to validator-engine-console 2022-10-17 13:24:59 +03:00
SpyCheese
1d42c38122
Move some constants to global config (#484) 2022-10-10 17:13:21 +03:00
SpyCheese
636348ffc2 Merge branch 'testnet' into block-generation
# Conflicts:
#	overlay/overlay-fec-broadcast.cpp
#	overlay/overlay.hpp
#	tl/generate/scheme/ton_api.tl
#	tl/generate/scheme/ton_api.tlo
#	validator-engine-console/validator-engine-console-query.cpp
#	validator-engine-console/validator-engine-console-query.h
#	validator-engine-console/validator-engine-console.cpp
#	validator-engine/validator-engine.cpp
#	validator-engine/validator-engine.hpp
#	validator/downloaders/wait-block-state.hpp
#	validator/impl/validate-query.cpp
#	validator/validator.h
2022-10-06 13:46:25 +03:00
EmelyanenkoK
8329a58994
Add GetPerfTimerStats (#451)
* Add getperfwarningtimeraverage query for validator-engine-console

* Fix for getperfwarningtimeraverage query

* getperftimerstats

* Add history max size: 1 hour

* PerfWarningTimer: move callback instead of copy

* PerfWarningTimer: fix move constructor bug

* PerfWarningTimer: fix bug: lifetime of the callback was greater than lifetime of the local variable 'manager' captured by reference

* Fix validate-query.cpp: 'manager' used after it was moved

* PerfWarningTimer: remove logs

* getperftimerstats: write to json file

* getperftimerstatsjson

Co-authored-by: legaii <jgates.ardux@gmail.com>
Co-authored-by: Ivan Siomash <106972486+legaii@users.noreply.github.com>
2022-09-10 12:57:13 +03:00
SpyCheese
ea7a5776fe Download persistent states when syncing new shards 2022-08-19 18:24:18 +03:00
SpyCheese
be2169e523 Reload validator options when needed 2022-08-19 18:24:17 +03:00
SpyCheese
662435462e Validators temporary join shard overlays 2022-08-19 18:24:17 +03:00
SpyCheese
7241522de2 Remove obsolete interface for importing blocks 2022-08-19 18:24:16 +03:00
SpyCheese
212c07f2f0 Change update_shard_configuration 2022-08-19 18:24:15 +03:00
SpyCheese
e43e235143 Get neighbors' msg queues from other nodes 2022-08-19 18:24:15 +03:00
SpyCheese
7ac60bea7d New shard overlays 2022-08-19 18:24:15 +03:00
SpyCheese
53270a00e6 Add CollatorNode and make validators request blocks from it 2022-08-19 18:24:15 +03:00
SpyCheese
415ace3da9 Exporting and importing candidates using validator-engine-console; lite mode for validator group 2022-08-19 18:24:14 +03:00
SpyCheese
849d995346 Add getvalidatorsessions query to validator-engine-console 2022-08-19 18:24:14 +03:00
SpyCheese
b3e6819f3f Log validator session stats to file 2022-08-18 21:51:57 +03:00
EmelyanenkoK
89b8717cf8
Persistent optimization (#382)
* State serialization consuming less RAM

* Move large-boc-serializer to crypto/vm, refactor code

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2022-05-24 21:17:11 +03:00
EmelyanenkoK
c07394aab5
Optimization of persistent state serialization (#364)
* Fix double serialization of masterchain; increase sync_blocks_before

* Improve logging in DownloadState

* Write persistent state directly to file instead of a buffer

* Don't keep ref to masterchain state in AsyncStateSerializer

* Sparse state serialization over longer period

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2022-05-15 17:51:24 +03:00
OmicronTau
678a8a6a13 Add overlay certificate checks 2022-05-06 10:32:55 +03:00
ton
9f008b129f updated submodules, bugfixes
- added new fift/func code for validator complaint creation
- bugfixes in validator
- updates in tonlib
- new versions of rocksdb/abseil
- hardfork support
2020-04-27 16:01:46 +04:00