1
0
Fork 0
mirror of https://github.com/ton-blockchain/ton synced 2025-02-13 11:42:18 +00:00
Commit graph

133 commits

Author SHA1 Message Date
SpyCheese
172c16ca2e Merge branch 'testnet' into block-generation 2024-05-13 16:38:48 +03:00
EmelyanenkoK
d5c09936cf
Block broadcasts in custom overlays (#986)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-10 17:04:49 +03:00
EmelyanenkoK
6fb2019a4f
Improve validator session stats (#982)
* Add list of validators
* Fix producer stats
* Make round and stats timestamps more meaningful

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-08 17:41:15 +03:00
SpyCheese
a745f6f679 Merge branch 'testnet' into block-generation 2024-04-26 12:08:28 +03:00
EmelyanenkoK
9a543c6b28
Add more info to session stats (#973)
* Improve validator session stats

* Improve validator session stats

Add got_submit_at, collation_cached, validation_cached.
Fix stats cleanup.

* Fix setting timestamp for cached blocks

* Add serialize/deserialize time, serialized size to validator session stats, fix setting is_accepted

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-25 18:47:15 +03:00
SpyCheese
1fda6902a9 Merge branch 'testnet' into block-generation 2024-04-01 18:32:44 +03:00
EmelyanenkoK
f7907bdd58
Rename private ext msg overlays to custom overlays, change interface a bit (#952)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-01 18:24:08 +03:00
SpyCheese
4916e4847a Merge branch 'testnet' into block-generation 2024-04-01 17:28:23 +03:00
EmelyanenkoK
0434eadc1f
Add custom overlays for external messages (#949)
* Private overlay for external messages

* Improve ext msg overlays

* Manage from validator console
* Bypass out queue size limit for high-priority messages
* Shuffle messages in get_external_messages

* Cleanup mempool when creating validator group

* Improve private overlays for externals

1. Allow using validator adnl ids in addition to fullnode ids
2. Set priority per sender, not per overlay
3. Require the same overlay name for all nodes
4. Enable lz4 in private block overlay

* Fix typo, add debug logs

* Enable lz4 in private block overlay by config

Change proto_version for lz4 in catchain overlays to 4

* Add logs for broadcasts in fullnode

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-01 16:44:08 +03:00
SpyCheese
f5cedc3b6e Enable compression in private overlays v2 and in collator node 2024-03-29 15:04:07 +03:00
SpyCheese
7999a7e2c1 Merge branch 'testnet' into block-generation 2024-03-26 16:19:34 +03:00
EmelyanenkoK
b07614335c
liteServer.getOutMsgQueueSizes method (#943)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-03-26 15:55:58 +03:00
EmelyanenkoK
0bcebe8a0e
Exp/compress candidates (#942)
* Compress block candidates in validator-session

* Compress blocks in full-node (disabled for now)

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-03-26 14:52:46 +03:00
EmelyanenkoK
9452c367e4
LS queries to nonfinal blocks (#941)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-03-26 14:51:06 +03:00
SpyCheese
3190c67f18 Simplify getCapabilities in fullnode 2024-03-26 11:42:30 +03:00
SpyCheese
2ee7cbc106 Merge branch 'testnet' into block-generation
# Conflicts:
#	tonlib/tonlib/TonlibClient.cpp
#	validator-engine/validator-engine.hpp
#	validator/validator-options.hpp
#	validator/validator.h
2024-03-25 18:40:47 +03:00
EmelyanenkoK
f1592641de
Tonlib improvement fixes (#934)
* 3.6. Fix directory traversal in KeyValueDir

* 3.9. Fix buffer_to_hex reversing nibbles

* 3.5. Fix error handling at blocks.getBlockHeader

* 3.11. Fix query.forget

* 3.12. Fix error handling in RemoteRunSmcMethod

* 4.1. Delete unused files

* 3.10. Use named constants instead hardcoded constants

* 3.4. Fix response block header verification

* 3.1. Check proof of blocks.getShards response

* fix td::buffer_to_hex + test

* 3.2. Add proof check for listBlockTransactions response in RunEmulator actor

* 3.8. Add proof checking for getLibraries method

* fix regression tests

* 3.3 Add proof checking for lookupBlock method

* Add publishers to proof of getLibrariesWithProof response  (#25)

* fix missing return, fix requesting mc block

* Fix requesting lookupBlock with client mc blk == mc ref block

* Fix duplicating lib data in proof and data, add mode 2 for not including the data

* Migration of LastBlockStorage with fixed td::buffer_to_hex

---------

Co-authored-by: ms <dungeon666master@protonmail.com>
Co-authored-by: Marat <98183742+dungeon-master-666@users.noreply.github.com>
2024-03-19 15:31:29 +03:00
SpyCheese
fc6e2ead29 Revert "Process adnl query errors"
This reverts commit 1869a25062.
2024-03-15 14:13:07 +03:00
SpyCheese
28699654fb New private block overlays 2024-02-21 18:38:43 +03:00
SpyCheese
a2eb3f3631 Merge branch 'testnet' into block-generation 2024-02-16 16:38:47 +03:00
EmelyanenkoK
59927ba534
Improve block broadcasts processing; add special overlay for blocks for validators (#885)
* Improve block broadcast processing

* ValidatorManagerImpl::written_handle
* Retry sending broadcasts in ValidatorGroup
* Fix setting channel_ready in AdnlPeerPair

* Add special overlay for validators for block broadcasting (#842)

* Private overlay for broadcasting blocks

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>

(cherry picked from commit a52045bd91)

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-02-01 20:20:45 +03:00
SpyCheese
f4fd3ff3be Merge branch 'testnet' into block-generation 2024-02-01 19:29:25 +03:00
SpyCheese
a68b5cbe62
Improve validator session stats (#861) 2024-01-16 14:24:46 +03:00
EmelyanenkoK
4303e49c93 Postpone addition of overlay for block broadcasting (#842)
This reverts commit a52045bd91.
2024-01-16 11:29:03 +03:00
EmelyanenkoK
a52045bd91
Add special overlay for validators for block broadcasting (#842)
* Private overlay for broadcasting blocks

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2023-12-28 21:54:55 +03:00
SpyCheese
2bfa6240dd
Support wallet-v4 in tonlib (#785) 2023-10-25 12:20:23 +03:00
SpyCheese
4d5b79224d Proxy liteserver 2023-09-21 15:14:34 +03:00
SpyCheese
5c02459fd8 Optimize importing out queues 2023-07-31 18:12:09 +03:00
SpyCheese
da137fecf5 Extra shard overlay stats 2023-07-26 13:05:16 +03:00
SpyCheese
f1e62d0075 Move msg queue limits to config 2023-07-24 15:29:55 +03:00
SpyCheese
8e85bfa6e6 Merge branch 'testnet' into block-generation 2023-07-14 16:00:39 +03:00
EmelyanenkoK
f923b96a2f
Add Tonlib emulator libs (#744)
* Use global libraries in RunEmulator

* Tonlib method smc.getLibrariesExt

* Process missing transaction in getStateByTransaction

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2023-07-14 15:38:21 +03:00
SpyCheese
5dd0c15d07 Limit imported msg queue size 2023-07-04 23:34:34 +03:00
SpyCheese
e4e77c16c5 Merge branch 'testnet' into block-generation 2023-06-02 13:34:00 +03:00
SpyCheese
5abfe2337e
Catchain improvements (#698)
* Fix "sgn" in fift; fix marking infinite loops as noterurn in func

* TON-P1-1: Remove unused catchain queries

* TON-P1-15: Avoid synchronization with self

* TON-P1-3, TON-P1-17: Disallow more than one candidate per src per round (to prevent flood), add checks to process_broadcast

* TON-P1-10: Fix fast/slow attempts

* TON-P1-14: Add named constants

* TON-P1-18, TON-P1-19: Alloc temporary memory in the same way as persistent memory

* TON-P1-20: Add comment to choose_blocks_to_approve

* TON-P1-16: Avoid creating two catchain blocks on need_new_block

* TON-P1-8: Add some validation to validator-engine parameters

* TON-P1-6: Don't allow sending the same block many times

Many requests for the same block are not unusual (however, there's no need to answer them all)

* TON-P1-2: Enable prohibiting dependencies from blamed nodes (2.7.5 in CatChain doc), fix processing blame proofs

* Best practices

bp-6: Fix incorrect warning
bp-7: Remove unused code
bp-8: Bring back PerfWarningTimer logging (only when no callback)
bp-9: Remove unnecessary condition
bp-11: Remove commented-out code
bp-13: Divide code in validator-session-state
Adherence to Specification: Fix typo
2023-05-10 12:57:57 +03:00
SpyCheese
bb21f732fd
Recent updates in storage (#667)
* Fix error handling in Torrent.cpp, improve choosing peers for upload

* Various improvements in storage daemon

"get-pieces-info"
Store "added at"
Improve calculating up/down speed
Improve TL protocol for future compatibility
Remove empty directories on "--remove-files"
Better windows support
Debug logs in PeerActor
More restrictions on TorrentInfo
Bugfixes

* Global speed limits for download and upload

+bugfix

* Reset download/upload speed on changing settings or completion

* Exclude some system files in TorrentCreator
2023-04-07 15:50:07 +03:00
SpyCheese
d082ac36b0 Merge branch 'testnet' into block-generation 2023-03-28 16:52:33 +03:00
EmelyanenkoK
47311d6e0e
Improve tweaking for high throughput (#610)
* Option "--disable-ext-msg-broadcast"

* "Get shard out queue size" query

* Move disabling ext msg broadcasts from command-line arguments to config

* Fix compilation error

* Asynchronous store_cell and gc in celldb

* Make GC in celldb work evenly over time

* Increase timeouts for downloading blocks

* Reuse blocks from previous rounds in validator session

* Use Rldp2 in FullNode for downloading persistent states and archives

* Improve logs in download-archive-slice and download-state

* Decrease delay between serializing shards

* Make CellDbIn::load_cell synchronous to avoid interfering with store_cell

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2023-03-15 10:47:35 +03:00
Marat
078aabe50e
Add method listBlockTransactionsExt to liteserver (#399)
* Verify proof for method blocks.getTransactions

* check completeness of response

* fix start_lt

* fix mode & 128, check bTxes->ids_ out of bounds

* Improve gitactions; separate cpp-check (#346)

* Use ninja build tool and compile blockchain-explorer

Ninja builds TON much faster;

* Use clang instead of gcc

* remove blockchain-explorer

since target not found on github action

* move ccpcheck to other gitaction

* run nativelib-java only against wallets branch for now

* rename gitaction

* Update windows2019x64-tonlib-java.yml

* Update windows2019x64-tonlib-java.yml

* Update macos-10.15-tonlib-java.yml

* Update windows2019x64-tonlib-java.yml

* Update windows2019x64-tonlib-java.yml

* rebase

* update tlo's

* Revert "Improve gitactions; separate cpp-check (#346)"

This reverts commit bd1d96e6d391e48840d81cfcf10d2692848e504e.

* add checks, simplify ls response

* Revert workflows

* Add verifying proofs

* fix win build

---------

Co-authored-by: neodiX42 <namlem@gmail.com>
2023-03-07 19:49:25 +03:00
EmelyanenkoK
c369127ae0
rldp2 support in rldp-http-proxy (#608)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2023-02-03 09:50:44 +03:00
EmelyanenkoK
3b3c25b654
Add account state by transaction and emulator (extended) (#592)
* account_state_by_transaction

* Correct time calculation

* Bug fixes

* Refactor

* namespace block::transaction

* smc.forget

* RunEmulator: remove wallet_id

* Refactor & fixes

* AccountStateByTransaction: use shardchain block instead of masterchain block

* transaction emulator core

* refactor

* tx emulator major functionality

* small format changes

* readme

* clean

* return json, add support for init messages

* tx emulator readme

* refactor getConfigParam and getConfigAll

* add shardchain_libs_boc parameter

* option to change verbosity level of transaction emulator

* fix deserializing ShardAccount with account_none

* add mode needSpecialSmc when unpack config

* emulator: block::Transaction -> block::transaction::Transaction

* Refactor

* emulator: Fix bug

* emulator: Support for emulator-extern

* emulator: Refactor

* Return vm log and vm exit code.

* fix build on macos, emulator_static added

* adjust documentation

* ignore_chksig for external messages

* tvm emulator, run get method

* Added more params for transaction emulator

* Added setters for optional transaction emulator params, moved libs to a new setter

* Added actions cell output to transaction emulator

* fix tonlib build

* refactoring, rand seed as hex size 64, tvm emulator send message

* tvm send message, small refactoring

* fix config decoding, rename

* improve documentation

* macos export symbols

* Added run_get_method to transaction emulator emscipten wrapper

* Fixed empty action list serialization

* Changed actions list cell to serialize as json null instead of empty string in transaction emulator

* stack as boc

* log gas remaining

* Fix prev_block_id

* fix build errors

* Refactor fetch_config_params

* fix failing unwrap of optional rand_seed

* lookup correct shard, choose prev_block based on account shard

* fix tonlib android jni build

---------

Co-authored-by: legaii <jgates.ardux@gmail.com>
Co-authored-by: ms <dungeon666master@protonmail.com>
Co-authored-by: krigga <krigga7@gmail.com>
2023-02-02 10:03:45 +03:00
SpyCheese
29851c38ef Various changes after merge 2023-01-12 17:32:59 +03:00
SpyCheese
2ea17ec03b Use partial liteservers in tonlib and lite-client 2023-01-11 20:16:32 +03:00
SpyCheese
1ccf25d6b7 Merge branch 'testnet' into block-generation 2023-01-11 16:10:57 +03:00
EmelyanenkoK
7347ec0b3b
Add TON Storage to Proxy via storage gateway (#577)
* Access ton storage from proxy, resolve "dns_storage_address" in tonlib

* Set storage gateway address in proxy args

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2022-12-29 17:28:50 +03:00
SpyCheese
ad736c6bc3
Add "--copy" to storage-daemon-cli create command (#576)
* Add "--copy" flag to "create", improve console output

* Hide excessive logs
2022-12-28 17:40:20 +03:00
SpyCheese
48d48e595e
Pause and resume upload in storage-daemon (#567) 2022-12-23 11:05:29 +03:00
SpyCheese
d652f7d706 Merge branch 'testnet' into block-generation
# Conflicts:
#	adnl/adnl-query.cpp
#	crypto/block/block.tlb
#	crypto/block/mc-config.h
#	lite-client/lite-client.cpp
#	overlay/overlay-manager.h
#	overlay/overlay-peers.cpp
#	overlay/overlay.cpp
#	overlay/overlay.h
#	overlay/overlay.hpp
#	overlay/overlays.h
#	rldp-http-proxy/DNSResolver.cpp
#	rldp-http-proxy/rldp-http-proxy.cpp
#	tl/generate/scheme/ton_api.tl
#	tl/generate/scheme/ton_api.tlo
#	tl/generate/scheme/tonlib_api.tlo
#	ton/ton-types.h
#	tonlib/tonlib/ExtClient.cpp
#	tonlib/tonlib/ExtClient.h
#	tonlib/tonlib/ExtClientLazy.cpp
#	tonlib/tonlib/ExtClientOutbound.h
#	tonlib/tonlib/ExtClientRaw.h
#	tonlib/tonlib/TonlibClient.cpp
#	tonlib/tonlib/TonlibClient.h
#	tonlib/tonlib/tonlib-cli.cpp
#	validator/impl/collator.cpp
#	validator/impl/validate-query.cpp
#	validator/impl/validate-query.hpp
#	validator/manager.cpp
#	validator/state-serializer.cpp
#	validator/state-serializer.hpp
#	validator/validator-group.cpp
#	validator/validator-group.hpp
#	validator/validator.h
2022-12-22 23:37:35 +03:00
EmelyanenkoK
360ef54e6b
TON Storage utilities (#564)
* Rename chunk to piece in MerkleTree for consistency

* Refactor PeerManager

* Make PeerState thread-safe

* Download torrent by hash

* First version of storage daemon

* Download torrents partially

* Improve storing and loading torrent state in DB

* Rewrite MerkleTree

* "Remove torrent" in storage daemon

* Process errors, fix bugs in storage

* Move TonlibClientWrapper from rldp-http-proxy to tonlib

* Initial version of storage provider

* Move interaction with contracts to smc-util

* Improve TonlibClientWrapper interface

* Various improvements in storage provider

* Fix TorrentCreator.cpp

* Improve interface for partial download

* Client mode in storage-daemon

* Improve interface of storage-daemon-cli

* Fix calculating speed, show peers in storage-daemon

* Use permanent adnl id in storage daemon

* Fix sending large "storage.addUpdate" messages

* Improve printing torrents in cli

* Update tlo

* Fix RldpSender::on_ack

* Update storage provider

* Add "address" parameter to get-provider-params

* Allow client to close storage contract

* Limit torrent description

* Add more logs to storage provider

* smc.forget tonlib method

* Use smc.forget in storage daemon

* Optimize sending messages in smc-util.cpp

* Fix verbosity, remove excessive logs

* Json output in storage-daemon-cli

* Update storage provider contracts

* Fix rldp2 acks

* Change verbosity of logs in rldp2

* Update help and output of commands and in storage-daemon-cli

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2022-12-22 12:24:13 +03:00
EmelyanenkoK
3ff50f5f47
Add network id to dht (#559)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2022-12-20 14:26:39 +03:00