1
0
Fork 0
mirror of https://github.com/ton-blockchain/ton synced 2025-02-15 04:32:21 +00:00
Commit graph

56 commits

Author SHA1 Message Date
SpyCheese
7bc50e63d7
tonNode.getOutMsgQueueProof query in public shard overlays (#1413)
* tonNode.getOutMsgQueueProof query in public shard overlays

* Allow responding to getOutMsgQueueProof requests one at a time only
2024-12-04 14:38:57 +03:00
SpyCheese
061c82f89c
Send validator telemetry to the private overlay (#1325)
* Send validator telemetry to the private overlay

* Improve rotating neighbours in overlays
2024-11-25 22:37:18 +03:00
birydrad
72020c04c4
celldb in-memory mode, stats for actors, perf counters, minor fix in rldp2 (#1164)
* getactorstats query for validator-engine-console

* celldb in-memory mode (--celldb-in-memory option)

* rldp2: bugfix - do not estimate speed while nothing is sent

* add simple ed25519 benchmark

* fix compilation errors of different platforms and move to c++20

* fix some warnings

* turn on TON_USE_ABSEIL for glibc 2.27 nix build

---------

Co-authored-by: birydrad <>
2024-09-23 17:34:37 +03:00
EmelyanenkoK
1a5bbf30f1
mintless-proof-generator (#1166)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-09-12 18:02:56 +03:00
SpyCheese
9c3dc22b78
Improve validator session stats (#1117)
* Improve validator session stats

* Collator stats: block limits, number of processed external messages
* Collator and validator work time
* Last key block seqno
* Approvers and signers

* End validator session stats
2024-08-20 19:54:16 +03:00
SpyCheese
9661676646
Improve dht lookup in overlays (#1104)
Continue dht lookup even if value was found
2024-08-15 15:26:35 +03:00
EmelyanenkoK
e985ac0358
Add --fast-state-serializer flag and responding to AdnlMessageCreateChannel with Nop (#1096)
* Add --fast-state-serializer and tools for jemalloc

* Disable fast state serializer by default unless RAM is >= 90GB
* Print jemalloc stats once a minute
* Dump jemalloc profile on request

* Respond to AdnlMessageCreateChannel with Nop

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-08-07 14:25:45 +03:00
SpyCheese
b9e89d4c66
"getcollatoroptionsjson" command in validator console (#1059)
* "getcollatoroptionsjson" command in validator console

* Improve state serializer

Use previous persistent state to speed up reading
2024-07-17 15:10:55 +03:00
EmelyanenkoK
0daee1d887
Merge message dispatch queue (#1030)
* Deferred messages and msg metadata

* Store out msg queue size in state

* Add checks for queue processing

1. Collator must process at least one message from AccountDispatchQueue (unless block is full)
2. The first message from a transaction is not counted, it cannot be deferred (unless AccountDispatchQueue is not empty)

* Return msg metadata from LS in listBlockTransactions[Ext]

* Enable new features by capabilities

* Changes in deferred messages

* Process deferred messages via new_msgs in collator
* Rework setting deferred_lt, bring back check_message_processing_order, check order of deferred_lt in validator

* Use have_unprocessed_account_dispatch_queue_ in collator

* Fix setting transaction lt for deferred messages

* Fix lite-client compilation error

* Changes in process_dispatch_queue, rename deferred_lt -> emitted_lt

* Fix compilation error

* Use uint64 for msg queue size

* Add liteServer.getBlockOutMsgQueueSize

* Fix compilation error

* Fix typos in comments

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-06-27 16:12:23 +03:00
EmelyanenkoK
74801d00b8
Command to disable state serializer (#1011)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-31 20:50:31 +03:00
EmelyanenkoK
d2b012c883
Add write time stats to celldb/db_stats.txt (#972)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-25 18:45:39 +03:00
EmelyanenkoK
0bcebe8a0e
Exp/compress candidates (#942)
* Compress block candidates in validator-session

* Compress blocks in full-node (disabled for now)

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-03-26 14:52:46 +03:00
EmelyanenkoK
f1592641de
Tonlib improvement fixes (#934)
* 3.6. Fix directory traversal in KeyValueDir

* 3.9. Fix buffer_to_hex reversing nibbles

* 3.5. Fix error handling at blocks.getBlockHeader

* 3.11. Fix query.forget

* 3.12. Fix error handling in RemoteRunSmcMethod

* 4.1. Delete unused files

* 3.10. Use named constants instead hardcoded constants

* 3.4. Fix response block header verification

* 3.1. Check proof of blocks.getShards response

* fix td::buffer_to_hex + test

* 3.2. Add proof check for listBlockTransactions response in RunEmulator actor

* 3.8. Add proof checking for getLibraries method

* fix regression tests

* 3.3 Add proof checking for lookupBlock method

* Add publishers to proof of getLibrariesWithProof response  (#25)

* fix missing return, fix requesting mc block

* Fix requesting lookupBlock with client mc blk == mc ref block

* Fix duplicating lib data in proof and data, add mode 2 for not including the data

* Migration of LastBlockStorage with fixed td::buffer_to_hex

---------

Co-authored-by: ms <dungeon666master@protonmail.com>
Co-authored-by: Marat <98183742+dungeon-master-666@users.noreply.github.com>
2024-03-19 15:31:29 +03:00
neodix42
e0a320f715
Improve TON build scripts and some tests (#855)
* fix macOS github actions

* fix android tonlib GH action;

* fixing wasm GH action

* strip binaries

* fix randomly failing ubuntu and wasm GH actions

* fix randomly failing ubuntu and wasm GH actions

* revert some changes

* adding more nix scripts and automated native build scripts;
debug static ton compilation

* minor fix

* do not use pkg_config if path specified

* move wasm script, run with sudo action script

* weird, fixing - cp: missing destination file operand after 'assembly/native/build-ubuntu-20.04-shared.sh'

* weird, fixing - cp: missing destination file operand after 'assembly/native/build-ubuntu-20.04-shared.sh'

* ok

* some adjustments for android and win builds

* some adjustments for android and win builds

* moving stripping inside the build script

* access rights handling; adding simple binaries' tests

* make lite-client-common, fift-lib and smc-envelope deliberately static;
add -a (artifacts) flag to build scripts;

* minor wasm build fix;
create separate tonlib android build script;
remove outdated __has_trivial_copy(T)

* add windows build - WIP

* adjust android build;
improve win build;

* adjust sodium paths for android build; use proper compiler for windows build;

* add github windows build auxiliary file

* adjust wasm build

* add portable ubuntu build

* exclude some unstable tests for some time

* compile portable binaries on ubuntu-20.04

* exclude some unstable tests

* include static gsl

* restart builds

* restart builds

* restart builds

* remove libreadline, gsl and blas dependencies in linux build

* add macos build script

* install missing autoconf in macos builds

* enable all tests and see what fails

* enable win tests and restart others

* enable win tests and fix test-smartcont.cpp

* enable win tests

* use clang-16 on mac builds, add blockchain-explorer for ubuntu builds, add portable macos build

* move sudo part outside a build scripts

* move sudo part outside a build scripts

* run llvm install with sudo

* remove libgnutls28-dev before ubuntu static compilation, include blockchain-explorer into artifacts;
remove warning: definition of implicit copy constructor for 'Stat' is deprecated because it has a user-declared copy assignment operator [-Wdeprecated-copy]

* rework wrong decision, put back system gnutls, but compile libmicrohttpd with --disable-https

* add jenkins pipeline sceleton

* WIP jenkins pipeline sceleton

* WIP jenkins pipeline changes

* WIP jenkins pipeline: add stage timout, zip and group artifacts

* WIP jenkins pipeline: macos portable build fix

* WIP jenkins pipeline: wording

* WIP jenkins pipeline: add android tonlib

* WIP jenkins pipeline: add was binaries

* WIP jenkins pipeline: add TOTAL_MEMORY 1.5gb to funcfiftlib wasm linking

* WIP jenkins pipeline: add nix build on linux aarch64

* WIP jenkins pipeline: funcfiftlib compilation fails that 16mb mem is not enough, increase to 32mb

* WIP jenkins pipeline: enable test in nix build

* WIP jenkins pipeline: add linux x86-64 nix build

* WIP jenkins pipeline: include libs in nix build

* WIP jenkins pipeline: include libs in nix build

* WIP jenkins pipeline: include mac nix build

* WIP jenkins pipeline: include mac nix build

* WIP jenkins pipeline: include mac nix build

* WIP jenkins pipeline: include mac nix build

* WIP jenkins pipeline: include mac nix build

* WIP jenkins pipeline: include mac nix build

* WIP jenkins pipeline: nix linux arm64 with openssl 1.1 for now

* WIP jenkins pipeline: working ubuntu arm64 libtonjson

* WIP jenkins pipeline: working ubuntu arm64 libtonjson + minor fix

* WIP jenkins pipeline: working ubuntu arm64 libtonjson + minor fix 2

* WIP jenkins pipeline: merry christmas

* WIP jenkins pipeline: merry christmas 2

* WIP jenkins pipeline: remove native static builds

* WIP jenkins pipeline: enable more tests

* WIP jenkins pipeline: zip artifacts better

* WIP jenkins pipeline: get rid of path in the final zip

* WIP jenkins pipeline: minor fix, include lib and smartcont folders

* WIP jenkins pipeline: minor fix, include lib and smartcont folders into nix artifacts also

* WIP jenkins pipeline: minor fix

* WIP jenkins pipeline: minor fix

* adjust github actions for new nix builds

* cleanup

* cleanup

* cleanup

* cleanup

* rename libtonlibjson.so.0.5 to libtonlibjson.so

* Add TON build instructions to README.md

* simplify

* fix test-tonlib-offline

* set timeout per test of 300 sec

* set timeout per test of 600 sec for non nix builds

* increase test timeout to 900 sec; minor changes

* use MS VS 2022 for win TON compilation; update README.md

* use MS VS 2022 for win TON compilation; update README.md

* change path to MSVC in github workflow

* change path to MSVC in groovy pipeline

* compile ton on win, with msvc 2022 community and enterprise versions

* minor fixes

* improve network tests

* remove TON compilation against macos-11 github runner

* add `choco feature enable -n allowEmptyChecksums` since pkg-config-lite-0.28-1 does not have a checksum

* abort win compilation if 3pp can't be downloaded

* increase test timeout to 30 min

* improving test-catchain
2024-01-15 23:48:04 +03:00
EmelyanenkoK
7fcf267717
Improve large OutMsgQueue clearance (#822)
* Improve Collator::opt_msg_queue_cleanup, increase collator timeout

* Disable importing ext msgs if queue is too big

* Extend timeout in collator if previous block is too old

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2023-12-06 19:34:01 +03:00
SpyCheese
5847897b37
Various changes in TVM, github builds and tests (#793)
* Bugfixes in TVM and node

* Upgrade to C++17

* Improve GitHub builds

* Fix existing tests and partially integrate them into builds

---------

Co-authored-by: neodiX42 <namlem@gmail.com>
Co-authored-by: EmelyanenkoK <emelyanenko.kirill@gmail.com>
2023-11-03 14:43:34 +03:00
SpyCheese
5abfe2337e
Catchain improvements (#698)
* Fix "sgn" in fift; fix marking infinite loops as noterurn in func

* TON-P1-1: Remove unused catchain queries

* TON-P1-15: Avoid synchronization with self

* TON-P1-3, TON-P1-17: Disallow more than one candidate per src per round (to prevent flood), add checks to process_broadcast

* TON-P1-10: Fix fast/slow attempts

* TON-P1-14: Add named constants

* TON-P1-18, TON-P1-19: Alloc temporary memory in the same way as persistent memory

* TON-P1-20: Add comment to choose_blocks_to_approve

* TON-P1-16: Avoid creating two catchain blocks on need_new_block

* TON-P1-8: Add some validation to validator-engine parameters

* TON-P1-6: Don't allow sending the same block many times

Many requests for the same block are not unusual (however, there's no need to answer them all)

* TON-P1-2: Enable prohibiting dependencies from blamed nodes (2.7.5 in CatChain doc), fix processing blame proofs

* Best practices

bp-6: Fix incorrect warning
bp-7: Remove unused code
bp-8: Bring back PerfWarningTimer logging (only when no callback)
bp-9: Remove unnecessary condition
bp-11: Remove commented-out code
bp-13: Divide code in validator-session-state
Adherence to Specification: Fix typo
2023-05-10 12:57:57 +03:00
EmelyanenkoK
47311d6e0e
Improve tweaking for high throughput (#610)
* Option "--disable-ext-msg-broadcast"

* "Get shard out queue size" query

* Move disabling ext msg broadcasts from command-line arguments to config

* Fix compilation error

* Asynchronous store_cell and gc in celldb

* Make GC in celldb work evenly over time

* Increase timeouts for downloading blocks

* Reuse blocks from previous rounds in validator session

* Use Rldp2 in FullNode for downloading persistent states and archives

* Improve logs in download-archive-slice and download-state

* Decrease delay between serializing shards

* Make CellDbIn::load_cell synchronous to avoid interfering with store_cell

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2023-03-15 10:47:35 +03:00
Dan Volkov
4590ed381b
Make funcfiftlib compilation compatible with modern compilers (#618)
* wip: make funcfiftlib compilation compatible with modern compilers

* wip: add methods needed for another compiler

* fix: tdutils port config if emscripten

* feat: func source and realpath callback

* fix: invalid fift compilation exceptions

---------

Co-authored-by: krigga <krigga7@gmail.com>
2023-03-09 17:37:15 +03:00
SpyCheese
1e4eecfdb0
Logs and size limits for incoming queries in FullNodeShard (#601)
* Log incoming queries in FullNodeShard

* Limit size for some queries in FullNodeShard
2023-02-02 10:04:19 +03:00
Andrey Pfau
f66c06143c
Fix pretty print json (remove unnecessary line breaks) (#515)
* add: TON Overflow

* Fix pretty print json (remove unnecessary line breaks)
2022-11-02 21:28:44 +03:00
AlexeyFSL
e2cca03a78
Func and Fift lib for WASM (#455)
* Add social badges

Add telegram, Twitter and Stack Overflow badges in Readme

* update README.md badges

* patch for wasm build

* fix narrowing conversion error for clang compiler

* refactor func code

* funcfift lib implementation

* fix funcfift lib
fix CMakeFile

* fix rvalue missing

* remove unused field from result json

* name fix
remove unused target

* rename

* added script for building funcfiftlib to wasm

* fix json fild names

* fix commit hash for script

* added version function to funcfiftlib

* update commit hash for script

* add realpath fail processing
fix DISABLE_EXCEPTION_CATCHING option

* update hash in script

Co-authored-by: Anthony Tsivarev <tsivarev.a@gmail.com>
Co-authored-by: EmelyanenkoK <emelyanenko.kirill@gmail.com>
Co-authored-by: tolya-yanot <1449561+tolya-yanot@users.noreply.github.com>
2022-09-14 12:36:01 +03:00
EmelyanenkoK
8329a58994
Add GetPerfTimerStats (#451)
* Add getperfwarningtimeraverage query for validator-engine-console

* Fix for getperfwarningtimeraverage query

* getperftimerstats

* Add history max size: 1 hour

* PerfWarningTimer: move callback instead of copy

* PerfWarningTimer: fix move constructor bug

* PerfWarningTimer: fix bug: lifetime of the callback was greater than lifetime of the local variable 'manager' captured by reference

* Fix validate-query.cpp: 'manager' used after it was moved

* PerfWarningTimer: remove logs

* getperftimerstats: write to json file

* getperftimerstatsjson

Co-authored-by: legaii <jgates.ardux@gmail.com>
Co-authored-by: Ivan Siomash <106972486+legaii@users.noreply.github.com>
2022-09-10 12:57:13 +03:00
SpyCheese
b398862cf8 Fix error handling in ServerSocketFd.cpp 2022-06-27 10:22:39 +03:00
OmicronTau
cb31a20206 Add methods to sign and import certificates 2022-05-06 10:42:49 +03:00
Alex Melman
5c992d6a93 Using Howard Hinnant's free, open-source header-only and thread-safe datetime library.
https://howardhinnant.github.io/date/date.html
All logs will display a datetime in format "2021-02-23 12:57:21.1023272" UTC, instead of simply displaying double formatted timestamp.

Update ccpp-linux.yml

Soon github actions will use ubuntu 20.04 as latest. Let's avoid surprised and stick to ubuntu-18.04 fow now.
2021-02-23 14:31:58 +03:00
AleksejMelman
b22611a123 Create ccpp-linux.yaml
Taken from https://github.com/akme/ton/blob/master/.github/workflows/ccpp-linux.yml

Fixing error LNK2019: unresolved external symbol _GetProcessMemoryInfo@12 when building windows desktop client

Fixes compilation error of lite-client on windows

Fixes compilation error:
error C2664: 'std::pair<td::int64,td::int64> std::make_pair<td::int64,td::int64>(_Ty1 &&,_Ty2 &&) noexcept (<expr>)': cannot convert argument 1 from 'const td::uint64' to '_Ty1 &&'

make_pair() has been changed in VS2012 to support a new C++11 feature called move semantics and I suspect that explicitly specifying the types for make_pair() is getting in the way.

The fix should be safe for linux distro as well.

Adding Github action - Win64 ton compile (#14)

* Add github action win64 compile
* manager.cpp - Strange though, but due to this kind of type usage win64 compilation fails. Let's make it more generic.
2021-02-09 11:15:50 +03:00
ton
24dc184a2e validator large state serialization bugfix + tdlib update for clients 2021-01-07 22:48:04 +03:00
ton
f064b1047a pow-testgiver support 2020-07-06 17:07:20 +03:00
ton
4e2624459b integrating the existing state of TON Storage / TON Payments / CPS Fift development branches 2020-05-27 22:10:46 +04:00
ton
eecf05ca59 bugfixes + tonlib update 2020-04-30 15:04:47 +04:00
ton
9f008b129f updated submodules, bugfixes
- added new fift/func code for validator complaint creation
- bugfixes in validator
- updates in tonlib
- new versions of rocksdb/abseil
- hardfork support
2020-04-27 16:01:46 +04:00
ton
be9c34c62d bugfixes 2020-04-10 23:06:01 +04:00
ton
8be3fc99ed bugfixes + decreased archive slice size 2020-04-08 23:49:28 +04:00
ton
a31f8d4424 bugfixed + crypto update
- compiles vs BoringSSL
- config proposal/vote fift code
- bugfixes in catchain
- other small fixes
2020-03-27 18:59:00 +04:00
ton
9f351fc29f emergency update 2020-03-24 03:32:16 +04:00
ton
e27fb1e09c updated vm (breaking compatibility)
- updated vm
- new actor scheduler
- updated tonlib
- updated DNS smartcontract
2020-02-28 14:28:47 +04:00
ton
a73d202ba2 updated func and tonlib 2020-02-15 20:03:17 +04:00
ton
1de39f5d7c bugfixes + doc update 2020-02-08 23:24:49 +04:00
ton
77842f9b63 updated tonlib
- updated tonlib
- updated validator
- updated documentation
- first version of http over rldp proxy
2020-02-06 21:56:46 +04:00
ton
53ec9684bd updated liteserver
- new methods for liteserver/liteclient
- added ADNL/DHT client-only work mode
- fixed crash in ADNL
2020-02-02 16:53:37 +04:00
ton
e30d98eb30 new db
new database
fift/func bugfixes
2019-11-15 18:02:37 +04:00
ton
950e292264 do not use builtin popcnt 2019-11-01 22:15:04 +04:00
ton
3002321eb7 updated tonlib 2019-10-31 14:26:04 +04:00
ton
c860ce3d1e updated smartcontracts
- updated smartcontracts
- updated fullnode database layout
- fixed memory leak in blockchain-explorer
- updated tonlib
2019-10-23 17:43:50 +04:00
ton
b978e27b2f updated smartcontract code
- updated smartcontract code
- fixed bug in liteserver listening socket
- updated documentation
2019-10-14 11:13:23 +04:00
ton
7ea00ebfcf updated tonlib, fixed bugs
updated tonlib
fixed bugs in func
validator: partial support for hardforks
liteserver: support for waitMasterchainBlock prefix
transactions: support for gas flat rate
2019-10-03 17:04:52 +04:00
ton
fd7a8de970 updated tonlib, block routing
- upated tonlib
- fixed bug in message routing
2019-09-28 11:44:38 +04:00
ton
a1e352d894 updated tonlib, new fullnode queries 2019-09-24 01:10:57 +04:00
ton
28df74178c updated tonlib
1. updated tonlib
2. fixed bug in state download
2019-09-22 10:14:09 +04:00