1
0
Fork 0
mirror of https://github.com/ton-blockchain/ton synced 2025-02-13 03:32:22 +00:00
Commit graph

54 commits

Author SHA1 Message Date
birydrad
72020c04c4
celldb in-memory mode, stats for actors, perf counters, minor fix in rldp2 (#1164)
* getactorstats query for validator-engine-console

* celldb in-memory mode (--celldb-in-memory option)

* rldp2: bugfix - do not estimate speed while nothing is sent

* add simple ed25519 benchmark

* fix compilation errors of different platforms and move to c++20

* fix some warnings

* turn on TON_USE_ABSEIL for glibc 2.27 nix build

---------

Co-authored-by: birydrad <>
2024-09-23 17:34:37 +03:00
EmelyanenkoK
1a5bbf30f1
mintless-proof-generator (#1166)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-09-12 18:02:56 +03:00
SpyCheese
9c3dc22b78
Improve validator session stats (#1117)
* Improve validator session stats

* Collator stats: block limits, number of processed external messages
* Collator and validator work time
* Last key block seqno
* Approvers and signers

* End validator session stats
2024-08-20 19:54:16 +03:00
SpyCheese
9661676646
Improve dht lookup in overlays (#1104)
Continue dht lookup even if value was found
2024-08-15 15:26:35 +03:00
EmelyanenkoK
e985ac0358
Add --fast-state-serializer flag and responding to AdnlMessageCreateChannel with Nop (#1096)
* Add --fast-state-serializer and tools for jemalloc

* Disable fast state serializer by default unless RAM is >= 90GB
* Print jemalloc stats once a minute
* Dump jemalloc profile on request

* Respond to AdnlMessageCreateChannel with Nop

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-08-07 14:25:45 +03:00
SpyCheese
b9e89d4c66
"getcollatoroptionsjson" command in validator console (#1059)
* "getcollatoroptionsjson" command in validator console

* Improve state serializer

Use previous persistent state to speed up reading
2024-07-17 15:10:55 +03:00
EmelyanenkoK
0daee1d887
Merge message dispatch queue (#1030)
* Deferred messages and msg metadata

* Store out msg queue size in state

* Add checks for queue processing

1. Collator must process at least one message from AccountDispatchQueue (unless block is full)
2. The first message from a transaction is not counted, it cannot be deferred (unless AccountDispatchQueue is not empty)

* Return msg metadata from LS in listBlockTransactions[Ext]

* Enable new features by capabilities

* Changes in deferred messages

* Process deferred messages via new_msgs in collator
* Rework setting deferred_lt, bring back check_message_processing_order, check order of deferred_lt in validator

* Use have_unprocessed_account_dispatch_queue_ in collator

* Fix setting transaction lt for deferred messages

* Fix lite-client compilation error

* Changes in process_dispatch_queue, rename deferred_lt -> emitted_lt

* Fix compilation error

* Use uint64 for msg queue size

* Add liteServer.getBlockOutMsgQueueSize

* Fix compilation error

* Fix typos in comments

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-06-27 16:12:23 +03:00
EmelyanenkoK
74801d00b8
Command to disable state serializer (#1011)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-05-31 20:50:31 +03:00
EmelyanenkoK
d2b012c883
Add write time stats to celldb/db_stats.txt (#972)
Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-04-25 18:45:39 +03:00
EmelyanenkoK
0bcebe8a0e
Exp/compress candidates (#942)
* Compress block candidates in validator-session

* Compress blocks in full-node (disabled for now)

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2024-03-26 14:52:46 +03:00
EmelyanenkoK
f1592641de
Tonlib improvement fixes (#934)
* 3.6. Fix directory traversal in KeyValueDir

* 3.9. Fix buffer_to_hex reversing nibbles

* 3.5. Fix error handling at blocks.getBlockHeader

* 3.11. Fix query.forget

* 3.12. Fix error handling in RemoteRunSmcMethod

* 4.1. Delete unused files

* 3.10. Use named constants instead hardcoded constants

* 3.4. Fix response block header verification

* 3.1. Check proof of blocks.getShards response

* fix td::buffer_to_hex + test

* 3.2. Add proof check for listBlockTransactions response in RunEmulator actor

* 3.8. Add proof checking for getLibraries method

* fix regression tests

* 3.3 Add proof checking for lookupBlock method

* Add publishers to proof of getLibrariesWithProof response  (#25)

* fix missing return, fix requesting mc block

* Fix requesting lookupBlock with client mc blk == mc ref block

* Fix duplicating lib data in proof and data, add mode 2 for not including the data

* Migration of LastBlockStorage with fixed td::buffer_to_hex

---------

Co-authored-by: ms <dungeon666master@protonmail.com>
Co-authored-by: Marat <98183742+dungeon-master-666@users.noreply.github.com>
2024-03-19 15:31:29 +03:00
neodix42
e0a320f715
Improve TON build scripts and some tests (#855)
* fix macOS github actions

* fix android tonlib GH action;

* fixing wasm GH action

* strip binaries

* fix randomly failing ubuntu and wasm GH actions

* fix randomly failing ubuntu and wasm GH actions

* revert some changes

* adding more nix scripts and automated native build scripts;
debug static ton compilation

* minor fix

* do not use pkg_config if path specified

* move wasm script, run with sudo action script

* weird, fixing - cp: missing destination file operand after 'assembly/native/build-ubuntu-20.04-shared.sh'

* weird, fixing - cp: missing destination file operand after 'assembly/native/build-ubuntu-20.04-shared.sh'

* ok

* some adjustments for android and win builds

* some adjustments for android and win builds

* moving stripping inside the build script

* access rights handling; adding simple binaries' tests

* make lite-client-common, fift-lib and smc-envelope deliberately static;
add -a (artifacts) flag to build scripts;

* minor wasm build fix;
create separate tonlib android build script;
remove outdated __has_trivial_copy(T)

* add windows build - WIP

* adjust android build;
improve win build;

* adjust sodium paths for android build; use proper compiler for windows build;

* add github windows build auxiliary file

* adjust wasm build

* add portable ubuntu build

* exclude some unstable tests for some time

* compile portable binaries on ubuntu-20.04

* exclude some unstable tests

* include static gsl

* restart builds

* restart builds

* restart builds

* remove libreadline, gsl and blas dependencies in linux build

* add macos build script

* install missing autoconf in macos builds

* enable all tests and see what fails

* enable win tests and restart others

* enable win tests and fix test-smartcont.cpp

* enable win tests

* use clang-16 on mac builds, add blockchain-explorer for ubuntu builds, add portable macos build

* move sudo part outside a build scripts

* move sudo part outside a build scripts

* run llvm install with sudo

* remove libgnutls28-dev before ubuntu static compilation, include blockchain-explorer into artifacts;
remove warning: definition of implicit copy constructor for 'Stat' is deprecated because it has a user-declared copy assignment operator [-Wdeprecated-copy]

* rework wrong decision, put back system gnutls, but compile libmicrohttpd with --disable-https

* add jenkins pipeline sceleton

* WIP jenkins pipeline sceleton

* WIP jenkins pipeline changes

* WIP jenkins pipeline: add stage timout, zip and group artifacts

* WIP jenkins pipeline: macos portable build fix

* WIP jenkins pipeline: wording

* WIP jenkins pipeline: add android tonlib

* WIP jenkins pipeline: add was binaries

* WIP jenkins pipeline: add TOTAL_MEMORY 1.5gb to funcfiftlib wasm linking

* WIP jenkins pipeline: add nix build on linux aarch64

* WIP jenkins pipeline: funcfiftlib compilation fails that 16mb mem is not enough, increase to 32mb

* WIP jenkins pipeline: enable test in nix build

* WIP jenkins pipeline: add linux x86-64 nix build

* WIP jenkins pipeline: include libs in nix build

* WIP jenkins pipeline: include libs in nix build

* WIP jenkins pipeline: include mac nix build

* WIP jenkins pipeline: include mac nix build

* WIP jenkins pipeline: include mac nix build

* WIP jenkins pipeline: include mac nix build

* WIP jenkins pipeline: include mac nix build

* WIP jenkins pipeline: include mac nix build

* WIP jenkins pipeline: nix linux arm64 with openssl 1.1 for now

* WIP jenkins pipeline: working ubuntu arm64 libtonjson

* WIP jenkins pipeline: working ubuntu arm64 libtonjson + minor fix

* WIP jenkins pipeline: working ubuntu arm64 libtonjson + minor fix 2

* WIP jenkins pipeline: merry christmas

* WIP jenkins pipeline: merry christmas 2

* WIP jenkins pipeline: remove native static builds

* WIP jenkins pipeline: enable more tests

* WIP jenkins pipeline: zip artifacts better

* WIP jenkins pipeline: get rid of path in the final zip

* WIP jenkins pipeline: minor fix, include lib and smartcont folders

* WIP jenkins pipeline: minor fix, include lib and smartcont folders into nix artifacts also

* WIP jenkins pipeline: minor fix

* WIP jenkins pipeline: minor fix

* adjust github actions for new nix builds

* cleanup

* cleanup

* cleanup

* cleanup

* rename libtonlibjson.so.0.5 to libtonlibjson.so

* Add TON build instructions to README.md

* simplify

* fix test-tonlib-offline

* set timeout per test of 300 sec

* set timeout per test of 600 sec for non nix builds

* increase test timeout to 900 sec; minor changes

* use MS VS 2022 for win TON compilation; update README.md

* use MS VS 2022 for win TON compilation; update README.md

* change path to MSVC in github workflow

* change path to MSVC in groovy pipeline

* compile ton on win, with msvc 2022 community and enterprise versions

* minor fixes

* improve network tests

* remove TON compilation against macos-11 github runner

* add `choco feature enable -n allowEmptyChecksums` since pkg-config-lite-0.28-1 does not have a checksum

* abort win compilation if 3pp can't be downloaded

* increase test timeout to 30 min

* improving test-catchain
2024-01-15 23:48:04 +03:00
EmelyanenkoK
7fcf267717
Improve large OutMsgQueue clearance (#822)
* Improve Collator::opt_msg_queue_cleanup, increase collator timeout

* Disable importing ext msgs if queue is too big

* Extend timeout in collator if previous block is too old

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2023-12-06 19:34:01 +03:00
SpyCheese
5847897b37
Various changes in TVM, github builds and tests (#793)
* Bugfixes in TVM and node

* Upgrade to C++17

* Improve GitHub builds

* Fix existing tests and partially integrate them into builds

---------

Co-authored-by: neodiX42 <namlem@gmail.com>
Co-authored-by: EmelyanenkoK <emelyanenko.kirill@gmail.com>
2023-11-03 14:43:34 +03:00
SpyCheese
5abfe2337e
Catchain improvements (#698)
* Fix "sgn" in fift; fix marking infinite loops as noterurn in func

* TON-P1-1: Remove unused catchain queries

* TON-P1-15: Avoid synchronization with self

* TON-P1-3, TON-P1-17: Disallow more than one candidate per src per round (to prevent flood), add checks to process_broadcast

* TON-P1-10: Fix fast/slow attempts

* TON-P1-14: Add named constants

* TON-P1-18, TON-P1-19: Alloc temporary memory in the same way as persistent memory

* TON-P1-20: Add comment to choose_blocks_to_approve

* TON-P1-16: Avoid creating two catchain blocks on need_new_block

* TON-P1-8: Add some validation to validator-engine parameters

* TON-P1-6: Don't allow sending the same block many times

Many requests for the same block are not unusual (however, there's no need to answer them all)

* TON-P1-2: Enable prohibiting dependencies from blamed nodes (2.7.5 in CatChain doc), fix processing blame proofs

* Best practices

bp-6: Fix incorrect warning
bp-7: Remove unused code
bp-8: Bring back PerfWarningTimer logging (only when no callback)
bp-9: Remove unnecessary condition
bp-11: Remove commented-out code
bp-13: Divide code in validator-session-state
Adherence to Specification: Fix typo
2023-05-10 12:57:57 +03:00
EmelyanenkoK
47311d6e0e
Improve tweaking for high throughput (#610)
* Option "--disable-ext-msg-broadcast"

* "Get shard out queue size" query

* Move disabling ext msg broadcasts from command-line arguments to config

* Fix compilation error

* Asynchronous store_cell and gc in celldb

* Make GC in celldb work evenly over time

* Increase timeouts for downloading blocks

* Reuse blocks from previous rounds in validator session

* Use Rldp2 in FullNode for downloading persistent states and archives

* Improve logs in download-archive-slice and download-state

* Decrease delay between serializing shards

* Make CellDbIn::load_cell synchronous to avoid interfering with store_cell

---------

Co-authored-by: SpyCheese <mikle98@yandex.ru>
2023-03-15 10:47:35 +03:00
Dan Volkov
4590ed381b
Make funcfiftlib compilation compatible with modern compilers (#618)
* wip: make funcfiftlib compilation compatible with modern compilers

* wip: add methods needed for another compiler

* fix: tdutils port config if emscripten

* feat: func source and realpath callback

* fix: invalid fift compilation exceptions

---------

Co-authored-by: krigga <krigga7@gmail.com>
2023-03-09 17:37:15 +03:00
SpyCheese
1e4eecfdb0
Logs and size limits for incoming queries in FullNodeShard (#601)
* Log incoming queries in FullNodeShard

* Limit size for some queries in FullNodeShard
2023-02-02 10:04:19 +03:00
Andrey Pfau
f66c06143c
Fix pretty print json (remove unnecessary line breaks) (#515)
* add: TON Overflow

* Fix pretty print json (remove unnecessary line breaks)
2022-11-02 21:28:44 +03:00
AlexeyFSL
e2cca03a78
Func and Fift lib for WASM (#455)
* Add social badges

Add telegram, Twitter and Stack Overflow badges in Readme

* update README.md badges

* patch for wasm build

* fix narrowing conversion error for clang compiler

* refactor func code

* funcfift lib implementation

* fix funcfift lib
fix CMakeFile

* fix rvalue missing

* remove unused field from result json

* name fix
remove unused target

* rename

* added script for building funcfiftlib to wasm

* fix json fild names

* fix commit hash for script

* added version function to funcfiftlib

* update commit hash for script

* add realpath fail processing
fix DISABLE_EXCEPTION_CATCHING option

* update hash in script

Co-authored-by: Anthony Tsivarev <tsivarev.a@gmail.com>
Co-authored-by: EmelyanenkoK <emelyanenko.kirill@gmail.com>
Co-authored-by: tolya-yanot <1449561+tolya-yanot@users.noreply.github.com>
2022-09-14 12:36:01 +03:00
EmelyanenkoK
8329a58994
Add GetPerfTimerStats (#451)
* Add getperfwarningtimeraverage query for validator-engine-console

* Fix for getperfwarningtimeraverage query

* getperftimerstats

* Add history max size: 1 hour

* PerfWarningTimer: move callback instead of copy

* PerfWarningTimer: fix move constructor bug

* PerfWarningTimer: fix bug: lifetime of the callback was greater than lifetime of the local variable 'manager' captured by reference

* Fix validate-query.cpp: 'manager' used after it was moved

* PerfWarningTimer: remove logs

* getperftimerstats: write to json file

* getperftimerstatsjson

Co-authored-by: legaii <jgates.ardux@gmail.com>
Co-authored-by: Ivan Siomash <106972486+legaii@users.noreply.github.com>
2022-09-10 12:57:13 +03:00
SpyCheese
b398862cf8 Fix error handling in ServerSocketFd.cpp 2022-06-27 10:22:39 +03:00
OmicronTau
cb31a20206 Add methods to sign and import certificates 2022-05-06 10:42:49 +03:00
Alex Melman
5c992d6a93 Using Howard Hinnant's free, open-source header-only and thread-safe datetime library.
https://howardhinnant.github.io/date/date.html
All logs will display a datetime in format "2021-02-23 12:57:21.1023272" UTC, instead of simply displaying double formatted timestamp.

Update ccpp-linux.yml

Soon github actions will use ubuntu 20.04 as latest. Let's avoid surprised and stick to ubuntu-18.04 fow now.
2021-02-23 14:31:58 +03:00
AleksejMelman
b22611a123 Create ccpp-linux.yaml
Taken from https://github.com/akme/ton/blob/master/.github/workflows/ccpp-linux.yml

Fixing error LNK2019: unresolved external symbol _GetProcessMemoryInfo@12 when building windows desktop client

Fixes compilation error of lite-client on windows

Fixes compilation error:
error C2664: 'std::pair<td::int64,td::int64> std::make_pair<td::int64,td::int64>(_Ty1 &&,_Ty2 &&) noexcept (<expr>)': cannot convert argument 1 from 'const td::uint64' to '_Ty1 &&'

make_pair() has been changed in VS2012 to support a new C++11 feature called move semantics and I suspect that explicitly specifying the types for make_pair() is getting in the way.

The fix should be safe for linux distro as well.

Adding Github action - Win64 ton compile (#14)

* Add github action win64 compile
* manager.cpp - Strange though, but due to this kind of type usage win64 compilation fails. Let's make it more generic.
2021-02-09 11:15:50 +03:00
ton
24dc184a2e validator large state serialization bugfix + tdlib update for clients 2021-01-07 22:48:04 +03:00
ton
f064b1047a pow-testgiver support 2020-07-06 17:07:20 +03:00
ton
4e2624459b integrating the existing state of TON Storage / TON Payments / CPS Fift development branches 2020-05-27 22:10:46 +04:00
ton
eecf05ca59 bugfixes + tonlib update 2020-04-30 15:04:47 +04:00
ton
9f008b129f updated submodules, bugfixes
- added new fift/func code for validator complaint creation
- bugfixes in validator
- updates in tonlib
- new versions of rocksdb/abseil
- hardfork support
2020-04-27 16:01:46 +04:00
ton
be9c34c62d bugfixes 2020-04-10 23:06:01 +04:00
ton
8be3fc99ed bugfixes + decreased archive slice size 2020-04-08 23:49:28 +04:00
ton
a31f8d4424 bugfixed + crypto update
- compiles vs BoringSSL
- config proposal/vote fift code
- bugfixes in catchain
- other small fixes
2020-03-27 18:59:00 +04:00
ton
9f351fc29f emergency update 2020-03-24 03:32:16 +04:00
ton
e27fb1e09c updated vm (breaking compatibility)
- updated vm
- new actor scheduler
- updated tonlib
- updated DNS smartcontract
2020-02-28 14:28:47 +04:00
ton
a73d202ba2 updated func and tonlib 2020-02-15 20:03:17 +04:00
ton
1de39f5d7c bugfixes + doc update 2020-02-08 23:24:49 +04:00
ton
77842f9b63 updated tonlib
- updated tonlib
- updated validator
- updated documentation
- first version of http over rldp proxy
2020-02-06 21:56:46 +04:00
ton
53ec9684bd updated liteserver
- new methods for liteserver/liteclient
- added ADNL/DHT client-only work mode
- fixed crash in ADNL
2020-02-02 16:53:37 +04:00
ton
e30d98eb30 new db
new database
fift/func bugfixes
2019-11-15 18:02:37 +04:00
ton
950e292264 do not use builtin popcnt 2019-11-01 22:15:04 +04:00
ton
3002321eb7 updated tonlib 2019-10-31 14:26:04 +04:00
ton
c860ce3d1e updated smartcontracts
- updated smartcontracts
- updated fullnode database layout
- fixed memory leak in blockchain-explorer
- updated tonlib
2019-10-23 17:43:50 +04:00
ton
b978e27b2f updated smartcontract code
- updated smartcontract code
- fixed bug in liteserver listening socket
- updated documentation
2019-10-14 11:13:23 +04:00
ton
7ea00ebfcf updated tonlib, fixed bugs
updated tonlib
fixed bugs in func
validator: partial support for hardforks
liteserver: support for waitMasterchainBlock prefix
transactions: support for gas flat rate
2019-10-03 17:04:52 +04:00
ton
fd7a8de970 updated tonlib, block routing
- upated tonlib
- fixed bug in message routing
2019-09-28 11:44:38 +04:00
ton
a1e352d894 updated tonlib, new fullnode queries 2019-09-24 01:10:57 +04:00
ton
28df74178c updated tonlib
1. updated tonlib
2. fixed bug in state download
2019-09-22 10:14:09 +04:00
ton
f40822b58a fullnode: support for TCP master/slave replication 2019-09-19 23:17:17 +04:00
ton
13140ddf29 updated block header
1. Updated block header, proofs now contain more data
   Notice, that old proofs may become invalid in the future
2. Fixed message routing
3. Fixed block creator id in block header
4. Support for full proofs in tonlib
5. Support for partial state download
6. Some other bugfixes
2019-09-18 21:46:32 +04:00