This commit contains some parts of https://github.com/ton-blockchain/ton/tree/accelerator
This is auxiliary code that mostly does not change node behavior.
1) Semiprivate overlays and other improvements in overlays code
2) Rename actual_min_split -> monitor_min_split, fix building shard overlays
3) Loading block candidates by block id from DB, fix accept_block after validator restart
4) Cells: ProofStorageStat and changes in CellUsageTree
5) Remove some unused code, other minor changes
* add github action for macOS 14 (arm64, M1)
* add github action (portable) for macOS 14 (arm64, M1)
* rename macOS arm64 output artifact
* Update libsodium on windows
* Compile libsodium
* Update build-windows.bat
* use upgraded libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;
* revert libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;
* use upgraded libsodium 1.0.20; use compiled static libsodium for Windows instead of precompiled;
* fix libsodium version 1.0.19; use compiled static libsodium for Windows instead of precompiled;
* try 1.0.20 libsodium precompiled on github
* try 1.0.18 libsodium precompiled on github
* try windows build on win server 2019
* and use PlatformToolset=v142
* use cmake -G "Visual Studio 16 2019"
* fix path to msvc 2019 on github
* separate github windows build on win server 2019 and build on win server 2022
* Update assembly/native/build-windows-2019.bat
add retry mechanism
Co-authored-by: Dr. Awesome Doge <doge@ton.org>
* add test-emulator; disable test groovy pipeline
* trigger all gh actions
* fix win build
* call test-emulator
* remove usage of rocksdb in fift-lib
* put back some code for test-db
* fix test-emulator
* remove usage of db-path parameter in fift
* some func adjustments
* fix checkout of openssl in fift-func-wasm-build-ubuntu.sh
* typo
* improve wasm build script for quicker turn around
* remove sENVIRONMENT=web,worker for funcfiftlib. will be added later.
* remove sENVIRONMENT=web,worker for funcfiftlib. will be added later.
* remove sENVIRONMENT=web,worker for funcfiftlib. will be added later.
* minor adjustments
* remove -d option in fift; optimize fift-lib usage
* reduce tondb usage
---------
Co-authored-by: neodiX <neodix42@ton.org>
Co-authored-by: Dr. Awesome Doge <doge@ton.org>
Co-authored-by: ms <dungeon666master@protonmail.com>
* Set collator options from validator console
* Fix compilation error in manager-disk
* Defer all messages if out msg queue is too big
* Fix checking queue size in collator
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* Deferred messages and msg metadata
* Store out msg queue size in state
* Add checks for queue processing
1. Collator must process at least one message from AccountDispatchQueue (unless block is full)
2. The first message from a transaction is not counted, it cannot be deferred (unless AccountDispatchQueue is not empty)
* Return msg metadata from LS in listBlockTransactions[Ext]
* Enable new features by capabilities
* Changes in deferred messages
* Process deferred messages via new_msgs in collator
* Rework setting deferred_lt, bring back check_message_processing_order, check order of deferred_lt in validator
* Use have_unprocessed_account_dispatch_queue_ in collator
* Fix setting transaction lt for deferred messages
* Fix lite-client compilation error
* Changes in process_dispatch_queue, rename deferred_lt -> emitted_lt
* Fix compilation error
* Use uint64 for msg queue size
* Add liteServer.getBlockOutMsgQueueSize
* Fix compilation error
* Fix typos in comments
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* add github action for macOS 14 (arm64, M1)
* add github action (portable) for macOS 14 (arm64, M1)
* rename macOS arm64 output artifact
* Colon cannot be used as a path separator for FIFTPATH or -I argument in fift on Windows when absolute paths are used (e.g. C:\path\lib:C:\path\smartcont).
Suggestion to use @ as a new path separator on Windows.
---------
Co-authored-by: neodiX <neodix42@ton.org>
* Utils for writing precompiled contracts
* Precompiled contracts in config, override gas_usage for them
* Add base class for precompiled contracts
* Improve utils for precompiled smc
* Implement GETPRECOMPILEDGAS
* Enable precompiles by flag
* Process null data in PrecompiledSmartContract
* Fix ton_block wasm build
* Fix vm::util::store_(u)long
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* Remove gas_prices_v3, enable new gas limits by GlobalVersion = 5
* Change final date for higher gas limit
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* fix macOS github actions
* fix android tonlib GH action;
* fixing wasm GH action
* strip binaries
* fix randomly failing ubuntu and wasm GH actions
* fix randomly failing ubuntu and wasm GH actions
* revert some changes
* adding more nix scripts and automated native build scripts;
debug static ton compilation
* minor fix
* do not use pkg_config if path specified
* move wasm script, run with sudo action script
* weird, fixing - cp: missing destination file operand after 'assembly/native/build-ubuntu-20.04-shared.sh'
* weird, fixing - cp: missing destination file operand after 'assembly/native/build-ubuntu-20.04-shared.sh'
* ok
* some adjustments for android and win builds
* some adjustments for android and win builds
* moving stripping inside the build script
* access rights handling; adding simple binaries' tests
* make lite-client-common, fift-lib and smc-envelope deliberately static;
add -a (artifacts) flag to build scripts;
* minor wasm build fix;
create separate tonlib android build script;
remove outdated __has_trivial_copy(T)
* add windows build - WIP
* adjust android build;
improve win build;
* adjust sodium paths for android build; use proper compiler for windows build;
* add github windows build auxiliary file
* adjust wasm build
* add portable ubuntu build
* exclude some unstable tests for some time
* compile portable binaries on ubuntu-20.04
* exclude some unstable tests
* include static gsl
* restart builds
* restart builds
* restart builds
* remove libreadline, gsl and blas dependencies in linux build
* add macos build script
* install missing autoconf in macos builds
* enable all tests and see what fails
* enable win tests and restart others
* enable win tests and fix test-smartcont.cpp
* enable win tests
* use clang-16 on mac builds, add blockchain-explorer for ubuntu builds, add portable macos build
* move sudo part outside a build scripts
* move sudo part outside a build scripts
* run llvm install with sudo
* remove libgnutls28-dev before ubuntu static compilation, include blockchain-explorer into artifacts;
remove warning: definition of implicit copy constructor for 'Stat' is deprecated because it has a user-declared copy assignment operator [-Wdeprecated-copy]
* rework wrong decision, put back system gnutls, but compile libmicrohttpd with --disable-https
* add jenkins pipeline sceleton
* WIP jenkins pipeline sceleton
* WIP jenkins pipeline changes
* WIP jenkins pipeline: add stage timout, zip and group artifacts
* WIP jenkins pipeline: macos portable build fix
* WIP jenkins pipeline: wording
* WIP jenkins pipeline: add android tonlib
* WIP jenkins pipeline: add was binaries
* WIP jenkins pipeline: add TOTAL_MEMORY 1.5gb to funcfiftlib wasm linking
* WIP jenkins pipeline: add nix build on linux aarch64
* WIP jenkins pipeline: funcfiftlib compilation fails that 16mb mem is not enough, increase to 32mb
* WIP jenkins pipeline: enable test in nix build
* WIP jenkins pipeline: add linux x86-64 nix build
* WIP jenkins pipeline: include libs in nix build
* WIP jenkins pipeline: include libs in nix build
* WIP jenkins pipeline: include mac nix build
* WIP jenkins pipeline: include mac nix build
* WIP jenkins pipeline: include mac nix build
* WIP jenkins pipeline: include mac nix build
* WIP jenkins pipeline: include mac nix build
* WIP jenkins pipeline: include mac nix build
* WIP jenkins pipeline: nix linux arm64 with openssl 1.1 for now
* WIP jenkins pipeline: working ubuntu arm64 libtonjson
* WIP jenkins pipeline: working ubuntu arm64 libtonjson + minor fix
* WIP jenkins pipeline: working ubuntu arm64 libtonjson + minor fix 2
* WIP jenkins pipeline: merry christmas
* WIP jenkins pipeline: merry christmas 2
* WIP jenkins pipeline: remove native static builds
* WIP jenkins pipeline: enable more tests
* WIP jenkins pipeline: zip artifacts better
* WIP jenkins pipeline: get rid of path in the final zip
* WIP jenkins pipeline: minor fix, include lib and smartcont folders
* WIP jenkins pipeline: minor fix, include lib and smartcont folders into nix artifacts also
* WIP jenkins pipeline: minor fix
* WIP jenkins pipeline: minor fix
* adjust github actions for new nix builds
* cleanup
* cleanup
* cleanup
* cleanup
* rename libtonlibjson.so.0.5 to libtonlibjson.so
* Add TON build instructions to README.md
* simplify
* fix test-tonlib-offline
* set timeout per test of 300 sec
* set timeout per test of 600 sec for non nix builds
* increase test timeout to 900 sec; minor changes
* use MS VS 2022 for win TON compilation; update README.md
* use MS VS 2022 for win TON compilation; update README.md
* change path to MSVC in github workflow
* change path to MSVC in groovy pipeline
* compile ton on win, with msvc 2022 community and enterprise versions
* minor fixes
* improve network tests
* remove TON compilation against macos-11 github runner
* add `choco feature enable -n allowEmptyChecksums` since pkg-config-lite-0.28-1 does not have a checksum
* abort win compilation if 3pp can't be downloaded
* increase test timeout to 30 min
* improving test-catchain
* Set higher gas limit for special accounts, don't add gas from special accounts to block total
* Make removing special accounts from block gas limits enabled by config
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* Improve handling outbound message queues
* Cleanup queue faster
* Calculate queue sizes in background
* Force or limit split/merge depending on queue size
* Increase validate_ref limit for transaction
* Add all changes of public libraries to block size estimation
* Don't crash on timeout in GC
* Don't import external messages when queue is too big
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* Add parameter --celldb-compress-depth to speed up celldb
* Fix collator timeout
* Add block_state_cache
* Adjust state cache ttl
* Don't merge shards when queue is too big
* Decrease lt limit if previous block is too old
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* Improve Collator::opt_msg_queue_cleanup, increase collator timeout
* Disable importing ext msgs if queue is too big
* Extend timeout in collator if previous block is too old
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* Check account size limits in unpack_msg_state
* Don't allow deploying a contract with public libs
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* New TVM instructions
* Remove PREVBLOCKS
* Separate target ton_crypto into TVM-related and -unrelared code
* Add fine for failed "send message"; rework SENDMSG
* Fix include
* Fix bugs, improve action fines
* Disable fines for special accounts
* Handle msg_balance_remaining.grams == null in transaction.cpp
* Bugfixes in SENDMSG
* Fix fee calculation in SENDMSG
* Fix CellStorageStat and transaction.cpp after merge
* SETBOUNCEONACTIONPHASEFAIL instruction
* ADDDIVMOD instructions
* RUNVM, RUNVMX instructions
* Changes in RUNVM
* Tests for adddiv and runvm
* HASHEXT instruction
* Improve opcode-timing
More iterations
Don't measure preliminary run
Remove logs and other excessive operations
Add "error" to output
* Increase RUNVM gas price
* Optimize HASHEXT, adjust gas price
* Add "bounce of action fail" flag to actions
* Stack operations with unlimited arguments
* Ristretto255 instructions
* Adjust gas consumption
* Optional fixed number of return values in RUNVM, fix exception handling
* Adjust gas consumption
* Simplify gas consumption logic
* Support of secp256k1 and sodium libraries in builds (#11)
* add support of secp256k1 library to the builds (linux, win)
* add support of secp256k1 library to the builds (linux, win)
* install secp256k1 via brew
* install libsodium via brew;
change sodium to upper case in FindSodium.cmake
* install libsodium via brew;
change sodium to upper case in FindSodium.cmake
* simplify FindSodium.cmake
* bug fixing
* bug fixing
* bug fixing
* add macro SODIUM_STATIC
* adjust build command for windows
* put back original FindSodium.cmake
* put back original FindSodium.cmake
* fix sodium unzipped path for windows;
add ninja
* fix sodium unzipped path for windows;
add ninja
* fix sodium unzipped path for windows;
add ninja
* Win32 github build for secp256k1
* x64 architecture github build for secp256k1
* fix sodium linking on linux
* enable docker buildx arm64 builds from forked repos
* enable docker buildx arm64 builds from forked repos
* enable docker buildx arm64 builds from forked repos
* adjust mac builds for secp2561k and sodium
* fix tonlib jni generation
* minor fix
* sync fixes across platforms
* add libsodium build script for android and precompiled static libraries
* build tonlib for android (fails)
* FindSodium uppercase
* remove system libsodium for android, use precompiled instead;
specify SECP256K1_INCLUDE_DIR fir mac 12.6
* uppercase sodium
* simplify FindSodium
* fix windows build sodium path;
use ninja for windows
* simplify sodium 2
* adjust windows sodium paths;
add paths to android jni
* add ninja build windows
* add ninja build windows
* add ninja build windows 2
* remove win ninja
* fix 1
* fix 2
* fix win 3
* fix linux compile 3
* fix jni 1
* fix jni 2 and mac
* fix jni 3
* fix jni 4
* fix jni 5
* fix mac 6
* fix mac 7 and jni paths
* fix jni 8
* rework sodium for android
* rework sodium for android
* rework sodium for android 2
* fixed sodium for android 2
* fixed sodium for android 3
* static secp256k1 for android
* add precompiled arm secp256k1
* add precompiled arm secp256k1
* build native-lib with secp256k1 x86-64 (non arm)
* update precompiled with NDK libsecp256k1.a
* update precompiled with NDK libsecp256k1.a
* update precompiled with NDK libsecp256k1.a
* refactor llvm-strip location
* refactor llvm-strip location
* add native-lib.so for armv7a, armv8a
* add native-lib.so for armv7a, armv8a
* test armv7a, armv8a
* armv7a - fails linking on sodium, test -> armv8a
* works x86-64, armv7a - fails linking on sodium, armv8a - fails linking secp256k1 (incompatible with aarch64linux)
* update libpsec256k1, sodium static libs
* test x86 android native-lib
* test armv7 android native-lib
* test armv8 android native-lib
* x86_64 and arm64 android native-lib works
* x86_64 and arm64 android native-lib works
* x86_64 and arm64 android native-lib works
* test armv7 android native-lib
* test all android native-libs
* test all android native-libs
* test all android native-libs
* test all android native-libs - without SodiumAndroid
* test all android native-libs - with FindSodiumAndroid.cmake
* win, with Sodium via SODIUM_DIR
* win, with Sodium via SODIUM_DIR env
* win, with Sodium via SODIUM_DIR env
* win, with Sodium via SODIUM_DIR env and SODIUM_USE_STATIC_LIBS
* win, with Sodium via SODIUM_DIR, SODIUM_USE_STATIC_LIBS and SODIUM_INCLUDE_DIR
* android, with FindSodium
* android, with FindSodium with SODIUM_USE_STATIC_LIBS
* remove if not apple
* target_link_libraries(ton_crypto_core PUBLIC secp256k1)
* android SECP256K1_INCLUDE_DIRS
* android SECP256K1_INCLUDE_DIR
* add libsecp256k1.a/so pre-compiled with ubuntu 22 x86-64
* add libsecp256k1.a/so pre-compiled with ubuntu 22 x86-64
* sodium dirs
* sodium dirs
* sodium dirs
* remove NOT APPLE and SodiumAndroid
* add NOT APPLE and remove SodiumAndroid
* add NOT APPLE and remove SodiumAndroid
* remove build scripts for 18.04, reduce CMakeLists.txt
* remove build scripts for 18.04, reduce CMakeLists.txt
* Fix cas consumption during library load
* Fix fetch_config_params after merge
* Add all ADDDIVMOD ops to Asm.fif
* Save unpaid storage fee to due_payment
* Add "set prev blocks info" to emulator
* Adjusted builds (#13)
* Update flake.nix
Add libsodium
* add libsecp256k1-dev and libsodium-dev into wasm build
* make back emulator a shared library;
put emulator to artifacts;
compile wasm artifacts with sodium and secp256k1.
* add secp256k1 to nix
* compile emulator statically with nix
* compile emulator statically with nix
* compile emulator lib statically with nix
* compile emulator lib statically with nix
* add libemulator to artifacts
* add shared libemulator library to artifacts
* minor release fix
* update set-output commands;
add recent_changelog.md
* releases fixes
* releases fixes, multiline
* releases fixes, multiline
* releases fixes, multiline
* put back multiline changelog
* put back multiline changelog
* ConfigParam 19 (global-id) and GLOBALID instruction
* Fix gas consumption in HASHEXT
* Add blst library
* Add bls instructions
* Allow passing long code to opcode-timing
* Add bls testcase
* More BLS instructions
* Fix tests, add bls tests
* Add more bls tests
* Improve some bls operations
* Adjust some BLS gas prices
* Adjust BLS gas prices
* Enable __BLST_PORTABLE__ flag only if PORTABLE flag is set
* Add tests for BLS_PAIRING
* GASCONSUMED instruction
* Fix compilation against docker with blst library; (#14)
* fix compilation against docker with blst library;
add precompiled libblst.a to android builds
* minor fix
* Adjust BLKSWX gas
* Fix comparison with NAN
* Allow arbitrary integers for scalars in ristretto multiplication, fix test
* Adjust nix builds according to PR 694 (#15)
* integrate and test PR-694
* integrate and test PR-694, test 2
* Add P256_CHKSIGN (secp256r1)
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
Co-authored-by: neodiX42 <namlem@gmail.com>
* Fix "sgn" in fift; fix marking infinite loops as noterurn in func
* TON-P1-1: Remove unused catchain queries
* TON-P1-15: Avoid synchronization with self
* TON-P1-3, TON-P1-17: Disallow more than one candidate per src per round (to prevent flood), add checks to process_broadcast
* TON-P1-10: Fix fast/slow attempts
* TON-P1-14: Add named constants
* TON-P1-18, TON-P1-19: Alloc temporary memory in the same way as persistent memory
* TON-P1-20: Add comment to choose_blocks_to_approve
* TON-P1-16: Avoid creating two catchain blocks on need_new_block
* TON-P1-8: Add some validation to validator-engine parameters
* TON-P1-6: Don't allow sending the same block many times
Many requests for the same block are not unusual (however, there's no need to answer them all)
* TON-P1-2: Enable prohibiting dependencies from blamed nodes (2.7.5 in CatChain doc), fix processing blame proofs
* Best practices
bp-6: Fix incorrect warning
bp-7: Remove unused code
bp-8: Bring back PerfWarningTimer logging (only when no callback)
bp-9: Remove unnecessary condition
bp-11: Remove commented-out code
bp-13: Divide code in validator-session-state
Adherence to Specification: Fix typo
* Fix UB in func
* Improve optimizing int consts and unused variables in FunC
* Bump funC version to 0.4.3
* Fix analyzing repeat loop
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* Option "--disable-ext-msg-broadcast"
* "Get shard out queue size" query
* Move disabling ext msg broadcasts from command-line arguments to config
* Fix compilation error
* Asynchronous store_cell and gc in celldb
* Make GC in celldb work evenly over time
* Increase timeouts for downloading blocks
* Reuse blocks from previous rounds in validator session
* Use Rldp2 in FullNode for downloading persistent states and archives
* Improve logs in download-archive-slice and download-state
* Decrease delay between serializing shards
* Make CellDbIn::load_cell synchronous to avoid interfering with store_cell
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
* Verify proof for method blocks.getTransactions
* check completeness of response
* fix start_lt
* fix mode & 128, check bTxes->ids_ out of bounds
* Improve gitactions; separate cpp-check (#346)
* Use ninja build tool and compile blockchain-explorer
Ninja builds TON much faster;
* Use clang instead of gcc
* remove blockchain-explorer
since target not found on github action
* move ccpcheck to other gitaction
* run nativelib-java only against wallets branch for now
* rename gitaction
* Update windows2019x64-tonlib-java.yml
* Update windows2019x64-tonlib-java.yml
* Update macos-10.15-tonlib-java.yml
* Update windows2019x64-tonlib-java.yml
* Update windows2019x64-tonlib-java.yml
* rebase
* update tlo's
* Revert "Improve gitactions; separate cpp-check (#346)"
This reverts commit bd1d96e6d391e48840d81cfcf10d2692848e504e.
* add checks, simplify ls response
* Revert workflows
* Add verifying proofs
* fix win build
---------
Co-authored-by: neodiX42 <namlem@gmail.com>
