TVM Upgrade (#686)
* New TVM instructions
* Remove PREVBLOCKS
* Separate target ton_crypto into TVM-related and -unrelared code
* Add fine for failed "send message"; rework SENDMSG
* Fix include
* Fix bugs, improve action fines
* Disable fines for special accounts
* Handle msg_balance_remaining.grams == null in transaction.cpp
* Bugfixes in SENDMSG
* Fix fee calculation in SENDMSG
* Fix CellStorageStat and transaction.cpp after merge
* SETBOUNCEONACTIONPHASEFAIL instruction
* ADDDIVMOD instructions
* RUNVM, RUNVMX instructions
* Changes in RUNVM
* Tests for adddiv and runvm
* HASHEXT instruction
* Improve opcode-timing
More iterations
Don't measure preliminary run
Remove logs and other excessive operations
Add "error" to output
* Increase RUNVM gas price
* Optimize HASHEXT, adjust gas price
* Add "bounce of action fail" flag to actions
* Stack operations with unlimited arguments
* Ristretto255 instructions
* Adjust gas consumption
* Optional fixed number of return values in RUNVM, fix exception handling
* Adjust gas consumption
* Simplify gas consumption logic
* Support of secp256k1 and sodium libraries in builds (#11)
* add support of secp256k1 library to the builds (linux, win)
* add support of secp256k1 library to the builds (linux, win)
* install secp256k1 via brew
* install libsodium via brew;
change sodium to upper case in FindSodium.cmake
* install libsodium via brew;
change sodium to upper case in FindSodium.cmake
* simplify FindSodium.cmake
* bug fixing
* bug fixing
* bug fixing
* add macro SODIUM_STATIC
* adjust build command for windows
* put back original FindSodium.cmake
* put back original FindSodium.cmake
* fix sodium unzipped path for windows;
add ninja
* fix sodium unzipped path for windows;
add ninja
* fix sodium unzipped path for windows;
add ninja
* Win32 github build for secp256k1
* x64 architecture github build for secp256k1
* fix sodium linking on linux
* enable docker buildx arm64 builds from forked repos
* enable docker buildx arm64 builds from forked repos
* enable docker buildx arm64 builds from forked repos
* adjust mac builds for secp2561k and sodium
* fix tonlib jni generation
* minor fix
* sync fixes across platforms
* add libsodium build script for android and precompiled static libraries
* build tonlib for android (fails)
* FindSodium uppercase
* remove system libsodium for android, use precompiled instead;
specify SECP256K1_INCLUDE_DIR fir mac 12.6
* uppercase sodium
* simplify FindSodium
* fix windows build sodium path;
use ninja for windows
* simplify sodium 2
* adjust windows sodium paths;
add paths to android jni
* add ninja build windows
* add ninja build windows
* add ninja build windows 2
* remove win ninja
* fix 1
* fix 2
* fix win 3
* fix linux compile 3
* fix jni 1
* fix jni 2 and mac
* fix jni 3
* fix jni 4
* fix jni 5
* fix mac 6
* fix mac 7 and jni paths
* fix jni 8
* rework sodium for android
* rework sodium for android
* rework sodium for android 2
* fixed sodium for android 2
* fixed sodium for android 3
* static secp256k1 for android
* add precompiled arm secp256k1
* add precompiled arm secp256k1
* build native-lib with secp256k1 x86-64 (non arm)
* update precompiled with NDK libsecp256k1.a
* update precompiled with NDK libsecp256k1.a
* update precompiled with NDK libsecp256k1.a
* refactor llvm-strip location
* refactor llvm-strip location
* add native-lib.so for armv7a, armv8a
* add native-lib.so for armv7a, armv8a
* test armv7a, armv8a
* armv7a - fails linking on sodium, test -> armv8a
* works x86-64, armv7a - fails linking on sodium, armv8a - fails linking secp256k1 (incompatible with aarch64linux)
* update libpsec256k1, sodium static libs
* test x86 android native-lib
* test armv7 android native-lib
* test armv8 android native-lib
* x86_64 and arm64 android native-lib works
* x86_64 and arm64 android native-lib works
* x86_64 and arm64 android native-lib works
* test armv7 android native-lib
* test all android native-libs
* test all android native-libs
* test all android native-libs
* test all android native-libs - without SodiumAndroid
* test all android native-libs - with FindSodiumAndroid.cmake
* win, with Sodium via SODIUM_DIR
* win, with Sodium via SODIUM_DIR env
* win, with Sodium via SODIUM_DIR env
* win, with Sodium via SODIUM_DIR env and SODIUM_USE_STATIC_LIBS
* win, with Sodium via SODIUM_DIR, SODIUM_USE_STATIC_LIBS and SODIUM_INCLUDE_DIR
* android, with FindSodium
* android, with FindSodium with SODIUM_USE_STATIC_LIBS
* remove if not apple
* target_link_libraries(ton_crypto_core PUBLIC secp256k1)
* android SECP256K1_INCLUDE_DIRS
* android SECP256K1_INCLUDE_DIR
* add libsecp256k1.a/so pre-compiled with ubuntu 22 x86-64
* add libsecp256k1.a/so pre-compiled with ubuntu 22 x86-64
* sodium dirs
* sodium dirs
* sodium dirs
* remove NOT APPLE and SodiumAndroid
* add NOT APPLE and remove SodiumAndroid
* add NOT APPLE and remove SodiumAndroid
* remove build scripts for 18.04, reduce CMakeLists.txt
* remove build scripts for 18.04, reduce CMakeLists.txt
* Fix cas consumption during library load
* Fix fetch_config_params after merge
* Add all ADDDIVMOD ops to Asm.fif
* Save unpaid storage fee to due_payment
* Add "set prev blocks info" to emulator
* Adjusted builds (#13)
* Update flake.nix
Add libsodium
* add libsecp256k1-dev and libsodium-dev into wasm build
* make back emulator a shared library;
put emulator to artifacts;
compile wasm artifacts with sodium and secp256k1.
* add secp256k1 to nix
* compile emulator statically with nix
* compile emulator statically with nix
* compile emulator lib statically with nix
* compile emulator lib statically with nix
* add libemulator to artifacts
* add shared libemulator library to artifacts
* minor release fix
* update set-output commands;
add recent_changelog.md
* releases fixes
* releases fixes, multiline
* releases fixes, multiline
* releases fixes, multiline
* put back multiline changelog
* put back multiline changelog
* ConfigParam 19 (global-id) and GLOBALID instruction
* Fix gas consumption in HASHEXT
* Add blst library
* Add bls instructions
* Allow passing long code to opcode-timing
* Add bls testcase
* More BLS instructions
* Fix tests, add bls tests
* Add more bls tests
* Improve some bls operations
* Adjust some BLS gas prices
* Adjust BLS gas prices
* Enable __BLST_PORTABLE__ flag only if PORTABLE flag is set
* Add tests for BLS_PAIRING
* GASCONSUMED instruction
* Fix compilation against docker with blst library; (#14)
* fix compilation against docker with blst library;
add precompiled libblst.a to android builds
* minor fix
* Adjust BLKSWX gas
* Fix comparison with NAN
* Allow arbitrary integers for scalars in ristretto multiplication, fix test
* Adjust nix builds according to PR 694 (#15)
* integrate and test PR-694
* integrate and test PR-694, test 2
* Add P256_CHKSIGN (secp256r1)
---------
Co-authored-by: SpyCheese <mikle98@yandex.ru>
Co-authored-by: neodiX42 <namlem@gmail.com>
2023-05-24 18:14:13 +00:00
# Global versions
Global version is a parameter specified in `ConfigParam 8` ([block.tlb](https://github.com/ton-blockchain/ton/blob/master/crypto/block/block.tlb#L595)).
Various features are enabled depending on the global version.
## Version 4
### New TVM instructions
* `PREVMCBLOCKS` , `PREVKEYBLOCK`
* `GLOBALID`
* `HASHEXT(A)(R)`
* `ECRECOVER`
* `SENDMSG`
* `RUNVM` , `RUNVMX`
* `GASCONSUMED`
* `RIST255_...` instructions
* `BLS_...` instructions
* `P256_CHKSIGNS` , `P256_CHKSIGNU`
### Division
[Division instruction ](https://ton.org/docs/learn/tvm-instructions/instructions#52-division ) can add a number to the
intermediate value before division (e.g. `(xy+w)/z` ).
### Stack operations
* Arguments of `PICK` , `ROLL` , `ROLLREV` , `BLKSWX` , `REVX` , `DROPX` , `XCHGX` , `CHKDEPTH` , `ONLYTOPX` , `ONLYX` are now unlimited.
* `ROLL` , `ROLLREV` , `BLKSWX` , `REVX` , `ONLYTOPX` consume more gas when arguments are big.
### c7 tuple
**c7** tuple extended from 10 to 14 elements:
* **10**: code of the smart contract.
* **11**: value of the incoming message.
* **12**: fees collected in the storage phase.
* **13**: information about previous blocks.
### Action phase
* If "send message" action fails, the account is required to pay for processing cells of the message.
* Flag +16 in actions "Send message", "Reserve", "Change library" causes bounce if action fails.
### Storage phase
2024-01-21 09:59:59 +00:00
* Unpaid storage fee is now saved to `due_payment`
## Version 5
2024-01-24 10:05:22 +00:00
### Gas limits
2024-01-21 09:59:59 +00:00
Version 5 enables higher gas limits for special contracts.
* Gas limit for all transactions on special contracts is set to `special_gas_limit` from `ConfigParam 20` (which is 35M at the moment of writing).
2024-01-24 10:05:22 +00:00
Previously only ticktock transactions had this limit, while ordinary transactions had a default limit of `gas_limit` gas (1M).
2024-01-21 09:59:59 +00:00
* Gas usage of special contracts is not taken into account when checking block limits. This allows keeping masterchain block limits low
while having high gas limits for elector.
* Gas limit on `EQD_v9j1rlsuHHw2FIhcsCFFSD367ldfDdCKcsNmNpIRzUlu` is increased to `special_gas_limit * 2` until 2024-02-29.
2024-01-24 10:05:22 +00:00
See [this post ](https://t.me/tonstatus/88 ) for details.
### Loading libraries
* Loading "nested libraries" (i.e. a library cell that points to another library cell) throws an exception.
* Loading a library consumes gas for cell load only once (for the library cell), not twice (both for the library cell and the cell in the library).
2024-01-26 12:43:53 +00:00
* `XLOAD` now works differently. When it takes a library cell, it returns the cell that it points to. This allows loading "nested libraries", if needed.
## Version 6
### c7 tuple
2024-03-05 13:54:49 +00:00
**c7** tuple extended from 14 to 17 elements:
2024-01-29 13:38:42 +00:00
* **14**: tuple that contains some config parameters as cell slices. If the parameter is absent from the config, the value is null. Asm opcode: `UNPACKEDCONFIGTUPLE` .
* **0**: `StoragePrices` from `ConfigParam 18` . Not the whole dict, but only the one StoragePrices entry (one which corresponds to the current time).
* **1**: `ConfigParam 19` (global id).
* **2**: `ConfigParam 20` (mc gas prices).
* **3**: `ConfigParam 21` (gas prices).
* **4**: `ConfigParam 24` (mc fwd fees).
* **5**: `ConfigParam 25` (fwd fees).
* **6**: `ConfigParam 43` (size limits).
* **15**: "[due payment](https://github.com/ton-blockchain/ton/blob/8a9ff339927b22b72819c5125428b70c406da631/crypto/block/block.tlb#L237)" - current debt for storage fee (nanotons). Asm opcode: `DUEPAYMENT` .
2024-03-05 13:54:49 +00:00
* **16**: "precompiled gas usage" - gas usage for the current contract if it is precompiled (see `ConfigParam 45` ), `null` otherwise. Asm opcode: `GETPRECOMPILEDGAS` .
2024-01-26 12:43:53 +00:00
### New TVM instructions
2024-01-26 15:24:39 +00:00
#### Fee calculation
2024-01-29 13:38:42 +00:00
* `GETGASFEE` (`gas_used is_mc - price`) - calculates gas fee.
* `GETSTORAGEFEE` (`cells bits seconds is_mc - price`) - calculates storage fees (only current StoragePrices entry is used).
* `GETFORWARDFEE` (`cells bits is_mc - price`) - calculates forward fee.
2024-03-05 13:54:49 +00:00
* `GETPRECOMPILEDGAS` (`- x`) - returns gas usage for the current contract if it is precompiled, `null` otherwise.
2024-01-29 13:38:42 +00:00
* `GETORIGINALFWDFEE` (`fwd_fee is_mc - orig_fwd_fee`) - calculate `fwd_fee * 2^16 / first_frac` . Can be used to get the original `fwd_fee` of the message.
* `GETGASFEESIMPLE` (`gas_used is_mc - price`) - same as `GETGASFEE` , but without flat price (just `(gas_used * price) / 2^16` ).
* `GETFORWARDFEESIMPLE` (`cells bits is_mc - price`) - same as `GETFORWARDFEE` , but without lump price (just `(bits*bit_price + cells*cell_price) / 2^16` ).
2024-01-26 15:24:39 +00:00
2024-01-26 12:43:53 +00:00
`gas_used` , `cells` , `bits` , `time_delta` are integers in range `0..2^63-1` .
2024-01-26 15:24:39 +00:00
#### Cell operations
Operations for working with Merkle proofs, where cells can have non-zero level and multiple hashes.
* `CLEVEL` (`cell - level`) - returns level of the cell.
* `CLEVELMASK` (`cell - level_mask`) - returns level mask of the cell.
* `i CHASHI` (`cell - hash`) - returns `i` th hash of the cell.
* `i CDEPTHI` (`cell - depth`) - returns `i` th depth of the cell.
* `CHASHIX` (`cell i - hash`) - returns `i` th hash of the cell.
* `CDEPTHIX` (`cell i - depth`) - returns `i` th depth of the cell.
`i` is in range `0..3` .
2024-01-26 12:43:53 +00:00
### Other changes
* `GLOBALID` gets `ConfigParam 19` from the tuple, not from the config dict. This decreases gas usage.
2024-06-11 12:08:08 +00:00
* `SENDMSG` gets `ConfigParam 24/25` (message prices) from the tuple, not from the config dict, and also uses `ConfigParam 43` to get max_msg_cells.
## Version 7
[Explicitly nullify ](https://github.com/ton-blockchain/ton/pull/957/files ) `due_payment` after due reimbursment.
## Version 8
- Check mode on invalid `action_send_msg` . Ignore action if `IGNORE_ERROR` (+2) bit is set, bounce if `BOUNCE_ON_FAIL` (+16) bit is set.
- Slightly change random seed generation to fix mix of `addr_rewrite` and `addr` .
- Fill in `skipped_actions` for both invalid and valid messages with `IGNORE_ERROR` mode that can't be sent.
- Allow unfreeze through external messages.
2024-09-20 12:16:46 +00:00
- Don't use user-provided `fwd_fee` and `ihr_fee` for internal messages.
## Version 9
- Fix `RAWRESERVE` action with flag `4` (use original balance of the account) by explicitly setting `original_balance` to `balance - msg_balance_remaining` .
2024-09-23 09:09:16 +00:00
- Previously it did not work if storage fee was greater than the original balance.
- Jumps to nested continuations of depth more than 8 consume 1 gas for eact subsequent continuation (this does not affect most of TVM code).