mirror of
https://github.com/ton-blockchain/ton
synced 2025-03-09 15:40:10 +00:00
TVM Upgrade (#686)
* New TVM instructions * Remove PREVBLOCKS * Separate target ton_crypto into TVM-related and -unrelared code * Add fine for failed "send message"; rework SENDMSG * Fix include * Fix bugs, improve action fines * Disable fines for special accounts * Handle msg_balance_remaining.grams == null in transaction.cpp * Bugfixes in SENDMSG * Fix fee calculation in SENDMSG * Fix CellStorageStat and transaction.cpp after merge * SETBOUNCEONACTIONPHASEFAIL instruction * ADDDIVMOD instructions * RUNVM, RUNVMX instructions * Changes in RUNVM * Tests for adddiv and runvm * HASHEXT instruction * Improve opcode-timing More iterations Don't measure preliminary run Remove logs and other excessive operations Add "error" to output * Increase RUNVM gas price * Optimize HASHEXT, adjust gas price * Add "bounce of action fail" flag to actions * Stack operations with unlimited arguments * Ristretto255 instructions * Adjust gas consumption * Optional fixed number of return values in RUNVM, fix exception handling * Adjust gas consumption * Simplify gas consumption logic * Support of secp256k1 and sodium libraries in builds (#11) * add support of secp256k1 library to the builds (linux, win) * add support of secp256k1 library to the builds (linux, win) * install secp256k1 via brew * install libsodium via brew; change sodium to upper case in FindSodium.cmake * install libsodium via brew; change sodium to upper case in FindSodium.cmake * simplify FindSodium.cmake * bug fixing * bug fixing * bug fixing * add macro SODIUM_STATIC * adjust build command for windows * put back original FindSodium.cmake * put back original FindSodium.cmake * fix sodium unzipped path for windows; add ninja * fix sodium unzipped path for windows; add ninja * fix sodium unzipped path for windows; add ninja * Win32 github build for secp256k1 * x64 architecture github build for secp256k1 * fix sodium linking on linux * enable docker buildx arm64 builds from forked repos * enable docker buildx arm64 builds from forked repos * enable docker buildx arm64 builds from forked repos * adjust mac builds for secp2561k and sodium * fix tonlib jni generation * minor fix * sync fixes across platforms * add libsodium build script for android and precompiled static libraries * build tonlib for android (fails) * FindSodium uppercase * remove system libsodium for android, use precompiled instead; specify SECP256K1_INCLUDE_DIR fir mac 12.6 * uppercase sodium * simplify FindSodium * fix windows build sodium path; use ninja for windows * simplify sodium 2 * adjust windows sodium paths; add paths to android jni * add ninja build windows * add ninja build windows * add ninja build windows 2 * remove win ninja * fix 1 * fix 2 * fix win 3 * fix linux compile 3 * fix jni 1 * fix jni 2 and mac * fix jni 3 * fix jni 4 * fix jni 5 * fix mac 6 * fix mac 7 and jni paths * fix jni 8 * rework sodium for android * rework sodium for android * rework sodium for android 2 * fixed sodium for android 2 * fixed sodium for android 3 * static secp256k1 for android * add precompiled arm secp256k1 * add precompiled arm secp256k1 * build native-lib with secp256k1 x86-64 (non arm) * update precompiled with NDK libsecp256k1.a * update precompiled with NDK libsecp256k1.a * update precompiled with NDK libsecp256k1.a * refactor llvm-strip location * refactor llvm-strip location * add native-lib.so for armv7a, armv8a * add native-lib.so for armv7a, armv8a * test armv7a, armv8a * armv7a - fails linking on sodium, test -> armv8a * works x86-64, armv7a - fails linking on sodium, armv8a - fails linking secp256k1 (incompatible with aarch64linux) * update libpsec256k1, sodium static libs * test x86 android native-lib * test armv7 android native-lib * test armv8 android native-lib * x86_64 and arm64 android native-lib works * x86_64 and arm64 android native-lib works * x86_64 and arm64 android native-lib works * test armv7 android native-lib * test all android native-libs * test all android native-libs * test all android native-libs * test all android native-libs - without SodiumAndroid * test all android native-libs - with FindSodiumAndroid.cmake * win, with Sodium via SODIUM_DIR * win, with Sodium via SODIUM_DIR env * win, with Sodium via SODIUM_DIR env * win, with Sodium via SODIUM_DIR env and SODIUM_USE_STATIC_LIBS * win, with Sodium via SODIUM_DIR, SODIUM_USE_STATIC_LIBS and SODIUM_INCLUDE_DIR * android, with FindSodium * android, with FindSodium with SODIUM_USE_STATIC_LIBS * remove if not apple * target_link_libraries(ton_crypto_core PUBLIC secp256k1) * android SECP256K1_INCLUDE_DIRS * android SECP256K1_INCLUDE_DIR * add libsecp256k1.a/so pre-compiled with ubuntu 22 x86-64 * add libsecp256k1.a/so pre-compiled with ubuntu 22 x86-64 * sodium dirs * sodium dirs * sodium dirs * remove NOT APPLE and SodiumAndroid * add NOT APPLE and remove SodiumAndroid * add NOT APPLE and remove SodiumAndroid * remove build scripts for 18.04, reduce CMakeLists.txt * remove build scripts for 18.04, reduce CMakeLists.txt * Fix cas consumption during library load * Fix fetch_config_params after merge * Add all ADDDIVMOD ops to Asm.fif * Save unpaid storage fee to due_payment * Add "set prev blocks info" to emulator * Adjusted builds (#13) * Update flake.nix Add libsodium * add libsecp256k1-dev and libsodium-dev into wasm build * make back emulator a shared library; put emulator to artifacts; compile wasm artifacts with sodium and secp256k1. * add secp256k1 to nix * compile emulator statically with nix * compile emulator statically with nix * compile emulator lib statically with nix * compile emulator lib statically with nix * add libemulator to artifacts * add shared libemulator library to artifacts * minor release fix * update set-output commands; add recent_changelog.md * releases fixes * releases fixes, multiline * releases fixes, multiline * releases fixes, multiline * put back multiline changelog * put back multiline changelog * ConfigParam 19 (global-id) and GLOBALID instruction * Fix gas consumption in HASHEXT * Add blst library * Add bls instructions * Allow passing long code to opcode-timing * Add bls testcase * More BLS instructions * Fix tests, add bls tests * Add more bls tests * Improve some bls operations * Adjust some BLS gas prices * Adjust BLS gas prices * Enable __BLST_PORTABLE__ flag only if PORTABLE flag is set * Add tests for BLS_PAIRING * GASCONSUMED instruction * Fix compilation against docker with blst library; (#14) * fix compilation against docker with blst library; add precompiled libblst.a to android builds * minor fix * Adjust BLKSWX gas * Fix comparison with NAN * Allow arbitrary integers for scalars in ristretto multiplication, fix test * Adjust nix builds according to PR 694 (#15) * integrate and test PR-694 * integrate and test PR-694, test 2 * Add P256_CHKSIGN (secp256r1) --------- Co-authored-by: SpyCheese <mikle98@yandex.ru> Co-authored-by: neodiX42 <namlem@gmail.com>
This commit is contained in:
EmelyanenkoK
GitHub
parent
d6b11d9d36
commit
9f93888cf4
374 changed files with 25800 additions and 387 deletions
|
|
@ -8,16 +8,22 @@
|
|||
#include "common/bigint.hpp"
|
||||
|
||||
#include "td/utils/base64.h"
|
||||
#include "td/utils/tests.h"
|
||||
#include "td/utils/ScopeGuard.h"
|
||||
#include "td/utils/StringBuilder.h"
|
||||
#include "td/utils/Timer.h"
|
||||
|
||||
td::Ref<vm::Cell> to_cell(const unsigned char *buff, int bits) {
|
||||
td::Ref<vm::Cell> to_cell(td::Slice s) {
|
||||
if (s.size() >= 4 && s.substr(0, 4) == "boc:") {
|
||||
s.remove_prefix(4);
|
||||
auto boc = td::base64_decode(s).move_as_ok();
|
||||
return vm::std_boc_deserialize(boc).move_as_ok();
|
||||
}
|
||||
unsigned char buff[128];
|
||||
const int bits = (int)td::bitstring::parse_bitstring_hex_literal(buff, sizeof(buff), s.begin(), s.end());
|
||||
CHECK(bits >= 0);
|
||||
return vm::CellBuilder().store_bits(buff, bits, 0).finalize();
|
||||
}
|
||||
|
||||
long double timingBaseline;
|
||||
|
||||
typedef struct {
|
||||
long double mean;
|
||||
long double stddev;
|
||||
|
|
@ -56,56 +62,60 @@ typedef struct {
|
|||
bool errored;
|
||||
} runtimeStats;
|
||||
|
||||
runInfo time_run_vm(td::Slice command) {
|
||||
unsigned char buff[128];
|
||||
const int bits = (int)td::bitstring::parse_bitstring_hex_literal(buff, sizeof(buff), command.begin(), command.end());
|
||||
CHECK(bits >= 0);
|
||||
|
||||
const auto cell = to_cell(buff, bits);
|
||||
|
||||
vm::Stack prepare_stack(td::Slice command) {
|
||||
const auto cell = to_cell(command);
|
||||
vm::init_op_cp0();
|
||||
vm::DictionaryBase::get_empty_dictionary();
|
||||
|
||||
class Logger : public td::LogInterface {
|
||||
public:
|
||||
void append(td::CSlice slice) override {
|
||||
res.append(slice.data(), slice.size());
|
||||
}
|
||||
std::string res;
|
||||
};
|
||||
static Logger logger;
|
||||
logger.res = "";
|
||||
td::set_log_fatal_error_callback([](td::CSlice message) { td::default_log_interface->append(logger.res); });
|
||||
vm::VmLog log{&logger, td::LogOptions::plain()};
|
||||
log.log_options.level = 4;
|
||||
log.log_options.fix_newlines = true;
|
||||
log.log_mask |= vm::VmLog::DumpStack;
|
||||
|
||||
vm::Stack stack;
|
||||
try {
|
||||
vm::GasLimits gas_limit(10000, 10000);
|
||||
|
||||
std::clock_t cStart = std::clock();
|
||||
vm::GasLimits gas_limit;
|
||||
int ret = vm::run_vm_code(vm::load_cell_slice_ref(cell), stack, 0 /*flags*/, nullptr /*data*/,
|
||||
std::move(log) /*VmLog*/, nullptr, &gas_limit);
|
||||
vm::VmLog{}, nullptr, &gas_limit, {}, {}, nullptr, 4);
|
||||
CHECK(ret == 0);
|
||||
} catch (...) {
|
||||
LOG(FATAL) << "catch unhandled exception";
|
||||
}
|
||||
return stack;
|
||||
}
|
||||
|
||||
runInfo time_run_vm(td::Slice command, td::Ref<vm::Stack> stack) {
|
||||
const auto cell = to_cell(command);
|
||||
vm::init_op_cp0();
|
||||
vm::DictionaryBase::get_empty_dictionary();
|
||||
CHECK(stack.is_unique());
|
||||
try {
|
||||
vm::GasLimits gas_limit;
|
||||
vm::VmState vm{vm::load_cell_slice_ref(cell), std::move(stack), gas_limit, 0, {}, vm::VmLog{}, {}, {}};
|
||||
vm.set_global_version(4);
|
||||
std::clock_t cStart = std::clock();
|
||||
int ret = ~vm.run();
|
||||
std::clock_t cEnd = std::clock();
|
||||
const auto time = (1000.0 * static_cast<long double>(cEnd - cStart) / CLOCKS_PER_SEC) - timingBaseline;
|
||||
return {time >= 0 ? time : 0, gas_limit.gas_consumed(), ret};
|
||||
const auto time = (1000.0 * static_cast<long double>(cEnd - cStart) / CLOCKS_PER_SEC);
|
||||
return {time >= 0 ? time : 0, vm.gas_consumed(), ret};
|
||||
} catch (...) {
|
||||
LOG(FATAL) << "catch unhandled exception";
|
||||
return {-1, -1, 1};
|
||||
}
|
||||
}
|
||||
|
||||
runtimeStats averageRuntime(td::Slice command) {
|
||||
const size_t samples = 5000;
|
||||
runtimeStats averageRuntime(td::Slice command, const vm::Stack& stack) {
|
||||
size_t samples = 100000;
|
||||
runInfo total;
|
||||
std::vector<runInfo> values;
|
||||
values.reserve(samples);
|
||||
for(size_t i=0; i<samples; ++i) {
|
||||
const auto value = time_run_vm(command);
|
||||
td::Timer t0;
|
||||
for(size_t i = 0; i < samples; ++i) {
|
||||
const auto value_empty = time_run_vm(td::Slice(""), td::Ref<vm::Stack>(true, stack));
|
||||
const auto value_code = time_run_vm(command, td::Ref<vm::Stack>(true, stack));
|
||||
runInfo value{value_code.runtime - value_empty.runtime, value_code.gasUsage - value_empty.gasUsage,
|
||||
value_code.vmReturnCode};
|
||||
values.push_back(value);
|
||||
total += value;
|
||||
if (t0.elapsed() > 2.0 && i + 1 >= 20) {
|
||||
samples = i + 1;
|
||||
values.resize(samples);
|
||||
break;
|
||||
}
|
||||
}
|
||||
const auto runtimeMean = total.runtime / static_cast<long double>(samples);
|
||||
const auto gasMean = static_cast<long double>(total.gasUsage) / static_cast<long double>(samples);
|
||||
|
|
@ -120,42 +130,39 @@ runtimeStats averageRuntime(td::Slice command) {
|
|||
errored = errored || value.errored();
|
||||
}
|
||||
return {
|
||||
{runtimeMean, sqrt(runtimeDiffSum / static_cast<long double>(samples))},
|
||||
{gasMean, sqrt(gasDiffSum / static_cast<long double>(samples))},
|
||||
{runtimeMean, sqrtl(runtimeDiffSum / static_cast<long double>(samples))},
|
||||
{gasMean, sqrtl(gasDiffSum / static_cast<long double>(samples))},
|
||||
errored
|
||||
};
|
||||
}
|
||||
|
||||
runtimeStats timeInstruction(const std::string& setupCode, const std::string& toMeasure) {
|
||||
const auto setupCodeTime = averageRuntime(setupCode);
|
||||
const auto totalCodeTime = averageRuntime(setupCode + toMeasure);
|
||||
return {
|
||||
{totalCodeTime.runtime.mean - setupCodeTime.runtime.mean, totalCodeTime.runtime.stddev},
|
||||
{totalCodeTime.gasUsage.mean - setupCodeTime.gasUsage.mean, totalCodeTime.gasUsage.stddev},
|
||||
false
|
||||
};
|
||||
vm::Stack stack = prepare_stack(setupCode);
|
||||
return averageRuntime(toMeasure, stack);
|
||||
}
|
||||
|
||||
int main(int argc, char** argv) {
|
||||
SET_VERBOSITY_LEVEL(verbosity_ERROR);
|
||||
if(argc != 2 && argc != 3) {
|
||||
std::cerr <<
|
||||
"This utility compares the timing of VM execution against the gas used.\n"
|
||||
"It can be used to discover opcodes or opcode sequences that consume an "
|
||||
"inordinate amount of computational resources relative to their gas cost.\n"
|
||||
"\n"
|
||||
"The utility expects two command line arguments, each a hex string: \n"
|
||||
"The utility expects two command line arguments: \n"
|
||||
"The TVM code used to set up the stack and VM state followed by the TVM code to measure.\n"
|
||||
"For example, to test the DIVMODC opcode:\n"
|
||||
"\t$ " << argv[0] << " 80FF801C A90E 2>/dev/null\n"
|
||||
"\tOPCODE,runtime mean,runtime stddev,gas mean,gas stddev\n"
|
||||
"\tA90E,0.0066416,0.00233496,26,0\n"
|
||||
"\n"
|
||||
"Usage: " << argv[0] <<
|
||||
" [TVM_SETUP_BYTECODE_HEX] TVM_BYTECODE_HEX" << std::endl << std::endl;
|
||||
"Usage: " << argv[0] << " [TVM_SETUP_BYTECODE] TVM_BYTECODE\n"
|
||||
"\tBYTECODE is either:\n"
|
||||
"\t1. hex-encoded string (e.g. A90E for DIVMODC)\n"
|
||||
"\t2. boc:<serialized boc in base64> (e.g. boc:te6ccgEBAgEABwABAogBAAJ7)" << std::endl << std::endl;
|
||||
return 1;
|
||||
}
|
||||
std::cout << "OPCODE,runtime mean,runtime stddev,gas mean,gas stddev" << std::endl;
|
||||
timingBaseline = averageRuntime("").runtime.mean;
|
||||
std::cout << "OPCODE,runtime mean,runtime stddev,gas mean,gas stddev,error" << std::endl;
|
||||
std::string setup, code;
|
||||
if(argc == 2) {
|
||||
setup = "";
|
||||
|
|
@ -165,7 +172,7 @@ int main(int argc, char** argv) {
|
|||
code = argv[2];
|
||||
}
|
||||
const auto time = timeInstruction(setup, code);
|
||||
std::cout << code << "," << time.runtime.mean << "," << time.runtime.stddev << "," <<
|
||||
time.gasUsage.mean << "," << time.gasUsage.stddev << std::endl;
|
||||
std::cout << std::fixed << std::setprecision(9) << code << "," << time.runtime.mean << "," << time.runtime.stddev
|
||||
<< "," << time.gasUsage.mean << "," << time.gasUsage.stddev << "," << (int)time.errored << std::endl;
|
||||
return 0;
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue