From e0a320f715a50a37ce177592e324e6044440d033 Mon Sep 17 00:00:00 2001 From: neodix42 Date: Mon, 15 Jan 2024 21:48:04 +0100 Subject: [PATCH] Improve TON build scripts and some tests (#855) * fix macOS github actions * fix android tonlib GH action; * fixing wasm GH action * strip binaries * fix randomly failing ubuntu and wasm GH actions * fix randomly failing ubuntu and wasm GH actions * revert some changes * adding more nix scripts and automated native build scripts; debug static ton compilation * minor fix * do not use pkg_config if path specified * move wasm script, run with sudo action script * weird, fixing - cp: missing destination file operand after 'assembly/native/build-ubuntu-20.04-shared.sh' * weird, fixing - cp: missing destination file operand after 'assembly/native/build-ubuntu-20.04-shared.sh' * ok * some adjustments for android and win builds * some adjustments for android and win builds * moving stripping inside the build script * access rights handling; adding simple binaries' tests * make lite-client-common, fift-lib and smc-envelope deliberately static; add -a (artifacts) flag to build scripts; * minor wasm build fix; create separate tonlib android build script; remove outdated __has_trivial_copy(T) * add windows build - WIP * adjust android build; improve win build; * adjust sodium paths for android build; use proper compiler for windows build; * add github windows build auxiliary file * adjust wasm build * add portable ubuntu build * exclude some unstable tests for some time * compile portable binaries on ubuntu-20.04 * exclude some unstable tests * include static gsl * restart builds * restart builds * restart builds * remove libreadline, gsl and blas dependencies in linux build * add macos build script * install missing autoconf in macos builds * enable all tests and see what fails * enable win tests and restart others * enable win tests and fix test-smartcont.cpp * enable win tests * use clang-16 on mac builds, add blockchain-explorer for ubuntu builds, add portable macos build * move sudo part outside a build scripts * move sudo part outside a build scripts * run llvm install with sudo * remove libgnutls28-dev before ubuntu static compilation, include blockchain-explorer into artifacts; remove warning: definition of implicit copy constructor for 'Stat' is deprecated because it has a user-declared copy assignment operator [-Wdeprecated-copy] * rework wrong decision, put back system gnutls, but compile libmicrohttpd with --disable-https * add jenkins pipeline sceleton * WIP jenkins pipeline sceleton * WIP jenkins pipeline changes * WIP jenkins pipeline: add stage timout, zip and group artifacts * WIP jenkins pipeline: macos portable build fix * WIP jenkins pipeline: wording * WIP jenkins pipeline: add android tonlib * WIP jenkins pipeline: add was binaries * WIP jenkins pipeline: add TOTAL_MEMORY 1.5gb to funcfiftlib wasm linking * WIP jenkins pipeline: add nix build on linux aarch64 * WIP jenkins pipeline: funcfiftlib compilation fails that 16mb mem is not enough, increase to 32mb * WIP jenkins pipeline: enable test in nix build * WIP jenkins pipeline: add linux x86-64 nix build * WIP jenkins pipeline: include libs in nix build * WIP jenkins pipeline: include libs in nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: nix linux arm64 with openssl 1.1 for now * WIP jenkins pipeline: working ubuntu arm64 libtonjson * WIP jenkins pipeline: working ubuntu arm64 libtonjson + minor fix * WIP jenkins pipeline: working ubuntu arm64 libtonjson + minor fix 2 * WIP jenkins pipeline: merry christmas * WIP jenkins pipeline: merry christmas 2 * WIP jenkins pipeline: remove native static builds * WIP jenkins pipeline: enable more tests * WIP jenkins pipeline: zip artifacts better * WIP jenkins pipeline: get rid of path in the final zip * WIP jenkins pipeline: minor fix, include lib and smartcont folders * WIP jenkins pipeline: minor fix, include lib and smartcont folders into nix artifacts also * WIP jenkins pipeline: minor fix * WIP jenkins pipeline: minor fix * adjust github actions for new nix builds * cleanup * cleanup * cleanup * cleanup * rename libtonlibjson.so.0.5 to libtonlibjson.so * Add TON build instructions to README.md * simplify * fix test-tonlib-offline * set timeout per test of 300 sec * set timeout per test of 600 sec for non nix builds * increase test timeout to 900 sec; minor changes * use MS VS 2022 for win TON compilation; update README.md * use MS VS 2022 for win TON compilation; update README.md * change path to MSVC in github workflow * change path to MSVC in groovy pipeline * compile ton on win, with msvc 2022 community and enterprise versions * minor fixes * improve network tests * remove TON compilation against macos-11 github runner * add `choco feature enable -n allowEmptyChecksums` since pkg-config-lite-0.28-1 does not have a checksum * abort win compilation if 3pp can't be downloaded * increase test timeout to 30 min * improving test-catchain --- .../build-ton-linux-android-tonlib.yml | 32 +++ .../build-ton-linux-x86-64-shared.yml | 40 +++ .../build-ton-macos-x86-64-shared.yml | 25 ++ .../workflows/build-ton-wasm-emscripten.yml | 30 +++ .github/workflows/create-release.yml | 6 +- .github/workflows/macos-11.7-compile.yml | 106 -------- .github/workflows/macos-12.6-compile.yml | 106 -------- .github/workflows/ton-aarch64-linux.yml | 50 ---- .github/workflows/ton-aarch64-macos.yml | 47 ---- .github/workflows/ton-ccpcheck.yml | 3 +- .github/workflows/ton-wasm-emscripten.yml | 46 ---- .github/workflows/ton-x86-64-linux.yml | 18 +- .github/workflows/ton-x86-64-macos.yml | 18 +- .github/workflows/ton-x86-64-windows.yml | 34 +++ .github/workflows/tonlib-android-jni.yml | 61 ----- .github/workflows/ubuntu-22.04-compile.yml | 78 ------ .github/workflows/ubuntu-compile.yml | 81 ------ .github/workflows/win-2019-compile.yml | 108 -------- .gitignore | 12 +- CMake/FindMHD.cmake | 29 ++- CMake/FindSecp256k1.cmake | 27 +- CMake/FindSodium.cmake | 1 + CMakeLists.txt | 19 +- README.md | 88 ++++++- assembly/android/build-android-tonlib.sh | 55 ++++ assembly/cicd/jenkins/test-builds.groovy | 236 ++++++++++++++++++ assembly/native/build-macos-portable.sh | 207 +++++++++++++++ assembly/native/build-macos-shared.sh | 136 ++++++++++ assembly/native/build-ubuntu-portable.sh | 198 +++++++++++++++ assembly/native/build-ubuntu-shared.sh | 122 +++++++++ assembly/native/build-windows-github.bat | 2 + assembly/native/build-windows.bat | 193 ++++++++++++++ assembly/nix/build-linux-arm64-nix.sh | 20 ++ assembly/nix/build-linux-x86-64-nix.sh | 20 ++ assembly/nix/build-macos-nix.sh | 17 ++ flake.lock => assembly/nix/flakes/flake.lock | 0 flake.nix => assembly/nix/flakes/flake.nix | 0 shell.nix => assembly/nix/flakes/shell.nix | 0 assembly/nix/linux-arm64-static.nix | 45 ++++ assembly/nix/linux-arm64-tonlib.nix | 44 ++++ assembly/nix/linux-x86-64-static.nix | 45 ++++ assembly/nix/linux-x86-64-tonlib.nix | 54 ++++ assembly/nix/macos-static.nix | 65 +++++ assembly/nix/macos-tonlib.nix | 55 ++++ assembly/nix/microhttpd.nix | 28 +++ assembly/nix/openssl.nix | 30 +++ .../wasm}/fift-func-wasm-build-ubuntu.sh | 53 +++- blockchain-explorer/CMakeLists.txt | 27 +- catchain/catchain-receiver.cpp | 4 +- crypto/CMakeLists.txt | 24 +- crypto/test/test-smartcont.cpp | 5 +- crypto/vm/boc.h | 1 + example/android/README.md | 24 +- example/android/build-all.sh | 2 + example/android/build.sh | 22 +- lite-client/CMakeLists.txt | 2 +- tdnet/td/net/TcpListener.cpp | 8 +- tdnet/test/net-test.cpp | 6 +- tdutils/td/utils/as.h | 5 - test/test-catchain.cpp | 9 +- tonlib/test/offline.cpp | 19 +- tonlib/tonlib/TonlibClient.cpp | 2 + 62 files changed, 2019 insertions(+), 831 deletions(-) create mode 100644 .github/workflows/build-ton-linux-android-tonlib.yml create mode 100644 .github/workflows/build-ton-linux-x86-64-shared.yml create mode 100644 .github/workflows/build-ton-macos-x86-64-shared.yml create mode 100644 .github/workflows/build-ton-wasm-emscripten.yml delete mode 100644 .github/workflows/macos-11.7-compile.yml delete mode 100644 .github/workflows/macos-12.6-compile.yml delete mode 100644 .github/workflows/ton-aarch64-linux.yml delete mode 100644 .github/workflows/ton-aarch64-macos.yml delete mode 100644 .github/workflows/ton-wasm-emscripten.yml create mode 100644 .github/workflows/ton-x86-64-windows.yml delete mode 100644 .github/workflows/tonlib-android-jni.yml delete mode 100644 .github/workflows/ubuntu-22.04-compile.yml delete mode 100644 .github/workflows/ubuntu-compile.yml delete mode 100644 .github/workflows/win-2019-compile.yml create mode 100644 assembly/android/build-android-tonlib.sh create mode 100644 assembly/cicd/jenkins/test-builds.groovy create mode 100644 assembly/native/build-macos-portable.sh create mode 100644 assembly/native/build-macos-shared.sh create mode 100644 assembly/native/build-ubuntu-portable.sh create mode 100644 assembly/native/build-ubuntu-shared.sh create mode 100644 assembly/native/build-windows-github.bat create mode 100644 assembly/native/build-windows.bat create mode 100644 assembly/nix/build-linux-arm64-nix.sh create mode 100644 assembly/nix/build-linux-x86-64-nix.sh create mode 100644 assembly/nix/build-macos-nix.sh rename flake.lock => assembly/nix/flakes/flake.lock (100%) rename flake.nix => assembly/nix/flakes/flake.nix (100%) rename shell.nix => assembly/nix/flakes/shell.nix (100%) create mode 100644 assembly/nix/linux-arm64-static.nix create mode 100644 assembly/nix/linux-arm64-tonlib.nix create mode 100644 assembly/nix/linux-x86-64-static.nix create mode 100644 assembly/nix/linux-x86-64-tonlib.nix create mode 100644 assembly/nix/macos-static.nix create mode 100644 assembly/nix/macos-tonlib.nix create mode 100644 assembly/nix/microhttpd.nix create mode 100644 assembly/nix/openssl.nix rename {.github/script => assembly/wasm}/fift-func-wasm-build-ubuntu.sh (71%) mode change 100755 => 100644 diff --git a/.github/workflows/build-ton-linux-android-tonlib.yml b/.github/workflows/build-ton-linux-android-tonlib.yml new file mode 100644 index 00000000..ae1be22f --- /dev/null +++ b/.github/workflows/build-ton-linux-android-tonlib.yml @@ -0,0 +1,32 @@ +name: Tonlib Android + +on: [push,workflow_dispatch,workflow_call] + +jobs: + build: + runs-on: ubuntu-22.04 + + steps: + - name: Check out repository + uses: actions/checkout@v3 + with: + submodules: 'recursive' + + - name: Install system libraries + run: | + sudo apt-get update + sudo apt-get install -y build-essential git cmake ninja-build automake libtool texinfo autoconf libgflags-dev \ + zlib1g-dev libssl-dev libreadline-dev libmicrohttpd-dev pkg-config libgsl-dev python3 python3-dev \ + libtool autoconf libsodium-dev libsecp256k1-dev + + - name: Build TON + run: | + cp assembly/android/build-android-tonlib.sh . + chmod +x build-android-tonlib.sh + ./build-android-tonlib.sh -a + + - name: Upload artifacts + uses: actions/upload-artifact@master + with: + name: tonlib-android + path: artifacts \ No newline at end of file diff --git a/.github/workflows/build-ton-linux-x86-64-shared.yml b/.github/workflows/build-ton-linux-x86-64-shared.yml new file mode 100644 index 00000000..166de660 --- /dev/null +++ b/.github/workflows/build-ton-linux-x86-64-shared.yml @@ -0,0 +1,40 @@ +name: Ubuntu TON build (shared, x86-64) + +on: [push,workflow_dispatch,workflow_call] + +jobs: + build: + strategy: + fail-fast: false + matrix: + os: [ubuntu-20.04, ubuntu-22.04] + runs-on: ${{ matrix.os }} + + steps: + - name: Check out repository + uses: actions/checkout@v3 + with: + submodules: 'recursive' + + - name: Install system libraries + run: | + sudo apt-get update + sudo apt-get install -y build-essential git cmake ninja-build zlib1g-dev libsecp256k1-dev libmicrohttpd-dev libsodium-dev + + - name: Install clang-16 + run: | + wget https://apt.llvm.org/llvm.sh + chmod +x llvm.sh + sudo ./llvm.sh 16 all + + - name: Build TON + run: | + cp assembly/native/build-ubuntu-shared.sh . + chmod +x build-ubuntu-shared.sh + ./build-ubuntu-shared.sh -t -a + + - name: Upload artifacts + uses: actions/upload-artifact@master + with: + name: ton-binaries-${{ matrix.os }} + path: artifacts diff --git a/.github/workflows/build-ton-macos-x86-64-shared.yml b/.github/workflows/build-ton-macos-x86-64-shared.yml new file mode 100644 index 00000000..c9331e3b --- /dev/null +++ b/.github/workflows/build-ton-macos-x86-64-shared.yml @@ -0,0 +1,25 @@ +name: MacOS TON build (shared, x86-64) + +on: [push,workflow_dispatch,workflow_call] + +jobs: + build: + runs-on: macos-12 + + steps: + - name: Check out repository + uses: actions/checkout@v3 + with: + submodules: 'recursive' + + - name: Build TON + run: | + cp assembly/native/build-macos-shared.sh . + chmod +x build-macos-shared.sh + ./build-macos-shared.sh -t -a + + - name: Upload artifacts + uses: actions/upload-artifact@master + with: + name: ton-binaries-macos-12 + path: artifacts diff --git a/.github/workflows/build-ton-wasm-emscripten.yml b/.github/workflows/build-ton-wasm-emscripten.yml new file mode 100644 index 00000000..16156b07 --- /dev/null +++ b/.github/workflows/build-ton-wasm-emscripten.yml @@ -0,0 +1,30 @@ +name: Emscripten TON build (wasm) + +on: [push,workflow_dispatch,workflow_call] + +jobs: + build: + runs-on: ubuntu-22.04 + + steps: + - name: Check out repository + uses: actions/checkout@v3 + with: + submodules: 'recursive' + + - name: Install system libraries + run: | + sudo apt-get update + sudo apt-get install -y build-essential git openssl cmake ninja-build zlib1g-dev libssl-dev libsecp256k1-dev libmicrohttpd-dev libsodium-dev + + - name: Build TON WASM artifacts + run: | + cd assembly/wasm + chmod +x fift-func-wasm-build-ubuntu.sh + ./fift-func-wasm-build-ubuntu.sh -a + + - name: Upload artifacts + uses: actions/upload-artifact@master + with: + name: ton-wasm-binaries + path: artifacts \ No newline at end of file diff --git a/.github/workflows/create-release.yml b/.github/workflows/create-release.yml index 10c20ac0..3d248cfe 100644 --- a/.github/workflows/create-release.yml +++ b/.github/workflows/create-release.yml @@ -46,7 +46,7 @@ jobs: - name: Download Windows artifacts uses: dawidd6/action-download-artifact@v2 with: - workflow: win-2019-compile.yml + workflow: ton-x86-64-windows.yml path: artifacts workflow_conclusion: success skip_unpack: true @@ -54,7 +54,7 @@ jobs: - name: Download and unzip Windows artifacts uses: dawidd6/action-download-artifact@v2 with: - workflow: win-2019-compile.yml + workflow: ton-x86-64-windows.yml path: artifacts workflow_conclusion: success skip_unpack: false @@ -62,7 +62,7 @@ jobs: - name: Download WASM artifacts uses: dawidd6/action-download-artifact@v2 with: - workflow: ton-wasm-emscripten.yml + workflow: build-ton-wasm-emscripten.yml path: artifacts workflow_conclusion: success skip_unpack: true diff --git a/.github/workflows/macos-11.7-compile.yml b/.github/workflows/macos-11.7-compile.yml deleted file mode 100644 index eb12db1b..00000000 --- a/.github/workflows/macos-11.7-compile.yml +++ /dev/null @@ -1,106 +0,0 @@ -name: MacOS 11.7 Big Sur x86-64 Compile - -on: [push,workflow_dispatch,workflow_call] - -jobs: - build: - - runs-on: macos-11 - - steps: - - name: Check out repository - uses: actions/checkout@v3 - with: - submodules: 'recursive' - - - name: Compile Secp256k1 - run: | - export NONINTERACTIVE=1 - brew install ninja secp256k1 libsodium libmicrohttpd pkg-config automake libtool - git clone https://github.com/libbitcoin/secp256k1.git - cd secp256k1 - ./autogen.sh - ./configure --enable-module-recovery - make - make install - - - name: Build all - run: | - brew unlink openssl@1.1 - brew install openssl@3 - brew unlink openssl@3 && brew link --overwrite openssl@3 - rootPath=`pwd` - mkdir build - cd build - cmake -GNinja -DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=11.7 -DCMAKE_CXX_FLAGS="-stdlib=libc++" -DCMAKE_BUILD_TYPE=Release .. - - ninja storage-daemon storage-daemon-cli blockchain-explorer fift func tonlib tonlibjson tonlib-cli validator-engine \ - lite-client pow-miner validator-engine-console generate-random-id json2tlo dht-server \ - http-proxy rldp-http-proxy adnl-proxy create-state create-hardfork tlbc emulator \ - test-ed25519 test-ed25519-crypto test-bigint test-vm test-fift test-cells test-smartcont test-net test-tdactor \ - test-tdutils test-tonlib-offline test-adnl test-dht test-rldp test-rldp2 test-catchain test-fec test-tddb test-db test-validator-session-state - - - name: Strip binaries - run: | - strip build/storage/storage-daemon/storage-daemon - strip build/storage/storage-daemon/storage-daemon-cli - strip build/blockchain-explorer/blockchain-explorer - strip build/crypto/fift - strip build/crypto/func - strip build/crypto/create-state - strip build/crypto/tlbc - strip build/validator-engine-console/validator-engine-console - strip build/tonlib/tonlib-cli - strip build/http/http-proxy - strip build/rldp-http-proxy/rldp-http-proxy - strip build/dht-server/dht-server - strip build/lite-client/lite-client - strip build/validator-engine/validator-engine - strip build/utils/generate-random-id - strip build/utils/json2tlo - strip build/adnl/adnl-proxy - - - name: Run tests - run: | - cd build - ctest --output-on-failure -E "test-catchain|test-actors" - - - name: Find & copy binaries - run: | - mkdir artifacts - cp build/storage/storage-daemon/storage-daemon artifacts/ - cp build/storage/storage-daemon/storage-daemon-cli artifacts/ - cp build/blockchain-explorer/blockchain-explorer artifacts/ - cp build/crypto/fift artifacts/ - cp build/crypto/func artifacts/ - cp build/crypto/create-state artifacts/ - cp build/crypto/tlbc artifacts/ - cp build/validator-engine-console/validator-engine-console artifacts/ - cp build/tonlib/tonlib-cli artifacts/ - cp build/tonlib/libtonlibjson.0.5.dylib artifacts/libtonlibjson.dylib - cp build/http/http-proxy artifacts/ - cp build/rldp-http-proxy/rldp-http-proxy artifacts/ - cp build/dht-server/dht-server artifacts/ - cp build/lite-client/lite-client artifacts/ - cp build/validator-engine/validator-engine artifacts/ - cp build/utils/generate-random-id artifacts/ - cp build/utils/json2tlo artifacts/ - cp build/adnl/adnl-proxy artifacts/ - cp build/emulator/*emulator.* artifacts/ - chmod +x artifacts/* - rsync -r crypto/smartcont artifacts/ - rsync -r crypto/fift/lib artifacts/ - ls -laRt artifacts - - - name: Simple binaries test - run: | - artifacts/validator-engine -V - artifacts/lite-client -V - artifacts/fift -V - artifacts/func -V - - - name: Upload artifacts - uses: actions/upload-artifact@master - with: - name: ton-macos-11.7 - path: artifacts diff --git a/.github/workflows/macos-12.6-compile.yml b/.github/workflows/macos-12.6-compile.yml deleted file mode 100644 index f41efc66..00000000 --- a/.github/workflows/macos-12.6-compile.yml +++ /dev/null @@ -1,106 +0,0 @@ -name: MacOS 12.6 Monterey x86-64 Compile - -on: [push,workflow_dispatch,workflow_call] - -jobs: - build: - - runs-on: macos-12 - - steps: - - name: Check out repository - uses: actions/checkout@v3 - with: - submodules: 'recursive' - - - name: Compile Secp256k1 - run: | - export NONINTERACTIVE=1 - brew install ninja secp256k1 libsodium libmicrohttpd pkg-config automake libtool - git clone https://github.com/libbitcoin/secp256k1.git - cd secp256k1 - ./autogen.sh - ./configure --enable-module-recovery - make - make install - - - name: Build all - run: | - brew unlink openssl@1.1 - brew install openssl@3 - brew unlink openssl@3 && brew link --overwrite openssl@3 - rootPath=`pwd` - mkdir build - cd build - cmake -GNinja -DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=12.6 -DCMAKE_CXX_FLAGS="-stdlib=libc++" -DCMAKE_BUILD_TYPE=Release .. - - ninja storage-daemon storage-daemon-cli blockchain-explorer fift func tonlib tonlibjson tonlib-cli \ - validator-engine lite-client pow-miner validator-engine-console generate-random-id json2tlo dht-server \ - http-proxy rldp-http-proxy adnl-proxy create-state create-hardfork tlbc emulator \ - test-ed25519 test-ed25519-crypto test-bigint test-vm test-fift test-cells test-smartcont test-net test-tdactor \ - test-tdutils test-tonlib-offline test-adnl test-dht test-rldp test-rldp2 test-catchain test-fec test-tddb test-db test-validator-session-state - - - name: Strip binaries - run: | - strip build/storage/storage-daemon/storage-daemon - strip build/storage/storage-daemon/storage-daemon-cli - strip build/blockchain-explorer/blockchain-explorer - strip build/crypto/fift - strip build/crypto/func - strip build/crypto/create-state - strip build/crypto/tlbc - strip build/validator-engine-console/validator-engine-console - strip build/tonlib/tonlib-cli - strip build/http/http-proxy - strip build/rldp-http-proxy/rldp-http-proxy - strip build/dht-server/dht-server - strip build/lite-client/lite-client - strip build/validator-engine/validator-engine - strip build/utils/generate-random-id - strip build/utils/json2tlo - strip build/adnl/adnl-proxy - - - name: Run tests - run: | - cd build - ctest --output-on-failure -E "test-catchain|test-actors" - - - name: Find & copy binaries - run: | - mkdir artifacts - cp build/storage/storage-daemon/storage-daemon artifacts/ - cp build/storage/storage-daemon/storage-daemon-cli artifacts/ - cp build/blockchain-explorer/blockchain-explorer artifacts/ - cp build/crypto/fift artifacts/ - cp build/crypto/func artifacts/ - cp build/crypto/create-state artifacts/ - cp build/crypto/tlbc artifacts/ - cp build/validator-engine-console/validator-engine-console artifacts/ - cp build/tonlib/tonlib-cli artifacts/ - cp build/tonlib/libtonlibjson.0.5.dylib artifacts/libtonlibjson.dylib - cp build/http/http-proxy artifacts/ - cp build/rldp-http-proxy/rldp-http-proxy artifacts/ - cp build/dht-server/dht-server artifacts/ - cp build/lite-client/lite-client artifacts/ - cp build/validator-engine/validator-engine artifacts/ - cp build/utils/generate-random-id artifacts/ - cp build/utils/json2tlo artifacts/ - cp build/adnl/adnl-proxy artifacts/ - cp build/emulator/*emulator.* artifacts/ - chmod +x artifacts/* - rsync -r crypto/smartcont artifacts/ - rsync -r crypto/fift/lib artifacts/ - ls -laRt artifacts - - - name: Simple binaries test - run: | - artifacts/validator-engine -V - artifacts/lite-client -V - artifacts/fift -V - artifacts/func -V - - - name: Upload artifacts - uses: actions/upload-artifact@master - with: - name: ton-macos-12.6 - path: artifacts diff --git a/.github/workflows/ton-aarch64-linux.yml b/.github/workflows/ton-aarch64-linux.yml deleted file mode 100644 index 3c600bee..00000000 --- a/.github/workflows/ton-aarch64-linux.yml +++ /dev/null @@ -1,50 +0,0 @@ -name: "TON aarch64 Linux binaries" - -on: [workflow_dispatch,workflow_call] - -jobs: - build: - runs-on: ubuntu-22.04 - - steps: - - run: | - sudo apt update - sudo apt install -y apt-utils - sudo apt install -q -y qemu-system-aarch64 qemu-efi binfmt-support qemu-user-static - - - uses: actions/checkout@v3 - with: - submodules: 'recursive' - - - uses: cachix/install-nix-action@v18 - with: - extra_nix_config: | - access-tokens = github.com=${{ secrets.GITHUB_TOKEN }} - - - name: Compile - run: nix build .?submodules=1#packages.aarch64-linux.ton-oldglibc_staticbinaries --print-build-logs --system aarch64-linux -o result-aarch64 - - - name: Copy binaries - run: | - ls -lart - mkdir artifacts - cp $PWD/result-aarch64-linux/bin/* artifacts/ - chmod +x artifacts/* - cp $PWD/result-aarch64-linux/lib/libtonlibjson.so.0.5 artifacts/libtonlibjson.so - cp $PWD/result-aarch64-linux/lib/libemulator.so artifacts/ - cp -R crypto/smartcont artifacts/ - cp -R crypto/fift/lib artifacts/ - - - name: Simple binaries test - run: | - sudo mv /nix/store /nix/store2 - artifacts/validator-engine -V - artifacts/lite-client -V - artifacts/fift -V - artifacts/func -V - - - name: Upload artifacts - uses: actions/upload-artifact@master - with: - name: ton-aarch64-linux-binaries - path: artifacts \ No newline at end of file diff --git a/.github/workflows/ton-aarch64-macos.yml b/.github/workflows/ton-aarch64-macos.yml deleted file mode 100644 index 75fcec78..00000000 --- a/.github/workflows/ton-aarch64-macos.yml +++ /dev/null @@ -1,47 +0,0 @@ -name: "TON aarch64 macOS binaries" - -on: [workflow_dispatch,workflow_call] - -jobs: - build: - runs-on: macos-12 - - steps: - - run: brew install qemu - - - uses: actions/checkout@v3 - with: - submodules: 'recursive' - - - uses: cachix/install-nix-action@v18 - with: - extra_nix_config: | - access-tokens = github.com=${{ secrets.GITHUB_TOKEN }} - - - name: Compile - run: nix build .?submodules=1#packages.aarch64-darwin.ton-staticbin-dylib --print-build-logs -o result-aarch64-darwin - - - name: Copy binaries - run: | - ls -lart - mkdir artifacts - cp $PWD/result-aarch64-darwin/bin/* artifacts/ - chmod +x artifacts/* - cp $PWD/result-aarch64-darwin/lib/libtonlibjson* artifacts/ - cp $PWD/result-aarch64-darwin/lib/libemulator* artifacts/ - cp -R crypto/smartcont artifacts/ - cp -R crypto/fift/lib artifacts/ - - - name: Simple binaries test - run: | - sudo mv /nix/store /nix/store2 - artifacts/validator-engine -V - artifacts/lite-client -V - artifacts/fift -V - artifacts/func -V - - - name: Upload artifacts - uses: actions/upload-artifact@master - with: - name: ton-aarch64-macos-binaries - path: artifacts diff --git a/.github/workflows/ton-ccpcheck.yml b/.github/workflows/ton-ccpcheck.yml index 8e9d6ad2..d2d8cf70 100644 --- a/.github/workflows/ton-ccpcheck.yml +++ b/.github/workflows/ton-ccpcheck.yml @@ -1,10 +1,9 @@ -name: TON Ccpcheck +name: TON Static Code Analysis on: [push,workflow_dispatch,workflow_call] jobs: build: - runs-on: ubuntu-22.04 steps: diff --git a/.github/workflows/ton-wasm-emscripten.yml b/.github/workflows/ton-wasm-emscripten.yml deleted file mode 100644 index a3167800..00000000 --- a/.github/workflows/ton-wasm-emscripten.yml +++ /dev/null @@ -1,46 +0,0 @@ -name: TON WASM Compile - -on: [push,workflow_dispatch,workflow_call] - -jobs: - build: - runs-on: ubuntu-22.04 - - steps: - - name: Check out repository - uses: actions/checkout@v3 - with: - submodules: 'recursive' - - - name: Install libraries - run: | - sudo apt update - sudo apt install -y build-essential git make cmake ninja-build clang libgflags-dev zlib1g-dev libssl-dev libreadline-dev libmicrohttpd-dev pkg-config libgsl-dev python3 python3-dev python3-pip nodejs libsecp256k1-dev libsodium-dev automake libtool - - - name: Setup compiler - run: | - wget https://apt.llvm.org/llvm.sh - chmod +x llvm.sh - sudo ./llvm.sh 16 all - - - name: Configure & Build - run: | - cd .github/script - ./fift-func-wasm-build-ubuntu.sh - - - name: Find & copy binaries - run: | - mkdir artifacts - ls build/crypto - cp build/crypto/fift* artifacts - cp build/crypto/func* artifacts - cp build/crypto/tlbc* artifacts - cp build/emulator/emulator-emscripten* artifacts - cp -R crypto/smartcont artifacts - cp -R crypto/fift/lib artifacts - - - name: Upload artifacts - uses: actions/upload-artifact@master - with: - name: ton-wasm-binaries - path: artifacts \ No newline at end of file diff --git a/.github/workflows/ton-x86-64-linux.yml b/.github/workflows/ton-x86-64-linux.yml index a4760dc4..fdd91000 100644 --- a/.github/workflows/ton-x86-64-linux.yml +++ b/.github/workflows/ton-x86-64-linux.yml @@ -1,4 +1,4 @@ -name: "TON x86_64 Linux binaries" +name: Ubuntu TON build (portable, x86-64) on: [push,workflow_dispatch,workflow_call] @@ -20,19 +20,11 @@ jobs: extra_nix_config: | access-tokens = github.com=${{ secrets.GITHUB_TOKEN }} - - name: Compile - run: nix build .?submodules=1#packages.x86_64-linux.ton-oldglibc_staticbinaries --print-build-logs --system x86_64-linux -o result-x86_64 - - - name: Copy binaries + - name: Build TON run: | - ls -lart - mkdir artifacts - cp $PWD/result-x86_64/bin/* artifacts/ - chmod +x artifacts/* - cp $PWD/result-x86_64/lib/libtonlibjson.so.0.5 artifacts/libtonlibjson.so - cp $PWD/result-x86_64/lib/libemulator.so artifacts/ - cp -R crypto/smartcont artifacts/ - cp -R crypto/fift/lib artifacts/ + cp assembly/nix/build-linux-x86-64-nix.sh . + chmod +x build-linux-x86-64-nix.sh + ./build-linux-x86-64-nix.sh - name: Simple binaries test run: | diff --git a/.github/workflows/ton-x86-64-macos.yml b/.github/workflows/ton-x86-64-macos.yml index cea2937a..c0f90718 100644 --- a/.github/workflows/ton-x86-64-macos.yml +++ b/.github/workflows/ton-x86-64-macos.yml @@ -1,4 +1,4 @@ -name: "TON x86_64 macOS binaries" +name: MacOS TON build (portable, x86-64) on: [push,workflow_dispatch,workflow_call] @@ -16,19 +16,11 @@ jobs: extra_nix_config: | access-tokens = github.com=${{ secrets.GITHUB_TOKEN }} - - name: Compile - run: nix build .?submodules=1#packages.x86_64-darwin.ton-staticbin-dylib --print-build-logs -o result-x86_64-darwin - - - name: Copy binaries + - name: Build TON run: | - ls -lart - mkdir artifacts - cp $PWD/result-x86_64-darwin/bin/* artifacts/ - chmod +x artifacts/* - cp $PWD/result-x86_64-darwin/lib/libtonlibjson.dylib artifacts/ - cp $PWD/result-x86_64-darwin/lib/libemulator.dylib artifacts/ - cp -R crypto/smartcont artifacts/ - cp -R crypto/fift/lib artifacts/ + cp assembly/nix/build-macos-nix.sh . + chmod +x build-macos-nix.sh + ./build-macos-nix.sh - name: Simple binaries test run: | diff --git a/.github/workflows/ton-x86-64-windows.yml b/.github/workflows/ton-x86-64-windows.yml new file mode 100644 index 00000000..67026183 --- /dev/null +++ b/.github/workflows/ton-x86-64-windows.yml @@ -0,0 +1,34 @@ +name: Windows TON build (portable, x86-64) + +on: [push,workflow_dispatch,workflow_call] + +defaults: + run: + shell: cmd + +jobs: + build: + + runs-on: windows-2022 + + steps: + - name: Get Current OS version + run: | + systeminfo | findstr /B /C:"OS Name" /C:"OS Version" + + - name: Check out current repository + uses: actions/checkout@v3 + with: + submodules: 'recursive' + + - name: Build TON + run: | + copy assembly\native\build-windows-github.bat . + copy assembly\native\build-windows.bat . + build-windows-github.bat Enterprise + + - name: Upload artifacts + uses: actions/upload-artifact@master + with: + name: ton-win-binaries + path: artifacts diff --git a/.github/workflows/tonlib-android-jni.yml b/.github/workflows/tonlib-android-jni.yml deleted file mode 100644 index 6e04f8b7..00000000 --- a/.github/workflows/tonlib-android-jni.yml +++ /dev/null @@ -1,61 +0,0 @@ -name: Tonlib Android JNI - -on: [push,workflow_dispatch,workflow_call] - -jobs: - build: - - runs-on: ubuntu-22.04 - - steps: - - name: Check out repository - uses: actions/checkout@v3 - with: - submodules: 'recursive' - - - name: Install libraries - run: | - sudo apt update - sudo apt install -y build-essential git make cmake clang libgflags-dev zlib1g-dev libssl-dev libreadline-dev libmicrohttpd-dev pkg-config libgsl-dev python3 python3-dev ninja-build - - - name: Configure & Build - run: | - wget -q https://dl.google.com/android/repository/android-ndk-r25b-linux.zip - unzip -q android-ndk-r25b-linux.zip - export JAVA_AWT_LIBRARY=NotNeeded - export JAVA_JVM_LIBRARY=NotNeeded - export JAVA_INCLUDE_PATH=${JAVA_HOME}/include - export JAVA_AWT_INCLUDE_PATH=${JAVA_HOME}/include - export JAVA_INCLUDE_PATH2=${JAVA_HOME}/include/linux - - export ANDROID_NDK_ROOT=$(pwd)/android-ndk-r25b - export NDK_PLATFORM="android-21" - export ANDROID_PLATFORM="android-21" - export OPENSSL_DIR=$(pwd)/example/android/third_party/crypto - - rm -rf example/android/src/drinkless/org/ton/TonApi.java - cd example/android/ - - sudo apt install -y libtool autoconf libsodium-dev libsecp256k1-dev - - cmake -GNinja -DTON_ONLY_TONLIB=ON . - - ninja prepare_cross_compiling - - sudo apt remove -y libsodium-dev libsecp256k1-dev - - rm CMakeCache.txt - ./build-all.sh - find . -name "*.debug" -type f -delete - - - name: Find & copy binaries - run: | - mkdir -p artifacts/tonlib-android-jni - cp example/android/src/drinkless/org/ton/TonApi.java artifacts/tonlib-android-jni/ - cp -R example/android/libs/* artifacts/tonlib-android-jni/ - - - name: Upload artifacts - uses: actions/upload-artifact@master - with: - name: tonlib-android-jni - path: artifacts \ No newline at end of file diff --git a/.github/workflows/ubuntu-22.04-compile.yml b/.github/workflows/ubuntu-22.04-compile.yml deleted file mode 100644 index af8943a1..00000000 --- a/.github/workflows/ubuntu-22.04-compile.yml +++ /dev/null @@ -1,78 +0,0 @@ -name: Ubuntu 22.04 Compile - -on: [push,workflow_dispatch,workflow_call] - -jobs: - build: - - runs-on: ubuntu-22.04 - - steps: - - name: Check out repository - uses: actions/checkout@v3 - with: - submodules: 'recursive' - - - name: Install libraries - run: | - sudo apt update - sudo apt install -y build-essential git make cmake clang libgflags-dev zlib1g-dev libssl-dev libreadline-dev libmicrohttpd-dev pkg-config libgsl-dev python3 python3-dev ninja-build libsecp256k1-dev libsodium-dev - - - name: Show CPU flags - run: | - cat /proc/cpuinfo - - - name: Configure & Build - run: | - export CC=$(which clang) - export CXX=$(which clang++) - export CCACHE_DISABLE=1 - - git clone https://github.com/openssl/openssl openssl_3 - cd openssl_3 - git checkout openssl-3.1.4 - ./config - make build_libs -j4 - - cd .. - rootPath=`pwd` - mkdir build - cd build - - cmake -GNinja -DOPENSSL_FOUND=1 -DOPENSSL_INCLUDE_DIR=$rootPath/openssl_3/include -DOPENSSL_CRYPTO_LIBRARY=$rootPath/openssl_3/libcrypto.a -DCMAKE_BUILD_TYPE=Release -DPORTABLE=1 -DTON_ARCH= .. - - ninja storage-daemon storage-daemon-cli fift func tonlib tonlibjson tonlib-cli validator-engine lite-client \ - pow-miner validator-engine-console generate-random-id json2tlo dht-server http-proxy rldp-http-proxy \ - adnl-proxy create-state emulator \ - test-ed25519 test-ed25519-crypto test-bigint test-vm test-fift test-cells test-smartcont test-net test-tdactor \ - test-tdutils test-tonlib-offline test-adnl test-dht test-rldp test-rldp2 test-catchain test-fec test-tddb test-db test-validator-session-state - - - name: Strip binaries - run: | - strip -g build/storage/storage-daemon/storage-daemon build/storage/storage-daemon/storage-daemon-cli build/crypto/fift build/crypto/tlbc build/crypto/func build/crypto/create-state build/validator-engine-console/validator-engine-console build/tonlib/tonlib-cli build/tonlib/libtonlibjson.so.0.5 build/http/http-proxy build/rldp-http-proxy/rldp-http-proxy build/dht-server/dht-server build/lite-client/lite-client build/validator-engine/validator-engine build/utils/generate-random-id build/utils/json2tlo build/adnl/adnl-proxy build/emulator/libemulator.* - - - name: Run tests - run: | - cd build - ctest --output-on-failure -E "test-catchain|test-actors" - - - name: Find & copy binaries - run: | - mkdir artifacts - cp build/storage/storage-daemon/storage-daemon build/storage/storage-daemon/storage-daemon-cli build/crypto/fift build/crypto/tlbc build/crypto/func build/crypto/create-state build/validator-engine-console/validator-engine-console build/tonlib/tonlib-cli build/tonlib/libtonlibjson.so.0.5 build/http/http-proxy build/rldp-http-proxy/rldp-http-proxy build/dht-server/dht-server build/lite-client/lite-client build/validator-engine/validator-engine build/utils/generate-random-id build/utils/json2tlo build/adnl/adnl-proxy build/emulator/libemulator.* artifacts - chmod +x artifacts/* - cp -R crypto/smartcont artifacts/ - cp -R crypto/fift/lib artifacts/ - - - name: Simple binaries test - run: | - artifacts/validator-engine -V - artifacts/lite-client -V - artifacts/fift -V - artifacts/func -V - - - name: Upload artifacts - uses: actions/upload-artifact@master - with: - name: ton-ubuntu-binaries - path: artifacts diff --git a/.github/workflows/ubuntu-compile.yml b/.github/workflows/ubuntu-compile.yml deleted file mode 100644 index 3c1e7bad..00000000 --- a/.github/workflows/ubuntu-compile.yml +++ /dev/null @@ -1,81 +0,0 @@ -name: Ubuntu Compile x86-64 - -on: [push,workflow_dispatch,workflow_call] - -jobs: - build: - strategy: - fail-fast: false - matrix: - os: [ubuntu-20.04, ubuntu-22.04] - runs-on: ${{ matrix.os }} - - steps: - - name: Check out repository - uses: actions/checkout@v3 - with: - submodules: 'recursive' - - - name: Install libraries - run: | - sudo apt update - sudo apt install -y build-essential git make cmake clang libgflags-dev zlib1g-dev libssl-dev libreadline-dev libmicrohttpd-dev pkg-config libgsl-dev python3 python3-dev ninja-build libsecp256k1-dev libsodium-dev - - - name: Show CPU flags - run: | - cat /proc/cpuinfo - - - name: Configure & Build - run: | - export CC=$(which clang) - export CXX=$(which clang++) - export CCACHE_DISABLE=1 - - mkdir build-${{ matrix.os }} - cd build-${{ matrix.os }} - - git clone https://github.com/openssl/openssl openssl_3 - cd openssl_3 - git checkout openssl-3.1.4 - ./config - make build_libs -j4 - - cd .. - rootPath=`pwd` - - cmake -GNinja -DOPENSSL_FOUND=1 -DOPENSSL_INCLUDE_DIR=$rootPath/openssl_3/include -DOPENSSL_CRYPTO_LIBRARY=$rootPath/openssl_3/libcrypto.a -DCMAKE_BUILD_TYPE=Release -DPORTABLE=1 -DTON_ARCH= .. - ninja storage-daemon storage-daemon-cli fift func tonlib tonlibjson tonlib-cli validator-engine lite-client \ - pow-miner validator-engine-console generate-random-id json2tlo dht-server http-proxy rldp-http-proxy adnl-proxy \ - create-state create-hardfork emulator \ - test-ed25519 test-ed25519-crypto test-bigint test-vm test-fift test-cells test-smartcont test-net test-tdactor \ - test-tdutils test-tonlib-offline test-adnl test-dht test-rldp test-rldp2 test-catchain test-fec test-tddb test-db test-validator-session-state - - - name: Strip binaries - run: | - strip -g build-${{ matrix.os }}/storage/storage-daemon/storage-daemon build-${{ matrix.os }}/storage/storage-daemon/storage-daemon-cli build-${{ matrix.os }}/crypto/fift build-${{ matrix.os }}/crypto/tlbc build-${{ matrix.os }}/crypto/func build-${{ matrix.os }}/crypto/create-state build-${{ matrix.os }}/validator-engine-console/validator-engine-console build-${{ matrix.os }}/tonlib/tonlib-cli build-${{ matrix.os }}/tonlib/libtonlibjson.so.0.5 build-${{ matrix.os }}/http/http-proxy build-${{ matrix.os }}/rldp-http-proxy/rldp-http-proxy build-${{ matrix.os }}/dht-server/dht-server build-${{ matrix.os }}/lite-client/lite-client build-${{ matrix.os }}/validator-engine/validator-engine build-${{ matrix.os }}/utils/generate-random-id build-${{ matrix.os }}/utils/json2tlo build-${{ matrix.os }}/adnl/adnl-proxy build-${{ matrix.os }}/emulator/libemulator.* - - - name: Run tests - run: | - cd build-${{ matrix.os }} - ctest --output-on-failure -E "test-catchain|test-actors" - - - name: Find & copy binaries - run: | - mkdir artifacts-${{ matrix.os }} - cp build-${{ matrix.os }}/storage/storage-daemon/storage-daemon build-${{ matrix.os }}/storage/storage-daemon/storage-daemon-cli build-${{ matrix.os }}/crypto/fift build-${{ matrix.os }}/crypto/tlbc build-${{ matrix.os }}/crypto/func build-${{ matrix.os }}/crypto/create-state build-${{ matrix.os }}/validator-engine-console/validator-engine-console build-${{ matrix.os }}/tonlib/tonlib-cli build-${{ matrix.os }}/tonlib/libtonlibjson.so.0.5 build-${{ matrix.os }}/http/http-proxy build-${{ matrix.os }}/rldp-http-proxy/rldp-http-proxy build-${{ matrix.os }}/dht-server/dht-server build-${{ matrix.os }}/lite-client/lite-client build-${{ matrix.os }}/validator-engine/validator-engine build-${{ matrix.os }}/utils/generate-random-id build-${{ matrix.os }}/utils/json2tlo build-${{ matrix.os }}/adnl/adnl-proxy build-${{ matrix.os }}/emulator/libemulator.* artifacts-${{ matrix.os }} - chmod +x artifacts-${{ matrix.os }}/* - cp -R crypto/smartcont artifacts-${{ matrix.os }} - cp -R crypto/fift/lib artifacts-${{ matrix.os }} - - - name: Simple binaries test - run: | - artifacts-${{ matrix.os }}/validator-engine -V - artifacts-${{ matrix.os }}/lite-client -V - artifacts-${{ matrix.os }}/fift -V - artifacts-${{ matrix.os }}/func -V - - - name: Upload artifacts - uses: actions/upload-artifact@master - with: - name: ton-binaries-${{ matrix.os }} - path: artifacts-${{ matrix.os }} diff --git a/.github/workflows/win-2019-compile.yml b/.github/workflows/win-2019-compile.yml deleted file mode 100644 index e94655f7..00000000 --- a/.github/workflows/win-2019-compile.yml +++ /dev/null @@ -1,108 +0,0 @@ -name: Windows Server 2019 x64 Compile - -on: [push,workflow_dispatch,workflow_call] - -defaults: - run: - shell: cmd - -jobs: - build: - - runs-on: windows-2019 - - steps: - - name: Get Current OS version - run: | - systeminfo | findstr /B /C:"OS Name" /C:"OS Version" - - - name: Check out current repository - uses: actions/checkout@v3 - with: - submodules: 'recursive' - - - name: Check out zlib repository - uses: actions/checkout@v3 - with: - repository: desktop-app/zlib - path: zlib - - - name: Setup msbuild.exe - uses: microsoft/setup-msbuild@v1.1 - - - name: Install Pkg-config Lite - run: choco install pkgconfiglite - - - name: Compile zlib Win64 - run: | - cd zlib\contrib\vstudio\vc14 - msbuild zlibstat.vcxproj /p:Configuration=ReleaseWithoutAsm /p:platform=x64 -p:PlatformToolset=v142 - - - name: Compile secp256k1 Win64 - run: | - git clone https://github.com/libbitcoin/secp256k1.git - cd secp256k1\builds\msvc\vs2017 - msbuild /p:Configuration=StaticRelease -p:PlatformToolset=v142 -p:Platform=x64 - - - name: Install pre-compiled libsodium Win64 - run: | - curl -Lo libsodium-1.0.18-stable-msvc.zip https://download.libsodium.org/libsodium/releases/libsodium-1.0.18-stable-msvc.zip - unzip libsodium-1.0.18-stable-msvc.zip - - - name: Install pre-compiled OpenSSL 3 Win64 - run: | - curl -Lo openssl-3.1.4.zip https://github.com/neodiX42/precompiled-openssl-win64/raw/main/openssl-3.1.4.zip - unzip openssl-3.1.4.zip - - - name: Install pre-compiled libmicrohttpd Win64 - run: | - curl -Lo libmicrohttpd-0.9.77-w32-bin.zip https://github.com/neodiX42/precompiled-openssl-win64/raw/main/libmicrohttpd-0.9.77-w32-bin.zip - unzip libmicrohttpd-0.9.77-w32-bin.zip - - - name: Install pre-compiled Readline Win64 - run: | - curl -Lo readline-5.0-1-lib.zip https://github.com/neodiX42/precompiled-openssl-win64/raw/main/readline-5.0-1-lib.zip - unzip readline-5.0-1-lib.zip - - - name: Compile - run: | - set root=%cd% - set SODIUM_DIR=%root%\libsodium - echo %root% - echo %SODIUM_DIR% - mkdir build - cd build - cmake -DSODIUM_USE_STATIC_LIBS=1 -DSECP256K1_INCLUDE_DIR=%root%\secp256k1\include -DSECP256K1_LIBRARY=%root%\secp256k1\bin\x64\Release\v142\static\secp256k1.lib -DREADLINE_INCLUDE_DIR=%root%\readline-5.0-1-lib\include\readline -DREADLINE_LIBRARY=%root%\readline-5.0-1-lib\lib\readline.lib -DPORTABLE=1 -DZLIB_FOUND=1 -DMHD_FOUND=1 -DMHD_LIBRARY=%root%\libmicrohttpd-0.9.77-w32-bin\x86_64\VS2019\Release-static\libmicrohttpd.lib -DMHD_INCLUDE_DIR=%root%\libmicrohttpd-0.9.77-w32-bin\x86_64\VS2019\Release-static -DZLIB_INCLUDE_DIR=%root%\zlib -DZLIB_LIBRARY=%root%\zlib\contrib\vstudio\vc14\x64\ZlibStatReleaseWithoutAsm\zlibstat.lib -DOPENSSL_FOUND=1 -DOPENSSL_INCLUDE_DIR=%root%/openssl-3.1.4/x64/include -DOPENSSL_CRYPTO_LIBRARY=%root%/openssl-3.1.4/x64/lib/libcrypto_static.lib -DCMAKE_CXX_FLAGS="/DTD_WINDOWS=1 /EHsc /bigobj /W0" .. - cmake --build . --config Release --target storage-daemon storage-daemon-cli blockchain-explorer fift func tonlib tonlibjson tonlib-cli validator-engine lite-client pow-miner validator-engine-console generate-random-id json2tlo dht-server http-proxy rldp-http-proxy adnl-proxy create-state create-hardfork emulator test-ed25519 test-ed25519-crypto test-bigint test-vm test-fift test-cells test-smartcont test-net test-tdactor test-tdutils test-tonlib-offline test-adnl test-dht test-rldp test-rldp2 test-catchain test-fec test-tddb test-db test-validator-session-state - - - name: Run tests - run: | - cd build - ctest -C Release --output-on-failure -E "test-catchain|test-actors|test-validator-session-state" - - - name: Show executables - run: | - cd build - del Release\test-* - dir *.exe /a-D /S /B - dir *.dll /a-D /S /B - - - name: Check if validator-engine.exe exists - run: | - copy %cd%\build\validator-engine\Release\validator-engine.exe test - - - name: Find & copy binaries - run: | - mkdir artifacts - mkdir artifacts\smartcont - mkdir artifacts\lib - - for %%I in (build\storage\storage-daemon\Release\storage-daemon.exe build\storage\storage-daemon\Release\storage-daemon-cli.exe build\blockchain-explorer\blockchain-explorer.exe build\crypto\Release\fift.exe build\crypto\Release\tlbc.exe build\crypto\Release\func.exe build\crypto\Release\create-state.exe build\validator-engine-console\Release\validator-engine-console.exe build\tonlib\Release\tonlib-cli.exe build\tonlib\Release\tonlibjson.dll build\http\Release\http-proxy.exe build\rldp-http-proxy\Release\rldp-http-proxy.exe build\dht-server\Release\dht-server.exe build\lite-client\Release\lite-client.exe build\validator-engine\Release\validator-engine.exe build\utils\Release\generate-random-id.exe build\utils\Release\json2tlo.exe build\adnl\Release\adnl-proxy.exe build\emulator\Release\emulator.dll) do (strip -g %%I & copy %%I artifacts\) - xcopy /e /k /h /i crypto\smartcont artifacts\smartcont - xcopy /e /k /h /i crypto\fift\lib artifacts\lib - - - name: Upload artifacts - uses: actions/upload-artifact@master - with: - name: ton-win-binaries - path: artifacts diff --git a/.gitignore b/.gitignore index 54d9ffc7..536918ab 100644 --- a/.gitignore +++ b/.gitignore @@ -12,4 +12,14 @@ test/regression-tests.cache/ *.swp **/*build*/ .idea -.vscode \ No newline at end of file +.vscode +zlib/ +libsodium/ +libmicrohttpd-0.9.77-w32-bin/ +readline-5.0-1-lib/ +secp256k1/ +openssl-3.1.4/ +libsodium-1.0.18-stable-msvc.zip +libmicrohttpd-0.9.77-w32-bin.zip +openssl-3.1.4.zip +readline-5.0-1-lib.zip diff --git a/CMake/FindMHD.cmake b/CMake/FindMHD.cmake index c4b94c0e..7d6dd5fd 100644 --- a/CMake/FindMHD.cmake +++ b/CMake/FindMHD.cmake @@ -2,23 +2,26 @@ # Once done this will define # # MHD_FOUND - system has MHD -# MHD_INCLUDE_DIRS - the MHD include directory +# MHD_INCLUDE_DIR - the MHD include directory # MHD_LIBRARY - Link these to use MHD -find_path( - MHD_INCLUDE_DIR - NAMES microhttpd.h - DOC "microhttpd include dir" -) +if (NOT MHD_LIBRARY) + find_path( + MHD_INCLUDE_DIR + NAMES microhttpd.h + DOC "microhttpd include dir" + ) -find_library( - MHD_LIBRARY - NAMES microhttpd microhttpd-10 libmicrohttpd libmicrohttpd-dll - DOC "microhttpd library" -) + find_library( + MHD_LIBRARY + NAMES microhttpd microhttpd-10 libmicrohttpd libmicrohttpd-dll + DOC "microhttpd library" + ) +endif() -set(MHD_INCLUDE_DIRS ${MHD_INCLUDE_DIR}) -set(MHD_LIBRARIES ${MHD_LIBRARY}) +if (MHD_LIBRARY) + message(STATUS "Found MHD: ${MHD_LIBRARY}") +endif() include(FindPackageHandleStandardArgs) find_package_handle_standard_args(MHD DEFAULT_MSG MHD_INCLUDE_DIR MHD_LIBRARY) diff --git a/CMake/FindSecp256k1.cmake b/CMake/FindSecp256k1.cmake index 11603f15..68a37c71 100644 --- a/CMake/FindSecp256k1.cmake +++ b/CMake/FindSecp256k1.cmake @@ -2,28 +2,27 @@ # Once done this will define # # SECP256K1_FOUND - system has SECP256K1 -# SECP256K1_INCLUDE_DIRS - the SECP256K1 include directory +# SECP256K1_INCLUDE_DIR - the SECP256K1 include directory # SECP256K1_LIBRARY - Link these to use SECP256K1 -find_path( - SECP256K1_INCLUDE_DIR - NAMES secp256k1_recovery.h - DOC "secp256k1_recovery.h include dir" -) +if (NOT SECP256K1_LIBRARY) + find_path( + SECP256K1_INCLUDE_DIR + NAMES secp256k1_recovery.h + DOC "secp256k1_recovery.h include dir" + ) -find_library( - SECP256K1_LIBRARY - NAMES secp256k1 libsecp256k1 - DOC "secp256k1 library" -) + find_library( + SECP256K1_LIBRARY + NAMES secp256k1 libsecp256k1 + DOC "secp256k1 library" + ) +endif() if (SECP256K1_LIBRARY) message(STATUS "Found Secp256k1: ${SECP256K1_LIBRARY}") endif() -set(SECP256K1_INCLUDE_DIRS ${SECP256K1_INCLUDE_DIR}) -set(SECP256K1_LIBRARIES ${SECP256K1_LIBRARY}) - include(FindPackageHandleStandardArgs) find_package_handle_standard_args(Secp256k1 DEFAULT_MSG SECP256K1_INCLUDE_DIR SECP256K1_LIBRARY) mark_as_advanced(SECP256K1_INCLUDE_DIR SECP256K1_LIBRARY) diff --git a/CMake/FindSodium.cmake b/CMake/FindSodium.cmake index 0053ac54..85194ee2 100644 --- a/CMake/FindSodium.cmake +++ b/CMake/FindSodium.cmake @@ -26,6 +26,7 @@ # Furthermore an imported "sodium" target is created. # + if (CMAKE_C_COMPILER_ID STREQUAL "GNU" OR CMAKE_C_COMPILER_ID STREQUAL "Clang") set(_GCC_COMPATIBLE 1) diff --git a/CMakeLists.txt b/CMakeLists.txt index c6d7ed87..89be3238 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -211,7 +211,13 @@ set(CMAKE_THREAD_PREFER_PTHREAD ON) set(THREADS_PREFER_PTHREAD_FLAG ON) find_package(Threads REQUIRED) find_package(PkgConfig REQUIRED) -find_package(ZLIB REQUIRED) + +if (NOT ZLIB_FOUND) + find_package(ZLIB REQUIRED) +else() + message(STATUS "Using zlib ${ZLIB_LIBRARIES}") +endif() + if (TON_ARCH AND NOT MSVC) CHECK_CXX_COMPILER_FLAG( "-march=${TON_ARCH}" COMPILER_OPT_ARCH_SUPPORTED ) @@ -371,6 +377,9 @@ if (LATEX_FOUND) add_latex_document(doc/fiftbase.tex TARGET_NAME fift_basic_description) add_latex_document(doc/catchain.tex TARGET_NAME catchain_consensus_description) endif() +if (NOT LATEX_FOUND) + message(STATUS "Could NOT find LATEX (this is NOT an error)") +endif() #END internal function(target_link_libraries_system target) @@ -580,8 +589,8 @@ if (NOT NIX) WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/crypto/func/auto-tests) if (WIN32) set_property(TEST test-func PROPERTY ENVIRONMENT - "FUNC_EXECUTABLE=${CMAKE_CURRENT_BINARY_DIR}/crypto/Release/func.exe" - "FIFT_EXECUTABLE=${CMAKE_CURRENT_BINARY_DIR}/crypto/Release/fift.exe" + "FUNC_EXECUTABLE=${CMAKE_CURRENT_BINARY_DIR}/crypto/func.exe" + "FIFT_EXECUTABLE=${CMAKE_CURRENT_BINARY_DIR}/crypto/fift.exe" "FIFTPATH=${CMAKE_CURRENT_SOURCE_DIR}/crypto/fift/lib/") else() set_property(TEST test-func PROPERTY ENVIRONMENT @@ -596,8 +605,8 @@ if (NOT NIX) WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/crypto/func/auto-tests) if (WIN32) set_property(TEST test-func-legacy PROPERTY ENVIRONMENT - "FUNC_EXECUTABLE=${CMAKE_CURRENT_BINARY_DIR}/crypto/Release/func.exe" - "FIFT_EXECUTABLE=${CMAKE_CURRENT_BINARY_DIR}/crypto/Release/fift.exe" + "FUNC_EXECUTABLE=${CMAKE_CURRENT_BINARY_DIR}/crypto/func.exe" + "FIFT_EXECUTABLE=${CMAKE_CURRENT_BINARY_DIR}/crypto/fift.exe" "FIFTPATH=${CMAKE_CURRENT_SOURCE_DIR}/crypto/fift/lib/") else() set_property(TEST test-func-legacy PROPERTY ENVIRONMENT diff --git a/README.md b/README.md index 7e78bb04..653f2f83 100644 --- a/README.md +++ b/README.md @@ -43,7 +43,7 @@ __The Open Network (TON)__ is a fast, secure, scalable blockchain focused on han - To work on TON check [wallets](https://ton.app/wallets), [explorers](https://ton.app/explorers), [DEXes](https://ton.app/dex) and [utilities](https://ton.app/utilities) - To interact with TON check [APIs](https://ton.org/docs/develop/dapps/apis/) -## Updates flow: +## Updates flow * **master branch** - mainnet is running on this stable branch. @@ -61,12 +61,90 @@ Usually, the response to your pull request will indicate which section it falls * Thou shall not merge your own PRs, at least one person should review the PR and merge it (4-eyes rule) * Thou shall make sure that workflows are cleanly completed for your PR before considering merge -## Workflows responsibility -If a CI workflow fails not because of your changes but workflow issues, try to fix it yourself or contact one of the persons listed below via Telegram messenger: +## Build TON blockchain -* **C/C++ CI (ccpp-linux.yml)**: TBD -* **C/C++ CI Win64 Compile (ccpp-win64.yml)**: TBD +### Ubuntu 20.4, 22.04 (x86-64, aarch64) +Install additional system libraries +```bash + sudo apt-get update + sudo apt-get install -y build-essential git cmake ninja-build zlib1g-dev libsecp256k1-dev libmicrohttpd-dev libsodium-dev + + wget https://apt.llvm.org/llvm.sh + chmod +x llvm.sh + sudo ./llvm.sh 16 all +``` +Compile TON binaries +```bash + cp assembly/native/build-ubuntu-shared.sh . + chmod +x build-ubuntu-shared.sh + ./build-ubuntu-shared.sh +``` +### MacOS 11, 12 (x86-64, aarch64) +```bash + cp assembly/native/build-macos-shared.sh . + chmod +x build-macos-shared.sh + ./build-macos-shared.sh +``` + +### Windows 10, 11, Server (x86-64) +You need to install `MS Visual Studio 2022` first. +Go to https://www.visualstudio.com/downloads/ and download `MS Visual Studio 2022 Community`. + +Launch installer and select `Desktop development with C++`. +After installation, also make sure that `cmake` is globally available by adding +`C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin` to the system `PATH` (adjust the path per your needs). + +Open an elevated (Run as Administrator) `x86-64 Native Tools Command Prompt for VS 2022`, go to the root folder and execute: +```bash + copy assembly\native\build-windows.bat . + build-windows.bat +``` + +### Building TON to WebAssembly +Install additional system libraries on Ubuntu +```bash + sudo apt-get update + sudo apt-get install -y build-essential git cmake ninja-build zlib1g-dev libsecp256k1-dev libmicrohttpd-dev libsodium-dev + + wget https://apt.llvm.org/llvm.sh + chmod +x llvm.sh + sudo ./llvm.sh 16 all +``` +Compile TON binaries with emscripten +```bash + cd assembly/wasm + chmod +x fift-func-wasm-build-ubuntu.sh + ./fift-func-wasm-build-ubuntu.sh +``` + +### Building TON tonlib library for Android (arm64-v8a, armeabi-v7a, x86, x86-64) +Install additional system libraries on Ubuntu +```bash + sudo apt-get update + sudo apt-get install -y build-essential git cmake ninja-build automake libtool texinfo autoconf libgflags-dev \ + zlib1g-dev libssl-dev libreadline-dev libmicrohttpd-dev pkg-config libgsl-dev python3 python3-dev \ + libtool autoconf libsodium-dev libsecp256k1-dev +``` +Compile TON tonlib library +```bash + cp assembly/android/build-android-tonlib.sh . + chmod +x build-android-tonlib.sh + ./build-android-tonlib.sh +``` + +### Build TON portable binaries with Nix package manager +You need to install Nix first. +```bash + sh <(curl -L https://nixos.org/nix/install) --daemon +``` +Then compile TON with Nix by executing below command from the root folder: +```bash + cp -r assembly/nix/* . + export NIX_PATH=nixpkgs=https://github.com/nixOS/nixpkgs/archive/23.05.tar.gz + nix-build linux-x86-64-static.nix +``` +More examples for other platforms can be found under `assembly/nix`. ## Running tests diff --git a/assembly/android/build-android-tonlib.sh b/assembly/android/build-android-tonlib.sh new file mode 100644 index 00000000..e470f602 --- /dev/null +++ b/assembly/android/build-android-tonlib.sh @@ -0,0 +1,55 @@ +with_artifacts=false + +while getopts 'a' flag; do + case "${flag}" in + a) with_artifacts=true ;; + *) break + ;; + esac +done + +if [ ! -d android-ndk-r25b ]; then + rm android-ndk-r25b-linux.zip + wget -q https://dl.google.com/android/repository/android-ndk-r25b-linux.zip + unzip -q android-ndk-r25b-linux.zip + test $? -eq 0 || { echo "Can't unzip android-ndk-r25b-linux.zip"; exit 1; } + echo Android NDK extracted +else + echo Using extracted Android NDK +fi + +export JAVA_AWT_LIBRARY=NotNeeded +export JAVA_JVM_LIBRARY=NotNeeded +export JAVA_INCLUDE_PATH=${JAVA_HOME}/include +export JAVA_AWT_INCLUDE_PATH=${JAVA_HOME}/include +export JAVA_INCLUDE_PATH2=${JAVA_HOME}/include/linux + +export ANDROID_NDK_ROOT=$(pwd)/android-ndk-r25b +export NDK_PLATFORM="android-21" +export ANDROID_PLATFORM="android-21" +export OPENSSL_DIR=$(pwd)/example/android/third_party/crypto + +rm -rf example/android/src/drinkless/org/ton/TonApi.java +cd example/android/ + +rm CMakeCache.txt .ninja_* +cmake -GNinja -DTON_ONLY_TONLIB=ON . + +test $? -eq 0 || { echo "Can't configure TON"; exit 1; } + +ninja prepare_cross_compiling + +test $? -eq 0 || { echo "Can't compile prepare_cross_compiling"; exit 1; } + +rm CMakeCache.txt .ninja_* + +. ./build-all.sh + +find . -name "*.debug" -type f -delete + +if [ "$with_artifacts" = true ]; then + cd ../.. + mkdir -p artifacts/tonlib-android-jni + cp example/android/src/drinkless/org/ton/TonApi.java artifacts/tonlib-android-jni/ + cp -R example/android/libs/* artifacts/tonlib-android-jni/ +fi diff --git a/assembly/cicd/jenkins/test-builds.groovy b/assembly/cicd/jenkins/test-builds.groovy new file mode 100644 index 00000000..380efedd --- /dev/null +++ b/assembly/cicd/jenkins/test-builds.groovy @@ -0,0 +1,236 @@ +pipeline { + agent none + stages { + stage('Run Builds') { + parallel { + stage('Ubuntu 20.04 x86-64 (shared)') { + agent { + label 'Ubuntu_x86-64' + } + steps { + timeout(time: 90, unit: 'MINUTES') { + sh ''' + cp assembly/native/build-ubuntu-shared.sh . + chmod +x build-ubuntu-shared.sh + ./build-ubuntu-shared.sh -t -a + ''' + sh ''' + cd artifacts + zip -9r ton-x86_64-linux-shared ./* + ''' + archiveArtifacts artifacts: 'artifacts/ton-x86_64-linux-shared.zip' + } + } + } + stage('Ubuntu 20.04 x86-64 (portable)') { + agent { + label 'Ubuntu_x86-64' + } + steps { + timeout(time: 90, unit: 'MINUTES') { + sh ''' + cp assembly/nix/build-linux-x86-64-nix.sh . + chmod +x build-linux-x86-64-nix.sh + ./build-linux-x86-64-nix.sh + ''' + sh ''' + cd artifacts + zip -9r ton-x86-64-linux-portable ./* + ''' + archiveArtifacts artifacts: 'artifacts/ton-x86-64-linux-portable.zip' + } + } + } + stage('Ubuntu 20.04 aarch64 (shared)') { + agent { + label 'Ubuntu_arm64' + } + steps { + timeout(time: 90, unit: 'MINUTES') { + sh ''' + cp assembly/native/build-ubuntu-shared.sh . + chmod +x build-ubuntu-shared.sh + ./build-ubuntu-shared.sh -t -a + ''' + sh ''' + cd artifacts + zip -9r ton-arm64-linux-shared ./* + ''' + archiveArtifacts artifacts: 'artifacts/ton-arm64-linux-shared.zip' + } + } + } + stage('Ubuntu 20.04 aarch64 (portable)') { + agent { + label 'Ubuntu_arm64' + } + steps { + timeout(time: 90, unit: 'MINUTES') { + sh ''' + cp assembly/nix/build-linux-arm64-nix.sh . + chmod +x build-linux-arm64-nix.sh + ./build-linux-arm64-nix.sh + ''' + sh ''' + cd artifacts + zip -9r ton-arm64-linux-portable ./* + ''' + archiveArtifacts artifacts: 'artifacts/ton-arm64-linux-portable.zip' + } + } + } + stage('macOS 12.7 x86-64 (shared)') { + agent { + label 'macOS_12.7_x86-64' + } + steps { + timeout(time: 90, unit: 'MINUTES') { + sh ''' + cp assembly/native/build-macos-shared.sh . + chmod +x build-macos-shared.sh + ./build-macos-shared.sh -t -a + ''' + sh ''' + cd artifacts + zip -9r ton-x86-64-macos-shared ./* + ''' + archiveArtifacts artifacts: 'artifacts/ton-x86-64-macos-shared.zip' + } + } + } + stage('macOS 12.7 x86-64 (portable)') { + agent { + label 'macOS_12.7_x86-64' + } + steps { + timeout(time: 90, unit: 'MINUTES') { + sh ''' + cp assembly/nix/build-macos-nix.sh . + chmod +x build-macos-nix.sh + ./build-macos-nix.sh + ''' + sh ''' + cd artifacts + zip -9r ton-x86-64-macos-portable ./* + ''' + archiveArtifacts artifacts: 'artifacts/ton-x86-64-macos-portable.zip' + } + } + } + stage('macOS 12.6 aarch64 (shared)') { + agent { + label 'macOS_12.6-arm64-m1' + } + steps { + timeout(time: 90, unit: 'MINUTES') { + sh ''' + cp assembly/native/build-macos-shared.sh . + chmod +x build-macos-shared.sh + ./build-macos-shared.sh -t -a + ''' + sh ''' + cd artifacts + zip -9r ton-arm64-macos-m1-shared ./* + ''' + archiveArtifacts artifacts: 'artifacts/ton-arm64-macos-m1-shared.zip' + } + } + } + stage('macOS 12.6 aarch64 (portable)') { + agent { + label 'macOS_12.6-arm64-m1' + } + steps { + timeout(time: 90, unit: 'MINUTES') { + sh ''' + cp assembly/nix/build-macos-nix.sh . + chmod +x build-macos-nix.sh + ./build-macos-nix.sh + ''' + sh ''' + cd artifacts + zip -9r ton-arm64-macos-portable ./* + ''' + archiveArtifacts artifacts: 'artifacts/ton-arm64-macos-portable.zip' + } + } + } + stage('macOS 13.2 aarch64 (shared)') { + agent { + label 'macOS_13.2-arm64-m2' + } + steps { + timeout(time: 90, unit: 'MINUTES') { + sh ''' + cp assembly/native/build-macos-shared.sh . + chmod +x build-macos-shared.sh + ./build-macos-shared.sh -t -a + ''' + sh ''' + cd artifacts + zip -9r ton-arm64-macos-m2-shared ./* + ''' + archiveArtifacts artifacts: 'artifacts/ton-arm64-macos-m2-shared.zip' + } + } + } + stage('Windows Server 2022 x86-64') { + agent { + label 'Windows_x86-64' + } + steps { + timeout(time: 90, unit: 'MINUTES') { + bat ''' + copy assembly\\native\\build-windows.bat . + build-windows.bat + ''' + bat ''' + cd artifacts + zip -9r ton-x86-64-windows ./* + ''' + archiveArtifacts artifacts: 'artifacts/ton-x86-64-windows.zip' + } + } + } + stage('Android Tonlib') { + agent { + label 'Ubuntu_x86-64' + } + steps { + timeout(time: 90, unit: 'MINUTES') { + sh ''' + cp assembly/android/build-android-tonlib.sh . + chmod +x build-android-tonlib.sh + ./build-android-tonlib.sh -a + ''' + sh ''' + cd artifacts/tonlib-android-jni + zip -9r ton-android-tonlib ./* + ''' + archiveArtifacts artifacts: 'artifacts/tonlib-android-jni/ton-android-tonlib.zip' + } + } + } + stage('WASM fift func emulator') { + agent { + label 'Ubuntu_x86-64' + } + steps { + timeout(time: 90, unit: 'MINUTES') { + sh ''' + cd assembly/wasm + chmod +x fift-func-wasm-build-ubuntu.sh + ./fift-func-wasm-build-ubuntu.sh -a + ''' + sh ''' + cd artifacts + zip -9r ton-wasm-binaries ./* + ''' + archiveArtifacts artifacts: 'artifacts/ton-wasm-binaries.zip' + } + } + } + } + } + } +} \ No newline at end of file diff --git a/assembly/native/build-macos-portable.sh b/assembly/native/build-macos-portable.sh new file mode 100644 index 00000000..a4187d4c --- /dev/null +++ b/assembly/native/build-macos-portable.sh @@ -0,0 +1,207 @@ +#/bin/bash + +with_tests=false +with_artifacts=false +OSX_TARGET=10.15 + + +while getopts 'tao:' flag; do + case "${flag}" in + t) with_tests=true ;; + a) with_artifacts=true ;; + o) OSX_TARGET=${OPTARG} ;; + *) break + ;; + esac +done + +if [ ! -d "build" ]; then + mkdir build + cd build +else + cd build + rm -rf .ninja* CMakeCache.txt +fi + +export NONINTERACTIVE=1 +brew install ninja pkg-config automake libtool autoconf +brew install llvm@16 + + +if [ -f /opt/homebrew/opt/llvm@16/bin/clang ]; then + export CC=/opt/homebrew/opt/llvm@16/bin/clang + export CXX=/opt/homebrew/opt/llvm@16/bin/clang++ +else + export CC=/usr/local/opt/llvm@16/bin/clang + export CXX=/usr/local/opt/llvm@16/bin/clang++ +fi +export CCACHE_DISABLE=1 + +if [ ! -d "secp256k1" ]; then +git clone https://github.com/bitcoin-core/secp256k1.git +cd secp256k1 +secp256k1Path=`pwd` +git checkout v0.3.2 +./autogen.sh +./configure --enable-module-recovery --enable-static --disable-tests --disable-benchmark --with-pic +make -j12 +test $? -eq 0 || { echo "Can't compile secp256k1"; exit 1; } +cd .. +else + secp256k1Path=$(pwd)/secp256k1 + echo "Using compiled secp256k1" +fi + +if [ ! -d "libsodium" ]; then + export LIBSODIUM_FULL_BUILD=1 + git clone https://github.com/jedisct1/libsodium.git + cd libsodium + sodiumPath=`pwd` + git checkout 1.0.18 + ./autogen.sh + ./configure --with-pic --enable-static + make -j12 + test $? -eq 0 || { echo "Can't compile libsodium"; exit 1; } + cd .. +else + sodiumPath=$(pwd)/libsodium + echo "Using compiled libsodium" +fi + +if [ ! -d "openssl_3" ]; then + git clone https://github.com/openssl/openssl openssl_3 + cd openssl_3 + opensslPath=`pwd` + git checkout openssl-3.1.4 + ./config -static + make build_libs -j12 + test $? -eq 0 || { echo "Can't compile openssl_3"; exit 1; } + cd .. +else + opensslPath=$(pwd)/openssl_3 + echo "Using compiled openssl_3" +fi + +if [ ! -d "zlib" ]; then + git clone https://github.com/madler/zlib.git + cd zlib + zlibPath=`pwd` + ./configure --static + make -j12 + test $? -eq 0 || { echo "Can't compile zlib"; exit 1; } + cd .. +else + zlibPath=$(pwd)/zlib + echo "Using compiled zlib" +fi + +if [ ! -d "libmicrohttpd" ]; then + git clone https://git.gnunet.org/libmicrohttpd.git + cd libmicrohttpd + libmicrohttpdPath=`pwd` + ./autogen.sh + ./configure --enable-static --disable-tests --disable-benchmark --disable-shared --disable-https --with-pic + make -j12 + test $? -eq 0 || { echo "Can't compile libmicrohttpd"; exit 1; } + cd .. +else + libmicrohttpdPath=$(pwd)/libmicrohttpd + echo "Using compiled libmicrohttpd" +fi + +cmake -GNinja .. \ +-DPORTABLE=1 \ +-DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=$OSX_TARGET \ +-DCMAKE_CXX_FLAGS="-stdlib=libc++" \ +-DCMAKE_BUILD_TYPE=Release \ +-DOPENSSL_FOUND=1 \ +-DOPENSSL_INCLUDE_DIR=$opensslPath/include \ +-DOPENSSL_CRYPTO_LIBRARY=$opensslPath/libcrypto.a \ +-DZLIB_FOUND=1 \ +-DZLIB_INCLUDE_DIR=$zlibPath \ +-DZLIB_LIBRARIES=$zlibPath/libz.a \ +-DSECP256K1_FOUND=1 \ +-DSECP256K1_INCLUDE_DIR=$secp256k1Path/include \ +-DSECP256K1_LIBRARY=$secp256k1Path/.libs/libsecp256k1.a \ +-DSODIUM_FOUND=1 \ +-DSODIUM_INCLUDE_DIR=$sodiumPath/src/libsodium/include \ +-DSODIUM_LIBRARY_RELEASE=$sodiumPath/src/libsodium/.libs/libsodium.a \ +-DMHD_FOUND=1 \ +-DMHD_INCLUDE_DIR=$libmicrohttpdPath/src/include \ +-DMHD_LIBRARY=$libmicrohttpdPath/src/microhttpd/.libs/libmicrohttpd.a + + +test $? -eq 0 || { echo "Can't configure ton"; exit 1; } + +if [ "$with_tests" = true ]; then + ninja storage-daemon storage-daemon-cli blockchain-explorer \ + tonlib tonlibjson tonlib-cli validator-engine func fift \ + lite-client pow-miner validator-engine-console generate-random-id json2tlo dht-server \ + http-proxy rldp-http-proxy adnl-proxy create-state create-hardfork tlbc emulator \ + test-ed25519 test-ed25519-crypto test-bigint test-vm test-fift test-cells test-smartcont \ + test-net test-tdactor test-tdutils test-tonlib-offline test-adnl test-dht test-rldp \ + test-rldp2 test-catchain test-fec test-tddb test-db test-validator-session-state + test $? -eq 0 || { echo "Can't compile ton"; exit 1; } +else + ninja storage-daemon storage-daemon-cli blockchain-explorer \ + tonlib tonlibjson tonlib-cli validator-engine func fift \ + lite-client pow-miner validator-engine-console generate-random-id json2tlo dht-server \ + http-proxy rldp-http-proxy adnl-proxy create-state create-hardfork tlbc emulator + test $? -eq 0 || { echo "Can't compile ton"; exit 1; } +fi + +strip storage/storage-daemon/storage-daemon +strip storage/storage-daemon/storage-daemon-cli +strip blockchain-explorer/blockchain-explorer +strip crypto/fift +strip crypto/func +strip crypto/create-state +strip crypto/tlbc +strip validator-engine-console/validator-engine-console +strip tonlib/tonlib-cli +strip http/http-proxy +strip rldp-http-proxy/rldp-http-proxy +strip dht-server/dht-server +strip lite-client/lite-client +strip validator-engine/validator-engine +strip utils/generate-random-id +strip utils/json2tlo +strip adnl/adnl-proxy + +cd .. + +if [ "$with_artifacts" = true ]; then + echo Creating artifacts... + rm -rf artifacts + mkdir artifacts + cp crypto/fift/lib artifacts/ + cp -R crypto/smartcont/ artifacts/ + cp build/storage/storage-daemon/storage-daemon artifacts/ + cp build/storage/storage-daemon/storage-daemon-cli artifacts/ + cp build/blockchain-explorer/blockchain-explorer artifacts/ + cp build/crypto/fift artifacts/ + cp build/crypto/func artifacts/ + cp build/crypto/create-state artifacts/ + cp build/crypto/tlbc artifacts/ + cp build/validator-engine-console/validator-engine-console artifacts/ + cp build/tonlib/tonlib-cli artifacts/ + cp build/tonlib/libtonlibjson.0.5.dylib artifacts/libtonlibjson.dylib + cp build/http/http-proxy artifacts/ + cp build/rldp-http-proxy/rldp-http-proxy artifacts/ + cp build/dht-server/dht-server artifacts/ + cp build/lite-client/lite-client artifacts/ + cp build/validator-engine/validator-engine artifacts/ + cp build/utils/generate-random-id artifacts/ + cp build/utils/json2tlo artifacts/ + cp build/adnl/adnl-proxy artifacts/ + cp build/emulator/libemulator.dylib artifacts/ + chmod +x artifacts/* + rsync -r crypto/smartcont artifacts/ + rsync -r crypto/fift/lib artifacts/ +fi + +if [ "$with_tests" = true ]; then + cd build +# ctest --output-on-failure -E "test-catchain|test-actors" + ctest --output-on-failure +fi diff --git a/assembly/native/build-macos-shared.sh b/assembly/native/build-macos-shared.sh new file mode 100644 index 00000000..7b4f90ee --- /dev/null +++ b/assembly/native/build-macos-shared.sh @@ -0,0 +1,136 @@ +#/bin/bash + +with_tests=false +with_artifacts=false +OSX_TARGET=10.15 + + +while getopts 'tao:' flag; do + case "${flag}" in + t) with_tests=true ;; + a) with_artifacts=true ;; + o) OSX_TARGET=${OPTARG} ;; + *) break + ;; + esac +done + +if [ ! -d "build" ]; then + mkdir build + cd build +else + cd build + rm -rf .ninja* CMakeCache.txt +fi + +export NONINTERACTIVE=1 +brew install ninja libsodium libmicrohttpd pkg-config automake libtool autoconf gnutls +brew install llvm@16 + +if [ -f /opt/homebrew/opt/llvm@16/bin/clang ]; then + export CC=/opt/homebrew/opt/llvm@16/bin/clang + export CXX=/opt/homebrew/opt/llvm@16/bin/clang++ +else + export CC=/usr/local/opt/llvm@16/bin/clang + export CXX=/usr/local/opt/llvm@16/bin/clang++ +fi +export CCACHE_DISABLE=1 + +if [ ! -d "secp256k1" ]; then + git clone https://github.com/bitcoin-core/secp256k1.git + cd secp256k1 + secp256k1Path=`pwd` + git checkout v0.3.2 + ./autogen.sh + ./configure --enable-module-recovery --enable-static --disable-tests --disable-benchmark + make -j12 + test $? -eq 0 || { echo "Can't compile secp256k1"; exit 1; } + cd .. +else + secp256k1Path=$(pwd)/secp256k1 + echo "Using compiled secp256k1" +fi + +brew unlink openssl@1.1 +brew install openssl@3 +brew unlink openssl@3 && brew link --overwrite openssl@3 + +cmake -GNinja -DCMAKE_BUILD_TYPE=Release .. \ +-DCMAKE_CXX_FLAGS="-stdlib=libc++" \ +-DSECP256K1_FOUND=1 \ +-DSECP256K1_INCLUDE_DIR=$secp256k1Path/include \ +-DSECP256K1_LIBRARY=$secp256k1Path/.libs/libsecp256k1.a + +test $? -eq 0 || { echo "Can't configure ton"; exit 1; } + +if [ "$with_tests" = true ]; then + ninja storage-daemon storage-daemon-cli blockchain-explorer \ + tonlib tonlibjson tonlib-cli validator-engine func fift \ + lite-client pow-miner validator-engine-console generate-random-id json2tlo dht-server \ + http-proxy rldp-http-proxy adnl-proxy create-state create-hardfork tlbc emulator \ + test-ed25519 test-ed25519-crypto test-bigint test-vm test-fift test-cells test-smartcont \ + test-net test-tdactor test-tdutils test-tonlib-offline test-adnl test-dht test-rldp \ + test-rldp2 test-catchain test-fec test-tddb test-db test-validator-session-state + test $? -eq 0 || { echo "Can't compile ton"; exit 1; } +else + ninja storage-daemon storage-daemon-cli blockchain-explorer \ + tonlib tonlibjson tonlib-cli validator-engine func fift \ + lite-client pow-miner validator-engine-console generate-random-id json2tlo dht-server \ + http-proxy rldp-http-proxy adnl-proxy create-state create-hardfork tlbc emulator + test $? -eq 0 || { echo "Can't compile ton"; exit 1; } +fi + + +strip storage/storage-daemon/storage-daemon +strip storage/storage-daemon/storage-daemon-cli +strip blockchain-explorer/blockchain-explorer +strip crypto/fift +strip crypto/func +strip crypto/create-state +strip crypto/tlbc +strip validator-engine-console/validator-engine-console +strip tonlib/tonlib-cli +strip http/http-proxy +strip rldp-http-proxy/rldp-http-proxy +strip dht-server/dht-server +strip lite-client/lite-client +strip validator-engine/validator-engine +strip utils/generate-random-id +strip utils/json2tlo +strip adnl/adnl-proxy + +cd .. + +if [ "$with_artifacts" = true ]; then + echo Creating artifacts... + rm -rf artifacts + mkdir artifacts + cp build/storage/storage-daemon/storage-daemon artifacts/ + cp build/storage/storage-daemon/storage-daemon-cli artifacts/ + cp build/blockchain-explorer/blockchain-explorer artifacts/ + cp build/crypto/fift artifacts/ + cp build/crypto/func artifacts/ + cp build/crypto/create-state artifacts/ + cp build/crypto/tlbc artifacts/ + cp build/validator-engine-console/validator-engine-console artifacts/ + cp build/tonlib/tonlib-cli artifacts/ + cp build/tonlib/libtonlibjson.0.5.dylib artifacts/libtonlibjson.dylib + cp build/http/http-proxy artifacts/ + cp build/rldp-http-proxy/rldp-http-proxy artifacts/ + cp build/dht-server/dht-server artifacts/ + cp build/lite-client/lite-client artifacts/ + cp build/validator-engine/validator-engine artifacts/ + cp build/utils/generate-random-id artifacts/ + cp build/utils/json2tlo artifacts/ + cp build/adnl/adnl-proxy artifacts/ + cp build/emulator/libemulator.dylib artifacts/ + chmod +x artifacts/* + rsync -r crypto/smartcont artifacts/ + rsync -r crypto/fift/lib artifacts/ +fi + +if [ "$with_tests" = true ]; then + cd build +# ctest --output-on-failure -E "test-catchain|test-actors" + ctest --output-on-failure --timeout 1800 +fi diff --git a/assembly/native/build-ubuntu-portable.sh b/assembly/native/build-ubuntu-portable.sh new file mode 100644 index 00000000..81dbe710 --- /dev/null +++ b/assembly/native/build-ubuntu-portable.sh @@ -0,0 +1,198 @@ +#/bin/bash + +#sudo apt-get update +#sudo apt-get install -y build-essential git cmake ninja-build automake libtool texinfo autoconf + +with_tests=false +with_artifacts=false + + +while getopts 'ta' flag; do + case "${flag}" in + t) with_tests=true ;; + a) with_artifacts=true ;; + *) break + ;; + esac +done + +if [ ! -d "build" ]; then + mkdir build + cd build +else + cd build + rm -rf .ninja* CMakeCache.txt +fi + +export CC=$(which clang-16) +export CXX=$(which clang++-16) +export CCACHE_DISABLE=1 + + +if [ ! -d "secp256k1" ]; then +git clone https://github.com/bitcoin-core/secp256k1.git +cd secp256k1 +secp256k1Path=`pwd` +git checkout v0.3.2 +./autogen.sh +./configure --enable-module-recovery --enable-static --disable-tests --disable-benchmark --with-pic +make -j12 +test $? -eq 0 || { echo "Can't compile secp256k1"; exit 1; } +cd .. +# ./.libs/libsecp256k1.a +# ./include +else + secp256k1Path=$(pwd)/secp256k1 + echo "Using compiled secp256k1" +fi + +if [ ! -d "libsodium" ]; then + export LIBSODIUM_FULL_BUILD=1 + git clone https://github.com/jedisct1/libsodium.git + cd libsodium + sodiumPath=`pwd` + git checkout 1.0.18 + ./autogen.sh + ./configure --with-pic --enable-static + make -j12 + test $? -eq 0 || { echo "Can't compile libsodium"; exit 1; } + cd .. +else + sodiumPath=$(pwd)/libsodium + echo "Using compiled libsodium" +fi + +if [ ! -d "openssl_3" ]; then + git clone https://github.com/openssl/openssl openssl_3 + cd openssl_3 + opensslPath=`pwd` + git checkout openssl-3.1.4 + ./config -static + make build_libs -j12 + test $? -eq 0 || { echo "Can't compile openssl_3"; exit 1; } + cd .. +else + opensslPath=$(pwd)/openssl_3 + echo "Using compiled openssl_3" +fi + +if [ ! -d "zlib" ]; then + git clone https://github.com/madler/zlib.git + cd zlib + zlibPath=`pwd` + ./configure --static + make -j12 + test $? -eq 0 || { echo "Can't compile zlib"; exit 1; } + cd .. +else + zlibPath=$(pwd)/zlib + echo "Using compiled zlib" +fi + +if [ ! -d "libmicrohttpd" ]; then + git clone https://git.gnunet.org/libmicrohttpd.git + cd libmicrohttpd + libmicrohttpdPath=`pwd` + ./autogen.sh + ./configure --enable-static --disable-tests --disable-benchmark --disable-shared --disable-https --with-pic + make -j12 + test $? -eq 0 || { echo "Can't compile libmicrohttpd"; exit 1; } + cd .. +else + libmicrohttpdPath=$(pwd)/libmicrohttpd + echo "Using compiled libmicrohttpd" +fi + +cmake -GNinja .. \ +-DPORTABLE=1 \ +-DCMAKE_BUILD_TYPE=Release \ +-DOPENSSL_FOUND=1 \ +-DOPENSSL_INCLUDE_DIR=$opensslPath/include \ +-DOPENSSL_CRYPTO_LIBRARY=$opensslPath/libcrypto.a \ +-DZLIB_FOUND=1 \ +-DZLIB_INCLUDE_DIR=$zlibPath \ +-DZLIB_LIBRARIES=$zlibPath/libz.a \ +-DSECP256K1_FOUND=1 \ +-DSECP256K1_INCLUDE_DIR=$secp256k1Path/include \ +-DSECP256K1_LIBRARY=$secp256k1Path/.libs/libsecp256k1.a \ +-DSODIUM_FOUND=1 \ +-DSODIUM_INCLUDE_DIR=$sodiumPath/src/libsodium/include \ +-DSODIUM_LIBRARY_RELEASE=$sodiumPath/src/libsodium/.libs/libsodium.a \ +-DMHD_FOUND=1 \ +-DMHD_INCLUDE_DIR=$libmicrohttpdPath/src/include \ +-DMHD_LIBRARY=$libmicrohttpdPath/src/microhttpd/.libs/libmicrohttpd.a + + +test $? -eq 0 || { echo "Can't configure ton"; exit 1; } + +if [ "$with_tests" = true ]; then +ninja storage-daemon storage-daemon-cli fift func tonlib tonlibjson tonlib-cli \ + validator-engine lite-client pow-miner validator-engine-console blockchain-explorer \ + generate-random-id json2tlo dht-server http-proxy rldp-http-proxy \ + adnl-proxy create-state emulator test-ed25519 test-ed25519-crypto test-bigint \ + test-vm test-fift test-cells test-smartcont test-net test-tdactor test-tdutils \ + test-tonlib-offline test-adnl test-dht test-rldp test-rldp2 test-catchain \ + test-fec test-tddb test-db test-validator-session-state + test $? -eq 0 || { echo "Can't compile ton"; exit 1; } +else +ninja storage-daemon storage-daemon-cli fift func tonlib tonlibjson tonlib-cli \ + validator-engine lite-client pow-miner validator-engine-console blockchain-explorer \ + generate-random-id json2tlo dht-server http-proxy rldp-http-proxy \ + adnl-proxy create-state emulator + test $? -eq 0 || { echo "Can't compile ton"; exit 1; } +fi + +strip -g storage/storage-daemon/storage-daemon \ + storage/storage-daemon/storage-daemon-cli \ + blockchain-explorer/blockchain-explorer \ + crypto/fift \ + crypto/tlbc \ + crypto/func \ + crypto/create-state \ + validator-engine-console/validator-engine-console \ + tonlib/tonlib-cli \ + tonlib/libtonlibjson.so.0.5 \ + http/http-proxy \ + rldp-http-proxy/rldp-http-proxy \ + dht-server/dht-server \ + lite-client/lite-client \ + validator-engine/validator-engine \ + utils/generate-random-id \ + utils/json2tlo \ + adnl/adnl-proxy \ + emulator/libemulator.* + +test $? -eq 0 || { echo "Can't strip final binaries"; exit 1; } + +# simple binaries' test +./storage/storage-daemon/storage-daemon -V || exit 1 +./validator-engine/validator-engine -V || exit 1 +./lite-client/lite-client -V || exit 1 +./crypto/fift -V || exit 1 + +cd .. + +if [ "$with_artifacts" = true ]; then + rm -rf artifacts + mkdir artifacts + cp crypto/fift/lib artifacts/ + cp -R crypto/smartcont/ artifacts/ + mv build/tonlib/libtonlibjson.so.0.5 build/tonlib/libtonlibjson.so + cp build/storage/storage-daemon/storage-daemon build/storage/storage-daemon/storage-daemon-cli \ + build/crypto/fift build/crypto/tlbc build/crypto/func build/crypto/create-state build/blockchain-explorer/blockchain-explorer \ + build/validator-engine-console/validator-engine-console build/tonlib/tonlib-cli \ + build/tonlib/libtonlibjson.so build/http/http-proxy build/rldp-http-proxy/rldp-http-proxy \ + build/dht-server/dht-server build/lite-client/lite-client build/validator-engine/validator-engine \ + build/utils/generate-random-id build/utils/json2tlo build/adnl/adnl-proxy build/emulator/libemulator.so \ + artifacts + test $? -eq 0 || { echo "Can't copy final binaries"; exit 1; } + chmod +x artifacts/* + cp -R crypto/smartcont artifacts + cp -R crypto/fift/lib artifacts +fi + +if [ "$with_tests" = true ]; then + cd build +# ctest --output-on-failure -E "test-catchain|test-actors|test-smartcont|test-adnl|test-validator-session-state|test-dht|test-rldp" + ctest --output-on-failure -E "test-adnl" +fi diff --git a/assembly/native/build-ubuntu-shared.sh b/assembly/native/build-ubuntu-shared.sh new file mode 100644 index 00000000..12c819cd --- /dev/null +++ b/assembly/native/build-ubuntu-shared.sh @@ -0,0 +1,122 @@ +#/bin/bash + +#sudo apt-get update +#sudo apt-get install -y build-essential git cmake ninja-build zlib1g-dev libsecp256k1-dev libmicrohttpd-dev libsodium-dev + +with_tests=false +with_artifacts=false + + +while getopts 'ta' flag; do + case "${flag}" in + t) with_tests=true ;; + a) with_artifacts=true ;; + *) break + ;; + esac +done + +if [ ! -d "build" ]; then + mkdir build + cd build +else + cd build + rm -rf .ninja* CMakeCache.txt +fi + +export CC=$(which clang-16) +export CXX=$(which clang++-16) +export CCACHE_DISABLE=1 + +if [ ! -d "openssl_3" ]; then + git clone https://github.com/openssl/openssl openssl_3 + cd openssl_3 + opensslPath=`pwd` + git checkout openssl-3.1.4 + ./config + make build_libs -j12 + test $? -eq 0 || { echo "Can't compile openssl_3"; exit 1; } + cd .. +else + opensslPath=$(pwd)/openssl_3 + echo "Using compiled openssl_3" +fi + +cmake -GNinja .. \ +-DCMAKE_BUILD_TYPE=Release \ +-DOPENSSL_ROOT_DIR=$opensslPath \ +-DOPENSSL_INCLUDE_DIR=$opensslPath/include \ +-DOPENSSL_CRYPTO_LIBRARY=$opensslPath/libcrypto.so + + +test $? -eq 0 || { echo "Can't configure ton"; exit 1; } + +if [ "$with_tests" = true ]; then +ninja storage-daemon storage-daemon-cli fift func tonlib tonlibjson tonlib-cli \ + validator-engine lite-client pow-miner validator-engine-console blockchain-explorer \ + generate-random-id json2tlo dht-server http-proxy rldp-http-proxy \ + adnl-proxy create-state emulator test-ed25519 test-ed25519-crypto test-bigint \ + test-vm test-fift test-cells test-smartcont test-net test-tdactor test-tdutils \ + test-tonlib-offline test-adnl test-dht test-rldp test-rldp2 test-catchain \ + test-fec test-tddb test-db test-validator-session-state + test $? -eq 0 || { echo "Can't compile ton"; exit 1; } +else +ninja storage-daemon storage-daemon-cli fift func tonlib tonlibjson tonlib-cli \ + validator-engine lite-client pow-miner validator-engine-console blockchain-explorer \ + generate-random-id json2tlo dht-server http-proxy rldp-http-proxy \ + adnl-proxy create-state emulator + test $? -eq 0 || { echo "Can't compile ton"; exit 1; } +fi + +strip -g storage/storage-daemon/storage-daemon \ + storage/storage-daemon/storage-daemon-cli \ + blockchain-explorer/blockchain-explorer \ + crypto/fift \ + crypto/tlbc \ + crypto/func \ + crypto/create-state \ + validator-engine-console/validator-engine-console \ + tonlib/tonlib-cli \ + tonlib/libtonlibjson.so.0.5 \ + http/http-proxy \ + rldp-http-proxy/rldp-http-proxy \ + dht-server/dht-server \ + lite-client/lite-client \ + validator-engine/validator-engine \ + utils/generate-random-id \ + utils/json2tlo \ + adnl/adnl-proxy \ + emulator/libemulator.* + +test $? -eq 0 || { echo "Can't strip final binaries"; exit 1; } + +# simple binaries' test +./storage/storage-daemon/storage-daemon -V || exit 1 +./validator-engine/validator-engine -V || exit 1 +./lite-client/lite-client -V || exit 1 +./crypto/fift -V || exit 1 + +cd .. + +if [ "$with_artifacts" = true ]; then + rm -rf artifacts + mkdir artifacts + mv build/tonlib/libtonlibjson.so.0.5 build/tonlib/libtonlibjson.so + cp build/storage/storage-daemon/storage-daemon build/storage/storage-daemon/storage-daemon-cli \ + build/crypto/fift build/crypto/tlbc build/crypto/func build/crypto/create-state build/blockchain-explorer/blockchain-explorer \ + build/validator-engine-console/validator-engine-console build/tonlib/tonlib-cli \ + build/tonlib/libtonlibjson.so build/http/http-proxy build/rldp-http-proxy/rldp-http-proxy \ + build/dht-server/dht-server build/lite-client/lite-client build/validator-engine/validator-engine \ + build/utils/generate-random-id build/utils/json2tlo build/adnl/adnl-proxy build/emulator/libemulator.so \ + artifacts + test $? -eq 0 || { echo "Can't copy final binaries"; exit 1; } + chmod +x artifacts/* + cp -R crypto/smartcont artifacts + cp -R crypto/fift/lib artifacts +fi + +if [ "$with_tests" = true ]; then + cd build +# ctest --output-on-failure -E "test-catchain|test-actors|test-smartcont|test-adnl|test-validator-session-state|test-dht|test-rldp" + ctest --output-on-failure --timeout 1800 +fi \ No newline at end of file diff --git a/assembly/native/build-windows-github.bat b/assembly/native/build-windows-github.bat new file mode 100644 index 00000000..7cad8c7e --- /dev/null +++ b/assembly/native/build-windows-github.bat @@ -0,0 +1,2 @@ +call "C:\Program Files\Microsoft Visual Studio\2022\%1\VC\Auxiliary\Build\vcvars64.bat" +call build-windows.bat -t \ No newline at end of file diff --git a/assembly/native/build-windows.bat b/assembly/native/build-windows.bat new file mode 100644 index 00000000..2e3f2082 --- /dev/null +++ b/assembly/native/build-windows.bat @@ -0,0 +1,193 @@ +REM execute this script inside elevated (Run as Administrator) console "x64 Native Tools Command Prompt for VS 2022" + +echo off + +echo Installing chocolatey windows package manager... +@"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin" +choco -? +IF %errorlevel% NEQ 0 ( + echo Can't install chocolatey + exit /b %errorlevel% +) + +choco feature enable -n allowEmptyChecksums + +echo Installing pkgconfiglite... +choco install -y pkgconfiglite +IF errorlevel 1 ( + echo Can't install pkgconfiglite + exit /b %errorlevel% +) + +echo Installing ninja... +choco install -y ninja +IF errorlevel 1 ( + echo Can't install ninja + exit /b %errorlevel% +) + +if not exist "zlib" ( +git clone https://github.com/madler/zlib.git +cd zlib\contrib\vstudio\vc14 +msbuild zlibstat.vcxproj /p:Configuration=ReleaseWithoutAsm /p:platform=x64 -p:PlatformToolset=v143 + +IF errorlevel 1 ( + echo Can't install zlib + exit /b %errorlevel% +) +cd ..\..\..\.. +) else ( +echo Using zlib... +) + +if not exist "secp256k1" ( +git clone https://github.com/libbitcoin/secp256k1.git +cd secp256k1\builds\msvc\vs2017 +msbuild /p:Configuration=StaticRelease -p:PlatformToolset=v143 -p:Platform=x64 +IF errorlevel 1 ( + echo Can't install secp256k1 + exit /b %errorlevel% +) +cd ..\..\..\.. +) else ( +echo Using secp256k1... +) + + +if not exist "libsodium" ( +curl -Lo libsodium-1.0.18-stable-msvc.zip https://download.libsodium.org/libsodium/releases/libsodium-1.0.18-stable-msvc.zip +IF errorlevel 1 ( + echo Can't download libsodium + exit /b %errorlevel% +) +unzip libsodium-1.0.18-stable-msvc.zip +) else ( +echo Using libsodium... +) + +if not exist "openssl-3.1.4" ( +curl -Lo openssl-3.1.4.zip https://github.com/neodiX42/precompiled-openssl-win64/raw/main/openssl-3.1.4.zip +IF errorlevel 1 ( + echo Can't download OpenSSL + exit /b %errorlevel% +) +unzip -q openssl-3.1.4.zip +) else ( +echo Using openssl... +) + +if not exist "libmicrohttpd-0.9.77-w32-bin" ( +curl -Lo libmicrohttpd-0.9.77-w32-bin.zip https://github.com/neodiX42/precompiled-openssl-win64/raw/main/libmicrohttpd-0.9.77-w32-bin.zip +IF errorlevel 1 ( + echo Can't download libmicrohttpd + exit /b %errorlevel% +) +unzip -q libmicrohttpd-0.9.77-w32-bin.zip +) else ( +echo Using libmicrohttpd... +) + +if not exist "readline-5.0-1-lib" ( +curl -Lo readline-5.0-1-lib.zip https://github.com/neodiX42/precompiled-openssl-win64/raw/main/readline-5.0-1-lib.zip +IF errorlevel 1 ( + echo Can't download readline + exit /b %errorlevel% +) +unzip -q -d readline-5.0-1-lib readline-5.0-1-lib.zip +) else ( +echo Using readline... +) + + +set root=%cd% +echo %root% +set SODIUM_DIR=%root%\libsodium + +mkdir build +cd build +cmake -GNinja -DCMAKE_BUILD_TYPE=Release ^ +-DPORTABLE=1 ^ +-DSODIUM_USE_STATIC_LIBS=1 ^ +-DSECP256K1_FOUND=1 ^ +-DSECP256K1_INCLUDE_DIR=%root%\secp256k1\include ^ +-DSECP256K1_LIBRARY=%root%\secp256k1\bin\x64\Release\v143\static\secp256k1.lib ^ +-DMHD_FOUND=1 ^ +-DMHD_LIBRARY=%root%\libmicrohttpd-0.9.77-w32-bin\x86_64\VS2019\Release-static\libmicrohttpd.lib ^ +-DMHD_INCLUDE_DIR=%root%\libmicrohttpd-0.9.77-w32-bin\x86_64\VS2019\Release-static ^ +-DZLIB_FOUND=1 ^ +-DZLIB_INCLUDE_DIR=%root%\zlib ^ +-DZLIB_LIBRARIES=%root%\zlib\contrib\vstudio\vc14\x64\ZlibStatReleaseWithoutAsm\zlibstat.lib ^ +-DOPENSSL_FOUND=1 ^ +-DOPENSSL_INCLUDE_DIR=%root%/openssl-3.1.4/x64/include ^ +-DOPENSSL_CRYPTO_LIBRARY=%root%/openssl-3.1.4/x64/lib/libcrypto_static.lib ^ +-DCMAKE_CXX_FLAGS="/DTD_WINDOWS=1 /EHsc /bigobj" .. +IF errorlevel 1 ( + echo Can't configure TON + exit /b %errorlevel% +) + +IF "%1"=="-t" ( +ninja storage-daemon storage-daemon-cli blockchain-explorer fift func tonlib tonlibjson ^ +tonlib-cli validator-engine lite-client pow-miner validator-engine-console generate-random-id ^ +json2tlo dht-server http-proxy rldp-http-proxy adnl-proxy create-state create-hardfork emulator ^ +test-ed25519 test-ed25519-crypto test-bigint test-vm test-fift test-cells test-smartcont test-net ^ +test-tdactor test-tdutils test-tonlib-offline test-adnl test-dht test-rldp test-rldp2 test-catchain ^ +test-fec test-tddb test-db test-validator-session-state +IF errorlevel 1 ( + echo Can't compile TON + exit /b %errorlevel% +) +) else ( +ninja storage-daemon storage-daemon-cli blockchain-explorer fift func tonlib tonlibjson ^ +tonlib-cli validator-engine lite-client pow-miner validator-engine-console generate-random-id ^ +json2tlo dht-server http-proxy rldp-http-proxy adnl-proxy create-state create-hardfork emulator +IF errorlevel 1 ( + echo Can't compile TON + exit /b %errorlevel% +) +) + +copy validator-engine\validator-engine.exe test +IF errorlevel 1 ( + echo validator-engine.exe does not exist + exit /b %errorlevel% +) + +IF "%1"=="-t" ( + echo Running tests... +REM ctest -C Release --output-on-failure -E "test-catchain|test-actors|test-validator-session-state" + ctest -C Release --output-on-failure --timeout 1800 + IF errorlevel 1 ( + echo Some tests failed + exit /b %errorlevel% + ) +) + + +echo Creating artifacts... +cd .. +mkdir artifacts +mkdir artifacts\smartcont +mkdir artifacts\lib + +for %%I in (build\storage\storage-daemon\storage-daemon.exe ^ +build\storage\storage-daemon\storage-daemon-cli.exe ^ +build\blockchain-explorer\blockchain-explorer.exe ^ +build\crypto\fift.exe ^ +build\crypto\tlbc.exe ^ +build\crypto\func.exe ^ +build\crypto\create-state.exe ^ +build\validator-engine-console\validator-engine-console.exe ^ +build\tonlib\tonlib-cli.exe ^ +build\tonlib\tonlibjson.dll ^ +build\http\http-proxy.exe ^ +build\rldp-http-proxy\rldp-http-proxy.exe ^ +build\dht-server\dht-server.exe ^ +build\lite-client\lite-client.exe ^ +build\validator-engine\validator-engine.exe ^ +build\utils\generate-random-id.exe ^ +build\utils\json2tlo.exe ^ +build\adnl\adnl-proxy.exe ^ +build\emulator\emulator.dll) do (strip -g %%I & copy %%I artifacts\) +xcopy /e /k /h /i crypto\smartcont artifacts\smartcont +xcopy /e /k /h /i crypto\fift\lib artifacts\lib diff --git a/assembly/nix/build-linux-arm64-nix.sh b/assembly/nix/build-linux-arm64-nix.sh new file mode 100644 index 00000000..bb859141 --- /dev/null +++ b/assembly/nix/build-linux-arm64-nix.sh @@ -0,0 +1,20 @@ +#/bin/bash + +nix-build --version +test $? -eq 0 || { echo "Nix is not installed!"; exit 1; } + +cp assembly/nix/linux-arm64* . +cp assembly/nix/microhttpd.nix . +cp assembly/nix/openssl.nix . +export NIX_PATH=nixpkgs=https://github.com/nixOS/nixpkgs/archive/23.05.tar.gz + +nix-build linux-arm64-static.nix +mkdir artifacts +cp ./result/bin/* artifacts/ +chmod +x artifacts/* +rm -rf result +nix-build linux-arm64-tonlib.nix +cp ./result/lib/libtonlibjson.so.0.5 artifacts/libtonlibjson.so +cp ./result/lib/libemulator.so artifacts/ +cp -r crypto/fift/lib artifacts/ +cp -r crypto/smartcont artifacts/ \ No newline at end of file diff --git a/assembly/nix/build-linux-x86-64-nix.sh b/assembly/nix/build-linux-x86-64-nix.sh new file mode 100644 index 00000000..eca6fe58 --- /dev/null +++ b/assembly/nix/build-linux-x86-64-nix.sh @@ -0,0 +1,20 @@ +#/bin/bash + +nix-build --version +test $? -eq 0 || { echo "Nix is not installed!"; exit 1; } + +cp assembly/nix/linux-x86-64* . +cp assembly/nix/microhttpd.nix . +cp assembly/nix/openssl.nix . +export NIX_PATH=nixpkgs=https://github.com/nixOS/nixpkgs/archive/23.05.tar.gz + +nix-build linux-x86-64-static.nix +mkdir artifacts +cp ./result/bin/* artifacts/ +chmod +x artifacts/* +rm -rf result +nix-build linux-x86-64-tonlib.nix +cp ./result/lib/libtonlibjson.so.0.5 artifacts/libtonlibjson.so +cp ./result/lib/libemulator.so artifacts/ +cp -r crypto/fift/lib artifacts/ +cp -r crypto/smartcont artifacts/ \ No newline at end of file diff --git a/assembly/nix/build-macos-nix.sh b/assembly/nix/build-macos-nix.sh new file mode 100644 index 00000000..fdf674a6 --- /dev/null +++ b/assembly/nix/build-macos-nix.sh @@ -0,0 +1,17 @@ +#/bin/bash + +nix-build --version +test $? -eq 0 || { echo "Nix is not installed!"; exit 1; } + +cp assembly/nix/macos-* . +export NIX_PATH=nixpkgs=https://github.com/nixOS/nixpkgs/archive/23.05.tar.gz +nix-build macos-static.nix +mkdir artifacts +cp ./result-bin/bin/* artifacts/ +chmod +x artifacts/* +rm -rf result-bin +nix-build macos-tonlib.nix +cp ./result/lib/libtonlibjson.dylib artifacts/ +cp ./result/lib/libemulator.dylib artifacts/ +cp -r crypto/fift/lib artifacts/ +cp -r crypto/smartcont artifacts/ \ No newline at end of file diff --git a/flake.lock b/assembly/nix/flakes/flake.lock similarity index 100% rename from flake.lock rename to assembly/nix/flakes/flake.lock diff --git a/flake.nix b/assembly/nix/flakes/flake.nix similarity index 100% rename from flake.nix rename to assembly/nix/flakes/flake.nix diff --git a/shell.nix b/assembly/nix/flakes/shell.nix similarity index 100% rename from shell.nix rename to assembly/nix/flakes/shell.nix diff --git a/assembly/nix/linux-arm64-static.nix b/assembly/nix/linux-arm64-static.nix new file mode 100644 index 00000000..616dfba5 --- /dev/null +++ b/assembly/nix/linux-arm64-static.nix @@ -0,0 +1,45 @@ +# export NIX_PATH=nixpkgs=https://github.com/nixOS/nixpkgs/archive/23.05.tar.gz + +{ pkgs ? import { system = builtins.currentSystem; } +, lib ? pkgs.lib +, stdenv ? pkgs.stdenv +}: +let + microhttpdmy = (import ./microhttpd.nix) {}; +in +with import microhttpdmy; +stdenv.mkDerivation { + pname = "ton"; + version = "dev-bin"; + + src = ./.; + + nativeBuildInputs = with pkgs; + [ + cmake ninja git pkg-config + ]; + + buildInputs = with pkgs; + [ + pkgsStatic.openssl microhttpdmy pkgsStatic.zlib pkgsStatic.libsodium.dev pkgsStatic.secp256k1 glibc.static + ]; + + makeStatic = true; + doCheck = true; + + cmakeFlags = [ + "-DTON_USE_ABSEIL=OFF" + "-DNIX=ON" + "-DBUILD_SHARED_LIBS=OFF" + "-DCMAKE_LINK_SEARCH_START_STATIC=ON" + "-DCMAKE_LINK_SEARCH_END_STATIC=ON" + "-DMHD_FOUND=1" + "-DMHD_INCLUDE_DIR=${microhttpdmy}/usr/local/include" + "-DMHD_LIBRARY=${microhttpdmy}/usr/local/lib/libmicrohttpd.a" + "-DCMAKE_CTEST_ARGUMENTS=--timeout;1800" + ]; + + LDFLAGS = [ + "-static-libgcc" "-static-libstdc++" "-static" + ]; +} diff --git a/assembly/nix/linux-arm64-tonlib.nix b/assembly/nix/linux-arm64-tonlib.nix new file mode 100644 index 00000000..a753423b --- /dev/null +++ b/assembly/nix/linux-arm64-tonlib.nix @@ -0,0 +1,44 @@ +# export NIX_PATH=nixpkgs=https://github.com/nixOS/nixpkgs/archive/23.05.tar.gz +{ + pkgs ? import { system = builtins.currentSystem; } +, lib ? pkgs.lib +, stdenv ? pkgs.stdenv +}: +let + microhttpdmy = (import ./microhttpd.nix) {}; +in +with import microhttpdmy; +pkgs.llvmPackages_16.stdenv.mkDerivation { + pname = "ton"; + version = "dev-lib"; + + src = ./.; + + nativeBuildInputs = with pkgs; + [ + cmake ninja git pkg-config + ]; + + buildInputs = with pkgs; + [ + pkgsStatic.openssl microhttpdmy pkgsStatic.zlib pkgsStatic.libsodium.dev pkgsStatic.secp256k1 + ]; + + dontAddStaticConfigureFlags = false; + + cmakeFlags = [ + "-DTON_USE_ABSEIL=OFF" + "-DNIX=ON" + "-DMHD_FOUND=1" + "-DMHD_INCLUDE_DIR=${microhttpdmy}/usr/local/include" + "-DMHD_LIBRARY=${microhttpdmy}/usr/local/lib/libmicrohttpd.a" + ]; + + LDFLAGS = [ + "-static-libgcc" "-static-libstdc++" "-fPIC" "-fcommon" + ]; + + ninjaFlags = [ + "tonlibjson" "emulator" + ]; +} diff --git a/assembly/nix/linux-x86-64-static.nix b/assembly/nix/linux-x86-64-static.nix new file mode 100644 index 00000000..616dfba5 --- /dev/null +++ b/assembly/nix/linux-x86-64-static.nix @@ -0,0 +1,45 @@ +# export NIX_PATH=nixpkgs=https://github.com/nixOS/nixpkgs/archive/23.05.tar.gz + +{ pkgs ? import { system = builtins.currentSystem; } +, lib ? pkgs.lib +, stdenv ? pkgs.stdenv +}: +let + microhttpdmy = (import ./microhttpd.nix) {}; +in +with import microhttpdmy; +stdenv.mkDerivation { + pname = "ton"; + version = "dev-bin"; + + src = ./.; + + nativeBuildInputs = with pkgs; + [ + cmake ninja git pkg-config + ]; + + buildInputs = with pkgs; + [ + pkgsStatic.openssl microhttpdmy pkgsStatic.zlib pkgsStatic.libsodium.dev pkgsStatic.secp256k1 glibc.static + ]; + + makeStatic = true; + doCheck = true; + + cmakeFlags = [ + "-DTON_USE_ABSEIL=OFF" + "-DNIX=ON" + "-DBUILD_SHARED_LIBS=OFF" + "-DCMAKE_LINK_SEARCH_START_STATIC=ON" + "-DCMAKE_LINK_SEARCH_END_STATIC=ON" + "-DMHD_FOUND=1" + "-DMHD_INCLUDE_DIR=${microhttpdmy}/usr/local/include" + "-DMHD_LIBRARY=${microhttpdmy}/usr/local/lib/libmicrohttpd.a" + "-DCMAKE_CTEST_ARGUMENTS=--timeout;1800" + ]; + + LDFLAGS = [ + "-static-libgcc" "-static-libstdc++" "-static" + ]; +} diff --git a/assembly/nix/linux-x86-64-tonlib.nix b/assembly/nix/linux-x86-64-tonlib.nix new file mode 100644 index 00000000..ac183d2b --- /dev/null +++ b/assembly/nix/linux-x86-64-tonlib.nix @@ -0,0 +1,54 @@ +# export NIX_PATH=nixpkgs=https://github.com/nixOS/nixpkgs/archive/23.11.tar.gz +# copy linux-x86-64-tonlib.nix to git root directory and execute: +# nix-build linux-x86-64-tonlib.nix +{ + pkgs ? import { system = builtins.currentSystem; } +, lib ? pkgs.lib +, stdenv ? pkgs.stdenv +}: +let + system = builtins.currentSystem; + + nixos1909 = (import (builtins.fetchTarball { + url = "https://channels.nixos.org/nixos-19.09/nixexprs.tar.xz"; + sha256 = "1vp1h2gkkrckp8dzkqnpcc6xx5lph5d2z46sg2cwzccpr8ay58zy"; + }) { inherit system; }); + glibc227 = nixos1909.glibc // { pname = "glibc"; }; + stdenv227 = let + cc = pkgs.wrapCCWith { + cc = nixos1909.buildPackages.gcc-unwrapped; + libc = glibc227; + bintools = pkgs.binutils.override { libc = glibc227; }; + }; + in (pkgs.overrideCC pkgs.stdenv cc); + +in +stdenv227.mkDerivation { + pname = "ton"; + version = "dev-lib"; + + src = ./.; + + nativeBuildInputs = with pkgs; + [ cmake ninja git pkg-config ]; + + buildInputs = with pkgs; + [ + pkgsStatic.openssl pkgsStatic.zlib pkgsStatic.libmicrohttpd.dev pkgsStatic.libsodium.dev pkgsStatic.secp256k1 + ]; + + dontAddStaticConfigureFlags = false; + + cmakeFlags = [ + "-DTON_USE_ABSEIL=OFF" + "-DNIX=ON" + ]; + + LDFLAGS = [ + "-static-libgcc" "-static-libstdc++" "-fPIC" + ]; + + ninjaFlags = [ + "tonlibjson" "emulator" + ]; +} diff --git a/assembly/nix/macos-static.nix b/assembly/nix/macos-static.nix new file mode 100644 index 00000000..e65ec1a8 --- /dev/null +++ b/assembly/nix/macos-static.nix @@ -0,0 +1,65 @@ +# export NIX_PATH=nixpkgs=https://github.com/nixOS/nixpkgs/archive/23.05.tar.gz + +{ pkgs ? import { system = builtins.currentSystem; } +, lib ? pkgs.lib +, stdenv ? pkgs.stdenv +}: + +pkgs.llvmPackages_14.stdenv.mkDerivation { + pname = "ton"; + version = "dev-bin"; + + src = ./.; + + nativeBuildInputs = with pkgs; + [ cmake ninja git pkg-config ]; + + buildInputs = with pkgs; + lib.forEach [ + secp256k1 libsodium.dev libmicrohttpd.dev gmp.dev nettle.dev libtasn1.dev libidn2.dev libunistring.dev gettext (gnutls.override { withP11-kit = false; }).dev + ] + (x: x.overrideAttrs(oldAttrs: rec { configureFlags = (oldAttrs.configureFlags or []) ++ [ "--enable-static" "--disable-shared" "--disable-tests" ]; dontDisableStatic = true; })) + ++ [ + darwin.apple_sdk.frameworks.CoreFoundation + (openssl.override { static = true; }).dev + (zlib.override { shared = false; }).dev + (libiconv.override { enableStatic = true; enableShared = false; }) + ]; + + + dontAddStaticConfigureFlags = true; + makeStatic = true; + doCheck = true; + + configureFlags = []; + + cmakeFlags = [ + "-DTON_USE_ABSEIL=OFF" + "-DNIX=ON" + "-DCMAKE_CROSSCOMPILING=OFF" + "-DCMAKE_LINK_SEARCH_START_STATIC=ON" + "-DCMAKE_LINK_SEARCH_END_STATIC=ON" + "-DBUILD_SHARED_LIBS=OFF" + "-DCMAKE_CXX_FLAGS=-stdlib=libc++" + "-DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=11.3" + "-DCMAKE_CTEST_ARGUMENTS=--timeout;1800" + ]; + + LDFLAGS = [ + "-static-libstdc++" + "-framework CoreFoundation" + ]; + + postInstall = '' + moveToOutput bin "$bin" + ''; + + preFixup = '' + for fn in "$bin"/bin/* "$out"/lib/*.dylib; do + echo Fixing libc++ in "$fn" + install_name_tool -change "$(otool -L "$fn" | grep libc++.1 | cut -d' ' -f1 | xargs)" libc++.1.dylib "$fn" + install_name_tool -change "$(otool -L "$fn" | grep libc++abi.1 | cut -d' ' -f1 | xargs)" libc++abi.dylib "$fn" + done + ''; + outputs = [ "bin" "out" ]; +} \ No newline at end of file diff --git a/assembly/nix/macos-tonlib.nix b/assembly/nix/macos-tonlib.nix new file mode 100644 index 00000000..c362de4e --- /dev/null +++ b/assembly/nix/macos-tonlib.nix @@ -0,0 +1,55 @@ +# export NIX_PATH=nixpkgs=https://github.com/nixOS/nixpkgs/archive/23.05.tar.gz + +{ pkgs ? import { system = builtins.currentSystem; } +, lib ? pkgs.lib +, stdenv ? pkgs.stdenv +}: + +pkgs.llvmPackages_14.stdenv.mkDerivation { + pname = "ton"; + version = "dev-lib"; + + src = ./.; + + nativeBuildInputs = with pkgs; + [ cmake ninja git pkg-config ]; + + buildInputs = with pkgs; + lib.forEach [ + secp256k1 libsodium.dev libmicrohttpd.dev gmp.dev nettle.dev libtasn1.dev libidn2.dev libunistring.dev gettext (gnutls.override { withP11-kit = false; }).dev + ] (x: x.overrideAttrs(oldAttrs: rec { configureFlags = (oldAttrs.configureFlags or []) ++ [ "--enable-static" "--disable-shared" "--disable-tests" ]; dontDisableStatic = true; })) + ++ [ + darwin.apple_sdk.frameworks.CoreFoundation + (openssl.override { static = true; }).dev + (zlib.override { shared = false; }).dev + (libiconv.override { enableStatic = true; enableShared = false; }) + ]; + + dontAddStaticConfigureFlags = true; + + configureFlags = []; + + cmakeFlags = [ + "-DTON_USE_ABSEIL=OFF" + "-DNIX=ON" + "-DCMAKE_CXX_FLAGS=-stdlib=libc++" + "-DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=11.3" + ]; + + LDFLAGS = [ + "-static-libstdc++" + "-framework CoreFoundation" + ]; + + ninjaFlags = [ + "tonlibjson" "emulator" + ]; + + preFixup = '' + for fn in $out/bin/* $out/lib/*.dylib; do + echo Fixing libc++ in "$fn" + install_name_tool -change "$(otool -L "$fn" | grep libc++.1 | cut -d' ' -f1 | xargs)" libc++.1.dylib "$fn" + install_name_tool -change "$(otool -L "$fn" | grep libc++abi.1 | cut -d' ' -f1 | xargs)" libc++abi.dylib "$fn" + done + ''; +} \ No newline at end of file diff --git a/assembly/nix/microhttpd.nix b/assembly/nix/microhttpd.nix new file mode 100644 index 00000000..4f871425 --- /dev/null +++ b/assembly/nix/microhttpd.nix @@ -0,0 +1,28 @@ +{ pkgs ? import { system = builtins.currentSystem; } +, stdenv ? pkgs.stdenv +, fetchgit ? pkgs.fetchgit +}: + +stdenv.mkDerivation rec { + name = "microhttpdmy"; + + + src = fetchgit { + url = "https://git.gnunet.org/libmicrohttpd.git"; + rev = "refs/tags/v0.9.77"; + sha256 = "sha256-x+nfB07PbZwBlFc6kZZFYiRpk0a3QN/ByHB+hC8na/o="; + }; + + nativeBuildInputs = with pkgs; [ automake libtool autoconf texinfo ]; + + buildInputs = with pkgs; [ ]; + + configurePhase = '' + ./autogen.sh + ./configure --enable-static --disable-tests --disable-benchmark --disable-shared --disable-https --with-pic + ''; + + installPhase = '' + make install DESTDIR=$out + ''; +} diff --git a/assembly/nix/openssl.nix b/assembly/nix/openssl.nix new file mode 100644 index 00000000..8d30aa50 --- /dev/null +++ b/assembly/nix/openssl.nix @@ -0,0 +1,30 @@ +{ pkgs ? import { system = builtins.currentSystem; } +, stdenv ? pkgs.stdenv +, fetchFromGitHub ? pkgs.fetchFromGitHub +}: + +stdenv.mkDerivation rec { + name = "opensslmy"; + + src = fetchFromGitHub { + owner = "openssl"; + repo = "openssl"; + rev = "refs/tags/openssl-3.1.4"; + sha256 = "sha256-Vvf1wiNb4ikg1lIS9U137aodZ2JzM711tSWMJFYWtWI="; + }; + + nativeBuildInputs = with pkgs; [ perl ]; + + buildInputs = with pkgs; [ ]; + + postPatch = '' + patchShebangs Configure + ''; + + configurePhase = '' + ./Configure no-shared + ''; + installPhase = '' + make install DESTDIR=$out + ''; +} diff --git a/.github/script/fift-func-wasm-build-ubuntu.sh b/assembly/wasm/fift-func-wasm-build-ubuntu.sh old mode 100755 new mode 100644 similarity index 71% rename from .github/script/fift-func-wasm-build-ubuntu.sh rename to assembly/wasm/fift-func-wasm-build-ubuntu.sh index feac19e3..9ca23cc0 --- a/.github/script/fift-func-wasm-build-ubuntu.sh +++ b/assembly/wasm/fift-func-wasm-build-ubuntu.sh @@ -1,7 +1,24 @@ # The script builds funcfift compiler to WASM -# dependencies: -#sudo apt-get install -y build-essential git make cmake clang libgflags-dev zlib1g-dev libssl-dev libreadline-dev libmicrohttpd-dev pkg-config libgsl-dev python3 python3-dev python3-pip nodejs libevent-dev +# Execute these prerequisites first +# sudo apt update +# sudo apt install -y build-essential git make cmake ninja-build clang libgflags-dev zlib1g-dev libssl-dev \ +# libreadline-dev libmicrohttpd-dev pkg-config libgsl-dev python3 python3-dev python3-pip \ +# nodejs libsecp256k1-dev libsodium-dev automake libtool + +# wget https://apt.llvm.org/llvm.sh +# chmod +x llvm.sh +# sudo ./llvm.sh 16 all + +with_artifacts=false + +while getopts 'a' flag; do + case "${flag}" in + a) with_artifacts=true ;; + *) break + ;; + esac +done export CC=$(which clang-16) export CXX=$(which clang++-16) @@ -39,8 +56,7 @@ mkdir build cd build cmake -GNinja -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_CXX_STANDARD=17 \ --DZLIB_LIBRARY=/usr/lib/x86_64-linux-gnu/libz.so \ --DZLIB_INCLUDE_DIR=$ZLIB_DIR \ +-DOPENSSL_FOUND=1 \ -DOPENSSL_ROOT_DIR=$OPENSSL_DIR \ -DOPENSSL_INCLUDE_DIR=$OPENSSL_DIR/include \ -DOPENSSL_CRYPTO_LIBRARY=$OPENSSL_DIR/libcrypto.so \ @@ -62,8 +78,9 @@ cd emsdk ./emsdk install 3.1.19 ./emsdk activate 3.1.19 EMSDK_DIR=`pwd` +ls $EMSDK_DIR -source $EMSDK_DIR/emsdk_env.sh +. $EMSDK_DIR/emsdk_env.sh export CC=$(which emcc) export CXX=$(which em++) export CCACHE_DISABLE=1 @@ -71,7 +88,7 @@ export CCACHE_DISABLE=1 cd ../openssl make clean -emconfigure ./Configure linux-generic32 no-shared no-dso no-engine no-unit-test no-ui +emconfigure ./Configure linux-generic32 no-shared no-dso no-engine no-unit-test sed -i 's/CROSS_COMPILE=.*/CROSS_COMPILE=/g' Makefile sed -i 's/-ldl//g' Makefile sed -i 's/-O3/-Os/g' Makefile @@ -101,22 +118,42 @@ test $? -eq 0 || { echo "Can't compile libsodium with emmake "; exit 1; } cd ../build emcmake cmake -DUSE_EMSCRIPTEN=ON -DCMAKE_BUILD_TYPE=Release \ --DZLIB_LIBRARY=$ZLIB_DIR/libz.a \ +-DZLIB_FOUND=1 \ +-DZLIB_LIBRARIES=$ZLIB_DIR/libz.a \ -DZLIB_INCLUDE_DIR=$ZLIB_DIR \ +-DOPENSSL_FOUND=1 \ -DOPENSSL_ROOT_DIR=$OPENSSL_DIR \ -DOPENSSL_INCLUDE_DIR=$OPENSSL_DIR/include \ -DOPENSSL_CRYPTO_LIBRARY=$OPENSSL_DIR/libcrypto.a \ -DOPENSSL_SSL_LIBRARY=$OPENSSL_DIR/libssl.a \ -DCMAKE_TOOLCHAIN_FILE=$EMSDK_DIR/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake \ -DCMAKE_CXX_FLAGS="-sUSE_ZLIB=1" \ +-DSECP256K1_FOUND=1 \ -DSECP256K1_INCLUDE_DIR=$SECP256K1_DIR/include \ -DSECP256K1_LIBRARY=$SECP256K1_DIR/.libs/libsecp256k1.a \ -DSODIUM_INCLUDE_DIR=$SODIUM_DIR/src/libsodium/include \ -DSODIUM_LIBRARY_RELEASE=$SODIUM_DIR/src/libsodium/.libs/libsodium.a \ --DSODIUM_LIBRARY_DEBUG=$SODIUM_DIR/src/libsodium/.libs/libsodium.a \ -DSODIUM_USE_STATIC_LIBS=ON .. test $? -eq 0 || { echo "Can't configure TON with emmake "; exit 1; } cp -R ../crypto/smartcont ../crypto/fift/lib crypto emmake make -j16 funcfiftlib func fift tlbc emulator-emscripten + +test $? -eq 0 || { echo "Can't compile TON with emmake "; exit 1; } + +if [ "$with_artifacts" = true ]; then + echo "Creating artifacts..." + cd .. + rm -rf artifacts + mkdir artifacts + ls build/crypto + cp build/crypto/fift* artifacts + cp build/crypto/func* artifacts + cp build/crypto/tlbc* artifacts + cp build/emulator/emulator-emscripten* artifacts + cp -R crypto/smartcont artifacts + cp -R crypto/fift/lib artifacts +fi + + diff --git a/blockchain-explorer/CMakeLists.txt b/blockchain-explorer/CMakeLists.txt index fc94e709..8aae8805 100644 --- a/blockchain-explorer/CMakeLists.txt +++ b/blockchain-explorer/CMakeLists.txt @@ -14,15 +14,28 @@ set(BLOCHAIN_EXPLORER_SOURCE add_executable(blockchain-explorer ${BLOCHAIN_EXPLORER_SOURCE}) if (NIX) - find_package(PkgConfig REQUIRED) - pkg_check_modules(MHD libmicrohttpd) - target_include_directories(blockchain-explorer PUBLIC ${MHD_INCLUDE_DIRS} ${MHD_STATIC_INCLUDE_DIRS}) - target_link_libraries(blockchain-explorer tdutils tdactor adnllite tl_lite_api tl-lite-utils ton_crypto ton_block ${MHD_LIBRARIES} ${MHD_STATIC_LIBRARIES}) + if (MHD_FOUND) + target_include_directories(blockchain-explorer PUBLIC ${MHD_INCLUDE_DIR}) + target_link_libraries(blockchain-explorer tdutils tdactor adnllite tl_lite_api tl-lite-utils ton_crypto ton_block ${MHD_LIBRARY}) + else() + find_package(PkgConfig REQUIRED) + pkg_check_modules(MHD libmicrohttpd) + target_include_directories(blockchain-explorer PUBLIC ${MHD_INCLUDE_DIR} ${MHD_STATIC_INCLUDE_DIRS}) + target_link_libraries(blockchain-explorer tdutils tdactor adnllite tl_lite_api tl-lite-utils ton_crypto ton_block ${MHD_LIBRARIES} ${MHD_STATIC_LIBRARIES}) + endif() else() - find_package(MHD) - target_include_directories(blockchain-explorer PUBLIC ${MHD_INCLUDE_DIRS}) - target_link_libraries(blockchain-explorer tdutils tdactor adnllite tl_lite_api tl-lite-utils ton_crypto ton_block ${MHD_LIBRARIES}) + if (MHD_FOUND) + target_include_directories(blockchain-explorer PUBLIC ${MHD_INCLUDE_DIR}) + target_link_libraries(blockchain-explorer tdutils tdactor adnllite tl_lite_api tl-lite-utils ton_crypto ton_block ${MHD_LIBRARY}) + else() + find_package(MHD) + target_include_directories(blockchain-explorer PUBLIC ${MHD_INCLUDE_DIR}) + target_link_libraries(blockchain-explorer tdutils tdactor adnllite tl_lite_api tl-lite-utils ton_crypto ton_block ${MHD_LIBRARY}) + endif() endif() +target_include_directories(blockchain-explorer PUBLIC ${MHD_INCLUDE_DIR}) +target_link_libraries(blockchain-explorer tdutils tdactor adnllite tl_lite_api tl-lite-utils ton_crypto ton_block ${MHD_LIBRARY}) + install(TARGETS blockchain-explorer RUNTIME DESTINATION bin) diff --git a/catchain/catchain-receiver.cpp b/catchain/catchain-receiver.cpp index 482cfb43..c8206de9 100644 --- a/catchain/catchain-receiver.cpp +++ b/catchain/catchain-receiver.cpp @@ -287,7 +287,9 @@ void CatChainReceiverImpl::add_block_cont_3(tl_object_ptrdelivered()); + LOG_CHECK(last_sent_block_->delivered()) + << "source=" << last_sent_block_->get_source_id() << " ill=" << last_sent_block_->is_ill() + << " height=" << last_sent_block_->get_height(); } active_send_ = false; diff --git a/crypto/CMakeLists.txt b/crypto/CMakeLists.txt index 62b0d216..0871d250 100644 --- a/crypto/CMakeLists.txt +++ b/crypto/CMakeLists.txt @@ -325,23 +325,20 @@ endif() if (MSVC) find_package(Sodium REQUIRED) target_compile_definitions(ton_crypto PUBLIC SODIUM_STATIC) - target_include_directories(ton_crypto_core PUBLIC $) - target_link_libraries(ton_crypto PUBLIC ${SECP256K1_LIBRARIES}) -elseif (ANDROID) + target_include_directories(ton_crypto_core PUBLIC $) + target_link_libraries(ton_crypto PUBLIC ${SECP256K1_LIBRARY}) +elseif (ANDROID OR EMSCRIPTEN) target_include_directories(ton_crypto_core PUBLIC $) target_link_libraries(ton_crypto PUBLIC $) else() - if (NOT USE_EMSCRIPTEN) + if (NOT SODIUM_FOUND) find_package(Sodium REQUIRED) - target_include_directories(ton_crypto_core PUBLIC $) - target_link_libraries(ton_crypto PUBLIC ${SECP256K1_LIBRARIES}) else() - target_include_directories(ton_crypto_core PUBLIC $) - target_link_libraries(ton_crypto PUBLIC ${SECP256K1_LIBRARY}) - endif() - if (NOT APPLE AND NOT USE_EMSCRIPTEN) - target_link_libraries(ton_crypto_core PUBLIC secp256k1) + message(STATUS "Using Sodium ${SODIUM_LIBRARY_RELEASE}") endif() + target_compile_definitions(ton_crypto PUBLIC SODIUM_STATIC) + target_include_directories(ton_crypto_core PUBLIC $) + target_link_libraries(ton_crypto PUBLIC ${SECP256K1_LIBRARY}) endif() target_include_directories(ton_crypto_core PUBLIC $) @@ -356,7 +353,7 @@ add_executable(test-ed25519-crypto test/test-ed25519-crypto.cpp) target_include_directories(test-ed25519-crypto PUBLIC $) target_link_libraries(test-ed25519-crypto PUBLIC ton_crypto) -add_library(fift-lib ${FIFT_SOURCE}) +add_library(fift-lib STATIC ${FIFT_SOURCE}) target_include_directories(fift-lib PUBLIC $) target_link_libraries(fift-lib PUBLIC ton_crypto ton_db tdutils ton_block) if (USE_EMSCRIPTEN) @@ -400,6 +397,7 @@ if (USE_EMSCRIPTEN) target_link_options(funcfiftlib PRIVATE -sIGNORE_MISSING_MAIN=1) target_link_options(funcfiftlib PRIVATE -sAUTO_NATIVE_LIBRARIES=0) target_link_options(funcfiftlib PRIVATE -sMODULARIZE=1) + target_link_options(funcfiftlib PRIVATE -sTOTAL_MEMORY=33554432) target_link_options(funcfiftlib PRIVATE -sALLOW_MEMORY_GROWTH=1) target_link_options(funcfiftlib PRIVATE -sALLOW_TABLE_GROWTH=1) target_link_options(funcfiftlib PRIVATE --embed-file ${CMAKE_CURRENT_SOURCE_DIR}/fift/lib@/fiftlib) @@ -506,7 +504,7 @@ if (NOT CMAKE_CROSSCOMPILING OR USE_EMSCRIPTEN) GenFif(DEST smartcont/auto/simple-wallet-ext-code SOURCE smartcont/simple-wallet-ext-code.fc NAME simple-wallet-ext) endif() -add_library(smc-envelope ${SMC_ENVELOPE_SOURCE}) +add_library(smc-envelope STATIC ${SMC_ENVELOPE_SOURCE}) target_include_directories(smc-envelope PUBLIC $) target_link_libraries(smc-envelope PUBLIC ton_crypto PRIVATE tdutils ton_block) if (NOT CMAKE_CROSSCOMPILING) diff --git a/crypto/test/test-smartcont.cpp b/crypto/test/test-smartcont.cpp index 98534bc5..7f512cea 100644 --- a/crypto/test/test-smartcont.cpp +++ b/crypto/test/test-smartcont.cpp @@ -489,7 +489,7 @@ void do_test_wallet(int revision) { auto address = std::move(res.address); auto iwallet = std::move(res.wallet); auto public_key = priv_key.get_public_key().move_as_ok().as_octet_string(); - ; + check_wallet_state(iwallet, 1, 123, public_key); // lets send a lot of messages @@ -1026,7 +1026,7 @@ class CheckedDns { } return action; }); - auto query = dns_->create_update_query(key_.value(), smc_actions).move_as_ok(); + auto query = dns_->create_update_query(key_.value(), smc_actions, query_id_++).move_as_ok(); CHECK(dns_.write().send_external_message(std::move(query)).code == 0); } map_dns_.update(entries); @@ -1081,6 +1081,7 @@ class CheckedDns { using ManualDns = ton::ManualDns; td::optional key_; td::Ref dns_; + td::uint32 query_id_ = 1; // Query id serve as "valid until", but in tests now() == 0 MapDns map_dns_; td::optional combined_map_dns_; diff --git a/crypto/vm/boc.h b/crypto/vm/boc.h index 1bff5b25..09ae1b66 100644 --- a/crypto/vm/boc.h +++ b/crypto/vm/boc.h @@ -52,6 +52,7 @@ class NewCellStorageStat { bool operator==(const Stat& other) const { return key() == other.key(); } + Stat(const Stat& other) = default; Stat& operator=(const Stat& other) = default; Stat& operator+=(const Stat& other) { cells += other.cells; diff --git a/example/android/README.md b/example/android/README.md index f17ba9d2..cf12ba30 100644 --- a/example/android/README.md +++ b/example/android/README.md @@ -6,27 +6,9 @@ Prerequisite: installed Java and set environment variable JAVA_HOME. ```bash git clone --recursive https://github.com/ton-blockchain/ton.git cd ton -wget https://dl.google.com/android/repository/android-ndk-r25b-linux.zip -unzip android-ndk-r25b-linux.zip -export JAVA_AWT_LIBRARY=NotNeeded -export JAVA_JVM_LIBRARY=NotNeeded -export JAVA_INCLUDE_PATH=${JAVA_HOME}/include -export JAVA_AWT_INCLUDE_PATH=${JAVA_HOME}/include -export JAVA_INCLUDE_PATH2=${JAVA_HOME}/include/linux - -export ANDROID_NDK_ROOT=$(pwd)/android-ndk-r25b -export OPENSSL_DIR=$(pwd)/example/android/third_party/crypto -export SECP256K1_INCLUDE_DIR=$(pwd)/example/android/third_party/secp256k1/include -export SECP256K1_LIBRARY=$(pwd)/example/android/third_party/secp256k1/.libs/libsecp256k1.a -export SODIUM_INCLUDE_DIR=$(pwd)/example/android/third_party/libsodium/libsodium-android-westmere/include -export SODIUM_LIBRARY=$(pwd)/example/android/third_party/libsodium/libsodium-android-westmere/lib/libsodium.a - -rm -rf example/android/src/drinkless/org/ton/TonApi.java -cd example/android/ -cmake -GNinja -DTON_ONLY_TONLIB=ON . -ninja prepare_cross_compiling -rm CMakeCache.txt -./build-all.sh +cp assembly/android/build-android-tonlib.sh . +chmod +x build-android-tonlib.sh +sudo -E ./build-android-tonlib.sh ``` # Generation of Tonlib libraries for iOS in Xcode diff --git a/example/android/build-all.sh b/example/android/build-all.sh index f436e361..6f97dec0 100755 --- a/example/android/build-all.sh +++ b/example/android/build-all.sh @@ -1,4 +1,6 @@ #!/bin/bash +echo ANDROID_NDK_ROOT = $ANDROID_NDK_ROOT + echo Building tonlib for x86... echo ARCH="x86" ./build.sh || exit 1 diff --git a/example/android/build.sh b/example/android/build.sh index 7f170dbc..06217255 100755 --- a/example/android/build.sh +++ b/example/android/build.sh @@ -42,14 +42,22 @@ ARCH=$ABI mkdir -p build-$ARCH cd build-$ARCH -cmake .. -GNinja -DPORTABLE=1 \ --DANDROID_ABI=x86 -DANDROID_PLATFORM=android-32 -DANDROID_NDK=${ANDROID_NDK_ROOT} \ --DCMAKE_TOOLCHAIN_FILE=${ANDROID_NDK_ROOT}/build/cmake/android.toolchain.cmake \ --DCMAKE_BUILD_TYPE=Release -DANDROID_ABI=${ABI} \ --DOPENSSL_ROOT_DIR=${OPENSSL_DIR}/${ORIG_ARCH} -DTON_ARCH="" \ +cmake .. -GNinja \ +-DPORTABLE=1 \ -DTON_ONLY_TONLIB=ON \ --DSECP256K1_INCLUDE_DIR=${SECP256K1_INCLUDE_DIR} -DSECP256K1_LIBRARY=${SECP256K1_LIBRARY} \ --DSODIUM_INCLUDE_DIR=${SODIUM_INCLUDE_DIR} -DSODIUM_LIBRARY_RELEASE=${SODIUM_LIBRARY_RELEASE} \ +-DTON_ARCH="" \ +-DANDROID_ABI=x86 \ +-DANDROID_PLATFORM=android-32 \ +-DANDROID_NDK=${ANDROID_NDK_ROOT} \ +-DCMAKE_TOOLCHAIN_FILE=${ANDROID_NDK_ROOT}/build/cmake/android.toolchain.cmake \ +-DCMAKE_BUILD_TYPE=Release \ +-DANDROID_ABI=${ABI} \ +-DOPENSSL_ROOT_DIR=${OPENSSL_DIR}/${ORIG_ARCH} \ +-DSECP256K1_FOUND=1 \ +-DSECP256K1_INCLUDE_DIR=${SECP256K1_INCLUDE_DIR} \ +-DSECP256K1_LIBRARY=${SECP256K1_LIBRARY} \ +-DSODIUM_INCLUDE_DIR=${SODIUM_INCLUDE_DIR} \ +-DSODIUM_LIBRARY_RELEASE=${SODIUM_LIBRARY_RELEASE} \ -DSODIUM_USE_STATIC_LIBS=1 \ -DBLST_LIB=${BLST_LIBRARY} || exit 1 diff --git a/lite-client/CMakeLists.txt b/lite-client/CMakeLists.txt index 53e09d77..598a8d28 100644 --- a/lite-client/CMakeLists.txt +++ b/lite-client/CMakeLists.txt @@ -1,6 +1,6 @@ cmake_minimum_required(VERSION 3.5 FATAL_ERROR) -add_library(lite-client-common lite-client-common.cpp lite-client-common.h) +add_library(lite-client-common STATIC lite-client-common.cpp lite-client-common.h) target_link_libraries(lite-client-common PUBLIC tdutils tdactor adnllite tl_api tl_lite_api tl-lite-utils ton_crypto ton_block) add_executable(lite-client lite-client.cpp lite-client.h) diff --git a/tdnet/td/net/TcpListener.cpp b/tdnet/td/net/TcpListener.cpp index 7b7364ba..e711cbbd 100644 --- a/tdnet/td/net/TcpListener.cpp +++ b/tdnet/td/net/TcpListener.cpp @@ -46,9 +46,11 @@ void TcpListener::start_up() { } void TcpListener::tear_down() { - // unsubscribe from socket updates - // nb: interface will be changed - td::actor::SchedulerContext::get()->get_poll().unsubscribe(server_socket_fd_.get_poll_info().get_pollable_fd_ref()); + if (!server_socket_fd_.empty()) { + // unsubscribe from socket updates + // nb: interface will be changed + td::actor::SchedulerContext::get()->get_poll().unsubscribe(server_socket_fd_.get_poll_info().get_pollable_fd_ref()); + } } void TcpListener::loop() { diff --git a/tdnet/test/net-test.cpp b/tdnet/test/net-test.cpp index bb084a67..d20be504 100644 --- a/tdnet/test/net-test.cpp +++ b/tdnet/test/net-test.cpp @@ -158,9 +158,11 @@ void run_server(int from_port, int to_port, bool is_first, bool use_tcp) { TEST(Net, PingPong) { SET_VERBOSITY_LEVEL(VERBOSITY_NAME(ERROR)); + int port1 = td::Random::fast(10000, 10999); + int port2 = td::Random::fast(11000, 11999); for (auto use_tcp : {false, true}) { - auto a = td::thread([use_tcp] { run_server(8091, 8092, true, use_tcp); }); - auto b = td::thread([use_tcp] { run_server(8092, 8091, false, use_tcp); }); + auto a = td::thread([=] { run_server(port1, port2, true, use_tcp); }); + auto b = td::thread([=] { run_server(port2, port1, false, use_tcp); }); a.join(); b.join(); } diff --git a/tdutils/td/utils/as.h b/tdutils/td/utils/as.h index c60c74e2..6015af29 100644 --- a/tdutils/td/utils/as.h +++ b/tdutils/td/utils/as.h @@ -76,12 +76,7 @@ class ConstAs { } // namespace detail -// no std::is_trivially_copyable in libstdc++ before 5.0 -#if __GLIBCXX__ -#define TD_IS_TRIVIALLY_COPYABLE(T) __has_trivial_copy(T) -#else #define TD_IS_TRIVIALLY_COPYABLE(T) std::is_trivially_copyable::value -#endif template = 0> diff --git a/test/test-catchain.cpp b/test/test-catchain.cpp index 149ea3e3..3131c2b9 100644 --- a/test/test-catchain.cpp +++ b/test/test-catchain.cpp @@ -186,6 +186,7 @@ class CatChainInst : public td::actor::Actor { void create_fork() { auto height = height_ - 1; //td::Random::fast(0, height_ - 1); + LOG(WARNING) << "Creating fork, source_id=" << idx_ << ", height=" << height; auto sum = prev_values_[height] + 1; td::uint64 x[2]; @@ -241,7 +242,8 @@ int main(int argc, char *argv[]) { td::actor::send_closure(adnl, &ton::adnl::Adnl::register_network_manager, network_manager.get()); }); - for (td::uint32 att = 0; att < 10; att++) { + for (td::uint32 att = 0; att < 20; att++) { + LOG(WARNING) << "Test #" << att; nodes.resize(total_nodes); scheduler.run_in_context([&] { @@ -296,7 +298,10 @@ int main(int argc, char *argv[]) { std::cout << "value=" << n.get_actor_unsafe().value() << std::endl; } - scheduler.run_in_context([&] { td::actor::send_closure(inst[0], &CatChainInst::create_fork); }); + td::uint32 fork_cnt = att < 10 ? 1 : (att - 10) / 5 + 2; + for (td::uint32 idx = 0; idx < fork_cnt; ++idx) { + scheduler.run_in_context([&] { td::actor::send_closure(inst[idx], &CatChainInst::create_fork); }); + } t = td::Timestamp::in(1.0); while (scheduler.run(1)) { diff --git a/tonlib/test/offline.cpp b/tonlib/test/offline.cpp index 35a95f5b..0fedc865 100644 --- a/tonlib/test/offline.cpp +++ b/tonlib/test/offline.cpp @@ -467,15 +467,20 @@ TEST(Tonlib, KeysApi) { make_object(make_object(key->public_key_, key->secret_.copy()))) .move_as_ok(); - auto err1 = sync_send(client, make_object( - new_local_password.copy(), td::SecureString("wrong password"), - make_object(copy_word_list()))) - .move_as_error(); + auto err1 = sync_send( + client, make_object(new_local_password.copy(), td::SecureString("wrong password"), + make_object(copy_word_list()))); + if (err1.is_ok()) { + if (err1.ok()->public_key_ != key->public_key_) { + err1 = td::Status::Error("imported key successfully, but the public key is different"); + } + } + err1.ensure_error(); auto err2 = sync_send(client, make_object(new_local_password.copy(), td::SecureString(), - make_object(copy_word_list()))) - .move_as_error(); - LOG(INFO) << err1 << " | " << err2; + make_object(copy_word_list()))); + err2.ensure_error(); + LOG(INFO) << err1.move_as_error() << " | " << err2.move_as_error(); auto imported_key = sync_send(client, make_object(new_local_password.copy(), mnemonic_password.copy(), make_object(copy_word_list()))) diff --git a/tonlib/tonlib/TonlibClient.cpp b/tonlib/tonlib/TonlibClient.cpp index 9ec664bb..bee7ef40 100644 --- a/tonlib/tonlib/TonlibClient.cpp +++ b/tonlib/tonlib/TonlibClient.cpp @@ -4995,6 +4995,8 @@ td::Status TonlibClient::do_request(const tonlib_api::importKey& request, if (!request.exported_key_) { return TonlibError::EmptyField("exported_key"); } + // Note: the mnemonic is considered valid if a certain hash starts with zero byte (see Mnemonic::is_basic_seed()) + // Therefore, importKey with invalid password has 1/256 chance to return OK TRY_RESULT(key, key_storage_.import_key(std::move(request.local_password_), std::move(request.mnemonic_password_), KeyStorage::ExportedKey{std::move(request.exported_key_->word_list_)})); TRY_RESULT(key_bytes, public_key_from_bytes(key.public_key.as_slice()));