1
0
Fork 0
mirror of https://github.com/ton-blockchain/ton synced 2025-02-13 11:42:18 +00:00
ton/assembly/native/build-ubuntu-portable.sh

217 lines
6.5 KiB
Bash
Raw Normal View History

Improve TON build scripts and some tests (#855) * fix macOS github actions * fix android tonlib GH action; * fixing wasm GH action * strip binaries * fix randomly failing ubuntu and wasm GH actions * fix randomly failing ubuntu and wasm GH actions * revert some changes * adding more nix scripts and automated native build scripts; debug static ton compilation * minor fix * do not use pkg_config if path specified * move wasm script, run with sudo action script * weird, fixing - cp: missing destination file operand after 'assembly/native/build-ubuntu-20.04-shared.sh' * weird, fixing - cp: missing destination file operand after 'assembly/native/build-ubuntu-20.04-shared.sh' * ok * some adjustments for android and win builds * some adjustments for android and win builds * moving stripping inside the build script * access rights handling; adding simple binaries' tests * make lite-client-common, fift-lib and smc-envelope deliberately static; add -a (artifacts) flag to build scripts; * minor wasm build fix; create separate tonlib android build script; remove outdated __has_trivial_copy(T) * add windows build - WIP * adjust android build; improve win build; * adjust sodium paths for android build; use proper compiler for windows build; * add github windows build auxiliary file * adjust wasm build * add portable ubuntu build * exclude some unstable tests for some time * compile portable binaries on ubuntu-20.04 * exclude some unstable tests * include static gsl * restart builds * restart builds * restart builds * remove libreadline, gsl and blas dependencies in linux build * add macos build script * install missing autoconf in macos builds * enable all tests and see what fails * enable win tests and restart others * enable win tests and fix test-smartcont.cpp * enable win tests * use clang-16 on mac builds, add blockchain-explorer for ubuntu builds, add portable macos build * move sudo part outside a build scripts * move sudo part outside a build scripts * run llvm install with sudo * remove libgnutls28-dev before ubuntu static compilation, include blockchain-explorer into artifacts; remove warning: definition of implicit copy constructor for 'Stat' is deprecated because it has a user-declared copy assignment operator [-Wdeprecated-copy] * rework wrong decision, put back system gnutls, but compile libmicrohttpd with --disable-https * add jenkins pipeline sceleton * WIP jenkins pipeline sceleton * WIP jenkins pipeline changes * WIP jenkins pipeline: add stage timout, zip and group artifacts * WIP jenkins pipeline: macos portable build fix * WIP jenkins pipeline: wording * WIP jenkins pipeline: add android tonlib * WIP jenkins pipeline: add was binaries * WIP jenkins pipeline: add TOTAL_MEMORY 1.5gb to funcfiftlib wasm linking * WIP jenkins pipeline: add nix build on linux aarch64 * WIP jenkins pipeline: funcfiftlib compilation fails that 16mb mem is not enough, increase to 32mb * WIP jenkins pipeline: enable test in nix build * WIP jenkins pipeline: add linux x86-64 nix build * WIP jenkins pipeline: include libs in nix build * WIP jenkins pipeline: include libs in nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: nix linux arm64 with openssl 1.1 for now * WIP jenkins pipeline: working ubuntu arm64 libtonjson * WIP jenkins pipeline: working ubuntu arm64 libtonjson + minor fix * WIP jenkins pipeline: working ubuntu arm64 libtonjson + minor fix 2 * WIP jenkins pipeline: merry christmas * WIP jenkins pipeline: merry christmas 2 * WIP jenkins pipeline: remove native static builds * WIP jenkins pipeline: enable more tests * WIP jenkins pipeline: zip artifacts better * WIP jenkins pipeline: get rid of path in the final zip * WIP jenkins pipeline: minor fix, include lib and smartcont folders * WIP jenkins pipeline: minor fix, include lib and smartcont folders into nix artifacts also * WIP jenkins pipeline: minor fix * WIP jenkins pipeline: minor fix * adjust github actions for new nix builds * cleanup * cleanup * cleanup * cleanup * rename libtonlibjson.so.0.5 to libtonlibjson.so * Add TON build instructions to README.md * simplify * fix test-tonlib-offline * set timeout per test of 300 sec * set timeout per test of 600 sec for non nix builds * increase test timeout to 900 sec; minor changes * use MS VS 2022 for win TON compilation; update README.md * use MS VS 2022 for win TON compilation; update README.md * change path to MSVC in github workflow * change path to MSVC in groovy pipeline * compile ton on win, with msvc 2022 community and enterprise versions * minor fixes * improve network tests * remove TON compilation against macos-11 github runner * add `choco feature enable -n allowEmptyChecksums` since pkg-config-lite-0.28-1 does not have a checksum * abort win compilation if 3pp can't be downloaded * increase test timeout to 30 min * improving test-catchain
2024-01-15 20:48:04 +00:00
#/bin/bash
#sudo apt-get update
#sudo apt-get install -y build-essential git cmake ninja-build automake libtool texinfo autoconf
with_tests=false
with_artifacts=false
while getopts 'ta' flag; do
case "${flag}" in
t) with_tests=true ;;
a) with_artifacts=true ;;
*) break
;;
esac
done
if [ ! -d "build" ]; then
mkdir build
cd build
else
cd build
rm -rf .ninja* CMakeCache.txt
fi
export CC=$(which clang-16)
export CXX=$(which clang++-16)
export CCACHE_DISABLE=1
if [ ! -d "lz4" ]; then
git clone https://github.com/lz4/lz4.git
cd lz4
lz4Path=`pwd`
git checkout v1.9.4
make -j12
test $? -eq 0 || { echo "Can't compile lz4"; exit 1; }
cd ..
# ./lib/liblz4.a
# ./lib
else
lz4Path=$(pwd)/lz4
echo "Using compiled lz4"
fi
Improve TON build scripts and some tests (#855) * fix macOS github actions * fix android tonlib GH action; * fixing wasm GH action * strip binaries * fix randomly failing ubuntu and wasm GH actions * fix randomly failing ubuntu and wasm GH actions * revert some changes * adding more nix scripts and automated native build scripts; debug static ton compilation * minor fix * do not use pkg_config if path specified * move wasm script, run with sudo action script * weird, fixing - cp: missing destination file operand after 'assembly/native/build-ubuntu-20.04-shared.sh' * weird, fixing - cp: missing destination file operand after 'assembly/native/build-ubuntu-20.04-shared.sh' * ok * some adjustments for android and win builds * some adjustments for android and win builds * moving stripping inside the build script * access rights handling; adding simple binaries' tests * make lite-client-common, fift-lib and smc-envelope deliberately static; add -a (artifacts) flag to build scripts; * minor wasm build fix; create separate tonlib android build script; remove outdated __has_trivial_copy(T) * add windows build - WIP * adjust android build; improve win build; * adjust sodium paths for android build; use proper compiler for windows build; * add github windows build auxiliary file * adjust wasm build * add portable ubuntu build * exclude some unstable tests for some time * compile portable binaries on ubuntu-20.04 * exclude some unstable tests * include static gsl * restart builds * restart builds * restart builds * remove libreadline, gsl and blas dependencies in linux build * add macos build script * install missing autoconf in macos builds * enable all tests and see what fails * enable win tests and restart others * enable win tests and fix test-smartcont.cpp * enable win tests * use clang-16 on mac builds, add blockchain-explorer for ubuntu builds, add portable macos build * move sudo part outside a build scripts * move sudo part outside a build scripts * run llvm install with sudo * remove libgnutls28-dev before ubuntu static compilation, include blockchain-explorer into artifacts; remove warning: definition of implicit copy constructor for 'Stat' is deprecated because it has a user-declared copy assignment operator [-Wdeprecated-copy] * rework wrong decision, put back system gnutls, but compile libmicrohttpd with --disable-https * add jenkins pipeline sceleton * WIP jenkins pipeline sceleton * WIP jenkins pipeline changes * WIP jenkins pipeline: add stage timout, zip and group artifacts * WIP jenkins pipeline: macos portable build fix * WIP jenkins pipeline: wording * WIP jenkins pipeline: add android tonlib * WIP jenkins pipeline: add was binaries * WIP jenkins pipeline: add TOTAL_MEMORY 1.5gb to funcfiftlib wasm linking * WIP jenkins pipeline: add nix build on linux aarch64 * WIP jenkins pipeline: funcfiftlib compilation fails that 16mb mem is not enough, increase to 32mb * WIP jenkins pipeline: enable test in nix build * WIP jenkins pipeline: add linux x86-64 nix build * WIP jenkins pipeline: include libs in nix build * WIP jenkins pipeline: include libs in nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: nix linux arm64 with openssl 1.1 for now * WIP jenkins pipeline: working ubuntu arm64 libtonjson * WIP jenkins pipeline: working ubuntu arm64 libtonjson + minor fix * WIP jenkins pipeline: working ubuntu arm64 libtonjson + minor fix 2 * WIP jenkins pipeline: merry christmas * WIP jenkins pipeline: merry christmas 2 * WIP jenkins pipeline: remove native static builds * WIP jenkins pipeline: enable more tests * WIP jenkins pipeline: zip artifacts better * WIP jenkins pipeline: get rid of path in the final zip * WIP jenkins pipeline: minor fix, include lib and smartcont folders * WIP jenkins pipeline: minor fix, include lib and smartcont folders into nix artifacts also * WIP jenkins pipeline: minor fix * WIP jenkins pipeline: minor fix * adjust github actions for new nix builds * cleanup * cleanup * cleanup * cleanup * rename libtonlibjson.so.0.5 to libtonlibjson.so * Add TON build instructions to README.md * simplify * fix test-tonlib-offline * set timeout per test of 300 sec * set timeout per test of 600 sec for non nix builds * increase test timeout to 900 sec; minor changes * use MS VS 2022 for win TON compilation; update README.md * use MS VS 2022 for win TON compilation; update README.md * change path to MSVC in github workflow * change path to MSVC in groovy pipeline * compile ton on win, with msvc 2022 community and enterprise versions * minor fixes * improve network tests * remove TON compilation against macos-11 github runner * add `choco feature enable -n allowEmptyChecksums` since pkg-config-lite-0.28-1 does not have a checksum * abort win compilation if 3pp can't be downloaded * increase test timeout to 30 min * improving test-catchain
2024-01-15 20:48:04 +00:00
if [ ! -d "secp256k1" ]; then
git clone https://github.com/bitcoin-core/secp256k1.git
cd secp256k1
secp256k1Path=`pwd`
git checkout v0.3.2
./autogen.sh
./configure --enable-module-recovery --enable-static --disable-tests --disable-benchmark --with-pic
make -j12
test $? -eq 0 || { echo "Can't compile secp256k1"; exit 1; }
cd ..
# ./.libs/libsecp256k1.a
# ./include
else
secp256k1Path=$(pwd)/secp256k1
echo "Using compiled secp256k1"
fi
if [ ! -d "libsodium" ]; then
export LIBSODIUM_FULL_BUILD=1
git clone https://github.com/jedisct1/libsodium.git
cd libsodium
sodiumPath=`pwd`
git checkout 1.0.18
./autogen.sh
./configure --with-pic --enable-static
make -j12
test $? -eq 0 || { echo "Can't compile libsodium"; exit 1; }
cd ..
else
sodiumPath=$(pwd)/libsodium
echo "Using compiled libsodium"
fi
if [ ! -d "openssl_3" ]; then
git clone https://github.com/openssl/openssl openssl_3
cd openssl_3
opensslPath=`pwd`
git checkout openssl-3.1.4
./config -static
make build_libs -j12
test $? -eq 0 || { echo "Can't compile openssl_3"; exit 1; }
cd ..
else
opensslPath=$(pwd)/openssl_3
echo "Using compiled openssl_3"
fi
if [ ! -d "zlib" ]; then
git clone https://github.com/madler/zlib.git
cd zlib
zlibPath=`pwd`
./configure --static
make -j12
test $? -eq 0 || { echo "Can't compile zlib"; exit 1; }
cd ..
else
zlibPath=$(pwd)/zlib
echo "Using compiled zlib"
fi
if [ ! -d "libmicrohttpd" ]; then
git clone https://git.gnunet.org/libmicrohttpd.git
cd libmicrohttpd
libmicrohttpdPath=`pwd`
./autogen.sh
./configure --enable-static --disable-tests --disable-benchmark --disable-shared --disable-https --with-pic
make -j12
test $? -eq 0 || { echo "Can't compile libmicrohttpd"; exit 1; }
cd ..
else
libmicrohttpdPath=$(pwd)/libmicrohttpd
echo "Using compiled libmicrohttpd"
fi
cmake -GNinja .. \
-DPORTABLE=1 \
-DCMAKE_BUILD_TYPE=Release \
-DOPENSSL_FOUND=1 \
-DOPENSSL_INCLUDE_DIR=$opensslPath/include \
-DOPENSSL_CRYPTO_LIBRARY=$opensslPath/libcrypto.a \
-DZLIB_FOUND=1 \
-DZLIB_INCLUDE_DIR=$zlibPath \
-DZLIB_LIBRARIES=$zlibPath/libz.a \
-DSECP256K1_FOUND=1 \
-DSECP256K1_INCLUDE_DIR=$secp256k1Path/include \
-DSECP256K1_LIBRARY=$secp256k1Path/.libs/libsecp256k1.a \
-DSODIUM_FOUND=1 \
-DSODIUM_INCLUDE_DIR=$sodiumPath/src/libsodium/include \
-DSODIUM_LIBRARY_RELEASE=$sodiumPath/src/libsodium/.libs/libsodium.a \
-DMHD_FOUND=1 \
-DMHD_INCLUDE_DIR=$libmicrohttpdPath/src/include \
-DMHD_LIBRARY=$libmicrohttpdPath/src/microhttpd/.libs/libmicrohttpd.a \
-DLZ4_FOUND=1 \
-DLZ4_INCLUDE_DIRS=$lz4Path/lib \
-DLZ4_LIBRARIES=$lz4Path/lib/liblz4.a
Improve TON build scripts and some tests (#855) * fix macOS github actions * fix android tonlib GH action; * fixing wasm GH action * strip binaries * fix randomly failing ubuntu and wasm GH actions * fix randomly failing ubuntu and wasm GH actions * revert some changes * adding more nix scripts and automated native build scripts; debug static ton compilation * minor fix * do not use pkg_config if path specified * move wasm script, run with sudo action script * weird, fixing - cp: missing destination file operand after 'assembly/native/build-ubuntu-20.04-shared.sh' * weird, fixing - cp: missing destination file operand after 'assembly/native/build-ubuntu-20.04-shared.sh' * ok * some adjustments for android and win builds * some adjustments for android and win builds * moving stripping inside the build script * access rights handling; adding simple binaries' tests * make lite-client-common, fift-lib and smc-envelope deliberately static; add -a (artifacts) flag to build scripts; * minor wasm build fix; create separate tonlib android build script; remove outdated __has_trivial_copy(T) * add windows build - WIP * adjust android build; improve win build; * adjust sodium paths for android build; use proper compiler for windows build; * add github windows build auxiliary file * adjust wasm build * add portable ubuntu build * exclude some unstable tests for some time * compile portable binaries on ubuntu-20.04 * exclude some unstable tests * include static gsl * restart builds * restart builds * restart builds * remove libreadline, gsl and blas dependencies in linux build * add macos build script * install missing autoconf in macos builds * enable all tests and see what fails * enable win tests and restart others * enable win tests and fix test-smartcont.cpp * enable win tests * use clang-16 on mac builds, add blockchain-explorer for ubuntu builds, add portable macos build * move sudo part outside a build scripts * move sudo part outside a build scripts * run llvm install with sudo * remove libgnutls28-dev before ubuntu static compilation, include blockchain-explorer into artifacts; remove warning: definition of implicit copy constructor for 'Stat' is deprecated because it has a user-declared copy assignment operator [-Wdeprecated-copy] * rework wrong decision, put back system gnutls, but compile libmicrohttpd with --disable-https * add jenkins pipeline sceleton * WIP jenkins pipeline sceleton * WIP jenkins pipeline changes * WIP jenkins pipeline: add stage timout, zip and group artifacts * WIP jenkins pipeline: macos portable build fix * WIP jenkins pipeline: wording * WIP jenkins pipeline: add android tonlib * WIP jenkins pipeline: add was binaries * WIP jenkins pipeline: add TOTAL_MEMORY 1.5gb to funcfiftlib wasm linking * WIP jenkins pipeline: add nix build on linux aarch64 * WIP jenkins pipeline: funcfiftlib compilation fails that 16mb mem is not enough, increase to 32mb * WIP jenkins pipeline: enable test in nix build * WIP jenkins pipeline: add linux x86-64 nix build * WIP jenkins pipeline: include libs in nix build * WIP jenkins pipeline: include libs in nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: include mac nix build * WIP jenkins pipeline: nix linux arm64 with openssl 1.1 for now * WIP jenkins pipeline: working ubuntu arm64 libtonjson * WIP jenkins pipeline: working ubuntu arm64 libtonjson + minor fix * WIP jenkins pipeline: working ubuntu arm64 libtonjson + minor fix 2 * WIP jenkins pipeline: merry christmas * WIP jenkins pipeline: merry christmas 2 * WIP jenkins pipeline: remove native static builds * WIP jenkins pipeline: enable more tests * WIP jenkins pipeline: zip artifacts better * WIP jenkins pipeline: get rid of path in the final zip * WIP jenkins pipeline: minor fix, include lib and smartcont folders * WIP jenkins pipeline: minor fix, include lib and smartcont folders into nix artifacts also * WIP jenkins pipeline: minor fix * WIP jenkins pipeline: minor fix * adjust github actions for new nix builds * cleanup * cleanup * cleanup * cleanup * rename libtonlibjson.so.0.5 to libtonlibjson.so * Add TON build instructions to README.md * simplify * fix test-tonlib-offline * set timeout per test of 300 sec * set timeout per test of 600 sec for non nix builds * increase test timeout to 900 sec; minor changes * use MS VS 2022 for win TON compilation; update README.md * use MS VS 2022 for win TON compilation; update README.md * change path to MSVC in github workflow * change path to MSVC in groovy pipeline * compile ton on win, with msvc 2022 community and enterprise versions * minor fixes * improve network tests * remove TON compilation against macos-11 github runner * add `choco feature enable -n allowEmptyChecksums` since pkg-config-lite-0.28-1 does not have a checksum * abort win compilation if 3pp can't be downloaded * increase test timeout to 30 min * improving test-catchain
2024-01-15 20:48:04 +00:00
test $? -eq 0 || { echo "Can't configure ton"; exit 1; }
if [ "$with_tests" = true ]; then
ninja storage-daemon storage-daemon-cli fift func tonlib tonlibjson tonlib-cli \
validator-engine lite-client pow-miner validator-engine-console blockchain-explorer \
generate-random-id json2tlo dht-server http-proxy rldp-http-proxy \
adnl-proxy create-state emulator test-ed25519 test-ed25519-crypto test-bigint \
test-vm test-fift test-cells test-smartcont test-net test-tdactor test-tdutils \
test-tonlib-offline test-adnl test-dht test-rldp test-rldp2 test-catchain \
test-fec test-tddb test-db test-validator-session-state
test $? -eq 0 || { echo "Can't compile ton"; exit 1; }
else
ninja storage-daemon storage-daemon-cli fift func tonlib tonlibjson tonlib-cli \
validator-engine lite-client pow-miner validator-engine-console blockchain-explorer \
generate-random-id json2tlo dht-server http-proxy rldp-http-proxy \
adnl-proxy create-state emulator
test $? -eq 0 || { echo "Can't compile ton"; exit 1; }
fi
strip -g storage/storage-daemon/storage-daemon \
storage/storage-daemon/storage-daemon-cli \
blockchain-explorer/blockchain-explorer \
crypto/fift \
crypto/tlbc \
crypto/func \
crypto/create-state \
validator-engine-console/validator-engine-console \
tonlib/tonlib-cli \
tonlib/libtonlibjson.so.0.5 \
http/http-proxy \
rldp-http-proxy/rldp-http-proxy \
dht-server/dht-server \
lite-client/lite-client \
validator-engine/validator-engine \
utils/generate-random-id \
utils/json2tlo \
adnl/adnl-proxy \
emulator/libemulator.*
test $? -eq 0 || { echo "Can't strip final binaries"; exit 1; }
# simple binaries' test
./storage/storage-daemon/storage-daemon -V || exit 1
./validator-engine/validator-engine -V || exit 1
./lite-client/lite-client -V || exit 1
./crypto/fift -V || exit 1
cd ..
if [ "$with_artifacts" = true ]; then
rm -rf artifacts
mkdir artifacts
cp crypto/fift/lib artifacts/
cp -R crypto/smartcont/ artifacts/
mv build/tonlib/libtonlibjson.so.0.5 build/tonlib/libtonlibjson.so
cp build/storage/storage-daemon/storage-daemon build/storage/storage-daemon/storage-daemon-cli \
build/crypto/fift build/crypto/tlbc build/crypto/func build/crypto/create-state build/blockchain-explorer/blockchain-explorer \
build/validator-engine-console/validator-engine-console build/tonlib/tonlib-cli \
build/tonlib/libtonlibjson.so build/http/http-proxy build/rldp-http-proxy/rldp-http-proxy \
build/dht-server/dht-server build/lite-client/lite-client build/validator-engine/validator-engine \
build/utils/generate-random-id build/utils/json2tlo build/adnl/adnl-proxy build/emulator/libemulator.so \
artifacts
test $? -eq 0 || { echo "Can't copy final binaries"; exit 1; }
chmod +x artifacts/*
cp -R crypto/smartcont artifacts
cp -R crypto/fift/lib artifacts
fi
if [ "$with_tests" = true ]; then
cd build
# ctest --output-on-failure -E "test-catchain|test-actors|test-smartcont|test-adnl|test-validator-session-state|test-dht|test-rldp"
ctest --output-on-failure -E "test-adnl"
fi