external/ton
1
0
Fork 0
mirror of https://github.com/ton-blockchain/ton synced 2025-02-12 11:12:16 +00:00
Code Issues Releases Wiki Activity
ton/tolk/builtins.cpp

1282 lines
45 KiB
C++
Raw Normal View History

[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
/*
This file is part of TON Blockchain Library.
TON Blockchain Library is free software: you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as published by
the Free Software Foundation, either version 2 of the License, or
(at your option) any later version.
TON Blockchain Library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License
along with TON Blockchain Library. If not, see <http://www.gnu.org/licenses/>.
*/
#include "tolk.h"
[Tolk] Embedded stdlib.tolk, CompilerState, strict includes Several related changes: - stdlib.tolk is embedded into a distribution (deb package or tolk-js), the user won't have to download it and store as a project file; it's an important step to maintain correct language versioning - stdlib.tolk is auto-included, that's why all its functions are available out of the box - strict includes: you can't use symbol `f` from another file unless you've #include'd this file - drop all C++ global variables holding compilation state, merge them into a single struct CompilerState located at compiler-state.h; for instance, stdlib filename is also there
2024-10-31 07:02:01 +00:00
#include "compiler-state.h"
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
#include "type-system.h"
#include "generics-helpers.h"
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
namespace tolk {
using namespace std::literals::string_literals;
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
// given func_type = `(slice, int) -> slice` and func flags, create SymLocalVarOrParameter
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
// currently (see at the bottom) parameters of built-in functions are unnamed:
// built-in functions are created using a resulting type
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
static std::vector<LocalVarData> define_builtin_parameters(const std::vector<TypePtr>& params_types, int func_flags) {
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
// `loadInt()`, `storeInt()`: they accept `self` and mutate it; no other options available in built-ins for now
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
bool is_mutate_self = func_flags & FunctionData::flagHasMutateParams;
std::vector<LocalVarData> parameters;
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
parameters.reserve(params_types.size());
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
for (int i = 0; i < static_cast<int>(params_types.size()); ++i) {
LocalVarData p_sym("", {}, params_types[i], (i == 0 && is_mutate_self) * LocalVarData::flagMutateParameter, i);
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
parameters.push_back(std::move(p_sym));
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
}
[Tolk] Tolk v0.5.0 as FunC v0.5.0 could have been like All changes from PR "FunC v0.5.0": https://github.com/ton-blockchain/ton/pull/1026 Instead of developing FunC, we decided to fork it. BTW, the first Tolk release will be v0.6, a metaphor of FunC v0.5 that missed a chance to occur.
2024-10-31 06:54:05 +00:00
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
return parameters;
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
}
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
static void define_builtin_func(const std::string& name, const std::vector<TypePtr>& params_types, TypePtr return_type, const GenericsDeclaration* genericTs, const simple_compile_func_t& func, int flags) {
auto* f_sym = new FunctionData(name, {}, return_type, define_builtin_parameters(params_types, flags), flags, genericTs, nullptr, new FunctionBodyBuiltin(func), nullptr);
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
G.symtable.add_function(f_sym);
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
}
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
static void define_builtin_func(const std::string& name, const std::vector<TypePtr>& params_types, TypePtr return_type, const GenericsDeclaration* genericTs, const AsmOp& macro, int flags) {
auto* f_sym = new FunctionData(name, {}, return_type, define_builtin_parameters(params_types, flags), flags, genericTs, nullptr, new FunctionBodyBuiltin(make_simple_compile(macro)), nullptr);
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
G.symtable.add_function(f_sym);
[Tolk] Tolk v0.5.0 as FunC v0.5.0 could have been like All changes from PR "FunC v0.5.0": https://github.com/ton-blockchain/ton/pull/1026 Instead of developing FunC, we decided to fork it. BTW, the first Tolk release will be v0.6, a metaphor of FunC v0.5 that missed a chance to occur.
2024-10-31 06:54:05 +00:00
}
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
static void define_builtin_func(const std::string& name, const std::vector<TypePtr>& params_types, TypePtr return_type, const GenericsDeclaration* genericTs, const simple_compile_func_t& func, int flags,
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
std::initializer_list<int> arg_order, std::initializer_list<int> ret_order) {
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
auto* f_sym = new FunctionData(name, {}, return_type, define_builtin_parameters(params_types, flags), flags, genericTs, nullptr, new FunctionBodyBuiltin(func), nullptr);
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
f_sym->arg_order = arg_order;
f_sym->ret_order = ret_order;
G.symtable.add_function(f_sym);
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
}
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
void FunctionBodyBuiltin::compile(AsmOpList& dest, std::vector<VarDescr>& out, std::vector<VarDescr>& in,
SrcLocation where) const {
dest.append(simple_compile(out, in, where));
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
}
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
void FunctionBodyAsm::compile(AsmOpList& dest) const {
dest.append(ops);
}
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
/*
*
* DEFINE BUILT-IN FUNCTIONS
*
*/
int emulate_negate(int a) {
int f = VarDescr::_Pos | VarDescr::_Neg;
if ((a & f) && (~a & f)) {
a ^= f;
}
return a;
}
int emulate_add(int a, int b) {
if (b & VarDescr::_Zero) {
return a;
} else if (a & VarDescr::_Zero) {
return b;
}
int u = a & b, v = a | b;
int r = VarDescr::_Int;
int t = u & (VarDescr::_Pos | VarDescr::_Neg);
if (v & VarDescr::_Nan) {
return r | VarDescr::_Nan;
}
// non-quiet addition always returns finite results!
r |= t | VarDescr::_Finite;
if (t) {
r |= v & VarDescr::_NonZero;
}
r |= v & VarDescr::_Nan;
if (u & (VarDescr::_Odd | VarDescr::_Even)) {
r |= VarDescr::_Even;
} else if (!(~v & (VarDescr::_Odd | VarDescr::_Even))) {
r |= VarDescr::_Odd | VarDescr::_NonZero;
}
return r;
}
int emulate_sub(int a, int b) {
return emulate_add(a, emulate_negate(b));
}
int emulate_mul(int a, int b) {
[Tolk] v0.6 syntax: `fun`, `import`, `var`, types on the right, etc. Lots of changes, actually. Most noticeable are: - traditional //comments - #include -> import - a rule "import what you use" - ~ found -> !found (for -1/0) - null() -> null - is_null?(v) -> v == null - throw is a keyword - catch with swapped arguments - throw_if, throw_unless -> assert - do until -> do while - elseif -> else if - drop ifnot, elseifnot - drop rarely used operators A testing framework also appears here. All tests existed earlier, but due to significant syntax changes, their history is useless.
2024-10-31 07:11:41 +00:00
if ((b & VarDescr::ConstOne) == VarDescr::ConstOne) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
return a;
[Tolk] v0.6 syntax: `fun`, `import`, `var`, types on the right, etc. Lots of changes, actually. Most noticeable are: - traditional //comments - #include -> import - a rule "import what you use" - ~ found -> !found (for -1/0) - null() -> null - is_null?(v) -> v == null - throw is a keyword - catch with swapped arguments - throw_if, throw_unless -> assert - do until -> do while - elseif -> else if - drop ifnot, elseifnot - drop rarely used operators A testing framework also appears here. All tests existed earlier, but due to significant syntax changes, their history is useless.
2024-10-31 07:11:41 +00:00
} else if ((a & VarDescr::ConstOne) == VarDescr::ConstOne) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
return b;
}
int u = a & b, v = a | b;
int r = VarDescr::_Int;
if (v & VarDescr::_Nan) {
return r | VarDescr::_Nan;
}
// non-quiet multiplication always yields finite results, if any
r |= VarDescr::_Finite;
if (v & VarDescr::_Zero) {
// non-quiet multiplication
// the result is zero, if any result at all
return VarDescr::ConstZero;
}
if (u & (VarDescr::_Pos | VarDescr::_Neg)) {
r |= VarDescr::_Pos;
} else if (!(~v & (VarDescr::_Pos | VarDescr::_Neg))) {
r |= VarDescr::_Neg;
}
r |= v & VarDescr::_Even;
r |= u & (VarDescr::_Odd | VarDescr::_NonZero);
return r;
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
int emulate_bitwise_and(int a, int b) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
int both = a & b, any = a | b;
int r = VarDescr::_Int;
if (any & VarDescr::_Nan) {
return r | VarDescr::_Nan;
}
r |= VarDescr::_Finite;
if (any & VarDescr::_Zero) {
return VarDescr::ConstZero;
}
r |= both & (VarDescr::_Even | VarDescr::_Odd);
if (both & VarDescr::_Odd) {
r |= VarDescr::_NonZero;
}
return r;
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
int emulate_bitwise_or(int a, int b) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
if (b & VarDescr::_Zero) {
return a;
} else if (a & VarDescr::_Zero) {
return b;
}
int both = a & b, any = a | b;
int r = VarDescr::_Int;
if (any & VarDescr::_Nan) {
return r | VarDescr::_Nan;
}
r |= VarDescr::_Finite;
r |= any & VarDescr::_NonZero;
r |= any & VarDescr::_Odd;
r |= both & VarDescr::_Even;
return r;
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
int emulate_bitwise_xor(int a, int b) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
if (b & VarDescr::_Zero) {
return a;
} else if (a & VarDescr::_Zero) {
return b;
}
int both = a & b, any = a | b;
int r = VarDescr::_Int;
if (any & VarDescr::_Nan) {
return r | VarDescr::_Nan;
}
r |= VarDescr::_Finite;
r |= both & VarDescr::_Even;
if (both & VarDescr::_Odd) {
r |= VarDescr::_Even;
}
return r;
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
int emulate_bitwise_not(int a) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
if ((a & VarDescr::ConstZero) == VarDescr::ConstZero) {
return VarDescr::ConstTrue;
}
if ((a & VarDescr::ConstTrue) == VarDescr::ConstTrue) {
return VarDescr::ConstZero;
}
int a2 = a;
int f = VarDescr::_Even | VarDescr::_Odd;
if ((a2 & f) && (~a2 & f)) {
a2 ^= f;
}
[Tolk] v0.6 syntax: `fun`, `import`, `var`, types on the right, etc. Lots of changes, actually. Most noticeable are: - traditional //comments - #include -> import - a rule "import what you use" - ~ found -> !found (for -1/0) - null() -> null - is_null?(v) -> v == null - throw is a keyword - catch with swapped arguments - throw_if, throw_unless -> assert - do until -> do while - elseif -> else if - drop ifnot, elseifnot - drop rarely used operators A testing framework also appears here. All tests existed earlier, but due to significant syntax changes, their history is useless.
2024-10-31 07:11:41 +00:00
a2 &= ~(VarDescr::_Zero | VarDescr::_NonZero | VarDescr::_Pos | VarDescr::_Neg);
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
if ((a & VarDescr::_Neg) && (a & VarDescr::_NonZero)) {
a2 |= VarDescr::_Pos;
}
if (a & VarDescr::_Pos) {
a2 |= VarDescr::_Neg;
}
return a2;
}
int emulate_lshift(int a, int b) {
if (((a | b) & VarDescr::_Nan) || !(~b & (VarDescr::_Neg | VarDescr::_NonZero))) {
return VarDescr::_Int | VarDescr::_Nan;
}
if (b & VarDescr::_Zero) {
return a;
}
int t = ((b & VarDescr::_NonZero) ? VarDescr::_Even : 0);
t |= b & VarDescr::_Finite;
[Tolk] Tolk v0.5.0 as FunC v0.5.0 could have been like All changes from PR "FunC v0.5.0": https://github.com/ton-blockchain/ton/pull/1026 Instead of developing FunC, we decided to fork it. BTW, the first Tolk release will be v0.6, a metaphor of FunC v0.5 that missed a chance to occur.
2024-10-31 06:54:05 +00:00
return emulate_mul(a, VarDescr::_Int | VarDescr::_Pos | VarDescr::_NonZero | t);
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
}
int emulate_div(int a, int b) {
[Tolk] v0.6 syntax: `fun`, `import`, `var`, types on the right, etc. Lots of changes, actually. Most noticeable are: - traditional //comments - #include -> import - a rule "import what you use" - ~ found -> !found (for -1/0) - null() -> null - is_null?(v) -> v == null - throw is a keyword - catch with swapped arguments - throw_if, throw_unless -> assert - do until -> do while - elseif -> else if - drop ifnot, elseifnot - drop rarely used operators A testing framework also appears here. All tests existed earlier, but due to significant syntax changes, their history is useless.
2024-10-31 07:11:41 +00:00
if ((b & VarDescr::ConstOne) == VarDescr::ConstOne) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
return a;
[Tolk] v0.6 syntax: `fun`, `import`, `var`, types on the right, etc. Lots of changes, actually. Most noticeable are: - traditional //comments - #include -> import - a rule "import what you use" - ~ found -> !found (for -1/0) - null() -> null - is_null?(v) -> v == null - throw is a keyword - catch with swapped arguments - throw_if, throw_unless -> assert - do until -> do while - elseif -> else if - drop ifnot, elseifnot - drop rarely used operators A testing framework also appears here. All tests existed earlier, but due to significant syntax changes, their history is useless.
2024-10-31 07:11:41 +00:00
} else if ((b & VarDescr::ConstOne) == VarDescr::ConstOne) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
return emulate_negate(a);
}
if (b & VarDescr::_Zero) {
return VarDescr::_Int | VarDescr::_Nan;
}
int u = a & b, v = a | b;
int r = VarDescr::_Int;
if (v & VarDescr::_Nan) {
return r | VarDescr::_Nan;
}
// non-quiet division always yields finite results, if any
r |= VarDescr::_Finite;
if (a & VarDescr::_Zero) {
// non-quiet division
// the result is zero, if any result at all
return VarDescr::ConstZero;
}
if (u & (VarDescr::_Pos | VarDescr::_Neg)) {
r |= VarDescr::_Pos;
} else if (!(~v & (VarDescr::_Pos | VarDescr::_Neg))) {
r |= VarDescr::_Neg;
}
return r;
}
int emulate_rshift(int a, int b) {
if (((a | b) & VarDescr::_Nan) || !(~b & (VarDescr::_Neg | VarDescr::_NonZero))) {
return VarDescr::_Int | VarDescr::_Nan;
}
if (b & VarDescr::_Zero) {
return a;
}
int t = ((b & VarDescr::_NonZero) ? VarDescr::_Even : 0);
t |= b & VarDescr::_Finite;
[Tolk] Tolk v0.5.0 as FunC v0.5.0 could have been like All changes from PR "FunC v0.5.0": https://github.com/ton-blockchain/ton/pull/1026 Instead of developing FunC, we decided to fork it. BTW, the first Tolk release will be v0.6, a metaphor of FunC v0.5 that missed a chance to occur.
2024-10-31 06:54:05 +00:00
return emulate_div(a, VarDescr::_Int | VarDescr::_Pos | VarDescr::_NonZero | t);
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
}
int emulate_mod(int a, int b, int round_mode = -1) {
[Tolk] v0.6 syntax: `fun`, `import`, `var`, types on the right, etc. Lots of changes, actually. Most noticeable are: - traditional //comments - #include -> import - a rule "import what you use" - ~ found -> !found (for -1/0) - null() -> null - is_null?(v) -> v == null - throw is a keyword - catch with swapped arguments - throw_if, throw_unless -> assert - do until -> do while - elseif -> else if - drop ifnot, elseifnot - drop rarely used operators A testing framework also appears here. All tests existed earlier, but due to significant syntax changes, their history is useless.
2024-10-31 07:11:41 +00:00
if ((b & VarDescr::ConstOne) == VarDescr::ConstOne) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
return VarDescr::ConstZero;
}
if (b & VarDescr::_Zero) {
return VarDescr::_Int | VarDescr::_Nan;
}
int r = VarDescr::_Int;
if ((a | b) & VarDescr::_Nan) {
return r | VarDescr::_Nan;
}
// non-quiet division always yields finite results, if any
r |= VarDescr::_Finite;
if (a & VarDescr::_Zero) {
// non-quiet division
// the result is zero, if any result at all
return VarDescr::ConstZero;
}
if (round_mode < 0) {
r |= b & (VarDescr::_Pos | VarDescr::_Neg);
} else if (round_mode > 0) {
r |= emulate_negate(b) & (VarDescr::_Pos | VarDescr::_Neg);
}
if (b & VarDescr::_Even) {
r |= a & (VarDescr::_Even | VarDescr::_Odd);
}
return r;
}
bool VarDescr::always_less(const VarDescr& other) const {
if (is_int_const() && other.is_int_const()) {
return int_const < other.int_const;
}
return (always_nonpos() && other.always_pos()) || (always_neg() && other.always_nonneg());
}
bool VarDescr::always_leq(const VarDescr& other) const {
if (is_int_const() && other.is_int_const()) {
return int_const <= other.int_const;
}
return always_nonpos() && other.always_nonneg();
}
bool VarDescr::always_greater(const VarDescr& other) const {
return other.always_less(*this);
}
bool VarDescr::always_geq(const VarDescr& other) const {
return other.always_leq(*this);
}
bool VarDescr::always_equal(const VarDescr& other) const {
return is_int_const() && other.is_int_const() && *int_const == *other.int_const;
}
bool VarDescr::always_neq(const VarDescr& other) const {
if (is_int_const() && other.is_int_const()) {
return *int_const != *other.int_const;
}
return always_greater(other) || always_less(other) || (always_even() && other.always_odd()) ||
(always_odd() && other.always_even());
}
AsmOp exec_op(std::string op) {
return AsmOp::Custom(op);
}
AsmOp exec_op(std::string op, int args, int retv = 1) {
return AsmOp::Custom(op, args, retv);
}
AsmOp exec_arg_op(std::string op, long long arg) {
std::ostringstream os;
os << arg << ' ' << op;
return AsmOp::Custom(os.str());
}
AsmOp exec_arg_op(std::string op, long long arg, int args, int retv) {
std::ostringstream os;
os << arg << ' ' << op;
return AsmOp::Custom(os.str(), args, retv);
}
AsmOp exec_arg_op(std::string op, td::RefInt256 arg) {
std::ostringstream os;
os << arg << ' ' << op;
return AsmOp::Custom(os.str());
}
AsmOp exec_arg_op(std::string op, td::RefInt256 arg, int args, int retv) {
std::ostringstream os;
os << arg << ' ' << op;
return AsmOp::Custom(os.str(), args, retv);
}
AsmOp exec_arg2_op(std::string op, long long imm1, long long imm2, int args, int retv) {
std::ostringstream os;
os << imm1 << ' ' << imm2 << ' ' << op;
return AsmOp::Custom(os.str(), args, retv);
}
AsmOp push_const(td::RefInt256 x) {
return AsmOp::IntConst(std::move(x));
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_add(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation where) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
tolk_assert(res.size() == 1 && args.size() == 2);
VarDescr &r = res[0], &x = args[0], &y = args[1];
if (x.is_int_const() && y.is_int_const()) {
r.set_const(x.int_const + y.int_const);
if (!r.int_const->is_valid()) {
throw ParseError(where, "integer overflow");
}
x.unused();
y.unused();
return push_const(r.int_const);
}
r.val = emulate_add(x.val, y.val);
if (y.is_int_const() && y.int_const->signed_fits_bits(8)) {
y.unused();
if (y.always_zero()) {
return AsmOp::Nop();
}
if (*y.int_const == 1) {
return exec_op("INC", 1);
}
if (*y.int_const == -1) {
return exec_op("DEC", 1);
}
return exec_arg_op("ADDCONST", y.int_const, 1);
}
if (x.is_int_const() && x.int_const->signed_fits_bits(8)) {
x.unused();
if (x.always_zero()) {
return AsmOp::Nop();
}
if (*x.int_const == 1) {
return exec_op("INC", 1);
}
if (*x.int_const == -1) {
return exec_op("DEC", 1);
}
return exec_arg_op("ADDCONST", x.int_const, 1);
}
return exec_op("ADD", 2);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_sub(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation where) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
tolk_assert(res.size() == 1 && args.size() == 2);
VarDescr &r = res[0], &x = args[0], &y = args[1];
if (x.is_int_const() && y.is_int_const()) {
r.set_const(x.int_const - y.int_const);
if (!r.int_const->is_valid()) {
throw ParseError(where, "integer overflow");
}
x.unused();
y.unused();
return push_const(r.int_const);
}
r.val = emulate_sub(x.val, y.val);
if (y.is_int_const() && (-y.int_const)->signed_fits_bits(8)) {
y.unused();
if (y.always_zero()) {
return {};
}
if (*y.int_const == 1) {
return exec_op("DEC", 1);
}
if (*y.int_const == -1) {
return exec_op("INC", 1);
}
return exec_arg_op("ADDCONST", -y.int_const, 1);
}
if (x.always_zero()) {
x.unused();
return exec_op("NEGATE", 1);
}
return exec_op("SUB", 2);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_unary_minus(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation where) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
tolk_assert(res.size() == 1 && args.size() == 1);
VarDescr &r = res[0], &x = args[0];
if (x.is_int_const()) {
r.set_const(-x.int_const);
if (!r.int_const->is_valid()) {
throw ParseError(where, "integer overflow");
}
x.unused();
return push_const(r.int_const);
}
r.val = emulate_negate(x.val);
return exec_op("NEGATE", 1);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_unary_plus(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation where) {
tolk_assert(res.size() == 1 && args.size() == 1);
VarDescr &r = res[0], &x = args[0];
if (x.is_int_const()) {
r.set_const(x.int_const);
x.unused();
return push_const(r.int_const);
}
r.val = x.val;
return AsmOp::Nop();
}
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
AsmOp compile_logical_not(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation where, bool for_int_arg) {
[Tolk] v0.6 syntax: `fun`, `import`, `var`, types on the right, etc. Lots of changes, actually. Most noticeable are: - traditional //comments - #include -> import - a rule "import what you use" - ~ found -> !found (for -1/0) - null() -> null - is_null?(v) -> v == null - throw is a keyword - catch with swapped arguments - throw_if, throw_unless -> assert - do until -> do while - elseif -> else if - drop ifnot, elseifnot - drop rarely used operators A testing framework also appears here. All tests existed earlier, but due to significant syntax changes, their history is useless.
2024-10-31 07:11:41 +00:00
tolk_assert(res.size() == 1 && args.size() == 1);
VarDescr &r = res[0], &x = args[0];
if (x.is_int_const()) {
r.set_const(x.int_const == 0 ? -1 : 0);
x.unused();
return push_const(r.int_const);
}
r.val = VarDescr::ValBool;
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
// for integers, `!var` is `var != 0`
// for booleans, `!var` can be shortened to `~var` (works the same for 0/-1 but consumes less)
return for_int_arg ? exec_op("0 EQINT", 1) : exec_op("NOT", 1);
[Tolk] v0.6 syntax: `fun`, `import`, `var`, types on the right, etc. Lots of changes, actually. Most noticeable are: - traditional //comments - #include -> import - a rule "import what you use" - ~ found -> !found (for -1/0) - null() -> null - is_null?(v) -> v == null - throw is a keyword - catch with swapped arguments - throw_if, throw_unless -> assert - do until -> do while - elseif -> else if - drop ifnot, elseifnot - drop rarely used operators A testing framework also appears here. All tests existed earlier, but due to significant syntax changes, their history is useless.
2024-10-31 07:11:41 +00:00
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_bitwise_and(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation where) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
tolk_assert(res.size() == 1 && args.size() == 2);
VarDescr &r = res[0], &x = args[0], &y = args[1];
if (x.is_int_const() && y.is_int_const()) {
r.set_const(x.int_const & y.int_const);
x.unused();
y.unused();
return push_const(r.int_const);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
r.val = emulate_bitwise_and(x.val, y.val);
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
return exec_op("AND", 2);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_bitwise_or(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation where) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
tolk_assert(res.size() == 1 && args.size() == 2);
VarDescr &r = res[0], &x = args[0], &y = args[1];
if (x.is_int_const() && y.is_int_const()) {
r.set_const(x.int_const | y.int_const);
x.unused();
y.unused();
return push_const(r.int_const);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
r.val = emulate_bitwise_or(x.val, y.val);
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
return exec_op("OR", 2);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_bitwise_xor(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation where) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
tolk_assert(res.size() == 1 && args.size() == 2);
VarDescr &r = res[0], &x = args[0], &y = args[1];
if (x.is_int_const() && y.is_int_const()) {
r.set_const(x.int_const ^ y.int_const);
x.unused();
y.unused();
return push_const(r.int_const);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
r.val = emulate_bitwise_xor(x.val, y.val);
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
return exec_op("XOR", 2);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_bitwise_not(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation where) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
tolk_assert(res.size() == 1 && args.size() == 1);
VarDescr &r = res[0], &x = args[0];
if (x.is_int_const()) {
r.set_const(~x.int_const);
x.unused();
return push_const(r.int_const);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
r.val = emulate_bitwise_not(x.val);
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
return exec_op("NOT", 1);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_mul_internal(VarDescr& r, VarDescr& x, VarDescr& y, SrcLocation where) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
if (x.is_int_const() && y.is_int_const()) {
r.set_const(x.int_const * y.int_const);
if (!r.int_const->is_valid()) {
throw ParseError(where, "integer overflow");
}
x.unused();
y.unused();
return push_const(r.int_const);
}
r.val = emulate_mul(x.val, y.val);
if (y.is_int_const()) {
int k = is_pos_pow2(y.int_const);
if (y.int_const->signed_fits_bits(8) && k < 0) {
y.unused();
if (y.always_zero() && x.always_finite()) {
// dubious optimization: NaN * 0 = ?
r.set_const(y.int_const);
x.unused();
return push_const(r.int_const);
}
if (*y.int_const == 1 && x.always_finite()) {
return AsmOp::Nop();
}
if (*y.int_const == -1) {
return exec_op("NEGATE", 1);
}
return exec_arg_op("MULCONST", y.int_const, 1);
}
if (k > 0) {
y.unused();
return exec_arg_op("LSHIFT#", k, 1);
}
if (k == 0) {
y.unused();
return AsmOp::Nop();
}
}
if (x.is_int_const()) {
int k = is_pos_pow2(x.int_const);
if (x.int_const->signed_fits_bits(8) && k < 0) {
x.unused();
if (x.always_zero() && y.always_finite()) {
// dubious optimization: NaN * 0 = ?
r.set_const(x.int_const);
y.unused();
return push_const(r.int_const);
}
if (*x.int_const == 1 && y.always_finite()) {
return AsmOp::Nop();
}
if (*x.int_const == -1) {
return exec_op("NEGATE", 1);
}
return exec_arg_op("MULCONST", x.int_const, 1);
}
if (k > 0) {
x.unused();
return exec_arg_op("LSHIFT#", k, 1);
}
if (k == 0) {
x.unused();
return AsmOp::Nop();
}
}
return exec_op("MUL", 2);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_mul(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation where) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
tolk_assert(res.size() == 1 && args.size() == 2);
return compile_mul_internal(res[0], args[0], args[1], where);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_lshift(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation where) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
tolk_assert(res.size() == 1 && args.size() == 2);
VarDescr &r = res[0], &x = args[0], &y = args[1];
if (y.is_int_const()) {
auto yv = y.int_const->to_long();
if (yv < 0 || yv > 256) {
throw ParseError(where, "lshift argument is out of range");
} else if (x.is_int_const()) {
r.set_const(x.int_const << (int)yv);
if (!r.int_const->is_valid()) {
throw ParseError(where, "integer overflow");
}
x.unused();
y.unused();
return push_const(r.int_const);
}
}
r.val = emulate_lshift(x.val, y.val);
if (y.is_int_const()) {
int k = (int)(y.int_const->to_long());
if (!k /* && x.always_finite() */) {
// dubious optimization: what if x=NaN ?
y.unused();
return AsmOp::Nop();
}
y.unused();
return exec_arg_op("LSHIFT#", k, 1);
}
if (x.is_int_const()) {
auto xv = x.int_const->to_long();
if (xv == 1) {
x.unused();
return exec_op("POW2", 1);
}
if (xv == -1) {
x.unused();
return exec_op("-1 PUSHINT SWAP LSHIFT", 1);
}
}
return exec_op("LSHIFT", 2);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_rshift(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation where,
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
int round_mode) {
tolk_assert(res.size() == 1 && args.size() == 2);
VarDescr &r = res[0], &x = args[0], &y = args[1];
if (y.is_int_const()) {
auto yv = y.int_const->to_long();
if (yv < 0 || yv > 256) {
throw ParseError(where, "rshift argument is out of range");
} else if (x.is_int_const()) {
r.set_const(td::rshift(x.int_const, (int)yv, round_mode));
x.unused();
y.unused();
return push_const(r.int_const);
}
}
r.val = emulate_rshift(x.val, y.val);
std::string rshift = (round_mode < 0 ? "RSHIFT" : (round_mode ? "RSHIFTC" : "RSHIFTR"));
if (y.is_int_const()) {
int k = (int)(y.int_const->to_long());
if (!k /* && x.always_finite() */) {
// dubious optimization: what if x=NaN ?
y.unused();
return AsmOp::Nop();
}
y.unused();
return exec_arg_op(rshift + "#", k, 1);
}
return exec_op(rshift, 2);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_div_internal(VarDescr& r, VarDescr& x, VarDescr& y, SrcLocation where, int round_mode) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
if (x.is_int_const() && y.is_int_const()) {
r.set_const(div(x.int_const, y.int_const, round_mode));
if (!r.int_const->is_valid()) {
throw ParseError(where, *y.int_const == 0 ? "division by zero" : "integer overflow");
}
x.unused();
y.unused();
return push_const(r.int_const);
}
r.val = emulate_div(x.val, y.val);
if (y.is_int_const()) {
if (*y.int_const == 0) {
throw ParseError(where, "division by zero");
}
if (*y.int_const == 1 && x.always_finite()) {
y.unused();
return AsmOp::Nop();
}
if (*y.int_const == -1) {
y.unused();
return exec_op("NEGATE", 1);
}
int k = is_pos_pow2(y.int_const);
if (k > 0) {
y.unused();
std::string op = "RSHIFT";
if (round_mode >= 0) {
op += (round_mode > 0 ? 'C' : 'R');
}
return exec_arg_op(op + '#', k, 1);
}
}
std::string op = "DIV";
if (round_mode >= 0) {
op += (round_mode > 0 ? 'C' : 'R');
}
return exec_op(op, 2);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_div(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation where, int round_mode) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
tolk_assert(res.size() == 1 && args.size() == 2);
return compile_div_internal(res[0], args[0], args[1], where, round_mode);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_mod(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation where,
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
int round_mode) {
tolk_assert(res.size() == 1 && args.size() == 2);
VarDescr &r = res[0], &x = args[0], &y = args[1];
if (x.is_int_const() && y.is_int_const()) {
r.set_const(mod(x.int_const, y.int_const, round_mode));
if (!r.int_const->is_valid()) {
throw ParseError(where, *y.int_const == 0 ? "division by zero" : "integer overflow");
}
x.unused();
y.unused();
return push_const(r.int_const);
}
r.val = emulate_mod(x.val, y.val);
if (y.is_int_const()) {
if (*y.int_const == 0) {
throw ParseError(where, "division by zero");
}
if ((*y.int_const == 1 || *y.int_const == -1) && x.always_finite()) {
x.unused();
y.unused();
r.set_const(td::zero_refint());
return push_const(r.int_const);
}
int k = is_pos_pow2(y.int_const);
if (k > 0) {
y.unused();
std::string op = "MODPOW2";
if (round_mode >= 0) {
op += (round_mode > 0 ? 'C' : 'R');
}
return exec_arg_op(op + '#', k, 1);
}
}
std::string op = "MOD";
if (round_mode >= 0) {
op += (round_mode > 0 ? 'C' : 'R');
}
return exec_op(op, 2);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_muldiv(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation where,
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
int round_mode) {
tolk_assert(res.size() == 1 && args.size() == 3);
VarDescr &r = res[0], &x = args[0], &y = args[1], &z = args[2];
if (x.is_int_const() && y.is_int_const() && z.is_int_const()) {
r.set_const(muldiv(x.int_const, y.int_const, z.int_const, round_mode));
if (!r.int_const->is_valid()) {
throw ParseError(where, *z.int_const == 0 ? "division by zero" : "integer overflow");
}
x.unused();
y.unused();
z.unused();
return push_const(r.int_const);
}
if (x.always_zero() || y.always_zero()) {
// dubious optimization for z=0...
x.unused();
y.unused();
z.unused();
r.set_const(td::make_refint(0));
return push_const(r.int_const);
}
char c = (round_mode < 0) ? 0 : (round_mode > 0 ? 'C' : 'R');
r.val = emulate_div(emulate_mul(x.val, y.val), z.val);
if (z.is_int_const()) {
if (*z.int_const == 0) {
throw ParseError(where, "division by zero");
}
if (*z.int_const == 1) {
z.unused();
return compile_mul_internal(r, x, y, where);
}
}
if (y.is_int_const() && *y.int_const == 1) {
y.unused();
return compile_div_internal(r, x, z, where, round_mode);
}
if (x.is_int_const() && *x.int_const == 1) {
x.unused();
return compile_div_internal(r, y, z, where, round_mode);
}
if (z.is_int_const()) {
int k = is_pos_pow2(z.int_const);
if (k > 0) {
z.unused();
std::string op = "MULRSHIFT";
if (c) {
op += c;
}
return exec_arg_op(op + '#', k, 2);
}
}
if (y.is_int_const()) {
int k = is_pos_pow2(y.int_const);
if (k > 0) {
y.unused();
std::string op = "LSHIFT#DIV";
if (c) {
op += c;
}
return exec_arg_op(op, k, 2);
}
}
if (x.is_int_const()) {
int k = is_pos_pow2(x.int_const);
if (k > 0) {
x.unused();
std::string op = "LSHIFT#DIV";
if (c) {
op += c;
}
return exec_arg_op(op, k, 2);
}
}
std::string op = "MULDIV";
if (c) {
op += c;
}
return exec_op(op, 3);
}
int compute_compare(td::RefInt256 x, td::RefInt256 y, int mode) {
int s = td::cmp(x, y);
if (mode == 7) {
return s;
} else {
return -((mode >> (1 - s)) & 1);
}
}
// return value:
// 4 -> constant 1
// 2 -> constant 0
// 1 -> constant -1
// 3 -> 0 or -1
int compute_compare(const VarDescr& x, const VarDescr& y, int mode) {
switch (mode) {
case 1: // >
return x.always_greater(y) ? 1 : (x.always_leq(y) ? 2 : 3);
case 2: // =
return x.always_equal(y) ? 1 : (x.always_neq(y) ? 2 : 3);
case 3: // >=
return x.always_geq(y) ? 1 : (x.always_less(y) ? 2 : 3);
case 4: // <
return x.always_less(y) ? 1 : (x.always_geq(y) ? 2 : 3);
case 5: // <>
return x.always_neq(y) ? 1 : (x.always_equal(y) ? 2 : 3);
case 6: // <=
return x.always_leq(y) ? 1 : (x.always_greater(y) ? 2 : 3);
case 7: // <=>
return x.always_less(y)
? 1
: (x.always_equal(y)
? 2
: (x.always_greater(y)
? 4
: (x.always_leq(y) ? 3 : (x.always_geq(y) ? 6 : (x.always_neq(y) ? 5 : 7)))));
default:
return 7;
}
}
AsmOp compile_cmp_int(std::vector<VarDescr>& res, std::vector<VarDescr>& args, int mode) {
tolk_assert(mode >= 1 && mode <= 7);
tolk_assert(res.size() == 1 && args.size() == 2);
VarDescr &r = res[0], &x = args[0], &y = args[1];
if (x.is_int_const() && y.is_int_const()) {
int v = compute_compare(x.int_const, y.int_const, mode);
r.set_const(v);
x.unused();
y.unused();
return mode == 7 ? push_const(r.int_const) : AsmOp::BoolConst(v != 0);
}
int v = compute_compare(x, y, mode);
// std::cerr << "compute_compare(" << x << ", " << y << ", " << mode << ") = " << v << std::endl;
tolk_assert(v);
if (!(v & (v - 1))) {
r.set_const(v - (v >> 2) - 2);
x.unused();
y.unused();
return mode == 7 ? push_const(r.int_const) : AsmOp::BoolConst(v & 1);
}
r.val = ~0;
if (v & 1) {
r.val &= VarDescr::ConstTrue;
}
if (v & 2) {
r.val &= VarDescr::ConstZero;
}
if (v & 4) {
r.val &= VarDescr::ConstOne;
}
// std::cerr << "result: " << r << std::endl;
static const char* cmp_int_names[] = {"", "GTINT", "EQINT", "GTINT", "LESSINT", "NEQINT", "LESSINT"};
static const char* cmp_names[] = {"", "GREATER", "EQUAL", "GEQ", "LESS", "NEQ", "LEQ", "CMP"};
static int cmp_int_delta[] = {0, 0, 0, -1, 0, 0, 1};
if (mode != 7) {
if (y.is_int_const() && y.int_const >= -128 && y.int_const <= 127) {
y.unused();
return exec_arg_op(cmp_int_names[mode], y.int_const + cmp_int_delta[mode], 1);
}
if (x.is_int_const() && x.int_const >= -128 && x.int_const <= 127) {
x.unused();
mode = ((mode & 4) >> 2) | (mode & 2) | ((mode & 1) << 2);
return exec_arg_op(cmp_int_names[mode], x.int_const + cmp_int_delta[mode], 1);
}
}
return exec_op(cmp_names[mode], 2);
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_throw(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
tolk_assert(res.empty() && args.size() == 1);
VarDescr& x = args[0];
if (x.is_int_const() && x.int_const->unsigned_fits_bits(11)) {
x.unused();
return exec_arg_op("THROW", x.int_const, 0, 0);
} else {
return exec_op("THROWANY", 1, 0);
}
}
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
AsmOp compile_throw_if_unless(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation) {
[Tolk] v0.6 syntax: `fun`, `import`, `var`, types on the right, etc. Lots of changes, actually. Most noticeable are: - traditional //comments - #include -> import - a rule "import what you use" - ~ found -> !found (for -1/0) - null() -> null - is_null?(v) -> v == null - throw is a keyword - catch with swapped arguments - throw_if, throw_unless -> assert - do until -> do while - elseif -> else if - drop ifnot, elseifnot - drop rarely used operators A testing framework also appears here. All tests existed earlier, but due to significant syntax changes, their history is useless.
2024-10-31 07:11:41 +00:00
tolk_assert(res.empty() && args.size() == 3);
VarDescr &x = args[0], &y = args[1], &z = args[2];
if (!z.always_true() && !z.always_false()) {
throw Fatal("invalid usage of built-in symbol");
}
bool mode = z.always_true();
z.unused();
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
std::string suff = (mode ? "IF" : "IFNOT");
bool skip_cond = false;
if (y.always_true() || y.always_false()) {
y.unused();
skip_cond = true;
if (y.always_true() != mode) {
x.unused();
return AsmOp::Nop();
}
}
if (x.is_int_const() && x.int_const->unsigned_fits_bits(11)) {
x.unused();
return skip_cond ? exec_arg_op("THROW", x.int_const, 0, 0) : exec_arg_op("THROW"s + suff, x.int_const, 1, 0);
} else {
return skip_cond ? exec_op("THROWANY", 1, 0) : exec_op("THROWANY"s + suff, 2, 0);
}
}
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_throw_arg(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
tolk_assert(res.empty() && args.size() == 2);
VarDescr &x = args[1];
if (x.is_int_const() && x.int_const->unsigned_fits_bits(11)) {
x.unused();
return exec_arg_op("THROWARG", x.int_const, 1, 0);
} else {
return exec_op("THROWARGANY", 2, 0);
}
}
AsmOp compile_bool_const(std::vector<VarDescr>& res, std::vector<VarDescr>& args, bool val) {
tolk_assert(res.size() == 1 && args.empty());
VarDescr& r = res[0];
r.set_const(val ? -1 : 0);
return AsmOp::Const(val ? "TRUE" : "FALSE");
}
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
// fun loadInt (mutate s: slice, len: int): int asm(s len -> 1 0) "LDIX";
// fun loadUint (mutate s: slice, len: int): int asm( -> 1 0) "LDUX";
// fun preloadInt (s: slice, len: int): int asm "PLDIX";
// fun preloadUint(s: slice, len: int): int asm "PLDUX";
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
AsmOp compile_fetch_int(std::vector<VarDescr>& res, std::vector<VarDescr>& args, bool fetch, bool sgnd) {
tolk_assert(args.size() == 2 && res.size() == 1 + (unsigned)fetch);
auto &y = args[1], &r = res.back();
r.val = (sgnd ? VarDescr::FiniteInt : VarDescr::FiniteUInt);
int v = -1;
if (y.is_int_const() && y.int_const >= 0 && y.int_const <= 256) {
v = (int)y.int_const->to_long();
if (!v) {
r.val = VarDescr::ConstZero;
}
if (v == 1) {
r.val = (sgnd ? VarDescr::ValBool : VarDescr::ValBit);
}
if (v > 0) {
y.unused();
return exec_arg_op((fetch ? "LD"s : "PLD"s) + (sgnd ? 'I' : 'U'), v, 1, 1 + (unsigned)fetch);
}
}
return exec_op((fetch ? "LD"s : "PLD"s) + (sgnd ? "IX" : "UX"), 2, 1 + (unsigned)fetch);
}
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
// fun storeInt (mutate self: builder, x: int, len: int): self asm(x b len) "STIX";
// fun storeUint (mutate self: builder, x: int, len: int): self asm(x b len) "STUX";
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
AsmOp compile_store_int(std::vector<VarDescr>& res, std::vector<VarDescr>& args, bool sgnd) {
tolk_assert(args.size() == 3 && res.size() == 1);
auto& z = args[2];
if (z.is_int_const() && z.int_const > 0 && z.int_const <= 256) {
z.unused();
return exec_arg_op("ST"s + (sgnd ? 'I' : 'U'), z.int_const, 2, 1);
}
return exec_op("ST"s + (sgnd ? "IX" : "UX"), 3, 1);
}
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
// fun loadBits (mutate self: slice, len: int): self asm(s len -> 1 0) "LDSLICEX"
// fun preloadBits(self: slice, len: int): slice asm(s len -> 1 0) "PLDSLICEX"
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
AsmOp compile_fetch_slice(std::vector<VarDescr>& res, std::vector<VarDescr>& args, bool fetch) {
tolk_assert(args.size() == 2 && res.size() == 1 + (unsigned)fetch);
auto& y = args[1];
int v = -1;
if (y.is_int_const() && y.int_const > 0 && y.int_const <= 256) {
v = (int)y.int_const->to_long();
if (v > 0) {
y.unused();
return exec_arg_op(fetch ? "LDSLICE" : "PLDSLICE", v, 1, 1 + (unsigned)fetch);
}
}
return exec_op(fetch ? "LDSLICEX" : "PLDSLICEX", 2, 1 + (unsigned)fetch);
}
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
// fun tupleAt<X>(t: tuple, index: int): X asm "INDEXVAR";
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_tuple_at(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
tolk_assert(args.size() == 2 && res.size() == 1);
auto& y = args[1];
if (y.is_int_const() && y.int_const >= 0 && y.int_const < 16) {
y.unused();
return exec_arg_op("INDEX", y.int_const, 1, 1);
}
return exec_op("INDEXVAR", 2, 1);
}
[Tolk] Support syntax `tensorVar.0` and `tupleVar.0` It works both for reading and writing: > var t = (1, 2); > t.0; // 1 > t.0 = 5; > t; // (5, 2) It also works for typed/untyped tuples, producing INDEX and SETINDEX. Global tensors and tuples works. Nesting `t.0.1.2` works. `mutate` works. Even mixing tuples inside tensors inside a global for writing works.
2025-01-27 07:29:17 +00:00
// fun tupleSetAt<X>(mutate self: tuple, value: X, index: int): void asm "SETINDEXVAR";
AsmOp compile_tuple_set_at(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation) {
tolk_assert(args.size() == 3 && res.size() == 1);
auto& y = args[2];
if (y.is_int_const() && y.int_const >= 0 && y.int_const < 16) {
y.unused();
return exec_arg_op("SETINDEX", y.int_const, 1, 1);
}
return exec_op("SETINDEXVAR", 2, 1);
}
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
// fun __isNull<X>(X arg): bool
[Tolk] Rewrite lexer, spaces are not mandatory anymore A new lexer is noticeably faster and memory efficient (although splitting a file to tokens is negligible in a whole pipeline). But the purpose of rewriting lexer was not just to speed up, but to allow writing code without spaces: `2+2` is now 4, not a valid identifier as earlier. The variety of symbols allowed in identifier has greatly reduced and is now similar to other languages. SrcLocation became 8 bytes on stack everywhere. Command-line flags were also reworked: - the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved - flags like -A -P and so on are no more needed, actually
2024-10-31 06:59:23 +00:00
AsmOp compile_is_null(std::vector<VarDescr>& res, std::vector<VarDescr>& args, SrcLocation) {
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
tolk_assert(args.size() == 1 && res.size() == 1);
res[0].val = VarDescr::ValBool;
return exec_op("ISNULL", 1, 1);
}
void define_builtins() {
using namespace std::placeholders;
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
TypePtr Unit = TypeDataVoid::create();
TypePtr Int = TypeDataInt::create();
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
TypePtr Bool = TypeDataBool::create();
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
TypePtr Slice = TypeDataSlice::create();
TypePtr Builder = TypeDataBuilder::create();
TypePtr Tuple = TypeDataTuple::create();
std::vector<GenericsDeclaration::GenericsItem> itemsT;
itemsT.emplace_back("T");
TypePtr typeT = TypeDataGenericT::create("T");
const GenericsDeclaration* declGenericT = new GenericsDeclaration(std::move(itemsT));
std::vector ParamsInt1 = {Int};
std::vector ParamsInt2 = {Int, Int};
std::vector ParamsInt3 = {Int, Int, Int};
std::vector ParamsSliceInt = {Slice, Int};
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
// builtin operators
// they are internally stored as functions, because at IR level, there is no difference
// between calling `userAdd(a,b)` and `_+_(a,b)`
// since they are registered in a global symtable, technically, they can even be referenced from Tolk code,
// though it's a "hidden feature" and won't work well for overloads (`==` for int and bool, for example)
// unary operators
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("-_", ParamsInt1, Int, nullptr,
compile_unary_minus,
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("+_", ParamsInt1, Int, nullptr,
compile_unary_plus,
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
define_builtin_func("!_", ParamsInt1, Bool, nullptr,
std::bind(compile_logical_not, _1, _2, _3, true),
FunctionData::flagMarkedAsPure);
define_builtin_func("!b_", {Bool}, Bool, nullptr, // "overloaded" separate version for bool
std::bind(compile_logical_not, _1, _2, _3, false),
FunctionData::flagMarkedAsPure);
define_builtin_func("~_", ParamsInt1, Int, nullptr,
compile_bitwise_not,
FunctionData::flagMarkedAsPure);
// binary operators
define_builtin_func("_+_", ParamsInt2, Int, nullptr,
compile_add,
FunctionData::flagMarkedAsPure);
define_builtin_func("_-_", ParamsInt2, Int, nullptr,
compile_sub,
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("_*_", ParamsInt2, Int, nullptr,
compile_mul,
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("_/_", ParamsInt2, Int, nullptr,
std::bind(compile_div, _1, _2, _3, -1),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("_~/_", ParamsInt2, Int, nullptr,
std::bind(compile_div, _1, _2, _3, 0),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("_^/_", ParamsInt2, Int, nullptr,
std::bind(compile_div, _1, _2, _3, 1),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("_%_", ParamsInt2, Int, nullptr,
std::bind(compile_mod, _1, _2, _3, -1),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("_<<_", ParamsInt2, Int, nullptr,
compile_lshift,
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("_>>_", ParamsInt2, Int, nullptr,
std::bind(compile_rshift, _1, _2, _3, -1),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("_~>>_", ParamsInt2, Int, nullptr,
std::bind(compile_rshift, _1, _2, _3, 0),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("_^>>_", ParamsInt2, Int, nullptr,
std::bind(compile_rshift, _1, _2, _3, 1),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
define_builtin_func("_&_", ParamsInt2, Int, nullptr, // also works for bool
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
compile_bitwise_and,
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
define_builtin_func("_|_", ParamsInt2, Int, nullptr, // also works for bool
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
compile_bitwise_or,
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
define_builtin_func("_^_", ParamsInt2, Int, nullptr, // also works for bool
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
compile_bitwise_xor,
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
define_builtin_func("_==_", ParamsInt2, Int, nullptr, // also works for bool
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
std::bind(compile_cmp_int, _1, _2, 2),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
define_builtin_func("_!=_", ParamsInt2, Int, nullptr, // also works for bool
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
std::bind(compile_cmp_int, _1, _2, 5),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("_<_", ParamsInt2, Int, nullptr,
std::bind(compile_cmp_int, _1, _2, 4),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("_>_", ParamsInt2, Int, nullptr,
std::bind(compile_cmp_int, _1, _2, 1),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("_<=_", ParamsInt2, Int, nullptr,
std::bind(compile_cmp_int, _1, _2, 6),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("_>=_", ParamsInt2, Int, nullptr,
std::bind(compile_cmp_int, _1, _2, 3),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("_<=>_", ParamsInt2, Int, nullptr,
std::bind(compile_cmp_int, _1, _2, 7),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
// special function used for internal compilation of some lexical constructs
// for example, `throw 123;` is actually calling `__throw(123)`
define_builtin_func("__true", {}, Bool, nullptr, /* AsmOp::Const("TRUE") */
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
std::bind(compile_bool_const, _1, _2, true),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
define_builtin_func("__false", {}, Bool, nullptr, /* AsmOp::Const("FALSE") */
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
std::bind(compile_bool_const, _1, _2, false),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("__null", {}, typeT, declGenericT,
AsmOp::Const("PUSHNULL"),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
define_builtin_func("__isNull", {typeT}, Bool, declGenericT,
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
compile_is_null,
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("__throw", ParamsInt1, Unit, nullptr,
compile_throw,
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
0);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("__throw_arg", {typeT, Int}, Unit, declGenericT,
compile_throw_arg,
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
0);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("__throw_if_unless", ParamsInt3, Unit, nullptr,
compile_throw_if_unless,
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
0);
[Tolk] `bool` type (-1/0 int under the hood) Comparison operators `== / >= /...` return `bool`. Logical operators `&& ||` return bool. Constants `true` and `false` have the `bool` type. Lots of stdlib functions return `bool`, not `int`. Operator `!x` supports both `int` and `bool`. Condition of `if` accepts both `int` and `bool`. Arithmetic operators are restricted to integers. Logical `&&` and `||` accept both `bool` and `int`. No arithmetic operations with bools allowed (only bitwise and logical).
2025-01-13 08:21:24 +00:00
// functions from stdlib marked as `builtin`, implemented at compiler level for optimizations
// (for example, `loadInt(1)` is `1 LDI`, but `loadInt(n)` for non-constant requires it be on a stack and `LDIX`)
define_builtin_func("mulDivFloor", ParamsInt3, Int, nullptr,
std::bind(compile_muldiv, _1, _2, _3, -1),
FunctionData::flagMarkedAsPure);
define_builtin_func("mulDivRound", ParamsInt3, Int, nullptr,
std::bind(compile_muldiv, _1, _2, _3, 0),
FunctionData::flagMarkedAsPure);
define_builtin_func("mulDivCeil", ParamsInt3, Int, nullptr,
std::bind(compile_muldiv, _1, _2, _3, 1),
FunctionData::flagMarkedAsPure);
define_builtin_func("mulDivMod", ParamsInt3, TypeDataTensor::create({Int, Int}), nullptr,
AsmOp::Custom("MULDIVMOD", 3, 2),
FunctionData::flagMarkedAsPure);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("loadInt", ParamsSliceInt, Int, nullptr,
std::bind(compile_fetch_int, _1, _2, true, true),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure | FunctionData::flagHasMutateParams | FunctionData::flagAcceptsSelf,
{}, {1, 0});
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("loadUint", ParamsSliceInt, Int, nullptr,
std::bind(compile_fetch_int, _1, _2, true, false),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure | FunctionData::flagHasMutateParams | FunctionData::flagAcceptsSelf,
{}, {1, 0});
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("loadBits", ParamsSliceInt, Slice, nullptr,
std::bind(compile_fetch_slice, _1, _2, true),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure | FunctionData::flagHasMutateParams | FunctionData::flagAcceptsSelf,
{}, {1, 0});
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("preloadInt", ParamsSliceInt, Int, nullptr,
std::bind(compile_fetch_int, _1, _2, false, true),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure | FunctionData::flagAcceptsSelf);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("preloadUint", ParamsSliceInt, Int, nullptr,
std::bind(compile_fetch_int, _1, _2, false, false),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure | FunctionData::flagAcceptsSelf);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("preloadBits", ParamsSliceInt, Slice, nullptr,
std::bind(compile_fetch_slice, _1, _2, false),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure | FunctionData::flagAcceptsSelf);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("storeInt", {Builder, Int, Int}, Unit, nullptr,
std::bind(compile_store_int, _1, _2, true),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure | FunctionData::flagHasMutateParams | FunctionData::flagAcceptsSelf | FunctionData::flagReturnsSelf,
{1, 0, 2}, {});
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("storeUint", {Builder, Int, Int}, Unit, nullptr,
std::bind(compile_store_int, _1, _2, false),
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure | FunctionData::flagHasMutateParams | FunctionData::flagAcceptsSelf | FunctionData::flagReturnsSelf,
{1, 0, 2}, {});
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("tupleAt", {Tuple, Int}, typeT, declGenericT,
compile_tuple_at,
[Tolk] AST-based semantic analysis, get rid of Expr This is a huge refactoring focusing on untangling compiler internals (previously forked from FunC). The goal is to convert AST directly to Op (a kind of IR representation), doing all code analysis at AST level. Noteable changes: - AST-based semantic kernel includes: registering global symbols, scope handling and resolving local/global identifiers, lvalue/rvalue calc and check, implicit return detection, mutability analysis, pure/impure validity checks, simple constant folding - values of `const` variables are calculated NOT based on CodeBlob, but via a newly-introduced AST-based constant evaluator - AST vertices are now inherited from expression/statement/other; expression vertices have common properties (TypeExpr, lvalue/rvalue) - symbol table is rewritten completely, SymDef/SymVal no longer exist, lexer now doesn't need to register identifiers - AST vertices have references to symbols, filled at different stages of pipeline - the remaining "FunC legacy part" is almost unchanged besides Expr which was fully dropped; AST is converted to Ops (IR) directly
2024-12-16 18:19:45 +00:00
FunctionData::flagMarkedAsPure | FunctionData::flagAcceptsSelf);
[Tolk] Support syntax `tensorVar.0` and `tupleVar.0` It works both for reading and writing: > var t = (1, 2); > t.0; // 1 > t.0 = 5; > t; // (5, 2) It also works for typed/untyped tuples, producing INDEX and SETINDEX. Global tensors and tuples works. Nesting `t.0.1.2` works. `mutate` works. Even mixing tuples inside tensors inside a global for writing works.
2025-01-27 07:29:17 +00:00
define_builtin_func("tupleSetAt", {Tuple, typeT, Int}, Unit, declGenericT,
compile_tuple_set_at,
FunctionData::flagMarkedAsPure | FunctionData::flagHasMutateParams | FunctionData::flagAcceptsSelf);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("debugPrint", {typeT}, Unit, declGenericT,
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
AsmOp::Custom("s0 DUMP DROP", 1, 1),
0);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("debugPrintString", {typeT}, Unit, declGenericT,
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
AsmOp::Custom("STRDUMP DROP", 1, 1),
0);
[Tolk] Rewrite the type system from Hindley-Milner to static typing FunC's (and Tolk's before this PR) type system is based on Hindley-Milner. This is a common approach for functional languages, where types are inferred from usage through unification. As a result, type declarations are not necessary: () f(a,b) { return a+b; } // a and b now int, since `+` (int, int) While this approach works for now, problems arise with the introduction of new types like bool, where `!x` must handle both int and bool. It will also become incompatible with int32 and other strict integers. This will clash with structure methods, struggle with proper generics, and become entirely impractical for union types. This PR completely rewrites the type system targeting the future. 1) type of any expression is inferred and never changed 2) this is available because dependent expressions already inferred 3) forall completely removed, generic functions introduced (they work like template functions actually, instantiated while inferring) 4) instantiation `<...>` syntax, example: `t.tupleAt<int>(0)` 5) `as` keyword, for example `t.tupleAt(0) as int` 6) methods binding is done along with type inferring, not before ("before", as worked previously, was always a wrong approach)
2024-12-30 15:31:27 +00:00
define_builtin_func("debugDumpStack", {}, Unit, nullptr,
[Tolk] Get rid of ~tilda with `mutate` and `self` methods This is a very big change. If FunC has `.methods()` and `~methods()`, Tolk has only dot, one and only way to call a `.method()`. A method may mutate an object, or may not. It's a behavioral and semantic difference from FunC. - `cs.loadInt(32)` modifies a slice and returns an integer - `b.storeInt(x, 32)` modifies a builder - `b = b.storeInt()` also works, since it not only modifies, but returns - chained methods also work, they return `self` - everything works exactly as expected, similar to JS - no runtime overhead, exactly same Fift instructions - custom methods are created with ease - tilda `~` does not exist in Tolk at all
2024-10-31 07:18:54 +00:00
AsmOp::Custom("DUMPSTK", 0, 0),
0);
[Tolk] Compiler built-in `__expect_type()` for testing purposes Currently, tolk-tester can test various "output" of the compiler: pass input and check output, validate fif codegen, etc. But it can not test compiler internals and AST representation. I've added an ability to have special functions to check/expose internal compiler state. The first (and the only now) is: > __expect_type(some_expr, "<type>"); Such a call has special treatment in a compilation process. Compilation fails if this expression doesn't have requested type. It's intended to be used in tests only. Not present in stdlib.
2025-01-14 18:41:15 +00:00
// functions not presented in stdlib at all
// used in tolk-tester to check/expose internal compiler state
// each of them is handled in a special way, search by its name
define_builtin_func("__expect_type", {TypeDataUnknown::create(), Slice}, Unit, nullptr,
AsmOp::Nop(),
FunctionData::flagMarkedAsPure);
[Tolk] Initial commit of TOLK Language: fork all sources from FunC The Tolk Language will be positioned as "next-generation FunC". It's literally a fork of a FunC compiler, introducing familiar syntax similar to TypeScript, but leaving all low-level optimizations untouched. Note, that FunC sources are partially stored in the parser/ folder (shared with TL/B). In Tolk, nothing is shared. Everything from parser/ is copied into tolk/ folder.
2024-10-31 06:51:07 +00:00
}
} // namespace tolk
Reference in a new issue Copy permalink