Several related changes:
- stdlib.tolk is embedded into a distribution (deb package or tolk-js),
the user won't have to download it and store as a project file;
it's an important step to maintain correct language versioning
- stdlib.tolk is auto-included, that's why all its functions are
available out of the box
- strict includes: you can't use symbol `f` from another file
unless you've #include'd this file
- drop all C++ global variables holding compilation state,
merge them into a single struct CompilerState located at
compiler-state.h; for instance, stdlib filename is also there
A new lexer is noticeably faster and memory efficient
(although splitting a file to tokens is negligible in a whole pipeline).
But the purpose of rewriting lexer was not just to speed up,
but to allow writing code without spaces:
`2+2` is now 4, not a valid identifier as earlier.
The variety of symbols allowed in identifier has greatly reduced
and is now similar to other languages.
SrcLocation became 8 bytes on stack everywhere.
Command-line flags were also reworked:
- the input for Tolk compiler is only a single file now, it's parsed, and parsing continues while new #include are resolved
- flags like -A -P and so on are no more needed, actually
