dvander@195: Massive refactoring to support better type systems. Read on for more. dvander@195: dvander@195: The previous compiler had a simple pipeline, best outlined as: dvander@195: (1) Parsing and name binding -> AST dvander@195: (2) Storage allocation -> AST dvander@195: (3) Semantic Analysis (SemA) -> AST dvander@195: (4) Bytecode Compilation (BC) -> Code dvander@195: dvander@195: This pipeline, unfortunately, had two problems. First, name binding cannot be performed in one pass if we wish to have multi-file support in the form of modules. Name binding must be two-pass. dvander@195: dvander@195: Second, because SemA was only capable of annotating AST nodes with a single type, most coercion work had to be duplicated in the BC phase, as coercion was not explicit in the SemA output. This made introducing new type rules extremely difficult, and all but precluded concepts like operator overloading (or overloading at all). dvander@195: dvander@195: This patch rewrites most of Keima's backend. Of interest are the following changes: dvander@195: (1) CompileContext has been refactored around future multi-file support. dvander@195: (2) Parsing no longer performs any name binding. dvander@195: (3) The grammar for type-and-name has been changed from: dvander@195: (Label | Identifier)? Identifier dvander@195: to: dvander@195: Type? Identifier dvander@195: where: dvander@195: Type ::= OldType | NewType dvander@195: OldType ::= Label dvander@195: NewType ::= Identifier (. NewType)* dvander@195: dvander@195: dvander@195: Although we do not implement the full production for Type yet, this dvander@195: distinction is important. A type identifier is now affixed to the AST dvander@195: as an Expression (right now, always a NameProxy), so it can fully dvander@195: participate in name binding. dvander@195: dvander@195: (4) Immediately after parsing, the AST goes through a NamePopulation phase. dvander@195: This phase creates Scope objects for every scope which declares a name, dvander@195: and if those names are declaring types or functions, a Symbol is created dvander@195: and entered into the scope. Symbols entirely replace the old BoundName dvander@195: classes. dvander@195: dvander@195: (5) After NamePopulation comes NameBinding. This phase performs three steps: dvander@195: (a) Links Scope objects together, to form a tree. dvander@195: (b) Binds any free name to an existing name in the scope chain. dvander@195: (c) Creates and registers any Symbols for names which have not yet been dvander@195: added to the scope (for example, local variables). dvander@195: dvander@195: In accordance, all "allocation" and "binding" concepts have been removed dvander@195: from Scopes, which are now lightweight container classes. dvander@195: dvander@195: (6) SemA now generates bytecode for each statement in the AST. Expression dvander@195: compilation is performed via a separate mechanism called HIR (High-level dvander@195: Intermediate Representation). To analyze an expression, SemA will walk dvander@195: the AST, and produce a HIR object for each node. HIR is typed, and SemA dvander@195: may insert coercion nodes, or even expand an AST node into multiple HIR dvander@195: nodes. The result of evaluating an expression in SemA is therefore the dvander@195: the root of a HIR tree, which SemA then sends to the HIRTranslator, which dvander@195: performs bytecode generation. dvander@195: dvander@195: As before, SemA still performs full semantic analysis. However, not it dvander@195: also produces each function's bytecode. dvander@195: dvander@195: This split allows us to decompose potentially complex semantics into dvander@195: a more fine-grained, AST-like structure, which can have very simple dvander@195: code-generation logic. For example, (1.0 + 5) might look like: dvander@195: dvander@195: HAdd(HFloat(1.0), HConvert(HInteger(5), )) dvander@195: dvander@195: As part of this decomposition, many opcodes have been removed. Rather than dvander@195: using typed jumps, we now expand jumps into longer tests. For example: dvander@195: jge.f