src/compiler/Parser.cpp
author David Anderson <dvander@alliedmods.net>
Sat Nov 17 18:41:01 2012 -0800 (2012-11-17)
changeset 195 c1ba166f3fc4
parent 194 src/Parser.cpp@b029d63565a2
child 209 5cbec15fadfe
permissions -rw-r--r--
Massive SemA/BC refactoring to support better type systems. Read on for more.

The previous compiler had a simple pipeline, best outlined as:
(1) Parsing and name binding -> AST
(2) Storage allocation -> AST
(3) Semantic Analysis (SemA) -> AST
(4) Bytecode Compilation (BC) -> Code

This pipeline, unfortunately, had two problems. First, name binding cannot be performed in one pass if we wish to have multi-file support in the form of modules. Name binding must be two-pass.

Second, because SemA was only capable of annotating AST nodes with a single type, most coercion work had to be duplicated in the BC phase, as coercion was not explicit in the SemA output. This made introducing new type rules extremely difficult, and all but precluded concepts like operator overloading (or overloading at all).

This patch rewrites most of Keima's backend. Of interest are the following changes:
(1) CompileContext has been refactored around future multi-file support.
(2) Parsing no longer performs any name binding.
(3) The grammar for type-and-name has been changed from:
(Label | Identifier)? Identifier
to:
Type? Identifier
where:
Type ::= OldType | NewType
OldType ::= Label
NewType ::= Identifier (. NewType)*


Although we do not implement the full production for Type yet, this
distinction is important. A type identifier is now affixed to the AST
as an Expression (right now, always a NameProxy), so it can fully
participate in name binding.

(4) Immediately after parsing, the AST goes through a NamePopulation phase.
This phase creates Scope objects for every scope which declares a name,
and if those names are declaring types or functions, a Symbol is created
and entered into the scope. Symbols entirely replace the old BoundName
classes.

(5) After NamePopulation comes NameBinding. This phase performs three steps:
(a) Links Scope objects together, to form a tree.
(b) Binds any free name to an existing name in the scope chain.
(c) Creates and registers any Symbols for names which have not yet been
added to the scope (for example, local variables).

In accordance, all "allocation" and "binding" concepts have been removed
from Scopes, which are now lightweight container classes.

(6) SemA now generates bytecode for each statement in the AST. Expression
compilation is performed via a separate mechanism called HIR (High-level
Intermediate Representation). To analyze an expression, SemA will walk
the AST, and produce a HIR object for each node. HIR is typed, and SemA
may insert coercion nodes, or even expand an AST node into multiple HIR
nodes. The result of evaluating an expression in SemA is therefore the
the root of a HIR tree, which SemA then sends to the HIRTranslator, which
performs bytecode generation.

As before, SemA still performs full semantic analysis. However, not it
also produces each function's bytecode.

This split allows us to decompose potentially complex semantics into
a more fine-grained, AST-like structure, which can have very simple
code-generation logic. For example, (1.0 + 5) might look like:

HAdd(HFloat(1.0), HConvert(HInteger(5), <Float>))

As part of this decomposition, many opcodes have been removed. Rather than
using typed jumps, we now expand jumps into longer tests. For example:
jge.f <label>
becomes:
ge.f
cvt.f2b
jt <label>

This simplifies the pipeline, and JITs should be able to melt the added
work away.

HIR is not used at a statement level, and HIR does not have any concept
of control-flow.

(7) The old BytecodeCompiler has been removed, as its work is now split
between SemA and HIRTranslator.


In addition, some minor refactorings have taken place:

(1) Opcodes.tbl no longer hardcodes numbers (aaah).
(2) Type has been split into smaller, typed structs.
(3) BytecodeEmitter no longer relies on Pools or RootScopes.
(4) SemA is now responsible for variable/storage allocation.
(5) Native table are now global, rather than per-module. As such, native
declarations now result in a Native object (which still requires a
CALLNATIVE opcode), which is an index into the global table. Natives
must be bound globally. This is in preparation for module support.
(6) Publics are no longer tracked, but rather registered via a global
callback upon module load. This callback can be set by embedders. This
is in preparation for multi-file support.
(7) The BytecodeEmitter now performs Symbol allocations itself, and it
also generates Code objects itself, which removes a good deal of
complexity.

Finally, a test harness has been added. This harness is a python script which finds *.test files, recursively, in the tests folder. For each file it runs the corresponding .sp. The contents of the .test file must match stdout+stderr.
     1 /* vim: set ts=4 sw=4 tw=99 et:
     2  *
     3  * Copyright (C) 2012 David Anderson
     4  *
     5  * This file is part of SourcePawn.
     6  *
     7  * SourcePawn is free software: you can redistribute it and/or modify it under
     8  * the terms of the GNU General Public License as published by the Free
     9  * Software Foundation, either version 3 of the License, or (at your option)
    10  * any later version.
    11  * 
    12  * SourcePawn is distributed in the hope that it will be useful, but WITHOUT ANY
    13  * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
    14  * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
    15  *
    16  * You should have received a copy of the GNU General Public License along with
    17  * SourcePawn. If not, see http://www.gnu.org/licenses/.
    18  */
    19 #include <string.h>
    20 #include "../Zone.h"
    21 #include "../FixedArray.h"
    22 #include "../Heap.h"
    23 #include "../Strings.h"
    24 #include "../TypeManager.h"
    25 #include "Parser.h"
    26 #include "../Heap-inl.h"
    27 
    28 using namespace ke;
    29 
    30 Parser::Parser(Zone *zone, CompileContext &cc, const char *stream, size_t length)
    31   : zone_(zone),
    32     cc_(cc),
    33     pool_(zone_->pool()),
    34     scanner_(cc, stream, length),
    35     unbound_(NULL)
    36 {
    37     unbound_ = new (pool_) PoolList<NameProxy *>();
    38     imports_ = new (pool_) PoolList<ImportStatement *>();
    39 }
    40 
    41 bool
    42 Parser::peek(TokenKind kind)
    43 {
    44     return scanner_.peek() == kind;
    45 }
    46 
    47 bool
    48 Parser::match(TokenKind kind)
    49 {
    50     if (!peek(kind))
    51         return false;
    52     scanner_.scan();
    53     return true;
    54 }
    55 
    56 bool
    57 Parser::matchSemicolon()
    58 {
    59     return expect(TOK_SEMICOLON);
    60 }
    61 
    62 bool
    63 Parser::expect(TokenKind kind)
    64 {
    65     TokenKind got = scanner_.scan();
    66     if (got == kind)
    67         return true;
    68     cc_.reportError(scanner_.begin(), Message_WrongToken, TokenNames[kind], TokenNames[got]);
    69     return false;
    70 }
    71 
    72 bool
    73 Parser::matchName(String **name, TokenKind kind)
    74 {
    75     assert(kind == TOK_NAME || kind == TOK_LABEL);
    76     *name = NULL;
    77     if (!match(kind))
    78         return false;
    79     *name = zone_->makeSymbol(scanner_.literal(), scanner_.literal_length());
    80     return true;
    81 }
    82 
    83 String *
    84 Parser::expectName()
    85 {
    86     if (!expect(TOK_NAME))
    87         return NULL;
    88     return zone_->makeSymbol(scanner_.literal(), scanner_.literal_length());
    89 }
    90 
    91 Expression *
    92 Parser::primitive()
    93 {
    94     TokenKind kind = scanner_.peek();
    95     switch (kind) {
    96       case TOK_FLOAT_LITERAL:
    97       {
    98         scanner_.scan();
    99         double value = Scanner::ParseDouble(scanner_.literal());
   100         return new (pool_) FloatLiteral(scanner_.begin(), value);
   101       }
   102 
   103       case TOK_HEX_LITERAL:
   104       {
   105         scanner_.scan();
   106         int value = strtol(scanner_.literal(), NULL, 16);
   107         return new (pool_) IntegerLiteral(scanner_.begin(), value);
   108       }
   109 
   110       case TOK_INTEGER_LITERAL:
   111       {
   112         scanner_.scan();
   113         int value = atoi(scanner_.literal());
   114         return new (pool_) IntegerLiteral(scanner_.begin(), value);
   115       }
   116 
   117       case TOK_TRUE:
   118       case TOK_FALSE:
   119         scanner_.scan();
   120         return new (pool_) BooleanLiteral(scanner_.begin(), kind);
   121 
   122       case TOK_STRING_LITERAL:
   123       {
   124         scanner_.scan();
   125         Local<ByteArray> array(zone_, ByteArray::New(zone_, scanner_.literal_length() + 1));
   126         if (!array)
   127             return NULL;
   128         memcpy(array->raw(), scanner_.literal(), scanner_.literal_length() + 1);
   129         return new (pool_) StringLiteral(scanner_.begin(), array);
   130       }
   131 
   132       default:
   133       {
   134         if (kind != TOK_ERROR)
   135             cc_.reportError(scanner_.begin(), Message_ExpectedExpression, TokenNames[kind]);
   136         return NULL;
   137       }
   138     }
   139 }
   140 
   141 Expression *
   142 Parser::arrayLiteral(TokenKind kind)
   143 {
   144     assert(kind == TOK_LBRACE || kind == TOK_LBRACKET);
   145 
   146     TokenKind close = (kind == TOK_LBRACE)
   147                       ? TOK_RBRACE
   148                       : TOK_RBRACKET;
   149 
   150     SourcePosition pos = scanner_.begin();
   151 
   152     ExpressionList *list = new (pool_) ExpressionList();
   153     do {
   154         if (match(close))
   155             break;
   156 
   157         Expression *expr = NULL;
   158         if (kind == TOK_LBRACE) {
   159             if (peek(TOK_LBRACE))
   160                 expr = arrayLiteral(TOK_LBRACE);
   161             else
   162                 expr = primitive();
   163         } else {
   164             expr = expression();
   165         }
   166         if (!expr)
   167             return NULL;
   168 
   169         if (!list->append(expr))
   170             return NULL;
   171 
   172         if (!match(TOK_COMMA)) {
   173             if (!expect(close))
   174                 return NULL;
   175             break;
   176         }
   177     } while (true);
   178 
   179     return new (pool_) ArrayLiteral(pos, kind, list);
   180 }
   181 
   182 Expression *
   183 Parser::prefix()
   184 {
   185     switch (scanner_.peek()) {
   186       case TOK_LPAREN:
   187       {
   188         scanner_.scan();
   189         Expression *expr = expression();
   190         if (!expr)
   191             return NULL;
   192         if (!expect(TOK_RPAREN))
   193             return NULL;
   194         return expr;
   195       }
   196 
   197       case TOK_NAME:
   198       {
   199         Local<String> name(zone_, expectName());
   200         if (!name)
   201             return NULL;
   202         return new (pool_) NameProxy(scanner_.begin(), name);
   203       }
   204 
   205       default:
   206         return primitive();
   207     }
   208 }
   209 
   210 // Parses only old-style declarations.
   211 bool
   212 Parser::typeAndName(Expression **type, String **namep)
   213 {
   214     if (match(TOK_LABEL)) {
   215         // Old-style declaration:
   216         //   new X:type
   217         Local<String> name(zone_, zone_->makeSymbol(scanner_.literal(), scanner_.literal_length()));
   218         if (!name)
   219             return false;
   220 
   221         *type = new (pool_) NameProxy(scanner_.begin(), name);
   222         *namep = expectName();
   223         return !!*namep;
   224     }
   225     
   226     *type = NULL;
   227     *namep = expectName();
   228 
   229     return !!*namep;
   230 }
   231 
   232 // Parses both old and new style declarations.
   233 bool
   234 Parser::fullTypeAndName(Expression **type, String **namep, DeclKind kind)
   235 {
   236     // Declaration:
   237     //   NewDecl ::= Type Ident
   238     //   OldDecl ::= Label? (Ident | "...")
   239     //   Type    ::= Ident
   240     //   Label   ::= Ident ":"
   241     //
   242     if (match(TOK_LABEL)) {
   243         // We something of the form:
   244         //     Label Ident
   245         // or: Label "..."
   246         Local<String> name(zone_, zone_->makeSymbol(scanner_.literal(), scanner_.literal_length()));
   247         if (!name)
   248             return false;
   249 
   250         *type = new (pool_) NameProxy(scanner_.begin(), name);
   251         if (kind == DeclArg && match(TOK_ELLIPSES)) {
   252             *namep = NULL;
   253         } else {
   254             if ((*namep = expectName()) == NULL)
   255                 return false;
   256         }
   257         return true;
   258     }
   259 
   260     Local<String> name(zone_, expectName());
   261     if (!name)
   262         return false;
   263 
   264     if (peek(TOK_NAME) || (kind == DeclArg && peek(TOK_ELLIPSES))) {
   265         // We got something of the form:
   266         //    Ident Ident
   267         *type = new (pool_) NameProxy(scanner_.begin(), name);
   268         if (peek(TOK_NAME)) {
   269             if ((*namep = expectName()) == NULL)
   270                 return false;
   271         } else {
   272             *namep = NULL;
   273         }
   274         return true;
   275     }
   276 
   277     // We got something of the form:
   278     //    Ident
   279     *type = NULL;
   280     *namep = name;
   281     return true;
   282 }
   283 
   284 Expression *
   285 Parser::call(Expression *callee)
   286 {
   287     ExpressionList *arguments = new (pool_) ExpressionList();
   288 
   289     SourcePosition pos = scanner_.begin();
   290     expect(TOK_LPAREN);
   291     
   292     if (!match(TOK_RPAREN)) {
   293         while (true) {
   294             Expression *expr = expression();
   295             if (!expr)
   296                 return NULL;
   297 
   298             if (!arguments->append(expr))
   299                 return NULL;
   300 
   301             if (!match(TOK_COMMA))
   302                 break;
   303         }
   304 
   305         if (!expect(TOK_RPAREN))
   306             return NULL;
   307     }
   308 
   309     return new (pool_) CallExpression(pos, callee, arguments);
   310 }
   311 
   312 Expression *
   313 Parser::index(Expression *left)
   314 {
   315     expect(TOK_LBRACKET);
   316     
   317     SourcePosition pos = scanner_.begin();
   318     Expression *expr = expression();
   319     if (!expr)
   320         return NULL;
   321 
   322     if (!expect(TOK_RBRACKET))
   323         return NULL;
   324 
   325     return new (pool_) IndexExpression(pos, left, expr);
   326 }
   327 
   328 Expression *
   329 Parser::field(Expression *left)
   330 {
   331     expect(TOK_DOT);
   332 
   333     SourcePosition pos = scanner_.begin();
   334     Local<String> name(zone_, expectName());
   335     if (!name)
   336         return NULL;
   337 
   338     return new (pool_) FieldExpression(pos, left, name);
   339 }
   340 
   341 Expression *
   342 Parser::primary()
   343 {
   344     Expression *expr = prefix();
   345     if (!expr)
   346         return NULL;
   347 
   348     for (;;) {
   349         switch (scanner_.peek()) {
   350           case TOK_LPAREN:
   351             if ((expr = call(expr)) == NULL)
   352                 return NULL;
   353             break;
   354 
   355           case TOK_LBRACKET:
   356             if ((expr = index(expr)) == NULL)
   357                 return NULL;
   358             break;
   359 
   360           case TOK_DOT:
   361             if ((expr = field(expr)) == NULL)
   362                 return NULL;
   363             break;
   364 
   365           default:
   366             return expr;
   367         }
   368     }
   369 }
   370 
   371 Expression *
   372 Parser::unary()
   373 {
   374     TokenKind token = scanner_.peek();
   375 
   376     SourcePosition pos = scanner_.begin();
   377     switch (token) {
   378       case TOK_INCREMENT:
   379       case TOK_DECREMENT:
   380       {
   381         scanner_.scan();
   382         Expression *expr = unary();
   383         if (!expr)
   384             return NULL;
   385         return new (pool_) IncDecExpression(pos, token, expr, false);
   386       }
   387 
   388       case TOK_MINUS:
   389       case TOK_NOT:
   390       case TOK_TILDE:
   391       {
   392         scanner_.scan();
   393         Expression *expr = unary();
   394         if (!expr)
   395             return NULL;
   396         return new (pool_) UnaryExpression(pos, token, expr);
   397       }
   398 
   399       case TOK_SIZEOF:
   400       {
   401         scanner_.scan();
   402         if (!expect(TOK_LPAREN))
   403             return NULL;
   404         Expression *expr = unary();
   405         if (!expr)
   406             return NULL;
   407         if (!expect(TOK_RPAREN))
   408             return NULL;
   409         return new (pool_) UnaryExpression(pos, token, expr);
   410       }
   411 
   412       case TOK_LABEL:
   413       {
   414         scanner_.scan();
   415         Local<String> tag(zone_, zone_->makeSymbol(scanner_.literal(), scanner_.literal_length()));
   416         if (!tag)
   417             return NULL;
   418 
   419         NameProxy *proxy = new (pool_) NameProxy(scanner_.begin(), tag);
   420 
   421         Expression *expr = unary();
   422         if (!expr)
   423             return NULL;
   424 
   425         return new (pool_) UnaryExpression(pos, TOK_LABEL, expr, proxy);
   426       }
   427 
   428       default:
   429         break;
   430     }
   431 
   432     Expression *expr = primary();
   433 
   434     token = scanner_.peek();
   435     if (token == TOK_INCREMENT || token == TOK_DECREMENT) {
   436         scanner_.scan();
   437         return new (pool_) IncDecExpression(pos, token, expr, true);
   438     }
   439 
   440     return expr;
   441 }
   442 
   443 Expression *
   444 Parser::multiplication()
   445 {
   446     Expression *left = unary();
   447     do {
   448         if (!match(TOK_SLASH) && !match(TOK_STAR) && !match(TOK_PERCENT))
   449             break;
   450         SourcePosition pos = scanner_.begin();
   451         TokenKind kind = scanner_.last();
   452         Expression *right = unary();
   453         if (!right)
   454             return NULL;
   455         left = new (pool_) BinaryExpression(pos, kind, left, right);
   456     } while (left);
   457     return left;
   458 }
   459 
   460 Expression *
   461 Parser::addition()
   462 {
   463     Expression *left = multiplication();
   464     while (left) {
   465         if (!match(TOK_PLUS) && !match(TOK_MINUS))
   466             break;
   467         SourcePosition pos = scanner_.begin();
   468         TokenKind kind = scanner_.last();
   469         Expression *right = multiplication();
   470         if (!right)
   471             return NULL;
   472         left = new (pool_) BinaryExpression(pos, kind, left, right);
   473     }
   474     return left;
   475 }
   476 
   477 Expression *
   478 Parser::shift()
   479 {
   480     Expression *left = addition();
   481     while (left) {
   482         if (!match(TOK_SHL) && !match(TOK_SHR) && !match(TOK_USHR))
   483             break;
   484         SourcePosition pos = scanner_.begin();
   485         TokenKind kind = scanner_.last();
   486         Expression *right = addition();
   487         if (!right)
   488             return NULL;
   489         left = new (pool_) BinaryExpression(pos, kind, left, right);
   490     }
   491     return left;
   492 }
   493 
   494 Expression *
   495 Parser::bitand_()
   496 {
   497     Expression *left = shift();
   498     while (left) {
   499         if (!match(TOK_BITAND))
   500             break;
   501         SourcePosition pos = scanner_.begin();
   502         Expression *right = shift();
   503         if (!right)
   504             return NULL;
   505         left = new (pool_) BinaryExpression(pos, TOK_BITAND, left, right);
   506     }
   507     return left;
   508 }
   509 
   510 Expression *
   511 Parser::bitxor()
   512 {
   513     Expression *left = bitand_();
   514     while (left) {
   515         if (!match(TOK_BITXOR))
   516             break;
   517         SourcePosition pos = scanner_.begin();
   518         Expression *right = shift();
   519         if (!right)
   520             return NULL;
   521         left = new (pool_) BinaryExpression(pos, TOK_BITXOR, left, right);
   522     }
   523     return left;
   524 }
   525 
   526 Expression *
   527 Parser::bitor_()
   528 {
   529     Expression *left = bitxor();
   530     while (left) {
   531         if (!match(TOK_BITOR))
   532             break;
   533         SourcePosition pos = scanner_.begin();
   534         Expression *right = bitxor();
   535         if (!right)
   536             return NULL;
   537         left = new (pool_) BinaryExpression(pos, TOK_BITOR, left, right);
   538     }
   539     return left;
   540 }
   541 
   542 Expression *
   543 Parser::relational()
   544 {
   545     Expression *left = bitor_();
   546     while (left) {
   547         TokenKind kind = scanner_.peek();
   548         if (kind < TOK_LT || kind > TOK_GE)
   549             break;
   550         scanner_.scan();
   551         SourcePosition pos = scanner_.begin();
   552         Expression *right = shift();
   553         if (!right)
   554             return NULL;
   555         left = new (pool_) BinaryExpression(pos, kind, left, right);
   556     }
   557     return left;
   558 }
   559 
   560 Expression *
   561 Parser::equals()
   562 {
   563     Expression *left = relational();
   564     while (left) {
   565         if (!match(TOK_EQUALS) && !match(TOK_NOTEQUALS))
   566             break;
   567         TokenKind kind = scanner_.last();
   568         SourcePosition pos = scanner_.begin();
   569         Expression *right = relational();
   570         if (!right)
   571             return NULL;
   572         left = new (pool_) BinaryExpression(pos, kind, left, right);
   573     }
   574     return left;
   575 }
   576 
   577 Expression *
   578 Parser::and_()
   579 {
   580     Expression *left = equals();
   581     while (left) {
   582         if (!match(TOK_AND))
   583             break;
   584         SourcePosition pos = scanner_.begin();
   585         Expression *right = equals();
   586         if (!right)
   587             return NULL;
   588         left = new (pool_) BinaryExpression(pos, TOK_AND, left, right);
   589     }
   590     return left;
   591 }
   592 
   593 Expression *
   594 Parser::or_()
   595 {
   596     Expression *left = and_();
   597     while (left) {
   598         if (!match(TOK_OR))
   599             break;
   600         SourcePosition pos = scanner_.begin();
   601         Expression *right = and_();
   602         if (!right)
   603             return NULL;
   604         left = new (pool_) BinaryExpression(pos, TOK_OR, left, right);
   605     }
   606     return left;
   607 }
   608 
   609 Expression *
   610 Parser::ternary()
   611 {
   612     Expression *cond = or_();
   613     if (!cond)
   614         return NULL;
   615     
   616     if (!match(TOK_QMARK))
   617         return cond;
   618 
   619     Expression *left;
   620     SourcePosition pos = scanner_.begin();
   621     AutoAllowTags<false> disableTags(scanner_);
   622     {
   623         if ((left = expression()) == NULL)
   624             return NULL;
   625     }
   626 
   627     if (!expect(TOK_COLON))
   628         return NULL;
   629 
   630     Expression *right = expression();
   631     if (!right)
   632         return NULL;
   633 
   634     return new (pool_) TernaryExpression(pos, cond, left, right);
   635 }
   636 
   637 Expression *
   638 Parser::assignment()
   639 {
   640     Expression *left = ternary();
   641     while (left) {
   642         TokenKind token = scanner_.peek();
   643         if (token != TOK_ASSIGN && (token < TOK_ASSIGN_ADD || token > TOK_ASSIGN_SHL))
   644             break;
   645         scanner_.scan();
   646         SourcePosition pos = scanner_.begin();
   647         Expression *expr = ternary();
   648         if (!expr)
   649             return NULL;
   650 
   651         left = new (pool_) Assignment(pos, token, left, expr);
   652     }
   653     return left;
   654 }
   655 
   656 Expression *
   657 Parser::expression()
   658 {
   659     return assignment();
   660 }
   661 
   662 Statement *
   663 Parser::while_()
   664 {
   665     SourcePosition pos = scanner_.begin();
   666     scanner_.scan();
   667 
   668     if (!expect(TOK_LPAREN))
   669         return NULL;
   670 
   671     Expression *condition = expression();
   672     if (!condition)
   673         return NULL;
   674 
   675     if (!expect(TOK_RPAREN))
   676         return NULL;
   677 
   678     Statement *body;
   679     if (scanner_.peek() == TOK_LBRACE)
   680         body = block();
   681     else
   682         body = localStatement();
   683     if (!body)
   684         return NULL;
   685 
   686     return new (pool_) WhileStatement(pos, TOK_WHILE, condition, body);
   687 }
   688 
   689 Statement *
   690 Parser::do_()
   691 {
   692     SourcePosition pos = scanner_.begin();
   693     scanner_.scan();
   694 
   695     Statement *body;
   696     if (scanner_.peek() == TOK_LBRACE)
   697         body = block();
   698     else
   699         body = localStatement();
   700     if (!body)
   701         return NULL;
   702 
   703     if (!expect(TOK_WHILE))
   704         return NULL;
   705 
   706     if (!expect(TOK_LPAREN))
   707         return NULL;
   708     Expression *condition = expression();
   709     if (!condition)
   710         return NULL;
   711     if (!expect(TOK_RPAREN))
   712         return NULL;
   713 
   714     return new (pool_) WhileStatement(pos, TOK_DO, condition, body);
   715 }
   716 
   717 Statement *
   718 Parser::switch_()
   719 {
   720     expect(TOK_SWITCH);
   721 
   722     SourcePosition pos = scanner_.begin();
   723 
   724     if (!expect(TOK_LPAREN))
   725         return NULL;
   726     Expression *expr = expression();
   727     if (!expr)
   728         return NULL;
   729     if (!expect(TOK_RPAREN))
   730         return NULL;
   731 
   732     if (!expect(TOK_LBRACE))
   733         return NULL;
   734 
   735     SourcePosition defaultPos;
   736     PoolList<Case *> *cases = new (pool_) PoolList<Case *>();
   737 
   738     Statement *defaultCase = NULL;
   739 
   740     while (!peek(TOK_RBRACE)) {
   741         Expression *expr = NULL;
   742         ExpressionList *others = NULL;
   743         if (match(TOK_DEFAULT)) {
   744             if (defaultCase) {
   745                 cc_.reportError(scanner_.begin(), Message_OneDefaultPerSwitch);
   746                 return NULL;
   747             }
   748 
   749             defaultPos = scanner_.begin();
   750         } else {
   751             if (defaultCase) {
   752                 cc_.reportError(defaultPos, Message_DefaultMustBeLastCase);
   753                 return NULL;
   754             }
   755 
   756             if (!expect(TOK_CASE))
   757                 return NULL;
   758 
   759             // A limitation in the grammar is that |case <NAME>:| will be
   760             // detected as a label.
   761             AutoAllowTags<false> disableTags(scanner_);
   762 
   763             if ((expr = expression()) == NULL)
   764                 return NULL;
   765 
   766             if (peek(TOK_COMMA)) {
   767                 others = new (pool_) ExpressionList();
   768                 while (match(TOK_COMMA)) {
   769                     Expression *other = expression();
   770                     if (!other)
   771                         return NULL;
   772                     if (!others->append(other))
   773                         return NULL;
   774                 }
   775             }
   776         }
   777 
   778         if (!expect(TOK_COLON))
   779             return NULL;
   780 
   781         Statement *statement = localStatement();
   782         if (!statement)
   783             return NULL;
   784 
   785         if (!peek(TOK_CASE) &&
   786             !peek(TOK_DEFAULT) &&
   787             !peek(TOK_RBRACE))
   788         {
   789             cc_.reportError(scanner_.begin(), Message_SingleStatementPerCase);
   790             return NULL;
   791         }
   792 
   793         if (expr) {
   794             Case *caze = new (pool_) Case(expr, others, statement);
   795             if (!cases->append(caze))
   796                 return NULL;
   797         } else {
   798             defaultCase = statement;
   799         }
   800     }
   801 
   802     if (!expect(TOK_RBRACE))
   803         return NULL;
   804 
   805     return new (pool_) SwitchStatement(pos, expr, cases, defaultCase);
   806 }
   807 
   808 Statement *
   809 Parser::forStatement()
   810 {
   811     SourcePosition pos = scanner_.begin();
   812     scanner_.scan();
   813     if (!expect(TOK_LPAREN))
   814         return NULL;
   815 
   816     Statement *decl = NULL;
   817     if (!match(TOK_SEMICOLON)) {
   818         if (match(TOK_NEW)) {
   819             if ((decl = vars(Qual_None)) == NULL)
   820                 return NULL;
   821         } else {
   822             if ((decl = expressionStatement()) == NULL)
   823                 return NULL;
   824         }
   825         if (!expect(TOK_SEMICOLON))
   826             return NULL;
   827     }
   828 
   829     Expression *condition = NULL;
   830     if (!match(TOK_SEMICOLON)) {
   831         if ((condition = expression()) == NULL)
   832             return NULL;
   833         if (!expect(TOK_SEMICOLON))
   834             return NULL;
   835     }
   836 
   837     Statement *update = NULL;
   838     if (!match(TOK_RPAREN)) {
   839         if ((update = expressionStatement()) == NULL)
   840             return NULL;
   841         if (!expect(TOK_RPAREN))
   842             return NULL;
   843     }
   844 
   845     Statement *body;
   846     if (scanner_.peek() == TOK_LBRACE)
   847         body = block();
   848     else
   849         body = localStatement();
   850     if (!body)
   851         return NULL;
   852 
   853     return new (pool_) ForStatement(pos, decl, condition, update, body);
   854 }
   855 
   856 ExpressionList *
   857 Parser::dimensions()
   858 {
   859     ExpressionList *postDimensions = new (pool_) ExpressionList();
   860     while (match(TOK_LBRACKET)) {
   861         Expression *dim = NULL;
   862         if (!match(TOK_RBRACKET)) {
   863             if ((dim = expression()) == NULL)
   864                 return NULL;
   865             if (!expect(TOK_RBRACKET))
   866                 return NULL;
   867         }
   868 
   869         if (!postDimensions->append(dim))
   870             return NULL;
   871     }
   872     return postDimensions;
   873 }
   874 
   875 VariableDeclaration *
   876 Parser::var(TypeQualifiers qual)
   877 {
   878     Expression *type;
   879     Local<String> name(zone_);
   880     
   881     if (!typeAndName(&type, name.ref()))
   882         return NULL;
   883 
   884     SourcePosition pos = scanner_.begin();
   885     
   886     Expression *expr = NULL;
   887     if (match(TOK_ASSIGN)) {
   888         switch (scanner_.peek()) {
   889           // Note: we don't support [] literals yet.
   890           case TOK_LBRACE:
   891             if ((expr = arrayLiteral(scanner_.scan())) == NULL)
   892                 return NULL;
   893             break;
   894 
   895           default:
   896             if ((expr = expression()) == NULL)
   897                 return NULL;
   898             break;
   899         }
   900     }
   901 
   902     return new (pool_) VariableDeclaration(pos, qual, name, type, expr);
   903 }
   904 
   905 Statement *
   906 Parser::vars(TypeQualifiers qual)
   907 {
   908     VariableDeclaration *decl = var(qual);
   909     if (!decl)
   910         return NULL;
   911 
   912     if (!peek(TOK_COMMA))
   913         return decl;
   914 
   915     StatementList *statements = new (pool_) StatementList();
   916     if (!statements->append(decl))
   917         return NULL;
   918 
   919     while (match(TOK_COMMA)) {
   920         Statement *decl = var(qual);
   921         if (!decl)
   922             return NULL;
   923         if (!statements->append(decl))
   924             return NULL;
   925     }
   926 
   927     // Create a fake-o block with no scope to create a statement list.
   928     return new (pool_) BlockStatement(decl->pos(), statements);
   929 }
   930 
   931 Statement *
   932 Parser::returnStatement()
   933 {
   934     SourcePosition pos = scanner_.begin();
   935     if (!expect(TOK_RETURN))
   936         return NULL;
   937 
   938     Expression *expr = NULL;
   939     if (scanner_.peek() != TOK_SEMICOLON) {
   940         if ((expr = expression()) == NULL)
   941             return NULL;
   942 
   943         // We only care about non-void returns when determining whether a
   944         // tagless function is non-void.
   945         encounteredReturn_ = true;
   946     }
   947 
   948     return new (pool_) ReturnStatement(pos, expr);
   949 }
   950 
   951 Statement *
   952 Parser::expressionStatement()
   953 {
   954     Expression *left = assignment();
   955     if (!left)
   956         return NULL;
   957 
   958     return new (pool_) ExpressionStatement(left);
   959 }
   960 
   961 Statement *
   962 Parser::block()
   963 {
   964     if (!expect(TOK_LBRACE))
   965         return NULL;
   966 
   967     SourcePosition pos = scanner_.begin();
   968 
   969     StatementList *statements = new (pool_) StatementList();
   970     while (!match(TOK_RBRACE)) {
   971         Statement *statement = localStatement();
   972         if (statement == NULL)
   973             return NULL;
   974         if (!statements->append(statement))
   975             return NULL;
   976     }
   977 
   978     BlockStatement *block = new (pool_) BlockStatement(pos, statements);
   979     return block;
   980 }
   981 
   982 Statement *
   983 Parser::if_()
   984 {
   985     expect(TOK_IF);
   986     
   987     SourcePosition pos = scanner_.begin();
   988     if (!expect(TOK_LPAREN))
   989         return NULL;
   990 
   991     Expression *cond = expression();
   992     if (!cond)
   993         return NULL;
   994 
   995     if (!expect(TOK_RPAREN))
   996         return NULL;
   997 
   998     Statement *ifTrue = localStatement();
   999     if (!ifTrue)
  1000         return NULL;
  1001 
  1002     IfStatement *outer = new (pool_) IfStatement(pos, cond, ifTrue);
  1003 
  1004     IfStatement *last = outer;
  1005     while (match(TOK_ELSE)) {
  1006         if (!match(TOK_IF)) {
  1007             Statement *ifFalse = localStatement();
  1008             if (!ifFalse)
  1009                 return NULL;
  1010 
  1011             last->setIfFalse(ifFalse);
  1012             break;
  1013         }
  1014 
  1015         SourcePosition pos = scanner_.begin();
  1016         if (!expect(TOK_LPAREN))
  1017             return NULL;
  1018 
  1019         Expression *otherCond = expression();
  1020         if (!otherCond)
  1021             return NULL;
  1022 
  1023         if (!expect(TOK_RPAREN))
  1024             return NULL;
  1025 
  1026         Statement *otherIfTrue = localStatement();
  1027         if (!otherIfTrue)
  1028             return NULL;
  1029 
  1030         IfStatement *inner = new (pool_) IfStatement(pos, otherCond, otherIfTrue);
  1031         last->setIfFalse(inner);
  1032         last = inner;
  1033     }
  1034 
  1035     return outer;
  1036 }
  1037 
  1038 Statement *
  1039 Parser::localStatement()
  1040 {
  1041     Statement *statement = NULL;
  1042 
  1043     // Statements which must be followed by a semicolon will break out of the
  1044     // switch. If they may end in BlockStatements, they'll immediately return.
  1045 
  1046     TokenKind kind = scanner_.peek();
  1047     switch (kind) {
  1048       case TOK_CONST:
  1049       case TOK_NEW:
  1050       {
  1051         TypeQualifiers quals = (kind == TOK_CONST)
  1052                               ? Qual_Const
  1053                               : Qual_None;
  1054         scanner_.scan();
  1055         if ((statement = vars(quals)) == NULL)
  1056           return NULL;
  1057         break;
  1058       }
  1059 
  1060       case TOK_FOR:
  1061         return forStatement();
  1062 
  1063       case TOK_WHILE:
  1064         return while_();
  1065 
  1066       case TOK_BREAK:
  1067         scanner_.scan();
  1068         statement = new (pool_) BreakStatement(scanner_.begin());
  1069         break;
  1070 
  1071       case TOK_CONTINUE:
  1072         scanner_.scan();
  1073         statement = new (pool_) ContinueStatement(scanner_.begin());
  1074         break;
  1075 
  1076       case TOK_DO:
  1077         if ((statement = do_()) == NULL)
  1078             return NULL;
  1079         break;
  1080 
  1081       case TOK_RETURN:
  1082         if ((statement = returnStatement()) == NULL)
  1083           return NULL;
  1084         break;
  1085 
  1086       case TOK_ENUM:
  1087         if ((statement = enum_()) == NULL)
  1088           return NULL;
  1089         break;
  1090 
  1091       case TOK_STRUCT:
  1092         if ((statement = struct_()) == NULL)
  1093           return NULL;
  1094         break;
  1095 
  1096       case TOK_SWITCH:
  1097         return switch_();
  1098 
  1099       case TOK_IF:
  1100         return if_();
  1101 
  1102       case TOK_LBRACE:
  1103         return block();
  1104 
  1105       default:
  1106         break;
  1107     }
  1108 
  1109     if (!statement) {
  1110         if ((statement = expressionStatement()) == NULL)
  1111             return NULL;
  1112     }
  1113 
  1114     if (!expect(TOK_SEMICOLON))
  1115         return NULL;
  1116 
  1117     return statement;
  1118 }
  1119 
  1120 Statement *
  1121 Parser::struct_()
  1122 {
  1123     expect(TOK_STRUCT);
  1124 
  1125     Local<String> name(zone_, expectName());
  1126     if (!name)
  1127         return NULL;
  1128 
  1129     SourcePosition pos = scanner_.begin();
  1130 
  1131     if (!expect(TOK_LBRACE))
  1132         return NULL;
  1133 
  1134     FieldList *fields = new (pool_) FieldList();
  1135 
  1136     while (!peek(TOK_RBRACE)) {
  1137         Expression *type;
  1138         Local<String> fieldName(zone_);
  1139 
  1140         if (!fullTypeAndName(&type, fieldName.ref(), DeclField))
  1141             return NULL;
  1142 
  1143         SourcePosition pos = scanner_.begin();
  1144 
  1145         if (!matchSemicolon())
  1146             return NULL;
  1147 
  1148         StructureStatement::Field *field =
  1149             new (pool_) StructureStatement::Field(pos, type, fieldName);
  1150         if (!field || !fields->append(field))
  1151             return NULL;
  1152     }
  1153 
  1154     if (!expect(TOK_RBRACE))
  1155         return NULL;
  1156 
  1157     if (fields->empty()) {
  1158         cc_.reportError(pos, Message_StructCantBeEmpty);
  1159         return NULL;
  1160     }
  1161 
  1162     return new (pool_) StructureStatement(pos, name, fields);
  1163 }
  1164 
  1165 Statement *
  1166 Parser::enum_()
  1167 {
  1168     expect(TOK_ENUM);
  1169 
  1170     SourcePosition pos = scanner_.begin();
  1171 
  1172     Local<String> name(zone_);
  1173     if (matchName(name.ref()) && !name)
  1174         return NULL;
  1175 
  1176     EnumStatement::EntryList *entries = new (pool_) EnumStatement::EntryList();
  1177 
  1178     if (!expect(TOK_LBRACE))
  1179         return NULL;
  1180 
  1181     do {
  1182         if (scanner_.peek() == TOK_RBRACE)
  1183             break;
  1184 
  1185         Local<String> name(zone_, expectName());
  1186         if (!name)
  1187             return NULL;
  1188 
  1189         Expression *expr = NULL;
  1190         if (match(TOK_ASSIGN)) {
  1191             if ((expr = expression()) == NULL)
  1192                 return NULL;
  1193         }
  1194 
  1195         if (!entries->append(EnumStatement::Entry(name, expr)))
  1196             return NULL;
  1197     } while (match(TOK_COMMA));
  1198     if (!expect(TOK_RBRACE))
  1199         return NULL;
  1200 
  1201     return new (pool_) EnumStatement(pos, name, entries);
  1202 }
  1203 
  1204 template <class T>
  1205 static inline ScopedRoot<T> *
  1206 NewScopedRoot(PoolAllocator &pool, Handle<T> obj)
  1207 {
  1208     ScopedRoot<T> *root = pool.alloc<ScopedRoot<T> >();
  1209     new (root) ScopedRoot<T>(obj);
  1210     return root;
  1211 }
  1212 
  1213 ImportStatement *
  1214 Parser::import()
  1215 {
  1216     NameList *names = new (pool_) NameList();
  1217 
  1218     SourcePosition pos = scanner_.begin();
  1219 
  1220     Local<String> name(zone_, expectName());
  1221     if (!name || !names->append(NewScopedRoot<String>(pool_, name)))
  1222         return NULL;
  1223 
  1224     TokenKind suffix = TOK_NONE;
  1225     while (match(TOK_DOT)) {
  1226         if (match(TOK_STAR)) {
  1227             suffix = TOK_STAR;
  1228             break;
  1229         }
  1230 
  1231         if ((name = expectName()) == NULL)
  1232             return NULL;
  1233 
  1234         if (!names->append(NewScopedRoot<String>(pool_, name)))
  1235             return NULL;
  1236     }
  1237 
  1238     return new (pool_) ImportStatement(pos, names, suffix);
  1239 }
  1240 
  1241 Statement *
  1242 Parser::function(TokenKind token)
  1243 {
  1244     Expression *returnType;
  1245     Local<String> name(zone_);
  1246 
  1247     if (!fullTypeAndName(&returnType, name.ref(), DeclFun))
  1248         return NULL;
  1249 
  1250     SourcePosition pos = scanner_.begin();
  1251     ParameterList *parameters = new (pool_) ParameterList();
  1252 
  1253     if (!expect(TOK_LPAREN))
  1254         return NULL;
  1255 
  1256     bool variadic = false;
  1257 
  1258     if (!match(TOK_RPAREN)) {
  1259         while (true) {
  1260             bool isConst = match(TOK_CONST);
  1261             bool ref = match(TOK_AMPERSAND);
  1262 
  1263             SourcePosition begin = scanner_.begin();
  1264             Expression *type;
  1265             Local<String> name(zone_);
  1266             if (!fullTypeAndName(&type, name.ref(), DeclArg))
  1267                 return NULL;
  1268 
  1269             if (!name) {
  1270                 if (isConst || ref) {
  1271                     cc_.reportError(begin, Message_InvalidVarArgs);
  1272                     return NULL;
  1273                 }
  1274 
  1275                 if (token != TOK_NATIVE) {
  1276                     cc_.reportError(begin, Message_VarArgsOutsideNative);
  1277                     return NULL;
  1278                 }
  1279 
  1280                 variadic = true;
  1281                 break;
  1282             }
  1283 
  1284             Expression *expr = NULL;
  1285             if (match(TOK_ASSIGN)) {
  1286                 if ((expr = expression()) == NULL)
  1287                     return NULL;
  1288             }
  1289 
  1290             Parameter *parameter = new (pool_) Parameter(name, type, ref, isConst, expr, begin);
  1291             if (!parameters->append(parameter))
  1292                 return NULL;
  1293 
  1294             if (!match(TOK_COMMA))
  1295                 break;
  1296         }
  1297         if (!expect(TOK_RPAREN))
  1298             return NULL;
  1299     }
  1300 
  1301     encounteredReturn_ = false;
  1302 
  1303     Statement *body = NULL;
  1304     if (token != TOK_FORWARD && token != TOK_NATIVE && token != TOK_TYPEDEF) {
  1305         body = block();
  1306         if (!body)
  1307             return NULL;
  1308     }
  1309 
  1310     FunctionSignature signature(returnType, parameters, variadic);
  1311     if (token == TOK_TYPEDEF)
  1312         return new (pool_) FunctionTypeStatement(pos, name, signature);
  1313 
  1314     return new (pool_) FunctionStatement(pos, name, token, body, signature, encounteredReturn_);
  1315 }
  1316 
  1317 static inline bool
  1318 DeclareIntrinsicType(Zone *zone, Scope *scope, const char *name, Handle<Type> type)
  1319 {
  1320     Local<String> tag(zone, zone->makeSymbol(name));
  1321     if (!tag)
  1322         return false;
  1323 
  1324     assert(false);
  1325     return false; //return !!scope->declareType(tag, type);
  1326 }
  1327 
  1328 static inline bool
  1329 DeclareIntrinsicType(Zone *zone, Scope *scope, const char *name, PrimitiveType type)
  1330 {
  1331     Local<Type> primitive(zone, zone->types()->getPrimitive(type));
  1332     assert(primitive);
  1333     return DeclareIntrinsicType(zone, scope, name, primitive);
  1334 }
  1335 
  1336 bool
  1337 Parser::setupIntrinsics()
  1338 {
  1339 #if 0
  1340     if (!DeclareIntrinsicType(zone_, scope_, "Float", PrimitiveType_Float))
  1341         return false;
  1342     if (!DeclareIntrinsicType(zone_, scope_, "_", PrimitiveType_Int32))
  1343         return false;
  1344     if (!DeclareIntrinsicType(zone_, scope_, "bool", PrimitiveType_Bool))
  1345         return false;
  1346     if (!DeclareIntrinsicType(zone_, scope_, "String", PrimitiveType_Char))
  1347         return false;
  1348 
  1349     Local<Type> unchecked(zone_, zone_->types()->getUnchecked());
  1350     if (!DeclareIntrinsicType(zone_, scope_, "unchecked", unchecked))
  1351         return false;
  1352 
  1353     Local<Type> voidType(zone_, zone_->types()->getVoid());
  1354     if (!DeclareIntrinsicType(zone_, scope_, "void", voidType))
  1355         return false;
  1356 
  1357     Local<Type> intType(zone_, zone_->types()->getPrimitive(PrimitiveType_Int32));
  1358     if (!DeclareIntrinsicType(zone_, scope_, "int", intType))
  1359         return false;
  1360 #endif
  1361     return true;
  1362 }
  1363 
  1364 ParseTree *
  1365 Parser::parse()
  1366 {
  1367     StatementList *statements = new (pool_) StatementList();
  1368 
  1369     if (!setupIntrinsics())
  1370         return NULL;
  1371 
  1372     for (;;) {
  1373         Statement *statement = NULL;
  1374 
  1375         TokenKind kind = scanner_.peek();
  1376         switch (kind) {
  1377           case TOK_ERROR:
  1378             return NULL;
  1379 
  1380           case TOK_EOF:
  1381             break;
  1382 
  1383           case TOK_NAME:
  1384             if ((statement = function(TOK_NAME)) == NULL)
  1385                 return NULL;
  1386             break;
  1387 
  1388           case TOK_IMPORT:
  1389           {
  1390             scanner_.scan();
  1391             if (statements->length())
  1392                 cc_.reportError(scanner_.begin(), Message_ImportsMustBeFirst);
  1393             ImportStatement *stmt = import();
  1394             if (!stmt)
  1395                 return NULL;
  1396             if (!imports_->append(stmt))
  1397                 return NULL;
  1398             if (!matchSemicolon())
  1399                 return NULL;
  1400             continue;
  1401           }
  1402 
  1403           case TOK_FORWARD:
  1404           case TOK_PUBLIC:
  1405           case TOK_NATIVE:
  1406           case TOK_STOCK:
  1407           case TOK_LABEL:
  1408           case TOK_TYPEDEF:
  1409           {
  1410             if (kind != TOK_LABEL)
  1411                 scanner_.scan();
  1412             if (kind == TOK_PUBLIC && peek(TOK_CONST)) {
  1413                 TypeQualifiers quals = TypeQualifiers(Qual_Public | Qual_Const);
  1414                 if ((statement = var(quals)) == NULL)
  1415                     return NULL;
  1416                 if (!matchSemicolon())
  1417                     return NULL;
  1418             } else {
  1419                 if ((statement = function(kind)) == NULL)
  1420                     return NULL;
  1421                 if ((kind == TOK_FORWARD || kind == TOK_NATIVE || kind == TOK_TYPEDEF) && !matchSemicolon())
  1422                     return NULL;
  1423             }
  1424             break;
  1425           }
  1426 
  1427           case TOK_CONST:
  1428           case TOK_NEW:
  1429           {
  1430             TypeQualifiers quals = (kind == TOK_NEW) ? Qual_Const : Qual_None;
  1431             // Note: at global scope, we don't accept comma-delimited
  1432             // declarations yet, to see if we can get away with forcing
  1433             // a nicer top-level style.
  1434             scanner_.scan();
  1435             if ((statement = var(quals)) == NULL)
  1436                 return NULL;
  1437             if (!matchSemicolon())
  1438                 return NULL;
  1439             break;
  1440           }
  1441 
  1442           case TOK_ENUM:
  1443             if ((statement = enum_()) == NULL)
  1444                 return NULL;
  1445             if (!matchSemicolon())
  1446                 return NULL;
  1447             break;
  1448 
  1449           case TOK_STRUCT:
  1450             if ((statement = struct_()) == NULL)
  1451                 return NULL;
  1452             if (!matchSemicolon())
  1453                 return NULL;
  1454             break;
  1455 
  1456           default:
  1457             cc_.reportError(scanner_.begin(), Message_ExpectedGlobal);
  1458             return NULL;
  1459         }
  1460 
  1461         if (!statement)
  1462             break;
  1463 
  1464         if (!statements->append(statement))
  1465             return NULL;
  1466     }
  1467 
  1468     return new (pool_) ParseTree(statements, unbound_, imports_);
  1469 }
  1470 
  1471 class AstPrinter : public AstVisitor
  1472 {
  1473     FILE *fp_;
  1474     size_t level_;
  1475 
  1476   private:
  1477     void prefix() {
  1478         for (size_t i = 0; i < level_; i++)
  1479             fprintf(fp_, "  ");
  1480     }
  1481     void indent() {
  1482         level_++;
  1483     }
  1484     void unindent() {
  1485         level_--;
  1486     }
  1487 
  1488   public:
  1489     AstPrinter(FILE *fp)
  1490       : fp_(fp),
  1491         level_(0)
  1492     {
  1493     }
  1494 
  1495     void dump(const FunctionSignature &sig) {
  1496         prefix();
  1497         fprintf(fp_, "()\n");
  1498     }
  1499     void visitNameProxy(NameProxy *name) {
  1500         prefix();
  1501         fprintf(fp_, "[ NameProxy (%s)\n", name->name()->chars());
  1502     }
  1503     void visitCallExpression(CallExpression *node) {
  1504         prefix();
  1505         fprintf(fp_, "[ CallExpression\n");
  1506         indent();
  1507         node->callee()->accept(this);
  1508         for (size_t i = 0; i < node->arguments()->length(); i++)
  1509             node->arguments()->at(i)->accept(this);
  1510         unindent();
  1511     }
  1512     void visitFunctionTypeStatement(FunctionTypeStatement *node) {
  1513         prefix();
  1514         fprintf(fp_, "[ FunctionTypeStatement (%s)\n", node->name()->chars());
  1515         indent();
  1516         dump(node->signature());
  1517         unindent();
  1518     }
  1519     void visitFunctionStatement(FunctionStatement *node) {
  1520         prefix();
  1521         fprintf(fp_, "[ FunctionStatement (%s)\n", node->name()->chars());
  1522         indent();
  1523         dump(node->signature());
  1524         if (node->body())
  1525             node->body()->accept(this);
  1526         unindent();
  1527     }
  1528     void visitExpressionStatement(ExpressionStatement *node) {
  1529         prefix();
  1530         fprintf(fp_, "[ ExpressionStatement\n");
  1531         indent();
  1532         node->expression()->accept(this);
  1533         unindent();
  1534     }
  1535     void visitAssignment(Assignment *node) {
  1536         prefix();
  1537         fprintf(fp_, "[ Assignment\n");
  1538         indent();
  1539         node->lvalue()->accept(this);
  1540         node->expression()->accept(this);
  1541         unindent();
  1542     }
  1543     void visitTernaryExpression(TernaryExpression *node) {
  1544         prefix();
  1545         fprintf(fp_, "[ TernaryExpression\n");
  1546         indent();
  1547         node->condition()->accept(this);
  1548         node->left()->accept(this);
  1549         node->right()->accept(this);
  1550         unindent();
  1551     }
  1552     void visitBinaryExpression(BinaryExpression *node) {
  1553         prefix();
  1554         fprintf(fp_, "[ BinaryExpression (%s)\n", TokenNames[node->token()]);
  1555         indent();
  1556         node->left()->accept(this);
  1557         node->right()->accept(this);
  1558         unindent();
  1559     }
  1560     void visitUnaryExpression(UnaryExpression *node) {
  1561         prefix();
  1562         fprintf(fp_, "[ UnaryExpression (%s)\n", TokenNames[node->token()]);
  1563         indent();
  1564         node->expression()->accept(this);
  1565         unindent();
  1566     }
  1567     void visitReturnStatement(ReturnStatement *node) {
  1568         prefix();
  1569         fprintf(fp_, "[ ReturnStatement\n");
  1570         indent();
  1571         if (node->expression())
  1572             node->expression()->accept(this);
  1573         unindent();
  1574     }
  1575     void visitForStatement(ForStatement *node) {
  1576         prefix();
  1577         fprintf(fp_, "[ ForStatement\n");
  1578         indent();
  1579         if (node->initialization())
  1580             node->initialization()->accept(this);
  1581         if (node->condition())
  1582             node->condition()->accept(this);
  1583         if (node->update())
  1584             node->update()->accept(this);
  1585         node->body()->accept(this);
  1586         unindent();
  1587     }
  1588     void visitBlockStatement(BlockStatement *node) {
  1589         prefix();
  1590         fprintf(fp_, "[ BlockStatement\n");
  1591         indent();
  1592         for (size_t i = 0; i < node->statements()->length(); i++)
  1593             node->statements()->at(i)->accept(this);
  1594         unindent();
  1595     }
  1596     void visitVariableDeclaration(VariableDeclaration *node) {
  1597         prefix();
  1598         fprintf(fp_, "[ VariableDeclaration (%s)\n", node->name()->chars());
  1599         indent();
  1600         if (node->initialization())
  1601             node->initialization()->accept(this);
  1602         unindent();
  1603     }
  1604     void visitIntegerLiteral(IntegerLiteral *node) {
  1605         prefix();
  1606         fprintf(fp_, "[ IntegerLiteral (%d)\n", node->value());
  1607     }
  1608     void visitBooleanLiteral(BooleanLiteral *node) {
  1609         prefix();
  1610         fprintf(fp_, "[ BooleanLiteral (%s)\n",
  1611                 (node->token() == TOK_TRUE) ? "true" : "false");
  1612     }
  1613     void visitFloatLiteral(FloatLiteral *node) {
  1614         prefix();
  1615         fprintf(fp_, "[ FloatLiteral (%f)\n", node->value());
  1616     }
  1617     void visitIfStatement(IfStatement *node) {
  1618         prefix();
  1619         fprintf(fp_, "[ IfStatement\n");
  1620         indent();
  1621         node->ifTrue()->accept(this);
  1622         if (node->ifFalse())
  1623             node->ifFalse()->accept(this);
  1624         unindent();
  1625     }
  1626     void visitIndexExpression(IndexExpression *node) {
  1627         prefix();
  1628         fprintf(fp_, "[ IndexExpression\n");
  1629         indent();
  1630         node->left()->accept(this);
  1631         node->right()->accept(this);
  1632         unindent();
  1633     }
  1634     void visitEnumStatement(EnumStatement *node) {
  1635         prefix();
  1636         fprintf(fp_, "[ EnumStatement (%s)\n", node->name() ? node->name()->chars() : "<anonymous>");
  1637         indent();
  1638         for (size_t i = 0; i < node->entries()->length(); i++) {
  1639             prefix();
  1640             fprintf(fp_, "%s =\n", node->entries()->at(i).name->chars());
  1641             if (node->entries()->at(i).expr) {
  1642                 indent();
  1643                 node->entries()->at(i).expr->accept(this);
  1644                 unindent();
  1645             }
  1646         }
  1647         unindent();
  1648     }
  1649     void visitWhileStatement(WhileStatement *node) {
  1650         prefix();
  1651         fprintf(fp_, "[ WhileStatement (%s)\n",
  1652                 (node->token() == TOK_DO) ? "do" : "while");
  1653         indent();
  1654         if (node->token() == TOK_DO) {
  1655             node->condition()->accept(this);
  1656             node->body()->accept(this);
  1657         } else {
  1658             node->condition()->accept(this);
  1659             node->body()->accept(this);
  1660         }
  1661         unindent();
  1662     }
  1663     void visitBreakStatement(BreakStatement *node) {
  1664         prefix();
  1665         fprintf(fp_, "[ BreakStatement\n");
  1666     }
  1667     void visitContinueStatement(ContinueStatement *node) {
  1668         prefix();
  1669         fprintf(fp_, "[ ContinueStatement\n");
  1670     }
  1671     void visitStringLiteral(StringLiteral *node) {
  1672         prefix();
  1673         fprintf(fp_, "[ StringLiteral\n");
  1674     }
  1675     void visitIncDecExpression(IncDecExpression *node) {
  1676         prefix();
  1677         fprintf(fp_, "[ IncDecExpression (postfix=%d)\n", node->postfix());
  1678         indent();
  1679         node->expression()->accept(this);
  1680         unindent();
  1681     }
  1682     void visitStructureStatement(StructureStatement *node) {
  1683         prefix();
  1684         fprintf(fp_, "[ StructureStatement %s\n", node->name()->chars());
  1685     }
  1686     void visitFieldExpression(FieldExpression *node) {
  1687         prefix();
  1688         fprintf(fp_, "[ FieldExpression %s\n", node->field()->chars());
  1689         indent();
  1690         node->left()->accept(this);
  1691         unindent();
  1692     }
  1693     void visitSwitchStatement(SwitchStatement *node) {
  1694         prefix();
  1695         fprintf(fp_, "[ SwitchStatement\n");
  1696         indent();
  1697         node->expression()->accept(this);
  1698         for (size_t i = 0; i < node->cases()->length(); i++) {
  1699             Case *c = node->cases()->at(i);
  1700             c->expression()->accept(this);
  1701             if (c->others()) {
  1702                 for (size_t j = 0; j < c->others()->length(); j++)
  1703                     c->others()->at(j)->accept(this);
  1704             }
  1705             indent();
  1706             c->statement()->accept(this);
  1707             unindent();
  1708         }
  1709         if (node->defaultCase())
  1710             node->defaultCase()->accept(this);
  1711         unindent();
  1712     }
  1713     void visitArrayLiteral(ArrayLiteral *node) {
  1714         prefix();
  1715         fprintf(fp_, "[ ArrayLiteral\n");
  1716         indent();
  1717         for (size_t i = 0; i < node->expressions()->length(); i++) {
  1718             Expression *expr = node->expressions()->at(i);
  1719             indent();
  1720             expr->accept(this);
  1721             unindent();
  1722         }
  1723     }
  1724     void visitImportStatement(ImportStatement *node) {
  1725         prefix();
  1726         fprintf(fp_, "[ ImportStatement\n");
  1727     }
  1728 };
  1729 
  1730 void
  1731 ParseTree::dump(FILE *fp)
  1732 {
  1733     AstPrinter printer(fp);
  1734 
  1735     for (size_t i = 0; i < statements_->length(); i++)
  1736         statements_->at(i)->accept(&printer);
  1737 }