Massive SemA/BC refactoring to support better type systems. Read on for more.
The previous compiler had a simple pipeline, best outlined as:
(1) Parsing and name binding -> AST
(2) Storage allocation -> AST
(3) Semantic Analysis (SemA) -> AST
(4) Bytecode Compilation (BC) -> Code
This pipeline, unfortunately, had two problems. First, name binding cannot be performed in one pass if we wish to have multi-file support in the form of modules. Name binding must be two-pass.
Second, because SemA was only capable of annotating AST nodes with a single type, most coercion work had to be duplicated in the BC phase, as coercion was not explicit in the SemA output. This made introducing new type rules extremely difficult, and all but precluded concepts like operator overloading (or overloading at all).
This patch rewrites most of Keima's backend. Of interest are the following changes:
(1) CompileContext has been refactored around future multi-file support.
(2) Parsing no longer performs any name binding.
(3) The grammar for type-and-name has been changed from:
(Label | Identifier)? Identifier
to:
Type? Identifier
where:
Type ::= OldType | NewType
OldType ::= Label
NewType ::= Identifier (. NewType)*
Although we do not implement the full production for Type yet, this
distinction is important. A type identifier is now affixed to the AST
as an Expression (right now, always a NameProxy), so it can fully
participate in name binding.
(4) Immediately after parsing, the AST goes through a NamePopulation phase.
This phase creates Scope objects for every scope which declares a name,
and if those names are declaring types or functions, a Symbol is created
and entered into the scope. Symbols entirely replace the old BoundName
classes.
(5) After NamePopulation comes NameBinding. This phase performs three steps:
(a) Links Scope objects together, to form a tree.
(b) Binds any free name to an existing name in the scope chain.
(c) Creates and registers any Symbols for names which have not yet been
added to the scope (for example, local variables).
In accordance, all "allocation" and "binding" concepts have been removed
from Scopes, which are now lightweight container classes.
(6) SemA now generates bytecode for each statement in the AST. Expression
compilation is performed via a separate mechanism called HIR (High-level
Intermediate Representation). To analyze an expression, SemA will walk
the AST, and produce a HIR object for each node. HIR is typed, and SemA
may insert coercion nodes, or even expand an AST node into multiple HIR
nodes. The result of evaluating an expression in SemA is therefore the
the root of a HIR tree, which SemA then sends to the HIRTranslator, which
performs bytecode generation.
As before, SemA still performs full semantic analysis. However, not it
also produces each function's bytecode.
This split allows us to decompose potentially complex semantics into
a more fine-grained, AST-like structure, which can have very simple
code-generation logic. For example, (1.0 + 5) might look like:
HAdd(HFloat(1.0), HConvert(HInteger(5), <Float>))
As part of this decomposition, many opcodes have been removed. Rather than
using typed jumps, we now expand jumps into longer tests. For example:
jge.f <label>
becomes:
ge.f
cvt.f2b
jt <label>
This simplifies the pipeline, and JITs should be able to melt the added
work away.
HIR is not used at a statement level, and HIR does not have any concept
of control-flow.
(7) The old BytecodeCompiler has been removed, as its work is now split
between SemA and HIRTranslator.
In addition, some minor refactorings have taken place:
(1) Opcodes.tbl no longer hardcodes numbers (aaah).
(2) Type has been split into smaller, typed structs.
(3) BytecodeEmitter no longer relies on Pools or RootScopes.
(4) SemA is now responsible for variable/storage allocation.
(5) Native table are now global, rather than per-module. As such, native
declarations now result in a Native object (which still requires a
CALLNATIVE opcode), which is an index into the global table. Natives
must be bound globally. This is in preparation for module support.
(6) Publics are no longer tracked, but rather registered via a global
callback upon module load. This callback can be set by embedders. This
is in preparation for multi-file support.
(7) The BytecodeEmitter now performs Symbol allocations itself, and it
also generates Code objects itself, which removes a good deal of
complexity.
Finally, a test harness has been added. This harness is a python script which finds *.test files, recursively, in the tests folder. For each file it runs the corresponding .sp. The contents of the .test file must match stdout+stderr.
1 /* vim: set ts=4 sw=4 tw=99 et:
3 * Copyright (C) 2012 David Anderson
5 * This file is part of SourcePawn.
7 * SourcePawn is free software: you can redistribute it and/or modify it under
8 * the terms of the GNU General Public License as published by the Free
9 * Software Foundation, either version 3 of the License, or (at your option)
12 * SourcePawn is distributed in the hope that it will be useful, but WITHOUT ANY
13 * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
14 * FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
16 * You should have received a copy of the GNU General Public License along with
17 * SourcePawn. If not, see http://www.gnu.org/licenses/.
21 #include "../FixedArray.h"
23 #include "../Strings.h"
24 #include "../TypeManager.h"
26 #include "../Heap-inl.h"
30 Parser::Parser(Zone *zone, CompileContext &cc, const char *stream, size_t length)
34 scanner_(cc, stream, length),
37 unbound_ = new (pool_) PoolList<NameProxy *>();
38 imports_ = new (pool_) PoolList<ImportStatement *>();
42 Parser::peek(TokenKind kind)
44 return scanner_.peek() == kind;
48 Parser::match(TokenKind kind)
57 Parser::matchSemicolon()
59 return expect(TOK_SEMICOLON);
63 Parser::expect(TokenKind kind)
65 TokenKind got = scanner_.scan();
68 cc_.reportError(scanner_.begin(), Message_WrongToken, TokenNames[kind], TokenNames[got]);
73 Parser::matchName(String **name, TokenKind kind)
75 assert(kind == TOK_NAME || kind == TOK_LABEL);
79 *name = zone_->makeSymbol(scanner_.literal(), scanner_.literal_length());
86 if (!expect(TOK_NAME))
88 return zone_->makeSymbol(scanner_.literal(), scanner_.literal_length());
94 TokenKind kind = scanner_.peek();
96 case TOK_FLOAT_LITERAL:
99 double value = Scanner::ParseDouble(scanner_.literal());
100 return new (pool_) FloatLiteral(scanner_.begin(), value);
103 case TOK_HEX_LITERAL:
106 int value = strtol(scanner_.literal(), NULL, 16);
107 return new (pool_) IntegerLiteral(scanner_.begin(), value);
110 case TOK_INTEGER_LITERAL:
113 int value = atoi(scanner_.literal());
114 return new (pool_) IntegerLiteral(scanner_.begin(), value);
120 return new (pool_) BooleanLiteral(scanner_.begin(), kind);
122 case TOK_STRING_LITERAL:
125 Local<ByteArray> array(zone_, ByteArray::New(zone_, scanner_.literal_length() + 1));
128 memcpy(array->raw(), scanner_.literal(), scanner_.literal_length() + 1);
129 return new (pool_) StringLiteral(scanner_.begin(), array);
134 if (kind != TOK_ERROR)
135 cc_.reportError(scanner_.begin(), Message_ExpectedExpression, TokenNames[kind]);
142 Parser::arrayLiteral(TokenKind kind)
144 assert(kind == TOK_LBRACE || kind == TOK_LBRACKET);
146 TokenKind close = (kind == TOK_LBRACE)
150 SourcePosition pos = scanner_.begin();
152 ExpressionList *list = new (pool_) ExpressionList();
157 Expression *expr = NULL;
158 if (kind == TOK_LBRACE) {
159 if (peek(TOK_LBRACE))
160 expr = arrayLiteral(TOK_LBRACE);
169 if (!list->append(expr))
172 if (!match(TOK_COMMA)) {
179 return new (pool_) ArrayLiteral(pos, kind, list);
185 switch (scanner_.peek()) {
189 Expression *expr = expression();
192 if (!expect(TOK_RPAREN))
199 Local<String> name(zone_, expectName());
202 return new (pool_) NameProxy(scanner_.begin(), name);
210 // Parses only old-style declarations.
212 Parser::typeAndName(Expression **type, String **namep)
214 if (match(TOK_LABEL)) {
215 // Old-style declaration:
217 Local<String> name(zone_, zone_->makeSymbol(scanner_.literal(), scanner_.literal_length()));
221 *type = new (pool_) NameProxy(scanner_.begin(), name);
222 *namep = expectName();
227 *namep = expectName();
232 // Parses both old and new style declarations.
234 Parser::fullTypeAndName(Expression **type, String **namep, DeclKind kind)
237 // NewDecl ::= Type Ident
238 // OldDecl ::= Label? (Ident | "...")
240 // Label ::= Ident ":"
242 if (match(TOK_LABEL)) {
243 // We something of the form:
246 Local<String> name(zone_, zone_->makeSymbol(scanner_.literal(), scanner_.literal_length()));
250 *type = new (pool_) NameProxy(scanner_.begin(), name);
251 if (kind == DeclArg && match(TOK_ELLIPSES)) {
254 if ((*namep = expectName()) == NULL)
260 Local<String> name(zone_, expectName());
264 if (peek(TOK_NAME) || (kind == DeclArg && peek(TOK_ELLIPSES))) {
265 // We got something of the form:
267 *type = new (pool_) NameProxy(scanner_.begin(), name);
268 if (peek(TOK_NAME)) {
269 if ((*namep = expectName()) == NULL)
277 // We got something of the form:
285 Parser::call(Expression *callee)
287 ExpressionList *arguments = new (pool_) ExpressionList();
289 SourcePosition pos = scanner_.begin();
292 if (!match(TOK_RPAREN)) {
294 Expression *expr = expression();
298 if (!arguments->append(expr))
301 if (!match(TOK_COMMA))
305 if (!expect(TOK_RPAREN))
309 return new (pool_) CallExpression(pos, callee, arguments);
313 Parser::index(Expression *left)
315 expect(TOK_LBRACKET);
317 SourcePosition pos = scanner_.begin();
318 Expression *expr = expression();
322 if (!expect(TOK_RBRACKET))
325 return new (pool_) IndexExpression(pos, left, expr);
329 Parser::field(Expression *left)
333 SourcePosition pos = scanner_.begin();
334 Local<String> name(zone_, expectName());
338 return new (pool_) FieldExpression(pos, left, name);
344 Expression *expr = prefix();
349 switch (scanner_.peek()) {
351 if ((expr = call(expr)) == NULL)
356 if ((expr = index(expr)) == NULL)
361 if ((expr = field(expr)) == NULL)
374 TokenKind token = scanner_.peek();
376 SourcePosition pos = scanner_.begin();
382 Expression *expr = unary();
385 return new (pool_) IncDecExpression(pos, token, expr, false);
393 Expression *expr = unary();
396 return new (pool_) UnaryExpression(pos, token, expr);
402 if (!expect(TOK_LPAREN))
404 Expression *expr = unary();
407 if (!expect(TOK_RPAREN))
409 return new (pool_) UnaryExpression(pos, token, expr);
415 Local<String> tag(zone_, zone_->makeSymbol(scanner_.literal(), scanner_.literal_length()));
419 NameProxy *proxy = new (pool_) NameProxy(scanner_.begin(), tag);
421 Expression *expr = unary();
425 return new (pool_) UnaryExpression(pos, TOK_LABEL, expr, proxy);
432 Expression *expr = primary();
434 token = scanner_.peek();
435 if (token == TOK_INCREMENT || token == TOK_DECREMENT) {
437 return new (pool_) IncDecExpression(pos, token, expr, true);
444 Parser::multiplication()
446 Expression *left = unary();
448 if (!match(TOK_SLASH) && !match(TOK_STAR) && !match(TOK_PERCENT))
450 SourcePosition pos = scanner_.begin();
451 TokenKind kind = scanner_.last();
452 Expression *right = unary();
455 left = new (pool_) BinaryExpression(pos, kind, left, right);
463 Expression *left = multiplication();
465 if (!match(TOK_PLUS) && !match(TOK_MINUS))
467 SourcePosition pos = scanner_.begin();
468 TokenKind kind = scanner_.last();
469 Expression *right = multiplication();
472 left = new (pool_) BinaryExpression(pos, kind, left, right);
480 Expression *left = addition();
482 if (!match(TOK_SHL) && !match(TOK_SHR) && !match(TOK_USHR))
484 SourcePosition pos = scanner_.begin();
485 TokenKind kind = scanner_.last();
486 Expression *right = addition();
489 left = new (pool_) BinaryExpression(pos, kind, left, right);
497 Expression *left = shift();
499 if (!match(TOK_BITAND))
501 SourcePosition pos = scanner_.begin();
502 Expression *right = shift();
505 left = new (pool_) BinaryExpression(pos, TOK_BITAND, left, right);
513 Expression *left = bitand_();
515 if (!match(TOK_BITXOR))
517 SourcePosition pos = scanner_.begin();
518 Expression *right = shift();
521 left = new (pool_) BinaryExpression(pos, TOK_BITXOR, left, right);
529 Expression *left = bitxor();
531 if (!match(TOK_BITOR))
533 SourcePosition pos = scanner_.begin();
534 Expression *right = bitxor();
537 left = new (pool_) BinaryExpression(pos, TOK_BITOR, left, right);
545 Expression *left = bitor_();
547 TokenKind kind = scanner_.peek();
548 if (kind < TOK_LT || kind > TOK_GE)
551 SourcePosition pos = scanner_.begin();
552 Expression *right = shift();
555 left = new (pool_) BinaryExpression(pos, kind, left, right);
563 Expression *left = relational();
565 if (!match(TOK_EQUALS) && !match(TOK_NOTEQUALS))
567 TokenKind kind = scanner_.last();
568 SourcePosition pos = scanner_.begin();
569 Expression *right = relational();
572 left = new (pool_) BinaryExpression(pos, kind, left, right);
580 Expression *left = equals();
584 SourcePosition pos = scanner_.begin();
585 Expression *right = equals();
588 left = new (pool_) BinaryExpression(pos, TOK_AND, left, right);
596 Expression *left = and_();
600 SourcePosition pos = scanner_.begin();
601 Expression *right = and_();
604 left = new (pool_) BinaryExpression(pos, TOK_OR, left, right);
612 Expression *cond = or_();
616 if (!match(TOK_QMARK))
620 SourcePosition pos = scanner_.begin();
621 AutoAllowTags<false> disableTags(scanner_);
623 if ((left = expression()) == NULL)
627 if (!expect(TOK_COLON))
630 Expression *right = expression();
634 return new (pool_) TernaryExpression(pos, cond, left, right);
640 Expression *left = ternary();
642 TokenKind token = scanner_.peek();
643 if (token != TOK_ASSIGN && (token < TOK_ASSIGN_ADD || token > TOK_ASSIGN_SHL))
646 SourcePosition pos = scanner_.begin();
647 Expression *expr = ternary();
651 left = new (pool_) Assignment(pos, token, left, expr);
665 SourcePosition pos = scanner_.begin();
668 if (!expect(TOK_LPAREN))
671 Expression *condition = expression();
675 if (!expect(TOK_RPAREN))
679 if (scanner_.peek() == TOK_LBRACE)
682 body = localStatement();
686 return new (pool_) WhileStatement(pos, TOK_WHILE, condition, body);
692 SourcePosition pos = scanner_.begin();
696 if (scanner_.peek() == TOK_LBRACE)
699 body = localStatement();
703 if (!expect(TOK_WHILE))
706 if (!expect(TOK_LPAREN))
708 Expression *condition = expression();
711 if (!expect(TOK_RPAREN))
714 return new (pool_) WhileStatement(pos, TOK_DO, condition, body);
722 SourcePosition pos = scanner_.begin();
724 if (!expect(TOK_LPAREN))
726 Expression *expr = expression();
729 if (!expect(TOK_RPAREN))
732 if (!expect(TOK_LBRACE))
735 SourcePosition defaultPos;
736 PoolList<Case *> *cases = new (pool_) PoolList<Case *>();
738 Statement *defaultCase = NULL;
740 while (!peek(TOK_RBRACE)) {
741 Expression *expr = NULL;
742 ExpressionList *others = NULL;
743 if (match(TOK_DEFAULT)) {
745 cc_.reportError(scanner_.begin(), Message_OneDefaultPerSwitch);
749 defaultPos = scanner_.begin();
752 cc_.reportError(defaultPos, Message_DefaultMustBeLastCase);
756 if (!expect(TOK_CASE))
759 // A limitation in the grammar is that |case <NAME>:| will be
760 // detected as a label.
761 AutoAllowTags<false> disableTags(scanner_);
763 if ((expr = expression()) == NULL)
766 if (peek(TOK_COMMA)) {
767 others = new (pool_) ExpressionList();
768 while (match(TOK_COMMA)) {
769 Expression *other = expression();
772 if (!others->append(other))
778 if (!expect(TOK_COLON))
781 Statement *statement = localStatement();
785 if (!peek(TOK_CASE) &&
786 !peek(TOK_DEFAULT) &&
789 cc_.reportError(scanner_.begin(), Message_SingleStatementPerCase);
794 Case *caze = new (pool_) Case(expr, others, statement);
795 if (!cases->append(caze))
798 defaultCase = statement;
802 if (!expect(TOK_RBRACE))
805 return new (pool_) SwitchStatement(pos, expr, cases, defaultCase);
809 Parser::forStatement()
811 SourcePosition pos = scanner_.begin();
813 if (!expect(TOK_LPAREN))
816 Statement *decl = NULL;
817 if (!match(TOK_SEMICOLON)) {
818 if (match(TOK_NEW)) {
819 if ((decl = vars(Qual_None)) == NULL)
822 if ((decl = expressionStatement()) == NULL)
825 if (!expect(TOK_SEMICOLON))
829 Expression *condition = NULL;
830 if (!match(TOK_SEMICOLON)) {
831 if ((condition = expression()) == NULL)
833 if (!expect(TOK_SEMICOLON))
837 Statement *update = NULL;
838 if (!match(TOK_RPAREN)) {
839 if ((update = expressionStatement()) == NULL)
841 if (!expect(TOK_RPAREN))
846 if (scanner_.peek() == TOK_LBRACE)
849 body = localStatement();
853 return new (pool_) ForStatement(pos, decl, condition, update, body);
859 ExpressionList *postDimensions = new (pool_) ExpressionList();
860 while (match(TOK_LBRACKET)) {
861 Expression *dim = NULL;
862 if (!match(TOK_RBRACKET)) {
863 if ((dim = expression()) == NULL)
865 if (!expect(TOK_RBRACKET))
869 if (!postDimensions->append(dim))
872 return postDimensions;
875 VariableDeclaration *
876 Parser::var(TypeQualifiers qual)
879 Local<String> name(zone_);
881 if (!typeAndName(&type, name.ref()))
884 SourcePosition pos = scanner_.begin();
886 Expression *expr = NULL;
887 if (match(TOK_ASSIGN)) {
888 switch (scanner_.peek()) {
889 // Note: we don't support [] literals yet.
891 if ((expr = arrayLiteral(scanner_.scan())) == NULL)
896 if ((expr = expression()) == NULL)
902 return new (pool_) VariableDeclaration(pos, qual, name, type, expr);
906 Parser::vars(TypeQualifiers qual)
908 VariableDeclaration *decl = var(qual);
912 if (!peek(TOK_COMMA))
915 StatementList *statements = new (pool_) StatementList();
916 if (!statements->append(decl))
919 while (match(TOK_COMMA)) {
920 Statement *decl = var(qual);
923 if (!statements->append(decl))
927 // Create a fake-o block with no scope to create a statement list.
928 return new (pool_) BlockStatement(decl->pos(), statements);
932 Parser::returnStatement()
934 SourcePosition pos = scanner_.begin();
935 if (!expect(TOK_RETURN))
938 Expression *expr = NULL;
939 if (scanner_.peek() != TOK_SEMICOLON) {
940 if ((expr = expression()) == NULL)
943 // We only care about non-void returns when determining whether a
944 // tagless function is non-void.
945 encounteredReturn_ = true;
948 return new (pool_) ReturnStatement(pos, expr);
952 Parser::expressionStatement()
954 Expression *left = assignment();
958 return new (pool_) ExpressionStatement(left);
964 if (!expect(TOK_LBRACE))
967 SourcePosition pos = scanner_.begin();
969 StatementList *statements = new (pool_) StatementList();
970 while (!match(TOK_RBRACE)) {
971 Statement *statement = localStatement();
972 if (statement == NULL)
974 if (!statements->append(statement))
978 BlockStatement *block = new (pool_) BlockStatement(pos, statements);
987 SourcePosition pos = scanner_.begin();
988 if (!expect(TOK_LPAREN))
991 Expression *cond = expression();
995 if (!expect(TOK_RPAREN))
998 Statement *ifTrue = localStatement();
1002 IfStatement *outer = new (pool_) IfStatement(pos, cond, ifTrue);
1004 IfStatement *last = outer;
1005 while (match(TOK_ELSE)) {
1006 if (!match(TOK_IF)) {
1007 Statement *ifFalse = localStatement();
1011 last->setIfFalse(ifFalse);
1015 SourcePosition pos = scanner_.begin();
1016 if (!expect(TOK_LPAREN))
1019 Expression *otherCond = expression();
1023 if (!expect(TOK_RPAREN))
1026 Statement *otherIfTrue = localStatement();
1030 IfStatement *inner = new (pool_) IfStatement(pos, otherCond, otherIfTrue);
1031 last->setIfFalse(inner);
1039 Parser::localStatement()
1041 Statement *statement = NULL;
1043 // Statements which must be followed by a semicolon will break out of the
1044 // switch. If they may end in BlockStatements, they'll immediately return.
1046 TokenKind kind = scanner_.peek();
1051 TypeQualifiers quals = (kind == TOK_CONST)
1055 if ((statement = vars(quals)) == NULL)
1061 return forStatement();
1068 statement = new (pool_) BreakStatement(scanner_.begin());
1073 statement = new (pool_) ContinueStatement(scanner_.begin());
1077 if ((statement = do_()) == NULL)
1082 if ((statement = returnStatement()) == NULL)
1087 if ((statement = enum_()) == NULL)
1092 if ((statement = struct_()) == NULL)
1110 if ((statement = expressionStatement()) == NULL)
1114 if (!expect(TOK_SEMICOLON))
1125 Local<String> name(zone_, expectName());
1129 SourcePosition pos = scanner_.begin();
1131 if (!expect(TOK_LBRACE))
1134 FieldList *fields = new (pool_) FieldList();
1136 while (!peek(TOK_RBRACE)) {
1138 Local<String> fieldName(zone_);
1140 if (!fullTypeAndName(&type, fieldName.ref(), DeclField))
1143 SourcePosition pos = scanner_.begin();
1145 if (!matchSemicolon())
1148 StructureStatement::Field *field =
1149 new (pool_) StructureStatement::Field(pos, type, fieldName);
1150 if (!field || !fields->append(field))
1154 if (!expect(TOK_RBRACE))
1157 if (fields->empty()) {
1158 cc_.reportError(pos, Message_StructCantBeEmpty);
1162 return new (pool_) StructureStatement(pos, name, fields);
1170 SourcePosition pos = scanner_.begin();
1172 Local<String> name(zone_);
1173 if (matchName(name.ref()) && !name)
1176 EnumStatement::EntryList *entries = new (pool_) EnumStatement::EntryList();
1178 if (!expect(TOK_LBRACE))
1182 if (scanner_.peek() == TOK_RBRACE)
1185 Local<String> name(zone_, expectName());
1189 Expression *expr = NULL;
1190 if (match(TOK_ASSIGN)) {
1191 if ((expr = expression()) == NULL)
1195 if (!entries->append(EnumStatement::Entry(name, expr)))
1197 } while (match(TOK_COMMA));
1198 if (!expect(TOK_RBRACE))
1201 return new (pool_) EnumStatement(pos, name, entries);
1205 static inline ScopedRoot<T> *
1206 NewScopedRoot(PoolAllocator &pool, Handle<T> obj)
1208 ScopedRoot<T> *root = pool.alloc<ScopedRoot<T> >();
1209 new (root) ScopedRoot<T>(obj);
1216 NameList *names = new (pool_) NameList();
1218 SourcePosition pos = scanner_.begin();
1220 Local<String> name(zone_, expectName());
1221 if (!name || !names->append(NewScopedRoot<String>(pool_, name)))
1224 TokenKind suffix = TOK_NONE;
1225 while (match(TOK_DOT)) {
1226 if (match(TOK_STAR)) {
1231 if ((name = expectName()) == NULL)
1234 if (!names->append(NewScopedRoot<String>(pool_, name)))
1238 return new (pool_) ImportStatement(pos, names, suffix);
1242 Parser::function(TokenKind token)
1244 Expression *returnType;
1245 Local<String> name(zone_);
1247 if (!fullTypeAndName(&returnType, name.ref(), DeclFun))
1250 SourcePosition pos = scanner_.begin();
1251 ParameterList *parameters = new (pool_) ParameterList();
1253 if (!expect(TOK_LPAREN))
1256 bool variadic = false;
1258 if (!match(TOK_RPAREN)) {
1260 bool isConst = match(TOK_CONST);
1261 bool ref = match(TOK_AMPERSAND);
1263 SourcePosition begin = scanner_.begin();
1265 Local<String> name(zone_);
1266 if (!fullTypeAndName(&type, name.ref(), DeclArg))
1270 if (isConst || ref) {
1271 cc_.reportError(begin, Message_InvalidVarArgs);
1275 if (token != TOK_NATIVE) {
1276 cc_.reportError(begin, Message_VarArgsOutsideNative);
1284 Expression *expr = NULL;
1285 if (match(TOK_ASSIGN)) {
1286 if ((expr = expression()) == NULL)
1290 Parameter *parameter = new (pool_) Parameter(name, type, ref, isConst, expr, begin);
1291 if (!parameters->append(parameter))
1294 if (!match(TOK_COMMA))
1297 if (!expect(TOK_RPAREN))
1301 encounteredReturn_ = false;
1303 Statement *body = NULL;
1304 if (token != TOK_FORWARD && token != TOK_NATIVE && token != TOK_TYPEDEF) {
1310 FunctionSignature signature(returnType, parameters, variadic);
1311 if (token == TOK_TYPEDEF)
1312 return new (pool_) FunctionTypeStatement(pos, name, signature);
1314 return new (pool_) FunctionStatement(pos, name, token, body, signature, encounteredReturn_);
1318 DeclareIntrinsicType(Zone *zone, Scope *scope, const char *name, Handle<Type> type)
1320 Local<String> tag(zone, zone->makeSymbol(name));
1325 return false; //return !!scope->declareType(tag, type);
1329 DeclareIntrinsicType(Zone *zone, Scope *scope, const char *name, PrimitiveType type)
1331 Local<Type> primitive(zone, zone->types()->getPrimitive(type));
1333 return DeclareIntrinsicType(zone, scope, name, primitive);
1337 Parser::setupIntrinsics()
1340 if (!DeclareIntrinsicType(zone_, scope_, "Float", PrimitiveType_Float))
1342 if (!DeclareIntrinsicType(zone_, scope_, "_", PrimitiveType_Int32))
1344 if (!DeclareIntrinsicType(zone_, scope_, "bool", PrimitiveType_Bool))
1346 if (!DeclareIntrinsicType(zone_, scope_, "String", PrimitiveType_Char))
1349 Local<Type> unchecked(zone_, zone_->types()->getUnchecked());
1350 if (!DeclareIntrinsicType(zone_, scope_, "unchecked", unchecked))
1353 Local<Type> voidType(zone_, zone_->types()->getVoid());
1354 if (!DeclareIntrinsicType(zone_, scope_, "void", voidType))
1357 Local<Type> intType(zone_, zone_->types()->getPrimitive(PrimitiveType_Int32));
1358 if (!DeclareIntrinsicType(zone_, scope_, "int", intType))
1367 StatementList *statements = new (pool_) StatementList();
1369 if (!setupIntrinsics())
1373 Statement *statement = NULL;
1375 TokenKind kind = scanner_.peek();
1384 if ((statement = function(TOK_NAME)) == NULL)
1391 if (statements->length())
1392 cc_.reportError(scanner_.begin(), Message_ImportsMustBeFirst);
1393 ImportStatement *stmt = import();
1396 if (!imports_->append(stmt))
1398 if (!matchSemicolon())
1410 if (kind != TOK_LABEL)
1412 if (kind == TOK_PUBLIC && peek(TOK_CONST)) {
1413 TypeQualifiers quals = TypeQualifiers(Qual_Public | Qual_Const);
1414 if ((statement = var(quals)) == NULL)
1416 if (!matchSemicolon())
1419 if ((statement = function(kind)) == NULL)
1421 if ((kind == TOK_FORWARD || kind == TOK_NATIVE || kind == TOK_TYPEDEF) && !matchSemicolon())
1430 TypeQualifiers quals = (kind == TOK_NEW) ? Qual_Const : Qual_None;
1431 // Note: at global scope, we don't accept comma-delimited
1432 // declarations yet, to see if we can get away with forcing
1433 // a nicer top-level style.
1435 if ((statement = var(quals)) == NULL)
1437 if (!matchSemicolon())
1443 if ((statement = enum_()) == NULL)
1445 if (!matchSemicolon())
1450 if ((statement = struct_()) == NULL)
1452 if (!matchSemicolon())
1457 cc_.reportError(scanner_.begin(), Message_ExpectedGlobal);
1464 if (!statements->append(statement))
1468 return new (pool_) ParseTree(statements, unbound_, imports_);
1471 class AstPrinter : public AstVisitor
1478 for (size_t i = 0; i < level_; i++)
1489 AstPrinter(FILE *fp)
1495 void dump(const FunctionSignature &sig) {
1497 fprintf(fp_, "()\n");
1499 void visitNameProxy(NameProxy *name) {
1501 fprintf(fp_, "[ NameProxy (%s)\n", name->name()->chars());
1503 void visitCallExpression(CallExpression *node) {
1505 fprintf(fp_, "[ CallExpression\n");
1507 node->callee()->accept(this);
1508 for (size_t i = 0; i < node->arguments()->length(); i++)
1509 node->arguments()->at(i)->accept(this);
1512 void visitFunctionTypeStatement(FunctionTypeStatement *node) {
1514 fprintf(fp_, "[ FunctionTypeStatement (%s)\n", node->name()->chars());
1516 dump(node->signature());
1519 void visitFunctionStatement(FunctionStatement *node) {
1521 fprintf(fp_, "[ FunctionStatement (%s)\n", node->name()->chars());
1523 dump(node->signature());
1525 node->body()->accept(this);
1528 void visitExpressionStatement(ExpressionStatement *node) {
1530 fprintf(fp_, "[ ExpressionStatement\n");
1532 node->expression()->accept(this);
1535 void visitAssignment(Assignment *node) {
1537 fprintf(fp_, "[ Assignment\n");
1539 node->lvalue()->accept(this);
1540 node->expression()->accept(this);
1543 void visitTernaryExpression(TernaryExpression *node) {
1545 fprintf(fp_, "[ TernaryExpression\n");
1547 node->condition()->accept(this);
1548 node->left()->accept(this);
1549 node->right()->accept(this);
1552 void visitBinaryExpression(BinaryExpression *node) {
1554 fprintf(fp_, "[ BinaryExpression (%s)\n", TokenNames[node->token()]);
1556 node->left()->accept(this);
1557 node->right()->accept(this);
1560 void visitUnaryExpression(UnaryExpression *node) {
1562 fprintf(fp_, "[ UnaryExpression (%s)\n", TokenNames[node->token()]);
1564 node->expression()->accept(this);
1567 void visitReturnStatement(ReturnStatement *node) {
1569 fprintf(fp_, "[ ReturnStatement\n");
1571 if (node->expression())
1572 node->expression()->accept(this);
1575 void visitForStatement(ForStatement *node) {
1577 fprintf(fp_, "[ ForStatement\n");
1579 if (node->initialization())
1580 node->initialization()->accept(this);
1581 if (node->condition())
1582 node->condition()->accept(this);
1584 node->update()->accept(this);
1585 node->body()->accept(this);
1588 void visitBlockStatement(BlockStatement *node) {
1590 fprintf(fp_, "[ BlockStatement\n");
1592 for (size_t i = 0; i < node->statements()->length(); i++)
1593 node->statements()->at(i)->accept(this);
1596 void visitVariableDeclaration(VariableDeclaration *node) {
1598 fprintf(fp_, "[ VariableDeclaration (%s)\n", node->name()->chars());
1600 if (node->initialization())
1601 node->initialization()->accept(this);
1604 void visitIntegerLiteral(IntegerLiteral *node) {
1606 fprintf(fp_, "[ IntegerLiteral (%d)\n", node->value());
1608 void visitBooleanLiteral(BooleanLiteral *node) {
1610 fprintf(fp_, "[ BooleanLiteral (%s)\n",
1611 (node->token() == TOK_TRUE) ? "true" : "false");
1613 void visitFloatLiteral(FloatLiteral *node) {
1615 fprintf(fp_, "[ FloatLiteral (%f)\n", node->value());
1617 void visitIfStatement(IfStatement *node) {
1619 fprintf(fp_, "[ IfStatement\n");
1621 node->ifTrue()->accept(this);
1622 if (node->ifFalse())
1623 node->ifFalse()->accept(this);
1626 void visitIndexExpression(IndexExpression *node) {
1628 fprintf(fp_, "[ IndexExpression\n");
1630 node->left()->accept(this);
1631 node->right()->accept(this);
1634 void visitEnumStatement(EnumStatement *node) {
1636 fprintf(fp_, "[ EnumStatement (%s)\n", node->name() ? node->name()->chars() : "<anonymous>");
1638 for (size_t i = 0; i < node->entries()->length(); i++) {
1640 fprintf(fp_, "%s =\n", node->entries()->at(i).name->chars());
1641 if (node->entries()->at(i).expr) {
1643 node->entries()->at(i).expr->accept(this);
1649 void visitWhileStatement(WhileStatement *node) {
1651 fprintf(fp_, "[ WhileStatement (%s)\n",
1652 (node->token() == TOK_DO) ? "do" : "while");
1654 if (node->token() == TOK_DO) {
1655 node->condition()->accept(this);
1656 node->body()->accept(this);
1658 node->condition()->accept(this);
1659 node->body()->accept(this);
1663 void visitBreakStatement(BreakStatement *node) {
1665 fprintf(fp_, "[ BreakStatement\n");
1667 void visitContinueStatement(ContinueStatement *node) {
1669 fprintf(fp_, "[ ContinueStatement\n");
1671 void visitStringLiteral(StringLiteral *node) {
1673 fprintf(fp_, "[ StringLiteral\n");
1675 void visitIncDecExpression(IncDecExpression *node) {
1677 fprintf(fp_, "[ IncDecExpression (postfix=%d)\n", node->postfix());
1679 node->expression()->accept(this);
1682 void visitStructureStatement(StructureStatement *node) {
1684 fprintf(fp_, "[ StructureStatement %s\n", node->name()->chars());
1686 void visitFieldExpression(FieldExpression *node) {
1688 fprintf(fp_, "[ FieldExpression %s\n", node->field()->chars());
1690 node->left()->accept(this);
1693 void visitSwitchStatement(SwitchStatement *node) {
1695 fprintf(fp_, "[ SwitchStatement\n");
1697 node->expression()->accept(this);
1698 for (size_t i = 0; i < node->cases()->length(); i++) {
1699 Case *c = node->cases()->at(i);
1700 c->expression()->accept(this);
1702 for (size_t j = 0; j < c->others()->length(); j++)
1703 c->others()->at(j)->accept(this);
1706 c->statement()->accept(this);
1709 if (node->defaultCase())
1710 node->defaultCase()->accept(this);
1713 void visitArrayLiteral(ArrayLiteral *node) {
1715 fprintf(fp_, "[ ArrayLiteral\n");
1717 for (size_t i = 0; i < node->expressions()->length(); i++) {
1718 Expression *expr = node->expressions()->at(i);
1724 void visitImportStatement(ImportStatement *node) {
1726 fprintf(fp_, "[ ImportStatement\n");
1731 ParseTree::dump(FILE *fp)
1733 AstPrinter printer(fp);
1735 for (size_t i = 0; i < statements_->length(); i++)
1736 statements_->at(i)->accept(&printer);