1.2.2 Adding Tokens

Minilang tokens consist of the reserved words of the language , the tokens for literals and the tokens for the operator and punctuation symbols.

The token file syntax is described in this document.

To add tokens to the minilang language:

• Right-click your project in the Solution Explorer and select 'Add | New Item...'.
• Select Visual C++ | Utility | Text File (.txt) item and name the file minilang.token .
• Add the following code to the token file:
  
  tokens minilang.token
  {
      (IF, "'if'"), (ELSE, "'else'"), (WHILE, "'while'"), (RETURN, "'return'"), (INT, "'int'"), (BOOL, "'bool'"), (VOID, "'void'"),
      (ID, "identifier"), (INTEGER_LITERAL, "integer literal"), (TRUE, "'true'"), (FALSE, "'false'"),
      (SEMICOLON, "';'"), (LPAREN, "'('"), (RPAREN, "')'"), (LBRACE, "'{'"), (RBRACE, "'}'"),
      (PLUS, "'+'"), (MINUS, "'-'"), (MUL, "'*'"), (DIV, "'/'"), (MOD, "'%''"),
      (NOT, "'!'"), (EQ, "'=='"), (NEQ, "'!='"), (LEQ, "'<='"), (GEQ, "'>='"), (LESS, "'<'"), (GREATER, "'>'"),
      (ASSIGN, "'='"), (COMMA, "','")
  }