top | up | prev | next

TokenLexer Class

Definition at line 9 of TokenLexer.hpp

Constructors

TokenLexer(const char32_t* start_, const char32_t* end_, const std::string& fileName_, int fileIndex_)

TokenLexer(const std::u32string& content_, const std::string& fileName_, int fileIndex_)

Member Functions

int	GetTokenId(int statementIndex)
int	NextState(int state, char32_t c) override

Member Variables

bool	inBlockComment

Constructor Details

TokenLexer Constructor

TokenLexer::TokenLexer(const char32_t * start_, const char32_t * end_, const std::string& fileName_, int fileIndex_)

Definition at line 19 of TokenLexer.cpp :
   19 : soulng::lexer::Lexer(start_, end_, fileName_, fileIndex_),
   20 inBlockComment()
   21 {
   22     SetKeywordMap(TokenLexerKeywords::GetKeywordMap());
   23 }

Declaration at line 13 of TokenLexer.hpp

TokenLexer Constructor

TokenLexer::TokenLexer(const std::u32string& content_, const std::string& fileName_, int fileIndex_)

Definition at line 13 of TokenLexer.cpp :
   13 : soulng::lexer::Lexer(content_, fileName_, fileIndex_),
   14 inBlockComment()
   15 {
   16     SetKeywordMap(TokenLexerKeywords::GetKeywordMap());
   17 }

Declaration at line 12 of TokenLexer.hpp

Member Function Details

GetTokenId Member Function

int TokenLexer::GetTokenId(int statementIndex)

Definition at line 19032 of TokenLexer.cpp
Declaration at line 17 of TokenLexer.hpp

Calls: soulng::lexer::Lexer::GetKeywordToken , soulng::lexer::Lexer::Retract

Called by: TokenLexer::NextState

NextState Member Function

int TokenLexer::NextState(int state, char32_t c) override

Definition at line 25 of TokenLexer.cpp
Declaration at line 14 of TokenLexer.hpp

Base class overridden functions: soulng::lexer::Lexer::NextState

Calls: TokenLexer::GetTokenId

top | up | prev | next