Now the Main.cpp source file of the minilang project contains the TestMinilangLexer function for testing the generated lexer.
We will implement a similar function for testing the parsers called TestMinilangParser. The task of the function is to parse the contents of a .minilang source file.
We begin by printing the path name of the .minilang file to stdout:
void TestMinilangParser(const std::string& minilangFilePath)
{
std::cout << "> " << minilangFilePath << std::endl;
// ...
}
Then we read the content of the test file to a std::string:
// ...
std::string s = soulng::util::ReadFile(minilangFilePath);
Then convert the content to a UTF-32 string:
// ...
std::u32string content = soulng::unicode::ToUtf32(s);
Then we construct a lexer with the content of the test file, the path name of it, and file index 0:
// ...
MinilangLexer lexer(content, minilangFilePath, 0);
The file index is not used here, so it can have any value.
The parser that parses a whole .minilang source file is generated from the SourceFileParser.parser file. Here's the current content of the SourceFileParser.parser:
#include <minilang/FunctionParser.hpp>
#include <minilang/MinilangLexer.hpp>
#include <minilang/MinilangTokens.hpp>
using namespace MinilangTokens;
parser SourceFileParser
{
uselexer MinilangLexer;
main;
using FunctionParser.Function;
SourceFile
::= Function:function*
;
}
We add an include directive for the SourceFileParser.hpp generated from the SourceFileParser.parser and call the Parse() function of the SourceFileParser with the constructed lexer:
// ...
#include <minilang/SourceFileParser.hpp>
// ...
void TestMinilangParser(const std::string& minilangFilePath)
{
// ...
SourceFileParser::Parse(lexer);
}
Here's the whole TestMinilangParser function:
void TestMinilangParser(const std::string& minilangFilePath)
{
std::cout << "> " << minilangFilePath << std::endl;
std::string s = soulng::util::ReadFile(minilangFilePath);
std::u32string content = soulng::unicode::ToUtf32(s);
MinilangLexer lexer(content, minilangFilePath, 0);
SourceFileParser::Parse(lexer);
std::cout << "end of file '" << minilangFilePath << "' reached" << std::endl;
}
The spg tool generates a Parse function for each parser that contains a main statement, such as this SourcedFileParser. The Parse function takes a lexer parameter and delegates the parsing to the first parsing rule of the parser, that is the one and only SourceFile rule for the SourcedFileParser.
Here's additions to the Main.cpp that implement parser testing:
void PrintUsage()
{
// ...
std::cout << "--parser-test | -p" << std::endl;
std::cout << " Test parser with <file.minilang>." << std::endl;
}
enum class Command
{
none, lexerTest, parserTest
};
int main(int argc, const char** argv)
{
// ...
if (soulng::util::StartsWith(arg, "--"))
{
// ...
else if (arg == "--parser-test")
{
command = Command::parserTest;
}
}
else if (soulng::util::StartsWith(arg, "-"))
{
// ...
for (char o : options)
{
// ...
else if (o == 'p')
{
command = Command::parserTest;
}
}
}
// ...
for (const std::string& filePath : files)
{
// ...
else if (command == Command::parserTest)
{
TestMinilangParser(filePath);
}
}
// ...
}
Here's the whole main function:
void PrintUsage()
{
std::cout << "Usage: minilang [options] { file.minilang }" << std::endl;
std::cout << "Options:" << std::endl;
std::cout << "--help | -h:" << std::endl;
std::cout << " Print help and exit." << std::endl;
std::cout << "--lexer-test | -l" << std::endl;
std::cout << " Test lexical analyzer with ." << std::endl;
std::cout << "--parser-test | -p" << std::endl;
std::cout << " Test parser with ." << std::endl;
}
enum class Command
{
none, lexerTest, parserTest
};
int main(int argc, const char** argv)
{
Initializer initializer;
try
{
std::vector files;
Command command = Command::none;
for (int i = 1; i < argc; ++i)
{
std::string arg = argv[i];
if (soulng::util::StartsWith(arg, "--"))
{
if (arg == "--help")
{
PrintUsage();
return 1;
}
else if (arg == "--lexer-test")
{
command = Command::lexerTest;
}
else if (arg == "--parser-test")
{
command = Command::parserTest;
}
else
{
throw std::runtime_error("unknown argument '" + arg + "'");
}
}
else if (soulng::util::StartsWith(arg, "-"))
{
std::string options = arg.substr(1);
if (options.empty())
{
throw std::runtime_error("unknown argument '" + arg + "'");
}
for (char o : options)
{
if (o == 'h')
{
PrintUsage();
return 1;
}
else if (o == 'l')
{
command = Command::lexerTest;
}
else if (o == 'p')
{
command = Command::parserTest;
}
else
{
throw std::runtime_error("unknown argument '-" + std::string(1, o) + "'");
}
}
}
else
{
files.push_back(soulng::util::GetFullPath(arg));
}
}
if (files.empty() || command == Command::none)
{
PrintUsage();
return 1;
}
for (const std::string& filePath : files)
{
if (command == Command::lexerTest)
{
TestMinilangLexer(filePath);
}
else if (command == Command::parserTest)
{
TestMinilangParser(filePath);
}
else
{
PrintUsage();
throw std::runtime_error("minilang: unknown command");
}
}
}
catch (const std::exception& ex)
{
std::cerr << soulng::unicode::ToUtf32(ex.what()) << std::endl;
return 1;
}
return 0;
}
We begin testing with empty input. The file empty.minilang has zero length.
We set the Configuration Properties / Debugging / Command Arguments to the value: --parser-test test\empty.minilang, and run the minilang project.
Here's the output:
> C:/soulng-1.0.0/examples/minilang/test/empty.minilang
end of file 'C:/soulng-1.0.0/examples/minilang/test/empty.minilang' reached
The parsing succeeded without an error, so the parser can handle an empty sequence of functions.
Next we will test with a minimal minimal.minilang source file that contains one function that does nothing. Here's the contents of the minimal.minilang:
void minimal(){}
We set the Configuration Properties / Debugging / Command Arguments to the value: --parser-test test\minimal.minilang, and run the minilang project.
Here's the output:
> C:/soulng-1.0.0/examples/minilang/test/minimal.minilang
end of file 'C:/soulng-1.0.0/examples/minilang/test/minimal.minilang' reached
The parsing succeeded again without an error, so the parser can handle a simple function.
Next we will test with a function that contains some computation: gcd.minilang: Here's the contents of the gcd.minilang:
int gcd(int a, int b)
{
while (b != 0)
{
a = a % b;
int t = a;
a = b;
b = t;
}
return a;
}
Here's the output:
> C:/soulng-1.0.0/examples/minilang/test/gcd.minilang
end of file 'C:/soulng-1.0.0/examples/minilang/test/gcd.minilang' reached
So far so good...
Now testing with a function that contains a parsing error, error_comma.minilang:
void foo,()
{
}
Here's the output:
> C:/soulng-1.0.0/examples/minilang/test/error_comma.minilang
parsing error in 'C:/soulng-1.0.0/examples/minilang/test/error_comma.minilang:1': end of file expected:
void foo,()
^^^^
The error message says that "end of file expected" and points to the void return value of the function, not the excessive comma as it should, so this test failed.
Now testing with a file with a missing semicolon, error_semicolon.minilang, that is modified from the gcd.minilang by removing the semicolon from the int t = a statement:
int gcd(int a, int b)
{
while (b != 0)
{
a = a % b;
int t = a
a = b;
b = t;
}
return a;
}
> C:/soulng-1.0.0/examples/minilang/test/error_semicolon.minilang
parsing error in 'C:/soulng-1.0.0/examples/minilang/test/error_semicolon.minilang:1': end of file expected:
int gcd(int a, int b)
^^^
The problem is same as in the fourth test: the error message says that "end of file expected" and points to the int return value of the function, not the missing semicolon as it should. This test failed also.
These problems lead to the next topic: generating parsing errors...
up: Table of contents | prev: Building the Parsers | next: Generating Parsing Errors