Create and Implement Token Parser

[ngraf3255]

The program currently has no way to parser tokens into an abstract syntax tree. As such a token parser must be implemented to allow for the program to compile BASIC code to machine code.

Required tasks:

• Use formalized token format as dictacted by what was determined by Create and implement token lexer #2
• Begin implementing a fast parser to convert tokens into an AST

Examples to look at:

• GNU Bison
• The Parser
• Abstract Syntax Tree
• Recursive Descent Parser

[ngraf3255]

This should be associated with branch dev/create-token-parser

[arkg7]

Created a branch for the token parser to be developed in

[ngraf3255]

@arkg7 , I'd like you to develop and implement the majority of this ticket on your own. Please create a detailed specification for the AST you plan to create first, so I can work on the LLVM IR generation, while you should plan and detail the development of the parsing internals on your own.

Working individually on each of these parts should allow us to build our skills seperately while still practicing working with others through setting up the AST specification.