It's will be better to use some kind of finite-state-machine(fsm) for lexer. For 2 main reasons: 1. Effective 2. Maintenance The second one means we should provide some tool to simply add new tokens to lexer by adding them to fsm.
It's will be better to use some kind of finite-state-machine(fsm) for lexer. For 2 main reasons:
The second one means we should provide some tool to simply add new tokens to lexer by adding them to fsm.