1
Q
Single Symbol Lookahead
A
first token in grammar unambiguously tells you which node you’re at
2
Q
5 steps to Tokenizing
A
- Read input as a single string
- kill all the newlines
- Replace all reserved words with RESERVEDWORDRESERVEDWORD
- Replace all RESERVEDWORDRESERVEDWORD with just RESERVEDWORD
- Split on RESERVEDWORD
