Lecturecise 03: Longest Match. Generating Lexical Analyzers

Slides: pptx, pdf

Key insights:

  • we follow the maximal munch rule: lexical analyzer should always eagerly accept the longest token that it can recognize from the current point
  • it is possible to automate the construction of lexical analyzers; the starting point of this construction is a conversion of regular expressions to deterministic automata
  • tools that automate this construction are part of compiler-compilers such as JavaCC

Hand-Written Scanner for While Language

Tools for Constructing Lexers

Compiler Construction Tools

Collection of Exercises


Background on regular languages and automata