Re: building a lexer generator

Hans-Peter Diettrich <>
3 Dec 2006 21:31:36 -0500

          From comp.compilers

Related articles
building a lexer generator (2006-11-29)
Re: building a lexer generator (Karsten Nyblad) (2006-12-01)
Re: building a lexer generator (2006-12-01)
Re: building a lexer generator (Hans-Peter Diettrich) (2006-12-03)
Re: building a lexer generator (2007-01-05)
| List of all articles for this month |

From: Hans-Peter Diettrich <>
Newsgroups: comp.compilers
Date: 3 Dec 2006 21:31:36 -0500
Organization: Compilers Central
References: 06-11-123 06-12-004
Keywords: lex
Posted-Date: 03 Dec 2006 21:31:36 EST

Karsten Nyblad wrote:

> of the regular expression you prefer to recognize, e.g., if a state can
> recognize a keyword and an identifier, you let the state recognize the
> keyword.

There exist cases where it is not wise to classify keywords immediately.
In C/C++ keywords can be used as macro identifiers, so that different
lexers would be required for an separate preprocessor, and for the
parser itself. Delphi/OPL defines "directives" along with keywords,
where directives are recognized only in a specific context, and
otherwise are ordinary identifiers.

In such cases it might be easier to detect keywords while looking up the
identifiers in the symbol table.


Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.