Re: building a lexer generator

torbenm@app-3.diku.dk (=?iso-8859-1?q?Torben_=C6gidius_Mogensen?=)
1 Dec 2006 09:18:56 -0500

          From comp.compilers
Related articles
building a lexer generator mips@cyberspace.org (2006-11-29)
Re: building a lexer generator 148f3wg02@sneakemail.com (Karsten Nyblad) (2006-12-01)
Re: building a lexer generator torbenm@app-3.diku.dk (2006-12-01)
Re: building a lexer generator DrDiettrich1@aol.com (Hans-Peter Diettrich) (2006-12-03)
Re: building a lexer generator mips@cyberspace.org (2007-01-05)
| List of all articles for this month |
From: torbenm@app-3.diku.dk (=?iso-8859-1?q?Torben_=C6gidius_Mogensen?=)
Newsgroups: comp.compilers
Date: 1 Dec 2006 09:18:56 -0500
Organization: Department of Computer Science, University of Copenhagen
References: 06-11-123
Keywords: lex
Posted-Date: 01 Dec 2006 09:18:56 EST

mips@cyberspace.org writes:




> I'M Trying To Build A Lexer Generator But I'Ve Only Been Hand Coded
> Lexer Before So I'M A Bit Stuck On The Design Side.
>
> I've already done searches on the various steps needed to build such
> tools but i still miss some points. For example, when i have five
> keywords and one identifier definition how do the lexer process them ?
> Does it have a DFA for each keywords and the identifier or does it
> process all with the same FSM ? I would bet on the second one but
> this mean that i must merge all the DFA (or i surely missed
> something).


As John said, the usual thing to do is to build a combined DFA -- not
by constructin individual DFAs and then combining these, but by
building individual NFAs, combining them and then convert the combined
NFA into a DFA.


http://www.diku.dk/undervisning/2006-2007/2006-2007_b2_ovs/Chapters1and2.pdf
(which are the first two chapters of my compiler notes/book) shows
more detail.


Torben


Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.