Re: Buffered input for a lexer?

Chris Lattner <sabre@nondot.org>
31 Mar 2002 23:20:09 -0500

          From comp.compilers

Related articles
[3 earlier articles]
Re: Buffered input for a lexer? sabre@nondot.org (Chris Lattner) (2002-03-24)
Re: Buffered input for a lexer? sabre@nondot.org (Chris Lattner) (2002-03-24)
Re: Buffered input for a lexer? rhyde@cs.ucr.edu (Randall Hyde) (2002-03-25)
Re: Buffered input for a lexer? cfc@world.std.com (Chris F Clark) (2002-03-25)
Re: Buffered input for a lexer? clint@0lsen.net (2002-03-31)
Re: Buffered input for a lexer? sabre@nondot.org (Chris Lattner) (2002-03-31)
Re: Buffered input for a lexer? sabre@nondot.org (Chris Lattner) (2002-03-31)
Re: Buffered input for a lexer? joachim_d@gmx.de (Joachim Durchholz) (2002-03-31)
Re: Buffered input for a lexer? cgweav@aol.com (2002-03-31)
Re: Buffered input for a lexer? bear@sonic.net (Ray Dillinger) (2002-04-10)
Re: Buffered input for a lexer? bear@sonic.net (Ray Dillinger) (2002-04-10)
Re: Buffered input for a lexer? cgweav@aol.com (2002-04-13)
Re: Buffered input for a lexer? ralph@inputplus.co.uk (2002-04-16)
[8 later articles]
| List of all articles for this month |

From: Chris Lattner <sabre@nondot.org>
Newsgroups: comp.compilers
Date: 31 Mar 2002 23:20:09 -0500
Organization: University of Illinois at Urbana-Champaign
References: 02-03-162 02-03-172
Keywords: lex, practice
Posted-Date: 31 Mar 2002 23:20:09 EST

Chris F Clark <cfc@world.std.com> wrote:
> The performance impact of a push v. a pull style lexer is negligible.
> It is below the threshold of what we've been able to ever measure. We


I definitely believe that.


> The one notable exception to that are the people using our system to
> do real event-driven state machine models. They need the push model
> and do some very interesting things with it. The push models combines
> very well with objects to make little machines that you can
> run/pause/save/restore/resume/.... all in parallel.


Interesting. How does the lookahead of the lexer and parser interact
with the push style of input? Did you have to change the parsing
algorithms to perform reductions as soon as possible?


Does your customer use your yacc++ grammar file to define the state
machine, or do they use something closer to a state machine description
language? It seems that describing a state machine with a grammar would
be kind of odd (although I haven't tried. :)


> More important are the details of the implementation. Those places
> where our code is even subtly different than [f]lex and yacc/bison can
> cause various hacks that people have developed over the years to fail.


Unfortunately, backwards compatibility is always an impediment to
progress. I imaging with lex and yacc that one of the big problems is
"undocumented" interfaces that people use, not to mention the more or
less hackish nature of the documented interfaces provided...


Thanks for the insight!


-Chris


Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.