Re: Buffered input for a lexer?

Chris F Clark <cfc@world.std.com> wrote:
> The performance impact of a push v. a pull style lexer is negligible.
> It is below the threshold of what we've been able to ever measure. We


I definitely believe that.


> The one notable exception to that are the people using our system to
> do real event-driven state machine models. They need the push model
> and do some very interesting things with it. The push models combines
> very well with objects to make little machines that you can
> run/pause/save/restore/resume/.... all in parallel.


Interesting. How does the lookahead of the lexer and parser interact
with the push style of input? Did you have to change the parsing
algorithms to perform reductions as soon as possible?


Does your customer use your yacc++ grammar file to define the state
machine, or do they use something closer to a state machine description
language? It seems that describing a state machine with a grammar would
be kind of odd (although I haven't tried. :)


> More important are the details of the implementation. Those places
> where our code is even subtly different than [f]lex and yacc/bison can
> cause various hacks that people have developed over the years to fail.


Unfortunately, backwards compatibility is always an impediment to
progress. I imaging with lex and yacc that one of the big problems is
"undocumented" interfaces that people use, not to mention the more or
less hackish nature of the documented interfaces provided...


Thanks for the insight!


-Chris

Return to the comp.compilers page.
Search the comp.compilers archives again.