Lexers in YACC?

I'm interested in parsing a language that is not LALR(1) using YACC. The
general approach taken in the past (by others) is to handle the non-LALR(1)
portions (which are fairly simple) by writing a hard-coded "intermediate
stage" between the lexer and the parser. This intermediate stage filters the
stream of lexer tokens and uses recursive descent to detect the problem
combinations and pack them up into single tokens which are then passed to the
parser.


It seems to me that it wouldn't be difficult to implement this intermediate
stage using YACC as well. In this version, the lexer would pass its tokens
to the first-stage YACC grammar, which would pass most of them unchanged.
Sequences that needed it would be reduced as above. The resulting stream
would then go to the second-stage YACC grammar, which would do most of the
parsing work.


What I need to know is, how does one write a set of YACC actions that cause
the resulting grammar to emulate yylex? I understand the preprocessor tricks
that allow two different YACC grammars to co-exist without name collision.
What I don't understand is how to make the generated parser of the first
stage "return" tokens to its caller. It seems to require generalized
co-routines, which of course C does not provide. The operating system is
Unix System V.


Has anyone tried to solve this sort of problem? Please send comments
directly to me, and I will summarize them to the net. I have set followups
to "poster".
--
cowan@marob.masa.com (aka ...!hombre!marob!cowan)
[I suppose you could run it as two processes connected by a pipe, the Unix
version of coroutines. -John]
--

Return to the comp.compilers page.
Search the comp.compilers archives again.