Re: problems with identifiers and keywords...

On 2004-10-22, Micha <micha-1@fantasymail.de> wrote:
> The second problem is, that there are (at some positions) special "words"
> allowed, which can consist of nearly all (printable) characters except
> spaces. To get this I wanted to switch the lexer-function for the next
> token in a parser-action, but that does not work all the time (I think
> because of the look-ahead, which is scanned with the wrong lexer). How
> can I get this switch right? Or is this a completly wrong way of handling
> this?


You cannot usually switch modes like this in the parser and have the
lexer see it in time. If the parser required lookahead, then the
token in question would have already been lexed - in your case
presumably incorrectly.


It's best if the lexer can switch modes w/o any intervention from the
parser. It's difficult to know exactly what you need unless you give
us a little more background info.


Can the lexer detect in all cases when the mode needs to be switched?
If so, you can use flex's feature of conditional matching called
"start conditions". This is probably your best bet for getting this
right since your 'everything but space' pattern could be put here and
predicated by the state <s>.


If the parser is the only feasible place to do this and the lexer
would've matched this token differently in this other mode, you can
access the current lookahead token in the parser via yychar and
re-inject it into your token stream (essentially forcing a re-lexing
of the token).


-Clint
[I've found in practice that LALR lookaheads rarely present a problem
when feeding hints back to the lexer. Although yacc can read ahead a
token, as often than not it does default reductions which don't need
one. -John]

Return to the comp.compilers page.
Search the comp.compilers archives again.