Re: coupling LALR with a scanner?

> IELR was exactly made for that reason, as a first step to PSLR: some
> grammars have no 'tokens' and 'grammar rules', they just have a
> 'grammar' where mutually exclusive tokens are present, e.g. you
> cannot make a Javascript single lexer as there are state where /
> (slash) means 'start of regular expression' (of course the content
> of the regular expression follows totally different lexing rules
> than the rest of the text) whereas in other states it means
> 'division' operator. If your parser cannot tell which of the two
> lexers to use, you are off.


IELR(1) does not solve this problem. It solves the problem of reduce-
reduce conflicts in an LR(0) state machine where a state has multiple
"lookback nonterminal transitions" which cause conflicts. IELR(1)
does state splitting (or duplicating) in order to remove the conflicts
(if possible). It has nothing to do with how many scanners you may
need. It only pertains to the language defined by the grammar. It
looks like you have two languages or two lexical languages (and need
two scanners).


To show that IELR(1) solves this problem you would have to have an
LALR(1) grammar that has reduce-reduce conflicts caused by the need of
two different lexers (scanners) and show that IELR(1) removes the
conflicts.


What you described with Javascript is a little language inside of
another language. Why not switch parsers when a slash ('/') is
encountered if necessary?


Parsing expression grammars are more appropriate for this Javascript
problem, if you don't want to hand-code a "fix" within an LALR parser.


A similar situation occurs when parsing C. You encounter a "#if ..."
while parsing C. At the '#' you must leave the C parser and enter the
C- preprocessor parser. After the conditional statement has been
parsed, you must switch back to the C parser.


/ Paul Mann

Return to the comp.compilers page.
Search the comp.compilers archives again.