Re: Context sensitive scanner ? (John D. Mitchell)
30 Nov 1997 22:57:48 -0500

          From comp.compilers

Related articles
[5 earlier articles]
Re: Context sensitive scanner ? (Scott Stanchfield) (1997-11-24)
Re: Context sensitive scanner ? (Chris F Clark) (1997-11-28)
Re: Context sensitive scanner ? (Henry Spencer) (1997-11-28)
Re: Context sensitive scanner ? (1997-11-29)
Re: Context sensitive scanner ? (Albert Theo Hofkamp) (1997-11-29)
Re: Context sensitive scanner ? (Scott Stanchfield) (1997-11-30)
Re: Context sensitive scanner ? (1997-11-30)
Re: Context sensitive scanner ? (Scott Stanchfield) (1997-11-30)
Re: Context sensitive scanner ? (Chris Clark USG) (1997-12-05)
Re: Context sensitive scanner ? (Mark Thiehatten) (1997-12-07)
Re: Context sensitive scanner ? (1997-12-07)
| List of all articles for this month |

From: (John D. Mitchell)
Newsgroups: comp.compilers
Date: 30 Nov 1997 22:57:48 -0500
Organization: Non, Inc.
References: 97-11-117 97-11-122 97-11-172
Keywords: lex (John D. Mitchell) writes:
>> Or you could switch to an LL(k) tool like PCCTS/ANTLR v1.x so you don't
>> have to worry so much about action placement.

Albert Theo Hofkamp <> wrote:
>Sorry, I should have stated my concern more clearly. If I would
>introduce states in the scanner such that the scanner would return
>only the `right' tokens, and the control of the state is done from
>within yacc, then I consider the moment when the parser executes the
>change of state as tricky.

Hmm... If I understand you correctly, I did not intend to imply the
use of parser feedback into the scanner is any sense a good solution.
On the contrary, I think it's a bad idea. If you're going to insist
on doing anyways, then using an LL parser rather than an LR parser
will make things much less tricky.

>I do not understand why LL(k) would solve this problem. It is true
>that it is easier to predict which action is executed, but the exact
>moment is still quite dependent on the implementation imho.

Huh? LL parsers don't have the fragility that LR parsers have
w.r.t. action placement. That makes action firing basically trivial to
predict in LL parsers.

>> You're asking for a predictive parser rather than a synthesizing parser.

>No, I am asking for a scanner which tries to do interpretation within
>the limits of what the parser expects. In other words, when the parser
>doesn't want a real, and the scanner gets a real as input, it will not
>return a real, but 2 integer numbers seperated by a dot.

There is an ambiguity in your language specification. You seem to
want some magical code to guess how to do the disambiguation for you.
Good luck. :-) You're going to have to figure out how and where you
can best add the appropriate amount of intelligence to your system to

If it's possible to lexically disambiguate (due to e.g., lexical
context) then you're set (as you can do everything in the lexer
without hacking the parser at all). This works great for e.g., the
various uses of '.' in C. ANTLR v2.x provides both syntactic and
semantic predicates in the lexer which can make this much easier to do
than in regexp based tools.

Otherwise, if you have to rely on parser state to disambiguate then I
suggest you follow the moderator's advice and just return the
constituent components and combine them as appropriate in your parser.
Again, the semantic and syntactic predicates available in ANTLR make
this easy.

Go wild,

Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.