Re: Context sensitive scanner ?

johnm@non.net (John D. Mitchell) writes:
[...]
>> Or you could switch to an LL(k) tool like PCCTS/ANTLR v1.x so you don't
>> have to worry so much about action placement.


Albert Theo Hofkamp <hat@se-46.wpa.wtb.tue.nl> wrote:
>Sorry, I should have stated my concern more clearly. If I would
>introduce states in the scanner such that the scanner would return
>only the `right' tokens, and the control of the state is done from
>within yacc, then I consider the moment when the parser executes the
>change of state as tricky.


Hmm... If I understand you correctly, I did not intend to imply the
use of parser feedback into the scanner is any sense a good solution.
On the contrary, I think it's a bad idea. If you're going to insist
on doing anyways, then using an LL parser rather than an LR parser
will make things much less tricky.




>I do not understand why LL(k) would solve this problem. It is true
>that it is easier to predict which action is executed, but the exact
>moment is still quite dependent on the implementation imho.


Huh? LL parsers don't have the fragility that LR parsers have
w.r.t. action placement. That makes action firing basically trivial to
predict in LL parsers.




[...]
>> You're asking for a predictive parser rather than a synthesizing parser.


>No, I am asking for a scanner which tries to do interpretation within
>the limits of what the parser expects. In other words, when the parser
>doesn't want a real, and the scanner gets a real as input, it will not
>return a real, but 2 integer numbers seperated by a dot.


There is an ambiguity in your language specification. You seem to
want some magical code to guess how to do the disambiguation for you.
Good luck. :-) You're going to have to figure out how and where you
can best add the appropriate amount of intelligence to your system to
disambiguate.


If it's possible to lexically disambiguate (due to e.g., lexical
context) then you're set (as you can do everything in the lexer
without hacking the parser at all). This works great for e.g., the
various uses of '.' in C. ANTLR v2.x provides both syntactic and
semantic predicates in the lexer which can make this much easier to do
than in regexp based tools.


Otherwise, if you have to rely on parser state to disambiguate then I
suggest you follow the moderator's advice and just return the
constituent components and combine them as appropriate in your parser.
Again, the semantic and syntactic predicates available in ANTLR make
this easy.


Go wild,
John
--

Return to the comp.compilers page.
Search the comp.compilers archives again.