Re: Context sensitive scanner ?

On 20 Nov 1997 22:38:40 -0500, Albert Theo Hofkamp
<hat@se-46.wpa.wtb.tue.nl> wrote:
[...]
>I am aware of the capability of creating states in Lex, but I do not
>really like it, since it heavily depends on the exact moment of executing
>when YACC statements (at the end of a rule).


Or you could switch to an LL(k) tool like PCCTS/ANTLR v1.x so you don't
have to worry so much about action placement.


>I'd rather want to know what tokens the parser is expecting, and recognize
>only those tokens (maybe with an exception on keywords) in the scanner.


You're asking for a predictive parser rather than a synthesizing parser.
Welcome to the wonderful world of LL. :-)




>I have never heard of such a tool, so my questions are:
>a) Does this exist, or I am trying something completely new here ?
>b) Are there tools (preferably compatible with Lex/Yacc grammar syntax) ?


I suggest http://java.magelang.com/antlr/


It's syntax is pretty different from lex & yacc but that's a good thing.




[...]
>[Well, disregarding the question of whether it's a good idea to write
>languages with lexical puns that practically beg people to write code
>that the compiler will misinterpret, I'd have the lexer return
>integers and dots as separate tokens and put reals together in the
>parser. -John]


In ANTLR v2.x, the lexing is not DFA-based but rather uses the same sort of
LL model as the parser. So, you can deal with such sick constructs right
in the lexer rather than pushing it up to the parser.


Hope this helps,
John
--

Return to the comp.compilers page.
Search the comp.compilers archives again.