Re: split tokens; was LR(1) Parsing : Error Handling & Recovery

On 8/29/14 12:13 PM, Hans Aberg wrote:
> On 2014/08/26 12:44, Evangelos Drikos wrote:
>> ... What the Bison team said about your idea?
>>...
> ... we discussed split tokens, which it cannot currently handle
> ... The main focus is on computer languages, though it
> would be nice to be able to experiment with natural languages.


Split tokens are also useful for computer languages like SQL which have
unreserved keywords; yet, an efficient solution requires enough code.


Bison & Flex are fast and mature tools that can be combined efficiently
in a simple and straightforward manner for some computer languages.


But, Bison's GLR probably cannot parse some context free grammars. One
such epsilon grammar given by Nozohoor-Farshi[NF91] has broken Tomita's
original algorithm; a slightly modified grammar for Bison is given here:


G1: S
    ;
S: A S 'b'
    | 'x'
    ;
A: {printf("Delayed forever!\n");}
    ;


Given the string "xbbb", a Bison (2.3) GLR parser created by the above
grammar goes into an infinitive loop. Perhaps, this well known issue is
either solved with a hack or not perceived as a bug exactly because the
main focus is on computer languages.


Probably, one could find a plethora of other open source tools for NLP.


Regards,
Ev. Drikos

Return to the comp.compilers page.
Search the comp.compilers archives again.