Re: coupling LALR with a scanner?

"Armel" <armelasselin@hotmail.com>
Sun, 2 Oct 2011 16:41:08 +0200

          From comp.compilers

Related articles
[7 earlier articles]
Re: coupling LALR with a scanner? paul@paulbmann.com (Paul B Mann) (2011-09-17)
Re: coupling LALR with a scanner? armelasselin@hotmail.com (Armel) (2011-09-19)
Re: coupling LALR with a scanner? paul@paulbmann.com (Paul B Mann) (2011-09-19)
Re: coupling LALR with a scanner? armelasselin@hotmail.com (Armel) (2011-09-20)
Re: coupling LALR with a scanner? cdodd@acm.org (Chris Dodd) (2011-09-23)
Re: coupling LALR with a scanner? cfc@shell01.TheWorld.com (Chris F Clark) (2011-09-29)
Re: coupling LALR with a scanner? armelasselin@hotmail.com (Armel) (2011-10-02)
Re: coupling LALR with a scanner? cfc@shell01.TheWorld.com (Chris F Clark) (2011-10-03)
Re: coupling LALR with a scanner? gah@ugcs.caltech.edu (glen herrmannsfeldt) (2011-10-03)
| List of all articles for this month |

From: "Armel" <armelasselin@hotmail.com>
Newsgroups: comp.compilers
Date: Sun, 2 Oct 2011 16:41:08 +0200
Organization: les newsgroups par Orange
References: 11-07-013 11-07-015 11-07-018 11-08-004 11-09-016 11-09-017 11-09-022 11-09-023 11-10-003
Keywords: parse
Posted-Date: 02 Oct 2011 22:48:53 EDT

> there is an argument for introducing such a gap to "segment"
> tokens into smaller chunks for both performance and expressibility
> reasons.


could you elaborate on this segmentation mechanism?


In my lexer generator, the developer can introduce start states by himself
and 'cut' complex expressions into smaller expressions which still respect
AFD capabilities and introduce dynamic regular expressions where absolutely
necessary, for languages allowing dynamic string delimiters for example
(e.g. doc-strings like << END_OF_STR_MARKER, some lines then a line with
END_OF_STR_MARKER only on the line), languages such as ruby are very funny
from that point of view if I remember well.


Regards
Armel



Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.