lexing backwards

"Stefan Monnier" <monnier+comp.compilers/news/@rum.cs.yale.edu>
5 Apr 2003 15:13:32 -0500

          From comp.compilers

Related articles
lexing backwards monnier+comp.compilers/news/@rum.cs.yale.edu (Stefan Monnier) (2003-04-05)
Re: lexing backwards haberg@math.su.se (2003-04-07)
Re: lexing backwards cfc@world.std.com (Chris F Clark) (2003-04-07)
Re: lexing backwards maratb@cs.berkeley.edu (Marat Boshernitsan) (2003-04-07)
Re: lexing backwards stan@zaborowski.org (Stan Zaborowski) (2003-04-13)
Re: lexing backwards Ron@Profit-Master.com (Ron Pinkas) (2003-04-13)
Re: lexing backwards monnier+comp.compilers/news/@rum.cs.yale.edu (Stefan Monnier) (2003-04-15)
[7 later articles]
| List of all articles for this month |

From: "Stefan Monnier" <monnier+comp.compilers/news/@rum.cs.yale.edu>
Newsgroups: comp.compilers
Date: 5 Apr 2003 15:13:32 -0500
Organization: Compilers Central
Keywords: lex, question
Posted-Date: 05 Apr 2003 15:13:31 EST

Could anyone point me to work and experience on lexing text locally
and backwards ?

Traditionally lexing is done left-to-right with longest-match regexps.
But such a lexing system obviously means that if you're in the middle
of a file and want to know what is the previous lexing token, you end
up (in the general case) having to lex from the beginning of the file.

Assuming one wants to be able to do such backward lexing and that one
wants to get the answer locally (without having to go back to the
beginning of the file), one needs to put some restrictions on the kind
of tokens (use shortest-match regexp, or use a subset of regexps) one
might accept, or one might just want to add some heuristics, or else
use caching, or ...

I can think of a few different things to do, but I'm hoping someone
out there has some experience with it.


PS: This is in the context of basic Emacs support for major modes, so it
        is not specific to a particular grammar although it's OK if it
        does not cover all possible language syntaxes.

Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.