Re: lexing backwards

"Chris F Clark" <cfc@world.std.com> wrote in message
> There is no trick to lexing backwards. Lexing locally, however, is in
> general impossible.


Chris, I was in total agreement with you until I got to thinking about
comments. And in particular I am thinking about languages that do not
have a closing comment delimiter but use end-of-line as the closing
delimiter. Examples would be "//" in C++ and "#" in Perl.


So attempting to parse backwards would create tokens that must be
thrown away once you realize that you are in a comment. This doesn't
seem too bad for C++. But consider what happens with Perl. You may
not use quoted strings which contain an end-of-line in C++. (Note you
can include a new-line "\n" which is different from end-of-line). But
there is no such restriction in Perl. And the comment delimiter can
be put inside a quoted string.


So you would have to parse back to the beginning of the program before
you can decide that a single quote mark is part of a comment and not a
string delimiter.


This might sound unlikely, but Perl programmers often create Perl
programs that write Perl programs. So a large number of what may seem
like Perl tokens are actually only part of a string. And people do
put comments for the generated program inside this string.

Return to the comp.compilers page.
Search the comp.compilers archives again.