Re: lexing backwards

"Ron Pinkas" <Ron@Profit-Master.com> wrote:


>>Self Contained
>>---------------
>>
>>These are tokens like the C language:
>>
>> -> ++ -- := ==
>>
>>That's to say no Delimiter is required to terminate such token. One may
>also
>>think of this class of tokens, as Multi-Character delimiters. Once found in
>>the input outside the context of a Stream, they serve as unconditional
>>terminator of the prior input, and are also tokens on their own.
>>
>> Counterexample:
>>
>> a/*comment*/=5;
>> ^^
>> If I follow you correctly "/=" is a self-contained token, but
>> that is not how the above would be parsed in C/C++.
>>
>> [snip]
>
>Please note that the text you quoted included this text:
>
>-------------------------------------------------------------
>Once found in the input outside the context of a Stream, they
> ^^^^^^^^^^^^^^^^^^^^^^^
>serve as unconditional terminator of the prior input, and are
>also tokens on their own.
>-------------------------------------------------------------
>
>Original post also included this:
>
>-------------------------------------------------------------
>[Comments may also be considered streams, though they
>may be commonly handled at a pre-lexing statge.]
>-------------------------------------------------------------.


          Take 2: (I still enjoy take 1, but my educated public demands
more sophisticated entertainment.) No comment, no comment:
                    a=b---c;
Parse that backwards. Again, self-contained in a forward direction,
but not in reverse where you still have to look at more characters
than are in the token to get the context. There is the danger of
backwards-lexing the above as:
                    a = b - -- c ;
when the correct lexing is:
                    a = b -- - c ;


          Corner cases have sharp points.


Sincerely,


Gene Wirchenko

Return to the comp.compilers page.
Search the comp.compilers archives again.