Re: lexing backwards

>Self Contained
>---------------
>
>These are tokens like the C language:
>
> -> ++ -- := ==
>
>That's to say no Delimiter is required to terminate such token. One may
also
>think of this class of tokens, as Multi-Character delimiters. Once found in
>the input outside the context of a Stream, they serve as unconditional
>terminator of the prior input, and are also tokens on their own.
>
> Counterexample:
>
> a/*comment*/=5;
> ^^
> If I follow you correctly "/=" is a self-contained token, but
> that is not how the above would be parsed in C/C++.
>
> [snip]


Please note that the text you quoted included this text:


-------------------------------------------------------------
Once found in the input outside the context of a Stream, they
                                                                        ^^^^^^^^^^^^^^^^^^^^^^^
serve as unconditional terminator of the prior input, and are
also tokens on their own.
-------------------------------------------------------------


Original post also included this:


-------------------------------------------------------------
[Comments may also be considered streams, though they
may be commonly handled at a pre-lexing statge.]
-------------------------------------------------------------.


Ron

Return to the comp.compilers page.
Search the comp.compilers archives again.