|Can Coco/R do multiple tokenizations email@example.com (2005-08-13)|
|Re: Can Coco/R do multiple tokenizations firstname.lastname@example.org (George Neuner) (2005-08-16)|
|Re: Can Coco/R do multiple tokenizations DrDiettrich@compuserve.de (Hans-Peter Diettrich) (2005-08-16)|
|Re: Can Coco/R do multiple tokenizations email@example.com (Gene Wirchenko) (2005-08-16)|
|Re: Can Coco/R do multiple tokenizations cfc@shell01.TheWorld.com (Chris F Clark) (2005-08-21)|
|Re: Can Coco/R do multiple tokenizations firstname.lastname@example.org (Darius Blasband) (2005-08-21)|
|From:||Gene Wirchenko <email@example.com>|
|Date:||16 Aug 2005 11:18:10 -0400|
|Posted-Date:||16 Aug 2005 11:18:10 EDT|
> Consider a langauage, which allows ! and = in its identifiers.
> Of course usual C operators like !,= etc are also allowed.
> Consder this string (note no whitespaces ):
> Valid tokenization/parsing can yield several posibbilityes
> 1. 'a!=b' .. a single token.
> 2. 'a!' '=' 'b' .. an assignment
> 3. 'a' '!=' 'b' .. an comparision.
> The accpetenace of a particular choice is influenced by
> 1. has a token already been defined (if 'a' and 'b' have been
> defined, than (3) gets priority )
What if a!b or a! have also been defined? In the latter case,
how does one assign the value of b to a!? (This is assuming that the
= operator in your language assigns right to left. (If you are going
to mess with other quasi-conventions, you might well mess with this.)
> 2. what are some tokens following this or preceding this string.
> .. if preceded by 'z =', than (3) can be omitted,
> .. if followed by '= z' then (1) is more probable.
I assume the spaces are significant. If not, there is even more
> In case of ambiguity I'ld idealy like to generate error and abort.
I know I would, at the language selection stage. It would
require force to get me to use a language like this. There is simply
too much room for error. C has
This is notably worse.
> Now can Coco/R, (can any other parser/lexer generator ) can do
> multiple tokenizations & parser-tree-generations , so that I can
>give a priotity to each of these three, and it can call my function
>to accept one over others.
Computerese Irregular Verb Conjugation:
I have preferences.
You have biases.
He/She has prejudices.
Return to the
Search the comp.compilers archives again.