Can Coco/R do multiple tokenizations

Hi,


    Consider a langauage, which allows ! and = in its identifiers.
    Of course usual C operators like !,= etc are also allowed.
    Consder this string (note no whitespaces ):
              'a!=b'
    Valid tokenization/parsing can yield several posibbilityes
    1. 'a!=b' .. a single token.
    2. 'a!' '=' 'b' .. an assignment
    3. 'a' '!=' 'b' .. an comparision.




  The accpetenace of a particular choice is influenced by
      1. has a token already been defined (if 'a' and 'b' have been
                          defined, than (3) gets priority )
      2. what are some tokens following this or preceding this string.
                      .. if preceded by 'z =', than (3) can be omitted,
                      .. if followed by '= z' then (1) is more probable.


    In case of ambiguity I'ld idealy like to generate error and abort.


    Now can Coco/R, (can any other parser/lexer generator ) can do
  multiple tokenizations & parser-tree-generations , so that I can
give a priotity to each of these three, and it can call my function
to accept one over others.


--TIA,
--Vardhan

Return to the comp.compilers page.
Search the comp.compilers archives again.