javaCC problem - special tokens handling

Hello,


My grammar contains some tokens that shall be interpreted as 'skipped'
tokens - so they may appear between any pair of 'ordinary' tokens.
For each skipped token, there is a special action that shall be
performed. Javacc allows this by using 'lexical action', but it has
one significant misfeature - the action is performed after the token
has been scanned in by the token manager but not when the token is
'consumed' by the parser. It means that the lexical action can be
performed during lookahead.


So if I implement some action for this 'skipped' tokens, the action is
performed earlier than it should be.


Does anyone know how to implement 'skipped tokens' with actions
invoked at the moment of parsing, but not at the moment of token
creation?


The grammar below contains an example:


Input is


a e f b e d


where 'e' and 'f' are 'special' tokens.


During to lookahead, the methods are called in this order:


actionE() actionF() actionE() doA() doB() doD()


but natural order would be


doA() actionE() actionF() doB() actionE() doD()


How can I achieve it?


I know one solution - patching of javacc-generated parser
(method jj_consume_token()), but I'd like to solve the problem in more
conventional fashion.


Thanks a lot,
Sergey Kuksenko.


P.S. This problem appears when parsing '#pragma', '#line' and other
directives in C/C++ parser


------------------- the grammar: -------------------------


SKIP: { " " | "\n" | "\r" }


SPECIAL_TOKEN:
{
        <"e"> { actionE(); }
    | <"f"> { actionF(); }
}


void Input():{}
{ (Input1())* }


void Input1():{}
{
  ( LOOKAHEAD(3)
      "a" {doA();} "b" {doB();} "c" {doC();}
  | "a" {doA();} "b" {doB();} "d" {doD();}
  )
}


----------------------- input: ----------------------------


a e f b e d


----------------------- output: ---------------------------


actionE() actionF() actionE() doA() doB() doD()


----------------------- desired output: -------------------


doA() actionE() actionF() doB() actionE() doD()

Return to the comp.compilers page.
Search the comp.compilers archives again.