Re: Preserve C/C++ comments in ANTLR v3

On Sun, 6 Jan 2008 18:02:27 -0500 (EST), Amal wrote:
> I need to extract comments from C/C++/Java source file. Normally
> comments are thrown away and they are defined as lexer rules
> (in ANTLR v3):


Is there a reason why you can't make the lexer capture the comments?
Then in the context where a comment may have some effect, check using
the parser action whether a comment was saved by the lexer.
If the comment always may have an action regardless of context, attach
the action to the lexer rule.


For example, in http://bisqwit.iki.fi/src/qbc.html, I used
the following lex rules (for QuickBasic):




\' { BEGIN(STATE_COMMENT); }


<STATE_COMMENT>{
    \$DYNAMIC {
        SetMetaDynamic(); /* Perform action */
        /* Don't return anything */
    }
    \$STATIC {
        SetMetaStatic(); /* Perform action */
        /* Don't return anything */
    }
    \$INCLUDE:\ *\' BEGIN(STATE_INCLUDE);
    \n+ BEGIN(INITIAL); Return CRLF;
    [^$\n]+ /* Ignore longest possible blobs until $ */
    . /* Ignore other */
}


And the parser remains oblivious to the fact that comments
were present, unless it chooses to check the status set by
those lexer actions.


--
Joel Yliluoma - http://iki.fi/bisqwit/

Return to the comp.compilers page.
Search the comp.compilers archives again.