Re: Java Comment-Preserving Grammar

Clint Olsen <clint@0lsen.net>
6 Jun 2004 15:24:52 -0400

          From comp.compilers

Related articles
Java Comment-Preserving Grammar matthew-google@faredge.com.au (2004-05-24)
Re: Java Comment-Preserving Grammar sreeni@viswanadha.net (Sreenivasa Viswanadha) (2004-05-30)
Re: Java Comment-Preserving Grammar martin@cs.uu.nl (Martin Bravenboer) (2004-05-30)
Re: Java Comment-Preserving Grammar cdodd@acm.org (Chris Dodd) (2004-05-30)
Re: Java Comment-Preserving Grammar dobes@dobesland.com (Dobes Vandermeer) (2004-05-30)
Re: Java Comment-Preserving Grammar tbauer@cadrc.calpoly.edu (Tim Bauer) (2004-05-30)
Re: Java Comment-Preserving Grammar jens.troeger@light-speed.de (2004-06-06)
Re: Java Comment-Preserving Grammar clint@0lsen.net (Clint Olsen) (2004-06-06)
Re: Java Comment-Preserving Grammar gah@ugcs.caltech.edu (glen herrmannsfeldt) (2004-06-09)
Re: Java Comment-Preserving Grammar cfc@shell01.TheWorld.com (Chris F Clark) (2004-06-11)
Re: Java Comment-Preserving Grammar alexc@std.com (Alex Colvin) (2004-06-13)
Re: Java Comment-Preserving Grammar gah@ugcs.caltech.edu (glen herrmannsfeldt) (2004-06-13)
Re: Java Comment-Preserving Grammar cfc@shell01.TheWorld.com (Chris F Clark) (2004-06-15)
| List of all articles for this month |

From: Clint Olsen <clint@0lsen.net>
Newsgroups: comp.compilers
Date: 6 Jun 2004 15:24:52 -0400
Organization: Comcast Online
References: 04-05-075
Keywords: parse, Java
Posted-Date: 06 Jun 2004 15:24:52 EDT

On 2004-05-24, Matthew Herrmann <matthew-google@faredge.com.au> wrote:
>
> Is there an available Java grammar which preserves comments in an output
> AST tree? I know there are plenty of java grammars out there, but all
> ignore comments. I'm happy to use any tool providing I can gain access to
> the comments in the tree.


The most straightforward way is to take a grammar and add the comment
annotation in the lexer as someone else suggested in this thread. We
are using such a technique on a language translator which is not only
attempting to preserve comments but also preprocessor constructs. Due
to time constraints we are using Parse::Yapp from the Perl CPAN site.
This is a pretty faithful re-implementation of yacc in Perl.


We use special fields in a token to flag whether or not a comment or
pre-processor construct is tied to this token, and the 'pre' tokens
should be handled first and 'post' operations should be handled
following. For comments, you don't really need 'post' field support
except when handling comments at the end of file condition:


....


a = b;


/* comment here */


In this example, 'comment here' would have to be a 'post' on the ';' token.
You could argue that EOF comments should be dropped in your case anyway.


If you have any more questions, please feel free to conact me.


-Clint


Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.