Re: Java Comment-Preserving Grammar

Dobes Vandermeer <dobes@dobesland.com>
30 May 2004 13:21:02 -0400

          From comp.compilers

Related articles
Java Comment-Preserving Grammar matthew-google@faredge.com.au (2004-05-24)
Re: Java Comment-Preserving Grammar sreeni@viswanadha.net (Sreenivasa Viswanadha) (2004-05-30)
Re: Java Comment-Preserving Grammar martin@cs.uu.nl (Martin Bravenboer) (2004-05-30)
Re: Java Comment-Preserving Grammar cdodd@acm.org (Chris Dodd) (2004-05-30)
Re: Java Comment-Preserving Grammar dobes@dobesland.com (Dobes Vandermeer) (2004-05-30)
RE: Java Comment-Preserving Grammar matt@faredge.com.au (Matthew Herrmann) (2004-05-30)
Re: Java Comment-Preserving Grammar tbauer@cadrc.calpoly.edu (Tim Bauer) (2004-05-30)
Re: Java Comment-Preserving Grammar jens.troeger@light-speed.de (2004-06-06)
Re: Java Comment-Preserving Grammar clint@0lsen.net (Clint Olsen) (2004-06-06)
Re: Java Comment-Preserving Grammar gah@ugcs.caltech.edu (glen herrmannsfeldt) (2004-06-09)
Re: Java Comment-Preserving Grammar cfc@shell01.TheWorld.com (Chris F Clark) (2004-06-11)
[3 later articles]
| List of all articles for this month |

From: Dobes Vandermeer <dobes@dobesland.com>
Newsgroups: comp.compilers
Date: 30 May 2004 13:21:02 -0400
Organization: Compilers Central
References: 04-05-075
Keywords: Java, parse
Posted-Date: 30 May 2004 13:21:02 EDT

On 24 May 2004 00:32:15 -0400, Matthew Herrmann
<matthew-google@faredge.com.au> wrote:


> Hi All,
>
> Is there an available Java grammar which preserves comments in an
> output AST tree? I know there are plenty of java grammars out there,
> but all ignore comments. I'm happy to use any tool providing I can
> gain access to the comments in the tree.
>
> My end goal is to parse commented-out extensions to the java language:
>
> class /*#immutable*/ Blah {
>
> public /*#input*/ int x;
> public /*#result*/ int y;
>
> }
>
> Any suggestions appreciated,


A simple thing to do would be to change the lexer grammar so that it
treated your special directive comments as non-comments, and change the
java grammar to correctly handle these in the places you want them.


A more flexible approach is to change the lexer so that comments are
attached to each token. You can add two fields to your lexer result
called "post_comment" and "pre_comment" (each comment would be referenced
to by two tokens). Then you can propagate this information so its still
available when the AST is finished contructing.


Implementing this in a grammar can be very complex; there are many
possible uses of comments in an expression:


a + /*var b*/ b


a + b /* sum */


a /* lets do a sum */ + b


This means your grammar has to take on a form like:


ADDITION: ( comment ) ? expr ( comment ) ? '+' ( comment ) ? expr
( comment ) ?


Which is a lot of work for very little gain.


CU
Dobes


Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.