Comments in a pre-compiler

I made a pre-compiler to add a couple of instructions I had in mind to
an existing language, basically to test whether they were usable or
not.


In trying to improve it I wanted to make the output nicecly human
readable. Up to then, the output of the pre-compiler worked nice for
the interpreter to understand ok, but if I wanted to read it, I had to
run a beautifier on it.


The problem came with comments. Comments are read and discarded by
the lexer, they never get to the parser. The parser and the lexer
often get out of synch with one another, the parser asking the lexer
to go ahead and then backtracking (I am probably not using the right
terms to refer to these actions, just take them in the more general
meaning). So, if I let the lexer let the comments through, by the
time the parser realises which code to output, there might be some
comments out ahead of the code. It is mostly noticeable with comments
interspersed within a statement.


So, I wanted to ask, is there a trivial solution which I am unawere
of, is it a non-trivial issue or it simply cannot be done? Any
references?


Thanks


Satyam
[It's a difficult problem to do well. A simple approach is to hang
the text of each comment on an adjacent token, and then output the
comment when you output the token. It's not foolproof, e.g., if the
token is duplicated or deleted, but it's easy. -John]

Return to the comp.compilers page.
Search the comp.compilers archives again.