Re: Testing strategy for compiler

On Jun 16, 10:18 am, kuangpma <kuang...@gmail.com> wrote:
> Folks,
>
> Say I have written a hand crafted lexer/parser/code gen.... to make a
> complete compiler. The question is how to test it? Since user can have
> millions possible ways of writing their program (with many different
> types of syntax errors) and it is difficult to test all the possible
> cases. So is there any good ways to test the compiler? How do those
> big guys (MS/Borland...) tested their compiler? Thanks.
> [Doing a thorough test of a manually written parser is extremely hard,
> particularly making sure that it rejects everything invalid that you
> might not have thought of. That's one of the main reasons to use scanner
> and parser generators, you can be reasonably confident that the language
> it accepts matches the grammar you gave it. -John]


There is essentially nothing different between testing a compiler and
testing any other program that includes fairly complicated algorithms.
All the usual procedures apply. Develop unit tests for each module.
Focus on edge cases. Start a regression testing mechanism along with
the first line of code; never throw away a test input, but add it to
the regression. Develop and check pre- and post-conditions within and
among modules. Insert code that checks them and never turn it off in
the production version unless absolutely necessary for performance
reasons. Develop consistency checks for internal data structures and
run them at strategic points as often as possible within peformance
constraints. Incorporate logging and internal state dump facilities
from the start of design, which can support all the above and helps by
making debugging more efficient.


The one advantage you have with a compiler is the possibility of large
amounts of existing source code with known functionality--effectively
tests that others have written for you! And of course if the compiler
is written in its own language, you have both a test and built-in
motivation toward correctness and higher performance.


I agree with John to a point. If e.g. you are hand coding a top down
recursive descent parser and follow the rules in a disciplined manner
for translating the LL grammar, then you are your own compiler
generator. You ought to get a correct parser if you can only suppress
the desire to be clever. Similarly if you write the lexer by sketching
a DFA and implementing it faithfully. You are your own scanner
generator, so you will get what you designed. The danger comes in
adding/modifying the rules in order to implement error reports and
recovery, improve efficiency, etc.


On the other hand, at least with yacc/bison where my experience lies,
getting good error handling seems often to require fiddling with the
grammar in ways as error prone as fiddling with the RD by-hand rules.
No doubt more modern tools are better.

Return to the comp.compilers page.
Search the comp.compilers archives again.