|Unit testing a compiler email@example.com (Rafael R. Sevilla) (2013-03-13)|
|Re: Unit testing a compiler firstname.lastname@example.org (BGB) (2013-03-13)|
|Re: Unit testing a compiler bobduff@shell01.TheWorld.com (Robert A Duff) (2013-03-13)|
|Re: Unit testing a compiler email@example.com (Walter Banks) (2013-03-19)|
|From:||Robert A Duff <bobduff@shell01.TheWorld.com>|
|Date:||Wed, 13 Mar 2013 18:58:42 -0400|
|Organization:||The World Public Access UNIX, Brookline, MA|
|Posted-Date:||13 Mar 2013 19:08:12 EDT|
"Rafael R. Sevilla" <firstname.lastname@example.org> writes:
> I am currently writing a simple byte compiler for a simple dialect of
> Lisp, and am wondering what is the best practice for testing such a
> compiler. It is certainly possible to run the compiler on some test
> code and then compare the code it generates with some reference code,
> but as the code samples become more complicated that rapidly becomes
Right, that won't work. (Unless you're testing an assembler,
which is a different story.)
>...Would it instead be better to test the compiler by actually
> running the code it generates, and then comparing the results with what
> the code snippet is supposed to evaluate to?
Yes, that's the right answer.
The specification of a high-level language says what programs are
supposed to do, not what machine code is generated by the compiler.
So test what they do -- compare outputs of compiled programs against
expected outputs. Any difference is a regression worth investigating.
Changes (such as new optimizations) will likely change the machine
code, but if the output of compiled programs is unchanged, that's OK.
>...Any other ideas on how to
> go about testing the code generator?
Lots, but too much to fit in the margin of this post. ;-)
Here's just one: Every time you fix a bug, add a regression test to
make sure that particular bug doesn't rear its ugly head again.
OK, one more: Every time one of your customers shows you some source
code, turn it into a regression test for your compiler. Get as much
test code as you can get your hands on.
>...What about when optimizations are
> being performed?
Test with optimization on and off.
Above is about testing that optimization (etc) didn't break something.
That's hard enough, but testing that "optimizations" don't actually
pessimize is even harder.
Return to the
Search the comp.compilers archives again.