Re: Testing strategy for compiler

On Wed, Jun 16, 2010 at 07:18:20AM -0700, kuangpma wrote:
> Folks,
>
> Say I have written a hand crafted lexer/parser/code gen.... to make a
> complete compiler. The question is how to test it? Since user can have
> millions possible ways of writing their program (with many different
> types of syntax errors) and it is difficult to test all the possible
> cases. So is there any good ways to test the compiler? How do those
> big guys (MS/Borland...) tested their compiler? Thanks.


If you have a formal (generative) grammar for your language, you can
generate all valid strings of language and test whether your parser
recognises them. So you have practical tests which also cover obscure
programmes. You could use yagg [1] for this purpose.


With a formal description of your language (e.g. structural operational
semantics) you could use a theorem prover to test whether the generated
programmes are semantically correct. But I doubt that this process
can be fully automated and I think it might be not as useful with a
compiler since the compiler translates the programme and processor
executes it, so you can only test the result of the computation and
not the intermediate steps.


If you want to see realistic examples, you could have a look at Perl 5.
They have a big test suite with many regression tests.


Regards,
Matthias-Christian


[1] http://sourceforge.net/projects/yagg/
[Any useful language has an infinite number of valid strings. I suppose you could
generate enough to test that it can match each rule in the grammar. And this doesn't
address the harder question of verifying that it'll reject invalid programs. -John]

Return to the comp.compilers page.
Search the comp.compilers archives again.