Re: Compiler stress tests?

Cliff Click wrote:


> In short, we throw the kitchen sink at the compiler.
> Beta testers still find bugs with amazing ease.


There's a good reason for this. Simply throwing a large test suite
at a compiler isn't enough - you also have to have some way of
verifying that you have actually exercised the entire compiler.
Ideally you would verify that every possible line of code the compiler
could produce has been produced during a test suite run. In practice
this is pretty hard, but you can come close.


I don't believe the test suite vendors can help you with this -
it's very specific to the compiler internals.


It's quite frightening when you apply this kind of verification
to an existing compiler - we find it's normal that the test suite
has only been testing about 70% of the compiler! Of the balance,
some turns out to be unreachable, much proves to be correct, but
some is bound to be wrong.


The time spend on conducting this kind of thing is considerable -
but is repaid in spades in reduced tech support calls.
--
  Clyde Smith-Stubbs | HI-TECH Software, | Voice: +61 7 3354 2411
  clyde@htsoft.com | P.O. Box 103, Alderley, | Fax: +61 7 3354 2422
http://www.htsoft.com | QLD, 4051, AUSTRALIA. |
--

Return to the comp.compilers page.
Search the comp.compilers archives again.