Re: Compiler stress tests?

Clyde Smith-Stubbs (clyde@hitech.com.au) wrote:


: It's quite frightening when you apply this kind of verification
: to an existing compiler - we find it's normal that the test suite
: has only been testing about 70% of the compiler! Of the balance,
: some turns out to be unreachable, much proves to be correct, but
: some is bound to be wrong.


It can be very difficult to find a case that will test some parts of
the compiler, yet it may be very difficult to prove that those parts
cannot be used in practice. I had a case where there were a number of
different special cases of register allocations that had to be allowed
for in some operations. I could not find code that would cause all of
them to happen, nor prove that the ones not tried would never come up.
I also found that small changes to the register allocation algorithms
would change which special cases the test cases came up with. So even
if one had a set of tests that had covered them all, they might not do
so in a later version. Testing this code at module level could be done
but would be very difficult. All one could do was throw as many
different examples as possible at the compiler and hope that it
covered all the cases that would come up in practice.


It is important to take your test data from as many sources as
possible as different sources will follow different patterns. A good
general rule of computing is each additional user will reveal a new
class of bugs (because they will do something different from everyone
else).


--
Stephen Baynes MBCS CEng Stephen.Baynes@soton.sc.philips.com
Philips Semiconductors Ltd
Southampton SO15 0DJ +44 (01703) 316431
United Kingdom
--

Return to the comp.compilers page.
Search the comp.compilers archives again.