Reliability (was: Re: Optimizing Across && And ||)

"Ronald F. Guilmette" <rfg@rahul.net>
Thu, 16 Mar 1995 18:46:33 GMT

          From comp.compilers

Related articles
Re: Optimizing Across && And || bart@cs.uoregon.edu (1995-02-23)
Re: Optimizing Across && And || pardo@cs.washington.edu (1995-03-13)
Reliability (was: Re: Optimizing Across && And ||) rfg@rahul.net (Ronald F. Guilmette) (1995-03-16)
| List of all articles for this month |

Newsgroups: comp.compilers
From: "Ronald F. Guilmette" <rfg@rahul.net>
Keywords: optimize, design, testing
Organization: a2i network
References: 95-02-179 95-03-078
Date: Thu, 16 Mar 1995 18:46:33 GMT

David Keppel <pardo@cs.washington.edu> wrote:
>
>Opinion alert:
>... [Stuff about why implementors spend so much time worring about their
> SPECmark numbers deleted]...
>
>I think we're starting to see changes here already, but it's an
>uphill battle to sell reliability...


Tell me about it! I sell compiler test suites for C and C++. It's a hard
sell, even at the best of times, even though I price these things dirt
cheap (relative to what it would cost for the compiler vendors to reproduce
even a small fraction of my test cases).


I even had a guy at one local semiconductor firm say to me once "If we buy
your test suite, then we will find some more bugs in our compiler. So what?"


I kid you not! This is really what he said.


>... which is unpleasant to talk about...


Compiler quality is only unplesant to talk about for those folks that don't
have it. (I could name names here, but I won't.)


>... and hard to measure.


Rubbish! I've developed sophisticated automated techniques for producing
randomized compiler test cases that root out even the most subtle compile-
time bugs. Free samples are available for anonymous ftp in the files:


ftp.rahul.net:pub/rfg/roadtest/sample.1
ftp.rahul.net:pub/rfg/roadtest/sample.2


If you want to see your favorite C compiler screw up and give errors on
some perfectly valid C code (and maybe even give up the ghost) just try
compiling these examples. They found bugs in every one of the 20+ C
compilers I've tried then on so far (although some verdors have since
fixed the bugs which are triggered by my test cases).


Suggestion: If you try these test cases on some C compiler(s), I suggest
disabling all warnings when doing the compiles. Any actual _errors_ that
you get will be due to compiler bugs.


>Performance is less hard (but not easy) to measure and obviously desirable.


I can't remember who said it, but I am often reminded of a wonderful quote:
"It is easy to get incorrect answers infinitely fast."
--


-- Ron Guilmette, Sunnyvale, CA ---------- RG Consulting -------------------
---- E-mail: rfg@segfault.us.com ----------- Purveyors of Compiler Test ----
-------------------------------------------- Suites and Bullet-Proof Shoes -
--


Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.