|Re: Optimizing Across && And || firstname.lastname@example.org (1995-02-23)|
|Re: Optimizing Across && And || email@example.com (1995-03-13)|
|Reliability (was: Re: Optimizing Across && And ||) firstname.lastname@example.org (Ronald F. Guilmette) (1995-03-16)|
|From:||"Ronald F. Guilmette" <email@example.com>|
|Keywords:||optimize, design, testing|
|Date:||Thu, 16 Mar 1995 18:46:33 GMT|
David Keppel <firstname.lastname@example.org> wrote:
>... [Stuff about why implementors spend so much time worring about their
> SPECmark numbers deleted]...
>I think we're starting to see changes here already, but it's an
>uphill battle to sell reliability...
Tell me about it! I sell compiler test suites for C and C++. It's a hard
sell, even at the best of times, even though I price these things dirt
cheap (relative to what it would cost for the compiler vendors to reproduce
even a small fraction of my test cases).
I even had a guy at one local semiconductor firm say to me once "If we buy
your test suite, then we will find some more bugs in our compiler. So what?"
I kid you not! This is really what he said.
>... which is unpleasant to talk about...
Compiler quality is only unplesant to talk about for those folks that don't
have it. (I could name names here, but I won't.)
>... and hard to measure.
Rubbish! I've developed sophisticated automated techniques for producing
randomized compiler test cases that root out even the most subtle compile-
time bugs. Free samples are available for anonymous ftp in the files:
If you want to see your favorite C compiler screw up and give errors on
some perfectly valid C code (and maybe even give up the ghost) just try
compiling these examples. They found bugs in every one of the 20+ C
compilers I've tried then on so far (although some verdors have since
fixed the bugs which are triggered by my test cases).
Suggestion: If you try these test cases on some C compiler(s), I suggest
disabling all warnings when doing the compiles. Any actual _errors_ that
you get will be due to compiler bugs.
>Performance is less hard (but not easy) to measure and obviously desirable.
I can't remember who said it, but I am often reminded of a wonderful quote:
"It is easy to get incorrect answers infinitely fast."
-- Ron Guilmette, Sunnyvale, CA ---------- RG Consulting -------------------
---- E-mail: email@example.com ----------- Purveyors of Compiler Test ----
-------------------------------------------- Suites and Bullet-Proof Shoes -
Return to the
Search the comp.compilers archives again.