Re: Reference source for C-compiler validation?

"Tzvetan Mikov" <ceco@no_smap_jupiter.com> wrote in message news:01-11-062...
> > Plum Hall: http://www.plumhall.com/suites.html
> > Perennial: http://www.peren.com/pages/cvsa_set.htm
> Does anyone have an approximate idea how expensive those are? There is
> no information on the websites (at least I couldn't find it) and
> didn't dare to e-mail their sales just to satisfy my silly curiosity.


At a previous company, we bought both of these, and I was amazed at
how cheap they were. I think they were both in the tens of thousands
of US dollars. (ie ?0,000 US$). At the time, I thought developing a
similar set of tests would take a couple of person years, so it was a
good deal. If I had to pinch pennies, I'd only get one of them. But
if you're goal is high quality software, get both.


> Another question: how effective are these suites? I mean, ANSI C/C++
> compliance problems are found in compilers all the time and I would
> assume that at least the major vendors can afford to buy the test
> suite.


I think they will find dozens, if not hundreds of bugs in your parser,
so the cost is maybe $1000, but more likely, $100 per bug. Cheap,
IMHO. That fact that people ship compilers with bugs, does not mean
that commercial compiler test suites are not cost effective.


> Also, are the suites attempting to validate the generated code


I don't remember. We didn't generate code, so didn't care about that
part.


> - I realize that this is very difficult and probably unrealistic to
> expect, but as the initial poster suggested, they could run "an
> algorithm with known results so that the validity of the created
> executable can be tested". Do you have an idea whether such a test
> would be useful in practice at all (beyond the very first stages of
> validating a compiler)?


Is this just running programs compiled by your compiler as part of the
test suite, and making sure that when run, they produce the expected
output?


Joshua Levy

Return to the comp.compilers page.
Search the comp.compilers archives again.