Re: Validation Suite Report

linus!mbunix!eachus@harvard.edu (Eachus)
Tue, 30 Aug 88 10:28:45 EDT

          From comp.compilers
Related articles
Validation Suite Report dg-rtp!bert!wood@uunet.uu.net (1988-08-26)
Re: Validation Suite Report linus!mbunix!eachus@harvard.edu (1988-08-30)
Validation Suite Report dg-rtp!bert!wood@uunet.uu.net (1991-06-13)
Re: Validation Suite Report johnl@iecc.cambridge.ma.us (John R. Levine) (1991-07-02)
Re: Validation Suite Report moss@cs.umass.edu (1991-07-03)
| List of all articles for this month |
Posted-Date: Tue, 30 Aug 88 10:28:45 EDT
Date: Tue, 30 Aug 88 10:28:45 EDT
From: linus!mbunix!eachus@harvard.edu (Eachus)
Newsgroups: comp.compilers
In-Reply-To: <2332@ima.ima.isc.com>
Organization: The MITRE Corporation, Bedford, Mass.

>If you can add any information to this posting, please let me know.


>=============================== ADA ===============================


>ACVC - Ada Compiler Validation


          This is the official validation test suite. Currently about 4000
files, my guess is about 20000 actual tests. (Some tests must be in
multiple files, but many of the test programs contain multiple tests.
A real fun experience -- if you are into S&M.


>ACEC - Suite to test Ada performance.


          This is being developed by the Boeing Military Airplane Company
under government contract. First release is scheduled to occur any day
now. It will be distributed by the Data and Analysis Center for
Software (DACS) at Griffiss AFB, Rome NY.


>The University of Michigan has a suite of Ada tests and benchmarks.


          These are useful, but for the most part require intrumentation to
Get accurate timings. A better set of benchmarks is the ACM SIGAda
Performance Issues Working Group (PIWG) test suite, which is the suite
most vendors use when publishing perfomance figures.


>Don't forget the DoD Ada validation suite! It's supposed to be one
>of the most strenuous around, and is applied to all Ada compilers
>before they are allowed to use the name (since Ada is a registered
>trademark of the DoD, they can do that).


          There is no question that the ACVC is the most strenuous around,
and the standards are MUCH stricter than for other validations.
Before the validation you submit a list of tests which you think don't
apply to your validation and why. (Most of these are usually for
floating point precision beyond what the compiler supports.) With this
you provide a listing from running all of the tests, and a few other
things. (This is the infamous "box of paper".)


          If the validation agency approves your submission, they come on
site to verify the results themselves. Failing any test means that
you fail the validation. No second chances, no fixing bugs on the
spot. If you pass you get to do it all again next year.


          Incidently, Ada is no longer a registered trademark of the DoD.
The trademark was intentionally allowed to lapse last year as having
accomplished its purpose.


Robert I. Eachus
--

Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.