Re: complaints about SPEC and a request for FORTRAN validation codes

moudgill@cs.cornell.EDU (Mayan Moudgill) writes:
>First, a caveat: I've only used the earlier version of SPEC, so I'm not
>sure my comments apply to CFP92.
>
>_BUT_
>
>Are the FORTRAN programs in SPEC supposed to be Fortran77 compliant? ...


No, the SPEC benchmarks are not guaranteed to be Fortran77 compliant.


>From the SPEC CFP92 Run Rules:
o Source code changes required for standards
                                  compliance should be described in the testing
                                  report. Appropriate standards documents should be
                                  cited. All such changes should be reported to
                                  SPEC. SPEC may consider incorporating such
                                  changes in future releases. Whenever possible,
                                  SPEC will strive to develop and enhance benchmarks
                                  to be POSIX and ANSI compliant.


SPEC's goal was to take existing programs/applications that people use and
develop workloads for them to provide a common reference point for
evaluation.


While a goal of compliance is note-worthy, three points were noticed:


- Making code compliant can be a lot of work (particularly when the authors
    of the code are not involved).
- These are benchmarks aimed at measuring CPU performance (not compiler
    compliance). It seemed realistic to include non-compliant code in the suite.
- Where is the line drawn at compliance? The goal was to reach the largest
    possible audience.


As mentioned in the Run Rules, SPEC is aware of the standards and as
applications and code move toward standard compliance so will the
benchmarks. If you have any input, please let SPEC know.


As mentioned, the SPEC suites were not targeted as a test of compiler
compliance. If the discrepencies with a standard are useful pieces of
information to have or you can provide that, let SPEC know
(spec-ncga@cup.portal.com or jwreilly@mipos2.intel.com).


Jeff
SPEC CINT92/CFP92 Release Manager


Jeff Reilly, Intel Corporation
jwreilly@mipos2.intel.com
(408) 765 - 5909
--

Return to the comp.compilers page.
Search the comp.compilers archives again.