What do benchmarks measure? (WAS: complaints about SPEC ...)

pardo@cs.washington.edu (David Keppel)
Sun, 14 Mar 1993 04:30:56 GMT

          From comp.compilers
Related articles
Some complaints about SPEC and a request for FORTRAN validation codes moudgill@cs.cornell.EDU (Mayan Moudgill) (1993-03-06)
Re: complaints about SPEC and a request for FORTRAN validation codes grover@brahmand.Eng.Sun.COM (1993-03-12)
What do benchmarks measure? (WAS: complaints about SPEC ...) pardo@cs.washington.edu (1993-03-14)
| List of all articles for this month |
Newsgroups: comp.benchmarks,comp.compilers
From: pardo@cs.washington.edu (David Keppel)
Keywords: benchmarks, standards
Organization: Computer Science & Engineering, U. of Washington, Seattle
References: 93-03-020 93-03-034
Date: Sun, 14 Mar 1993 04:30:56 GMT

grover@brahmand.Eng.Sun.COM (Vinod Grover) writes:
>[SPEC benchmarks measure *system* not *cpu* performance.]


According to Cmelik, et. al, the older SPEC benchmarks (what was available
when the study was done) issue 5% of their instructions in libraries on a
MIPS platform and 9% on a SPARC platform. Most of the discrepancy can be
attributed to MIPS-platform `malloc' and `free' routines that optimize for
speed and SPARC-platform routines that optimize for space, to block memory
operations (memset, mcopy, memcopy, bzero, bcopy) that have more efficient
implementations on the MIPS, and to the need for (existing) SPARCs to call
library routines to perform multiplies. The first two are essentially
system and not CPU issues.


;-D on ( Doing benchmark presses ) Pardo


%A Robert F. Cmelik
%A Shing I. Kong
%A David R. Ditzel
%A Edmund J. Kelly
%T An Analysis of MIPS and SPARC Instruction Set Utilization on the
SPEC Benchmarks
%J Proceedings of the 1991 Symposium on Architectural Support for
Programming Languages and Operating Systems (ASPLOS)
%D 1991
--

Post a followup to this message

Return to the comp.compilers page.
Search the comp.compilers archives again.