|For Review: C++ Performance Benchmarks email@example.com (Chris Cox) (2008-05-05)|
|From:||Chris Cox <firstname.lastname@example.org>|
|Date:||Mon, 05 May 2008 17:25:07 -0700|
|Keywords:||C++, benchmarks, available|
|Posted-Date:||05 May 2008 23:15:37 EDT|
I would like to get your help in reviewing the initial release of an
open source C++ Performance Benchmark.
These benchmarks will test compiler optimizations, language features,
common programming idioms, as well as some compiler runtime and OS
This is not SPEC or any sort of executive summary benchmark. This is
about details: what is the penalty for using ___, does my compiler
optimize ____, etc.
The primary goals of these benchmarks are:
1) To help compiler vendors identify places where they may be able to
improve the performance of the code they generate.
2) To help developers understand the performance impact of using different
data types, operations, and language features with their target compilers
The initial tests included in this release are:
Test to see if compilers will correctly unroll loops to hide
instruction latency. Some compilers have problems expanding the
templates, and most compilers tested have problems correctly
unrolling the loops for best performance.
A test to see if the compiler will move loop invariant calculations
out of the loop. Most compilers I tested have room for improvement.
An expanded version of the original abstraction penalty test,
answering "what happens to performance when I wrap a value in curly
braces?" Almost all compilers do well on the original summation
tests, but they don't do nearly so well on simple sort routines
using the same abstractions.
A utility program to print out information about the compiler
version, OS, and machine environment.
You can download the source from
I'll be posting updates and news on
http://stlab.adobe.com/performance/index.html and blogging results at
Where I need help:
More eyes checking the code for correctness.
Double checking the optimization tests, to verify that they really
test what they claim to test and don't run afoul of other
optimizations. The tests may expose bugs other than what they
are intended to test, but that's ok (as long as you fix the bugs).
Help testing and improving the portability of the code and
Help identifying and reporting more information about platforms
and compilers in machine.cpp. This is important in trying to
differentiate reports from different builds, compiler options, etc.
This will be even more important when running against a one-off
build of your own compiler on one of the 30 machines in your
What is still in progress:
More tests, of course. Suggestions are welcome.
Extracting conclusions from the raw numbers.
I want results from more platforms before nailing down
Completing the web site with results, conclusions, forum,
documentation, references, etc.
A public perforce depot for the current release source and all work
Thank you for your assistance,
(yeah, our internal netnews server is down...)
Return to the
Search the comp.compilers archives again.