Re: Parallelizing C/C++ code

On Mon, 03 May 2010 04:09:41 -0700, Raphael Fonte Boa wrote:


> On May 1, 7:00 am, russell kym horsell <k...@sdf.lonestar.org> wrote:
>> According to google there are 4.4 million hits for "parallelizing
>> compiler". :) So you know it's a huge area. Perhaps start with
>> wikipedia for an introduction.
>
> Hi Russel,
> Thanks for googling it for me :)
>
> Nevertheless, I think the problem for me lies more in the analysis area.
> Are the analysis for parallelism worth the effort for compilers
> technology? Googling for a tool that accomplishes such parallelization
> gives no result. I therefore imagine that it has no simple solution.
>
> Since I'm no compilers researcher I thought of asking here, maybe I
> could get an answer like:
> It has not been adopted by most compilers because it is not worth it,
> does not bring benefits, too complex or whatever.
>
> I know that it is a very broad area, and I don't think that some google
> searches will give me something I can use to answer my question.




The best evidence for whether a compiler feature is worth implementing
is whether it sells. Does the marketplace demand it? Does it make
one compiler outsell another? I've seen compilers that implicitly
split loops across multiple cores and shared memory, but parallelism
more ambitious than that has been left to the programmer.


Some of Intel's compilers attempt more paralleliztion than most because
the Itanium is so much more dependent on the compiler for hiding
latency. But when HP finally pulls the plug on the IA64 instruction set,
I doubt they'll continue to invest as heavily in compiler-driven
concurrency. There was a time when Cray too bought into autoparallelism,
but with the company's recent spate of insolvencies and buyouts, the
writing is on the wall for HPC in general -- it doesn't sell either.


With the the advent of larger-than-4x core CPUs or GPUs, will compiler
vendors implement novel forms of parallelism beyond loop splits or SSE
instructions? I doubt it. I've seldom seen cases where more than 8 SMP
cores can add further speedup to task auto-parallelism. Further speedup
requires explicit programmer intervention. Given the ubiquity of data
interdependency in serial languages like C/Java/C++, I think it's
unlikely that auto-parallelism will ever benefit from more than 4-8 cores/
processes. And GPUs will never benefit from auto-parallelism. They're
fundamentally data-parallel, which adds value only when you operate very
regularly on large blocks of data without interruption -- not something
that can ever arise automatically in languages like C/fortran/etc.


Is there a market for autoparallelism on hardware other than RISC CPUs
like System on Chip systems? Maybe, but I doubt it. These serve
embedded hardware markets which are usually programmed using specialized
fine-grain parallel libraries/hardware. Until the average programmer
starts to program embedded hardware for high performance, the
autoparallelism compiler market will remain unprofitably small.


A final thought. There are lots of research papers and textbooks on the
topic of autoparallelzing compilers. Academically it sells. But when it
comes to its day-to-day utility, the crux of the biscuit is whether
autoparallelism is valued by IT and business. Does It Sell?


        Randy

Return to the comp.compilers page.
Search the comp.compilers archives again.