Re: CACM article (Feb 2009): "Compiler research: the next 50 years"

"Ira Baxter" <idbaxter@semdesigns.com> writes:


> As far as I can tell, far too many groups (not just compiler types)
> suffer from the Not Invented Here syndrome and *lots* of stuff gets
> reinvented. (We just had a discussion about a fellow that wants to
> produce yet another parser generator. To build a toy for education
> reasons is fine, but he seemed pretty serious about trying to build a
> real one. The point being .... what?) [I'll cheerfully acknowledge
> that as an implementer of what I think of as a "compiler framework"
> puts me directly in the NIH camp, too, but I'd like to think we added
> enough interesting ideas so NIH wasn't the point],


Yes, NIH seems to be a major problem. I'm not sure the root cause for
NIH is always ego. And that may go to the articles content--I don't
seemed to have that copy of CACM.


One problem that I've seen is the incomposability of software. We
didn't start out to write Yacc++. We wanted to use the output of yacc
to make a better syntax-directed mode from emacs, one where
forward-word meant forward token. (For those curious, one can read
about our prototype in the '86 Sigplan conference on interpreters.)
However, the output of yacc was too optimized with its comb-vector
packing for our use. That combined with the lack of ELR support,
caused us to write our own incompatible version, because we couldn't
decompoose the two parts (yacc and emacs) into reusable pieces and
recompose them as we needed. After realizing our initial goal was too
ambiitious, the parser generator is the portion that survived.


Unfortunately, I think that points in part to one source of the
problem. There is a certain size project a single person or small
team can undertake. A parser generator (at least the core
functionality) is a good example of that. The core of Yacc++ was
coded up (including the ELR features) in about 3 hours from a vague
recollection of the algorithms. That means you can take such a thing
and easily build another one customized to a specific purpose. On the
other hand the engineering to make Yacc++ useable and robust over a
wide class of uses has taken some 20 years and there are still parts
that aren't "quite right". Thus, a lot of people build their own tool
to solve some specific problem, but never get the tool to be useful
for (or often even useable by) others.


Unfortunately, I don't hold much hope for large infrastructure
projects. The following are my impressions: ELI didn't become a
standard. ANDF didn't get very far. SUIF looked promosing but seemed
to be bypassed by people working on "newer" stuff before it took
complete root. JVM seems to have some traction.


However, these infrastructure project don't produce more composable
software. Not in general. If you have something, you've done in the
ANDF framework, there is zero chance it will work out-of-the-box in
connection with the JVM.


On the other hand, add-on projects for tools like Bison don't seem to
be a solution either. I have read no end to papers which have
"solved" one problem or another with a custom version of Bison.
However, such improvements rarely seem to make it mainstream. At the
same time, there have been several forks that have appeared and died.


I don't have a solution. I tried offering the poster that started
this thread the use of Yacc++ as a framework to do his extensions,
with the promise that I would integrate them (and keep them working)
if they were suitable. And, I will make that a general offer to
potential authors who want to collaborate. (I have found a couple of
such candidates who I need to pursue a little more.) To my mind that
solves the fork problem as there is someone who reintrgrates the forks
and figures out how to compose them. I don't know if that will gain
any traction.


More opinions,
-Chris


******************************************************************************
Chris Clark Internet: christopher.f.clark@compiler-resources.com
Compiler Resources, Inc. or: compres@world.std.com
23 Bailey Rd Web Site: http://world.std.com/~compres
Berlin, MA 01503 voice: (508) 435-5016
USA fax: (978) 838-0263 (24 hours)

Return to the comp.compilers page.
Search the comp.compilers archives again.