About task-specific programming languages

A friend of mine wrote on his blog about how he's working on a text
adventure, and I asked him whether he was using a general-purpose
programming language to develop the program or whether he was using a
text adventure development system. I pointed out to him about how the
applications which have been written to develop text adventures do
most of the "grunt work" of handling the overhead of running an
adventure game, and leave the game designer free to write the
descriptions and the puzzles and logic of the game itself, rather than
having to worry about the routines to handle the displaying of text,
and so on. You use a kind of programming language to describe the
world, including what text to display, what objects are present, what
commands are available and what those commands do.


It then kind of hit me. I've been involved in the development of
compilers for years, and I now understand the idea of such
compiler-writing tools as Yacc, Bison, and some of the others. I
never understood the reason for using them, they seemed more like
either toys that some people like to play around with, or some sort of
"ivory tower intellectual" construct that people in academia like to
write papers about and hype as the "next big thing" but not as
something that people in the "real world" can use to be able to get
"real work" accomplished.


Consider the world of operating systems; there are two schools of
thought on kernels (the base internal part of the operating system):
microkernel and monolithic kernel. In the academic world they've
more-or-less gone "ga-ga" over microkernels and small operating system
design. But for better than 99% of the operating systems people use in
computers, they use full-blown monolithic kernels because that is what
currently works successfully to get real-world work done.


A microkernel is where you have a very tiny operating system and
anything that isn't absolutely necessary for base machine operation
is, in effect, a loadable module, and thus you can pick-and-choose
what pieces to add or remove as part of the kernel. We have
microkernel-style when it comes to applications, you pick and choose
what to install on your computer. But when it comes to an operating
system, for real-world use, it's just one big program and it comes in
"one size fits all," thus the term "monolithic." (There has been some
movement in the microkernel direction with the idea of moving device
drivers out of the kernel and making them loadable modules, but not
much else.)


We also don't have any working microkernels available; the work on
HURD has been going on for years and if we had to wait for it (or some
other microkernel-based operating system architecture) to be
available, Microsoft Windows would have 96% of the operating system
market instead of probably about 84% (Apple has about 4%, Linux has
about 11%, and everyone else combined has maybe 1%). The point being
that in all of these cases they all use monolithic kernels,
microkernel architecture has (so far) failed to be anything but an
academic pipe dream.


Which brings me back to my original point about text adventure writing
systems. When you use a text-adventure programming system, you write
the specifications for the adventure, what commands it will accept,
what things will happen and so on. You don't worry about the
low-level details of how to parse what the user types-in, how to
implement objects in the programming language, how to turn various
switches on or off, the adventure system does all that for you. It
won't do all of the work, you still have to specify the logic
depending on what is happening in the game, but it does a very
important part of the job, freeing the programmer from handling the
overhead involved and allows him or her to focus on the more important
and critical parts of the work.


Which explains the use of compiler-writing tools and compiler
languages (a programming language designed specifically for specifying
the syntax and state of a computer program as opposed to a
general-purpose programming language). One uses these systems to
allow you to specify a programming language's syntax and how to
recognize the source language the compiler is supposed to understand.


Instead of writing the underlying code for creating a compiler, you
write the specifications for the language the compiler is going to
handle, and the compiler-writing tool handles the overhead and
bookeeping of what your language is supposed to specify. Again, it
won't do all of the work, you still have to specify the logic
depending on what a particular construct in that programming language
does, but the compiler-writing tools provide an important part of what
you have to do, freeing the programmer from handing the overhead
involved and allows him or her to focus on the more important and
critical parts of the work.


I think about the fact that I've been involved in compiler writing
(among other things) going back perhaps 20 years or more. And I never
thought about it - the realization never hit me - until now.


Joel Spolsky, who writes the Joel on Software blog, talks in one of
his articles about programming using pointers (a feature which is of
critical importance if you use C and C++) and how, it's one of the
things that he will have job applicants try doing a little as part of
their interview. Some people get pointers, and some don't. I mean, I
used to use them somewhat in building linked lists, but I never really
"got" them until one day, "the scales fell from my eyes" and I made
the connection. But it took me several years before I really
understood them.


Which also explains the development of some of the web-oriented
programming languages such as PHP, Perl, Python, and so on. These
languages have a large number of constructs designed specifically for
handling the tasks involved in solving the problems they target. And
what they do is something I've realized is extremely important in
programming; they increase the level of abstraction one can use and
thus make people more productive.


But then you can go even further, if you look at what is possible
using the Wikimedia system, the set of software modules that are used
to operate Wikipedia, that system itself supports a number of
programming constructs which are extremely useful for what they do,
allow the dynamic substitution of text depending on what is supplied
as a parameter. If you looked at the Wiki source of a complicated
Wikipedia page and translated it back into the PHP code it generates,
or if we went into what the PHP interpreter - written in C - executes
as a result of that PHP code, it would be a tremendous amount of work
to write directly, e.g. if it had to be written as programs executed
as CGI applications. Or even as the underlying HTML that is generated
by Wikimedia.


And if we go back to earlier times, we can look at the macro functions
in Lotus 1-2-3 as another task-specific language that was extremely
useful for the purpose it was intended: the manipulation of numbers
(and labels) for expressing a spreadsheet. I had just never thought
about task-specific programming languages as being significant or in
some cases more important than the full-blown "full service"
third-generation programming languages.


And so we've come full circle: at the application level, "micro
languages" or task-specific languages and programming systems such as
macro and scripting languages, Perl, PHP, Python, Ruby/Ruby on Rails,
Tcl, and SQL (and if I've left out your favorite, presume it's
included), are just as important for some problem domains as
"monolithic languages" (the third-generation languages) such as Basic,
C/C++, Cobol, Fortran, Pascal, etc.


I don't know why I never realized it before. But sometimes when you're
too close to the problem, you can't always see other things.


--
Paul Robinson - <a href="http://paul-robinson.us">My Blog</a>
"The lessons of history teach us - if they teach us anything - that
nobody learns the lessons that history teaches us."
[Actually, there are some perfectly good microkernels like QNX, but
they aren't freeware. Not sure what that tells us other than that
some kinds of tasks aren't easy to do in your spare time. -John]

Return to the comp.compilers page.
Search the comp.compilers archives again.