Re: Embeddable and Extensible Languages

On 25 Jan 2005 23:22:44 -0500, "PlayDough" <petela@gocougs.wsu.edu>
wrote:


>You mention syntax macros and input parsers. You seem to imply that
>it is possible to create my own syntax, and have it "converted" or
>"interpreted" by scheme. Is that correct?


That's exactly right. Scheme programs can generate and execute code
on the fly. Some implementations can only interpret such code, so if
the code is going to reused a lot or execution speed is important you
need to choose a version that can also compile on the fly.




>Could you expand on this a bit? Or point me to some good references?
>I've digged around a bit and found things such as "define-syntax name
>transformer-spec" and "let-syntax ((name transformer-spec) ...)
>body". Is this along the lines you are speaking of?


You're on the right track.


The syntax "transformers" are used to extend the language with custom
keywords and control constructs. They don't change the look however,
their input and ouput are both still Lispy.


As you typically want to control user interaction with the scripting
engine, the easiest route is simply to embed a parser for the input
syntax into your application. There are a number of good parsers
available for Scheme including general regex, LL and LR (lex/yacc
workalikes). Your program can take user input in whatever form it
likes, generate Scheme code and directly execute the results.


If you are very familar with Scheme (or Lisp), many implementations
allow customization of the system "reader", which is the standard
input facility. By manipulating the reader, you can input code
directly to Scheme using your favorite syntax rather than the normal
Lisp syntax. Typically this is used to created internal APIs or to
allow programming in domain specific terminology - AFAICT it isn't
something generally offered directly to users.


Reader customization is implementation dependent in Scheme. It's a
feature borrowed from Common Lisp, and most/all high quality Schemes
offer a way to do it ... but there is no standard for it yet. Most
are similar to CL but not all offer all the options. If you think you
want to be able to do this, you need to find out what your Scheme
allows and how exactly to do it.






As for Scheme references, you can check out comp.lang.scheme and visit
www.schemers.org. Schemers maintains a good list of available
versions, libraries and reference materials. As for implementations,
I would start by looking at some of these:


Chez - http://www.scheme.com/


Chez Scheme is a commercial native compiler version. There is also a
freely available threaded interpreter version called Petite Chez which
is fully compatible with the commercial version but does not include
the compiler.


PLT - http://www.plt-scheme.org/


PLT (aka DrScheme) is a free, mostly open source university teaching
version which was built around a core known as mzScheme. mzScheme is
a byte code compiler, is commercial grade, separable from PLT and
fully embeddable.


Guile - http://www.gnu.org/software/guile/
Elk - http://sam.zoy.org/projects/elk/
TinyScheme - http://tinyscheme.sourceforge.net/home.html


These are library implementations designed for scripting and
embedding. Guile and Elk are both well known embeddable versions,
TinyScheme is a well regarded experimental version.




Chez and PLT are both up to date and have extensive library support.
They are also most compatible with 3rd party code libraries you may
find. Scheme has no standard module facility so foreign libraries
usually need some spit and polish to run.


The other versions are lighter weight (by design) and don't come with
extensive libraries. They also tend to be a behind state of the art
and thus will have more gotcha's to porting code. However, they *are*
lighter weight and that may be important to your application.






Also I should mention that Common Lisp can do all of this as well and
you can pick up a lot of useful information on these techniques in
comp.lang.lisp. I didn't mention CL before it is not typically used
in an embedded setting (I am only aware of 2 embeddable versions) and
is much more heavyweight than Scheme. However, as a reference source,
there's a lot of mindshare common to both Scheme and CL and though
they are different languages, there is quite a bit of crossover in
applicable programming technique.


Hope some of this helps.
George

Return to the comp.compilers page.
Search the comp.compilers archives again.