Re: Languages with well-integrated Foreign Function Interface to learn from?

> In my opinion, the style of FFI used by scripting languages (Perl,
> python, lua, PHP, Ruby) is awful, and should never be emulated. It
> is particularly bad since most standard libraries are written using
> the FFI (as opposed to being written in user-code), which cannot be
> reused by other implementations of the scripting language.


agreed...




At least luckily the JVM (and to a lesser extent, Mono), implement
much of their libraries in Java (and C#, although, from what I can
tell, it is not as good in Mono since a lot of stuff is thinly wrapped
C libraries, meaning that one left cloning the project would likely
either have to clone a lot of the C libs, or being stuck using many of
the same libraries...).


As I see it, the ideal would be to implement a lot of the
functionality in terms of the language itself, and what things do
depend on the host VM or OS, would be done via abstract calls.




for example:
a GUI framework is mostly written in the language, and defers to the host VM
mostly for low-level calls (such as drawing operations, ...);
or, on the other extreme, the API can be almost entirely in the VM (and
largely independent of said VM), and at the script level this is mostly
exposed as an abstract API (similarly allowing clone implenentations to
implement it as they see fit...).


however, "half-and-half" implementations are less ideal, namely where the
GUI implementation (within the specific language) is a big mess of special
cases for wrapping various C-level API's (such as X11, GTK, Win32, ...).


forcing one, if they want to reuse it, to both use the supported GUI
frameworks, and also to implement the original VM's specific mechanisms for
wrapping said frameworks.


I have seen some of this before, it is ugly...




actually, related to this:
recently there have been some mild thoughts about having a GUI split into
several pieces:
a frontend API, a protocol, and a backend.


this would then be sort of like an X11 server, but focused more on the level
of GUI's and widgets...
this protocol would then be sent over a file-like stream (possibly a socket,
or a stream internal to the process).


the main point here is that then the GUI would be more easily redirectable
(without having to rewrite or recompile the code). the GUI could be plugged
into a GL-based drawing backend, GTK, ... (in my case, I would probably use
GL possibly, and have been partly considering using render-to-texture for
GUI purposes...).


but, alas, all this is going too far off topic...






> Consider PHP. Its FFI is simply an API which exposed quite a lot of
> the internals of the PHP interpreter. If you wish to provide another
> implementation of PHP, you must copy or wrap the API (which is the
> approach of Phalanger, a .Net version of PHP), ignore it (Roadsend)
> and miss out on or reimplement all the libraries, or try horrendous
> preprocessor and linker tricks to provide the same API (ProjectZero).
>
> Every scripting language I've come across has the same style, as the
> cited paper describes (although Ruby has at least put thought into
> their API, and made it quite attractive). Some have come up with
> better approaches, but they are not standard (Ruby's libFFI
> [http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html] or
> Python's cython/pyrex [http://cython.org/])
>
>
> In my opinion, the best approach is to provide a
> domain-specific-language which allows libraries to call C functions
> directly, along with a data type to wrap void-pointers. This is
> similar to Cython/pyrex, which I consider the current best-of-breed
> for scripting language FFIs.
>


I once did this (long ago) to allow one of my languages (in this case,
Scheme based), to interface with C. the cost was, however, that it was
limited to statically compiled code, and had all sorts of integration issues
with the interpreted version (for example, they could not directly share
binding environments, ...).




more recently, my project has taken a very different approach:
both the VM and native code are C.


using C as the VM language greatly reduces the costs of integration with the
outside world (since they can agree on matters of typesystem, data
representation, ...). and also gives the VM a good deal more reflective
power (by supporting C, it gains a good level of "understanding" of native
C-land as well...).


however, this is not the most easy strategy, and it has another cost:
many of C's technical issues and limitations end up gaining a hold on the VM
itself, making high-level language features MUCH more difficult to get
implemented...




though a good portion of the complexity may also be that my VM is not
interpreted, rather, it is currently a combination of run-time compilation
and dynamic code generation. (although, it is not proper JIT at present,
given that the code generation processes tend to work differently than
traditional JIT compilers...).


...




however, all this effort does have a payoff:
plain C to plain C integration.


AKA: there is no (or at least, directly visible) ugly glue code...




similarly, this may be extendable to Java and JavaScript as well, namely
that direct C<->JS calls would be possible, although likely with a few
restrictions/issues (mostly due to the drastically different typesystems,
... between C and JS...).


it would be less severe with Java, since the typesystems are a little closer
(only, there will be exposed language-specific stuff in some cases).




ammusingly, both Java and JS would have the option of using JNI on the C
end, should this be desired (although, at present there is not "complete"
integration between my framework and JNI at present...).




but, this does bring up another "generic" option:
VMs can generally go and implement JNI as their FFI (even if the VM has
little to do with the JVM), since one can at least pretend that JNI is a
"sort of" standardized C-side FFI...

Return to the comp.compilers page.
Search the comp.compilers archives again.