Re: Languages with well-integrated Foreign Function Interface to learn from?

Christoffer Lernv wrote:


> I'd like to research FFI in various languages, basically to find the
> best FFI-solution and copy from that one.


I think the current trends are towards what could be described as a mix
of aspect orientation and dynamic code generation. I've seen this in
..NET, Java, and Ruby.


..NET: P/Invoke


http://www.pinvoke.net/ has lots of example syntax and declarations.


These are basically external static method signatures with attributes
(user-definable metadata) attached that describes to the runtime
environment how to marshal arguments and what name to import from the
linked-to library.


Even though in practice it's the CLR, i.e. the actual runtime, which
does this linking and marshaling, it principle it could be done entirely
by a third-party library using a small handful of primitives. Since the
(full-version, non-browser/mobile) platform has the capability of
generating code dynamically at runtime, efficient wrappers that grab
arguments can be generated, juggling them about as necessary (guided by
metadata), and dispatching the final call, almost entirely written in
the language itself and in a replaceable and extendable way.


Java: JNA


https://jna.dev.java.net/


The idea is you create an interface that represents the exported
routines you want to access, and let a library take care of mapping
through reflection and annotations (user-definable metadata). Much like
P/Invoke and far easier to use than JNI (though it uses JNI under the
hood from what I understand).


Ruby: FFI


http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html


> What I'm looking for is syntax, to what extent automatic conversion of
> arguments are done, how to handle callbacks, memory management, how to
> create structured data (i.e. structs in the case of C) etc.


* Conversion of arguments: when interacting with C, certain things
almost certainly need to be done, such as strings and untyped
polymorphic return buffers (e.g. Windows APIs where there's a cbSize at
the start of the struct which determines which version will be
returned). The details of correspondences with C-level primitive types
depend on the type system and library, of course.


For example, C# uses mutable StringBuffer instances to represent C
non-const char*, and uses attributes on the parameter / return value in
the P/Invoke declaration to indicate the encoding. Similarly, structures
which are used for interop must be blittable (no GC'able references) and
can have explicit layout (again, done with attributes) for cases like
unions.


* Callbacks: this is "just" the inverse of imports, but may require
dynamic stubs for things like function pointer -> closure/method-pointer
conversion, so that both self/environment and code address can be
exported.


* Memory management: in a precise GC environment, there normally needs
to be either some mechanism for pinning memory (prevent move or
collection), or an "out" to manual memory allocation, or else all
transfer must be copies both ways. For example, .NET supports all three;
pinning via the C# 'fixed' keyword and GCHandle framework type, manual
allocation via the Marshal framework class, and generally defaults to
copying.


* Structs: some kind of way of declaratively specifying the layout of
the C-visible type, possibly using attributes / annotations or perhaps
even dynamic imperative specification at runtime like Ruby/FFI. With
runtime support, attribute metadata can be used to make the internal
type match the external type, but it's not absolutely necessary, if the
FFI library is going to do the conversion work.


If I were you, I'd try and focus on ensuring that the FFI work can be
done using libraries that may possibly be extendable for certain user
scenarios (using some kind of metadata), over and above some ad-hoc
tooling that needs to use the same language as the runtime (typically C)
and compiled and linked against the runtime for every FFI linkage. To
make rich interaction with the platform easy, make the FFI very easy to
use and play with.


-- Barry


--
http://barrkel.blogspot.com/

Return to the comp.compilers page.
Search the comp.compilers archives again.