For the second time, I’m changing the foreign function interface. (Actually, I already did.) It’s a rather sudden announcement, but since no one else besides me is using the language that I know of, I can get away with things like this.

The question is “why?” Short answer: Needs became more apparent. Long answer: When I first wrote the interface, I wasn’t certain of how to present the information, so I conceived of some mechanisms by which they would work. I found this tedious, and I thought that giving the user simulated access to the arguments list contained in the engine would streamline the process. But what happened was that the needs changed. While it was suitable (for the sake of speed) to give system function direct access to the list of arguments passed to them, foreign functions must be easy to create and easy to read.

Time and again, I found myself writing out boilerplate code to make checking types convenient. I hadn’t yet decided between using string types and enumeration values for types. The latter won out because it turned out to be easy to create non-conflicting enumeration values and the type checking was faster (for obvious reasons).

Worse than writing out boilerplate code is writing boilerplate code scattered throughout your other code. Simple functions become really ugly (even if fast). But perhaps the clincher was the fact that I was writing lots of processing code that had no guarantee of completing correctly until the final argument was processed. (And all of this was done to “streamline” the processing.) While I hadn’t yet encountered a case where this would be disastrous, I could see that eventually, it would be.

Ironically, what actually stimulated my thoughts in regards to the FFI was the fact that logging was still ugly. I had just managed to get system functions to output their error messages with object types (both given and expected). I wanted to do the same for foreign functions, but the existing interface was clunky. The foreign function would need associated functions that would return the types needed by the foreign function. But creating these extra functions and associating them with one specific foreign function was tons of extra work and led to me abusing the isVariadic() aspect of foreign functions due to the fact that ranges of argument counts were not supported. (i.e. Some foreign functions allowed 2 to 3 arguments, but the FFI only let them prepare for either 2 or 3, not both.)

To fix these issues, the new foreign function interface is nearly a complete revision of the original, and will have the following features:

  1. Argument count checking, including checking the number of arguments is within a certain range.
  2. Access to arguments by index (instead of sequentially, “out-of-a-pipe”, as before).
  3. Automatic error printing when argument count or argument types are checked via methods in the instance of FFIServices passed to the foreign function.

One of the beauties of this is the last point: automatic error printing. This means that the count and type errors will be passed through the system as LogMessages and be translatable in the printers and loggers. No more English-only printing. Program-specific error codes also be sent. The original printing functions have been preserved for logging errors that don’t have custom error codes.

What does the new interface look like?

UInteger getArgCount() - Necessary for variadic functions.
bool demandArgCount( UInteger count, bool imperative ) - For standard functions.
bool demandArgCountRange( UInteger min, UInteger max, bool imperative ) - For functions that can accept a range of arguments.
bool demandArgType( UInteger index, ObjectType::Value type ) - For verifying an argument type before obtaining it, allowing for safe casting.
Object& arg( UInteger index ) - For obtaining the argument directly. Throws an exception if the index is out of range.
void printCustomInfoCode( UInteger ) - Sends a custom error code.
void printCustomWarningCode( UInteger ) - Sends a custom error code.
void printCustomErrorCode( UInteger ) - Sends a custom error code.

Here’s what it will look like in practice:

bool myForeignFunc( FFIServices&  ffi ) {
if ( ! ffi.demandArgCount(2) ) {
  return false;
if ( ! ffi.demandArgType(0, getMyTypeAsObjectType(MyType) )
  || ! ffi.demandArgType(1, getMyTypeAsObjectType(MyType2) )
) {
  return false;
// Perform normal operations
 MyObject& myVar = (MyObject&)( ffi.arg(0) ); // Safe cast
 MyObject2& myVar = (MyObject2&)( ffi.arg(1) ); // Safe cast
// ...
  // Clean code here

return true;

Notice that all of the argument count and type handling is performed in the beginning, like it should be.

One of the other benefits of this change is that all of the type checking code within the Engine itself meant to be for foreign functions has now been removed. This declutters that particular section of code as well as speeds up the execution of functions that don’t need arguments (such as those meant to return constants like pi).

Benchmark tests revealed significant time savings. Those results will be published in the updated documentation.

To save more effort on the part of the user, I’ve made it such that demandArgType() works even if the index is out of bounds. This is nice for variadic functions that require certain arguments to begin.

The Moral of the Story

When I created the FFI, I was trying to do too much within the Engine itself. I was trying to solve the problem for all possible scenarios even before it was created. I should have – as I’ve done now – simply provided tools within the FFIServices for making it easier.

This story has some real life analogies. For example, it was like trying to create an automatic feeding machine that can pass food into every person’s throat (and thus needing to calculate throat sizes) instead of just giving them a fork and spoon and letting them do it.

The other moral is a confirmation of the awesomeness of a design principle regarding software architecture that I’ve learned on this project: Instead of sending data by means of multiple arguments in a function call, package it up all of the related data and send it. Then you won’t need to change your API and the many implementations of it. In this case, FFIServices and LogMessage are those packages (which I now call “shuttles” or “buses” after an analogy I use), and using them has made it a piece of cake to make changes.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s