D/Objective-C: hit a dead end, start anew

About three years ago, I was working on a pet project of mine which consisted of using the D programming language’s template capabilities to create a fully transparent bridge with the Objective-C runtime. It worked fine: objects were transparently exchanged between the two worlds and it was quite easy to write D bindings for Objective-C classes. It wasn’t too long however before some flaws started to appear…

The Flaws

The first problem was that it was awfully slow to compile anything using the bridge. While I did manage to reduce the compile time by a factor of 2 or 3, it’s still not fast. But that’s not a fatal flaw.

A second problem was the small inefficiencies everywhere. Normally, a compiled Objective-C object is stored in a particular way in the generated object file, and the dynamic linker and the Objective-C runtime automatically take care of initializing things. But I can’t do that from D, I need to initialize everything at runtime, and lazily to avoid the problem of circular dependencies between module static constructors. There’s a lot of code for that in the bridge.

As a consequence of the lazy initialization, bindings sometime need to be initialized manually in advance in some cases. This is the case before loading a nib file for instance. This is less than ideal.

But the real show stopper is the size of the generated code for the bindings. The system I created needs to generate a lot of code for each and every one of the functions and classes you define bindings for. This makes the final executable awfully big (and might also explain in part the slowness while compiling). The small test application included with the bridge takes about 60 Mb of code, thats for something that’d be only a few kilobytes when written in Objective-C.

This wasn’t the case at first. This bloat appeared as I added bindings for more and more classes in Cocoa. It is a real problem because no one in his right mind will want to ship a 60 Mb application that contains so much binding code that stay unused most of the time. A solution could be that everyone would have to write bindings that only contains the classes and methods he needs, but this is of rather limited interest.

A New Approach

So how to make the bindings more lightweight and without this lazy initialization nonsense? I see one way: make the compiler aware of the Objective-C object model. If the compiler becomes aware of what is an Objective-C object, how to call its methods, and how to emit the object code for it, it’d become as good as a real Objective-C compiler with the only difference being the syntax used would be the D syntax.

So that’s what I’ve been playing with this last month. I played with the source of DMD and tried to make it generate code à la Objective-C.

There’s only one thing that works right now: declaring extern (Objective-C) interfaces and calling functions on them. It works like this:

extern (Objective-C)
interface NSObject {
    NSObject description();
}

Now, when you have an instance of the interface NSObject in variable object, you can write object.description() and it’d be equivalent to writing [object description] in Objective-C. Internally, it is transforming the call to objc_msgSend(object, @selector(description)). I’ve made compiler put the selector in the __OBJC,__message_refs segment where it automatically get registered with the runtime while loading the executable. That’s exactly what does the real Objective-C compiler, and it’s much better than generating code to register the selector lazily at runtime.

For methods with more than one argument, the compiler can’t deduce the selector’s name from the function name since the selector is generally split in two or more parts. So I had to come with a syntax to bind a manually-specified selector with the function. Here’s what it looks like:

extern (Objective-C)
interface NSString {
    void getCharacters(wchar* buffer, NSRange range) [getCharacters:range:];
}

And that works too! With this, calling string.getCharacters(b, r) emits the same code as the Objective-C method call [string getCharacters:b range:r].

All this is great, but still incomplete. As it stands now you need to somehow get a pointer to an Objective-C object, cast it to the right interface and call functions on it. It’s definitely not bad, but it’s not very useful in itself either. There’s a lot more work to do before Cocoa gets usable with D.

But the ground work is there, so it’ll be easier to continue. It’ll certainly be at couple of months before it works right, more or less depending on how much time I can devote to this project. I want the final result to allow mixing Objective-C and D objects pretty much seamlessly, including subclassing Objective-C objects.

If you’d like to use D with Cocoa on the Mac, I’d be pleased to hear from you. And if you want to encourage me further with this, you can make a donation: if I get enough support it might get easier for me to put more time advancing all this.