osa1

Loading dynamic Haskell libs in Lua

January 16, 2015 - Tagged as: en, haskell, lua, hslua.

Last year I wrote a blog post in which I explained how to call Lua from Haskell and Haskell from Lua using hslua library. At the end of that blog post I mentioned that it should be possible to compile Haskell code to shared library and load that in Lua.

Today a friend in our research group parfunc asked a question about compiling Haskell to shared libraries and loading generated libraries in other programs and I thought while I’m at it I can just update my blog post as well. So in this post I’m going to explain how to compile Haskell functions to shared libraries and load them in Lua.

Before diving into the code, a few remarks:

All the code in this blog post is tried on Linux, with Lua 5.1 and latest LuaJIT.
To be able to load our function in Lua and register it, our functions should have C linkage and lua_CFunction type. We can either write Haskell functions directly using this type, or write C wrapper functions around our Haskell functions to be able to use them in Lua. In this post I’m going to do first one.
We’ll need some intermediate C code to expose some Haskell RTS functions to Lua, like hs_init to start Haskell runtime and hs_exit to stop it.
To be able to require our shared library in Lua, we need to implement a int luaopen_<ourlibrary>(lua_State *L) function. While in theory it should be possible to implement that function in Haskell, I’ll implement it in C in this post, because I’m not sure how to write Lua wrappers for hs_init and hs_exit in Haskell.
To keep the code as simple as possible, our Haskell function will be a very dumb addition function.

Let’s start.

Defining Lua function in Haskell

This is exactly the same as before: We just define a function with type: LuaState -> IO Int. To keep the code simple, we don’t do error handling at all.

module LibArith where

import Data.Maybe
import Scripting.Lua -- this one from hslua

foreign export ccall
  add :: LuaState -> IO ()

add :: LuaState -> IO ()
add l = do
  i1 <- fromJust `fmap` peek l 1
  i2 <- fromJust `fmap` peek l 2
  pop l 2
  push l (i1 + i2 :: Int)
  return 1

Implementing intermediate C

In our C glue code, we do two things:

Wrap hs_init and hs_exit Haskell runtime functions.
Implement Lua C module interface in which we register our functions to Lua. (see related docs for details)

Here’s the code:

#include "LibArith_stub.h"
#include "lua.h"

int hs_init_lua(lua_State *L)
{
  hs_init(NULL, NULL);
  return 0;
}

int hs_exit_lua(lua_State *L)
{
  hs_exit();
  return 0;
}

int luaopen_lualibhelper(lua_State *L)
{
  lua_pushcfunction(L, add);
  lua_setglobal(L, "add_in_haskell");
  lua_pushcfunction(L, hs_init_lua);
  lua_setglobal(L, "hs_init");
  lua_pushcfunction(L, hs_exit_lua);
  lua_setglobal(L, "hs_exit");
  return 0;
}

Some things to note:

LibArith_stub.h is generated by GHC. I’ll explain how to compile and link next.
Our Haskell function actually has type HsInt (*)(void *). While this is not what Lua API expected(it expects int (*)(lua_State *L)), in my x86_64 Linux machine this is working fine. In the worst case, you may need to wrap the Haskell function in C and convert the types using Haskell RTS C API and Lua C API.

Compiling and linking

This is the tricky part, I wasted a good 2 hours trying to figure how to compile to .so and link it with correct set of libraries.

First step is to compile hslua in a sandbox, or at least make it reachable by GHC(by installing globally, using nix environments etc.). I’ll be giving commands assuming that you’re in a sandbox that has hslua installed, if you’re not, then just replace cabal exec ghc -- part with ghc and it should just work.

Step 1, compile and link the Haskell code to generate a shared library:

$ cabal exec ghc -- LibArith.hs -shared -dynamic -fPIC -o libarith.so -lHSrts-ghc7.8.3

Note that if you’re using a different version of GHC, you’ll need to modify the last argument to make it link it with corrent GHC RTS library.(alternatively, you can link with debug or profiling versions etc.)

Step 2, compile the Lua module written in C(the C code above) and link it with our shared Haskell library:

$ cabal exec ghc -- libarithhelper.c -no-hs-main -optl -larith -o lualibhelper.so -shared -fPIC -dynamic

Note that you may need to pass extra linker parameters if you have Lua library/headers in non-standard locations. If that’s the case, -optl argument of GHC is used to add linker arguments, just use standard linker arguments with that(-L, -I etc.).

This command should print a warning like this:

/home/omer/opt/luajit_bin/include/luajit-2.0/lua.h:168:16:
     note: expected ‘lua_CFunction’ but argument is of type ‘HsInt (*)(void *)’
     LUA_API void  (lua_pushcclosure) (lua_State *L, lua_CFunction fn, int n);

Like mentioned above, this doesn’t make any difference on my x86_64 Linux machine. If that’s being a problem on your system, just wrap your Haskell function in intermediate C code above using Haskell RTS API.

Now you should have two shared libraries, one for our Haskell code and one for the intermediate C code. One problem is that the shared library generated from C is now depending on the one generated from Haskell. So Haskell library should be in your LD_LIBRARY_PATH.

A good improvement here would be to compile Haskell code to static library, and generate one dynamic library only. (which has Haskell library statically linked to it)

Loading the code in Lua

Before loading it, make sure that the dynamic linker can really find the shared library generated from Haskell. Run this:

$ ldd lualibhelper.so | grep "not found"

Make sure it’s not printing anything.

Now just run Lua and enjoy the library:

$ luajit-2.0.3
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall. http://luajit.org/
JIT: ON CMOV SSE2 SSE3 SSE4.1 fold cse dce fwd dse narrow loop abc sink fuse
> require "lualibhelper"
> hs_init()
> print(add_in_haskell(1, 2))
3
> print(add_in_haskell(-10, 20))
10

Just for the amusement, let’s crash it by running Haskell function after stopping the Haskell runtime:

> hs_exit()
> add_in_haskell(1, 2)
newBoundTask: RTS is not initialised; call hs_init() first

Fun :)

Conclusion

It turns out that extending Lua using Haskell is almost as easy as the doing it using the technique I explained in my previous blog post on this topic.

This post also demonstrates one other thing, namely, compiling Haskell libraries to shared libraries and dynamically loading them in different programs. I’m hoping that this post helps fellow Haskellers to extend their programs written in different languages with Haskell.

(Show comments)