osa1

Calling Haskell from Lua and Lua from Haskell

April 27, 2014 - Tagged as: haskell, en, lua, hslua.

TL;DR: It’s possible to call Haskell functions from Lua and Lua function from Haskell using hslua. “Callbacks” example in hslua repository shows how to do this.

UPDATE: Also see the follow-up post here.

hslua provides Lua 5.1 C API to the programmer, and using it we can call Lua functions from Haskell and Haskell functions from Lua. In this post, I’m going to give two example Lua function implementations in Haskell. This functions will be mapped to a global name in Lua and one of them will be getting Lua functions as it’s arguments.

Two ways of writing Lua functions in Haskell

There are two ways of writing Lua functions in Haskell using hslua. First is the high-level method, where the Haskell function is just any function, provided that it’s type is an instance of LuaImport(we’ll come to this later). Second method is what I’d like to call the raw Haskell function method. Raw Haskell functions should have the type LuaState -> IO CInt, where LuaState is Lua interpreter state provided by hslua and return type CInt represents the amount of return values(e.g. values that are left on the stack by this function). In raw functions, you need to do Lua stack manipulation using standard Lua 5.1 API.

While first method gives you a nice, abstracted way of writing Lua functions in Haskell, raw function method gives you the maximum amount of flexibility that Lua can provide. First method has lots of limitations when compared with raw method.

Higher-level Haskell functions in Lua

In first method all you have to do is to write a Haskell function with it’s type is an instance of LuaImport. So let’s see what types are instances:

instance (StackValue a) => LuaImport (IO a) where
instance (StackValue a, LuaImport b) => LuaImport (a -> b)

So basically every function type where it’s argument types are instances of StackValue and return type is IO a where a is also an instance of StackValue is an instance of LuaImport and thus can be used in this higher-level method of writing Lua functions in Haskell. Let’s see which types are StackValues:

instance StackValue LuaInteger
instance StackValue LuaNumber
instance StackValue Int
instance StackValue Double
instance StackValue String
instance StackValue Bool
instance StackValue (FunPtr LuaCFunction)
instance StackValue (Ptr a)
instance StackValue LuaState
instance StackValue ()

We have basic Haskell types Int, Double, String, and Bool as instances. Other types are for more advanced use, for example, Ptr a is used for userdata(basically any binary data that you want to pass to Lua stack and later get back, see reference manual for more details).

This two functions are instances of LuaImport:

concat' :: String -> String -> IO String
concat' s1 s2 = return $ s1 ++ s2

pow :: Double -> Double -> IO Double
pow d1 d2 = return $ d1 ** d2

And we can push this functions to Lua stack using pushhsfunction or directly assign them to a global variable using registerhsfunction. In this post I’ll use register functions:

import Scripting.Lua as Lua

main :: IO ()
main = do
    l <- newstate
    openlibs l
    registerhsfunction l "concat" concat'
    registerhsfunction l "pow" pow
    loadfile l "haskellfun.lua"
    call l 0 0
    close l

With 11 lines of code, we can create a Lua state and map this Haskell functions to some names and call them from Lua. This program runs Lua file haskellfun.lua, which you can see in examples folder of hslua repository.

Apart from the simplicity, another good thing about this method is that it handles type checking of Lua values automatically. Internally, this functions are wrapped with another function which gets LuaState as parameter and collects Lua values from stack, checks their types(and throws error in case of a type mismatch), and push return value of the function to the Lua stack again. Here’s an example call with wrong type of values:

print(pow("wrong"))
...
bad argument #1 to '?' (number expected, got string)

Major limitation of this method is that you can only get basic Lua types from the Lua stack. For example, you can’t get a Lua table automatically like you get a Lua string. This because Lua needs to keep track of tables and some other values for garbage collection.

Working on more complex Lua types and the registry

The Lua way of using Lua tables, Lua functions etc. in Lua API is to register that values to the Lua table called registry, and refer to that values using their index at registry. Using registry, Lua keeps tracks of references to Lua values that are available for garbage collection. See reference manual section 3.5 for more details.

Raw Haskell function method

We can do this writing raw Haskell functions. Raw functions have type LuaState -> IO CInt and LuaState allows us to run any C API function. raw functions are pushed to Lua stack using pushrawhsfunction and registered as global variable using registerrawhsfunction.

callbacks example in the hslua repository takes Lua callbacks in Haskell functions and later call them in FIFO order and return their return values as a Lua array(table with int keys). You can see the complete program in the repository and here I’ll give only the tricky parts.

Raw Haskell functions should return number of values left on the Lua stack as return values. As an example, addLuaCallbacks function uses this for simple error reporting, it puts the error string to the Lua stack and return 1 in case of an error, and return 0 otherwise:

addLuaCallbacks l = do
    ...
    case as of
      Nothing -> do
        -- arguments are functions, add them to callback queue and return
        -- nothing
        addCallbacks 1 args
        return 0
      Just errArg -> do
        -- error: argument at `errArg` is not a function, return error
        -- string
        pushstring l $ "argument " ++ show errArg ++ " is not a function"
        return 1

This example program keeps track of passed Lua callbacks in an IORef. Here’s the part that handles getting Lua callbacks from Lua stack:

addCallbacks n max
  | n > max = return ()
  | otherwise = do
      -- move nth argument to top of the stack
      pushvalue l n
      -- add function reference to registry
      refId <- ref l registryindex
      -- add registry index to IORef
      modifyIORef cs (++ [refId])
      -- continue adding other arguments
      addCallbacks (n+1) max

Note how we’re adding the function to the registry and getting it’s index at the registry in Haskell. We can now refer to this functions(e.g. push this function to Lua stack) using this index. Here’s the relevant code:

-- | Call Lua callbacks collected with `addLuaCallbacks`.
callLuaCallbacks :: LuaState -> IO CInt
callLuaCallbacks l = do
    ...
  where
    iter [] = return ()
    iter (c : rest) = do
      ...
      pushinteger l (fromIntegral c)
      gettable l registryindex
      -- call the callback
      call l 0 1
      ...
      iter rest

We’re pushing the index to the Lua stack, and calling gettable to push actual function to the stack using the index. Complete program is longish, so I’m omitting it here, you can see it in hsluas Github repository with an example Lua program that uses defined Haskell functions to pass Lua callbacks to Haskell.

A note about safety

Lua tolerates some incorrect stack operations and if you do that while writing raw Haskell functions, you can have hard times debugging your programs. Fortunately, Lua also provides a compile time flag to enable checking API usage for safety. If you install hslua using -fapicheck Cabal flag, it compiled Lua with API checking enabled and Lua gives you errors instead of silently doing something not intended. For example, if you refer to a Lua value at stack index -4 while your stack has only 3 elements, you get something like:

callbacks: src/lapi.c:57: index2adr: Assertion `idx != 0 && -idx <= L->top - L->base' failed.

This helps making sure that your API usage is correct.

Using hslua for reading configuration files written in Lua

hslua also provides a module for reading configuration files. As an example, using Scripting.Lua.ConfigFile.getNestedAssocLists, you can execute the Lua file given below and get resulting nested table as Haskell [(String, [(String, String)])]:

 someVal = {
    something = {
       foo = "aaa",
       bar = "bbb",
       baz = "ccc"
    },
    somethingElse = {
       ...
    }
 }

Running Lua programs using Lua interpreters

In all hslua examples, we needed to create Lua state in Haskell, register Haskell functions inside Haskell and then run Lua files using Lua C API again in Haskell. I think in theory it is also possible to run Lua programs using standard Lua interpreter executables(instead of running the interpreter using Lua C API inside Haskell) and load Haskell functions compiled to *.so shared library files.

About one year ago, I did something similar using C instead of Haskell. You can see the code here. The library compiled to a .so file and by using Lua’s package.loadlib, you can load functions defined in the .so.

Compiling raw Haskell hslua functions to a .so should not be tricky. LuaState type is just a wrapper around Ptr (), and CInt is just a C integer, so it’s signature is already compatible. We probably need to use foreign export ... of GHC FFI and find GHC parameters to compile to .so. I’ll investigate this further and post updates.

(Show comments)