Semi-automatic conversion of C++ exceptions to Lua errors

 0. Error handling in Lua and C++

Lua and C++ actually use a similar error handling scheme: an error is signalled at some point of the execution and the stack unrolls until a handler is found. If you ask google how to "simulate C++ exception in C" you will be shown several results all using setjmp/longjmp and this is exactly the mechanism used by the Lua interpreter. I'm not going to enter into details here but basically you call setjmp to mark an execution point and call longjmp to return to it, no matter how deep in the stack you are. If you think of setjmp as a try...catch and longjmp as throw, then you have a primitive exception handling scheme.

Back to Lua, when you call lua_pcall in C or pcall in Lua, the interpreter calls setjmp and the executes the function you passed it. If the interpreter detects an error, or the user code calls lua_error, longjmp is called and the execution resumes inside pcall.

The problem with longjmp is that it just discards the entire execution stack (not Lua's stack) without doing any cleanup. If you called malloc and then did a longjmp you just leaked a chunk of memory. So you have to be very careful to clean up everything. In C++ when you throw an exception, not only is the stack unrolled but the destructor of every object you allocated on the stack is called. So if you want to protect yourself you tie each allocation of resources to the lifetime of an object on the stack.  This practice is known as C++'s resource allocation is initialization (RAII) idiom. In the case of a memory buffer, you would use a smart pointer object to hold the raw pointer that calls delete when it is destructed.

Now that we have covered how the exception handling works in C++ and in Lua, let's see what happens if you mix them.

1. Signaling errors from C++ to Lua

Suppose you have a piece of C++ code that is called by lua. What happens if the C++ code throws an exception? Not really what we would like to happen: nobody catches the exceptions and the process terminates. Even if there is a pcall on the stack, Lua will not catch the exception.

Ok, I know that if you compile the PUC Lua interpreter with a C++ compiler it will use try...catch instead of setjmp...longjmp, but unless you are willing to recompile Lua and link it statically to you application, you can't count on that. Also PUC Lua is not the only interpreter out there.

The solution, as you might guess, is to convert the exceptions to Lua errors at the boundary between the two languages. Typically you have some nifty C++ code that you want to use in Lua and you add a layer to make it callable from Lua. Let's call this layer a binding. This binding usually consists of a bunch of functions that conform to the lua_CFunction signature.

Consider the following scenarios:

1. Deep in the C++ code an exception is raised that can't be corrected by the binding
2. A error is detected at the binding level that must must be reported to the calling Lua code

How should we handle these cases? A possible solution is:

"To protect ourselves from the first case, we surround every call to C++ with a try catch and call lua_error in the catch block. In the second case it doesn't make sense to throw an exception so we call lua_error directly."

It makes sense, doesn't it? But it actually has two severe problems.

The first problem is that if you call lua_error inside of the catch block you are guaranteed to leak at least one resource: the exception object itself. lua_error calls longjmp and the exception's destructor is never called. Usually exception objects contain an error message string so this is leaked as well. A way out would be to structure your exception handling code so that lua_error is called outside of the catch block. In the case where an error is detected in the binding and you call lua_error directly you also must be very careful to clean-up everything.

The second problem is that it is hard to organize your code around lua_error. The slightest mistake and you are leaking resources. Ok, you say, but I am a very disciplined programmer and I'll make sure that every object is destroyed before I call lua_error. But what if someone else who is unaware of all this subtleties edits your code? This solution is very hard to maintain. Also your C++ code doesn't feel very natural anymore.

2. Solution

The solution I propose is:

1. Handle all the conversions of exceptions to lua_error in one place
2. Never call lua_error except for this one place
3. Always use exceptions, even in binding lua_CFunctions

The trick, of course, is implementing item 1. We want to wrap every call to a lua_CFunction inside a try...catch statement. As an example let's use a modified version of listing 26.1 of Programming in Lua. We implement a function to list directories but instead of returning error codes we throw an exception (this is a bad idea, but it's just to illustrate our case).

#include <dirent.h>
#include <errno.h>
#include <stdexcept>

static int l_dir(lua_State* l) {

  const char* path = luaL_checkstring(L, 1);

  DIR* dir = opendir(path);
  if (dir == nullptr) {
    throw std::logic_error(strerror(errno));
  }

  lua_newtable(L);
  int i = 1;
  dirent* entry;

  while((entry = readdir(dir)) != NULL) {
    lua_pushnumber(L, i++);
    lua_pushstring(L, entry->d_name);
    lua_settable(L, -3);
  }
  closedir(dir);
  return 1;
}

Now we must register the function to call it from lua:

static const struct luaL_Reg dirlib[] = {
  { "dir", l_dir },
  { nullptr, nullptr }
};

int luaopen_dirlib(lua_State* L) {
  luaL_register(L, "dirlib", dirlib);
  return 1;
}

Our first try is:

static int wrap_dir(lua_State* L)
{
  try {
    return l_dir(L);
  } catch(std::exception& ex) {
    lua_pushstring(L, "caught C++ exception: ");
    lua_pushstring(L, ex.what());
    lua_concat(L,2);
  } catch (...) {
    lua_pushstring(L, "caught unknown C++ exception");
  }

  // call lua_error out of the catch block to make sure
  // that the exception's destructor is called
  // this is because lua_error calls longjmp and discards the error
  lua_error(L);

  return 0; //just to silence warnings
}

And we register this wrapper function instead of l_dir

static const struct luaL_Reg dirlib[] = {
  { "dir", wrap_dir },
  { nullptr, nullptr }
};

Notice that we don't call lua_error in the catch blocks to avoid leaking the exceptions. This solution works but it's not practical to require a wrapper function for every lua_CFunction, programmers are lazy people!

What we would like to do is to create a generic wrapper function that takes a lua_CFunction pointer as argument, calls it and handles potential exceptions. The problem is that with this new signature our function wouldn't conform to the lua_CFunction signature and therefore could not be registered with luaL_register. We need another way to pass the function pointer to our wrapper function.

As usual, in my posts, C++ templates come to our rescue when the situation looks most desperate. What few people know is that function pointers can be used as template arguments in C++. We can write a function template that takes as template argument a pointer to lua_CFuntion. Once instantied, it is a lua_CFunction like any other, that can be registered with luaL_register.

template<lua_CFunction func>
static int exception_translator(lua_State* L)
{
  try {
    return func(L);
  } catch(std::exception& ex) {
    lua_pushstring(L, "caught C++ exception: ");
    lua_pushstring(L, ex.what());
    lua_concat(L,2);
  } catch (...) {
    lua_pushstring(L, "caught unknown C++ exception");
  }

  // call lua_error out of the catch block to make sure
  // that the exception's destructor is called
  // this is because lua_error calls longjmp and discards the error
  lua_error(L);
  return 0; //just to silence warnings
}

static const struct luaL_Reg dirlib[] = {
  { "dir", exception_translator<l_dir> },
  { nullptr, nullptr }
};

3. Conclusion

If you are careful to wrap every lua_CFunction with this function you can use exceptions as much as you like in your C++ code knowing that they will be converted to Lua errors if they aren't caught before. Because you are able to use exceptions instead of lua_error, you know that everything that you carefully allocated using RAII will get disallocated in the case of errors. Also you don't have to rely on the compilation settings of your Lua interpreter, this works with any conforming interpreter.