Add option to limit the memory usage of Lua code

[Le0Developer]

Closes #211.

[scoder]

I admit that jumping to address 1 (as done before) isn't exactly a great error handler, but it feels like we should generally improve the way fatal errors are handled, rather than making them fatal but just in other ways.

OTOH, this is an improvement all by itself, so… shouldn't it be a separate PR?

[Le0Developer]

This error handler is the one that luaL_newstate registers

[scoder]

Not sure if we need this. Do you see a use case for changing the memory limit later on?

[Le0Developer]

I'm using this in the tests atm

[Le0Developer]

Using this to temporarily remove the limit to load trusted library and code and then restricting it once loading a user's code

[Le0Developer]

It seems like LuaJIT does not support a custom allocator in lua_newstate on 64bit.

See the following and failing tests:

Make sure you use luaL_newstate. Avoid using lua_newstate, since this uses the (slower) default memory allocator from your system (no support for this on 64 bit architectures).

• https://luajit.org/install.html

[scoder]

It seems like LuaJIT does not support a custom allocator in lua_newstate on 64bit.

Hmm. That is annoying. Then we should raise an exception when we detect that case and users request a custom allocator.

[Le0Developer]

Setting max_memory to None will now use the default lua allocator. Also added documentation for set_max_memory() and max_memory.

set_max_memory() also a strict option now, which will raise an exception if you try to set max_memory to a value lower than currently used.

[Le0Developer]

Just tested running LuaJIT 2.1.0-beta3 and it works on 64bit. 🤷

Following is a patch that disallows LuaJIT on 64bit, if needed:

diff --git a/lupa/lua.pxd b/lupa/lua.pxd
index 35d4b6e15a4fc18e3deb20353be72cae4ebaee86..72cdce030bfaf13db38af6d97aed66c85f55eacf 100644
--- a/lupa/lua.pxd
+++ b/lupa/lua.pxd
@@ -266,6 +266,13 @@ cdef extern from "lua.h" nogil:
         int i_ci               #           active function */
 
 
+################################################################################
+# luajit.h
+################################################################################
+
+cdef extern from "luajit.h" nogil:
+    int LUAJIT_VERSION_NUM
+
 ################################################################################
 # lauxlib.h
 ################################################################################
@@ -465,11 +472,18 @@ cdef extern from * nogil:
     #if LUA_VERSION_NUM < 502
     #define lua_pushglobaltable(L)  lua_pushvalue(L, LUA_GLOBALSINDEX)
     #endif
+
+    #if defined LUAJIT_VERSION_NUM && UINTPTR_MAX == 0xffffffffffffffff
+    #define IS_LUAJIT64 1
+    #else
+    #define IS_LUAJIT64 0
+    #endif
     """
     int read_lua_version(lua_State *L)
     int lua_isinteger(lua_State *L, int idx)
     lua_Integer lua_tointegerx (lua_State *L, int idx, int *isnum)
     void lua_pushglobaltable (lua_State *L)
+    char IS_LUAJIT64
 
 cdef extern from *:
     # Limits for Lua integers (in Lua<5.3: PTRDIFF_MIN, PTRDIFF_MAX)
diff --git a/lupa/_lupa.pyx b/lupa/_lupa.pyx
index 22656680343b1c6e9f97ef8adf972df9958e614a..c173d9a9376339e2dfec982e04ca5ca9b7a8b6d2 100644
--- a/lupa/_lupa.pyx
+++ b/lupa/_lupa.pyx
@@ -263,6 +263,9 @@ cdef class LuaRuntime:
                   bint register_eval=True, bint unpack_returned_tuples=False,
                   bint register_builtins=True, overflow_handler=None,
                   max_memory=None):
+        if lua.IS_LUAJIT64 and max_memory is not None:
+            raise RuntimeError("max_memory is not supported on 64bit LuaJIT")
+        
         cdef lua_State* L
 
         self._memory_left = 0

[scoder]

`set_max_memory()` also a strict option now, which will raise an exception if you try to set max_memory to a value lower than currently used.

I'd rather expose the current memory usage as readonly attribute and let users do their own checks. That seems more generally useful.

[Le0Developer]

Added memory_used

[scoder]

There's still a test failure. Maybe you could look into that? I'd like to release Lupa 2.0 soon and this would be a nice addition.

[Le0Developer]

I'm a Cython noob, so no idea how to fix this 🤷

Error compiling Cython file:
------------------------------------------------------------
...
    cdef bytes _source_encoding
    cdef object _attribute_filter
    cdef object _attribute_getter
    cdef object _attribute_setter
    cdef bint _unpack_returned_tuples
    cdef MemoryStatus _memory_status
                     ^
------------------------------------------------------------

lupa/lua53.pyx:260:22: Variable type 'MemoryStatus' is incomplete

[Le0Developer]

All test succeeding locally now! 🎉

[scoder]

Just two safety fixes.

[scoder]

Thanks!