ModSecurity 3 no longer compiles with LuaJIT 2.1

[mdunc]

A recent change in ModSecurity seems to have broken compatibility with LuaJIT (which is compatible with Lua 5.1). The following is the error when compiling:

In file included from ../src/rule_script.h:22:0,
                 from seclang-parser.yy:20,
                 from seclang-parser.cc:46:
../src/engine/lua.h: In function 'void luaL_setfuncs(lua_State*, const luaL_Reg*, int)':
../src/engine/lua.h:111:13: error: 'void luaL_setfuncs(lua_State*, const luaL_Reg*, int)' was declared 'extern' and later 'static' [-fpermissive]
 static void luaL_setfuncs(lua_State *L, const luaL_Reg *l, int nup) {
             ^~~~~~~~~~~~~
In file included from /usr/include/luajit-2.1/lua.hpp:5:0,
                 from ../src/engine/lua.h:17,
                 from ../src/rule_script.h:22,
                 from seclang-parser.yy:20,
                 from seclang-parser.cc:46:
/usr/include/luajit-2.1/lauxlib.h:88:18: note: previous declaration of 'void luaL_setfuncs(lua_State*, const luaL_Reg*, int)'
 LUALIB_API void (luaL_setfuncs) (lua_State *L, const luaL_Reg *l, int nup);
                  ^~~~~~~~~~~~~
make[3]: *** [Makefile:2013: parser/libmodsecurity_la-seclang-parser.lo] Error 1

[victorhora]

Hi @mdunc

I'm wondering if you are running the latest version and that the configure script correctly ran on your environment setting the flags for LuaJIT correctly?

On one of my test environments (Debian 8.9) it compiles and runs fine with LuaJIT:

ModSecurity - v3.0.2-100-g0d53111 for Linux

 Mandatory dependencies
   + libInjection                                  ....v3.0.2-100-g0d53111
   + SecLang tests                                 ....0d53111

 Optional dependencies
   + GeoIP/MaxMind                                 ....found
      * (MaxMind) v1.2.0
         -lmaxminddb , -DWITH_MAXMIND -I/usr/include/x86_64-linux-gnu
   + LibCURL                                       ....found v7.38.0
      -L/usr/lib/x86_64-linux-gnu -lcurl,  -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL
   + YAJL                                          ....found v2.1.0
      -lyajl , -DWITH_YAJL -I/usr/include/yajl
   + LMDB                                          ....not found
   + LibXML2                                       ....found v2.9.1
      -lxml2, -I/usr/include/libxml2 -DWITH_LIBXML2
   + SSDEEP                                        ....not found
   + LUA                                           ....found v501
      -lluajit-5.1 -L/usr/lib/x86_64-linux-gnu/, -DWITH_LUA -DWITH_LUA_5_1 -I/usr/include/luajit-2.0

 Other Options
   + Test Utilities                                ....enabled
   + SecDebugLog                                   ....enabled
   + afl fuzzer                                    ....disabled
   + library examples                              ....enabled
   + Building parser                               ....disabled
   + Treating pm operations as critical section    ....disabled

I would suggest double-checking that other Lua versions (5.1, 5.2, 5.3) are not screwing up the configure script by checking the output of the configure script.

[mdunc]

Hi @victorhora . I am compiling this in Alpine Linux 3.6 as part of a Docker container. The Dockerfile I am using has not been modified in several months and used to work fine (I have also tried Alpine 3.8 with the same results). The luajit and luajit-dev packages are installed (no other Lua versions are installed) then ModSecurity is pulled from Github and built with the following environment variables set:

export LUAJIT_LIB=/usr/lib
export LUAJIT_INC=/usr/include/luajit-2.1

These are the commands I'm using to build:

git clone https://github.com/SpiderLabs/ModSecurity.git
git clone https://github.com/SpiderLabs/ModSecurity-nginx
cd /ModSecurity
git checkout v3/master
git submodule init
git submodule update
./build.sh && ./configure
sed -i "16i #include <sys/types.h>" src/actions/transformations/transformation.h
make && make install

That sed line seems to be required in Alpine due to it using musl instead of glibc and a slight difference there (It's been a while, but IIRC, it was an issue with some type not being recognized and including that types.h file resolved it).

[mdunc]

This is the configure output:

ModSecurity - v3.0.2-100-g0d53111c for Linux
 
 Mandatory dependencies
   + libInjection                                  ....v3.0.2-100-g0d53111c
   + SecLang tests                                 ....0d53111c
 
 Optional dependencies
   + GeoIP/MaxMind                                 ....found 
      * (GeoIP) v1.6.12
         -lGeoIP , -I/usr/include/ 
   + LibCURL                                       ....found v7.61.1 
      -lcurl -lnghttp2 -lssh2 -lssh2 -lssl -lcrypto -lssl -lcrypto -lz,  -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL
   + YAJL                                          ....found v2.1.0
      -lyajl , -DWITH_YAJL -I/usr/include/yajl 
   + LMDB                                          ....disabled
   + LibXML2                                       ....found v2.9.8
      -lxml2 -L/lib -lz -lm, -I/usr/include/libxml2 -DWITH_LIBXML2
   + SSDEEP                                        ....not found
   + LUA                                           ....found 
      -lluajit-5.1  , -DWITH_LUA -DWITH_LUA_5_1 -I/usr/include/luajit-2.1 
 
 Other Options
   + Test Utilities                                ....enabled
   + SecDebugLog                                   ....enabled
   + afl fuzzer                                    ....disabled
   + library examples                              ....enabled
   + Building parser                               ....disabled
   + Treating pm operations as critical section    ....disabled

[mdunc]

Also, I see from the output you provided that Debian is using LuaJIT 2.0, not 2.1. I haven't gone through the code in LuaJIT, but this is what is causing the conflict in ModSecurity - dee9898#diff-f120b67754e2c3ff8a98d4e3007f12efR111. I had no problems compiling prior to that commit.
I can checkout eed6b5f and compile just fine.

[victorhora]

Indeed. LuaJIT 2.1 is breaking.

I believe this is due luaL_setfuncs() already being defined in LuaJIT 2.1 (see lauxlib.h#L88).

Commit dee9898#diff-f120b67754e2c3ff8a98d4e3007f12efR111 defines this func as it is being used by the Lua engine inside ModSecurity (lua.cc#L138) but it is lacking in Lua 5.1 to make it compatible.

It might just need an extra ifdef here dee9898#diff-f120b67754e2c3ff8a98d4e3007f12efR105 to cover for LuaJIT 2.1.

[victorhora]

@mdunc As for:

That sed line seems to be required in Alpine due to it using musl instead of glibc and a slight difference there (It's been a while, but IIRC, it was an issue with some type not being recognized and including that types.h file resolved it).

I've also checked this one and it wasn't necessary on the latest version of Alpine Linux (3.8.1). I did get: linux/if.h: No such file or directory error, but this can be fixed by simply installing the linux-headers package.

Can you maybe try the same on Alpine 3.6 to see if it fixes the issue? Otherwise we can consider including it by default as long as it doesn't break our buildbots or other mainstream distros.

Thanks.

[victorhora]

Pull request #1910 should fix this issue.

Thanks for the report :)

[StrathCole]

There is still a problem when using Luajit-2.1
the configure script outputs as CFLAGS:

      -lluajit-5.1 -L/usr/lib/x86_64-linux-gnu/, -DWITH_LUA -DWITH_LUA_5_1 -I/usr/include/lua

as /usr/include/lua is a symlink to luajit-2.1 this leads to non-working make.
The -DWITH_LUA_JIT_2_1 is only added inside an if block from line 6383

    if test -z "${LUA_CFLAGS}"; then
...
        case $LUA_PKG_VERSION in
           (5.1*) LUA_CFLAGS="-DWITH_LUA_5_1 ${LUA_CFLAGS}" ; lua_5_1=1 ;;
           (5.2*) LUA_CFLAGS="-DWITH_LUA_5_2 ${LUA_CFLAGS}" ; lua_5_2=1 ;;
           (2.0*) LUA_CFLAGS="-DWITH_LUA_5_1 ${LUA_CFLAGS}" ; lua_5_1=1 ;;
           (2.1*) LUA_CFLAGS="-DWITH_LUA_5_1 -DWITH_LUA_JIT_2_1 ${LUA_CFLAGS}" ; lua_5_1=1 ;;

But at this point the LUA_CFLAGS is already set so the if block is never executed.
When I disable the check (change test -z to test -n temporarily) configure correctly outputs

      -lluajit-5.1    , -DWITH_LUA -DWITH_LUA_5_1 -DWITH_LUA_JIT_2_1 -I/usr/include/luajit-2.1  

[victorhora]

Hi @StrathCole,

Thanks for the report.

Can you let us know if your environment have more than one version of Lua (or LuaJIT) installed? Sometimes the configure script can get a bit confused if that's the case.

If the issue still persists it would be great if you could provide details on your environment (e.g. distro).

Thanks!

[stanhu]

I'm also seeing this problem while trying to use Amazon Linux 2 to build LuaJIT 2.1 with ModSecurity (via ingress-nginx).

The workaround: yum install lua53 lua53-devel.

Details:

lua v5.1.4 is installed, but I can't uninstall it because yum needs it. Notice the configure.log gets:

configure:6152: LUA headers found at: /usr/local/include/lua
configure:6152: LUA headers found at: /usr/local/include/lua
configure:6152: LUA headers found at: /usr/local/include/lua
configure:6147: LUA library found at: /usr/lib64//liblua-5.1.so
configure:6152: LUA headers found at: /usr/local/include/lua

configure correctly picks up /usr/local/include/lua, but it incorrectly uses /usr/lib64:

   + LUA                                           ....found v501
      -llua-5.1 -L/usr/lib64/, -DWITH_LUA -DWITH_LUA_5_1 -I/usr/local/include/lua

lua-static owns /usr/lib64/liblua-5.1, but LuaJIT is actually in /usr/local/lib under libluajit:

bash-4.2# ls -al /usr/local/lib | grep lua
-rw-r--r-- 1 root root 5993328 Apr 18 23:07 libluajit-5.1.a
lrwxrwxrwx 1 root root      22 Apr 18 23:07 libluajit-5.1.so -> libluajit-5.1.so.2.1.0
lrwxrwxrwx 1 root root      22 Apr 18 23:07 libluajit-5.1.so.2 -> libluajit-5.1.so.2.1.0
-rwxr-xr-x 1 root root 3165952 Apr 18 23:07 libluajit-5.1.so.2.1.0
drwxr-xr-x 3 root root    4096 Apr 18 23:07 lua

[stanhu]

I should note fe98ce4 reverted searching for luajit.

[stanhu]

#2718 should help here.