liboqs is not ready for cross-compilation

[embetrix]

Describe the bug
liboqs CMake has a lot of hardocded settings and not ready to cross-compile e.g for armv8 on x86-64 hosts for example by setting target cpu features :

https://github.com/open-quantum-safe/liboqs/blob/main/CONFIGURE.md#oqs_use_cpufeature_instructions

To Reproduce

$ cmake -DCMAKE_C_COMPILER=aarch64-linux-gnu-gcc  -DCMAKE_SYSTEM_NAME=Linux -DCMAKE_SYSTEM_PROCESSOR=arm64 .
-- The C compiler identification is GNU 13.3.0
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/aarch64-linux-gnu-gcc
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/aarch64-linux-gnu-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test CC_SUPPORTS_WA_NOEXECSTACK
-- Performing Test CC_SUPPORTS_WA_NOEXECSTACK - Success
-- Performing Test LD_SUPPORTS_WL_Z_NOEXECSTACK
-- Performing Test LD_SUPPORTS_WL_Z_NOEXECSTACK - Success
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Alg enablement unchanged
-- Found OpenSSL: /opt/veld/2.0.0/sysroots/cortexa53-crypto-veld-linux/usr/lib/libcrypto.so (found suitable version "3.0.15", minimum required is "1.1.1")  
-- Looking for aligned_alloc
-- Looking for aligned_alloc - found
-- Looking for posix_memalign
-- Looking for posix_memalign - found
-- Looking for memalign
-- Looking for memalign - found
-- Looking for explicit_bzero
-- Looking for explicit_bzero - found
-- Looking for explicit_memset
-- Looking for explicit_memset - not found
-- Looking for memset_s
-- Looking for memset_s - not found
-- Found Doxygen: /opt/veld/2.0.0/sysroots/x86_64-pokysdk-linux/usr/bin/doxygen (found version "1.9.3") found components: doxygen missing components: dot
-- Configuring done (1.4s)
-- Generating done (0.2s)
-- Build files have been written to: /home/zaki/Projects/Playground/liboqs

Compile:

$ make
[  0%] Building C object src/sig/cross/CMakeFiles/cross_rsdpg_256_small_clean.dir/sig_cross_rsdpg_256_small.c.o
[  0%] Building C object src/sig/cross/CMakeFiles/cross_rsdpg_256_small_clean.dir/upcross_cross-rsdpg-256-small_clean/CROSS.c.o
In file included from /home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/CROSS.c:29:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/csprng_hash.h: In function ‘csprng_randombytes’:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/csprng_hash.h:49:44: error: conversion from ‘uint64_t’ {aka ‘long unsigned int’} to ‘uint32_t’ {aka ‘unsigned int’} may change value [-Werror=conversion]
   49 |         xof_shake_extract(csprng_state, x, xlen);
      |                                            ^~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/csprng_hash.h: In function ‘csprng_randombytes_x2’:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/csprng_hash.h:88:52: error: conversion from ‘uint64_t’ {aka ‘long unsigned int’} to ‘uint32_t’ {aka ‘unsigned int’} may change value [-Werror=conversion]
   88 |         xof_shake_x2_extract(csprng_state, x1, x2, xlen);
      |                                                    ^~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/csprng_hash.h: In function ‘csprng_randombytes_x3’:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/csprng_hash.h:93:60: error: conversion from ‘uint64_t’ {aka ‘long unsigned int’} to ‘uint32_t’ {aka ‘unsigned int’} may change value [-Werror=conversion]
   93 |         xof_shake_x4_extract(csprng_state, x1, x2, x3, x3, xlen);
      |                                                            ^~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/csprng_hash.h: In function ‘csprng_randombytes_x4’:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/csprng_hash.h:97:60: error: conversion from ‘uint64_t’ {aka ‘long unsigned int’} to ‘uint32_t’ {aka ‘unsigned int’} may change value [-Werror=conversion]
   97 |         xof_shake_x4_extract(csprng_state, x1, x2, x3, x4, xlen);
      |                                                            ^~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/csprng_hash.h: In function ‘hash’:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/csprng_hash.h:169:44: error: conversion from ‘uint64_t’ {aka ‘long unsigned int’} to ‘uint32_t’ {aka ‘unsigned int’} may change value [-Werror=conversion]
  169 |         xof_shake_update(&csprng_state, m, mlen);
      |                                            ^~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/csprng_hash.h: In function ‘par_hash’:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/csprng_hash.h:193:63: error: conversion from ‘uint64_t’ {aka ‘long unsigned int’} to ‘uint32_t’ {aka ‘const unsigned int’} may change value [-Werror=conversion]
  193 |         par_xof_input(par_level, &states, m_1, m_2, m_3, m_4, mlen);
      |                                                               ^~~~
In file included from /home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:33,
                 from /home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/CROSS.c:30:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/restr_arith.h: In function ‘restr_vec_sub’:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/restr_arith.h:30:27: error: conversion from ‘int’ to ‘uint8_t’ {aka ‘unsigned char’} may change value [-Werror=conversion]
   30 | #define FZRED_SINGLE(x)   (((x) & 0x7f) + ((x) >> 7))
      |                           ^
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/restr_arith.h:49:26: note: in expansion of macro ‘FZRED_SINGLE’
   49 |                 res[i] = FZRED_SINGLE( a[i] + FZRED_OPPOSITE(b[i]) );
      |                          ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/restr_arith.h: In function ‘fz_inf_w_by_fz_matrix’:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/restr_arith.h:30:27: error: conversion from ‘int’ to ‘uint8_t’ {aka ‘unsigned char’} may change value [-Werror=conversion]
   30 | #define FZRED_SINGLE(x)   (((x) & 0x7f) + ((x) >> 7))
      |                           ^
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/restr_arith.h:31:25: note: in expansion of macro ‘FZRED_SINGLE’
   31 | #define FZRED_DOUBLE(x) FZRED_SINGLE(FZRED_SINGLE(x))
      |                         ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/restr_arith.h:92:34: note: in expansion of macro ‘FZRED_DOUBLE’
   92 |                         res[j] = FZRED_DOUBLE( (FZ_DOUBLEPREC) res[j] +
      |                                  ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/restr_arith.h: In function ‘restr_inf_w_sub’:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/restr_arith.h:30:27: error: conversion from ‘int’ to ‘uint8_t’ {aka ‘unsigned char’} may change value [-Werror=conversion]
   30 | #define FZRED_SINGLE(x)   (((x) & 0x7f) + ((x) >> 7))
      |                           ^
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/restr_arith.h:104:26: note: in expansion of macro ‘FZRED_SINGLE’
  104 |                 res[i] = FZRED_SINGLE( a[i] + FZRED_OPPOSITE(b[i]) );
      |                          ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h: In function ‘RESTR_TO_VAL’:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:37:25: error: conversion from ‘int’ to ‘uint16_t’ {aka ‘short unsigned int’} may change value [-Werror=conversion]
   37 | #define FQRED_SINGLE(x) ((x)% Q)
      |                         ^
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:63:16: note: in expansion of macro ‘FQRED_SINGLE’
   63 |         res1 = FQRED_SINGLE(
      |                ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:37:25: error: conversion from ‘int’ to ‘uint16_t’ {aka ‘short unsigned int’} may change value [-Werror=conversion]
   37 | #define FQRED_SINGLE(x) ((x)% Q)
      |                         ^
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:66:16: note: in expansion of macro ‘FQRED_SINGLE’
   66 |         res2 = FQRED_SINGLE(
      |                ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:37:25: error: conversion from ‘int’ to ‘uint16_t’ {aka ‘short unsigned int’} may change value [-Werror=conversion]
   37 | #define FQRED_SINGLE(x) ((x)% Q)
      |                         ^
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:69:16: note: in expansion of macro ‘FQRED_SINGLE’
   69 |         res3 = FQRED_SINGLE(
      |                ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:37:29: error: conversion from ‘int’ to ‘uint16_t’ {aka ‘short unsigned int’} may change value [-Werror=conversion]
   37 | #define FQRED_SINGLE(x) ((x)% Q)
      |                         ~~~~^~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:74:16: note: in expansion of macro ‘FQRED_SINGLE’
   74 |         return FQRED_SINGLE( FQRED_SINGLE(res1 * res2) *
      |                ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h: In function ‘restr_vec_by_fq_matrix’:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:37:25: error: conversion from ‘unsigned int’ to ‘uint16_t’ {aka ‘short unsigned int’} may change value [-Werror=conversion]
   37 | #define FQRED_SINGLE(x) ((x)% Q)
      |                         ^
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:38:25: note: in expansion of macro ‘FQRED_SINGLE’
   38 | #define FQRED_DOUBLE(x) FQRED_SINGLE(FQRED_SINGLE(x))
      |                         ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:106:34: note: in expansion of macro ‘FQRED_DOUBLE’
  106 |                         res[j] = FQRED_DOUBLE( (FQ_DOUBLEPREC) res[j] +
      |                                  ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h: In function ‘fq_vec_by_fq_matrix’:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:37:25: error: conversion from ‘unsigned int’ to ‘uint16_t’ {aka ‘short unsigned int’} may change value [-Werror=conversion]
   37 | #define FQRED_SINGLE(x) ((x)% Q)
      |                         ^
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:38:25: note: in expansion of macro ‘FQRED_SINGLE’
   38 | #define FQRED_DOUBLE(x) FQRED_SINGLE(FQRED_SINGLE(x))
      |                         ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:120:34: note: in expansion of macro ‘FQRED_DOUBLE’
  120 |                         res[j] = FQRED_DOUBLE( (FQ_DOUBLEPREC) res[j] +
      |                                  ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h: In function ‘fq_vec_by_fq_vec_pointwise’:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:37:25: error: conversion from ‘unsigned int’ to ‘uint16_t’ {aka ‘short unsigned int’} may change value [-Werror=conversion]
   37 | #define FQRED_SINGLE(x) ((x)% Q)
      |                         ^
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:38:25: note: in expansion of macro ‘FQRED_SINGLE’
   38 | #define FQRED_DOUBLE(x) FQRED_SINGLE(FQRED_SINGLE(x))
      |                         ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:132:26: note: in expansion of macro ‘FQRED_DOUBLE’
  132 |                 res[i] = FQRED_DOUBLE( (FQ_DOUBLEPREC) in1[i] *
      |                          ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h: In function ‘restr_by_fq_vec_pointwise’:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:37:25: error: conversion from ‘unsigned int’ to ‘uint16_t’ {aka ‘short unsigned int’} may change value [-Werror=conversion]
   37 | #define FQRED_SINGLE(x) ((x)% Q)
      |                         ^
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:38:25: note: in expansion of macro ‘FQRED_SINGLE’
   38 | #define FQRED_DOUBLE(x) FQRED_SINGLE(FQRED_SINGLE(x))
      |                         ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:142:26: note: in expansion of macro ‘FQRED_DOUBLE’
  142 |                 res[i] = FQRED_DOUBLE( (FQ_DOUBLEPREC) RESTR_TO_VAL(in1[i]) *
      |                          ^~~~~~~~~~~~
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h: In function ‘fq_vec_by_restr_vec_scaled’:
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:37:25: error: conversion from ‘unsigned int’ to ‘uint16_t’ {aka ‘short unsigned int’} may change value [-Werror=conversion]
   37 | #define FQRED_SINGLE(x) ((x)% Q)
      |                         ^
/home/zaki/Projects/Playground/liboqs/src/sig/cross/upcross_cross-rsdpg-256-small_clean/fq_arith.h:38:25: note: in expansion of macro ‘FQRED_SINGLE’
   38 | #define FQRED_DOUBLE(x) FQRED_SINGLE(FQRED_SINGLE(x))

Expected behavior
liboqs CMake shoould be adapted to enable cross-compilation : host arch != from target arch

Environment (please complete the following information):

• OS: Ubuntu 24.04
• liboqs version main branch

[baentsch]

Thanks for this report @embetrix . Could you please be specific what you'd expect to work (and doesn't)? Please see the bug report template to make this "actionable" (incl. improve CI testing).

[embetrix]

@baentsch : Thanks I updated the description

[baentsch]

Thanks for the clarification @embetrix . Would you be OK renaming the issue to "cross-compiling to arm64 on x64 not using emulation" then as otherwise using this toolchain file should do the trick, no?

[embetrix]

@baentsch the problem is more generic, I'm working on the integration of OQS for yocto/openembedded : https://en.wikipedia.org/wiki/OpenEmbedded

The cross-compilation should be supported on any host (x86, x86-64, arm, arm64, riscv) to any target, right now it build and work but enabling the oqs_use_cpufeature_instructions is not possible, since the CmakeList.txt seems not to have planned this kind of situation.

[SWilson4]

@baentsch the problem is more generic, I'm working on the integration of OQS for yocto/openembedded : https://en.wikipedia.org/wiki/OpenEmbedded

The cross-compilation should be supported on any host (x86, x86-64, arm, arm64, riscv) to any target, right now it build and work but enabling the oqs_use_cpufeature_instructions is not possible, since the CmakeList.txt seems not to have planned this kind of situation.

@embetrix Would you be willing to contribute a PR to support this feature as you develop the integration?

[embetrix]

@SWilson4 : sure I will look into it