Error: found no usable tomli

[karaketir16]

Error while building with build-sim SIM=qemu in Ubuntu 22.04

 => => # python version: Python 3.10.12
 => => # mkvenv: Creating non-isolated virtual environment at 'pyvenv'
 => => # *** Ouch! ***
 => => # found no usable tomli, please install it
 => => # make: *** [Makefile:1124: stamps/build-qemu] Error 1

Dockerfile to reproduce

# Start with Ubuntu 22.04 as the base image
FROM ubuntu:22.04

# Set environment variables to avoid interactive prompts during build
ENV DEBIAN_FRONTEND=noninteractive

# Update and install required dependencies
RUN apt-get update && apt-get install -y \
    autoconf \
    automake \
    autotools-dev \
    curl \
    python3 \
    python3-pip \
    libmpc-dev \
    libmpfr-dev \
    libgmp-dev \
    gawk \
    build-essential \
    bison \
    flex \
    texinfo \
    gperf \
    libtool \
    patchutils \
    bc \
    zlib1g-dev \
    libexpat-dev \
    ninja-build \
    git \
    cmake \
    libglib2.0-dev \
    && rm -rf /var/lib/apt/lists/*  # Clean up apt cache to reduce image size

# Clone the riscv-gnu-toolchain repository
RUN git clone https://github.com/riscv/riscv-gnu-toolchain /riscv-gnu-toolchain

# Build the toolchain
WORKDIR /riscv-gnu-toolchain
RUN ./configure --prefix=/opt/riscv --enable-multilib && \
    make -j$(nproc) newlib linux build-sim SIM=qemu && \
    make install && \
    rm -rf /riscv-gnu-toolchain # Clean up to reduce image size

# Set the PATH environment variable to include the RISC-V toolchain
ENV PATH=/opt/riscv/bin:$PATH

# Set the working directory (optional)
WORKDIR /workspace

# Set the default command to run (you can override this when running the container)
CMD ["bash"]

[mickflemm]

I couldn't reproduce this, QEMU's build script creates a python venv and installs needed packages inside, this is the output in my case on a clean ubuntu 22.04 container (instantiated using lxc -n ubuntu-test -t download):

Using './build' as the directory for build output
python determined to be '/usr/bin/python3'
python version: Python 3.10.12
mkvenv: Creating non-isolated virtual environment at 'pyvenv'
mkvenv: checking for tomli>=1.2.0
mkvenv: installing tomli>=1.2.0
mkvenv: checking for meson>=0.63.0
mkvenv: installing meson==1.2.3
The Meson build system
Version: 1.2.3
Source dir: /yarvt/sources/riscv-qemu
Build dir: /yarvt/sources/riscv-qemu/build
Build type: native build
Project name: qemu
Project version: 8.2.7
C compiler for the host machine: cc -m64 -mcx16 (gcc 11.4.0 "cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0")
C linker for the host machine: cc -m64 -mcx16 ld.bfd 2.38
Host machine cpu family: x86_64
Host machine cpu: x86_64

As you can see it installed tomli inside the virtual environment, so there is no need to install it on the host OS as well. Maybe there it can't install the package in your case, network issues ?

[karaketir16]

I will try again with the same Dockerfile. It might have been a network issue, as you suggested.

I'll let you know.

[karaketir16]

It happened again, and it's on GitHub Actions. It's probably not a network issue on their side. Here's the link to the logs: https://github.com/karaketir16/test-docker/actions/runs/11913130748/job/33198181961#step:3:12935.

#8 688.8 checking that generated files are newer than configure... done
#8 688.8 configure: creating ./config.status
#8 688.8 mkvenv: Creating non-isolated virtual environment at 'pyvenv'
#8 688.8 config.status: creating Makefile
#8 688.9 mkdir -p -- ./libbacktrace
#8 688.9 Configuring in ./libbacktrace
#8 689.1 
#8 689.1 *** Ouch! ***
#8 689.1 
#8 689.1 found no usable tomli, please install it 
#8 689.1 
#8 689.1 
#8 689.1 make: *** [Makefile:1124: stamps/build-qemu] Error 1
#8 689.1 make: *** Waiting for unfinished jobs....
#8 689.1 mkdir -p -- ./libdecnumber

[karaketir16]

QEMU's build script creates a python venv and installs needed packages inside,

That's exactly where it fails—it cannot create the virtual environment (venv)
https://gitlab.com/qemu-project/qemu/-/blob/master/python/scripts/mkvenv.py?ref_type=heads#L719

[mickflemm]

Ah now I managed to reproduce this...

https://wiki.qemu.org/ChangeLog/8.2 :
"Building QEMU now uses the tomli library if Python is older than version 3.11. However, version 2.0.1 is bundled in case tomli is not installed on the host."

https://wiki.qemu.org/ChangeLog/9.1 :
"When using Python 3.10 or older, building QEMU requires the tomli package to be installed on the host. (The dependency was introduced in QEMU 8.2 but until now QEMU included a vendored copy of the library)."

In our case we only run make_report (which compiles QEMU etc) on ubuntu 24.04 runners that come with python 3.12 (so it doesn't require tomli). However your dockerfile is for ubuntu 22.04 that has python 3.10, in my setup (with the ubuntu 22.04 container) I noticed I was still building 8.2, that's why it worked fine.

So if someone does make_report on ubuntu 22.04 (which is disabled in our CI) they'll need to install tomli. I guess it makes sense to add this to the README, it should be true for other distros that use python < 3.11 as well.

[karaketir16]

I checked the following Docker images:

• fedora:40: has Python 3.12 (pre-installed).
• fedora:41: has Python 3.13 (does not have Python pre-installed).
• archlinux:latest: has Python 3.12.
• ubuntu:22.04: has Python 3.10.
• ubuntu:24.04: has Python 3.12.

I mean "has" to indicate the default Python3 version available when installed.

I couldn’t find Docker images for other distributions.

Installing python3-tomli on ubuntu:24.04 seems to have no side effects. The script first checks for import tomllib, so #1614 is valid for both Ubuntu versions.

Here’s a sample script for clarity:

HAVE_TOMLLIB = True
try:
    import tomllib
except ImportError:
    try:
        import tomli as tomllib
    except ImportError:
        HAVE_TOMLLIB = False

if not HAVE_TOMLLIB:
    raise Exception("Neither tomllib nor tomli found.")
else:
    print("Tomli or tomllib found. No problem.")

[TommyMurphyTM1234]

Can somebody clarify what, if anything, needs to be done to address and close this issue? E.g.

1. Nothing? 🙂
2. Modify setup-apt.sh?
3. Update README.md?
4. Something else?

[karaketir16]

Can somebody clarify what, if anything, needs to be done to address and close this

1. Nothing? 🙂
2. Modify setup-apt.sh?
3. Update README.md?
4. Something else?

2. If this file is just for github-actions, no, no need to modify it. But if it's an example of requirements, it would be better to add.
3. is needs to be done. Add requirement python3-tomli  #1614

[TommyMurphyTM1234]

Thanks @karaketir16.

Can somebody clarify what, if anything, needs to be done to address and close this

1. Nothing? 🙂
2. Modify setup-apt.sh?
3. Update README.md?
4. Something else?

2. If this file is just for github-actions, no, no need to modify it. But if it's an example of requirements, it would be better to add.

In my opinion if setup-apt-sh can encapsulate all dependencies then that's a good thing.
I'm assuming that if the image used by the GitHub actions already includes some of the packages then specifying them here will simply be redundant but otherwise innocuous?
In that case a few others (e.g. gcc/g++) should also be added for completeness.

3. is needs to be done. Add requirement python3-tomli  #1614

My personal preference would be, if possible, for the README to simply reference setup-apt.sh as the authoritative list of packages (at least for Ubuntu) and anybody manually installing the prerequisites needs to install these or the equivalents for their Linux distro flavour.