Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix image build #69

Merged
merged 3 commits into from
Jan 31, 2025
Merged

Fix image build #69

merged 3 commits into from
Jan 31, 2025

Conversation

theihor
Copy link
Contributor

@theihor theihor commented Jan 31, 2025

Review and clean up the list of packages installed during runner image build: remove duplicates, use common list between main and s390x Dockerfiles.

  • remove linux-image-generic
  • replace python3 with python3-minimal.
  • install LLVM with an official automatic script.
  • set UBUNTU_VERSION to noble by default

This addresses recent build failures:

@theihor theihor force-pushed the packages-list-reorg branch 2 times, most recently from 030b6d9 to c835311 Compare January 31, 2025 00:56
@theihor
Copy link
Contributor Author

theihor commented Jan 31, 2025

Looks like we are not the only ones getting the python error on apt install software-properties-common:

2025-01-31T00:45:50.1293461Z #6 379.5 Setting up python3-lazr.restfulclient (0.14.6-1) ...
2025-01-31T00:45:50.8910192Z #6 380.4 Traceback (most recent call last):
2025-01-31T00:45:51.0794488Z #6 380.4   File "/usr/bin/py3compile", line 323, in <module>
2025-01-31T00:45:51.0796688Z #6 380.4     main()
2025-01-31T00:45:51.0797106Z #6 380.4   File "/usr/bin/py3compile", line 302, in main
2025-01-31T00:45:51.0797592Z #6 380.4     compile(files, versions,
2025-01-31T00:45:51.0798096Z #6 380.4   File "/usr/bin/py3compile", line 203, in compile
2025-01-31T00:45:51.0798626Z #6 380.4     interpreter.magic_number(version),
2025-01-31T00:45:51.0799132Z #6 380.4     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-31T00:45:51.0799813Z #6 380.4   File "/usr/share/python3/debpython/interpreter.py", line 233, in magic_number
2025-01-31T00:45:51.0800624Z #6 380.4     result = self._execute('import importlib.util; print(importlib.util.MAGIC_NUMBER)', version)
2025-01-31T00:45:51.0801455Z #6 380.4              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-31T00:45:51.0801983Z #6 380.4   File "/usr/share/python3/debpython/interpreter.py", line 359, in _execute
2025-01-31T00:45:51.0802628Z #6 380.4     raise Exception('{} failed with status code {}'.format(command, output['returncode']))
2025-01-31T00:45:51.0803373Z #6 380.4 Exception: ('python3.12', '-c', 'import importlib.util; print(importlib.util.MAGIC_NUMBER)') failed with status code -11
2025-01-31T00:45:51.0804079Z #6 380.5 dpkg: error processing package python3-lazr.restfulclient (--configure):
2025-01-31T00:45:51.0804854Z #6 380.5  installed python3-lazr.restfulclient package post-installation script subprocess returned error exit status 1
2025-01-31T00:45:51.0805716Z #6 380.5 Setting up libgtk-3-common (3.24.41-4ubuntu1.2) ...
2025-01-31T00:45:51.0806308Z #6 380.5 Setting up libgs10:s390x (10.02.1~dfsg1-0ubuntu7.4) ...
2025-01-31T00:45:51.0806805Z #6 380.5 Setting up gsettings-desktop-schemas (46.1-0ubuntu1) ...
2025-01-31T00:45:51.0807527Z #6 380.5 dpkg: dependency problems prevent configuration of python3-launchpadlib:
2025-01-31T00:45:51.0808373Z #6 380.5  python3-launchpadlib depends on python3-lazr.restfulclient; however:
2025-01-31T00:45:51.0809153Z #6 380.5   Package python3-lazr.restfulclient is not configured yet.
2025-01-31T00:45:51.0809465Z #6 380.5 
2025-01-31T00:45:51.0809890Z #6 380.5 dpkg: error processing package python3-launchpadlib (--configure):
2025-01-31T00:45:51.0810367Z #6 380.5  dependency problems - leaving unconfigured
2025-01-31T00:45:51.0810950Z #6 380.5 Setting up ghostscript (10.02.1~dfsg1-0ubuntu7.4) ...
2025-01-31T00:45:51.0811758Z #6 380.6 dpkg: dependency problems prevent configuration of python3-software-properties:
2025-01-31T00:45:51.0812673Z #6 380.6  python3-software-properties depends on python3-launchpadlib; however:
2025-01-31T00:45:51.0813429Z #6 380.6   Package python3-launchpadlib is not configured yet.
2025-01-31T00:45:51.0814269Z #6 380.6 
2025-01-31T00:45:51.2830740Z #6 380.6 dpkg: error processing package python3-software-properties (--configure):
2025-01-31T00:45:51.2833046Z #6 380.6  dependency problems - leaving unconfigured
2025-01-31T00:45:51.2833641Z #6 380.6 Setting up at-spi2-core (2.52.0-1build1) ...
2025-01-31T00:45:51.2834246Z #6 380.6 Setting up glib-networking:s390x (2.80.0-1build1) ...
2025-01-31T00:45:51.2834925Z #6 380.6 Setting up libsoup-3.0-0:s390x (3.4.4-5ubuntu0.1) ...
2025-01-31T00:45:51.2835757Z #6 380.6 dpkg: dependency problems prevent configuration of software-properties-common:
2025-01-31T00:45:51.2837044Z #6 380.6  software-properties-common depends on python3-software-properties (= 0.99.49.1); however:
2025-01-31T00:45:51.2837955Z #6 380.6   Package python3-software-properties is not configured yet.
2025-01-31T00:45:51.2838513Z #6 380.6 
2025-01-31T00:45:51.2839009Z #6 380.6 dpkg: error processing package software-properties-common (--configure):
2025-01-31T00:45:51.2839733Z #6 380.6  dependency problems - leaving unconfigured

Review and clean up the list of packages installed during runner image
build: remove duplicates, use common list between main and s390x
Dockerfiles.

* remove linux-image-generic
* replace python3 with python3-minimal.
* install LLVM with an official automatic script.
* set UBUNTU_VERSION to noble by default

This addresses recent build failures:
* https://github.com/kernel-patches/runner/actions/runs/13058686735
* https://github.com/kernel-patches/runner/actions/runs/13037837932

Signed-off-by: Ihor Solodrai <[email protected]>
Add theihor (Ihor Solodrai) to reviewers in version_bump workflow.

Signed-off-by: Ihor Solodrai <[email protected]>
@theihor theihor force-pushed the packages-list-reorg branch from 730e943 to 2fe66e9 Compare January 31, 2025 01:17
@theihor theihor changed the title Reorganize list of packages installed on runners Fix image build Jan 31, 2025
@theihor
Copy link
Contributor Author

theihor commented Jan 31, 2025

Ok, this comment was very helpful: docker/setup-qemu-action#188 (comment)

Looks like "default" qemu installed by docker/setup-qemu-action is at fault. Specifying more recent qemu version fixed our build.

@theihor theihor merged commit a8ec125 into kernel-patches:main Jan 31, 2025
3 checks passed
theihor added a commit to libbpf/libbpf that referenced this pull request Jan 31, 2025
The run-on-arch action is using docker to run things on various
platforms. Docker in turn relies on qemu-user-static installed on the
system.

Recently there were various reports about multi-arch docker builds
failing with seemingly random issues, and it appears to boil down to
qemu [1]. I stumbled on this problem while updating s390x runners [2],
and setting up more recent version of qemu helped.

[1] docker/setup-qemu-action#188
[2] kernel-patches/runner#69

Signed-off-by: Ihor Solodrai <[email protected]>
theihor added a commit to libbpf/libbpf that referenced this pull request Jan 31, 2025
The run-on-arch action is using docker to run things on various
platforms. Docker in turn relies on qemu-user-static installed on the
system.

Recently there were various reports about multi-arch docker builds
failing with seemingly random issues, and it appears to boil down to
qemu [1]. I stumbled on this problem while updating s390x runners [2],
and setting up more recent version of qemu helped.

Install docker buildx and enforce it's usage with DOCKER_BUILDKIT=1
[3], so that it's used by run-on-arch action.

[1] docker/setup-qemu-action#188
[2] kernel-patches/runner#69
[3] https://docs.docker.com/build/buildkit/#getting-started

Signed-off-by: Ihor Solodrai <[email protected]>
theihor added a commit to theihor/libbpf that referenced this pull request Jan 31, 2025
run-on-arch-action is simply a wrapper around docker. There is no
value in using it in libbpf, as it is not complicated to run
non-native arch docker images directly on github-hosted runners.

Docker relies on qemu-user-static installed on the system to emulate
different architectures.

Recently there were various reports about multi-arch docker builds
failing with seemingly random issues, and it appears to boil down to
qemu [1]. I stumbled on this problem while updating s390x runners [2]
for BPF CI, and setting up more recent version of qemu helped.

This change addresses recent build failures on s390x and ppc64le.

[1] docker/setup-qemu-action#188
[2] kernel-patches/runner#69
[3] https://docs.docker.com/build/buildkit/#getting-started

Signed-off-by: Ihor Solodrai <[email protected]>
anakryiko pushed a commit to libbpf/libbpf that referenced this pull request Feb 1, 2025
run-on-arch-action is simply a wrapper around docker. There is no
value in using it in libbpf, as it is not complicated to run
non-native arch docker images directly on github-hosted runners.

Docker relies on qemu-user-static installed on the system to emulate
different architectures.

Recently there were various reports about multi-arch docker builds
failing with seemingly random issues, and it appears to boil down to
qemu [1]. I stumbled on this problem while updating s390x runners [2]
for BPF CI, and setting up more recent version of qemu helped.

This change addresses recent build failures on s390x and ppc64le.

[1] docker/setup-qemu-action#188
[2] kernel-patches/runner#69
[3] https://docs.docker.com/build/buildkit/#getting-started

Signed-off-by: Ihor Solodrai <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant