Skip to content

Commit

Permalink
unix: call ulimit -n to avoid overheads (#468)
Browse files Browse the repository at this point in the history
See inline comment in `base.Dockerfile` for why this hack is necessary.

When upgrading my machine recently, the `rlim_max` value increased to
~1B, making `apt-get` invocations take multiple minutes to complete.
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1332440 seems to
describe this issue.

The cross building images have a sufficiently new apt that doesn't seem
impacted.
  • Loading branch information
indygreg authored Jan 8, 2025
1 parent 8999425 commit 61e7c99
Show file tree
Hide file tree
Showing 4 changed files with 12 additions and 4 deletions.
10 changes: 9 additions & 1 deletion cpython-unix/base.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,12 @@ RUN ( echo 'amd64'; \
echo 'i386'; \
) > /var/lib/dpkg/arch

RUN apt-get update
# apt iterates all available file descriptors up to rlim_max and calls
# fcntl(fd, F_SETFD, FD_CLOEXEC). This can result in millions of system calls
# (we've seen 1B in the wild) and cause operations to take seconds to minutes.
# Setting a fd limit mitigates.
#
# Attempts at enforcing the limit globally via /etc/security/limits.conf and
# /root/.bashrc were not successful. Possibly because container image builds
# don't perform a login or use a shell the way we expect.
RUN ulimit -n 10000 && apt-get update
2 changes: 1 addition & 1 deletion cpython-unix/build.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
# Compression packages are needed to extract archives.
#
# Various other build tools are needed for various building.
RUN apt-get install \
RUN ulimit -n 10000 && apt-get install \
bzip2 \
file \
libc6-dev \
Expand Down
2 changes: 1 addition & 1 deletion cpython-unix/gcc.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{% include 'base.Dockerfile' %}
RUN apt-get install \
RUN ulimit -n 10000 && apt-get install \
autoconf \
automake \
bison \
Expand Down
2 changes: 1 addition & 1 deletion cpython-unix/xcb.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
{% include 'build.Dockerfile' %}
RUN apt-get install \
RUN ulimit -n 10000 && apt-get install \
python

0 comments on commit 61e7c99

Please sign in to comment.