Can't build docker images based on slim-2.8.0 or higher #64583
-
|
Our Airflow deployment is woefully behind the times and we're finally getting around to upgrading. First step before v3 is to upgrade from 2.7.3 to 2.11.2. We currently use the base image The build runs successfully on my local machine (running docker v28.1.1), but only fails in Gitlab. The Gitlab CI/CD job is defined with image: docker:28.5.2-cli
services:
- name: docker:28.5.2-dindI've seen many answers to similar issues that suggest something along the lines of including in the Dockerfile, but this fails with "Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8))." (supposedly just a warning but it fails without any other message) I've also tried But that just tells me "debian-archive-keyring is already the newest version (2023.3+deb12u2). debian-archive-keyring set to manually installed." then continues to fail as normal. Finally, I attempted to just run So it seems maybe there's some kind of issue with DNS inside the docker-in-docker container. What I'm curious about is what changed between 2.7.3 and 2.8.0 so that earlier images were not affected by this DNS issue? Also, if anybody has any suggestions I would be very grateful. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 4 replies
-
|
This is a classic symptom of the transition from Debian 11 (Bullseye) to Debian 12 (Bookworm). Airflow 2.7.3 images were Bullseye-based, but 2.8.0 shifted the default to Bookworm. The GPG "NO_PUBKEY" and "invalid signature" errors are usually a red herring in CI environments. The real issue is almost always that your GitLab runner's host is running an older version of Docker or You can usually fix this by updating your GitLab runner's Docker version (to 20.10.10+ at a minimum) or ensuring the If you can't touch the runner infrastructure, you might be able to work around it by temporarily using a more permissive security policy in your build command, though this depends on your runner's configuration: docker build --security-opt seccomp=unconfined .Also, a quick check on your Dockerfile: make sure you're switching to FROM apache/airflow:slim-2.11.2-python3.10
USER root
RUN apt-get update && apt-get install -y ...
USER airflowI'd bet on the Docker-in-Docker version/host compatibility being the primary culprit here, though. Bookworm is much pickier about the container runtime than Bullseye was. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @Nickalus12. Not having access to the runner itself, I tried your It seems that the runner's Docker version is too low, so I've requested it be upgraded to the latest. Once that goes through I will try again and update here. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, the If the upgrade stalls or takes a while, one other thing you could try in the meantime: pin your base image back to a Bullseye variant if one is still available for your Airflow version. Something like Drop me a tag (@Nickalus12) once the runner upgrade lands and I'll help troubleshoot if anything else comes up. |
Beta Was this translation helpful? Give feedback.
-
|
@Nickalus12, I just ran a build on an upgraded Docker runner and it worked. Thank you for the help! |
Beta Was this translation helpful? Give feedback.
-
|
No worries!! Glad to help. Ping me anytime if you need something! |
Beta Was this translation helpful? Give feedback.
This is a classic symptom of the transition from Debian 11 (Bullseye) to Debian 12 (Bookworm). Airflow 2.7.3 images were Bullseye-based, but 2.8.0 shifted the default to Bookworm.
The GPG "NO_PUBKEY" and "invalid signature" errors are usually a red herring in CI environments. The real issue is almost always that your GitLab runner's host is running an older version of Docker or
libseccompthat doesn't support the syscalls (likefaccessat2orclone3) that Bookworm’saptandgpgnow rely on. When the syscall is blocked by an outdated seccomp profile,aptfails to verify signatures and throws these misleading errors.You can usually fix this by updating your GitLab runner's Docker version (t…