You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "wolfdn (via GitHub)" <gi...@apache.org> on 2026/04/17 09:46:56 UTC
[PR] Embed GPG keys in repo for Docker build [airflow]
wolfdn opened a new pull request, #65408:
URL: https://github.com/apache/airflow/pull/65408
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
<!--
Thank you for contributing!
Please provide above a brief description of the changes made in this pull request.
Write a good git commit message following this guide: http://chris.beams.io/posts/git-commit/
Please make sure that your code changes are covered with tests.
And in case of new features or big changes remember to adjust the documentation.
Feel free to ping (in general) for the review if you do not see reaction for a few days
(72 Hours is the minimum reaction time you can expect from volunteers) - we sometimes miss notifications.
In case of an existing issue, reference it using one of the following:
* closes: #ISSUE
* related: #ISSUE
-->
## Problem
Building Airflow Docker images fails in environments where GPG's `dirmngr` cannot reach keyservers — most commonly behind corporate HTTP proxies.
`dirmngr` (GnuPG's network daemon, used for `gpg --recv-keys`) resolves DNS **locally** before connecting. In corporate networks where an intercepting DNS server returns NXDOMAIN for external domains (routing all internet access exclusively through an HTTP proxy), `dirmngr` fails with `keyserver receive failed: No keyserver available` before it ever attempts a connection. The `--keyserver-options http-proxy` flag does not help because DNS resolution happens before the proxy is contacted.
## Solution
Embed the GPG public keys directly in the repository under `scripts/docker/keys/` as ASCII-armored `.asc` files. During the Docker build, the scripts use the embedded keys instead of fetching them from keyservers. This is both faster (no network round-trip) and more robust (no dependency on keyserver availability or DNS resolution).
The keys are static — once published for a given key ID, they don't change. The key IDs are already hardcoded in the build scripts, so embedding the full keys is the natural next step.
A prek manual hook (`update-docker-gpg-keys`) is provided to refresh the keys from keyservers when needed. It uses `gpg --recv-keys` with the original keyservers and requires direct internet access.
## Changes
- **`scripts/docker/keys/*.asc`** — new directory containing 4 ASCII-armored GPG public key files
- **`scripts/docker/common.sh`** — `common::import_trusted_gpg()` simplified to read embedded key via `gpg --dearmor`; no keyserver fallback
- **`scripts/docker/install_os_dependencies.sh`** — `install_python()` simplified to import embedded key via `gpg --batch --import`
- **`Dockerfile.ci`, `Dockerfile`** — `COPY scripts/docker/keys/` added; inlined script copies updated to match simplified scripts
- **`scripts/ci/prek/inline_scripts_in_docker.py`** — skip directories when iterating `scripts/docker/` (the new `keys/` subdirectory)
- **`scripts/ci/prek/update_docker_gpg_keys.py`** — new prek manual hook to fetch/refresh keys
- **`.pre-commit-config.yaml`** — register `update-docker-gpg-keys` as a manual stage hook
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR, please
change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by".
-->
- [x] Yes (please specify the tool below)
GitHub Copilot - Claude Opus 4.6
<!--
Generated-by: [Tool Name] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
---
* Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Embed GPG keys in repo for Docker build [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #65408:
URL: https://github.com/apache/airflow/pull/65408
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Embed GPG keys in repo for Docker build [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #65408:
URL: https://github.com/apache/airflow/pull/65408#discussion_r3102662627
##########
.pre-commit-config.yaml:
##########
@@ -926,6 +927,14 @@ repos:
pass_filenames: false
files: ^Dockerfile$|^Dockerfile\.ci$|^scripts/docker/.*$
require_serial: true
+ - id: update-docker-gpg-keys
+ name: Update Docker build GPG keys (manual)
+ entry: ./scripts/ci/prek/update_docker_gpg_keys.py
+ stages: ['manual']
+ language: python
+ pass_filenames: false
+ files: ^scripts/docker/keys/.*$|^scripts/ci/prek/update_docker_gpg_keys\.py$
+ require_serial: true
Review Comment:
nit: I assume we do not need the serial execution here?
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Embed GPG keys in repo for Docker build [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #65408:
URL: https://github.com/apache/airflow/pull/65408#issuecomment-4274470904
### Backport successfully created: v3-2-test
Note: As of [Merging PRs targeted for Airflow 3.X](https://github.com/apache/airflow/blob/main/dev/README_AIRFLOW3_DEV.md#merging-prs-targeted-for-airflow-3x)
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
In matter of doubt please ask in [#release-management](https://apache-airflow.slack.com/archives/C03G9H97MM2) Slack channel.
<table>
<tr>
<th>Status</th>
<th>Branch</th>
<th>Result</th>
</tr>
<tr>
<td>✅</td>
<td>v3-2-test</td>
<td><a href="https://github.com/apache/airflow/pull/65470"><img src="https://img.shields.io/badge/PR-65470-blue" alt="PR Link"></a></td>
</tr>
</table>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Embed GPG keys in repo for Docker build [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #65408:
URL: https://github.com/apache/airflow/pull/65408#issuecomment-4274468350
> Now I probably understand what @potiuk referred to in #65361
Yes. That was **exactly** what I proposed. Thanks @wolfdn :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Embed GPG keys in repo for Docker build [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #65408:
URL: https://github.com/apache/airflow/pull/65408#discussion_r3105754323
##########
.pre-commit-config.yaml:
##########
@@ -926,6 +927,14 @@ repos:
pass_filenames: false
files: ^Dockerfile$|^Dockerfile\.ci$|^scripts/docker/.*$
require_serial: true
+ - id: update-docker-gpg-keys
+ name: Update Docker build GPG keys (manual)
+ entry: ./scripts/ci/prek/update_docker_gpg_keys.py
+ stages: ['manual']
+ language: python
+ pass_filenames: false
+ files: ^scripts/docker/keys/.*$|^scripts/ci/prek/update_docker_gpg_keys\.py$
+ require_serial: true
Review Comment:
I think pass_filename: false implies require_serial, but it does not hurt.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org