[airflow] branch main updated: Updates to check_files to produce a more useful Dockerfile (#29469)

[po...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
     new 9845d7b565 Updates to check_files to produce a more useful Dockerfile (#29469)
9845d7b565 is described below

commit 9845d7b5650f3d9b35874fa424ef2a3d820f8fe8
Author: Niko Oliveira <on...@amazon.com>
AuthorDate: Sat Mar 4 14:41:54 2023 -0800

    Updates to check_files to produce a more useful Dockerfile (#29469)
    
    - Also add more documentation on how to use the generated
    Dockerfile for testing installation
---
 dev/README_RELEASE_PROVIDER_PACKAGES.md | 14 ++++++++++++--
 dev/check_files.py                      |  7 +++++--
 2 files changed, 17 insertions(+), 4 deletions(-)

diff --git a/dev/README_RELEASE_PROVIDER_PACKAGES.md b/dev/README_RELEASE_PROVIDER_PACKAGES.md
index 44ef698213..97c7aa7794 100644
--- a/dev/README_RELEASE_PROVIDER_PACKAGES.md
+++ b/dev/README_RELEASE_PROVIDER_PACKAGES.md
@@ -569,14 +569,24 @@ Or update it if you already checked it out:
 svn update .
 ```
 
-Optionally you can use `check_files.py` script to verify that all expected files are
-present in SVN. This script may help also with verifying installation of the packages.
+Optionally you can use the [`check_files.py`](https://github.com/apache/airflow/blob/main/dev/check_files.py)
+script to verify that all expected files are present in SVN. This script will produce a `Dockerfile.pmc` which
+may help with verifying installation of the packages.
 
 ```shell script
 # Copy the list of packages (pypi urls) into `packages.txt` then run:
 python check_files.py providers -p {PATH_TO_SVN}
 ```
 
+After the above script completes you can build `Dockerfile.pmc` to trigger an installation of each provider
+package and verify the correct versions are installed:
+
+```shell script
+docker build -f Dockerfile.pmc --tag local/airflow .
+docker run --rm --entrypoint "airflow" local/airflow info
+docker image rm local/airflow
+```
+
 ### Licences check
 
 This can be done with the Apache RAT tool.
diff --git a/dev/check_files.py b/dev/check_files.py
index 8a013b3672..158ac5c046 100644
--- a/dev/check_files.py
+++ b/dev/check_files.py
@@ -24,7 +24,9 @@ import rich_click as click
 from rich import print
 
 PROVIDERS_DOCKER = """\
-FROM apache/airflow:latest
+FROM ghcr.io/apache/airflow/main/ci/python3.10
+RUN rm -rf /opt/airflow/airflow/providers
+
 
 # Install providers
 {}
@@ -81,7 +83,8 @@ def create_docker(txt: str):
     print(
         """\
         docker build -f Dockerfile.pmc --tag local/airflow .
-        docker run --rm local/airflow airflow info
+        docker run --rm --entrypoint "airflow" local/airflow info
+        docker image rm local/airflow
         """
     )