You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "kurtrwall (via GitHub)" <gi...@apache.org> on 2025/03/12 18:48:55 UTC
[D] Building Docker image seems to ignore constraints [airflow]
GitHub user kurtrwall created a discussion: Building Docker image seems to ignore constraints
### Apache Airflow version
Other Airflow 2 version (please specify below)
### If "Other Airflow 2 version" selected, which one?
2.10.3
### What happened?
I built an Airflow Docker image using this repo as the docker context on the 2.10.3 tag and I'm running into incompatibility issues with the version of the Celery provider package. The constraints.txt file at `/home/airflow/constraints.txt` baked in the image pins celery at 3.7.2 but `pip` says I have `3.10.2`.
Docker build configuration of the airflow image:
```
build:
context: https://github.com/apache/airflow.git#2.10.3
dockerfile: Dockerfile
args:
ADDITIONAL_DEV_APT_DEPS: gcc
AIRFLOW_BRANCH: 2.10.3
AIRFLOW_EXTRAS: apache-hdfs,apache-kafka,apache-spark,atlassian-jira,celery,docker,elasticsearch,http,ldap,postgres,redis,papermill,virtualenv
AIRFLOW_UID: "2060"
AIRFLOW_VERSION_SPECIFICATION: ==2.10.3
INSTALL_MSSQL_CLIENT: "false"
INSTALL_MYSQL_CLIENT: "false"
INSTALL_POSTGRES_CLIENT: "true"
PYTHON_BASE_IMAGE: python:3.12-slim-bookworm
```
Thanks for any assistance!
### What you think should happen instead?
I expect that the constraints file used would constrain the versions of the libraries installed, but that doesn't seem to be the case.
### How to reproduce
Create a compose file with a service that has a build configuration like so:
```
airflow:
build:
context: https://github.com/apache/airflow.git#2.10.3
dockerfile: Dockerfile
args:
ADDITIONAL_DEV_APT_DEPS: gcc
AIRFLOW_BRANCH: 2.10.3
AIRFLOW_EXTRAS: apache-hdfs,apache-kafka,apache-spark,atlassian-jira,celery,docker,elasticsearch,http,ldap,postgres,redis,papermill,virtualenv
AIRFLOW_UID: "2060"
AIRFLOW_VERSION_SPECIFICATION: ==2.10.3
INSTALL_MSSQL_CLIENT: "false"
INSTALL_MYSQL_CLIENT: "false"
INSTALL_POSTGRES_CLIENT: "true"
PYTHON_BASE_IMAGE: python:3.12-slim-bookworm
```
### Operating System
OSX
### Versions of Apache Airflow Providers
_No response_
### Deployment
Other Docker-based deployment
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
GitHub link: https://github.com/apache/airflow/discussions/47693
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Building Docker image seems to ignore constraints [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
GitHub user potiuk added a comment to the discussion: Building Docker image seems to ignore constraints
If you get that information out, I might be able to help you
GitHub link: https://github.com/apache/airflow/discussions/47693#discussioncomment-12479234
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Building Docker image seems to ignore constraints [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
GitHub user potiuk added a comment to the discussion: Building Docker image seems to ignore constraints
Also I think likely `AIRFLOW_VERSION_SPECIFICATION` should not be used. This is what might cause problems - it's optional and I am not sure what effect it has on constraints, we do not use it in any of the examples )
GitHub link: https://github.com/apache/airflow/discussions/47693#discussioncomment-12479257
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Building Docker image seems to ignore constraints [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
GitHub user potiuk added a comment to the discussion: Building Docker image seems to ignore constraints
Specifically please take a close look at the exact `pip install` command that are used and which files are downloaded as constraints and what packages are installed after every step.
GitHub link: https://github.com/apache/airflow/discussions/47693#discussioncomment-12479230
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Building Docker image seems to ignore constraints [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
GitHub user potiuk edited a comment on the discussion: Building Docker image seems to ignore constraints
This is interesting discussion, but not clear if this is isssue.
The way how you should build image and what build args to pass is described here with `docker build` commands - https://airflow.apache.org/docs/docker-stack/build.html#examples-of-image-customizing - se also arguments reference
You miss `AIRFLOW_VERSION` to be specified likely (but hard to say). If you need help with debugging stuff, please take a look at detailed output of the build process where you will see the exact steps and what commands are run at every step and see if you can figure out what is wrong. Generally, self analyse the output. If still in doubt, please copy && paste relevant output fragments showing the wrong constraints being used.
GitHub link: https://github.com/apache/airflow/discussions/47693#discussioncomment-12479188
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Building Docker image seems to ignore constraints [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
GitHub user potiuk added a comment to the discussion: Building Docker image seems to ignore constraints
This is interesting discussion, but not clear if this is isssue.
The way how you should build image and what build args to pass is described here with `docker build` commands - https://airflow.apache.org/docs/docker-stack/build.html#examples-of-image-customizing - se also arguments reference
You miss `AIRFLOW_VERSION` to be specified likely. If you need help with debugging stuff, please take a look at detailed output of the build process where you will see the exact steps and what commands are run at every step and see if you can figure out what is wrong. Generally, self analyse the output. If still in doubt, please copy && paste relevant output fragments showing the wrong constraints being used.
GitHub link: https://github.com/apache/airflow/discussions/47693#discussioncomment-12479188
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org