You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "snicki13 (via GitHub)" <gi...@apache.org> on 2026/04/09 10:56:39 UTC
[I] Constraints for Airflow 3.2.0 and Python 3.14 out of sync in Gitlab / Dockerfile [airflow]
snicki13 opened a new issue, #64954:
URL: https://github.com/apache/airflow/issues/64954
### Under which category would you file this issue?
Providers
### Apache Airflow version
3.2.0
### What happened and how to reproduce it?
I installed Airflow 3.2.0 into my dev-environment with Python 3.14 via this command:
`uv pip install apache-airflow[celery,http,pandas,postgres,sentry]=="3.2.0" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-3.2.0/constraints-3.14.txt"`
I then build my own Docker image by using the frozen requirements.txt from my dev environment and run pip install with it. The image build failes, because it cannot find a compiler to install numpy.
The requirements.txt resolved from this command my initial installation specifies numpy 2.2.6.
However, the base Docker image at docker.io/apache/airflow:3.2.0 (3cbaa4753752) comes with numpy 2.4.3.
When I use the constraint list for python3.13 at "https://raw.githubusercontent.com/apache/airflow/constraints-3.2.0/constraints-3.13.txt", the image builds.
### What you think should happen instead?
The constraints of Airflow 3.2.0 and Python 3.14 should be the same in the Docker image and via pip install.
When installing my requirements.txt - based on airflow requirements - the numpy version should match.
### Operating System
MacOS 26.4
### Deployment
Official Apache Airflow Helm Chart
### Apache Airflow Provider(s)
_No response_
### Versions of Apache Airflow Providers
_No response_
### Official Helm Chart version
1.20.0 (latest released)
### Kubernetes Version
Not Applicable
### Helm Chart configuration
Not Applicable
### Docker Image customizations
FROM ${AIRFLOW_IMAGE_NAME}
ARG AIRFLOW_IMAGE_NAME=docker.io/apache/airflow:${AIRFLOW_VERSION}
COPY airflow/uv.lock .
COPY airflow/pyproject.toml .
RUN uv export --format requirements.txt > requirements.txt && \
pip install --no-cache-dir "apache-airflow==${AIRFLOW_VERSION}" -r requirements.txt
### Anything else?
This is my full script to generate the uv.lock and pyproject.toml. The requirements.txt already contains the wrong version. The error is the same, when I just add the requirements.txt to the Docker image.
```
PYTHON_VERSION=$(python3 -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')")
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
EXTRAS=celery,http,pandas,postgres,sentry
uv pip install apache-airflow[$EXTRAS]=="$AIRFLOW_VERSION" --constraint "$CONSTRAINT_URL"
uv pip freeze > requirements.txt
uv add -r requirements.txt
```
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Constraints for Airflow 3.2.0 and Python 3.14 out of sync in Github and Dockerfile [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #64954:
URL: https://github.com/apache/airflow/issues/64954#issuecomment-4216826950
> It is unexpected for me, that the newer Python version expects the older version of numpy and I considered this an error in the constraint file. Especially seeing, that the Docker image of Airflow 3.2.0 with Python 3.14 brings numpy 2.4.3.
Not really:
```
⏚ [jarekpotiuk:~/code/airflow] main+ 127 ± docker run -it apache/airflow:3.2.0-python3.14 bash -c "pip freeze | grep numpy"
numpy==2.2.6
⏚ [jarekpotiuk:~/code/airflow] main+ ±
```
But I am glad you asked - I understand you are curious, that picked my curiosity as well and made me review if all is good in our pipeline. It looks like it is, but. will likely add some protection on making sure that release manager double checks that the constraints are regenerated "just" before release (this time they were generated few days before release - and we got some timing issue).
The reason why in 3.14 we have numpy 2.2.6 is because some of the providers held it from upgraded. If you look a the commits to the constraints-3-2 branch numpy is already upgraded to newer version for 3.14 for the upcoming 3.2.1 release - and it came with a number of providers upgraded - as they were released after the constraints have been resolved automatically by the resolution algorithm:
You can see the update in this commit - few days ago - after the constraints for 3.2.0 were generated (this was after new wave of providers was released and after 3.2.0rc2 was prepared for voting.
https://github.com/apache/airflow/actions/runs/24048921103/job/70144926268#step:11:2547
Most likely - if you would really like to understand why you might dig deeper - you will find that some of the updated provider's previous version held it back. Most likely amazon provider. We had some dependencies updated recently there for improved Python 3.14 compatibilty and likely that one freed numpy from upgrading. Python dependencies (and dependencies in general) and resolving them is a very complex task that we delegate to uv to resolve and actively work on freeing some of the versions that are held by other dependencies.
We even have this nice tool I developed to tell us why we cannot upgrade something and when you run it v3-2-test it nicely shows we are not held back any more (but it was)
<img width="1569" height="398" alt="Image" src="https://github.com/user-attachments/assets/093a4a32-460a-4d21-8e3c-630c7d814349" />
So no - this is not a bug - just timing of preparation of the image and constraints.
And just a comment for you - it's likely fine (in case your combination of providers) - to **not** use constraints when you are using your requirements - you are not limited by constraints. Constraints (as clearly explained in our docs: https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#constraints-files are not "requirements" - they are constraints and they are useful to do a reproducible installation of airlfow and our container image - that's about it. And there is absolutely no guarantee that you should not use compilers for that - many of our dependencies need compilation step at various Python versions or systems. So there is absolutely no guarantee compilation is not needed when you install airflow. This is not airflow - this is how python installation works. And if you install it in an environment where you have no compiler, you were simply lucky that you have not needed it for anything else. But it's neither expected nor guaran
teed - especially if you choose to build your own image from scratch.
Also - you are absolutely not blocked. This is explained in our documentation https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#using-your-own-constraints - you can download airflow constraints, and modify them in the way that you see fit and use them instead of airflow constraints - for example in this case updating numpy version, testing it, and using your own constraints file. It should be pretty easy to modify your Dockerfile to handle the case - even if it is exceptional.
We could theorethically regenrate those constraints - but since we have not tested it during release, we cannot guarantee it works - so us updating the constraints is not really an option. There are very good reasons why we are fixing constraints at release time and we are updating them only in very exceptiona situations - when airflow stops instaling at all due to setuptools release or some other "breaking" events.
This is explained here - what's the reasoning and why we will not do when single user (or even few) has problem they can very easily workaround:
https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#fixing-constraints-at-release-time
Also the reasons we actuallly regenerated constraints and rebuild our images are summarized here:
https://airflow.apache.org/docs/docker-stack/changelog.html#changes-after-publishing-the-images
There were 9 such events since 2022.
Re-generating constraints and rebuilding the image is extra effort for maintainers and we only do that when our users are completely blocked and have no known workarounds to fix serious installation problems with Airflow.
This is certainly not the case - you have ways to workaround your problem,
Summarising:
* Numpy in constraints is **working** in Python 3.14 - and some of our providers kept it to lower version when we released Airflow.
* It will be upgraded in the future release when the condition that held it was removed
* you have ways how to install newer version of Numpy for Python 3.14
I hope this explanation is sufficient?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Constraints for Airflow 3.2.0 and Python 3.14 out of sync in Github and Dockerfile [airflow]
Posted by "snicki13 (via GitHub)" <gi...@apache.org>.
snicki13 commented on issue #64954:
URL: https://github.com/apache/airflow/issues/64954#issuecomment-4215147719
@potiuk Thank you for you explanation. However, let me rephrase my issue statement as I may have convoluted it.
The constraints for Airflow 3.2.0 with Python 3.13 contain numpy==2.4.3, while Airflow 3.2.0 with Python 3.14 constrains numpy at 2.2.6.
See here: https://github.com/apache/airflow/blob/cd9049cdc66ffb51fd021fab9c28a5234bd3735f/constraints-3.13.txt#L478
And here: https://github.com/apache/airflow/blob/cd9049cdc66ffb51fd021fab9c28a5234bd3735f/constraints-3.14.txt#L475
It is unexpected for me, that the newer Python version expects the older version of numpy and I considered this an error in the constraint file. Especially seeing, that the Docker image of Airflow 3.2.0 with Python 3.14 brings numpy 2.4.3.
I currently have no dependencies in my requirements.txt or pyproject.toml that are not already pulled by the Airflow dependencies.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Constraints for Airflow 3.2.0 and Python 3.14 out of sync in Github and Dockerfile [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #64954: Constraints for Airflow 3.2.0 and Python 3.14 out of sync in Github and Dockerfile
URL: https://github.com/apache/airflow/issues/64954
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Constraints for Airflow 3.2.0 and Python 3.14 out of sync in Github and Dockerfile [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #64954:
URL: https://github.com/apache/airflow/issues/64954#issuecomment-4213721140
cc @potiuk @vatsrahul1001 I think discrepancy happened because constraints were not refreshed when RC2 was generated?
https://raw.githubusercontent.com/apache/airflow/constraints-3.2.0/constraints-3.14.txt
shows `2026-03-31T10:16:48.198634` which is when we cut RC1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Constraints for Airflow 3.2.0 and Python 3.14 out of sync in Gitlab / Dockerfile [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #64954:
URL: https://github.com/apache/airflow/issues/64954#issuecomment-4213600985
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Constraints for Airflow 3.2.0 and Python 3.14 out of sync in Github and Dockerfile [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #64954:
URL: https://github.com/apache/airflow/issues/64954#issuecomment-4215015429
Not really - while the constraints were not super refreshed - because apparently v3-2-test build did not succeed just before releasing, they are distinct for 3.2.0rc2 and rc1:
And yes @snicki13 - if you have a dependency, that requires complilation - for example the numpy version of yours does not have wheels prepared for 3.14, then that dependency needs to be built from sources using compiler - and this is something that is **expected** for dependencies in Python - nothing new - it's been like that for, practically, ever.
You have forgotten to mention which python version you use or post content of your requirements.txt
Note that Airflow image is a reference image that is optimized for size - and it does not have compilers installed - which saves a lot of size of the image (200 MB or so).
This is all described in detail in the documentaiton for the image: https://airflow.apache.org/docs/docker-stack/build.html
And this chapter and following explains the approaches you can take - either make your image biger with adding compllers or rebuilding your image from scratch adding your dependencies that makes your image similarly optimized as Airflow in case you need to add packages that require compilation:
Just read the docs https://airflow.apache.org/docs/docker-stack/build.html#extending-vs-customizing-the-image
Here is the log of constraints (git log apache/constrainst-3-2):
```
commit cd9049cdc66ffb51fd021fab9c28a5234bd3735f (tag: constraints-latest, tag: constraints-3.2.0rc2, tag: constraints-3.2.0)
Author: Automated GitHub Actions commit <de...@airflow.apache.org>
Date: Tue Mar 31 10:24:20 2026 +0000
Updating constraints. GitHub run id:23790129915
This update in constraints is automatically committed by the CI 'constraints-push' step based on
'refs/heads/v3-2-test' in the 'apache/airflow' repository with commit sha 23b5c3bd8fa9ebec11265427f8564914b46ac1c3.
The action that build those constraints can be found at https://github.com/apache/airflow/actions/runs/23790129915/
All tests passed in this build so we determined we can push the updated constraints.
See https://github.com/apache/airflow/blob/main/README.md#installing-from-pypi for details.
commit c7271569490049480169ca6369279a7f1b23efa5 (tag: constraints-3.2.0rc1)
Author: Automated GitHub Actions commit <de...@airflow.apache.org>
Date: Mon Mar 30 17:49:01 2026 +0000
Updating constraints. GitHub run id:23756874655
This update in constraints is automatically committed by the CI 'constraints-push' step based on
'refs/heads/v3-2-test' in the 'apache/airflow' repository with commit sha 9af7e89f5702db34f377742a5071f03472128077.
The action that build those constraints can be found at https://github.com/apache/airflow/actions/runs/23756874655/
All tests passed in this build so we determined we can push the updated constraints.
See https://github.com/apache/airflow/blob/main/README.md#installing-from-pypi for details.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Constraints for Airflow 3.2.0 and Python 3.14 out of sync in Github and Dockerfile [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #64954:
URL: https://github.com/apache/airflow/issues/64954#issuecomment-4215025881
BTW. @eladkal -> as others might have similar issues - maybe a good idea to send them to that answer.
This is quite expected to happen where people use old versions of packages and expect them to install on newer Python version - when the version of their dependencies was released when Python 3.14 was not even a thing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org