You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2023/03/12 02:05:17 UTC
[GitHub] [airflow] potiuk opened a new pull request, #30050: Installing non-editable airflow for constraint generation
potiuk opened a new pull request, #30050:
URL: https://github.com/apache/airflow/pull/30050
When we are generating constraints including released version of providers or "no providers" (which effectively means preinstalled version of providers only), we should use non-editable installation of airflow.
Normally in CI, we have editable version of airflow and we install providers from sources, but this leads to skipping installation of the preinstalled providers (it is connected to the fact that we were earlier installing as editable and "sources only" and those packages are marked as required, which confuses newer version of pip. It prints that the preinstalled packages are installed, but it does not specify the version - apparently thinking the editable version of the install already contains them.
Since our behaviour is pretty non-standard, this is not really a bug in pip, but it is a bug in our approach when we are adding providers for constraint generation. Rather than using editable installation when we add providers, we should use "regular" installation. This seems to remove the confusion and the preinstalled packages are also installed.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #30050: Installing non-editable airflow for constraint generation
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #30050:
URL: https://github.com/apache/airflow/pull/30050
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #30050: Installing non-editable airflow for constraint generation
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #30050:
URL: https://github.com/apache/airflow/pull/30050#issuecomment-1465073861
(but It worked for me locally)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #30050: Installing non-editable airflow for constraint generation
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #30050:
URL: https://github.com/apache/airflow/pull/30050#issuecomment-1465073824
we will find out after we merge :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #30050: Installing non-editable airflow for constraint generation
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #30050:
URL: https://github.com/apache/airflow/pull/30050#issuecomment-1465074149
> Do we need to regenerate constraints for 2.5.2rc1 after this is merged ?
Yes. we should (when we find it works).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #30050: Installing non-editable airflow for constraint generation
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #30050:
URL: https://github.com/apache/airflow/pull/30050#issuecomment-1465076391
Yep. I manually run constraint generation using the image from the current "main" build running after the push and it seems to work. I will cherry-pick that one to v2-5-test branch and it should regenerate the constrainst when it succeeds, then we could just move the tag
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #30050: Installing non-editable airflow for constraint generation
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #30050:
URL: https://github.com/apache/airflow/pull/30050#issuecomment-1465076816
Build running here (and hopefully it will push new constraints) https://github.com/apache/airflow/actions/runs/4395379619
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org