You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/10/08 22:18:57 UTC
[GitHub] [airflow] kaxil opened a new pull request #11363: Use only-if-needed upgrade strategy for PRs
kaxil opened a new pull request #11363:
URL: https://github.com/apache/airflow/pull/11363
Currently, upgrading dependencies in setup.py still run with previous versions of the package for the PR which fails.
This PR will change it to only upgrade the package if a requirement is not satisfied (i.e. if we change the upper or lower bound of dependency in setup.py).
This will avoid the situation like https://github.com/apache/airflow/pull/11020 and https://github.com/apache/airflow/pull/11313 where we can't test upgrading the package in the PR itself. And we were "deadlocked" even after merging the PR as the constraints won't allow installing the new version of the package, I had to generate new constraints myself and push to constraints-master branch (https://github.com/apache/airflow/commit/e6e840f16c33225bb80cf7836ef6a355262e6594), this is not ideal and won't be possible for just contributors.
Using `only-if-needed` upgrade strategy - packages are only upgraded if they are named in the pip command or a requirement file (i.e, they are direct requirements), or an upgraded parent needs a later version of the dependency than is currently installed.
Example:
**Using `only-if-needed`**:
```
root@822984cd3cf4:/opt/airflow# pip install -e . --upgrade --upgrade-strategy only-if-needed
Obtaining file:///opt/airflow
Requirement already satisfied, skipping upgrade: alembic<2.0,>=1.2 in /usr/local/lib/python3.7/site-packages (from apache-airflow==2.0.0.dev0) (1.4.3)
Requirement already satisfied, skipping upgrade: jinja2<2.12.0,>=2.10.1 in /usr/local/lib/python3.7/site-packages (from apache-airflow==2.0.0.dev0) (2.11.2)
Requirement already satisfied, skipping upgrade: json-merge-patch==0.2 in /usr/local/lib/python3.7/site-packages (from apache-airflow==2.0.0.dev0) (0.2)
Requirement already satisfied, skipping upgrade: tabulate<0.9,>=0.7.5 in /usr/local/lib/python3.7/site-packages (from apache-airflow==2.0.0.dev0) (0.8.7)
Collecting tenacity~=6.2.0
Downloading tenacity-6.2.0-py2.py3-none-any.whl (24 kB)
```
vs
**Using `eager`**:
```
root@822984cd3cf4:/opt/airflow# pip install -e . --upgrade --upgrade-strategy eager
Obtaining file:///opt/airflow
Requirement already up-to-date: alembic<2.0,>=1.2 in /usr/local/lib/python3.7/site-packages (from apache-airflow==2.0.0.dev0) (1.4.3)
Requirement already up-to-date: argcomplete~=1.10 in /usr/local/lib/python3.7/site-packages (from apache-airflow==2.0.0.dev0) (1.12.1)
Requirement already up-to-date: cryptography>=0.9.3 in /usr/local/lib/python3.7/site-packages (from apache-airflow==2.0.0.dev0) (3.1.1)
Requirement already up-to-date: dill<0.4,>=0.2.2 in /usr/local/lib/python3.7/site-packages (from apache-airflow==2.0.0.dev0) (0.3.2)
Requirement already up-to-date: funcsigs<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/site-packages (from apache-airflow==2.0.0.dev0) (1.0.2)
Collecting graphviz>=0.12
Downloading graphviz-0.14.2-py2.py3-none-any.whl (18 kB)
....
....
```
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #11363: Use only-if-needed upgrade strategy to install deps for PRs
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #11363:
URL: https://github.com/apache/airflow/pull/11363
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #11363: Use only-if-needed upgrade strategy to install deps for PRs
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #11363:
URL: https://github.com/apache/airflow/pull/11363#issuecomment-706032422
Cool!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11363: Use only-if-needed upgrade strategy to install deps for PRs
Posted by GitBox <gi...@apache.org>.
kaxil commented on pull request #11363:
URL: https://github.com/apache/airflow/pull/11363#issuecomment-705854254
https://github.com/apache/airflow/pull/11020 is blocked without this PR
cc @baolsen
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #11363: Use only-if-needed upgrade strategy to install deps for PRs
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #11363:
URL: https://github.com/apache/airflow/pull/11363#issuecomment-706032422
Cool!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #11363: Use only-if-needed upgrade strategy to install deps for PRs
Posted by GitBox <gi...@apache.org>.
kaxil commented on pull request #11363:
URL: https://github.com/apache/airflow/pull/11363#issuecomment-705854254
https://github.com/apache/airflow/pull/11020 is blocked without this PR
cc @baolsen
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #11363: Use only-if-needed upgrade strategy to install deps for PRs
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #11363:
URL: https://github.com/apache/airflow/pull/11363
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org