You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/02/22 23:15:13 UTC
[GitHub] [airflow] potiuk opened a new pull request #14368: Use latest Pip version when building image.
potiuk opened a new pull request #14368:
URL: https://github.com/apache/airflow/pull/14368
The initial problem with the new PIP resolved in 20.3 version have
been successfully solved. Seems that version 21.* is much more
stable and actually works in all cases, so we are switching back
to it.
Also changed pip and wheel dependencies to ~= (compatible) version
hoping that the experience of backwards incomptible release in
major version update have been adopted by the PIP team with
21* series release.
Closes #12838
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #14368: Use latest Pip version when building image.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14368:
URL: https://github.com/apache/airflow/pull/14368#issuecomment-784105647
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #14368: Use latest Pip version when building image.
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #14368:
URL: https://github.com/apache/airflow/pull/14368#issuecomment-784047657
Just as a coment - still we will not merge it yet - PIP 21.* still have some problems with installing providers (it again downloads multiple version of some packages and it takes > 30 minutes) and I need to take a look :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #14368: Use latest Pip version when building image.
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #14368:
URL: https://github.com/apache/airflow/pull/14368#issuecomment-784049013
> Just as a coment - still we will not merge it yet - PIP 21.* still have some problems with installing providers (it again downloads multiple version of some packages and it takes > 30 minutes) and I need to take a look :)
I've converted this PR to draft then so it doesn't get merged by mistake.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #14368: Use latest Pip version when building image.
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #14368:
URL: https://github.com/apache/airflow/pull/14368#issuecomment-815178948
Blockers. Apache Beam needs to be released including this PR : https://github.com/apache/beam/pull/14328
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #14368: Use latest Pip version when building image.
Posted by GitBox <gi...@apache.org>.
uranusjr commented on pull request #14368:
URL: https://github.com/apache/airflow/pull/14368#issuecomment-814834932
Is this no longer relevant, or are there blockers making this inviable?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #14368: Use latest Pip version when building image.
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #14368:
URL: https://github.com/apache/airflow/pull/14368#discussion_r580898233
##########
File path: Dockerfile
##########
@@ -190,7 +190,7 @@ ENV AIRFLOW_INSTALL_USER_FLAG="--user"
ENV AIRFLOW_INSTALL_EDITABLE_FLAG=""
# Upgrade to specific PIP version
-RUN pip install --upgrade "pip==${AIRFLOW_PIP_VERSION}"
+RUN pip install --upgrade "pip~=${AIRFLOW_PIP_VERSION}"
Review comment:
Well that what I actually intended :) ~= is defined as "compatible" release (at least that's the name PIP definition uses) - and I think this makes sense to always increment patchlevel automatically but then make a deliberate decisions about major level increases (looking at PIP history this is a reasonable) - even if it means that only new features are added.
So in short `>=21.0.1, ==21.0.*` is exactly what I wanted to achieve.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #14368: Use latest Pip version when building image.
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #14368:
URL: https://github.com/apache/airflow/pull/14368#discussion_r580895024
##########
File path: Dockerfile
##########
@@ -190,7 +190,7 @@ ENV AIRFLOW_INSTALL_USER_FLAG="--user"
ENV AIRFLOW_INSTALL_EDITABLE_FLAG=""
# Upgrade to specific PIP version
-RUN pip install --upgrade "pip==${AIRFLOW_PIP_VERSION}"
+RUN pip install --upgrade "pip~=${AIRFLOW_PIP_VERSION}"
Review comment:
https://www.python.org/dev/peps/pep-0440/#compatible-release
`pip~=21.0.1` is equivalent to `piip>=21.0.1, ==21.0.*`
Which from the PR description is not what I think you intended.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #14368: Use latest Pip version when building image.
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #14368:
URL: https://github.com/apache/airflow/pull/14368#discussion_r580899334
##########
File path: Dockerfile
##########
@@ -190,7 +190,7 @@ ENV AIRFLOW_INSTALL_USER_FLAG="--user"
ENV AIRFLOW_INSTALL_EDITABLE_FLAG=""
# Upgrade to specific PIP version
-RUN pip install --upgrade "pip==${AIRFLOW_PIP_VERSION}"
+RUN pip install --upgrade "pip~=${AIRFLOW_PIP_VERSION}"
Review comment:
Cool. I thought you wanted 21.* when you said "21* series release". LGTM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #14368: Use latest Pip version when building image.
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #14368:
URL: https://github.com/apache/airflow/pull/14368#issuecomment-815179389
But i will do it when they release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #14368: Use latest Pip version when building image.
Posted by GitBox <gi...@apache.org>.
uranusjr commented on pull request #14368:
URL: https://github.com/apache/airflow/pull/14368#issuecomment-815181036
Thanks for the context.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed pull request #14368: Use latest Pip version when building image.
Posted by GitBox <gi...@apache.org>.
potiuk closed pull request #14368:
URL: https://github.com/apache/airflow/pull/14368
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #14368: Use latest Pip version when building image.
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #14368:
URL: https://github.com/apache/airflow/pull/14368#issuecomment-784047657
Just as a comment - still we will not merge it yet - PIP 21.* still has some problems with installing providers (it again downloads multiple version of some packages and it takes > 30 minutes) and I need to take a look :). It's likely something that we can improve in our scripts, but it also means that installing using constraints is very important as PIP resolver still does not handle everything as smoothly as we would like to (but maybe this has something to do with conflicting dependencies in some cases)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org