You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/20 10:59:14 UTC
[GitHub] [airflow] ashb opened a new pull request #20979: Remove un-needed deps/version requirements
ashb opened a new pull request #20979:
URL: https://github.com/apache/airflow/pull/20979
This is in an aim to reduce the search space pip will hve to look at
when backtracking
- iso8601 hasn't been in use since 2017(!)
- pyjwt has been fixed upstream (flask-jwt-extended, via FAB), so we
don't need to require it ourselfs # Please enter the commit message
for your changes.
- wtforms and python3-openid issues have been fixed in FAB 3.4.0, so lets just update to that
version
When looking at another dep backtrack issue I noticed a couple of restrictions that we are placing that we don't need (and one dep we install that just isn't need at all)
I don't know if this will help or hurt backtracking problems, but removing deps/pins we don't need feels like a good thing overall.
I haven't tested these changes (just tested that it installed without build errors.)
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #20979: Remove un-needed deps/version requirements
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #20979:
URL: https://github.com/apache/airflow/pull/20979#issuecomment-1017429326
Very nice. What we really need to remove one more limit. We have another limit in Dockerfile.ci:
```
# * lazy-object-proxy<1.5.0: required by astroid
lazy-object-proxy<1.5.0
```
This can likely be removed as well (we will find out when we change it. And it might be the root cause of the issue.
It's been introduced to account for `pip` behaviour with --eager-upgrade. When eager-upgrade is run it will first upgrade "airflow" dependencies without any extras and only then will apply extras which caused similar backtracking because astroid had < 1.5.0 limitation and by the time it was added by extras we already had higher version.
But this limitation is gone and astroid we use now has:
```
lazy_object_proxy>=1.4.0
```
So we can safely remove it too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #20979: Remove un-needed deps/version requirements
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #20979:
URL: https://github.com/apache/airflow/pull/20979#issuecomment-1017437154
> Ah cool. Pushed that fix, lets see if it builds
:crossed_fingers: -> if it does (and test pass), then it means we can safely merge it. This change uses "eager upgrade" and has "full tests needed" so we will find out :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #20979: Remove un-needed deps/version requirements
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #20979:
URL: https://github.com/apache/airflow/pull/20979#issuecomment-1017429326
Very nice. What we really need to remove one more limit. We have another limit in Dockerfile.ci:
```
# * lazy-object-proxy<1.5.0: required by astroid
lazy-object-proxy<1.5.0
```
This can likely be removed as well (we will find out when we change it. And it might be the root cause of the issue.
It's been introduced to account for `pip` behaviour with --eager-upgrade (when eager-upgrade is run it will first upgrade "airflow" dependencies without any extras and only then will apply extras which caused similar backtracking because astroid had < 1.5.0 limitation and by the time it was added by extras we already had higher version.
But this limitation is gone and astroid we use now has:
```
lazy_object_proxy>=1.4.0
```
So we can safely remove it too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #20979: Remove un-needed deps/version requirements
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #20979:
URL: https://github.com/apache/airflow/pull/20979#issuecomment-1017429326
Very nice. What we really need to remove one more limit. We have another limit in Dockerfile.ci:
```
# * lazy-object-proxy<1.5.0: required by astroid
lazy-object-proxy<1.5.0
```
This can likely be removed as well (we will find out when we change it. And it might be the root cause of the issue.
It's been introduced to account for `pip` behaviour with --eager-upgrade. When eager-upgrade is run it will first upgrade "airflow" dependencies without any extras and only then will apply extras.
This caused similar backtracking because astroid had < 1.5.0 limitation and by the time it was added by extras we already had higher version.
But this limitation is gone and astroid we use now has:
```
lazy_object_proxy>=1.4.0
```
So we can safely remove it too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #20979: Remove un-needed deps/version requirements
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #20979:
URL: https://github.com/apache/airflow/pull/20979#discussion_r788768663
##########
File path: setup.cfg
##########
@@ -127,18 +120,12 @@ install_requires =
pluggy~=1.0
psutil>=4.2.0, <6.0.0
pygments>=2.0.1, <3.0
- pyjwt<3
Review comment:
>pyjwt has been fixed upstream (flask-jwt-extended, via FAB), so we
don't need to require it ourselfs # Please enter the commit message
for your changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb merged pull request #20979: Remove un-needed deps/version requirements
Posted by GitBox <gi...@apache.org>.
ashb merged pull request #20979:
URL: https://github.com/apache/airflow/pull/20979
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #20979: Remove un-needed deps/version requirements
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #20979:
URL: https://github.com/apache/airflow/pull/20979#issuecomment-1017429326
Very nice. What we really need to remove one more limit. We have another limit in Dockerfile.ci:
```
# * lazy-object-proxy<1.5.0: required by astroid
lazy-object-proxy<1.5.0
```
This can likely be removed as wel l(we will find out when we change it. And it might be the root cause of the issue.
It's been introduced to account for `pip` behaviour with --eager-upgrade (when eager-upgrade is run it will first upgrade "airflow" dependencies without any extras and only then will apply extras which caused similar backtracking because astroid had < 1.5.0 limitation and by the time it was added by extras we already had higher version.
But this limitation is gone and astroid we use now has:
```
lazy_object_proxy>=1.4.0
```
So we can safely remove it too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #20979: Remove un-needed deps/version requirements
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #20979:
URL: https://github.com/apache/airflow/pull/20979#issuecomment-1017572454
One flaky failure:
```
tests/executors/test_dask_executor.py::TestDaskExecutorQueue::test_dask_queues: ValueError: The futures should have finished; there is probably an error communicating with the Dask cluster.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #20979: Remove un-needed deps/version requirements
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #20979:
URL: https://github.com/apache/airflow/pull/20979#issuecomment-1017434762
Ah cool. Pushed that fix, lets see if it builds
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #20979: Remove un-needed deps/version requirements
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #20979:
URL: https://github.com/apache/airflow/pull/20979#discussion_r788768442
##########
File path: setup.cfg
##########
@@ -127,18 +120,12 @@ install_requires =
pluggy~=1.0
psutil>=4.2.0, <6.0.0
pygments>=2.0.1, <3.0
- pyjwt<3
Review comment:
What required pyjwt3 before?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org