You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/08/12 17:25:17 UTC
[GitHub] [airflow] potiuk opened a new pull request, #25697: Help pip resolved to make better decisions on Pyarrow version
potiuk opened a new pull request, #25697:
URL: https://github.com/apache/airflow/pull/25697
The pip resolver in `eager upgrade` mode on Python 3.10 decides
to downgrade Pyarrow to 5.0.0 which triggers numpy not being
compatible with 3.10 - even if Pyarrow 6.* is perfectly fine.
Adding the limit to eager upgrade directly helps the resolver to
make better decisions on it.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] susodapop commented on pull request #25697: Help pip resolver to make better decisions on Pyarrow version
Posted by GitBox <gi...@apache.org>.
susodapop commented on PR #25697:
URL: https://github.com/apache/airflow/pull/25697#issuecomment-1218403608
Hey team,`databricks-sql-connector` maintainer here: we just pushed `v2.0.4` with Python 3.10 support. LMK if this fixes things for you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25697: Help pip resolver to make better decisions on Pyarrow version
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25697:
URL: https://github.com/apache/airflow/pull/25697#issuecomment-1218449911
Yep. looks good @alexott @susodapop :) . All the images got built so it looks like it is going to work :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25697: Help pip resolver to make better decisions on Pyarrow version
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25697:
URL: https://github.com/apache/airflow/pull/25697#issuecomment-1219345396
Unfortunately it came back: https://github.com/apache/airflow/pull/25789 so that was not the reason (or maybe not the only reason @uranusjr).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25697: Help pip resolver to make better decisions on Pyarrow version
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25697:
URL: https://github.com/apache/airflow/pull/25697#issuecomment-1213373821
In case we decided to do 2.3.4 - we will have to cherry-pick that one - marked it as 2.3.4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #25697: Help pip resolver to make better decisions on Pyarrow version
Posted by GitBox <gi...@apache.org>.
uranusjr commented on PR #25697:
URL: https://github.com/apache/airflow/pull/25697#issuecomment-1218126453
A collegue in Astronomer just identified the root cause to this. [databricks-sql-python](https://github.com/databricks/databricks-sql-python), a dependency of the databricks provider, does not support 3.10. It also does not support pyarrow>=6, so pip is correct to report a resolution is impossible, although that conflict is the less important issue of the two and thus sort of misleading. But in any case, this means the latest constraint files will _probably_ _maybe_ have issues with the databricks extra, but I’m willing to bet on there’s nothing noticable unless someone complains on the issue tracker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25697: Help pip resolver to make better decisions on Pyarrow version
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25697:
URL: https://github.com/apache/airflow/pull/25697#issuecomment-1219346176
(but it does not actually mean the 2.0.4 upgrade was bad :) @susodapop @alexott :)).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25697: Help pip resolver to make better decisions on Pyarrow version
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25697:
URL: https://github.com/apache/airflow/pull/25697#issuecomment-1218216483
> A colleague in Astronomer just identified the root cause to this. [databricks-sql-python](https://github.com/databricks/databricks-sql-python), a dependency of the databricks provider, does not support 3.10. It also does not support pyarrow>=6, so pip is correct to report a resolution is impossible, although that conflict is the less important issue of the two and thus sort of misleading. But in any case, this means the latest constraint files will _probably_ _maybe_ have issues with the databricks extra, but I’m willing to bet on there’s nothing noticable unless someone complains on the issue tracker.
That must be it, indeed. https://pypi.org/project/databricks-sql-connector/2.0.3/ was released 7 days ago so that adds-up. @alexott - maybe you could ping the right people at Databricks and help to make the connector 3.10 compliant ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25697: Help pip resolver to make better decisions on Pyarrow version
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25697:
URL: https://github.com/apache/airflow/pull/25697#issuecomment-1218430167
> Hey team,`databricks-sql-connector` maintainer here: we just pushed `v2.0.4` with Python 3.10 support. LMK if this fixes things for you.
Woa.. That was QUICK @susodapop (And @alexott). PR running here: https://github.com/apache/airflow/pull/25777 - if it succeeds we are good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #25697: Help pip resolver to make better decisions on Pyarrow version
Posted by GitBox <gi...@apache.org>.
potiuk merged PR #25697:
URL: https://github.com/apache/airflow/pull/25697
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] alexott commented on pull request #25697: Help pip resolver to make better decisions on Pyarrow version
Posted by GitBox <gi...@apache.org>.
alexott commented on PR #25697:
URL: https://github.com/apache/airflow/pull/25697#issuecomment-1218217564
Ok, let me point team to it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org