You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/03/01 12:39:34 UTC
[GitHub] [airflow] potiuk opened a new pull request #21893: Add Python 3.9 support to Hive
potiuk opened a new pull request #21893:
URL: https://github.com/apache/airflow/pull/21893
Hive support for Python 3.9 has been removed in #15515 but cloudera
released new ssl 0.3.1 library version to support it and we should
be able to get Hive provider working for Python 3.9 too.
Fixes: #21891
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21893:
URL: https://github.com/apache/airflow/pull/21893#issuecomment-1055912947
Just rundom "maybe ? " test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk closed pull request #21893:
URL: https://github.com/apache/airflow/pull/21893
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21893:
URL: https://github.com/apache/airflow/pull/21893#issuecomment-1055416540
Closed/Reopened to ACTUALLY run Python 3.9 tests (with `full tests needed` label :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #21893:
URL: https://github.com/apache/airflow/pull/21893
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21893:
URL: https://github.com/apache/airflow/pull/21893#issuecomment-1055422838
And :facepalm: I also removed the skipIfs to actually RUN the tests on Python 3.9 :D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21893:
URL: https://github.com/apache/airflow/pull/21893#issuecomment-1055430468
> Ah, so the 0.3 series of sasl supports 3.9, solving our issue.
>
> Perhaps it’d be worthwhile to add some additional specifiers to describe this? e.g.
>
> ```
> # Some comment explaining 3.0 added 3.9 support...
> sasl; python_version<"3.9"
> sasl>=3.0; python_version>="3.9"
> ```
Good idea. I prefer not to complicate our setup if not needed sasl is a very specific library and pragmatically speaking anyone installing hive provider on Python 3.9 wll get latest (0.3.1) version installed. But yeah that might be a good idea as there **might** be some cases where someone has an older version of sasl library installed.
Adding it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21893:
URL: https://github.com/apache/airflow/pull/21893#issuecomment-1055414430
> I see [dropbox/PyHive#380](https://github.com/dropbox/PyHive/issues/380) is still open. What changed to make this viable?
Yeah, I thought the same initially and commented the same - but I looked closer - see #21891 discussion. Rather than switching to another implementation (pure-sasl based) cloudera released the `sasl` library with 3.9 support - this was the `5 years old` library I mentioned in the original PR https://pypi.org/project/sasl/#history
I am going to close https://github.com/dropbox/PyHive/issues/380 as soon as we merge this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #21893:
URL: https://github.com/apache/airflow/pull/21893#issuecomment-1055433986
Added it now @uranusjr ! Thanks for pointing it out ! I actually made it >=3.0.1 as apparently 3.0.1 was released a day after 3.0.0 (after 5 years) and this might only mean one thing even if 3.0.0 is not yanked :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #21893:
URL: https://github.com/apache/airflow/pull/21893#issuecomment-1055433986
Added it now @uranusjr ! Thanks for pointing it out ! I actually made it >=3.0.1 as apparently 3.0.1 was released a day after 3.0.0 and this might only mean one thing even if it is not yanked :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21893:
URL: https://github.com/apache/airflow/pull/21893#issuecomment-1055433986
Added it now @uranusjr ! Thanks for pointing it out ! I actually made it >=3.0.1 as apparently 3.0.1 was released a day after 3.0.0 and this might only mean one thing :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #21893:
URL: https://github.com/apache/airflow/pull/21893#issuecomment-1055414430
> I see [dropbox/PyHive#380](https://github.com/dropbox/PyHive/issues/380) is still open. What changed to make this viable?
Yeah, I thought the same initially and commented the same - but I looked closer - see #21891 discussion. Rather than PyHive switching to another implementation (pure-sasl based) cloudera released the `sasl` library with 3.9 support - this was the `5 years old` library I mentioned in the original PR https://pypi.org/project/sasl/#history
I am going to close https://github.com/dropbox/PyHive/issues/380 as soon as we merge this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #21893:
URL: https://github.com/apache/airflow/pull/21893#issuecomment-1055433986
Added it now @uranusjr ! Thanks for pointing it out ! I actually made it >=3.0.1 as apparently ~3.0.1~ 0.3.1 was released a day after ~3.0.0~ 0.3.0 (after 5 years) and this might only mean one thing even if ~3.0.0~ 0.3.0 is not yanked :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk closed pull request #21893:
URL: https://github.com/apache/airflow/pull/21893
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21893:
URL: https://github.com/apache/airflow/pull/21893#issuecomment-1055455412
:facepalm: :facepalm: :facepalm: 0.3.1 not 3.0.1 @potiuk. Try to copy&paste next time :D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
uranusjr commented on pull request #21893:
URL: https://github.com/apache/airflow/pull/21893#issuecomment-1055406718
I see dropbox/PyHive#380 is still open. What changed to make this viable?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
uranusjr commented on pull request #21893:
URL: https://github.com/apache/airflow/pull/21893#issuecomment-1055423741
Ah, so the 0.3 series of sasl supports 3.9, solving our issue.
Perhaps it’d be worthwhile to add some additional specifiers to describe this? e.g.
```
# Some comment explaining 3.0 added 3.9 support...
sasl; python_version<"3.9"
sasl>=3.0; python_version>="3.9"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #21893:
URL: https://github.com/apache/airflow/pull/21893#issuecomment-1055433986
Added it now @uranusjr ! Thanks for pointing it out ! I actually made it >=3.0.1 as apparently 3.0.1 was released a day after 3.0.0 and this might only mean one thing even if 3.0.0 is not yanked :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21893: Add Python 3.9 support to Hive
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21893:
URL: https://github.com/apache/airflow/pull/21893#issuecomment-1055402287
I would love this one merged before I release a new wave of providers for Feb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org