You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "bkyryliuk (via GitHub)" <gi...@apache.org> on 2023/11/13 20:02:47 UTC
[PR] Donate pyhive to apache kyuubi [kyuubi]
bkyryliuk opened a new pull request, #5686:
URL: https://github.com/apache/kyuubi/pull/5686
This pr: moves https://github.com/dropbox/PyHive to the pyhive folder in apache kyuubi project at [d4ae481675ac5588ba9101596fa26f22ef0e77c4](https://github.com/dropbox/PyHive/commit/d4ae481675ac5588ba9101596fa26f22ef0e77c4) commit
and changes the license text to the standard Apache License Version 2.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Donate PyHive to Apache Kyuubi [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #5686:
URL: https://github.com/apache/kyuubi/pull/5686#issuecomment-2026849179
SGA has been received, and I have sent the IP CLEARANCE to the Incubator mailing list - https://lists.apache.org/thread/8nlbpo2ckwshsn9rvnstmzpgns2tn4fl
We can start the code merging process now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Donate PyHive to Apache Kyuubi [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #5686:
URL: https://github.com/apache/kyuubi/pull/5686#discussion_r1544223521
##########
pyhive/setup.py:
##########
@@ -0,0 +1,78 @@
+#!/usr/bin/env python
+
+from setuptools import setup
+from setuptools.command.test import test as TestCommand
+import pyhive
+import sys
+
+
+class PyTest(TestCommand):
+ def finalize_options(self):
+ TestCommand.finalize_options(self)
+ self.test_args = []
+ self.test_suite = True
+
+ def run_tests(self):
+ # import here, cause outside the eggs aren't loaded
+ import pytest
+ errno = pytest.main(self.test_args)
+ sys.exit(errno)
+
+
+with open('README.rst') as readme:
+ long_description = readme.read()
+
+setup(
+ name="PyHive",
+ version=pyhive.__version__,
+ description="Python interface to Hive",
+ long_description=long_description,
+ url='https://github.com/dropbox/PyHive',
+ author="Jing Wang",
+ author_email="jing@dropbox.com",
Review Comment:
```suggestion
url='https://github.com/apache/kyuubi/python',
author="Apache Kyuubi Developers",
author_email="dev@kyuubi.apache.org",
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Donate PyHive to Apache Kyuubi [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 closed pull request #5686: Donate PyHive to Apache Kyuubi
URL: https://github.com/apache/kyuubi/pull/5686
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Donate PyHive to Apache Kyuubi [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #5686:
URL: https://github.com/apache/kyuubi/pull/5686#discussion_r1544221003
##########
pyhive/README.rst:
##########
@@ -0,0 +1,212 @@
+================================
+Project is currently unsupported
+================================
+
+
+
+
+.. image:: https://travis-ci.org/dropbox/PyHive.svg?branch=master
+ :target: https://travis-ci.org/dropbox/PyHive
+.. image:: https://img.shields.io/codecov/c/github/dropbox/PyHive.svg
+
Review Comment:
Please remove this part
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Donate pyhive to apache kyuubi [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #5686:
URL: https://github.com/apache/kyuubi/pull/5686#issuecomment-1809488777
Many thanks, @bkyryliuk. As we discussed, let's hold on a while to wait for SGA to get approved by ASF.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Donate PyHive to Apache Kyuubi [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #5686:
URL: https://github.com/apache/kyuubi/pull/5686#issuecomment-2033457754
kindly ping @bkyryliuk, appreciate it if you could take minutes to tune this commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Donate PyHive to Apache Kyuubi [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #5686:
URL: https://github.com/apache/kyuubi/pull/5686#issuecomment-2026876821
@bkyryliuk would you mind making the following changes? Then we can merge this patch.
1. rename the top-level dir `pyhive` to `python`
2. add `python/**` into `.rat-exlucdes` to temporarily skip the license header check, we can fix it later.
3. as we discussed in the mailing list before, you may want to update `NOTICE` to add such info
```patch
Apache Kyuubi
Copyright 2021-2024 The Apache Software Foundation.
This product includes software developed at
The Apache Software Foundation (https://www.apache.org/).
+ (This is an example, please tune the words by yourself)
+ Portions of this software were developed at Dropbox, Inc.
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Donate PyHive to Apache Kyuubi [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #5686:
URL: https://github.com/apache/kyuubi/pull/5686#issuecomment-2066298924
@bkyryliuk thanks for the reply, you may want to update the NOTICE as I mentioned [here](https://github.com/apache/kyuubi/pull/5686#issuecomment-2026876821). Also would be good to update the PyHive README to mention the donation.
Now, we are working on setup the CI https://github.com/apache/kyuubi/issues/6281, and the next step is to finish the first PyPi release.
We are likely to create a new project(I'm going to open a discussion on the mailing list) instead of an existing PyHive. So the following steps are optional
- grant PyPi PyHive publish permission to `apachekyuubi`(`chengpan@apache.org`)
- grant PyHive repo permission to @pan3793(me) and @yaooqinn(Kent Yao, Kyuubi PMC chair)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Donate PyHive to Apache Kyuubi [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #5686:
URL: https://github.com/apache/kyuubi/pull/5686#discussion_r1544221807
##########
pyhive/README.rst:
##########
@@ -0,0 +1,212 @@
+================================
+Project is currently unsupported
+================================
+
+
+
+
+.. image:: https://travis-ci.org/dropbox/PyHive.svg?branch=master
+ :target: https://travis-ci.org/dropbox/PyHive
+.. image:: https://img.shields.io/codecov/c/github/dropbox/PyHive.svg
+
+======
+PyHive
+======
+
+PyHive is a collection of Python `DB-API <http://www.python.org/dev/peps/pep-0249/>`_ and
+`SQLAlchemy <http://www.sqlalchemy.org/>`_ interfaces for `Presto <http://prestodb.io/>`_ ,
+`Hive <http://hive.apache.org/>`_ and `Trino <https://trino.io/>`_.
+
+Usage
+=====
+
+DB-API
+------
+.. code-block:: python
+
+ from pyhive import presto # or import hive or import trino
+ cursor = presto.connect('localhost').cursor() # or use hive.connect or use trino.connect
+ cursor.execute('SELECT * FROM my_awesome_data LIMIT 10')
+ print cursor.fetchone()
+ print cursor.fetchall()
+
+DB-API (asynchronous)
+---------------------
+.. code-block:: python
+
+ from pyhive import hive
+ from TCLIService.ttypes import TOperationState
+ cursor = hive.connect('localhost').cursor()
+ cursor.execute('SELECT * FROM my_awesome_data LIMIT 10', async=True)
+
+ status = cursor.poll().operationState
+ while status in (TOperationState.INITIALIZED_STATE, TOperationState.RUNNING_STATE):
+ logs = cursor.fetch_logs()
+ for message in logs:
+ print message
+
+ # If needed, an asynchronous query can be cancelled at any time with:
+ # cursor.cancel()
+
+ status = cursor.poll().operationState
+
+ print cursor.fetchall()
+
+In Python 3.7 `async` became a keyword; you can use `async_` instead:
+
+.. code-block:: python
+
+ cursor.execute('SELECT * FROM my_awesome_data LIMIT 10', async_=True)
+
+
+SQLAlchemy
+----------
+First install this package to register it with SQLAlchemy, see ``entry_points`` in ``setup.py``.
+
+.. code-block:: python
+
+ from sqlalchemy import *
+ from sqlalchemy.engine import create_engine
+ from sqlalchemy.schema import *
+ # Presto
+ engine = create_engine('presto://localhost:8080/hive/default')
+ # Trino
+ engine = create_engine('trino+pyhive://localhost:8080/hive/default')
+ # Hive
+ engine = create_engine('hive://localhost:10000/default')
+
+ # SQLAlchemy < 2.0
+ logs = Table('my_awesome_data', MetaData(bind=engine), autoload=True)
+ print select([func.count('*')], from_obj=logs).scalar()
+
+ # Hive + HTTPS + LDAP or basic Auth
+ engine = create_engine('hive+https://username:password@localhost:10000/')
+ logs = Table('my_awesome_data', MetaData(bind=engine), autoload=True)
+ print select([func.count('*')], from_obj=logs).scalar()
+
+ # SQLAlchemy >= 2.0
+ metadata_obj = MetaData()
+ books = Table("books", metadata_obj, Column("id", Integer), Column("title", String), Column("primary_author", String))
+ metadata_obj.create_all(engine)
+ inspector = inspect(engine)
+ inspector.get_columns('books')
+
+ with engine.connect() as con:
+ data = [{ "id": 1, "title": "The Hobbit", "primary_author": "Tolkien" },
+ { "id": 2, "title": "The Silmarillion", "primary_author": "Tolkien" }]
+ con.execute(books.insert(), data[0])
+ result = con.execute(text("select * from books"))
+ print(result.fetchall())
+
+Note: query generation functionality is not exhaustive or fully tested, but there should be no
+problem with raw SQL.
+
+Passing session configuration
+-----------------------------
+
+.. code-block:: python
+
+ # DB-API
+ hive.connect('localhost', configuration={'hive.exec.reducers.max': '123'})
+ presto.connect('localhost', session_props={'query_max_run_time': '1234m'})
+ trino.connect('localhost', session_props={'query_max_run_time': '1234m'})
+ # SQLAlchemy
+ create_engine(
+ 'presto://user@host:443/hive',
+ connect_args={'protocol': 'https',
+ 'session_props': {'query_max_run_time': '1234m'}}
+ )
+ create_engine(
+ 'trino+pyhive://user@host:443/hive',
+ connect_args={'protocol': 'https',
+ 'session_props': {'query_max_run_time': '1234m'}}
+ )
+ create_engine(
+ 'hive://user@host:10000/database',
+ connect_args={'configuration': {'hive.exec.reducers.max': '123'}},
+ )
+ # SQLAlchemy with LDAP
+ create_engine(
+ 'hive://user:password@host:10000/database',
+ connect_args={'auth': 'LDAP'},
+ )
+
+Requirements
+============
+
+Install using
+
+- ``pip install 'pyhive[hive]'`` or ``pip install 'pyhive[hive_pure_sasl]'`` for the Hive interface
+- ``pip install 'pyhive[presto]'`` for the Presto interface
+- ``pip install 'pyhive[trino]'`` for the Trino interface
+
+Note: ``'pyhive[hive]'`` extras uses `sasl <https://pypi.org/project/sasl/>`_ that doesn't support Python 3.11, See `github issue <https://github.com/cloudera/python-sasl/issues/30>`_.
+Hence PyHive also supports `pure-sasl <https://pypi.org/project/pure-sasl/>`_ via additional extras ``'pyhive[hive_pure_sasl]'`` which support Python 3.11.
+
+PyHive works with
+
+- Python 2.7 / Python 3
+- For Presto: `Presto installation <https://prestodb.io/docs/current/installation.html>`_
+- For Trino: `Trino installation <https://trino.io/docs/current/installation.html>`_
+- For Hive: `HiveServer2 <https://cwiki.apache.org/confluence/display/Hive/Setting+up+HiveServer2>`_ daemon
+
+Changelog
+=========
+See https://github.com/dropbox/PyHive/releases.
+
+Contributing
+============
+- Please fill out the Dropbox Contributor License Agreement at https://opensource.dropbox.com/cla/ and note this in your pull request.
Review Comment:
Please remove this line
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Donate PyHive to Apache Kyuubi [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #5686:
URL: https://github.com/apache/kyuubi/pull/5686#issuecomment-2044770430
Merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Donate pyhive to apache kyuubi [kyuubi]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #5686:
URL: https://github.com/apache/kyuubi/pull/5686#issuecomment-1960551902
Thanks for the PR! This PR is being closed due to inactivity. This isn't a judgement on the merit of the PR in any way. If this is still an issue with the latest version of Kyuubi, please reopen it and ask a committer to remove the Stale tag!
Thank you for using Kyuubi!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Donate pyhive to apache kyuubi [kyuubi]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #5686: Donate pyhive to apache kyuubi
URL: https://github.com/apache/kyuubi/pull/5686
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Donate PyHive to Apache Kyuubi [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #5686:
URL: https://github.com/apache/kyuubi/pull/5686#issuecomment-2044605605
To unblock the ongoing development, I'm going to merge this PR as-is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Donate PyHive to Apache Kyuubi [kyuubi]
Posted by "bkyryliuk (via GitHub)" <gi...@apache.org>.
bkyryliuk commented on PR #5686:
URL: https://github.com/apache/kyuubi/pull/5686#issuecomment-2064310751
@pan3793 apologies for the delay, I had no free time until recently. Is there anything else needed or are you unblocked ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Donate pyhive to apache kyuubi [kyuubi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #5686:
URL: https://github.com/apache/kyuubi/pull/5686#issuecomment-1960629842
## [Codecov](https://app.codecov.io/gh/apache/kyuubi/pull/5686?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 61.40%. Comparing base [(`b783233`)](https://app.codecov.io/gh/apache/kyuubi/commit/b7832330b7a6a23d9eb9cacd070a14e837fa76ad?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`569d74e`)](https://app.codecov.io/gh/apache/kyuubi/pull/5686?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 216 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #5686 +/- ##
============================================
- Coverage 61.49% 61.40% -0.09%
Complexity 23 23
============================================
Files 604 604
Lines 35689 35689
Branches 4884 4884
============================================
- Hits 21948 21916 -32
- Misses 11375 11392 +17
- Partials 2366 2381 +15
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/kyuubi/pull/5686?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org