You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "NikhilUjjwal7 (via GitHub)" <gi...@apache.org> on 2026/07/29 04:13:13 UTC
[PR] [KYUUBI #7360] Fix flaky PyHive test connection [kyuubi]
NikhilUjjwal7 opened a new pull request, #7595:
URL: https://github.com/apache/kyuubi/pull/7595
### Why are the changes needed?
Fixes #7360.
`python/pyhive/tests/test_hive.py::TestHive::test_description` can fail intermittently while opening the Hive connection with `TTransportException: TSocket read 0 bytes`.
This patch makes the test connection helper retry transient `TTransportException`s before failing, which makes the PyHive integration tests more tolerant of short HiveServer2 startup or connection timing issues.
### How was this patch tested?
Added a unit test that simulates a transient `TTransportException` from `hive.connect` and verifies that `TestHive.connect()` retries and succeeds on the next attempt.
Tested with:
```bash
python -m pytest python/pyhive/tests/test_hive.py::TestHive::test_connect_retries_transient_transport_failure -q
Result:
1 passed in 1.07s
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7360] Fix flaky PyHive test connection [kyuubi]
Posted by "NikhilUjjwal7 (via GitHub)" <gi...@apache.org>.
NikhilUjjwal7 commented on code in PR #7595:
URL: https://github.com/apache/kyuubi/pull/7595#discussion_r3679726808
##########
python/pyhive/tests/test_hive.py:
##########
@@ -30,13 +30,37 @@
from pyhive.tests.dbapi_test_case import with_cursor
_HOST = 'localhost'
+_CONNECT_ATTEMPTS = 5
+_CONNECT_RETRY_DELAY_SECONDS = 2
class TestHive(unittest.TestCase, DBAPITestCase):
__test__ = True
def connect(self):
- return hive.connect(host=_HOST, port=10000, configuration={'mapred.job.tracker': 'local'})
+ for attempt in range(_CONNECT_ATTEMPTS):
+ try:
+ return hive.connect(
+ host=_HOST, port=10000, configuration={'mapred.job.tracker': 'local'})
+ except TTransportException:
Review Comment:
Thanks for the review. I narrowed the retry condition to only transient transport failures (`END_OF_FILE`, `TIMED_OUT`, or `TSocket read 0 bytes`) and added a negative test to verify non-transient failures like login validation errors are not retried.
Tested with:
```bash
python -m pytest \
python/pyhive/tests/test_hive.py::TestHive::test_connect_retries_transient_transport_failure \
python/pyhive/tests/test_hive.py::TestHive::test_connect_does_not_retry_non_transient_transport_failure \
-q
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7360] Fix flaky PyHive test connection [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #7595:
URL: https://github.com/apache/kyuubi/pull/7595#issuecomment-5205016344
thanks, merged to master/1.12.1/1.11.2/1.10.4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7360] Fix flaky PyHive test connection [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 closed pull request #7595: [KYUUBI #7360] Fix flaky PyHive test connection
URL: https://github.com/apache/kyuubi/pull/7595
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7360] Fix flaky PyHive test connection [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7595:
URL: https://github.com/apache/kyuubi/pull/7595#discussion_r3672976883
##########
python/pyhive/tests/test_hive.py:
##########
@@ -30,13 +30,37 @@
from pyhive.tests.dbapi_test_case import with_cursor
_HOST = 'localhost'
+_CONNECT_ATTEMPTS = 5
+_CONNECT_RETRY_DELAY_SECONDS = 2
class TestHive(unittest.TestCase, DBAPITestCase):
__test__ = True
def connect(self):
- return hive.connect(host=_HOST, port=10000, configuration={'mapred.job.tracker': 'local'})
+ for attempt in range(_CONNECT_ATTEMPTS):
+ try:
+ return hive.connect(
+ host=_HOST, port=10000, configuration={'mapred.job.tracker': 'local'})
+ except TTransportException:
Review Comment:
could you narrow the catch (e.g. to `TTransportException` with `END_OF_FILE`/`TIMED_OUT` types, or checking the message) to avoid slowing real failures? e.g., a wrong LDAP password raises `TTransportException: Error validating the login`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7360] Fix flaky PyHive test connection [kyuubi]
Posted by "NikhilUjjwal7 (via GitHub)" <gi...@apache.org>.
NikhilUjjwal7 commented on PR #7595:
URL: https://github.com/apache/kyuubi/pull/7595#issuecomment-5180493384
Thanks for the update. I synced the latest commit and verified the focused retry tests locally.
```bash
python -m pytest \
python/pyhive/tests/test_hive.py::TestHive::test_connect_retries_transient_transport_failure \
python/pyhive/tests/test_hive.py::TestHive::test_connect_retries_connection_refused \
python/pyhive/tests/test_hive.py::TestHive::test_connect_does_not_retry_non_transient_transport_failure \
-q
Result:
3 passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] [KYUUBI #7360] Fix flaky PyHive test connection [kyuubi]
Posted by "NikhilUjjwal7 (via GitHub)" <gi...@apache.org>.
NikhilUjjwal7 commented on PR #7595:
URL: https://github.com/apache/kyuubi/pull/7595#issuecomment-5112975694
The `triage` workflow failure appears to be unrelated to this PR. It fails while parsing `.github/labeler.yml` in `actions/labeler@v7`, while this PR only changes `python/pyhive/tests/test_hive.py`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org