You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by ta...@apache.org on 2019/06/13 06:37:19 UTC
[impala] 05/07: IMPALA-7154: Increase metastore client time out for
tests
This is an automated email from the ASF dual-hosted git repository.
tarmstrong pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit e3117aa6645fec89ef587d6e57b38b981080cb03
Author: Vihang Karajgaonkar <vi...@cloudera.com>
AuthorDate: Tue Jun 11 14:24:26 2019 -0700
IMPALA-7154: Increase metastore client time out for tests
Some of the e2e tests create tables and databases on S3. HMS operations
like dropDatabaseCascade can take significant time (3-5 min) in such
tests. If the HMS Client times out, it retries for a configurable number
of times. Unfortunately, HMS client is not smart enough to understand
which operations should be retried. Eg, a drop database operation which
is timedout on the client side, should not be retried since the server
thread may have succeeded before the next retry. This would cause the
client to receive a NoSuchObjectException which would look like a flaky
test failure on the Impala side.
This patch increases the metastore client.socket.timeout for
the tests so that such flaky test failures are reduced.
Note:
1. The timeout value of 10 min was chosen based on looking at logs for
one such failure where the HMS API took about 3.5 min. Once this patch
is deployed we should continue to see if such failures reoccur and
readjust the client timeout value accordingly.
Change-Id: I253fc00618b28d8d6e626e1b6bf9c5c8290006d7
Reviewed-on: http://gerrit.cloudera.org:8080/13586
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
fe/src/test/resources/hive-site.xml.py | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/fe/src/test/resources/hive-site.xml.py b/fe/src/test/resources/hive-site.xml.py
index e46adc3..d8acccd 100644
--- a/fe/src/test/resources/hive-site.xml.py
+++ b/fe/src/test/resources/hive-site.xml.py
@@ -31,8 +31,15 @@ CONFIG.update({
# IMPALA-781: Impala doesn't support the default LazyBinaryColumnarSerde for RCFile.
'hive.default.rcfile.serde': 'org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe',
- 'hive.metastore.client.connect.retry.delay': '0',
- 'hive.metastore.client.socket.timeout': '120',
+ # IMPALA-7154: Some of the HMS operations on S3 can take long time and any timeouts
+ # on the HMS Client side can result in flaky test failures
+ # Following configuration specifies the time in seconds between successive retry
+ # attempts by metastore client in case of failures. By default, metastore client
+ # retries once based on the config value of hive.metastore.failure.retries
+ 'hive.metastore.client.connect.retry.delay': '1',
+ # set the metastore client timeout to 10 min
+ 'hive.metastore.client.socket.timeout': '600',
+
'hive.metastore.uris': 'thrift://${INTERNAL_LISTEN_HOST}:9083',
'hive.metastore.warehouse.dir': '${WAREHOUSE_LOCATION_PREFIX}/test-warehouse',