You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by ar...@apache.org on 2019/08/23 01:39:53 UTC
[impala] branch master updated (3787e47 -> 661d2d6)
This is an automated email from the ASF dual-hosted git repository.
arodoni pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git.
from 3787e47 IMPALA-8516: Fix the sha512sum check for the Maven download
new f6117c9 IMPALA-8875: fix test_drop_column_maintains_stats with Hive 3
new 661d2d6 Update version to 3.4.0-SNAPSHOT
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
bin/save-version.sh | 2 +-
tests/metadata/test_hms_integration.py | 19 +++++++++++++------
2 files changed, 14 insertions(+), 7 deletions(-)
[impala] 01/02: IMPALA-8875: fix test_drop_column_maintains_stats
with Hive 3
Posted by ar...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
arodoni pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit f6117c9580731b290d3566fef420f9c4a02273d9
Author: Csaba Ringhofer <cs...@cloudera.com>
AuthorDate: Thu Aug 22 20:35:22 2019 +0200
IMPALA-8875: fix test_drop_column_maintains_stats with Hive 3
Hive 3 adds 'COLUMN_STATS_ACCURATE': '{}' to column statistics
during ALTER TABLE which lead to breaking this test, as it
expected the stats to be completely equal. Removing this
property before comparision solves the issue.
Change-Id: Ic2937e7634eca01e10492733102c834237ab6d6a
Reviewed-on: http://gerrit.cloudera.org:8080/14123
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
tests/metadata/test_hms_integration.py | 19 +++++++++++++------
1 file changed, 13 insertions(+), 6 deletions(-)
diff --git a/tests/metadata/test_hms_integration.py b/tests/metadata/test_hms_integration.py
index 8df23cb..59de250 100644
--- a/tests/metadata/test_hms_integration.py
+++ b/tests/metadata/test_hms_integration.py
@@ -217,13 +217,17 @@ class TestHmsIntegration(ImpalaTestSuite):
result[line_elements[0]] = line_elements[1]
return result
- def hive_column_stats(self, table, column):
+ def hive_column_stats(self, table, column, remove_stats_accurate=False):
"""Returns a dictionary of stats for a column according to Hive."""
output = self.run_stmt_in_hive('describe formatted %s %s' % (table, column))
+ result = {}
if HIVE_MAJOR_VERSION == 2:
- return self.parse_hive2_describe_formatted_output(output)
+ result = self.parse_hive2_describe_formatted_output(output)
else:
- return self.parse_hive3_describe_formatted_output(output)
+ result = self.parse_hive3_describe_formatted_output(output)
+ if remove_stats_accurate:
+ result.pop('COLUMN_STATS_ACCURATE', None)
+ return result
def impala_columns(self, table_name):
"""
@@ -387,8 +391,10 @@ class TestHmsIntegration(ImpalaTestSuite):
hive_y_stats = self.hive_column_stats(table_name, 'y')
impala_stats = self.impala_all_column_stats(table_name)
self.client.execute('alter table %s drop column z' % table_name)
- assert hive_x_stats == self.hive_column_stats(table_name, 'x')
- assert hive_y_stats == self.hive_column_stats(table_name, 'y')
+ assert hive_x_stats == self.hive_column_stats(table_name, 'x',
+ remove_stats_accurate=True)
+ assert hive_y_stats == self.hive_column_stats(table_name, 'y',
+ remove_stats_accurate=True)
assert impala_stats['x'] == self.impala_all_column_stats(table_name)[
'x']
assert impala_stats['y'] == self.impala_all_column_stats(table_name)[
@@ -396,7 +402,8 @@ class TestHmsIntegration(ImpalaTestSuite):
self.run_stmt_in_hive(
'alter table %s replace columns (x int)' %
table_name)
- assert hive_x_stats == self.hive_column_stats(table_name, 'x')
+ assert hive_x_stats == self.hive_column_stats(table_name, 'x',
+ remove_stats_accurate=True)
assert impala_stats['x'] == self.impala_all_column_stats(table_name)[
'x']
[impala] 02/02: Update version to 3.4.0-SNAPSHOT
Posted by ar...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
arodoni pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 661d2d6bc73042adb7ece6be69661b4869def99e
Author: stiga-huang <hu...@gmail.com>
AuthorDate: Thu Aug 22 12:08:52 2019 -0700
Update version to 3.4.0-SNAPSHOT
Change-Id: I84e5b33dafe4715077cb2aeb5f1ecbb8fd227f27
Reviewed-on: http://gerrit.cloudera.org:8080/14125
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
bin/save-version.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/bin/save-version.sh b/bin/save-version.sh
index f170db3..7c02636 100755
--- a/bin/save-version.sh
+++ b/bin/save-version.sh
@@ -21,7 +21,7 @@
# Note: for internal (aka pre-release) versions, the version should have
# "-INTERNAL" appended. Parts of the code will look for this to distinguish
# between released and internal versions.
-VERSION=3.3.0-SNAPSHOT
+VERSION=3.4.0-SNAPSHOT
GIT_HASH=$(git rev-parse HEAD 2> /dev/null)
if [ -z $GIT_HASH ]
then