You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Csaba Ringhofer (Jira)" <ji...@apache.org> on 2019/08/22 16:26:00 UTC
[jira] [Commented] (IMPALA-8875)
TestHmsIntegration.test_drop_column_maintains_stats seems flaky
[ https://issues.apache.org/jira/browse/IMPALA-8875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913471#comment-16913471 ]
Csaba Ringhofer commented on IMPALA-8875:
-----------------------------------------
I looked at the test failures and the issue seems to be that Hive 3 HMS adds 'COLUMN_STATS_ACCURATE': '{}' to the column stats objects during ALTER TABLE DROP COLUMN. This can be easily reproduced and I think that it is not flaky. One way to solve this is to ignore COLUMN_STATS_ACCURATE when comparing stats, the other is to add this 'COLUMN_STATS_ACCURATE': '{}' in Impala too.
> TestHmsIntegration.test_drop_column_maintains_stats seems flaky
> ---------------------------------------------------------------
>
> Key: IMPALA-8875
> URL: https://issues.apache.org/jira/browse/IMPALA-8875
> Project: IMPALA
> Issue Type: Bug
> Reporter: Fang-Yu Rao
> Assignee: Gabor Kaszab
> Priority: Blocker
> Labels: broken-build
>
> The test of TestHmsIntegration.test_drop_column_maintains_stats seems flaky. The related test file was updated recently due to https://issues.apache.org/jira/browse/IMPALA-8823. Create this JIRA to track this failed test. Maybe [~gaborkaszab] you could take a brief look at this? Thanks!
> The error messages are provided in the following.
> {code:java}
> Error Message
> assert {'avg_col_len...ializer', ...} == {'COLUMN_STATS...me': 'x', ...} Common items: {'avg_col_len': '', 'bitVector': '', 'col_name': 'x', 'comment': 'from deserializer', 'data_type': 'int', 'distinct_count': '0', 'max': '0', 'max_col_len': '', 'min': '0', 'num_falses': '', 'num_nulls': '0', 'num_trues': ''} Right contains more items: {'COLUMN_STATS_ACCURATE': '{}'} Full diff: + {'COLUMN_STATS_ACCURATE': '{}', - {'avg_col_len': '', ? ^ + 'avg_col_len': '', ? ^ 'bitVector': '', 'col_name': 'x', 'comment': 'from deserializer', 'data_type': 'int', 'distinct_count': '0', 'max': '0', 'max_col_len': '', 'min': '0', 'num_falses': '', 'num_nulls': '0', 'num_trues': ''}
> {code}
> The stack trace is given as follows.
> {code:java}
> Stacktrace
> metadata/test_hms_integration.py:390: in test_drop_column_maintains_stats
> assert hive_x_stats == self.hive_column_stats(table_name, 'x')
> E assert {'avg_col_len...ializer', ...} == {'COLUMN_STATS...me': 'x', ...}
> E Common items:
> E {'avg_col_len': '',
> E 'bitVector': '',
> E 'col_name': 'x',
> E 'comment': 'from deserializer',
> E 'data_type': 'int',
> E 'distinct_count': '0',
> E 'max': '0',
> E 'max_col_len': '',
> E 'min': '0',
> E 'num_falses': '',
> E 'num_nulls': '0',
> E 'num_trues': ''}
> E Right contains more items:
> E {'COLUMN_STATS_ACCURATE': '{}'}
> E Full diff:
> E + {'COLUMN_STATS_ACCURATE': '{}',
> E - {'avg_col_len': '',
> E ? ^
> E + 'avg_col_len': '',
> E ? ^
> E 'bitVector': '',
> E 'col_name': 'x',
> E 'comment': 'from deserializer',
> E 'data_type': 'int',
> E 'distinct_count': '0',
> E 'max': '0',
> E 'max_col_len': '',
> E 'min': '0',
> E 'num_falses': '',
> E 'num_nulls': '0',
> E 'num_trues': ''}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org