You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by jo...@apache.org on 2022/11/09 06:22:40 UTC
[impala] branch master updated (7e934b1f4 -> 4bc86ac63)
This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
from 7e934b1f4 IMPALA-11702: add resource metrics for kudu scanner to profile
new 4e6692b02 IMPALA-11686: Fix test_corrupt_stat after IMPALA-11666
new 4bc86ac63 IMPALA-11714: Fix resolve_minidumps.py on Ubuntu 16
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
bin/resolve_minidumps.py | 2 +-
.../java/org/apache/impala/planner/Planner.java | 4 +--
.../queries/QueryTest/corrupt-stats.test | 36 +++++++++++++++++-----
3 files changed, 31 insertions(+), 11 deletions(-)
[impala] 02/02: IMPALA-11714: Fix resolve_minidumps.py on Ubuntu 16
Posted by jo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 4bc86ac638fb85deda317b9b1dfd1f574fa0553b
Author: Joe McDonnell <jo...@cloudera.com>
AuthorDate: Tue Nov 8 16:54:28 2022 -0800
IMPALA-11714: Fix resolve_minidumps.py on Ubuntu 16
resolve_minidumps.py uses /bin/env. Some distributions
support that, but that path does not exist on Ubuntu 16.
This switches to use /usr/bin/env, which is available
everywhere.
Change-Id: Iecde8713bc141f2a15babf3886848eacad8910aa
Reviewed-on: http://gerrit.cloudera.org:8080/19225
Reviewed-by: Michael Smith <mi...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
bin/resolve_minidumps.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/bin/resolve_minidumps.py b/bin/resolve_minidumps.py
index 0a0ba0c61..e3f00be70 100755
--- a/bin/resolve_minidumps.py
+++ b/bin/resolve_minidumps.py
@@ -1,4 +1,4 @@
-#!/bin/env python3
+#!/usr/bin/env python3
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
[impala] 01/02: IMPALA-11686: Fix test_corrupt_stat after IMPALA-11666
Posted by jo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 4e6692b024b3e53afa0feae94b0def58e0ac8655
Author: Fang-Yu Rao <fa...@cloudera.com>
AuthorDate: Tue Nov 8 12:18:37 2022 -0800
IMPALA-11686: Fix test_corrupt_stat after IMPALA-11666
IMPALA-11666 revised the message in the query plans when there are
potentially corrupt statistics, which broke test_corrupt_stat, an E2E
test only run in the exhaustive tests. This patch fixes the test file
accordingly.
Testing:
- Verified locally that the patch passes test_corrupt_stat.
Change-Id: I817c7807a07bb89b93d795bce958b9872eff2eef
Reviewed-on: http://gerrit.cloudera.org:8080/19224
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
.../java/org/apache/impala/planner/Planner.java | 4 +--
.../queries/QueryTest/corrupt-stats.test | 36 +++++++++++++++++-----
2 files changed, 30 insertions(+), 10 deletions(-)
diff --git a/fe/src/main/java/org/apache/impala/planner/Planner.java b/fe/src/main/java/org/apache/impala/planner/Planner.java
index d34201156..9c9056206 100644
--- a/fe/src/main/java/org/apache/impala/planner/Planner.java
+++ b/fe/src/main/java/org/apache/impala/planner/Planner.java
@@ -334,8 +334,8 @@ public class Planner {
"the partition(s) is positive.\n" +
"The latter case does not necessarily imply the existence of corrupt \n" +
"statistics when the corresponding tables are transactional.\n" +
- "If it is suspected that there may be corrupt statistics, drop and \n" +
- "re-compute statistics could resolve this problem.\n" +
+ "If it is suspected that there may be corrupt statistics, dropping and \n" +
+ "re-computing statistics could resolve this problem.\n" +
Joiner.on(", ").join(tableNames) + "\n");
hasHeader = true;
}
diff --git a/testdata/workloads/functional-query/queries/QueryTest/corrupt-stats.test b/testdata/workloads/functional-query/queries/QueryTest/corrupt-stats.test
index 6f2c69645..d365dfc71 100644
--- a/testdata/workloads/functional-query/queries/QueryTest/corrupt-stats.test
+++ b/testdata/workloads/functional-query/queries/QueryTest/corrupt-stats.test
@@ -53,8 +53,13 @@ STRING, BIGINT, BIGINT, STRING, STRING, STRING, STRING, STRING, STRING
---- QUERY
explain select count(*) from corrupted where org = 1;
---- RESULTS: VERIFY_IS_SUBSET
-'WARNING: The following tables have potentially corrupt table statistics.'
-'Drop and re-compute statistics to resolve this problem.'
+'The row count in one or more partitions in the following tables '
+'is either a) less than -1, or b) 0 but the size of all the files inside '
+'the partition(s) is positive.'
+'The latter case does not necessarily imply the existence of corrupt '
+'statistics when the corresponding tables are transactional.'
+'If it is suspected that there may be corrupt statistics, dropping and '
+'re-computing statistics could resolve this problem.'
'$DATABASE.corrupted'
''
'03:AGGREGATE [FINALIZE]'
@@ -132,8 +137,13 @@ STRING
alter table corrupted partition(org=2) set tblproperties('numRows'='-1234', 'STATS_GENERATED_VIA_STATS_TASK'='true');
explain select count(*) from corrupted where org = 2;
---- RESULTS: VERIFY_IS_SUBSET
-'WARNING: The following tables have potentially corrupt table statistics.'
-'Drop and re-compute statistics to resolve this problem.'
+'The row count in one or more partitions in the following tables '
+'is either a) less than -1, or b) 0 but the size of all the files inside '
+'the partition(s) is positive.'
+'The latter case does not necessarily imply the existence of corrupt '
+'statistics when the corresponding tables are transactional.'
+'If it is suspected that there may be corrupt statistics, dropping and '
+'re-computing statistics could resolve this problem.'
'$DATABASE.corrupted'
''
'03:AGGREGATE [FINALIZE]'
@@ -192,8 +202,13 @@ BIGINT, BIGINT, STRING, STRING, STRING, STRING, STRING, STRING
---- QUERY
explain select count(*) from corrupted_no_part;
---- RESULTS: VERIFY_IS_SUBSET
-'WARNING: The following tables have potentially corrupt table statistics.'
-'Drop and re-compute statistics to resolve this problem.'
+'The row count in one or more partitions in the following tables '
+'is either a) less than -1, or b) 0 but the size of all the files inside '
+'the partition(s) is positive.'
+'The latter case does not necessarily imply the existence of corrupt '
+'statistics when the corresponding tables are transactional.'
+'If it is suspected that there may be corrupt statistics, dropping and '
+'re-computing statistics could resolve this problem.'
'$DATABASE.corrupted_no_part'
''
'03:AGGREGATE [FINALIZE]'
@@ -215,8 +230,13 @@ STRING
alter table corrupted_no_part set tblproperties('numRows'='-1234', 'STATS_GENERATED_VIA_STATS_TASK'='true');
explain select count(*) from corrupted_no_part;
---- RESULTS: VERIFY_IS_SUBSET
-'WARNING: The following tables have potentially corrupt table statistics.'
-'Drop and re-compute statistics to resolve this problem.'
+'The row count in one or more partitions in the following tables '
+'is either a) less than -1, or b) 0 but the size of all the files inside '
+'the partition(s) is positive.'
+'The latter case does not necessarily imply the existence of corrupt '
+'statistics when the corresponding tables are transactional.'
+'If it is suspected that there may be corrupt statistics, dropping and '
+'re-computing statistics could resolve this problem.'
'$DATABASE.corrupted_no_part'
''
'03:AGGREGATE [FINALIZE]'