You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by ta...@apache.org on 2021/02/10 00:38:46 UTC
[impala] branch master updated (81d5377 -> 1f7b413)
This is an automated email from the ASF dual-hosted git repository.
tarmstrong pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git.
from 81d5377 IMPALA-10475: [DOCS] elaborate SYNC_DDL option
new 701714b IMPALA-10379: Add missing HiveLexer classes to shared-deps
new 8551805 IMPALA-9586: update query option docs for mt_dop
new 1f7b413 IMPALA-8721: re-enable test_hive_impala_interop
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/topics/impala_num_nodes.xml | 5 +++++
docs/topics/impala_num_scanner_threads.xml | 4 ++++
java/shaded-deps/hive-exec/pom.xml | 3 +++
testdata/workloads/functional-query/queries/QueryTest/misc.test | 9 +++++++++
tests/custom_cluster/test_hive_parquet_codec_interop.py | 6 ++----
5 files changed, 23 insertions(+), 4 deletions(-)
[impala] 02/03: IMPALA-9586: update query option docs for mt_dop
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tarmstrong pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 8551805875fcd2c701d988df887c1173520fca12
Author: Tim Armstrong <ta...@cloudera.com>
AuthorDate: Mon Feb 8 20:44:16 2021 -0800
IMPALA-9586: update query option docs for mt_dop
There are interactions between mt_dop and num_nodes and
num_scanner_threads. Mention these in the docs.
Change-Id: I3d9a6f56ffaf211d7d3ca1fad506ee83d516ccbd
Reviewed-on: http://gerrit.cloudera.org:8080/17043
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
---
docs/topics/impala_num_nodes.xml | 5 +++++
docs/topics/impala_num_scanner_threads.xml | 4 ++++
2 files changed, 9 insertions(+)
diff --git a/docs/topics/impala_num_nodes.xml b/docs/topics/impala_num_nodes.xml
index f885124..0bbf782 100644
--- a/docs/topics/impala_num_nodes.xml
+++ b/docs/topics/impala_num_nodes.xml
@@ -55,6 +55,11 @@ under the License.
<p conref="../shared/impala_common.xml#common/usage_notes_blurb"/>
<p>
+ Setting <codeph>NUM_NODES</codeph> to 1 disables multithreading, i.e. if
+ <codeph>MT_DOP</codeph> is greater than 1, it is effectively reduced to 1.
+ </p>
+
+ <p>
If you are diagnosing a problem that you suspect is due to a timing issue due to
distributed query processing, you can set <codeph>NUM_NODES=1</codeph> to verify
if the problem still occurs when all the work is done on a single node.
diff --git a/docs/topics/impala_num_scanner_threads.xml b/docs/topics/impala_num_scanner_threads.xml
index b7a47ae..dd111da 100644
--- a/docs/topics/impala_num_scanner_threads.xml
+++ b/docs/topics/impala_num_scanner_threads.xml
@@ -42,6 +42,10 @@ under the License.
</p>
<p>
+ Has no effect if multi-threaded query execution is enabled, i.e.
+ <codeph>MT_DOP</codeph> is greater than 0.
+ </p>
+ <p>
<b>Type:</b> numeric
</p>
[impala] 03/03: IMPALA-8721: re-enable test_hive_impala_interop
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tarmstrong pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 1f7b413d11321bd74aaa1a9ea9ed30e42222d80d
Author: Tim Armstrong <ta...@cloudera.com>
AuthorDate: Mon Feb 8 20:18:24 2021 -0800
IMPALA-8721: re-enable test_hive_impala_interop
The test now passes because HIVE-21290 was fixed.
Revert "IMPALA-8689: test_hive_impala_interop failing with "Timeout >7200s""
This reverts commit 5d8c99ce74c45a7d04f11e1f252b346d654f02bf.
Change-Id: I7e2beabd7082a45a0fc3b60d318cf698079768ff
Reviewed-on: http://gerrit.cloudera.org:8080/17042
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
tests/custom_cluster/test_hive_parquet_codec_interop.py | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/tests/custom_cluster/test_hive_parquet_codec_interop.py b/tests/custom_cluster/test_hive_parquet_codec_interop.py
index e8dbefb..e1b43e2 100644
--- a/tests/custom_cluster/test_hive_parquet_codec_interop.py
+++ b/tests/custom_cluster/test_hive_parquet_codec_interop.py
@@ -58,11 +58,9 @@ class TestParquetInterop(CustomClusterTestSuite):
def test_hive_impala_interop(self, vector, unique_database, cluster_properties):
# Setup source table.
source_table = "{0}.{1}".format(unique_database, "t1_source")
- # TODO: Once IMPALA-8721 is fixed add coverage for TimeStamp data type.
self.execute_query_expect_success(self.client,
- "create table {0} as select id, bool_col, tinyint_col, smallint_col, int_col, "
- "bigint_col, float_col, double_col, date_string_col, string_col, year, month "
- "from functional_parquet.alltypes".format(source_table))
+ "create table {0} as select * from functional_parquet.alltypes"
+ .format(source_table))
self.execute_query_expect_success(self.client,
"insert into {0}(id) values (7777), (8888), (9999), (11111), (22222), (33333)"
.format(source_table))
[impala] 01/03: IMPALA-10379: Add missing HiveLexer classes to
shared-deps
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tarmstrong pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 701714b10a77aee62cf2ad3e25db9e2dfd418780
Author: Tamas Mate <tm...@cloudera.com>
AuthorDate: Mon Feb 1 10:41:40 2021 +0100
IMPALA-10379: Add missing HiveLexer classes to shared-deps
HIVE-19064 introduced additional lexer classes that are required during
runtime. This commit adds the missing HiveLexer lexer classes to the
shared-deps. Without these classes queries such as 'select 1 as "``"'
would fail with 'NoClassDefFoundError'.
Testing:
- added a misc.test to verify that the classes are available and that
IMPALA-9641 is fixed by HIVE-19064
Change-Id: I6e3a00335983f26498c1130ab9f109f6e67256f5
Reviewed-on: http://gerrit.cloudera.org:8080/17019
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
java/shaded-deps/hive-exec/pom.xml | 3 +++
testdata/workloads/functional-query/queries/QueryTest/misc.test | 9 +++++++++
2 files changed, 12 insertions(+)
diff --git a/java/shaded-deps/hive-exec/pom.xml b/java/shaded-deps/hive-exec/pom.xml
index 6c6d070..639abc6 100644
--- a/java/shaded-deps/hive-exec/pom.xml
+++ b/java/shaded-deps/hive-exec/pom.xml
@@ -97,6 +97,9 @@ the same dependencies
<include>org/apache/hadoop/hive/ql/exec/FunctionUtils*</include>
<include>org/apache/hadoop/hive/ql/parse/GenericHiveLexer*</include>
<include>org/apache/hadoop/hive/ql/parse/HiveLexer*</include>
+ <include>org/apache/hadoop/hive/ql/parse/ANTLRNoCaseStringStream*</include>
+ <include>org/apache/hadoop/hive/ql/parse/ParseError*</include>
+ <include>org/apache/hadoop/hive/ql/parse/Quotation*</include>
<include>org/apache/hadoop/hive/ql/udf/**/*</include>
<!-- Many of the UDFs are annotated with their vectorized counter-parts.
Including them makes sure that we don't break -->
diff --git a/testdata/workloads/functional-query/queries/QueryTest/misc.test b/testdata/workloads/functional-query/queries/QueryTest/misc.test
index 903101c..2f48788 100644
--- a/testdata/workloads/functional-query/queries/QueryTest/misc.test
+++ b/testdata/workloads/functional-query/queries/QueryTest/misc.test
@@ -109,6 +109,15 @@ SELECT "quote \"", 'quote \''
string, string
====
---- QUERY
+# IMPALA-9641: Empty double quotation should not cause infinite loop in the
+# ImpalaD Frontend
+SELECT 1 as "``"
+---- RESULTS
+1
+---- TYPES
+int
+====
+---- QUERY
# Select from table that contains unsupported primitive types
SELECT int_col, str_col, bigint_col from functional.unsupported_types
---- RESULTS