You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by st...@apache.org on 2022/03/14 01:02:50 UTC
[impala] branch branch-3.4.1 created (now 5ecba2d)
This is an automated email from the ASF dual-hosted git repository.
stigahuang pushed a change to branch branch-3.4.1
in repository https://gitbox.apache.org/repos/asf/impala.git.
at 5ecba2d IMPALA-10426: Fix crash when inserting invalid timestamps
This branch includes the following new commits:
new 5ecba2d IMPALA-10426: Fix crash when inserting invalid timestamps
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[impala] 01/01: IMPALA-10426: Fix crash when inserting invalid timestamps
Posted by st...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
stigahuang pushed a commit to branch branch-3.4.1
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 5ecba2d46b48854921be24864689bf216e7fb4d8
Author: Zoltan Borok-Nagy <bo...@cloudera.com>
AuthorDate: Thu Jan 14 12:44:22 2021 +0100
IMPALA-10426: Fix crash when inserting invalid timestamps
Insertion of invalid timestamps causes Impala to crash when it uses
the INT64 Parquet timestamp types.
This patch fixes the error by checking for null values in
Int64TimestampColumnWriterBase::ConvertValue().
Testing:
* added e2e tests
Change-Id: I74fb754580663c99e1d8c3b73f8d62ea3305ac93
Reviewed-on: http://gerrit.cloudera.org:8080/16951
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
(cherry picked from commit 696dafed662cf7ef06b2a84bb4622eab4863f717)
---
be/src/exec/parquet/hdfs-parquet-table-writer.cc | 1 +
.../QueryTest/parquet-int64-timestamps.test | 40 ++++++++++++++++++++++
2 files changed, 41 insertions(+)
diff --git a/be/src/exec/parquet/hdfs-parquet-table-writer.cc b/be/src/exec/parquet/hdfs-parquet-table-writer.cc
index 7225384..708f3d5 100644
--- a/be/src/exec/parquet/hdfs-parquet-table-writer.cc
+++ b/be/src/exec/parquet/hdfs-parquet-table-writer.cc
@@ -594,6 +594,7 @@ public:
protected:
virtual void* ConvertValue(void* value) override {
+ if (UNLIKELY(value == nullptr)) return nullptr;
const TimestampValue* ts = reinterpret_cast<TimestampValue*>(value);
return ConvertTimestamp(*ts, &result_) ? &result_ : nullptr;
}
diff --git a/testdata/workloads/functional-query/queries/QueryTest/parquet-int64-timestamps.test b/testdata/workloads/functional-query/queries/QueryTest/parquet-int64-timestamps.test
index ce44e7b..9b36353 100644
--- a/testdata/workloads/functional-query/queries/QueryTest/parquet-int64-timestamps.test
+++ b/testdata/workloads/functional-query/queries/QueryTest/parquet-int64-timestamps.test
@@ -185,3 +185,43 @@ NULL
NULL
NULL
====
+---- QUERY
+# IMPALA-10426: invalid timestamps should be NULLs when inserted.
+create table invalid_timestamps (ts timestamp) stored as parquet;
+insert into invalid_timestamps
+values ('1021-01-14 11:36:44.916732000'), ('2021-02-34 11:36:44.916732000');
+select * from invalid_timestamps;
+---- RESULTS
+NULL
+NULL
+====
+---- QUERY
+truncate table invalid_timestamps;
+set parquet_timestamp_type=INT64_MILLIS;
+insert into invalid_timestamps
+values ('1021-01-14 11:36:44.916732000'), ('2021-02-34 11:36:44.916732000');
+select * from invalid_timestamps;
+---- RESULTS
+NULL
+NULL
+====
+---- QUERY
+truncate table invalid_timestamps;
+set parquet_timestamp_type=INT64_MICROS;
+insert into invalid_timestamps
+values ('1021-01-14 11:36:44.916732000'), ('2021-02-34 11:36:44.916732000');
+select * from invalid_timestamps;
+---- RESULTS
+NULL
+NULL
+====
+---- QUERY
+truncate table invalid_timestamps;
+set parquet_timestamp_type=INT64_NANOS;
+insert into invalid_timestamps
+values ('1021-01-14 11:36:44.916732000'), ('2021-02-34 11:36:44.916732000');
+select * from invalid_timestamps;
+---- RESULTS
+NULL
+NULL
+====