You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Zihao Ye (Jira)" <ji...@apache.org> on 2024/06/03 10:30:00 UTC
[jira] [Commented] (IMPALA-12322) return wrong timestamp when scan kudu timestamp with timezone
[ https://issues.apache.org/jira/browse/IMPALA-12322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17851574#comment-17851574 ]
Zihao Ye commented on IMPALA-12322:
-----------------------------------
[~csringhofer] I believe that there should be a separate query option to control the writing behavior, such as write_kudu_utc_timestamps is good. Here are two reasons for this way:
Backward Compatibility: Existing users who have configured convert_kudu_utc_timestamps for reading and have independently managed the timezone conversion for writing may experience unexpected data inconsistencies if the existing option is changed to also control writing. Introducing a separate option ensures that the current behavior remains stable and avoids unintended consequences during upgrades.
Flexibility: A separate option allows for greater flexibility. Users can independently control the behavior of reading and writing operations, thereby accommodating different use cases and requirements without imposing a one-size-fits-all solution.
> return wrong timestamp when scan kudu timestamp with timezone
> -------------------------------------------------------------
>
> Key: IMPALA-12322
> URL: https://issues.apache.org/jira/browse/IMPALA-12322
> Project: IMPALA
> Issue Type: Bug
> Affects Versions: Impala 4.1.1
> Environment: impala 4.1.1
> Reporter: daicheng
> Assignee: Zihao Ye
> Priority: Major
> Attachments: image-2022-04-24-00-01-05-746-1.png, image-2022-04-24-00-01-05-746.png, image-2022-04-24-00-01-37-520.png, image-2022-04-24-00-03-14-467-1.png, image-2022-04-24-00-03-14-467.png, image-2022-04-24-00-04-16-240-1.png, image-2022-04-24-00-04-16-240.png, image-2022-04-24-00-04-52-860-1.png, image-2022-04-24-00-04-52-860.png, image-2022-04-24-00-05-52-086-1.png, image-2022-04-24-00-05-52-086.png, image-2022-04-24-00-07-09-776-1.png, image-2022-04-24-00-07-09-776.png, image-2023-07-28-20-31-09-457.png, image-2023-07-28-22-27-38-521.png, image-2023-07-28-22-29-40-083.png, image-2023-07-28-22-36-17-460.png, image-2023-07-28-22-36-37-884.png, image-2023-07-28-22-38-19-728.png
>
>
> impala version is 3.1.0-cdh6.1
> i have set system timezone=Asia/Shanghai:
> !image-2022-04-24-00-01-37-520.png!
> !image-2022-04-24-00-01-05-746.png!
> here is the bug:
> *step 1*
> i have parquet file with two columns like below,and read it with impala-shell and spark (timezone=shanghai)
> !image-2022-04-24-00-03-14-467.png|width=1016,height=154!
> !image-2022-04-24-00-04-16-240.png|width=944,height=367!
> the result both exactly right。
> *step two*
> create kudu table with impala-shell:
> CREATE TABLE default.test_{_}test{_}_test_time2 (id BIGINT,t TIMESTAMP,PRIMARY KEY (id) ) STORED AS KUDU;
> note: kudu version:1.8
> and insert 2 row into the table with spark :
> !image-2022-04-24-00-04-52-860.png|width=914,height=279!
> *stop 3*
> read it with spark (timezone=shanghai)，spark read kudu table with kudu-client api,here is the result:
> !image-2022-04-24-00-05-52-086.png|width=914,height=301!
> the result is still exactly right。
> but read it with impala-shell:
> !image-2022-04-24-00-07-09-776.png|width=915,height=154!
> the result show late 8hour
> *conclusion*
> it seems like impala timezone didn't work when kudu column type is timestamp, but it work fine in parquet file,I don't know why?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org