You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "David Mollitor (Jira)" <ji...@apache.org> on 2021/01/27 03:00:00 UTC
[jira] [Commented] (HIVE-24353) performance: Refactor TimestampTZ
parsing
[ https://issues.apache.org/jira/browse/HIVE-24353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272527#comment-17272527 ]
David Mollitor commented on HIVE-24353:
---------------------------------------
I just saw this issue in the wild. I'll take a look at this.
The issue I am seeing involves a scenario where Hive is parsing/formatting many timestamp strings, more than it probably should. However, it is compounded by the fact that each string it parocesses is in the format {{2021-01-26 10:32:32.000000000}}. This is an issue because the current parsing code expects the time zone to be specified, or else it fails like you said.
I believe there are some areas in the Hive code base that generates timestamp string without timezone information and then those end up getting parsed here (slowly).
> performance: Refactor TimestampTZ parsing
> -----------------------------------------
>
> Key: HIVE-24353
> URL: https://issues.apache.org/jira/browse/HIVE-24353
> Project: Hive
> Issue Type: Improvement
> Reporter: Vincenz Priesnitz
> Assignee: Vincenz Priesnitz
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> I found that for datasets that contain a lot of timestamps (without timezones) hive spends the majority of time in TimestampTZUtil.parse, in particular constructing stractraces for the try-catch blocks.
> When parsing TimestampTZ we are currently using a fallback chain with several try-catch blocks. For a common timestamp string without a timezone, we currently throw and catch 2 exceptions, and actually parse the string twice.
> I propose a refactor, that parses the string once and then expresses the fallback chain with queries to the parsed TemporalAccessor.
>
> Update: I added a PR that resolves this issue: [https://github.com/apache/hive/pull/1650]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)