You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/11/13 13:33:00 UTC
[jira] [Work logged] (HIVE-24353) performance: Refactor TimestampTZ
parsing
[ https://issues.apache.org/jira/browse/HIVE-24353?focusedWorklogId=511359&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-511359 ]
ASF GitHub Bot logged work on HIVE-24353:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Nov/20 13:32
Start Date: 13/Nov/20 13:32
Worklog Time Spent: 10m
Work Description: pgaref commented on a change in pull request #1650:
URL: https://github.com/apache/hive/pull/1650#discussion_r522950225
##########
File path: common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java
##########
@@ -84,6 +86,18 @@ public void testVariations() {
TimestampTZUtil.parse("2017-05-08 07:45:00-3:00");
}
+ @Test
+ public void testPerformance() {
+ for (int i = 0; i < 100; i++) {
Review comment:
First loop redundant?
##########
File path: common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java
##########
@@ -84,6 +86,18 @@ public void testVariations() {
TimestampTZUtil.parse("2017-05-08 07:45:00-3:00");
}
+ @Test
+ public void testPerformance() {
+ for (int i = 0; i < 100; i++) {
+ TimestampTZUtil.parse("2017-01-01 13:33:00", ZoneId.of("UTC"));
+ }
+ Stopwatch sw = Stopwatch.createStarted();
+ for (int i = 0; i < 10000; i++) {
+ TimestampTZUtil.parse("2017-01-01 13:33:00", ZoneId.of("UTC"));
Review comment:
Can we add some randomness on the parsed timestamp? Maybe use i as part of time?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 511359)
Remaining Estimate: 0h
Time Spent: 10m
> performance: Refactor TimestampTZ parsing
> -----------------------------------------
>
> Key: HIVE-24353
> URL: https://issues.apache.org/jira/browse/HIVE-24353
> Project: Hive
> Issue Type: Improvement
> Reporter: Vincenz Priesnitz
> Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I found that for datasets that contain a lot of timestamps (without timezones) hive spends the majority of time in TimestampTZUtil.parse, in particular constructing stractraces for the try-catch blocks.
> When parsing TimestampTZ we are currently using a fallback chain with several try-catch blocks. For a common timestamp string without a timezone, we currently throw and catch 2 exceptions, and actually parse the string twice.
> I propose a refactor, that parses the string once and then expresses the fallback chain with queries to the parsed TemporalAccessor.
>
> Update: I added a PR that resolves this issue: [https://github.com/apache/hive/pull/1650]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)