You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/05/24 10:18:25 UTC
[GitHub] [spark] rluta edited a comment on pull request #32640: [SPARK-35494][SQL] Timestamp cast performance issue
rluta edited a comment on pull request #32640:
URL: https://github.com/apache/spark/pull/32640#issuecomment-846941541
I'm aware 2.4 is EOL but since 2.4.8 has just been released, I figured this patch may be included in a possible 2.4.9 since it has a pretty dramatic effect on some of our Oracle workloads.
3.x branches have a completely different (and better) implementation with only 1 usual path with lower performance:
```
OpenJDK 64-Bit Server VM 1.8.0_292-b10 on Mac OS X 10.16
Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Timestamp Conversion: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
no tz 917 940 20 4,6 218,6 1,0X
with UTC tz 2172 2232 84 1,9 517,8 0,4X
with hours tz 937 946 9 4,5 223,5 1,0X
```
I can also provide a different patch on master to fix the UTC performance and provide the benchmark class if you're interested but maybe it should be attached to another issue in Jira.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org