You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/05/24 10:18:25 UTC

[GitHub] [spark] rluta edited a comment on pull request #32640: [SPARK-35494][SQL] Timestamp cast performance issue

rluta edited a comment on pull request #32640:
URL: https://github.com/apache/spark/pull/32640#issuecomment-846941541


   I'm aware 2.4 is EOL but since 2.4.8 has just been released, I figured this patch may be included in a possible 2.4.9 since it has a pretty dramatic effect on some of our Oracle workloads.
   
   3.x branches have a completely different (and better) implementation with only 1 usual path with lower performance:
   
   ```
   OpenJDK 64-Bit Server VM 1.8.0_292-b10 on Mac OS X 10.16
   Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
   
   Timestamp Conversion:                     Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
   ------------------------------------------------------------------------------------------------------------------------
   no tz                                               917            940          20          4,6         218,6       1,0X
   with UTC tz                                        2172           2232          84          1,9         517,8       0,4X
   with hours tz                                       937            946           9          4,5         223,5       1,0X
   ```
   
   I can also provide a different patch on master to fix the UTC performance and provide the benchmark class if you're interested but maybe it should be attached to another issue in Jira.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org