You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/02/15 21:48:19 UTC
[GitHub] [spark] MaxGekk opened a new pull request #27596: [WIP] Fix getting
of time components before 1582 year
MaxGekk opened a new pull request #27596: [WIP] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
-->
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
### Does this PR introduce any user-facing change?
<!--
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If no, write 'No'.
-->
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27596: [SPARK-30843][SQL] Fix
getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586830698
LGTM, merging to master/3.0!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27596: [SPARK-30843][SQL]
Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586695847
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #27596: [SPARK-30843][SQL]
Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27596:
[SPARK-30843][SQL] Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#discussion_r380901657
##########
File path: sql/core/benchmarks/DateTimeBenchmark-results.txt
##########
@@ -2,428 +2,428 @@
Extract components
================================================================================================
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
cast to timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to timestamp wholestage off 425 447 30 23.5 42.5 1.0X
-cast to timestamp wholestage on 368 401 29 27.2 36.8 1.2X
+cast to timestamp wholestage off 447 462 21 22.4 44.7 1.0X
+cast to timestamp wholestage on 390 426 54 25.7 39.0 1.1X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
year of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-year of timestamp wholestage off 1158 1215 80 8.6 115.8 1.0X
-year of timestamp wholestage on 1158 1179 31 8.6 115.8 1.0X
+year of timestamp wholestage off 1189 1285 135 8.4 118.9 1.0X
+year of timestamp wholestage on 1146 1156 9 8.7 114.6 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
quarter of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-quarter of timestamp wholestage off 1285 1295 15 7.8 128.5 1.0X
-quarter of timestamp wholestage on 1243 1257 11 8.0 124.3 1.0X
+quarter of timestamp wholestage off 1290 1293 4 7.8 129.0 1.0X
+quarter of timestamp wholestage on 1237 1251 13 8.1 123.7 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
month of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-month of timestamp wholestage off 1076 1082 8 9.3 107.6 1.0X
-month of timestamp wholestage on 1088 1098 9 9.2 108.8 1.0X
+month of timestamp wholestage off 1096 1101 7 9.1 109.6 1.0X
+month of timestamp wholestage on 1088 1095 7 9.2 108.8 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
weekofyear of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-weekofyear of timestamp wholestage off 1649 1659 14 6.1 164.9 1.0X
-weekofyear of timestamp wholestage on 1648 1656 8 6.1 164.8 1.0X
+weekofyear of timestamp wholestage off 1635 1636 1 6.1 163.5 1.0X
+weekofyear of timestamp wholestage on 1711 1714 4 5.8 171.1 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
day of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-day of timestamp wholestage off 1083 1084 3 9.2 108.3 1.0X
-day of timestamp wholestage on 1082 1089 13 9.2 108.2 1.0X
+day of timestamp wholestage off 1094 1108 20 9.1 109.4 1.0X
+day of timestamp wholestage on 1083 1092 8 9.2 108.3 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
dayofyear of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-dayofyear of timestamp wholestage off 1102 1103 1 9.1 110.2 1.0X
-dayofyear of timestamp wholestage on 1123 1138 14 8.9 112.3 1.0X
+dayofyear of timestamp wholestage off 1145 1145 1 8.7 114.5 1.0X
+dayofyear of timestamp wholestage on 1131 1141 12 8.8 113.1 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
dayofmonth of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-dayofmonth of timestamp wholestage off 1068 1073 7 9.4 106.8 1.0X
-dayofmonth of timestamp wholestage on 1082 1095 13 9.2 108.2 1.0X
+dayofmonth of timestamp wholestage off 1112 1124 17 9.0 111.2 1.0X
+dayofmonth of timestamp wholestage on 1082 1096 14 9.2 108.2 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
dayofweek of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-dayofweek of timestamp wholestage off 1265 1294 41 7.9 126.5 1.0X
-dayofweek of timestamp wholestage on 1253 1262 11 8.0 125.3 1.0X
+dayofweek of timestamp wholestage off 1254 1255 2 8.0 125.4 1.0X
+dayofweek of timestamp wholestage on 1250 1260 10 8.0 125.0 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
weekday of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-weekday of timestamp wholestage off 1189 1191 3 8.4 118.9 1.0X
-weekday of timestamp wholestage on 1193 1199 6 8.4 119.3 1.0X
+weekday of timestamp wholestage off 1190 1196 8 8.4 119.0 1.0X
+weekday of timestamp wholestage on 1204 1218 11 8.3 120.4 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
hour of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-hour of timestamp wholestage off 366 368 3 27.3 36.6 1.0X
-hour of timestamp wholestage on 360 364 6 27.8 36.0 1.0X
+hour of timestamp wholestage off 859 862 4 11.6 85.9 1.0X
+hour of timestamp wholestage on 825 827 2 12.1 82.5 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
minute of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-minute of timestamp wholestage off 348 350 2 28.7 34.8 1.0X
-minute of timestamp wholestage on 355 361 9 28.1 35.5 1.0X
+minute of timestamp wholestage off 841 851 15 11.9 84.1 1.0X
+minute of timestamp wholestage on 817 824 6 12.2 81.7 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
second of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-second of timestamp wholestage off 347 352 7 28.8 34.7 1.0X
-second of timestamp wholestage on 351 359 10 28.5 35.1 1.0X
+second of timestamp wholestage off 901 905 5 11.1 90.1 1.0X
+second of timestamp wholestage on 830 844 12 12.1 83.0 1.1X
================================================================================================
Current date and time
================================================================================================
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
current_date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-current_date wholestage off 284 287 4 35.2 28.4 1.0X
-current_date wholestage on 312 318 6 32.1 31.2 0.9X
+current_date wholestage off 290 292 2 34.4 29.0 1.0X
+current_date wholestage on 301 309 10 33.3 30.1 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
current_timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-current_timestamp wholestage off 291 292 2 34.4 29.1 1.0X
-current_timestamp wholestage on 297 333 40 33.6 29.7 1.0X
+current_timestamp wholestage off 300 301 0 33.3 30.0 1.0X
+current_timestamp wholestage on 316 348 36 31.6 31.6 0.9X
================================================================================================
Date arithmetic
================================================================================================
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
cast to date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to date wholestage off 903 903 1 11.1 90.3 1.0X
-cast to date wholestage on 897 900 7 11.2 89.7 1.0X
+cast to date wholestage off 964 973 13 10.4 96.4 1.0X
+cast to date wholestage on 900 905 4 11.1 90.0 1.1X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
last_day: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-last_day wholestage off 1082 1082 1 9.2 108.2 1.0X
-last_day wholestage on 1107 1118 16 9.0 110.7 1.0X
+last_day wholestage off 1125 1138 19 8.9 112.5 1.0X
+last_day wholestage on 1111 1122 12 9.0 111.1 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
next_day: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-next_day wholestage off 968 974 8 10.3 96.8 1.0X
-next_day wholestage on 958 959 1 10.4 95.8 1.0X
+next_day wholestage off 970 982 16 10.3 97.0 1.0X
+next_day wholestage on 955 958 4 10.5 95.5 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
date_add: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-date_add wholestage off 894 895 1 11.2 89.4 1.0X
-date_add wholestage on 882 890 9 11.3 88.2 1.0X
+date_add wholestage off 894 914 29 11.2 89.4 1.0X
+date_add wholestage on 880 884 4 11.4 88.0 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
date_sub: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-date_sub wholestage off 892 896 6 11.2 89.2 1.0X
-date_sub wholestage on 881 888 7 11.3 88.1 1.0X
+date_sub wholestage off 901 901 1 11.1 90.1 1.0X
+date_sub wholestage on 883 892 16 11.3 88.3 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
add_months: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-add_months wholestage off 1221 1223 3 8.2 122.1 1.0X
-add_months wholestage on 1212 1217 5 8.2 121.2 1.0X
+add_months wholestage off 1212 1213 0 8.2 121.2 1.0X
+add_months wholestage on 1212 1219 11 8.3 121.2 1.0X
================================================================================================
Formatting dates
================================================================================================
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
format date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-format date wholestage off 4989 5009 29 2.0 498.9 1.0X
-format date wholestage on 5037 5055 26 2.0 503.7 1.0X
+format date wholestage off 4973 5069 136 2.0 497.3 1.0X
+format date wholestage on 5061 5075 18 2.0 506.1 1.0X
================================================================================================
Formatting timestamps
================================================================================================
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
from_unixtime: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-from_unixtime wholestage off 9157 9164 10 1.1 915.7 1.0X
-from_unixtime wholestage on 9101 9120 16 1.1 910.1 1.0X
+from_unixtime wholestage off 8827 8835 11 1.1 882.7 1.0X
+from_unixtime wholestage on 8840 8845 5 1.1 884.0 1.0X
================================================================================================
Convert timestamps
================================================================================================
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
from_utc_timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-from_utc_timestamp wholestage off 732 739 10 13.7 73.2 1.0X
-from_utc_timestamp wholestage on 767 776 8 13.0 76.7 1.0X
+from_utc_timestamp wholestage off 1143 1145 2 8.7 114.3 1.0X
+from_utc_timestamp wholestage on 1174 1187 12 8.5 117.4 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
to_utc_timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-to_utc_timestamp wholestage off 802 805 3 12.5 80.2 1.0X
-to_utc_timestamp wholestage on 776 781 5 12.9 77.6 1.0X
+to_utc_timestamp wholestage off 1721 1725 7 5.8 172.1 1.0X
+to_utc_timestamp wholestage on 1605 1613 5 6.2 160.5 1.1X
================================================================================================
Intervals
================================================================================================
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
cast interval: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast interval wholestage off 328 330 3 30.5 32.8 1.0X
-cast interval wholestage on 319 326 7 31.3 31.9 1.0X
+cast interval wholestage off 342 345 5 29.3 34.2 1.0X
+cast interval wholestage on 329 336 13 30.4 32.9 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
datediff: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-datediff wholestage off 1762 1764 3 5.7 176.2 1.0X
-datediff wholestage on 1495 1502 7 6.7 149.5 1.2X
+datediff wholestage off 1643 1645 3 6.1 164.3 1.0X
+datediff wholestage on 1645 1657 8 6.1 164.5 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
months_between: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-months_between wholestage off 1338 1339 1 7.5 133.8 1.0X
-months_between wholestage on 1334 1339 5 7.5 133.4 1.0X
+months_between wholestage off 3369 3375 8 3.0 336.9 1.0X
+months_between wholestage on 3356 3362 5 3.0 335.6 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
window: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-window wholestage off 2023 2094 100 0.5 2023.2 1.0X
-window wholestage on 43505 43551 33 0.0 43504.8 0.0X
+window wholestage off 2017 2029 18 0.5 2016.8 1.0X
+window wholestage on 44825 44909 56 0.0 44824.9 0.0X
Review comment:
@rednaxelafx @hvanhovell Any ideas why codegen 22 times slower?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27596: [WIP] Fix getting of
time components before 1582 year
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27596: [WIP] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586657190
**[Test build #118483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118483/testReport)** for PR 27596 at commit [`4759725`](https://github.com/apache/spark/commit/475972516fc2fe50496d480eab334a29e05c50bd).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27596: [WIP] Fix
getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27596: [WIP] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586669521
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on issue #27596: [SPARK-30843][SQL] Fix
getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on issue #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586734677
@cloud-fan @HyukjinKwon May I ask you to review this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27596: [WIP] Fix getting
of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27596: [WIP] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586657293
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27596: [SPARK-30843][SQL] Fix
getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586720249
**[Test build #118503 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118503/testReport)** for PR 27596 at commit [`5949128`](https://github.com/apache/spark/commit/59491286e6a034b7f2dd1d8a9f79f84c62b51d23).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27596:
[SPARK-30843][SQL] Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#discussion_r379888227
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/DateExpressionsSuite.scala
##########
@@ -1060,22 +1060,31 @@ class DateExpressionsSuite extends SparkFunSuite with ExpressionEvalHelper {
outstandingTimezonesIds.foreach { timezone =>
var timestamp = MakeTimestamp(Literal(2019), Literal(8), Literal(10),
Literal(0), Literal(0), Literal(Decimal(BigDecimal(10.123456789), 8, 6)),
- Some(Literal(timezone)))
+ Some(Literal(timezone)), Some(timezone))
+ def millis(ts: MakeTimestamp): Milliseconds = Milliseconds(timestamp, Some(timezone))
+ def micros(ts: MakeTimestamp): Microseconds = Microseconds(timestamp, Some(timezone))
- checkEvaluation(Milliseconds(timestamp), Decimal(BigDecimal(10123.457), 8, 3))
- checkEvaluation(Microseconds(timestamp), 10123457)
+ checkEvaluation(millis(timestamp), Decimal(BigDecimal(10123.457), 8, 3))
+ checkEvaluation(
Review comment:
New test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27596: [WIP] Fix getting
of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27596: [WIP] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586669523
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118483/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27596: [WIP] Fix
getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27596: [WIP] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586657293
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27596: [WIP]
Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #27596: [WIP] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#discussion_r379880203
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala
##########
@@ -51,7 +51,6 @@ trait TimeZoneAwareExpression extends Expression {
/** Returns a copy of this expression with the specified timeZoneId. */
def withTimeZone(timeZoneId: String): TimeZoneAwareExpression
- @transient lazy val timeZone: TimeZone = DateTimeUtils.getTimeZone(timeZoneId.get)
Review comment:
Finally, all expressions bound on legacy `TimeZone` have been gone.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27596: [WIP] Fix
getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27596: [WIP] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586669523
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118483/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27596:
[SPARK-30843][SQL] Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#discussion_r379888343
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala
##########
@@ -290,32 +293,38 @@ class DateTimeUtilsSuite extends SparkFunSuite with Matchers with SQLHelper {
}
test("hours") {
- var input = date(2015, 3, 18, 13, 2, 11, 0, TimeZonePST)
- assert(getHours(input, TimeZonePST) === 13)
- assert(getHours(input, TimeZoneGMT) === 20)
- input = date(2015, 12, 8, 2, 7, 9, 0, TimeZonePST)
- assert(getHours(input, TimeZonePST) === 2)
- assert(getHours(input, TimeZoneGMT) === 10)
+ var input = date(2015, 3, 18, 13, 2, 11, 0, zonePST)
+ assert(getHours(input, zonePST) === 13)
+ assert(getHours(input, zoneGMT) === 20)
+ input = date(2015, 12, 8, 2, 7, 9, 0, zonePST)
+ assert(getHours(input, zonePST) === 2)
+ assert(getHours(input, zoneGMT) === 10)
+ input = date(10, 1, 1, 0, 0, 0, 0, zonePST)
+ assert(getHours(input, zonePST) === 0)
}
test("minutes") {
- var input = date(2015, 3, 18, 13, 2, 11, 0, TimeZonePST)
- assert(getMinutes(input, TimeZonePST) === 2)
- assert(getMinutes(input, TimeZoneGMT) === 2)
- assert(getMinutes(input, TimeZone.getTimeZone("Australia/North")) === 32)
- input = date(2015, 3, 8, 2, 7, 9, 0, TimeZonePST)
- assert(getMinutes(input, TimeZonePST) === 7)
- assert(getMinutes(input, TimeZoneGMT) === 7)
- assert(getMinutes(input, TimeZone.getTimeZone("Australia/North")) === 37)
+ var input = date(2015, 3, 18, 13, 2, 11, 0, zonePST)
+ assert(getMinutes(input, zonePST) === 2)
+ assert(getMinutes(input, zoneGMT) === 2)
+ assert(getMinutes(input, getZoneId("Australia/North")) === 32)
+ input = date(2015, 3, 8, 2, 7, 9, 0, zonePST)
+ assert(getMinutes(input, zonePST) === 7)
+ assert(getMinutes(input, zoneGMT) === 7)
+ assert(getMinutes(input, getZoneId("Australia/North")) === 37)
+ input = date(10, 1, 1, 0, 0, 0, 0, zonePST)
Review comment:
New test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27596:
[SPARK-30843][SQL] Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#discussion_r379888348
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala
##########
@@ -290,32 +293,38 @@ class DateTimeUtilsSuite extends SparkFunSuite with Matchers with SQLHelper {
}
test("hours") {
- var input = date(2015, 3, 18, 13, 2, 11, 0, TimeZonePST)
- assert(getHours(input, TimeZonePST) === 13)
- assert(getHours(input, TimeZoneGMT) === 20)
- input = date(2015, 12, 8, 2, 7, 9, 0, TimeZonePST)
- assert(getHours(input, TimeZonePST) === 2)
- assert(getHours(input, TimeZoneGMT) === 10)
+ var input = date(2015, 3, 18, 13, 2, 11, 0, zonePST)
+ assert(getHours(input, zonePST) === 13)
+ assert(getHours(input, zoneGMT) === 20)
+ input = date(2015, 12, 8, 2, 7, 9, 0, zonePST)
+ assert(getHours(input, zonePST) === 2)
+ assert(getHours(input, zoneGMT) === 10)
+ input = date(10, 1, 1, 0, 0, 0, 0, zonePST)
+ assert(getHours(input, zonePST) === 0)
}
test("minutes") {
- var input = date(2015, 3, 18, 13, 2, 11, 0, TimeZonePST)
- assert(getMinutes(input, TimeZonePST) === 2)
- assert(getMinutes(input, TimeZoneGMT) === 2)
- assert(getMinutes(input, TimeZone.getTimeZone("Australia/North")) === 32)
- input = date(2015, 3, 8, 2, 7, 9, 0, TimeZonePST)
- assert(getMinutes(input, TimeZonePST) === 7)
- assert(getMinutes(input, TimeZoneGMT) === 7)
- assert(getMinutes(input, TimeZone.getTimeZone("Australia/North")) === 37)
+ var input = date(2015, 3, 18, 13, 2, 11, 0, zonePST)
+ assert(getMinutes(input, zonePST) === 2)
+ assert(getMinutes(input, zoneGMT) === 2)
+ assert(getMinutes(input, getZoneId("Australia/North")) === 32)
+ input = date(2015, 3, 8, 2, 7, 9, 0, zonePST)
+ assert(getMinutes(input, zonePST) === 7)
+ assert(getMinutes(input, zoneGMT) === 7)
+ assert(getMinutes(input, getZoneId("Australia/North")) === 37)
+ input = date(10, 1, 1, 0, 0, 0, 0, zonePST)
+ assert(getMinutes(input, zonePST) === 0)
}
test("seconds") {
- var input = date(2015, 3, 18, 13, 2, 11, 0, TimeZonePST)
- assert(getSeconds(input, TimeZonePST) === 11)
- assert(getSeconds(input, TimeZoneGMT) === 11)
- input = date(2015, 3, 8, 2, 7, 9, 0, TimeZonePST)
- assert(getSeconds(input, TimeZonePST) === 9)
- assert(getSeconds(input, TimeZoneGMT) === 9)
+ var input = date(2015, 3, 18, 13, 2, 11, 0, zonePST)
+ assert(getSeconds(input, zonePST) === 11)
+ assert(getSeconds(input, zoneGMT) === 11)
+ input = date(2015, 3, 8, 2, 7, 9, 0, zonePST)
+ assert(getSeconds(input, zonePST) === 9)
+ assert(getSeconds(input, zoneGMT) === 9)
+ input = date(10, 1, 1, 0, 0, 0, 0, zonePST)
Review comment:
new test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27596: [WIP] Fix
getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27596: [WIP] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586657190
**[Test build #118483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118483/testReport)** for PR 27596 at commit [`4759725`](https://github.com/apache/spark/commit/475972516fc2fe50496d480eab334a29e05c50bd).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27596: [WIP] Fix
getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27596: [WIP] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586657294
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23242/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27596: [SPARK-30843][SQL] Fix
getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586695710
**[Test build #118503 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118503/testReport)** for PR 27596 at commit [`5949128`](https://github.com/apache/spark/commit/59491286e6a034b7f2dd1d8a9f79f84c62b51d23).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27596: [WIP] Fix getting
of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27596: [WIP] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586669521
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27596: [WIP]
Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #27596: [WIP] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#discussion_r379881509
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala
##########
@@ -290,32 +293,38 @@ class DateTimeUtilsSuite extends SparkFunSuite with Matchers with SQLHelper {
}
test("hours") {
- var input = date(2015, 3, 18, 13, 2, 11, 0, TimeZonePST)
- assert(getHours(input, TimeZonePST) === 13)
- assert(getHours(input, TimeZoneGMT) === 20)
- input = date(2015, 12, 8, 2, 7, 9, 0, TimeZonePST)
- assert(getHours(input, TimeZonePST) === 2)
- assert(getHours(input, TimeZoneGMT) === 10)
+ var input = date(2015, 3, 18, 13, 2, 11, 0, zonePST)
+ assert(getHours(input, zonePST) === 13)
+ assert(getHours(input, zoneGMT) === 20)
+ input = date(2015, 12, 8, 2, 7, 9, 0, zonePST)
+ assert(getHours(input, zonePST) === 2)
+ assert(getHours(input, zoneGMT) === 10)
+ input = date(10, 1, 1, 0, 0, 0, 0, zonePST)
+ assert(getHours(input, zonePST) === 0)
Review comment:
This is new test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27596:
[SPARK-30843][SQL] Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586720432
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118503/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27596: [WIP]
Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #27596: [WIP] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#discussion_r379881626
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala
##########
@@ -290,32 +293,38 @@ class DateTimeUtilsSuite extends SparkFunSuite with Matchers with SQLHelper {
}
test("hours") {
- var input = date(2015, 3, 18, 13, 2, 11, 0, TimeZonePST)
- assert(getHours(input, TimeZonePST) === 13)
- assert(getHours(input, TimeZoneGMT) === 20)
- input = date(2015, 12, 8, 2, 7, 9, 0, TimeZonePST)
- assert(getHours(input, TimeZonePST) === 2)
- assert(getHours(input, TimeZoneGMT) === 10)
+ var input = date(2015, 3, 18, 13, 2, 11, 0, zonePST)
+ assert(getHours(input, zonePST) === 13)
+ assert(getHours(input, zoneGMT) === 20)
+ input = date(2015, 12, 8, 2, 7, 9, 0, zonePST)
+ assert(getHours(input, zonePST) === 2)
+ assert(getHours(input, zoneGMT) === 10)
+ input = date(10, 1, 1, 0, 0, 0, 0, zonePST)
+ assert(getHours(input, zonePST) === 0)
Review comment:
Before the changes:
```sql
spark-sql> select hour(timestamp '0010-01-01 00:00:00');
23
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27596:
[SPARK-30843][SQL] Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#discussion_r379888232
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/DateExpressionsSuite.scala
##########
@@ -1060,22 +1060,31 @@ class DateExpressionsSuite extends SparkFunSuite with ExpressionEvalHelper {
outstandingTimezonesIds.foreach { timezone =>
var timestamp = MakeTimestamp(Literal(2019), Literal(8), Literal(10),
Literal(0), Literal(0), Literal(Decimal(BigDecimal(10.123456789), 8, 6)),
- Some(Literal(timezone)))
+ Some(Literal(timezone)), Some(timezone))
+ def millis(ts: MakeTimestamp): Milliseconds = Milliseconds(timestamp, Some(timezone))
+ def micros(ts: MakeTimestamp): Microseconds = Microseconds(timestamp, Some(timezone))
- checkEvaluation(Milliseconds(timestamp), Decimal(BigDecimal(10123.457), 8, 3))
- checkEvaluation(Microseconds(timestamp), 10123457)
+ checkEvaluation(millis(timestamp), Decimal(BigDecimal(10123.457), 8, 3))
+ checkEvaluation(
+ millis(timestamp.copy(year = Literal(10))),
+ Decimal(BigDecimal(10123.457), 8, 3))
+
+ checkEvaluation(micros(timestamp), 10123457)
+ checkEvaluation(
Review comment:
New test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27596: [SPARK-30843][SQL]
Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586720432
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118503/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27596: [WIP] Fix getting
of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27596: [WIP] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586657294
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23242/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27596:
[SPARK-30843][SQL] Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586720430
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27596: [SPARK-30843][SQL]
Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586695849
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23258/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27596:
[SPARK-30843][SQL] Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586695849
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23258/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27596: [SPARK-30843][SQL]
Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586720430
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #27596:
[SPARK-30843][SQL] Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#discussion_r379888297
##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/DateExpressionsSuite.scala
##########
@@ -1103,15 +1112,19 @@ class DateExpressionsSuite extends SparkFunSuite with ExpressionEvalHelper {
outstandingTimezonesIds.foreach { timezone =>
val timestamp = MakeTimestamp(Literal(2019), Literal(8), Literal(10),
Literal(0), Literal(0), Literal(Decimal(10.123456, 8, 6)),
- Some(Literal(timezone)))
+ Some(Literal(timezone)), Some(timezone))
+ def secFrac(ts: MakeTimestamp): SecondWithFraction = SecondWithFraction(ts, Some(timezone))
Review comment:
I added the function to use the tested time zone instead of the default one.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27596:
[SPARK-30843][SQL] Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586695710
**[Test build #118503 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118503/testReport)** for PR 27596 at commit [`5949128`](https://github.com/apache/spark/commit/59491286e6a034b7f2dd1d8a9f79f84c62b51d23).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27596:
[SPARK-30843][SQL] Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586695847
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27596:
[SPARK-30843][SQL] Fix getting of time components before 1582 year
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27596: [SPARK-30843][SQL] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#discussion_r381081926
##########
File path: sql/core/benchmarks/DateTimeBenchmark-results.txt
##########
@@ -2,428 +2,428 @@
Extract components
================================================================================================
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
cast to timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to timestamp wholestage off 425 447 30 23.5 42.5 1.0X
-cast to timestamp wholestage on 368 401 29 27.2 36.8 1.2X
+cast to timestamp wholestage off 447 462 21 22.4 44.7 1.0X
+cast to timestamp wholestage on 390 426 54 25.7 39.0 1.1X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
year of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-year of timestamp wholestage off 1158 1215 80 8.6 115.8 1.0X
-year of timestamp wholestage on 1158 1179 31 8.6 115.8 1.0X
+year of timestamp wholestage off 1189 1285 135 8.4 118.9 1.0X
+year of timestamp wholestage on 1146 1156 9 8.7 114.6 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
quarter of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-quarter of timestamp wholestage off 1285 1295 15 7.8 128.5 1.0X
-quarter of timestamp wholestage on 1243 1257 11 8.0 124.3 1.0X
+quarter of timestamp wholestage off 1290 1293 4 7.8 129.0 1.0X
+quarter of timestamp wholestage on 1237 1251 13 8.1 123.7 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
month of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-month of timestamp wholestage off 1076 1082 8 9.3 107.6 1.0X
-month of timestamp wholestage on 1088 1098 9 9.2 108.8 1.0X
+month of timestamp wholestage off 1096 1101 7 9.1 109.6 1.0X
+month of timestamp wholestage on 1088 1095 7 9.2 108.8 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
weekofyear of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-weekofyear of timestamp wholestage off 1649 1659 14 6.1 164.9 1.0X
-weekofyear of timestamp wholestage on 1648 1656 8 6.1 164.8 1.0X
+weekofyear of timestamp wholestage off 1635 1636 1 6.1 163.5 1.0X
+weekofyear of timestamp wholestage on 1711 1714 4 5.8 171.1 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
day of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-day of timestamp wholestage off 1083 1084 3 9.2 108.3 1.0X
-day of timestamp wholestage on 1082 1089 13 9.2 108.2 1.0X
+day of timestamp wholestage off 1094 1108 20 9.1 109.4 1.0X
+day of timestamp wholestage on 1083 1092 8 9.2 108.3 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
dayofyear of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-dayofyear of timestamp wholestage off 1102 1103 1 9.1 110.2 1.0X
-dayofyear of timestamp wholestage on 1123 1138 14 8.9 112.3 1.0X
+dayofyear of timestamp wholestage off 1145 1145 1 8.7 114.5 1.0X
+dayofyear of timestamp wholestage on 1131 1141 12 8.8 113.1 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
dayofmonth of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-dayofmonth of timestamp wholestage off 1068 1073 7 9.4 106.8 1.0X
-dayofmonth of timestamp wholestage on 1082 1095 13 9.2 108.2 1.0X
+dayofmonth of timestamp wholestage off 1112 1124 17 9.0 111.2 1.0X
+dayofmonth of timestamp wholestage on 1082 1096 14 9.2 108.2 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
dayofweek of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-dayofweek of timestamp wholestage off 1265 1294 41 7.9 126.5 1.0X
-dayofweek of timestamp wholestage on 1253 1262 11 8.0 125.3 1.0X
+dayofweek of timestamp wholestage off 1254 1255 2 8.0 125.4 1.0X
+dayofweek of timestamp wholestage on 1250 1260 10 8.0 125.0 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
weekday of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-weekday of timestamp wholestage off 1189 1191 3 8.4 118.9 1.0X
-weekday of timestamp wholestage on 1193 1199 6 8.4 119.3 1.0X
+weekday of timestamp wholestage off 1190 1196 8 8.4 119.0 1.0X
+weekday of timestamp wholestage on 1204 1218 11 8.3 120.4 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
hour of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-hour of timestamp wholestage off 366 368 3 27.3 36.6 1.0X
-hour of timestamp wholestage on 360 364 6 27.8 36.0 1.0X
+hour of timestamp wholestage off 859 862 4 11.6 85.9 1.0X
+hour of timestamp wholestage on 825 827 2 12.1 82.5 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
minute of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-minute of timestamp wholestage off 348 350 2 28.7 34.8 1.0X
-minute of timestamp wholestage on 355 361 9 28.1 35.5 1.0X
+minute of timestamp wholestage off 841 851 15 11.9 84.1 1.0X
+minute of timestamp wholestage on 817 824 6 12.2 81.7 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
second of timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-second of timestamp wholestage off 347 352 7 28.8 34.7 1.0X
-second of timestamp wholestage on 351 359 10 28.5 35.1 1.0X
+second of timestamp wholestage off 901 905 5 11.1 90.1 1.0X
+second of timestamp wholestage on 830 844 12 12.1 83.0 1.1X
================================================================================================
Current date and time
================================================================================================
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
current_date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-current_date wholestage off 284 287 4 35.2 28.4 1.0X
-current_date wholestage on 312 318 6 32.1 31.2 0.9X
+current_date wholestage off 290 292 2 34.4 29.0 1.0X
+current_date wholestage on 301 309 10 33.3 30.1 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
current_timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-current_timestamp wholestage off 291 292 2 34.4 29.1 1.0X
-current_timestamp wholestage on 297 333 40 33.6 29.7 1.0X
+current_timestamp wholestage off 300 301 0 33.3 30.0 1.0X
+current_timestamp wholestage on 316 348 36 31.6 31.6 0.9X
================================================================================================
Date arithmetic
================================================================================================
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
cast to date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to date wholestage off 903 903 1 11.1 90.3 1.0X
-cast to date wholestage on 897 900 7 11.2 89.7 1.0X
+cast to date wholestage off 964 973 13 10.4 96.4 1.0X
+cast to date wholestage on 900 905 4 11.1 90.0 1.1X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
last_day: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-last_day wholestage off 1082 1082 1 9.2 108.2 1.0X
-last_day wholestage on 1107 1118 16 9.0 110.7 1.0X
+last_day wholestage off 1125 1138 19 8.9 112.5 1.0X
+last_day wholestage on 1111 1122 12 9.0 111.1 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
next_day: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-next_day wholestage off 968 974 8 10.3 96.8 1.0X
-next_day wholestage on 958 959 1 10.4 95.8 1.0X
+next_day wholestage off 970 982 16 10.3 97.0 1.0X
+next_day wholestage on 955 958 4 10.5 95.5 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
date_add: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-date_add wholestage off 894 895 1 11.2 89.4 1.0X
-date_add wholestage on 882 890 9 11.3 88.2 1.0X
+date_add wholestage off 894 914 29 11.2 89.4 1.0X
+date_add wholestage on 880 884 4 11.4 88.0 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
date_sub: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-date_sub wholestage off 892 896 6 11.2 89.2 1.0X
-date_sub wholestage on 881 888 7 11.3 88.1 1.0X
+date_sub wholestage off 901 901 1 11.1 90.1 1.0X
+date_sub wholestage on 883 892 16 11.3 88.3 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
add_months: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-add_months wholestage off 1221 1223 3 8.2 122.1 1.0X
-add_months wholestage on 1212 1217 5 8.2 121.2 1.0X
+add_months wholestage off 1212 1213 0 8.2 121.2 1.0X
+add_months wholestage on 1212 1219 11 8.3 121.2 1.0X
================================================================================================
Formatting dates
================================================================================================
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
format date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-format date wholestage off 4989 5009 29 2.0 498.9 1.0X
-format date wholestage on 5037 5055 26 2.0 503.7 1.0X
+format date wholestage off 4973 5069 136 2.0 497.3 1.0X
+format date wholestage on 5061 5075 18 2.0 506.1 1.0X
================================================================================================
Formatting timestamps
================================================================================================
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
from_unixtime: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-from_unixtime wholestage off 9157 9164 10 1.1 915.7 1.0X
-from_unixtime wholestage on 9101 9120 16 1.1 910.1 1.0X
+from_unixtime wholestage off 8827 8835 11 1.1 882.7 1.0X
+from_unixtime wholestage on 8840 8845 5 1.1 884.0 1.0X
================================================================================================
Convert timestamps
================================================================================================
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
from_utc_timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-from_utc_timestamp wholestage off 732 739 10 13.7 73.2 1.0X
-from_utc_timestamp wholestage on 767 776 8 13.0 76.7 1.0X
+from_utc_timestamp wholestage off 1143 1145 2 8.7 114.3 1.0X
+from_utc_timestamp wholestage on 1174 1187 12 8.5 117.4 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
to_utc_timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-to_utc_timestamp wholestage off 802 805 3 12.5 80.2 1.0X
-to_utc_timestamp wholestage on 776 781 5 12.9 77.6 1.0X
+to_utc_timestamp wholestage off 1721 1725 7 5.8 172.1 1.0X
+to_utc_timestamp wholestage on 1605 1613 5 6.2 160.5 1.1X
================================================================================================
Intervals
================================================================================================
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
cast interval: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast interval wholestage off 328 330 3 30.5 32.8 1.0X
-cast interval wholestage on 319 326 7 31.3 31.9 1.0X
+cast interval wholestage off 342 345 5 29.3 34.2 1.0X
+cast interval wholestage on 329 336 13 30.4 32.9 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
datediff: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-datediff wholestage off 1762 1764 3 5.7 176.2 1.0X
-datediff wholestage on 1495 1502 7 6.7 149.5 1.2X
+datediff wholestage off 1643 1645 3 6.1 164.3 1.0X
+datediff wholestage on 1645 1657 8 6.1 164.5 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
months_between: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-months_between wholestage off 1338 1339 1 7.5 133.8 1.0X
-months_between wholestage on 1334 1339 5 7.5 133.4 1.0X
+months_between wholestage off 3369 3375 8 3.0 336.9 1.0X
+months_between wholestage on 3356 3362 5 3.0 335.6 1.0X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
+OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 on Linux 4.15.0-1044-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
window: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-window wholestage off 2023 2094 100 0.5 2023.2 1.0X
-window wholestage on 43505 43551 33 0.0 43504.8 0.0X
+window wholestage off 2017 2029 18 0.5 2016.8 1.0X
+window wholestage on 44825 44909 56 0.0 44824.9 0.0X
Review comment:
maybe the generated method is too large?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27596: [WIP] Fix getting of
time components before 1582 year
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27596: [WIP] Fix getting of time components before 1582 year
URL: https://github.com/apache/spark/pull/27596#issuecomment-586669414
**[Test build #118483 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118483/testReport)** for PR 27596 at commit [`4759725`](https://github.com/apache/spark/commit/475972516fc2fe50496d480eab334a29e05c50bd).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org