This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 76874cf [SPARK-31111][SQL][TESTS] Fix interval output issue in ExtractBenchmark
76874cf is described below
commit 76874cfa4a9f6ada746f9c80a3a5a4d569f1a5a8
Author: Kent Yao <ya...@hotmail.com>
AuthorDate: Wed Mar 11 20:13:59 2020 +0800
[SPARK-31111][SQL][TESTS] Fix interval output issue in ExtractBenchmark
### What changes were proposed in this pull request?
fix the error caused by interval output in ExtractBenchmark
### Why are the changes needed?
fix a bug in the test
```scala
[info] Running case: cast to interval
[error] Exception in thread "main" org.apache.spark.sql.AnalysisException: Cannot use interval type in the table schema.;;
[error] OverwriteByExpression RelationV2[] noop-table, true, true
[error] +- Project [(subtractdates(cast(cast(id#0L as timestamp) as date), -719162) + subtracttimestamps(cast(id#0L as timestamp), -30610249419876544)) AS ((CAST(CAST(id AS TIMESTAMP) AS DATE) - DATE '0001-01-01') + (CAST(id AS TIMESTAMP) - TIMESTAMP '1000-01-01 01:02:03.123456'))#2]
[error] +- Range (1262304000, 1272304000, step=1, splits=Some(1))
[error]
[error] at org.apache.spark.sql.catalyst.util.TypeUtils$.failWithIntervalType(TypeUtils.scala:106)
[error] at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$25(CheckAnalysis.scala:389)
[error] at org.a
```
### Does this PR introduce any user-facing change?
no
### How was this patch tested?
re-run benchmark
Closes #27867 from yaooqinn/SPARK-31111.
Authored-by: Kent Yao <ya...@hotmail.com>
Signed-off-by: Wenchen Fan <we...@databricks.com>
(cherry picked from commit 2b46662bd0335320a668d2247a9c62552da07336)
Signed-off-by: Wenchen Fan <we...@databricks.com>
---
.../benchmarks/ExtractBenchmark-jdk11-results.txt | 208 ++++++++++-----------
sql/core/benchmarks/ExtractBenchmark-results.txt | 208 ++++++++++-----------
.../sql/execution/benchmark/ExtractBenchmark.scala | 4 +-
3 files changed, 211 insertions(+), 209 deletions(-)
diff --git a/sql/core/benchmarks/ExtractBenchmark-jdk11-results.txt b/sql/core/benchmarks/ExtractBenchmark-jdk11-results.txt
index 0572cc2..0873215 100644
--- a/sql/core/benchmarks/ExtractBenchmark-jdk11-results.txt
+++ b/sql/core/benchmarks/ExtractBenchmark-jdk11-results.txt
@@ -1,119 +1,119 @@
-OpenJDK 64-Bit Server VM 11.0.5+10-post-Ubuntu-0ubuntu1.118.04 on Linux 4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke extract for timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to timestamp 380 456 67 26.3 38.0 1.0X
-MILLENNIUM of timestamp 1274 1361 138 7.9 127.4 0.3X
-CENTURY of timestamp 1119 1132 19 8.9 111.9 0.3X
-DECADE of timestamp 1076 1083 6 9.3 107.6 0.4X
-YEAR of timestamp 1066 1098 43 9.4 106.6 0.4X
-ISOYEAR of timestamp 1190 1194 4 8.4 119.0 0.3X
-QUARTER of timestamp 1269 1273 4 7.9 126.9 0.3X
-MONTH of timestamp 1060 1075 22 9.4 106.0 0.4X
-WEEK of timestamp 1560 1565 8 6.4 156.0 0.2X
-DAY of timestamp 1039 1046 8 9.6 103.9 0.4X
-DAYOFWEEK of timestamp 1248 1274 24 8.0 124.8 0.3X
-DOW of timestamp 1252 1273 25 8.0 125.2 0.3X
-ISODOW of timestamp 1195 1204 9 8.4 119.5 0.3X
-DOY of timestamp 1081 1086 6 9.3 108.1 0.4X
-HOUR of timestamp 778 781 5 12.9 77.8 0.5X
-MINUTE of timestamp 779 780 1 12.8 77.9 0.5X
-SECOND of timestamp 597 611 20 16.7 59.7 0.6X
-MILLISECONDS of timestamp 636 642 6 15.7 63.6 0.6X
-MICROSECONDS of timestamp 498 504 5 20.1 49.8 0.8X
-EPOCH of timestamp 946 956 9 10.6 94.6 0.4X
+cast to timestamp 343 348 5 29.1 34.3 1.0X
+MILLENNIUM of timestamp 888 899 14 11.3 88.8 0.4X
+CENTURY of timestamp 835 846 15 12.0 83.5 0.4X
+DECADE of timestamp 824 834 9 12.1 82.4 0.4X
+YEAR of timestamp 818 824 10 12.2 81.8 0.4X
+ISOYEAR of timestamp 879 890 17 11.4 87.9 0.4X
+QUARTER of timestamp 965 1073 96 10.4 96.5 0.4X
+MONTH of timestamp 787 793 8 12.7 78.7 0.4X
+WEEK of timestamp 1277 1341 57 7.8 127.7 0.3X
+DAY of timestamp 780 785 8 12.8 78.0 0.4X
+DAYOFWEEK of timestamp 956 962 7 10.5 95.6 0.4X
+DOW of timestamp 995 1034 36 10.1 99.5 0.3X
+ISODOW of timestamp 885 898 11 11.3 88.5 0.4X
+DOY of timestamp 808 810 3 12.4 80.8 0.4X
+HOUR of timestamp 622 627 7 16.1 62.2 0.6X
+MINUTE of timestamp 629 636 8 15.9 62.9 0.5X
+SECOND of timestamp 800 819 30 12.5 80.0 0.4X
+MILLISECONDS of timestamp 743 755 21 13.5 74.3 0.5X
+MICROSECONDS of timestamp 692 730 53 14.5 69.2 0.5X
+EPOCH of timestamp 773 798 40 12.9 77.3 0.4X
-OpenJDK 64-Bit Server VM 11.0.5+10-post-Ubuntu-0ubuntu1.118.04 on Linux 4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke date_part for timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to timestamp 356 362 9 28.1 35.6 1.0X
-MILLENNIUM of timestamp 1260 1297 41 7.9 126.0 0.3X
-CENTURY of timestamp 1082 1085 3 9.2 108.2 0.3X
-DECADE of timestamp 1056 1068 19 9.5 105.6 0.3X
-YEAR of timestamp 1045 1053 13 9.6 104.5 0.3X
-ISOYEAR of timestamp 1300 1316 25 7.7 130.0 0.3X
-QUARTER of timestamp 1279 1280 2 7.8 127.9 0.3X
-MONTH of timestamp 1037 1046 11 9.6 103.7 0.3X
-WEEK of timestamp 1539 1557 28 6.5 153.9 0.2X
-DAY of timestamp 1032 1038 6 9.7 103.2 0.3X
-DAYOFWEEK of timestamp 1241 1244 4 8.1 124.1 0.3X
-DOW of timestamp 1237 1241 7 8.1 123.7 0.3X
-ISODOW of timestamp 1155 1158 3 8.7 115.5 0.3X
-DOY of timestamp 1075 1080 4 9.3 107.5 0.3X
-HOUR of timestamp 766 770 5 13.1 76.6 0.5X
-MINUTE of timestamp 764 769 4 13.1 76.4 0.5X
-SECOND of timestamp 590 592 2 16.9 59.0 0.6X
-MILLISECONDS of timestamp 627 636 10 16.0 62.7 0.6X
-MICROSECONDS of timestamp 493 505 15 20.3 49.3 0.7X
-EPOCH of timestamp 962 966 4 10.4 96.2 0.4X
+cast to timestamp 303 323 24 33.0 30.3 1.0X
+MILLENNIUM of timestamp 841 852 9 11.9 84.1 0.4X
+CENTURY of timestamp 818 821 4 12.2 81.8 0.4X
+DECADE of timestamp 821 836 15 12.2 82.1 0.4X
+YEAR of timestamp 858 928 61 11.7 85.8 0.4X
+ISOYEAR of timestamp 933 939 6 10.7 93.3 0.3X
+QUARTER of timestamp 979 1004 26 10.2 97.9 0.3X
+MONTH of timestamp 755 784 25 13.2 75.5 0.4X
+WEEK of timestamp 1189 1228 64 8.4 118.9 0.3X
+DAY of timestamp 768 770 4 13.0 76.8 0.4X
+DAYOFWEEK of timestamp 918 948 27 10.9 91.8 0.3X
+DOW of timestamp 933 983 53 10.7 93.3 0.3X
+ISODOW of timestamp 884 928 61 11.3 88.4 0.3X
+DOY of timestamp 787 797 12 12.7 78.7 0.4X
+HOUR of timestamp 611 664 58 16.4 61.1 0.5X
+MINUTE of timestamp 616 622 5 16.2 61.6 0.5X
+SECOND of timestamp 732 748 14 13.7 73.2 0.4X
+MILLISECONDS of timestamp 705 716 17 14.2 70.5 0.4X
+MICROSECONDS of timestamp 642 664 33 15.6 64.2 0.5X
+EPOCH of timestamp 760 781 18 13.2 76.0 0.4X
-OpenJDK 64-Bit Server VM 11.0.5+10-post-Ubuntu-0ubuntu1.118.04 on Linux 4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke extract for date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to date 886 905 17 11.3 88.6 1.0X
-MILLENNIUM of date 1253 1261 7 8.0 125.3 0.7X
-CENTURY of date 1068 1079 10 9.4 106.8 0.8X
-DECADE of date 1040 1068 25 9.6 104.0 0.9X
-YEAR of date 1032 1043 11 9.7 103.2 0.9X
-ISOYEAR of date 1304 1313 12 7.7 130.4 0.7X
-QUARTER of date 1284 1301 16 7.8 128.4 0.7X
-MONTH of date 1033 1036 4 9.7 103.3 0.9X
-WEEK of date 1535 1545 8 6.5 153.5 0.6X
-DAY of date 1023 1033 11 9.8 102.3 0.9X
-DAYOFWEEK of date 1230 1236 6 8.1 123.0 0.7X
-DOW of date 1238 1247 9 8.1 123.8 0.7X
-ISODOW of date 1159 1169 17 8.6 115.9 0.8X
-DOY of date 1082 1084 3 9.2 108.2 0.8X
-HOUR of date 1879 1891 11 5.3 187.9 0.5X
-MINUTE of date 1881 1905 21 5.3 188.1 0.5X
-SECOND of date 1718 1724 5 5.8 171.8 0.5X
-MILLISECONDS of date 1733 1737 6 5.8 173.3 0.5X
-MICROSECONDS of date 1629 1644 23 6.1 162.9 0.5X
-EPOCH of date 2085 2090 5 4.8 208.5 0.4X
+cast to date 712 786 120 14.0 71.2 1.0X
+MILLENNIUM of date 846 888 73 11.8 84.6 0.8X
+CENTURY of date 781 792 11 12.8 78.1 0.9X
+DECADE of date 770 824 48 13.0 77.0 0.9X
+YEAR of date 804 832 27 12.4 80.4 0.9X
+ISOYEAR of date 909 931 24 11.0 90.9 0.8X
+QUARTER of date 957 975 22 10.4 95.7 0.7X
+MONTH of date 789 794 7 12.7 78.9 0.9X
+WEEK of date 1141 1165 29 8.8 114.1 0.6X
+DAY of date 784 800 22 12.8 78.4 0.9X
+DAYOFWEEK of date 907 916 14 11.0 90.7 0.8X
+DOW of date 931 958 25 10.7 93.1 0.8X
+ISODOW of date 852 857 7 11.7 85.2 0.8X
+DOY of date 831 870 45 12.0 83.1 0.9X
+HOUR of date 1574 1598 26 6.4 157.4 0.5X
+MINUTE of date 1525 1590 58 6.6 152.5 0.5X
+SECOND of date 1728 1739 10 5.8 172.8 0.4X
+MILLISECONDS of date 1715 1727 10 5.8 171.5 0.4X
+MICROSECONDS of date 1559 1669 95 6.4 155.9 0.5X
+EPOCH of date 1766 1774 9 5.7 176.6 0.4X
-OpenJDK 64-Bit Server VM 11.0.5+10-post-Ubuntu-0ubuntu1.118.04 on Linux 4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke date_part for date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to date 888 891 6 11.3 88.8 1.0X
-MILLENNIUM of date 1250 1260 17 8.0 125.0 0.7X
-CENTURY of date 1070 1076 6 9.3 107.0 0.8X
-DECADE of date 1036 1041 5 9.6 103.6 0.9X
-YEAR of date 1037 1038 1 9.6 103.7 0.9X
-ISOYEAR of date 1300 1307 9 7.7 130.0 0.7X
-QUARTER of date 1267 1277 9 7.9 126.7 0.7X
-MONTH of date 1034 1037 4 9.7 103.4 0.9X
-WEEK of date 1543 1554 10 6.5 154.3 0.6X
-DAY of date 1022 1030 12 9.8 102.2 0.9X
-DAYOFWEEK of date 1230 1232 4 8.1 123.0 0.7X
-DOW of date 1227 1242 15 8.1 122.7 0.7X
-ISODOW of date 1157 1173 20 8.6 115.7 0.8X
-DOY of date 1073 1083 18 9.3 107.3 0.8X
-HOUR of date 1873 1878 7 5.3 187.3 0.5X
-MINUTE of date 1861 1876 14 5.4 186.1 0.5X
-SECOND of date 1717 1724 6 5.8 171.7 0.5X
-MILLISECONDS of date 1729 1736 7 5.8 172.9 0.5X
-MICROSECONDS of date 1622 1627 5 6.2 162.2 0.5X
-EPOCH of date 2066 2079 19 4.8 206.6 0.4X
+cast to date 803 825 23 12.5 80.3 1.0X
+MILLENNIUM of date 918 924 8 10.9 91.8 0.9X
+CENTURY of date 855 879 25 11.7 85.5 0.9X
+DECADE of date 777 795 19 12.9 77.7 1.0X
+YEAR of date 772 779 9 13.0 77.2 1.0X
+ISOYEAR of date 903 927 21 11.1 90.3 0.9X
+QUARTER of date 983 1339 381 10.2 98.3 0.8X
+MONTH of date 764 780 20 13.1 76.4 1.1X
+WEEK of date 1239 1316 112 8.1 123.9 0.6X
+DAY of date 769 857 91 13.0 76.9 1.0X
+DAYOFWEEK of date 900 965 61 11.1 90.0 0.9X
+DOW of date 990 1041 62 10.1 99.0 0.8X
+ISODOW of date 974 977 5 10.3 97.4 0.8X
+DOY of date 862 918 50 11.6 86.2 0.9X
+HOUR of date 1491 1517 25 6.7 149.1 0.5X
+MINUTE of date 1514 1520 9 6.6 151.4 0.5X
+SECOND of date 1657 1677 27 6.0 165.7 0.5X
+MILLISECONDS of date 1646 1669 26 6.1 164.6 0.5X
+MICROSECONDS of date 1538 1556 23 6.5 153.8 0.5X
+EPOCH of date 1699 1718 28 5.9 169.9 0.5X
-OpenJDK 64-Bit Server VM 11.0.5+10-post-Ubuntu-0ubuntu1.118.04 on Linux 4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke date_part for interval: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to interval 1273 1279 6 7.9 127.3 1.0X
-MILLENNIUM of interval 1323 1352 36 7.6 132.3 1.0X
-CENTURY of interval 1353 1356 4 7.4 135.3 0.9X
-DECADE of interval 1326 1339 11 7.5 132.6 1.0X
-YEAR of interval 1341 1345 3 7.5 134.1 0.9X
-QUARTER of interval 1368 1372 4 7.3 136.8 0.9X
-MONTH of interval 1320 1326 6 7.6 132.0 1.0X
-DAY of interval 1306 1310 4 7.7 130.6 1.0X
-HOUR of interval 1341 1347 8 7.5 134.1 0.9X
-MINUTE of interval 1337 1349 11 7.5 133.7 1.0X
-SECOND of interval 1450 1451 1 6.9 145.0 0.9X
-MILLISECONDS of interval 1476 1490 23 6.8 147.6 0.9X
-MICROSECONDS of interval 1316 1331 25 7.6 131.6 1.0X
-EPOCH of interval 1461 1462 1 6.8 146.1 0.9X
+cast to interval 971 976 5 10.3 97.1 1.0X
+MILLENNIUM of interval 983 1009 23 10.2 98.3 1.0X
+CENTURY of interval 972 984 12 10.3 97.2 1.0X
+DECADE of interval 954 962 10 10.5 95.4 1.0X
+YEAR of interval 954 973 17 10.5 95.4 1.0X
+QUARTER of interval 1009 1020 11 9.9 100.9 1.0X
+MONTH of interval 946 963 16 10.6 94.6 1.0X
+DAY of interval 952 963 12 10.5 95.2 1.0X
+HOUR of interval 948 960 11 10.5 94.8 1.0X
+MINUTE of interval 1035 1040 4 9.7 103.5 0.9X
+SECOND of interval 1085 1105 31 9.2 108.5 0.9X
+MILLISECONDS of interval 1065 1088 24 9.4 106.5 0.9X
+MICROSECONDS of interval 992 1007 17 10.1 99.2 1.0X
+EPOCH of interval 1087 1103 25 9.2 108.7 0.9X
diff --git a/sql/core/benchmarks/ExtractBenchmark-results.txt b/sql/core/benchmarks/ExtractBenchmark-results.txt
index 317f107..36bdbd0 100644
--- a/sql/core/benchmarks/ExtractBenchmark-results.txt
+++ b/sql/core/benchmarks/ExtractBenchmark-results.txt
@@ -1,119 +1,119 @@
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 1.8.0_231-b11 on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke extract for timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to timestamp 409 457 45 24.5 40.9 1.0X
-MILLENNIUM of timestamp 1240 1295 57 8.1 124.0 0.3X
-CENTURY of timestamp 1186 1240 49 8.4 118.6 0.3X
-DECADE of timestamp 1083 1104 20 9.2 108.3 0.4X
-YEAR of timestamp 1061 1073 15 9.4 106.1 0.4X
-ISOYEAR of timestamp 1198 1213 25 8.3 119.8 0.3X
-QUARTER of timestamp 1304 1322 23 7.7 130.4 0.3X
-MONTH of timestamp 1052 1067 19 9.5 105.2 0.4X
-WEEK of timestamp 1534 1558 25 6.5 153.4 0.3X
-DAY of timestamp 1038 1057 26 9.6 103.8 0.4X
-DAYOFWEEK of timestamp 1226 1239 22 8.2 122.6 0.3X
-DOW of timestamp 1212 1224 13 8.3 121.2 0.3X
-ISODOW of timestamp 1148 1165 24 8.7 114.8 0.4X
-DOY of timestamp 1066 1075 14 9.4 106.6 0.4X
-HOUR of timestamp 358 362 6 27.9 35.8 1.1X
-MINUTE of timestamp 364 369 4 27.4 36.4 1.1X
-SECOND of timestamp 453 471 26 22.1 45.3 0.9X
-MILLISECONDS of timestamp 497 500 2 20.1 49.7 0.8X
-MICROSECONDS of timestamp 360 363 4 27.8 36.0 1.1X
-EPOCH of timestamp 1100 1104 5 9.1 110.0 0.4X
+cast to timestamp 287 308 19 34.8 28.7 1.0X
+MILLENNIUM of timestamp 896 918 20 11.2 89.6 0.3X
+CENTURY of timestamp 849 856 7 11.8 84.9 0.3X
+DECADE of timestamp 765 777 17 13.1 76.5 0.4X
+YEAR of timestamp 754 756 2 13.3 75.4 0.4X
+ISOYEAR of timestamp 843 849 5 11.9 84.3 0.3X
+QUARTER of timestamp 867 873 9 11.5 86.7 0.3X
+MONTH of timestamp 758 762 4 13.2 75.8 0.4X
+WEEK of timestamp 1049 1054 6 9.5 104.9 0.3X
+DAY of timestamp 750 763 11 13.3 75.0 0.4X
+DAYOFWEEK of timestamp 890 918 25 11.2 89.0 0.3X
+DOW of timestamp 879 887 8 11.4 87.9 0.3X
+ISODOW of timestamp 862 869 11 11.6 86.2 0.3X
+DOY of timestamp 811 868 55 12.3 81.1 0.4X
+HOUR of timestamp 627 638 11 16.0 62.7 0.5X
+MINUTE of timestamp 600 606 6 16.7 60.0 0.5X
+SECOND of timestamp 743 799 51 13.5 74.3 0.4X
+MILLISECONDS of timestamp 723 737 22 13.8 72.3 0.4X
+MICROSECONDS of timestamp 648 653 5 15.4 64.8 0.4X
+EPOCH of timestamp 780 800 17 12.8 78.0 0.4X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 1.8.0_231-b11 on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke date_part for timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to timestamp 315 317 2 31.8 31.5 1.0X
-MILLENNIUM of timestamp 1155 1162 8 8.7 115.5 0.3X
-CENTURY of timestamp 1146 1152 5 8.7 114.6 0.3X
-DECADE of timestamp 1031 1043 11 9.7 103.1 0.3X
-YEAR of timestamp 1033 1041 10 9.7 103.3 0.3X
-ISOYEAR of timestamp 1274 1278 5 7.8 127.4 0.2X
-QUARTER of timestamp 1326 1346 30 7.5 132.6 0.2X
-MONTH of timestamp 1027 1031 7 9.7 102.7 0.3X
-WEEK of timestamp 1529 1535 6 6.5 152.9 0.2X
-DAY of timestamp 1024 1031 9 9.8 102.4 0.3X
-DAYOFWEEK of timestamp 1197 1201 5 8.4 119.7 0.3X
-DOW of timestamp 1201 1218 15 8.3 120.1 0.3X
-ISODOW of timestamp 1143 1149 6 8.8 114.3 0.3X
-DOY of timestamp 1074 1085 9 9.3 107.4 0.3X
-HOUR of timestamp 354 354 0 28.3 35.4 0.9X
-MINUTE of timestamp 358 361 5 27.9 35.8 0.9X
-SECOND of timestamp 445 456 17 22.5 44.5 0.7X
-MILLISECONDS of timestamp 504 514 12 19.8 50.4 0.6X
-MICROSECONDS of timestamp 361 370 8 27.7 36.1 0.9X
-EPOCH of timestamp 1111 1117 9 9.0 111.1 0.3X
+cast to timestamp 238 248 12 42.0 23.8 1.0X
+MILLENNIUM of timestamp 862 875 12 11.6 86.2 0.3X
+CENTURY of timestamp 833 847 22 12.0 83.3 0.3X
+DECADE of timestamp 759 765 7 13.2 75.9 0.3X
+YEAR of timestamp 744 755 15 13.4 74.4 0.3X
+ISOYEAR of timestamp 937 1019 73 10.7 93.7 0.3X
+QUARTER of timestamp 1011 1091 69 9.9 101.1 0.2X
+MONTH of timestamp 846 888 40 11.8 84.6 0.3X
+WEEK of timestamp 1210 1239 41 8.3 121.0 0.2X
+DAY of timestamp 932 979 41 10.7 93.2 0.3X
+DAYOFWEEK of timestamp 1174 1200 42 8.5 117.4 0.2X
+DOW of timestamp 1131 1172 37 8.8 113.1 0.2X
+ISODOW of timestamp 896 903 7 11.2 89.6 0.3X
+DOY of timestamp 805 818 12 12.4 80.5 0.3X
+HOUR of timestamp 596 597 2 16.8 59.6 0.4X
+MINUTE of timestamp 582 597 17 17.2 58.2 0.4X
+SECOND of timestamp 697 709 16 14.4 69.7 0.3X
+MILLISECONDS of timestamp 700 710 10 14.3 70.0 0.3X
+MICROSECONDS of timestamp 612 631 21 16.3 61.2 0.4X
+EPOCH of timestamp 755 760 7 13.2 75.5 0.3X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 1.8.0_231-b11 on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke extract for date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to date 849 851 3 11.8 84.9 1.0X
-MILLENNIUM of date 1129 1139 11 8.9 112.9 0.8X
-CENTURY of date 1136 1143 7 8.8 113.6 0.7X
-DECADE of date 1039 1043 5 9.6 103.9 0.8X
-YEAR of date 1030 1037 10 9.7 103.0 0.8X
-ISOYEAR of date 1269 1278 9 7.9 126.9 0.7X
-QUARTER of date 1323 1330 6 7.6 132.3 0.6X
-MONTH of date 1021 1023 2 9.8 102.1 0.8X
-WEEK of date 1541 1549 8 6.5 154.1 0.6X
-DAY of date 1021 1033 13 9.8 102.1 0.8X
-DAYOFWEEK of date 1196 1209 11 8.4 119.6 0.7X
-DOW of date 1214 1229 13 8.2 121.4 0.7X
-ISODOW of date 1148 1153 7 8.7 114.8 0.7X
-DOY of date 1073 1079 5 9.3 107.3 0.8X
-HOUR of date 1311 1314 4 7.6 131.1 0.6X
-MINUTE of date 1311 1311 1 7.6 131.1 0.6X
-SECOND of date 1420 1434 13 7.0 142.0 0.6X
-MILLISECONDS of date 1426 1442 14 7.0 142.6 0.6X
-MICROSECONDS of date 1312 1318 6 7.6 131.2 0.6X
-EPOCH of date 2034 2050 16 4.9 203.4 0.4X
+cast to date 633 641 7 15.8 63.3 1.0X
+MILLENNIUM of date 824 845 26 12.1 82.4 0.8X
+CENTURY of date 864 878 13 11.6 86.4 0.7X
+DECADE of date 746 763 17 13.4 74.6 0.8X
+YEAR of date 752 785 39 13.3 75.2 0.8X
+ISOYEAR of date 900 905 5 11.1 90.0 0.7X
+QUARTER of date 906 930 23 11.0 90.6 0.7X
+MONTH of date 747 752 6 13.4 74.7 0.8X
+WEEK of date 1089 1100 17 9.2 108.9 0.6X
+DAY of date 795 803 13 12.6 79.5 0.8X
+DAYOFWEEK of date 937 944 10 10.7 93.7 0.7X
+DOW of date 927 1003 66 10.8 92.7 0.7X
+ISODOW of date 977 980 7 10.2 97.7 0.6X
+DOY of date 827 877 45 12.1 82.7 0.8X
+HOUR of date 1525 1547 34 6.6 152.5 0.4X
+MINUTE of date 1473 1499 23 6.8 147.3 0.4X
+SECOND of date 1600 1615 15 6.2 160.0 0.4X
+MILLISECONDS of date 1666 1765 156 6.0 166.6 0.4X
+MICROSECONDS of date 1554 1627 127 6.4 155.4 0.4X
+EPOCH of date 1615 1646 27 6.2 161.5 0.4X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 1.8.0_231-b11 on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke date_part for date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to date 852 879 42 11.7 85.2 1.0X
-MILLENNIUM of date 1131 1136 7 8.8 113.1 0.8X
-CENTURY of date 1138 1145 6 8.8 113.8 0.7X
-DECADE of date 1030 1043 13 9.7 103.0 0.8X
-YEAR of date 1022 1028 8 9.8 102.2 0.8X
-ISOYEAR of date 1260 1265 6 7.9 126.0 0.7X
-QUARTER of date 1326 1330 7 7.5 132.6 0.6X
-MONTH of date 1014 1034 26 9.9 101.4 0.8X
-WEEK of date 1523 1526 5 6.6 152.3 0.6X
-DAY of date 1022 1023 2 9.8 102.2 0.8X
-DAYOFWEEK of date 1197 1203 9 8.4 119.7 0.7X
-DOW of date 1188 1198 16 8.4 118.8 0.7X
-ISODOW of date 1143 1153 9 8.8 114.3 0.7X
-DOY of date 1052 1058 7 9.5 105.2 0.8X
-HOUR of date 1309 1311 4 7.6 130.9 0.7X
-MINUTE of date 1302 1305 6 7.7 130.2 0.7X
-SECOND of date 1414 1432 16 7.1 141.4 0.6X
-MILLISECONDS of date 1441 1450 11 6.9 144.1 0.6X
-MICROSECONDS of date 1292 1301 8 7.7 129.2 0.7X
-EPOCH of date 2030 2036 8 4.9 203.0 0.4X
+cast to date 676 690 20 14.8 67.6 1.0X
+MILLENNIUM of date 868 882 16 11.5 86.8 0.8X
+CENTURY of date 875 880 8 11.4 87.5 0.8X
+DECADE of date 762 776 16 13.1 76.2 0.9X
+YEAR of date 788 800 11 12.7 78.8 0.9X
+ISOYEAR of date 903 917 12 11.1 90.3 0.7X
+QUARTER of date 983 1018 40 10.2 98.3 0.7X
+MONTH of date 836 857 19 12.0 83.6 0.8X
+WEEK of date 1137 1168 28 8.8 113.7 0.6X
+DAY of date 768 817 82 13.0 76.8 0.9X
+DAYOFWEEK of date 890 926 36 11.2 89.0 0.8X
+DOW of date 1007 1033 39 9.9 100.7 0.7X
+ISODOW of date 962 969 7 10.4 96.2 0.7X
+DOY of date 797 882 80 12.6 79.7 0.8X
+HOUR of date 1449 1482 29 6.9 144.9 0.5X
+MINUTE of date 1536 1610 69 6.5 153.6 0.4X
+SECOND of date 1675 1823 128 6.0 167.5 0.4X
+MILLISECONDS of date 1605 1622 18 6.2 160.5 0.4X
+MICROSECONDS of date 1481 1504 32 6.8 148.1 0.5X
+EPOCH of date 1656 1843 296 6.0 165.6 0.4X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux 4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 1.8.0_231-b11 on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke date_part for interval: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to interval 1249 1254 6 8.0 124.9 1.0X
-MILLENNIUM of interval 1310 1316 9 7.6 131.0 1.0X
-CENTURY of interval 1304 1315 10 7.7 130.4 1.0X
-DECADE of interval 1306 1313 7 7.7 130.6 1.0X
-YEAR of interval 1304 1313 11 7.7 130.4 1.0X
-QUARTER of interval 1310 1317 7 7.6 131.0 1.0X
-MONTH of interval 1311 1319 12 7.6 131.1 1.0X
-DAY of interval 1295 1304 13 7.7 129.5 1.0X
-HOUR of interval 1301 1306 8 7.7 130.1 1.0X
-MINUTE of interval 1316 1319 3 7.6 131.6 0.9X
-SECOND of interval 1437 1440 3 7.0 143.7 0.9X
-MILLISECONDS of interval 1435 1449 16 7.0 143.5 0.9X
-MICROSECONDS of interval 1304 1314 9 7.7 130.4 1.0X
-EPOCH of interval 1440 1453 19 6.9 144.0 0.9X
+cast to interval 919 959 45 10.9 91.9 1.0X
+MILLENNIUM of interval 959 1001 36 10.4 95.9 1.0X
+CENTURY of interval 974 998 21 10.3 97.4 0.9X
+DECADE of interval 975 989 13 10.3 97.5 0.9X
+YEAR of interval 987 1012 24 10.1 98.7 0.9X
+QUARTER of interval 1014 1031 25 9.9 101.4 0.9X
+MONTH of interval 978 1018 45 10.2 97.8 0.9X
+DAY of interval 1012 1095 92 9.9 101.2 0.9X
+HOUR of interval 1129 1143 14 8.9 112.9 0.8X
+MINUTE of interval 1038 1060 25 9.6 103.8 0.9X
+SECOND of interval 1062 1289 356 9.4 106.2 0.9X
+MILLISECONDS of interval 1142 1226 83 8.8 114.2 0.8X
+MICROSECONDS of interval 1038 1129 79 9.6 103.8 0.9X
+EPOCH of interval 1090 1127 34 9.2 109.0 0.8X
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/ExtractBenchmark.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/ExtractBenchmark.scala
index de23132..f1315d4 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/ExtractBenchmark.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/ExtractBenchmark.scala
@@ -42,7 +42,9 @@ object ExtractBenchmark extends SqlBasedBenchmark {
spark
.range(sinceSecond, sinceSecond + cardinality, 1, 1)
.selectExpr(exprs: _*)
- .noop()
+ .queryExecution
+ .toRdd
+ .foreach(_ => ())
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org