You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by db...@apache.org on 2019/03/13 20:27:28 UTC
[spark] branch master updated:
[SPARK-27034][SPARK-27123][SQL][FOLLOWUP] Update Nested Schema Pruning BM
result with EC2
This is an automated email from the ASF dual-hosted git repository.
dbtsai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 3221bf4 [SPARK-27034][SPARK-27123][SQL][FOLLOWUP] Update Nested Schema Pruning BM result with EC2
3221bf4 is described below
commit 3221bf4cd5199a366aa1041e1a8bf9de9ed2eb89
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Wed Mar 13 20:27:10 2019 +0000
[SPARK-27034][SPARK-27123][SQL][FOLLOWUP] Update Nested Schema Pruning BM result with EC2
## What changes were proposed in this pull request?
This is a follow up PR for #23943 in order to update the benchmark result with EC2 `r3.xlarge` instance.
## How was this patch tested?
N/A. (Manually compare the diff)
Closes #24078 from dongjoon-hyun/SPARK-27034.
Authored-by: Dongjoon Hyun <dh...@apple.com>
Signed-off-by: DB Tsai <d_...@apple.com>
---
.../OrcNestedSchemaPruningBenchmark-results.txt | 40 +++++++++++-----------
.../OrcV2NestedSchemaPruningBenchmark-results.txt | 40 +++++++++++-----------
...ParquetNestedSchemaPruningBenchmark-results.txt | 40 +++++++++++-----------
3 files changed, 60 insertions(+), 60 deletions(-)
diff --git a/sql/core/benchmarks/OrcNestedSchemaPruningBenchmark-results.txt b/sql/core/benchmarks/OrcNestedSchemaPruningBenchmark-results.txt
index fdd35cd..078fbbb 100644
--- a/sql/core/benchmarks/OrcNestedSchemaPruningBenchmark-results.txt
+++ b/sql/core/benchmarks/OrcNestedSchemaPruningBenchmark-results.txt
@@ -2,39 +2,39 @@
Nested Schema Pruning Benchmark For ORC v1
================================================================================================
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Selection: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 116 151 36 8.6 116.3 1.0X
-Nested column 544 604 31 1.8 544.5 0.2X
+Top-level column 117 154 23 8.5 117.5 1.0X
+Nested column 1271 1295 26 0.8 1270.5 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Limiting: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 360 397 32 2.8 360.4 1.0X
-Nested column 3322 3503 166 0.3 3322.4 0.1X
+Top-level column 431 488 73 2.3 431.2 1.0X
+Nested column 1738 1777 24 0.6 1738.3 0.2X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Repartitioning: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 292 334 32 3.4 291.8 1.0X
-Nested column 3306 3489 123 0.3 3305.7 0.1X
+Top-level column 349 381 87 2.9 348.7 1.0X
+Nested column 4374 4456 125 0.2 4373.6 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Repartitioning by exprs: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 302 333 27 3.3 302.0 1.0X
-Nested column 2697 3347 390 0.4 2697.4 0.1X
+Top-level column 358 410 85 2.8 358.5 1.0X
+Nested column 4447 4517 125 0.2 4447.3 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Sorting: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 316 440 146 3.2 315.8 1.0X
-Nested column 2728 2928 205 0.4 2727.9 0.1X
+Top-level column 553 566 13 1.8 552.8 1.0X
+Nested column 5115 5248 84 0.2 5115.1 0.1X
diff --git a/sql/core/benchmarks/OrcV2NestedSchemaPruningBenchmark-results.txt b/sql/core/benchmarks/OrcV2NestedSchemaPruningBenchmark-results.txt
index ad43ffb..eaaf3b0 100644
--- a/sql/core/benchmarks/OrcV2NestedSchemaPruningBenchmark-results.txt
+++ b/sql/core/benchmarks/OrcV2NestedSchemaPruningBenchmark-results.txt
@@ -2,39 +2,39 @@
Nested Schema Pruning Benchmark For ORC v2
================================================================================================
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Selection: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 91 102 9 11.0 91.2 1.0X
-Nested column 1459 1548 80 0.7 1458.5 0.1X
+Top-level column 120 148 24 8.3 120.0 1.0X
+Nested column 2367 2415 43 0.4 2367.0 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Limiting: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 101 112 10 9.9 100.7 1.0X
-Nested column 1459 1619 109 0.7 1458.9 0.1X
+Top-level column 129 153 16 7.8 128.5 1.0X
+Nested column 2368 2400 32 0.4 2367.7 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Repartitioning: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 268 284 12 3.7 268.2 1.0X
-Nested column 2781 2865 73 0.4 2780.8 0.1X
+Top-level column 359 396 59 2.8 358.9 1.0X
+Nested column 4100 4147 59 0.2 4099.9 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Repartitioning by exprs: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 309 318 6 3.2 309.2 1.0X
-Nested column 2426 2891 253 0.4 2425.8 0.1X
+Top-level column 367 394 56 2.7 366.8 1.0X
+Nested column 4182 4236 64 0.2 4181.5 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Sorting: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 179 194 8 5.6 179.3 1.0X
-Nested column 2084 2277 243 0.5 2083.7 0.1X
+Top-level column 262 269 6 3.8 262.5 1.0X
+Nested column 2950 3021 65 0.3 2949.8 0.1X
diff --git a/sql/core/benchmarks/ParquetNestedSchemaPruningBenchmark-results.txt b/sql/core/benchmarks/ParquetNestedSchemaPruningBenchmark-results.txt
index d51ebc6..1f1296e 100644
--- a/sql/core/benchmarks/ParquetNestedSchemaPruningBenchmark-results.txt
+++ b/sql/core/benchmarks/ParquetNestedSchemaPruningBenchmark-results.txt
@@ -2,39 +2,39 @@
Nested Schema Pruning Benchmark For Parquet
================================================================================================
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Selection: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 88 114 16 11.4 87.5 1.0X
-Nested column 201 223 27 5.0 200.5 0.4X
+Top-level column 145 174 23 6.9 145.1 1.0X
+Nested column 325 346 19 3.1 324.8 0.4X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Limiting: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 263 315 36 3.8 263.2 1.0X
-Nested column 2111 2622 613 0.5 2111.1 0.1X
+Top-level column 434 508 108 2.3 434.3 1.0X
+Nested column 625 647 23 1.6 624.8 0.7X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Repartitioning: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 222 250 34 4.5 222.2 1.0X
-Nested column 2084 2339 266 0.5 2084.2 0.1X
+Top-level column 357 368 9 2.8 356.9 1.0X
+Nested column 2897 2976 88 0.3 2897.4 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Repartitioning by exprs: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 238 306 96 4.2 238.1 1.0X
-Nested column 2080 2373 218 0.5 2079.5 0.1X
+Top-level column 365 413 77 2.7 364.9 1.0X
+Nested column 2902 2969 99 0.3 2902.4 0.1X
-Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
-Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
+OpenJDK 64-Bit Server VM 1.8.0_201-b09 on Linux 3.10.0-862.3.2.el7.x86_64
+Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Sorting: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Top-level column 328 383 57 3.1 327.6 1.0X
-Nested column 2595 3136 638 0.4 2595.1 0.1X
+Top-level column 555 600 80 1.8 555.4 1.0X
+Nested column 3448 3490 45 0.3 3447.9 0.2X
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org