You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2019/09/29 22:13:00 UTC
[jira] [Resolved] (SPARK-29282) Investigate the performance
regression in `core` module on JDK11 benchmark
[ https://issues.apache.org/jira/browse/SPARK-29282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun resolved SPARK-29282.
-----------------------------------
Fix Version/s: 3.0.0
Resolution: Fixed
Issue resolved by pull request 25966
[https://github.com/apache/spark/pull/25966]
> Investigate the performance regression in `core` module on JDK11 benchmark
> --------------------------------------------------------------------------
>
> Key: SPARK-29282
> URL: https://issues.apache.org/jira/browse/SPARK-29282
> Project: Spark
> Issue Type: Sub-task
> Components: Tests
> Affects Versions: 3.0.0
> Reporter: Dongjoon Hyun
> Assignee: Dongjoon Hyun
> Priority: Major
> Fix For: 3.0.0
>
>
> JDK11 versions are slower than JDK8.
> {code}
> #!/bin/bash
> git clean -fdx
> export JAVA_HOME=/Users/dongjoon/APACHE/jdk-release-adoptopenjdk/jdk8u222-b10/Contents/Home
> SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "core/test:runMain org.apache.spark.serializer.KryoSerializerBenchmark"
> cp core/benchmarks/KryoSerializerBenchmark-results.txt ~/KryoSerializerBenchmark-jdk8-results.txt
> git clean -fdx
> export JAVA_HOME=/Users/dongjoon/APACHE/jdk-release-adoptopenjdk/jdk-11.0.2+9/Contents/Home
> SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "core/test:runMain org.apache.spark.serializer.KryoSerializerBenchmark"
> cp core/benchmarks/KryoSerializerBenchmark-jdk11-results.txt ~/KryoSerializerBenchmark-jdk11.0.2-results.txt
> git clean -fdx
> export JAVA_HOME=/Users/dongjoon/APACHE/jdk-release-adoptopenjdk/jdk-11.0.3+7/Contents/Home
> SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "core/test:runMain org.apache.spark.serializer.KryoSerializerBenchmark"
> cp core/benchmarks/KryoSerializerBenchmark-jdk11-results.txt ~/KryoSerializerBenchmark-jdk11.0.3-results.txt
> git clean -fdx
> export JAVA_HOME=/Users/dongjoon/APACHE/jdk-release-adoptopenjdk/jdk-11.0.4+11/Contents/Home
> SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "core/test:runMain org.apache.spark.serializer.KryoSerializerBenchmark"
> cp core/benchmarks/KryoSerializerBenchmark-jdk11-results.txt ~/KryoSerializerBenchmark-jdk11.0.4-results.txt
> {code}
> {code}
> ================================================================================================
> Benchmark KryoPool vs old"pool of 1" implementation
> ================================================================================================
> OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Mac OS X 10.14.6
> Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz
> Benchmark KryoPool vs old"pool of 1" implementation: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
> ------------------------------------------------------------------------------------------------------------------------
> KryoPool:true 2015 2647 710 0.0 4029936.9 1.0X
> KryoPool:false 3526 4230 639 0.0 7052045.1 0.6X
> ================================================================================================
> Benchmark KryoPool vs old"pool of 1" implementation
> ================================================================================================
> OpenJDK 64-Bit Server VM 11.0.2+9 on Mac OS X 10.14.6
> Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz
> Benchmark KryoPool vs old"pool of 1" implementation: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
> ------------------------------------------------------------------------------------------------------------------------
> KryoPool:true 2694 3402 955 0.0 5387690.7 1.0X
> KryoPool:false 4050 4927 761 0.0 8099091.3 0.7X
> ================================================================================================
> Benchmark KryoPool vs old"pool of 1" implementation
> ================================================================================================
> OpenJDK 64-Bit Server VM 11.0.3+7 on Mac OS X 10.14.6
> Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz
> Benchmark KryoPool vs old"pool of 1" implementation: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
> ------------------------------------------------------------------------------------------------------------------------
> KryoPool:true 2638 3482 NaN 0.0 5275226.1 1.0X
> KryoPool:false 4165 4954 763 0.0 8329580.9 0.6X
> ================================================================================================
> Benchmark KryoPool vs old"pool of 1" implementation
> ================================================================================================
> OpenJDK 64-Bit Server VM 11.0.4+11 on Mac OS X 10.14.6
> Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz
> Benchmark KryoPool vs old"pool of 1" implementation: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
> ------------------------------------------------------------------------------------------------------------------------
> KryoPool:true 2613 3523 NaN 0.0 5225510.3 1.0X
> KryoPool:false 4037 4989 927 0.0 8074904.6 0.6X
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org