You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2026/08/09 20:53:12 UTC
(spark) branch branch-4.3 updated: [SPARK-58678][CORE][TEST] Add `OpenHashMapBenchmark`
This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.3 by this push:
new 62e79ede03b7 [SPARK-58678][CORE][TEST] Add `OpenHashMapBenchmark`
62e79ede03b7 is described below
commit 62e79ede03b79a26b53b3130aa9935a1b14e78ed
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Sun Aug 9 13:52:43 2026 -0700
[SPARK-58678][CORE][TEST] Add `OpenHashMapBenchmark`
### What changes were proposed in this pull request?
This PR aims to add `OpenHashMapBenchmark` to measure the performance of
`org.apache.spark.util.collection.OpenHashMap` against `java.util.HashMap` in three scenarios
modeled on Spark's real usage:
- **Insert**: inserting 1M distinct String keys via `update`/`put`
- **Aggregate**: 5M `changeValue`/`merge` operations over 1M String keys (the typical
aggregation pattern, e.g. `Mode`, `countByValue`)
- **Lookup**: looking up 1M String keys in random order
### Why are the changes needed?
`OpenHashMap` claims that it is "about 5X faster than java.util.HashMap".
https://github.com/apache/spark/blob/8645e2c08929bbc1ef2ef66b442f525814f92220/core/src/main/scala/org/apache/spark/util/collection/OpenHashMap.scala#L24
However, This claim dates from 2013 (pre-JDK 8), and there has been no benchmark in the repository to
verify it. On modern JDKs, `java.util.HashMap` has improved significantly. According to this benchmark, Java is much faster for **Insert** and **Lookup**.
- In case of `Insert`, `java.util.HashMap` is 4.6X, 5.5X, 6.5X faster in Java 17, 21, and 25, respectively.
- Note that `OpenHashMap` still uses about 2.2x less memory than `java.util.HashMap` for
`String -> Long` entries thanks to its specialized primitive value storage. This benchmark
provides a consistent way to track the trade-off across JDK versions and future improvements.
### Does this PR introduce _any_ user-facing change?
No. This is a test-only change.
### How was this patch tested?
Manually ran the benchmark:
```
build/sbt "core/Test/runMain org.apache.spark.util.collection.OpenHashMapBenchmark"
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Fable 5
Closes #57881 from dongjoon-hyun/SPARK-58678-2.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
(cherry picked from commit 9a20f8b3693c527485e432a57d3fcac6df8eda86)
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
.../OpenHashMapBenchmark-jdk21-results.txt | 26 ++++
.../OpenHashMapBenchmark-jdk25-results.txt | 26 ++++
core/benchmarks/OpenHashMapBenchmark-results.txt | 26 ++++
.../util/collection/OpenHashMapBenchmark.scala | 133 +++++++++++++++++++++
4 files changed, 211 insertions(+)
diff --git a/core/benchmarks/OpenHashMapBenchmark-jdk21-results.txt b/core/benchmarks/OpenHashMapBenchmark-jdk21-results.txt
new file mode 100644
index 000000000000..c98d2b88560e
--- /dev/null
+++ b/core/benchmarks/OpenHashMapBenchmark-jdk21-results.txt
@@ -0,0 +1,26 @@
+================================================================================================
+OpenHashMap vs java.util.HashMap
+================================================================================================
+
+OpenJDK 64-Bit Server VM 21.0.12+8-LTS on Linux 6.17.0-1020-azure
+AMD EPYC 7763 64-Core Processor
+Insert 1000000 distinct String keys: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
+------------------------------------------------------------------------------------------------------------------------
+OpenHashMap 262 280 15 3.8 262.0 1.0X
+java.util.HashMap 48 84 57 21.0 47.7 5.5X
+
+OpenJDK 64-Bit Server VM 21.0.12+8-LTS on Linux 6.17.0-1020-azure
+AMD EPYC 7763 64-Core Processor
+Aggregate 5000000 ops on 1000000 String keys: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
+----------------------------------------------------------------------------------------------------------------------------
+OpenHashMap changeValue 858 885 28 5.8 171.7 1.0X
+java.util.HashMap merge 772 811 60 6.5 154.5 1.1X
+
+OpenJDK 64-Bit Server VM 21.0.12+8-LTS on Linux 6.17.0-1020-azure
+AMD EPYC 7763 64-Core Processor
+Look up 1000000 String keys in random order: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
+---------------------------------------------------------------------------------------------------------------------------
+OpenHashMap 101 106 3 9.9 100.9 1.0X
+java.util.HashMap 48 53 3 20.7 48.3 2.1X
+
+
diff --git a/core/benchmarks/OpenHashMapBenchmark-jdk25-results.txt b/core/benchmarks/OpenHashMapBenchmark-jdk25-results.txt
new file mode 100644
index 000000000000..f21fd323a84f
--- /dev/null
+++ b/core/benchmarks/OpenHashMapBenchmark-jdk25-results.txt
@@ -0,0 +1,26 @@
+================================================================================================
+OpenHashMap vs java.util.HashMap
+================================================================================================
+
+OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure
+AMD EPYC 7763 64-Core Processor
+Insert 1000000 distinct String keys: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
+------------------------------------------------------------------------------------------------------------------------
+OpenHashMap 326 345 19 3.1 326.3 1.0X
+java.util.HashMap 50 96 67 19.8 50.5 6.5X
+
+OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure
+AMD EPYC 7763 64-Core Processor
+Aggregate 5000000 ops on 1000000 String keys: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
+----------------------------------------------------------------------------------------------------------------------------
+OpenHashMap changeValue 1103 1208 148 4.5 220.6 1.0X
+java.util.HashMap merge 1273 1286 19 3.9 254.5 0.9X
+
+OpenJDK 64-Bit Server VM 25.0.4+7-LTS on Linux 6.17.0-1020-azure
+AMD EPYC 7763 64-Core Processor
+Look up 1000000 String keys in random order: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
+---------------------------------------------------------------------------------------------------------------------------
+OpenHashMap 120 136 15 8.3 120.0 1.0X
+java.util.HashMap 58 64 3 17.1 58.4 2.1X
+
+
diff --git a/core/benchmarks/OpenHashMapBenchmark-results.txt b/core/benchmarks/OpenHashMapBenchmark-results.txt
new file mode 100644
index 000000000000..6bdd08f4ca19
--- /dev/null
+++ b/core/benchmarks/OpenHashMapBenchmark-results.txt
@@ -0,0 +1,26 @@
+================================================================================================
+OpenHashMap vs java.util.HashMap
+================================================================================================
+
+OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure
+AMD EPYC 7763 64-Core Processor
+Insert 1000000 distinct String keys: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
+------------------------------------------------------------------------------------------------------------------------
+OpenHashMap 228 243 13 4.4 227.9 1.0X
+java.util.HashMap 50 88 53 20.1 49.8 4.6X
+
+OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure
+AMD EPYC 7763 64-Core Processor
+Aggregate 5000000 ops on 1000000 String keys: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
+----------------------------------------------------------------------------------------------------------------------------
+OpenHashMap changeValue 772 778 7 6.5 154.3 1.0X
+java.util.HashMap merge 740 763 37 6.8 148.1 1.0X
+
+OpenJDK 64-Bit Server VM 17.0.20+8-LTS on Linux 6.17.0-1020-azure
+AMD EPYC 7763 64-Core Processor
+Look up 1000000 String keys in random order: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
+---------------------------------------------------------------------------------------------------------------------------
+OpenHashMap 109 111 2 9.2 108.5 1.0X
+java.util.HashMap 52 54 1 19.1 52.3 2.1X
+
+
diff --git a/core/src/test/scala/org/apache/spark/util/collection/OpenHashMapBenchmark.scala b/core/src/test/scala/org/apache/spark/util/collection/OpenHashMapBenchmark.scala
new file mode 100644
index 000000000000..4aa5ee8b4b84
--- /dev/null
+++ b/core/src/test/scala/org/apache/spark/util/collection/OpenHashMapBenchmark.scala
@@ -0,0 +1,133 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.util.collection
+
+import java.util.{HashMap => JHashMap, Random}
+
+import org.apache.spark.benchmark.{Benchmark, BenchmarkBase}
+
+/**
+ * Benchmark for OpenHashMap vs java.util.HashMap.
+ * Measures insert, aggregate (changeValue/merge), and random-order lookup performance
+ * with String keys and Long values.
+ * {{{
+ * To run this benchmark:
+ * 1. without sbt: bin/spark-submit --class <this class> <spark core test jar>
+ * 2. build/sbt "core/Test/runMain <this class>"
+ * 3. generate result: SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "core/Test/runMain <this class>"
+ * Results will be written to "benchmarks/OpenHashMapBenchmark-results.txt".
+ * }}}
+ */
+object OpenHashMapBenchmark extends BenchmarkBase {
+
+ private val numKeys = 1000000
+ private val numAggOps = 5000000
+
+ override def runBenchmarkSuite(mainArgs: Array[String]): Unit = {
+ runBenchmark("OpenHashMap vs java.util.HashMap") {
+ insertBenchmark()
+ aggregateBenchmark()
+ lookupBenchmark()
+ }
+ }
+
+ private def stringKeys: Array[String] = Array.tabulate(numKeys)(i => "key_" + i)
+
+ private def insertBenchmark(): Unit = {
+ val keys = stringKeys
+ val benchmark = new Benchmark(s"Insert $numKeys distinct String keys", numKeys,
+ output = output)
+ benchmark.addCase("OpenHashMap") { _ =>
+ val map = new OpenHashMap[String, Long](64)
+ var i = 0
+ while (i < numKeys) {
+ map.update(keys(i), i.toLong)
+ i += 1
+ }
+ }
+ benchmark.addCase("java.util.HashMap") { _ =>
+ val map = new JHashMap[String, java.lang.Long](16)
+ var i = 0
+ while (i < numKeys) {
+ map.put(keys(i), i.toLong)
+ i += 1
+ }
+ }
+ benchmark.run()
+ }
+
+ private def aggregateBenchmark(): Unit = {
+ val keys = stringKeys
+ val random = new Random(42)
+ val aggIndices = Array.fill(numAggOps)(random.nextInt(numKeys))
+ val benchmark = new Benchmark(s"Aggregate $numAggOps ops on $numKeys String keys", numAggOps,
+ output = output)
+ benchmark.addCase("OpenHashMap changeValue") { _ =>
+ val map = new OpenHashMap[String, Long](64)
+ var i = 0
+ while (i < numAggOps) {
+ map.changeValue(keys(aggIndices(i)), 1L, _ + 1L)
+ i += 1
+ }
+ }
+ benchmark.addCase("java.util.HashMap merge") { _ =>
+ val map = new JHashMap[String, java.lang.Long](16)
+ var i = 0
+ while (i < numAggOps) {
+ map.merge(keys(aggIndices(i)), 1L, (a, b) => a + b)
+ i += 1
+ }
+ }
+ benchmark.run()
+ }
+
+ private def lookupBenchmark(): Unit = {
+ val keys = stringKeys
+ val random = new Random(42)
+ // Look up in random order so that neither map benefits from the memory layout produced by
+ // sequential insertion.
+ val lookupIndices = Array.fill(numKeys)(random.nextInt(numKeys))
+ val openHashMap = new OpenHashMap[String, Long](64)
+ val jHashMap = new JHashMap[String, java.lang.Long](16)
+ var i = 0
+ while (i < numKeys) {
+ openHashMap.update(keys(i), i.toLong)
+ jHashMap.put(keys(i), i.toLong)
+ i += 1
+ }
+ val benchmark = new Benchmark(s"Look up $numKeys String keys in random order", numKeys,
+ output = output)
+ benchmark.addCase("OpenHashMap") { _ =>
+ var sum = 0L
+ var i = 0
+ while (i < numKeys) {
+ sum += openHashMap(keys(lookupIndices(i)))
+ i += 1
+ }
+ }
+ benchmark.addCase("java.util.HashMap") { _ =>
+ var sum = 0L
+ var i = 0
+ while (i < numKeys) {
+ sum += jHashMap.get(keys(lookupIndices(i)))
+ i += 1
+ }
+ }
+ benchmark.run()
+ }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org