(spark) branch branch-4.3 updated: [SPARK-58947][CORE] Handle Char in createArray and add tests for SparkCollectionUtils

[ur...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

uros-b pushed a commit to branch branch-4.3
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.3 by this push:
     new 4efb4787564e [SPARK-58947][CORE] Handle Char in createArray and add tests for SparkCollectionUtils
4efb4787564e is described below

commit 4efb4787564e6941a6c2a76179c5bfd4d4f5d1c2
Author: Uros <22...@users.noreply.github.com>
AuthorDate: Tue Aug 25 15:47:49 2026 +0000

    [SPARK-58947][CORE] Handle Char in createArray and add tests for SparkCollectionUtils
    
    ### What changes were proposed in this pull request?
    
    Adds a unit test suite for `SparkCollectionUtils` (`common/utils`, `org.apache.spark.util`), which had no test coverage, and fixes a gap the tests surfaced.
    
    Tests cover all four methods: `toMapWithIndex` (empty input, duplicate keys, equivalence to `zipWithIndex.toMap`); `isEmpty` / `isNotEmpty` (null, empty, non-empty `java.util.Map`); and `createArray` for every primitive type plus reference types and `size == 0`.
    
    Adding the `createArray` coverage exposed a latent bug: it dispatched on 7 of the 8 primitive `classOf` types (Boolean/Byte/Short/Int/Long/Float/Double) but not `Char`, so a `Char` argument fell through to the `Array[AnyRef]` cast and threw `ClassCastException` (a `char[]` is not an `Object[]`). This adds the missing `Char` case so `createArray[Char]` fills the array like the other primitives.
    
    ### Why are the changes needed?
    
    `SparkCollectionUtils` is used in production (e.g. `StructType` field indexing via `toMapWithIndex`) but had no tests. The `Char` gap is currently unreachable (no caller passes `Char`), but it is a latent crash for any future `Char` caller; adding the case makes the dispatch complete and symmetric with the other primitives.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No. `SparkCollectionUtils` is `private[spark]`; this adds tests and a missing internal `Char` case.
    
    ### How was this patch tested?
    
    New `SparkCollectionUtilsSuite` (extends `AnyFunSuite`, the base used by the sibling `common/utils` suites), covering all four methods including `createArray` for all eight primitive types, reference types, and `size == 0`.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Code (Opus 4.8)
    
    Closes #58227 from uros-b/test-sparkcollectionutils.
    
    Authored-by: Uros <22...@users.noreply.github.com>
    Signed-off-by: Uros Bojanic <22...@users.noreply.github.com>
    (cherry picked from commit 0fb568c220dc1e1f797757a9392024e5244f346f)
    Signed-off-by: Uros Bojanic <22...@users.noreply.github.com>
---
 .../apache/spark/util/SparkCollectionUtils.scala   |  2 +
 .../spark/util/SparkCollectionUtilsSuite.scala     | 64 ++++++++++++++++++++++
 2 files changed, 66 insertions(+)

diff --git a/common/utils/src/main/scala/org/apache/spark/util/SparkCollectionUtils.scala b/common/utils/src/main/scala/org/apache/spark/util/SparkCollectionUtils.scala
index 9c255daa0522..956d9c8bc9e3 100644
--- a/common/utils/src/main/scala/org/apache/spark/util/SparkCollectionUtils.scala
+++ b/common/utils/src/main/scala/org/apache/spark/util/SparkCollectionUtils.scala
@@ -52,6 +52,8 @@ private[spark] trait SparkCollectionUtils {
         Arrays.fill(arr.asInstanceOf[Array[Byte]], defaultValue.asInstanceOf[Byte])
       case c if c == classOf[Short] =>
         Arrays.fill(arr.asInstanceOf[Array[Short]], defaultValue.asInstanceOf[Short])
+      case c if c == classOf[Char] =>
+        Arrays.fill(arr.asInstanceOf[Array[Char]], defaultValue.asInstanceOf[Char])
       case c if c == classOf[Int] =>
         Arrays.fill(arr.asInstanceOf[Array[Int]], defaultValue.asInstanceOf[Int])
       case c if c == classOf[Long] =>
diff --git a/common/utils/src/test/scala/org/apache/spark/util/SparkCollectionUtilsSuite.scala b/common/utils/src/test/scala/org/apache/spark/util/SparkCollectionUtilsSuite.scala
new file mode 100644
index 000000000000..8040587e873a
--- /dev/null
+++ b/common/utils/src/test/scala/org/apache/spark/util/SparkCollectionUtilsSuite.scala
@@ -0,0 +1,64 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.util
+
+import org.scalatest.funsuite.AnyFunSuite // scalastyle:ignore funsuite
+
+class SparkCollectionUtilsSuite extends AnyFunSuite { // scalastyle:ignore funsuite
+
+  test("toMapWithIndex maps each key to its position, matching zipWithIndex.toMap") {
+    assert(SparkCollectionUtils.toMapWithIndex(Seq("a", "b", "c")) ===
+      Map("a" -> 0, "b" -> 1, "c" -> 2))
+    assert(SparkCollectionUtils.toMapWithIndex(Seq.empty[String]) === Map.empty[String, Int])
+    // Duplicate keys keep the last index, matching zipWithIndex.toMap.
+    assert(SparkCollectionUtils.toMapWithIndex(Seq("a", "a")) === Map("a" -> 1))
+    val keys = Seq(10, 20, 30, 40)
+    assert(SparkCollectionUtils.toMapWithIndex(keys) === keys.zipWithIndex.toMap)
+  }
+
+  test("isEmpty and isNotEmpty handle null, empty and non-empty maps") {
+    val nullMap: java.util.Map[String, Int] = null
+    assert(SparkCollectionUtils.isEmpty(nullMap))
+    assert(!SparkCollectionUtils.isNotEmpty(nullMap))
+
+    val empty = new java.util.HashMap[String, Int]()
+    assert(SparkCollectionUtils.isEmpty(empty))
+    assert(!SparkCollectionUtils.isNotEmpty(empty))
+
+    val nonEmpty = new java.util.HashMap[String, Int]()
+    nonEmpty.put("a", 1)
+    assert(!SparkCollectionUtils.isEmpty(nonEmpty))
+    assert(SparkCollectionUtils.isNotEmpty(nonEmpty))
+  }
+
+  test("createArray fills primitive-typed arrays with the default value") {
+    assert(SparkCollectionUtils.createArray(3, 7) === Array(7, 7, 7))
+    assert(SparkCollectionUtils.createArray(2, true) === Array(true, true))
+    assert(SparkCollectionUtils.createArray(2, 1L) === Array(1L, 1L))
+    assert(SparkCollectionUtils.createArray(2, 2.5d) === Array(2.5d, 2.5d))
+    assert(SparkCollectionUtils.createArray(2, 1.5f) === Array(1.5f, 1.5f))
+    assert(SparkCollectionUtils.createArray(2, 3.toByte) === Array(3.toByte, 3.toByte))
+    assert(SparkCollectionUtils.createArray(2, 4.toShort) === Array(4.toShort, 4.toShort))
+    assert(SparkCollectionUtils.createArray(2, 'a') === Array('a', 'a'))
+    assert(SparkCollectionUtils.createArray(0, 7).isEmpty)
+  }
+
+  test("createArray fills reference-typed arrays and returns empty for size 0") {
+    assert(SparkCollectionUtils.createArray(2, "x") === Array("x", "x"))
+    assert(SparkCollectionUtils.createArray(0, "x").isEmpty)
+  }
+}


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org