(spark) branch master updated: [SPARK-59273][SQL][TEST][FOLLOWUP] Cover string fill with mixed schemas

[ur...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

uros-b pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new e71d3c11464d [SPARK-59273][SQL][TEST][FOLLOWUP] Cover string fill with mixed schemas
e71d3c11464d is described below

commit e71d3c11464d17b61dd220af7d5e8404589c14ba
Author: Xinyao Zhang <43...@users.noreply.github.com>
AuthorDate: Tue Sep 22 18:40:19 2026 +0000

    [SPARK-59273][SQL][TEST][FOLLOWUP] Cover string fill with mixed schemas
    
    ### What changes were proposed in this pull request?
    
    Extend the existing [SPARK-59273](https://issues.apache.org/jira/browse/SPARK-59273) CHAR/VARCHAR `na.fill("x")` test with an integer column.
    
    The test verifies that string fill values still fill CHAR and VARCHAR columns, while a non-string column is left unchanged.
    
    ### Why are the changes needed?
    
    SPARK-59273 broadened string matching to include the CHAR/VARCHAR family. The follow-up fix in #58646 restored skipping non-string columns.
    
    The existing tests cover those behaviors separately. This test covers the mixed-schema case directly, so a future change cannot reintroduce the interaction that caused the regression.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    The targeted test was started locally, but its first-run SQL test compilation was interrupted before execution completed. GitHub Actions will run the full suite.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Codex (GPT-5)
    
    Closes #58682 from zhangxinyao88/codex/SPARK-59273-mixed-schema-fill-test.
    
    Authored-by: Xinyao Zhang <43...@users.noreply.github.com>
    Signed-off-by: Uros Bojanic <22...@users.noreply.github.com>
---
 .../apache/spark/sql/DataFrameNaFunctionsSuite.scala    | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/sql/core/src/test/scala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala
index 43e2673bdd4a..3dcbe4e60a5b 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala
@@ -22,7 +22,14 @@ import scala.jdk.CollectionConverters._
 import org.apache.spark.SparkUnsupportedOperationException
 import org.apache.spark.sql.internal.SQLConf
 import org.apache.spark.sql.test.SharedSparkSession
-import org.apache.spark.sql.types.{CharType, StringType, StructField, StructType, VarcharType}
+import org.apache.spark.sql.types.{
+  CharType,
+  IntegerType,
+  StringType,
+  StructField,
+  StructType,
+  VarcharType
+}
 
 class DataFrameNaFunctionsSuite extends SharedSparkSession {
   import testImplicits._
@@ -219,10 +226,12 @@ class DataFrameNaFunctionsSuite extends SharedSparkSession {
     withSQLConf(SQLConf.CHAR_VARCHAR_STANDARD_SEMANTICS.key -> "true") {
       val schema = StructType(Seq(
         StructField("c", CharType(3)),
-        StructField("v", VarcharType(3))))
-      val input = spark.createDataFrame(sparkContext.parallelize(Seq(Row(null, null))), schema)
+        StructField("v", VarcharType(3)),
+        StructField("i", IntegerType)))
+      val input = spark.createDataFrame(
+        sparkContext.parallelize(Seq(Row(null, null, null))), schema)
 
-      checkAnswer(input.na.fill("x"), Row("x  ", "x"))
+      checkAnswer(input.na.fill("x"), Row("x  ", "x", null))
     }
   }
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org