(spark) branch master updated: [SPARK-56614][SQL][CONNECT][TESTS][FOLLOWUP] Pin strictDataFrameColumnResolution=true for lazy column validation test

[ru...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

zhengruifeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 661afd4d8a17 [SPARK-56614][SQL][CONNECT][TESTS][FOLLOWUP] Pin strictDataFrameColumnResolution=true for lazy column validation test
661afd4d8a17 is described below

commit 661afd4d8a17175597aa7bfb5cf721fa62de55e0
Author: Ruifeng Zheng <ru...@apache.org>
AuthorDate: Wed Apr 29 18:20:12 2026 +0800

    [SPARK-56614][SQL][CONNECT][TESTS][FOLLOWUP] Pin strictDataFrameColumnResolution=true for lazy column validation test
    
    ### What changes were proposed in this pull request?
    Pin `spark.sql.analyzer.strictDataFrameColumnResolution=true` around the body of the `lazy column validation` test in `DataFrameSuite`. The config is set via `spark.conf.set/unset` rather than `withSQLConf` because the lazy SQLConf entry trips `withSQLConf`'s `isModifiable` check on the Connect server.
    
    ### Why are the changes needed?
    The test asserts that `df4.schema` throws `AnalysisException` for `df1("x")` when `df1` does not contain `x`. This holds only under strict plan-id-based resolution; if the name-based fallback path is enabled, `df1("x")` resolves to `df2.x` from the join output and `df4.schema` succeeds. Today this works because `STRICT_DATAFRAME_COLUMN_RESOLUTION` defaults to `true`, but the test should not silently rely on that default; pinning it makes the assumption explicit and keeps the test robu [...]
    
    ### Does this PR introduce _any_ user-facing change?
    No. Test-only change.
    
    ### How was this patch tested?
    Existing `DataFrameSuite."lazy column validation"`.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    Generated-by: Claude Code (Anthropic), claude-opus-4-7
    
    Closes #55604 from zhengruifeng/SPARK-strict-df-col-resolution-test-pin.
    
    Authored-by: Ruifeng Zheng <ru...@apache.org>
    Signed-off-by: Ruifeng Zheng <ru...@foxmail.com>
---
 .../apache/spark/sql/connect/DataFrameSuite.scala  | 28 +++++++++++++++-------
 1 file changed, 19 insertions(+), 9 deletions(-)

diff --git a/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/DataFrameSuite.scala b/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/DataFrameSuite.scala
index 890245fdd2fb..57b8080c4b13 100644
--- a/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/DataFrameSuite.scala
+++ b/sql/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/DataFrameSuite.scala
@@ -68,14 +68,24 @@ class DataFrameSuite extends QueryTest with RemoteSparkSession {
   }
 
   test("lazy column validation") {
-    val session = spark
-    import session.implicits._
-
-    val df1 = Seq(1 -> "y").toDF("a", "y")
-    val df2 = Seq(1 -> "x").toDF("a", "x")
-    val df3 = df1.join(df2, df1("a") === df2("a"))
-    val df4 = df3.select(df1("x")) // <- No exception here
-
-    intercept[AnalysisException] { df4.schema }
+    // The test relies on strict plan-id-based resolution: with the name-based fallback
+    // enabled, df1("x") would resolve to df2.x via the join output and df4.schema would
+    // succeed. Pin the config directly via spark.conf.set/unset; the lazy SQLConf entry
+    // trips withSQLConf's isModifiable check on the Connect server, so we cannot use that
+    // helper here.
+    spark.conf.set("spark.sql.analyzer.strictDataFrameColumnResolution", "true")
+    try {
+      val session = spark
+      import session.implicits._
+
+      val df1 = Seq(1 -> "y").toDF("a", "y")
+      val df2 = Seq(1 -> "x").toDF("a", "x")
+      val df3 = df1.join(df2, df1("a") === df2("a"))
+      val df4 = df3.select(df1("x")) // <- No exception here
+
+      intercept[AnalysisException] { df4.schema }
+    } finally {
+      spark.conf.unset("spark.sql.analyzer.strictDataFrameColumnResolution")
+    }
   }
 }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org