(spark) branch master updated: [SPARK-50785][SQL][FOLLOWUP] Remove overrideIfExists check in SqlScriptingLocalVariableManager

[we...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

wenchen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 20c60d982de0 [SPARK-50785][SQL][FOLLOWUP] Remove overrideIfExists check in SqlScriptingLocalVariableManager
20c60d982de0 is described below

commit 20c60d982de0ca42259d9a6d76ec0c25f48dfb69
Author: Dušan Tišma <du...@databricks.com>
AuthorDate: Thu Sep 4 08:34:28 2025 +0800

    [SPARK-50785][SQL][FOLLOWUP] Remove overrideIfExists check in SqlScriptingLocalVariableManager
    
    ### What changes were proposed in this pull request?
    
    Even though DECLARE OR REPLACE is not allowed for local variables, we supported this in the execution code because `ForStatementExec` internally used this. Now that it is properly implemented, we should respect the specification everywhere.
    
    ### Why are the changes needed?
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Existing tests
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No.
    
    Closes #52221 from dusantism-db/create-variable-cleanup.
    
    Authored-by: Dušan Tišma <du...@databricks.com>
    Signed-off-by: Wenchen Fan <we...@databricks.com>
---
 .../spark/sql/scripting/SqlScriptingLocalVariableManager.scala     | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingLocalVariableManager.scala b/sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingLocalVariableManager.scala
index fb3560ddb9f3..c9a243db4f08 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingLocalVariableManager.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/scripting/SqlScriptingLocalVariableManager.scala
@@ -37,11 +37,8 @@ class SqlScriptingLocalVariableManager(context: SqlScriptingExecutionContext)
       overrideIfExists: Boolean): Unit = {
     val name = nameParts.last
 
-    // overrideIfExists should not be supported because local variables don't support
-    // DECLARE OR REPLACE. However ForStatementExec currently uses this to handle local vars,
-    // so we support it for now.
-    // TODO [SPARK-50785]: Refactor ForStatementExec to use local variables properly.
-    if (!overrideIfExists && context.currentScope.variables.contains(name)) {
+    // Sanity check, this should already be thrown by CreateVariableExec.run
+    if (context.currentScope.variables.contains(name)) {
       throw new AnalysisException(
         errorClass = "VARIABLE_ALREADY_EXISTS",
         messageParameters = Map(


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org