(spark) branch branch-3.5 updated: [SPARK-48843] Prevent infinite loop with BindParameters

[we...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

wenchen pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.5 by this push:
     new 1e15e3f3eb2b [SPARK-48843] Prevent infinite loop with BindParameters
1e15e3f3eb2b is described below

commit 1e15e3f3eb2bd92f3a3c5365a99afea10c392084
Author: Nemanja Boric <ne...@databricks.com>
AuthorDate: Wed Jul 10 20:39:00 2024 +0800

    [SPARK-48843] Prevent infinite loop with BindParameters
    
    ### What changes were proposed in this pull request?
    
    In order to resolve the named parameters on the subtree, BindParameters recurses into the subtrees and tries to match the pattern with the named parameters. If there's no named parameter in the current level, the rule tries to return the unchanged plan. However, instead of returning the current plan object, the rule always returns the captured root plan node, leading into the infinite recursion.
    
    ### Why are the changes needed?
    
    Infinite recursion with the named parameters and the global limit.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Added unit tests.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No.
    
    Closes #47271 from nemanja-boric-databricks/fix-bind.
    
    Lead-authored-by: Nemanja Boric <ne...@databricks.com>
    Co-authored-by: Wenchen Fan <cl...@gmail.com>
    Signed-off-by: Wenchen Fan <we...@databricks.com>
    (cherry picked from commit a39f70d0c8e85c9911d9a15445fd2a136a66ae4b)
    Signed-off-by: Wenchen Fan <we...@databricks.com>
---
 python/pyspark/sql/tests/connect/test_connect_basic.py             | 7 +++++++
 .../scala/org/apache/spark/sql/catalyst/analysis/parameters.scala  | 2 +-
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/python/pyspark/sql/tests/connect/test_connect_basic.py b/python/pyspark/sql/tests/connect/test_connect_basic.py
index 48e5248e28f5..d1f6994edba7 100644
--- a/python/pyspark/sql/tests/connect/test_connect_basic.py
+++ b/python/pyspark/sql/tests/connect/test_connect_basic.py
@@ -1242,6 +1242,13 @@ class SparkConnectBasicTests(SparkConnectSQLTestCase):
         df2 = self.spark.sql("SELECT * FROM range(10) WHERE id > :minId", args={"minId": 7})
         self.assert_eq(df.toPandas(), df2.toPandas())
 
+    def test_namedargs_with_global_limit(self):
+        sqlText = """SELECT * FROM VALUES (TIMESTAMP('2022-12-25 10:30:00'), 1) as tab(date, val)
+         where val = :val"""
+        df = self.connect.sql(sqlText, args={"val": 1})
+        df2 = self.spark.sql(sqlText, args={"val": 1})
+        self.assert_eq(df.toPandas(), df2.toPandas())
+
     def test_sql_with_pos_args(self):
         df = self.connect.sql("SELECT * FROM range(10) WHERE id > ?", args=[7])
         df2 = self.spark.sql("SELECT * FROM range(10) WHERE id > ?", args=[7])
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/parameters.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/parameters.scala
index 2e3cabce24a4..20cc9a2ab545 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/parameters.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/parameters.scala
@@ -136,7 +136,7 @@ object BindParameters extends Rule[LogicalPlan] with QueryErrorsBase {
             args(posToIndex(pos))
         }
 
-      case _ => plan
+      case other => other
     }
   }
 }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org