(spark) branch master updated: [SPARK-46532][CONNECT][PYTHON][FOLLOW-UP] Pass message parameters in Python Spark Connect client

[gu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 2e918e8b5c11 [SPARK-46532][CONNECT][PYTHON][FOLLOW-UP] Pass message parameters in Python Spark Connect client
2e918e8b5c11 is described below

commit 2e918e8b5c1178c918a0ee81cde3a261ca6cb72c
Author: Hyukjin Kwon <gu...@apache.org>
AuthorDate: Fri Dec 29 16:30:51 2023 +0900

    [SPARK-46532][CONNECT][PYTHON][FOLLOW-UP] Pass message parameters in Python Spark Connect client
    
    ### What changes were proposed in this pull request?
    
    This PR is a followup of https://github.com/apache/spark/pull/44468 that addresses the additional metadata in Python Spark Connect client.
    
    ### Why are the changes needed?
    
    For feature parity.
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes, when `spark.sql.connect.enrichError.enabled` is disabled, users are sill able to get the message parameters.
    
    ### How was this patch tested?
    
    Unittest was added.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No.
    
    Closes #44528 from HyukjinKwon/SPARK-46532-followup.
    
    Authored-by: Hyukjin Kwon <gu...@apache.org>
    Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
 python/pyspark/errors/exceptions/connect.py            |  3 +++
 python/pyspark/sql/tests/connect/test_connect_basic.py | 11 +++++++++++
 2 files changed, 14 insertions(+)

diff --git a/python/pyspark/errors/exceptions/connect.py b/python/pyspark/errors/exceptions/connect.py
index ba172135cb64..0cffe7268753 100644
--- a/python/pyspark/errors/exceptions/connect.py
+++ b/python/pyspark/errors/exceptions/connect.py
@@ -69,6 +69,9 @@ def convert_exception(
     if "errorClass" in info.metadata:
         error_class = info.metadata["errorClass"]
 
+    if "messageParameters" in info.metadata:
+        message_parameters = json.loads(info.metadata["messageParameters"])
+
     stacktrace: Optional[str] = None
     if resp is not None and resp.HasField("root_error_idx"):
         message = resp.errors[resp.root_error_idx].message
diff --git a/python/pyspark/sql/tests/connect/test_connect_basic.py b/python/pyspark/sql/tests/connect/test_connect_basic.py
index 7275f40b39a9..045ba8f0060d 100755
--- a/python/pyspark/sql/tests/connect/test_connect_basic.py
+++ b/python/pyspark/sql/tests/connect/test_connect_basic.py
@@ -3486,6 +3486,17 @@ class SparkConnectSessionTests(ReusedConnectTestCase):
             PySparkSession.builder.create()
             self.assertIn("Create a new SparkSession is only supported with SparkConnect.", str(e))
 
+    def test_get_message_parameters_without_enriched_error(self):
+        with self.sql_conf({"spark.sql.connect.enrichError.enabled": False}):
+            exception = None
+            try:
+                self.spark.sql("""SELECT a""")
+            except AnalysisException as e:
+                exception = e
+
+            self.assertIsNotNone(exception)
+            self.assertEqual(exception.getMessageParameters(), {"objectName": "`a`"})
+
 
 class SparkConnectSessionWithOptionsTest(unittest.TestCase):
     def setUp(self) -> None:


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org