You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2025/06/09 10:09:54 UTC
(spark) branch master updated: [SPARK-52420][PYTHON][TESTS] Make test_udtf_with_invalid_return_type compatible with Python only client
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new c75e7c78b2d1 [SPARK-52420][PYTHON][TESTS] Make test_udtf_with_invalid_return_type compatible with Python only client
c75e7c78b2d1 is described below
commit c75e7c78b2d15b741698ce2c2f98e3c7e877ab91
Author: Hyukjin Kwon <gu...@apache.org>
AuthorDate: Mon Jun 9 03:09:42 2025 -0700
[SPARK-52420][PYTHON][TESTS] Make test_udtf_with_invalid_return_type compatible with Python only client
### What changes were proposed in this pull request?
This PR is a followup of https://github.com/apache/spark/pull/50997 that backports the test changes into branch-3.5.
### Why are the changes needed?
To make the build pass (https://github.com/apache/spark/actions/runs/15522509263/job/43697654408).
```
======================================================================
FAIL [0.015s]: test_udtf_with_invalid_return_type (pyspark.sql.tests.connect.test_parity_udtf.UDTFParityTests.test_udtf_with_invalid_return_type)
----------------------------------------------------------------------
pyspark.errors.exceptions.connect.SparkConnectGrpcException: (org.apache.spark.sql.connect.common.InvalidPlanInput) [INVALID_SCHEMA_TYPE_NON_STRUCT] Invalid schema type. Expect a struct type, but got "INT". SQLSTATE: 42K09
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/runner/work/spark/spark-3.5/python/pyspark/sql/tests/connect/test_parity_udtf.py", line 57, in test_udtf_with_invalid_return_type
with self.assertRaisesRegex(
AssertionError: "Invalid Python user-defined table function return type." does not match "(org.apache.spark.sql.connect.common.InvalidPlanInput) [INVALID_SCHEMA_TYPE_NON_STRUCT] Invalid schema type. Expect a struct type, but got "INT". SQLSTATE: 42K09"
----------------------------------------------------------------------
```
Now that the error message has changed, the tests in branch-3.5 fails.
### Does this PR introduce _any_ user-facing change?
No, test-only.
### How was this patch tested?
Manually.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #51121 from HyukjinKwon/SPARK-52420.
Authored-by: Hyukjin Kwon <gu...@apache.org>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
python/pyspark/sql/tests/connect/test_parity_udtf.py | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/python/pyspark/sql/tests/connect/test_parity_udtf.py b/python/pyspark/sql/tests/connect/test_parity_udtf.py
index 9ad65f22707f..208f9ae53898 100644
--- a/python/pyspark/sql/tests/connect/test_parity_udtf.py
+++ b/python/pyspark/sql/tests/connect/test_parity_udtf.py
@@ -60,9 +60,7 @@ class UDTFParityTests(BaseUDTFTestsMixin, ReusedConnectTestCase):
def eval(self, a: int):
yield a + 1,
- with self.assertRaisesRegex(
- InvalidPlanInput, "Invalid schema type. Expect a struct type, but got"
- ):
+ with self.assertRaisesRegex(InvalidPlanInput, "Invalid.*type"):
TestUDTF(lit(1)).collect()
@unittest.skip("Spark Connect does not support broadcast but the test depends on it.")
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org