You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2025/12/01 09:47:59 UTC
(spark) branch branch-4.1 updated: [SPARK-54481][SPARK-54495][SPARK-54483][TESTS][PYTHON][CONNECT] Reenable a few spark connect tests
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new 2e4dedf6171a [SPARK-54481][SPARK-54495][SPARK-54483][TESTS][PYTHON][CONNECT] Reenable a few spark connect tests
2e4dedf6171a is described below
commit 2e4dedf6171ad6a429ced2c33394801e1b09e4e7
Author: xianzhe-databricks <xi...@databricks.com>
AuthorDate: Mon Dec 1 18:47:36 2025 +0900
[SPARK-54481][SPARK-54495][SPARK-54483][TESTS][PYTHON][CONNECT] Reenable a few spark connect tests
### What changes were proposed in this pull request?
* This PR re-enables 3 tests, which only fail during the cross-version scenario: Spark Connect 4.0 client <-> 4.2 Spark Server
* We fixed these tests on the 4.0 branch separately in another PR https://github.com/apache/spark/pull/53250
* ‼️ This PR needs to be backported to `branch-4.1`. How can I make it possible? HyukjinKwon
### Why are the changes needed?
To regain test coverage
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #53235 from xianzhe-databricks/enable-tests.
Authored-by: xianzhe-databricks <xi...@databricks.com>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
(cherry picked from commit 54cde812c2657717651156636041a84556619a5b)
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
python/pyspark/pandas/tests/test_typedef.py | 3 ---
python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py | 4 ----
python/pyspark/sql/tests/pandas/test_pandas_map.py | 3 ---
3 files changed, 10 deletions(-)
diff --git a/python/pyspark/pandas/tests/test_typedef.py b/python/pyspark/pandas/tests/test_typedef.py
index 29d1c170f065..54a03200c05c 100644
--- a/python/pyspark/pandas/tests/test_typedef.py
+++ b/python/pyspark/pandas/tests/test_typedef.py
@@ -311,9 +311,6 @@ class TypeHintTestsMixin:
self.assertRaisesRegex(TypeError, "object.*not understood", try_infer_return_type)
- @unittest.skipIf(
- os.environ.get("SPARK_SKIP_CONNECT_COMPAT_TESTS") == "1", "SPARK-54495: To be reenabled"
- )
def test_as_spark_type_pandas_on_spark_dtype(self):
type_mapper = {
# binary
diff --git a/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py b/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py
index 0a77334ff85e..0d91da035497 100644
--- a/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py
+++ b/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py
@@ -18,7 +18,6 @@
import unittest
import logging
from typing import cast
-import os
from pyspark.sql import functions as sf
from pyspark.sql.functions import pandas_udf, udf
@@ -244,9 +243,6 @@ class CogroupedApplyInPandasTestsMixin:
self._test_merge_empty(fn=merge_pandas)
- @unittest.skipIf(
- os.environ.get("SPARK_SKIP_CONNECT_COMPAT_TESTS") == "1", "SPARK-54481: To be reenabled"
- )
def test_apply_in_pandas_returning_incompatible_type(self):
with self.quiet():
self.check_apply_in_pandas_returning_incompatible_type()
diff --git a/python/pyspark/sql/tests/pandas/test_pandas_map.py b/python/pyspark/sql/tests/pandas/test_pandas_map.py
index f816b09fe5ae..946d56f2fe63 100644
--- a/python/pyspark/sql/tests/pandas/test_pandas_map.py
+++ b/python/pyspark/sql/tests/pandas/test_pandas_map.py
@@ -273,9 +273,6 @@ class MapInPandasTestsMixin:
actual = df.repartition(1).mapInPandas(f, "id long, value long").collect()
self.assertEqual(actual, expected)
- @unittest.skipIf(
- os.environ.get("SPARK_SKIP_CONNECT_COMPAT_TESTS") == "1", "SPARK-54483: To be reenabled"
- )
def test_dataframes_with_incompatible_types(self):
with self.quiet():
self.check_dataframes_with_incompatible_types()
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org