(spark) branch master updated: [SPARK-56861][PS][TESTS] Fix `ArrowInterfaceTests` being silently skipped by unittest

[ru...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

zhengruifeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 9ab8e43a940e [SPARK-56861][PS][TESTS] Fix `ArrowInterfaceTests` being silently skipped by unittest
9ab8e43a940e is described below

commit 9ab8e43a940ebe6adb9cf32336b59c6aff105eff
Author: Ruifeng Zheng <ru...@apache.org>
AuthorDate: Thu May 14 15:56:05 2026 +0800

    [SPARK-56861][PS][TESTS] Fix `ArrowInterfaceTests` being silently skipped by unittest
    
    ### What changes were proposed in this pull request?
    
    Fix the base class of `ArrowInterfaceTests` in `python/pyspark/pandas/tests/test_arrow_interface.py` so the test is actually discovered by `unittest`.
    
    The class was declared as:
    
    ```python
    class ArrowInterfaceTests(ArrowInterfaceTestsMixin, SQLTestUtils):
        pass
    ```
    
    `SQLTestUtils` is a plain mixin that does not extend `unittest.TestCase`, so this class itself is not a `TestCase` subclass and `unittest`'s discovery (`TestLoader.loadTestsFromModule`) silently skips it.
    
    Verified with `loader.loadTestsFromModule(...).countTestCases()`:
    
    | | tests collected | `issubclass(TestCase)` |
    | --- | --- | --- |
    | Before fix | 0 | `False` |
    | After fix  | 3 | `True`  |
    
    The sibling Spark Connect parity test `ArrowInterfaceParityTests` in `tests/connect/test_parity_arrow_interface.py` extends `ReusedConnectTestCase` (which does chain to `TestCase`) and so was unaffected, masking the issue.
    
    Fix: switch the base to `PandasOnSparkTestCase`, matching the other `pyspark.pandas.tests.test_*` classes (e.g. `NamespaceTests`, `UtilsTests`, `InternalFrameTests`, `SQLTests`). `PandasOnSparkTestCase` chains through `ReusedSQLTestCase -> ReusedPySparkTestCase -> PySparkBaseTestCase -> unittest.TestCase`.
    
    ### Why are the changes needed?
    
    `test_spark_arrow_c_streamer_arrow_consumer` (added in SPARK-55124) has not actually been running in CI since it was introduced; it is a latent gap in coverage of the pandas-on-Spark Arrow C-stream interface.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No. Test-only change.
    
    ### How was this patch tested?
    
    - Discovery check (above): `unittest.TestLoader.loadTestsFromModule` now collects the test.
    - Ran the test locally:
    
    ```
    $ python -m unittest -v pyspark.pandas.tests.test_arrow_interface
    test_assert_classic_mode (...ArrowInterfaceTests) ... ok
    test_spark_arrow_c_streamer_arrow_consumer (...ArrowInterfaceTests) ... ok
    test_assert_classic_mode (...PandasOnSparkTestCase) ... ok
    ----------------------------------------------------------------------
    Ran 3 tests in 10.683s
    OK
    ```
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude opus-4-7
    
    Closes #55870 from zhengruifeng/SPARK-arrow-interface-tests.
    
    Authored-by: Ruifeng Zheng <ru...@apache.org>
    Signed-off-by: Ruifeng Zheng <ru...@apache.org>
---
 python/pyspark/pandas/tests/test_arrow_interface.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/python/pyspark/pandas/tests/test_arrow_interface.py b/python/pyspark/pandas/tests/test_arrow_interface.py
index d38462dbe9e1..304c38c756c4 100644
--- a/python/pyspark/pandas/tests/test_arrow_interface.py
+++ b/python/pyspark/pandas/tests/test_arrow_interface.py
@@ -21,7 +21,7 @@ from pyspark.testing.utils import (
     pyarrow_requirement_message,
 )
 from pyspark import pandas as ps
-from pyspark.testing.sqlutils import SQLTestUtils
+from pyspark.testing.pandasutils import PandasOnSparkTestCase
 
 import pandas as pd
 
@@ -60,7 +60,7 @@ class ArrowInterfaceTestsMixin:
         self.assertEqual(result, expected)
 
 
-class ArrowInterfaceTests(ArrowInterfaceTestsMixin, SQLTestUtils):
+class ArrowInterfaceTests(ArrowInterfaceTestsMixin, PandasOnSparkTestCase):
     pass
 
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org