You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2026/05/14 03:20:16 UTC
[PR] [PS][TESTS] Fix `ArrowInterfaceTests` being silently skipped by unittest [spark]
zhengruifeng opened a new pull request, #55870:
URL: https://github.com/apache/spark/pull/55870
### What changes were proposed in this pull request?
Fix the base class of `ArrowInterfaceTests` in `python/pyspark/pandas/tests/test_arrow_interface.py` so the test is actually discovered by `unittest`.
The class was declared as:
```python
class ArrowInterfaceTests(ArrowInterfaceTestsMixin, SQLTestUtils):
pass
```
`SQLTestUtils` is a plain mixin that does not extend `unittest.TestCase`, so this class itself is not a `TestCase` subclass and `unittest`'s discovery (`TestLoader.loadTestsFromModule`) silently skips it.
Verified with `loader.loadTestsFromModule(...).countTestCases()`:
| | tests collected | `issubclass(TestCase)` |
| --- | --- | --- |
| Before fix | 0 | `False` |
| After fix | 3 | `True` |
The sibling Spark Connect parity test `ArrowInterfaceParityTests` in `tests/connect/test_parity_arrow_interface.py` extends `ReusedConnectTestCase` (which does chain to `TestCase`) and so was unaffected, masking the issue.
Fix: switch the base to `PandasOnSparkTestCase`, matching the other `pyspark.pandas.tests.test_*` classes (e.g. `NamespaceTests`, `UtilsTests`, `InternalFrameTests`, `SQLTests`). `PandasOnSparkTestCase` chains through `ReusedSQLTestCase -> ReusedPySparkTestCase -> PySparkBaseTestCase -> unittest.TestCase`.
### Why are the changes needed?
`test_spark_arrow_c_streamer_arrow_consumer` (added in SPARK-55124) has not actually been running in CI since it was introduced; it is a latent gap in coverage of the pandas-on-Spark Arrow C-stream interface.
### Does this PR introduce _any_ user-facing change?
No. Test-only change.
### How was this patch tested?
- Discovery check (above): `unittest.TestLoader.loadTestsFromModule` now collects the test.
- Ran the test locally:
```
$ python -m unittest -v pyspark.pandas.tests.test_arrow_interface
test_assert_classic_mode (...ArrowInterfaceTests) ... ok
test_spark_arrow_c_streamer_arrow_consumer (...ArrowInterfaceTests) ... ok
test_assert_classic_mode (...PandasOnSparkTestCase) ... ok
----------------------------------------------------------------------
Ran 3 tests in 10.683s
OK
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude opus-4-7
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-56861][PS][TESTS] Fix `ArrowInterfaceTests` being silently skipped by unittest [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #55870:
URL: https://github.com/apache/spark/pull/55870#issuecomment-4448815983
merged to master/4.x
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-56861][PS][TESTS] Fix `ArrowInterfaceTests` being silently skipped by unittest [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng closed pull request #55870: [SPARK-56861][PS][TESTS] Fix `ArrowInterfaceTests` being silently skipped by unittest
URL: https://github.com/apache/spark/pull/55870
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org