You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "Spenserrrr (via GitHub)" <gi...@apache.org> on 2026/07/24 20:06:05 UTC
[PR] [SPARK-58332][PYTHON][TEST] Move compare_or_generate_golden_matrix into GoldenFileTestMixin [spark]
Spenserrrr opened a new pull request, #57510:
URL: https://github.com/apache/spark/pull/57510
### What changes were proposed in this pull request?
`compare_or_generate_golden_matrix` was duplicated verbatim across three PyArrow golden-file test files:
- `python/pyspark/tests/upstream/pyarrow/test_pyarrow_array_cast.py`
- `python/pyspark/tests/upstream/pyarrow/test_pyarrow_arrow_to_pandas_default.py`
- `python/pyspark/tests/upstream/pyarrow/test_pyarrow_arrow_to_pandas_non_default.py`
This PR moves it into `GoldenFileTestMixin` (`python/pyspark/testing/goldenutils.py`), which all three suites already inherit, and removes the local copies. Imports that became unused after the removal (`inspect`, `os`, `typing.Callable/List/Optional`) are dropped from the test files.
This is a follow-up to #57435, where reviewers asked to centralize the duplicated matrix driver into the mixin.
### Why are the changes needed?
Removes duplicated test machinery so the golden-file matrix driver has a single implementation, making it easier to maintain and reuse for future golden-file suites.
### Does this PR introduce _any_ user-facing change?
No. Test-only, behavior-preserving refactor.
### How was this patch tested?
Existing suites pass in compare mode (no golden files regenerated):
```
python -m pytest \
python/pyspark/tests/upstream/pyarrow/test_pyarrow_array_cast.py \
python/pyspark/tests/upstream/pyarrow/test_pyarrow_arrow_to_pandas_default.py \
python/pyspark/tests/upstream/pyarrow/test_pyarrow_arrow_to_pandas_non_default.py
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58332][PYTHON][TEST] Move compare_or_generate_golden_matrix into GoldenFileTestMixin [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #57510:
URL: https://github.com/apache/spark/pull/57510#issuecomment-5100531330
**Merge Summary:**
- merged into master https://github.com/apache/spark/commit/863694202aef47863bbdef35816d3c0e8f3066b9
- merged into branch-4.x https://github.com/apache/spark/commit/f767b1c81c69021869c1cc501c66aa1dcdb6ee72
*Posted by `merge_spark_pr.py`*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58332][PYTHON][TEST] Move compare_or_generate_golden_matrix into GoldenFileTestMixin [spark]
Posted by "Spenserrrr (via GitHub)" <gi...@apache.org>.
Spenserrrr commented on PR #57510:
URL: https://github.com/apache/spark/pull/57510#issuecomment-5073922182
Hi @Yicong-Huang and @uros-b! This is the follow-up to #57435 that removes the duplicated `compare_or_generate_golden_matrix`. Please let me know if there is anything you want me to revise. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58332][PYTHON][TEST] Move compare_or_generate_golden_matrix into GoldenFileTestMixin [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng closed pull request #57510: [SPARK-58332][PYTHON][TEST] Move compare_or_generate_golden_matrix into GoldenFileTestMixin
URL: https://github.com/apache/spark/pull/57510
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58332][PYTHON][TEST] Move compare_or_generate_golden_matrix into GoldenFileTestMixin [spark]
Posted by "Spenserrrr (via GitHub)" <gi...@apache.org>.
Spenserrrr commented on PR #57510:
URL: https://github.com/apache/spark/pull/57510#issuecomment-5095790702
Thanks for the review @uros-b @HyukjinKwon! Would one of you be able to merge this when you have a moment? I think Yicong is OOO this week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58332][PYTHON][TEST] Move compare_or_generate_golden_matrix into GoldenFileTestMixin [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #57510:
URL: https://github.com/apache/spark/pull/57510#issuecomment-5075631855
Thank you @Spenserrrr! cc @Yicong-Huang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org