You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "201573 (via GitHub)" <gi...@apache.org> on 2026/05/07 17:02:22 UTC
[PR] [SPARK-56760][PYTHON] Remove dead numpy version check in pandas typehints [spark]
201573 opened a new pull request, #55743:
URL: https://github.com/apache/spark/pull/55743
### What changes were proposed in this pull request?
This PR removes the obsolete NumPy version guard around `numpy.typing.NDArray` handling in pandas-on-Spark type hints. The current minimum NumPy version is already newer than 1.21, so the branch is always enabled.
The associated typedef test now imports `numpy.typing` directly and continues to assert `NDArray[...]` conversion.
### Why are the changes needed?
The version check is dead code now that PySpark requires NumPy 1.22 or newer. Removing it simplifies the type hint path without changing behavior.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Verified locally:
- `SPARK_HOME=/tmp/spark_codex_56760 ./python/run-tests.py --python-executables python3 --testnames pyspark.pandas.tests.test_typedef`
- `SPARK_HOME=/tmp/spark_codex_56760 PYTHON_EXECUTABLE=python3 ./dev/lint-python --compile`
- `git diff --check`
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Codex (GPT-5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-56760][PYTHON] Remove dead numpy version check in pandas typehints [spark]
Posted by "201573 (via GitHub)" <gi...@apache.org>.
201573 closed pull request #55743: [SPARK-56760][PYTHON] Remove dead numpy version check in pandas typehints
URL: https://github.com/apache/spark/pull/55743
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-56760][PYTHON] Remove dead numpy version check in pandas typehints [spark]
Posted by "201573 (via GitHub)" <gi...@apache.org>.
201573 commented on PR #55743:
URL: https://github.com/apache/spark/pull/55743#issuecomment-4399673614
Thanks for pointing this out. I removed that remaining NumPy version guard as well and fixed the ruff formatting failure from the fork run.
Verified locally:
- `PYTHON_EXECUTABLE=/tmp/spark-56765-mypy-venv/bin/python ./dev/lint-python --compile`
- `PATH=/tmp/spark-56765-mypy-venv/bin:$PATH PYTHON_EXECUTABLE=/tmp/spark-56765-mypy-venv/bin/python ./dev/lint-python --ruff`
- `SPARK_HOME=/tmp/spark-pr55743-worktree-1bc6 python3 python/run-tests.py --python-executables=/tmp/spark-56765-mypy-venv/bin/python --testnames pyspark.pandas.tests.computation.test_apply_func`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org