You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "Yicong-Huang (via GitHub)" <gi...@apache.org> on 2026/02/09 15:14:07 UTC
[PR] [SPARK-55405][PYTHON][TESTS][FOLLOWUP] Fix float16 cast test override for numpy compatibility [spark]
Yicong-Huang opened a new pull request, #54228:
URL: https://github.com/apache/spark/pull/54228
### What changes were proposed in this pull request?
Compute float16 version overrides dynamically instead of hardcoding string values. PyArrow < 21 uses numpy's float16 formatting for `str(scalar)`, which varies across numpy versions (scientific notation in >= 2.4, decimal in < 2.4). This fixes CI failures on environments with numpy 1.26.
### Why are the changes needed?
CI (Python 3.10, PyArrow 18, numpy 1.26) fails with:
```
int16:max_min -> float16: expected '[3.277e+04, -3.277e+04, None]@float16', got '[32770.0, -32770.0, None]@float16'
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Tested locally across Python 3.10/3.11/3.12 with PyArrow 18 and 23.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55405][PYTHON][TESTS][FOLLOWUP] Skip PyArrow array cast tests when numpy < 2.0 [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng closed pull request #54228: [SPARK-55405][PYTHON][TESTS][FOLLOWUP] Skip PyArrow array cast tests when numpy < 2.0
URL: https://github.com/apache/spark/pull/54228
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55405][PYTHON][TESTS][FOLLOWUP] Skip PyArrow array cast tests when numpy < 2.0 [spark]
Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
Yicong-Huang commented on PR #54228:
URL: https://github.com/apache/spark/pull/54228#issuecomment-3880301651
> @Yicong-Huang actually, I think we can just skip the test when numpy < 2, like the udf type tests
Make sense. I have updated it to skip when numpy < 2.0.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55405][PYTHON][TESTS][FOLLOWUP] Skip PyArrow array cast tests when numpy < 2.0 [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #54228:
URL: https://github.com/apache/spark/pull/54228#issuecomment-3881611283
thanks, merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-55405][PYTHON][TESTS][FOLLOWUP] Fix float16 cast test override for numpy compatibility [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #54228:
URL: https://github.com/apache/spark/pull/54228#issuecomment-3874809973
@Yicong-Huang actually, I think we can just skip the test when numpy < 2, like the udf type tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org