You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "bmschmidt (via GitHub)" <gi...@apache.org> on 2025/03/21 02:22:41 UTC
[I] safe casting fails on large integer when called through pa.array [arrow]
bmschmidt opened a new issue, #45875:
URL: https://github.com/apache/arrow/issues/45875
### Describe the bug, including details regarding any error messages, version, and platform.
Thanks to everyone who has contributed to this project.
I'm finding a discrepancy in behavior between creating an array directly and casting it. The `safe` argument does not behave the same way in pa.array as in `Array.cast`, although the docs make it seem that they should both allow casting.
```python
import pyarrow as pa
# works, rounding to 100000000
pa.array([99999999]).cast(pa.float32(), safe=False)
# fails
pa.array([99999999], pa.float32(), safe=False)
```
Platforms:
* Python 3.12.5/OS X/pyarrow 19.0.1
Full output below.
```
Python 3.12.5 (main, Aug 14 2024, 04:32:18) [Clang 18.1.8 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyarrow as pa; pa.array([99999999]).cast(pa.float32(), safe=False)
<pyarrow.lib.FloatArray object at 0x107a44100>
[
100000000
]
>>> pa.array([99999999], pa.float32(), safe=False)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "pyarrow/array.pxi", line 372, in pyarrow.lib.array
File "pyarrow/array.pxi", line 42, in pyarrow.lib._sequence_to_array
File "pyarrow/error.pxi", line 155, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Integer value 99999999 is outside of the range exactly representable by a IEEE 754 single precision value
>>> pa.__version__
'19.0.1'
```
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Safe casting fails on large integer when called through pa.array [arrow]
Posted by "bmschmidt (via GitHub)" <gi...@apache.org>.
bmschmidt commented on issue #45875:
URL: https://github.com/apache/arrow/issues/45875#issuecomment-2775765263
Thanks for the update, makes sense!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Safe casting fails on large integer when called through pa.array [arrow]
Posted by "bmschmidt (via GitHub)" <gi...@apache.org>.
bmschmidt closed issue #45875: [Python] Safe casting fails on large integer when called through pa.array
URL: https://github.com/apache/arrow/issues/45875
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Safe casting fails on large integer when called through pa.array [arrow]
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on issue #45875:
URL: https://github.com/apache/arrow/issues/45875#issuecomment-2771606180
Thank you for opening the issue, @bmschmidt!
Unfortunately, this is a known bug. We've created an umbrella issue that tracks all cases related to the Python to PyArrow array conversion where `safe=False` is ignored. You can follow the progress of this bug here: https://github.com/apache/arrow/issues/36202.
The issue arises because the code path for the Python to PyArrow conversion does not implement the different notions of `safe=True` vs `safe=False`. If you would start with a NumPy array, which uses a different code path (numpy_to_arrow.cc ), the above behaviour would match:
```python
In [10]: pa.array(np.array([99999999]), pa.float32(), safe=False)
Out[10]:
<pyarrow.lib.FloatArray object at 0x10c074ca0>
[
100000000
]
```
To fix this, the code in python_to_arrow.cc will need to be updated to use the Arrow C++ casting machinery, including the correct handling of safe.
I’ll be closing this issue in favour of the one linked above (https://github.com/apache/arrow/issues/36202), but please don’t hesitate to ask any additional questions or add comments there!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org