[GitHub] [arrow] jorisvandenbossche commented on issue #26199: [Python] Array.to_numpy() with type fixed_size_list(int64(), 1) doesn't roundtrip for large integer values

["jorisvandenbossche (via GitHub)" <gi...@apache.org>]

jorisvandenbossche commented on issue #26199:
URL: https://github.com/apache/arrow/issues/26199#issuecomment-1481065636

   I am not sure we can do anything about this, since this is an inherent limitation of numpy not having missing value support for integers, and so that we have to use float64 to represent those. The same happens for primitive, non-nested arrays as well:
   
   ```python
   data = [None, 9007199254740993]
   arr = pa.array(data, type=pa.uint64())
   ndarray = arr.to_numpy(zero_copy_only=False)
   
   >>> arr
   <pyarrow.lib.UInt64Array object at 0x7fa6efe29400>
   [
     null,
     9007199254740993
   ]
   >>> ndarray
   array([           nan, 9.00719925e+15])
   ```
   
   One difference, though, is that when trying to recreate, we raise an error instead of silently roundtripping a different value:
   
   ```python
   >>> restored = pa.array(ndarray, type=arr.type)
   ...
   ArrowInvalid: Float value nan was truncated converting to uint64
   ```
   
   That's related to the fact that we don't do safe casting when converting nested data, which is discussed in https://github.com/apache/arrow/issues/31857. 
   
   Will add this issue as an extra example there, and then we can close this.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org