[GitHub] [arrow] wjones127 commented on issue #12652: [Python] How to avoid overflow error.

[GitBox <gi...@apache.org>]

wjones127 commented on issue #12652:
URL: https://github.com/apache/arrow/issues/12652#issuecomment-1071004210


   @kato1208 The preferred way to solve overflow is to change your schema to use types that are appropriate for the data. Look at your first example: The `"id"` value starts as `3046682132`, but when converted to Arrow (with `safe=False`) comes out as `-1248285164`. I doubt that's what you actually want.
   
   That id value is just too large to fit into an int32. If you change the schema to use int64 you can run this without any overflow errors and without needing `safe=False`. And it will give you correct data.
   
   ```python
   schema = pa.schema([
       pa.field('name', pa.string()),
       pa.field('id', pa.int64()),
       pa.field("points", pa.list_(pa.int32())),
       pa.field('groups', pa.struct([
           pa.field("group_name", pa.string()),
           pa.field("group_id", pa.int64()),
       ])),
   ])
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org