[jira] [Commented] (ARROW-15026) [Python] datetime.timedelta to pyarrow.duration('us') silently overflows

["Joris Van den Bossche (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-15026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17456348#comment-17456348 ] 

Joris Van den Bossche commented on ARROW-15026:
-----------------------------------------------

Thanks for the report!

The conversion of a datetime.timedelta object to the underlying integer value (how it is stored in Arrow) happens here: https://github.com/apache/arrow/blob/53ab5a000431d97fe8b2afec40034b8287bd44be/cpp/src/arrow/python/datetime.h#L149-L153 
That indeed doesn't check for overflow. We do have some utilities to do additions / multiplications with checking for overflow, see eg {{MultiplyWithOverflow}}.

If you are interested, a PR is certainly welcome!

> [Python] datetime.timedelta to pyarrow.duration('us') silently overflows
> ------------------------------------------------------------------------
>
>                 Key: ARROW-15026
>                 URL: https://issues.apache.org/jira/browse/ARROW-15026
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>            Reporter: Andreas Rappold
>            Priority: Major
>
>  
> Hi! This reproduces the issue:
> {code:java}
> # python 3.9.9
> # pyarrow 6.0.1
> import datetime
> import pyarrow
> d = datetime.timedelta(days=-106751992, seconds=71945, microseconds=224192)
> pyarrow.scalar(d)
> # <pyarrow.DurationScalar: datetime.timedelta(days=-106751992, seconds=71945, microseconds=224192)>
> pyarrow.scalar(d).as_py() == d
> # True
> d2 = d - datetime.timedelta(microseconds=1)
> pyarrow.scalar(d2)
> # <pyarrow.DurationScalar: datetime.timedelta(days=106751991, seconds=14454, microseconds=775807)>
> pyarrow.scalar(d2).as_py() == d2
> # False{code}
> Other conversions (e.g. to int*) raise an exception instead. I didn't check if duration overflows for too large timedeltas. If its easy to fix, point me in the right direction and I try to create a PR. Thanks
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)