[jira] [Created] (ARROW-9768) Pyarrow allows for unsafe conversions of datetime objects to timestamp nanoseconds

["Joshua Lay (Jira)" <ji...@apache.org>]

Joshua Lay created ARROW-9768:
---------------------------------

             Summary: Pyarrow allows for unsafe conversions of datetime objects to timestamp nanoseconds
                 Key: ARROW-9768
                 URL: https://issues.apache.org/jira/browse/ARROW-9768
             Project: Apache Arrow
          Issue Type: Bug
    Affects Versions: 1.0.0
            Reporter: Joshua Lay


Hi, 

I've been working on a project that requires the use of pyarrow. In general, it pulls data from database into a pandas dataframe, converts this dataframe into pyarrow table, then writes this table into parquet. 

In parquet, I want to store date values as timestamp format with nanoseconds precision. This works fine with most dates except those past pandas.Timestamp.max: [https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Timestamp.max.html.|https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Timestamp.max.html]

I was expecting some exception to be raised (like in Pandas), however this did not happen and the value was processed incorrectly. Note that this is with safe=True. Can this please be looked into? Thanks

{{Example Code:}}

{{pa.array([datetime(2262,4,12)], type=pa.timestamp("ns"))}}
{{}}

{{Return:}}

{{[}}
{{ 1677-09-21 00:25:26.290448384}}
{{]}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)