[jira] [Commented] (ARROW-5895) [Python] New version stores timestamps as epoch ms instead of ISO timestamp string

["Joris Van den Bossche (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16881549#comment-16881549 ] 

Joris Van den Bossche commented on ARROW-5895:
----------------------------------------------

[~johwilso1] Thanks for the report. Can you provide a small reproducible example? 

Also, what do you mean with "write ISO strings". I don't think pyarrow 0.13.0 was actually writing timestamps as strings (rather as integers), so it might be an issue with the representation of the value.

> [Python] New version stores timestamps as epoch ms instead of ISO timestamp string
> ----------------------------------------------------------------------------------
>
>                 Key: ARROW-5895
>                 URL: https://issues.apache.org/jira/browse/ARROW-5895
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>    Affects Versions: 0.14.0
>         Environment: Linux dev.office.whoop.com 3.10.0-957.21.3.el7.x86_64 #1 SMP Tue Jun 18 16:35:19 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
>            Reporter: John Wilson
>            Priority: Major
>
> Just upgraded from pyarrow 0.13 to 0.14.
> Columns of type TimestampType(timestmap[ns]) now get written as epoch ms values: 
> 1561939200507
> Where 0.13 wrote TimestampType(timestamp[ns]) as an ISO string:
> 2019-07-01T00:00:00.507Z
> This broke my implementation.  How do I get pyarrow to write ISO strings again in 0.14?
>  
> Here is my table write:
> {{ pyarrow.parquet.write_to_dataset(table=tbl, root_path=local_path,}}
> {{ partition_cols=['env', 'dt'],}}
> {{ coerce_timestamps='ms',}}
> {{ allow_truncated_timestamps=True,}}
> {{ version='2.0',}}
> {{ compression='SNAPPY')}}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)