You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@arrow.apache.org by "Wes McKinney (JIRA)" <ji...@apache.org> on 2018/03/13 02:48:00 UTC

[jira] [Updated] (ARROW-1425) [Python] Document semantic differences between Spark timestamps and Arrow timestamps

     [ https://issues.apache.org/jira/browse/ARROW-1425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wes McKinney updated ARROW-1425:
--------------------------------
    Fix Version/s:     (was: 0.9.0)
                   0.10.0

> [Python] Document semantic differences between Spark timestamps and Arrow timestamps
> ------------------------------------------------------------------------------------
>
>                 Key: ARROW-1425
>                 URL: https://issues.apache.org/jira/browse/ARROW-1425
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: Python
>            Reporter: Wes McKinney
>            Assignee: Li Jin
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 0.10.0
>
>
> The way that Spark treats non-timezone-aware timestamps as session local can be problematic when using pyarrow which may view the data coming from toPandas() as time zone naive (but with fields as though it were UTC, not session local). We should document carefully how to properly handle the data coming from Spark to avoid problems.
> cc [~bryanc] [~holdenkarau]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)