You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Bryan Cutler (Jira)" <ji...@apache.org> on 2020/01/24 23:05:00 UTC

[jira] [Created] (SPARK-30640) Prevent unnessary copies of data in Arrow to Pandas conversion with Timestamps

Bryan Cutler created SPARK-30640:
------------------------------------

             Summary: Prevent unnessary copies of data in Arrow to Pandas conversion with Timestamps
                 Key: SPARK-30640
                 URL: https://issues.apache.org/jira/browse/SPARK-30640
             Project: Spark
          Issue Type: Improvement
          Components: PySpark, SQL
    Affects Versions: 2.4.4
            Reporter: Bryan Cutler


During conversion of Arrow to Pandas, timestamp columns are modified to localize for the current timezone. If there are no timestamp columns, this can sometimes result in unnecessary copies of the data. See [https://www.mail-archive.com/dev@arrow.apache.org/msg17008.html] for discussion.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org