You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Davies Liu (JIRA)" <ji...@apache.org> on 2016/03/02 23:20:18 UTC

[jira] [Commented] (SPARK-13353) Use UnsafeRowSerializer to collect DataFrame

    [ https://issues.apache.org/jira/browse/SPARK-13353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176598#comment-15176598 ] 

Davies Liu commented on SPARK-13353:
------------------------------------

[~rxin] Any idea on this one? One workaround could be packing rows as batches before collect a DataFrame, then deserialize them in collect() on driver.

> Use UnsafeRowSerializer to collect DataFrame
> --------------------------------------------
>
>                 Key: SPARK-13353
>                 URL: https://issues.apache.org/jira/browse/SPARK-13353
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>            Reporter: Davies Liu
>
> UnsafeRowSerializer should be more efficient than JavaSerializer or KyroSerializer for DataFrame.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org