You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2016/03/16 07:28:33 UTC

[jira] [Assigned] (SPARK-13926) Automatically use Kryo serializer when shuffling RDDs with simple types

     [ https://issues.apache.org/jira/browse/SPARK-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-13926:
------------------------------------

    Assignee: Josh Rosen  (was: Apache Spark)

> Automatically use Kryo serializer when shuffling RDDs with simple types
> -----------------------------------------------------------------------
>
>                 Key: SPARK-13926
>                 URL: https://issues.apache.org/jira/browse/SPARK-13926
>             Project: Spark
>          Issue Type: Improvement
>          Components: Shuffle
>            Reporter: Josh Rosen
>            Assignee: Josh Rosen
>
> Because ClassTags are available when constructing ShuffledRDD we can use  them to automatically use Kryo for shuffle serialization when the RDD's types are guaranteed to be compatible with Kryo (e.g. RDDs whose key, value, and/or combiner types are primitives, arrays of primitives, or strings). This is likely to result in a large performance gain for many RDD API workloads.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org