You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "leocook (JIRA)" <ji...@apache.org> on 2017/02/08 11:20:41 UTC
[jira] [Updated] (SPARK-19513) Split partition without shuffle（means repartition locally）
[ https://issues.apache.org/jira/browse/SPARK-19513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
leocook updated SPARK-19513:
----------------------------
Description:
provide a method， that can split partition without shuffle， just repartition locally.
Sample: there is a partition with to much data, it will be OOM to spark task,
if we increase the num of partition, we usually call repartition，but it will shuffle.
Now I want to split partitions locally, just make less data in each partition。
!http://7xriy2.com1.z0.glb.clouddn.com/spark%20split%20partition%20no%20shuffer.png!
was:
provide a method， that can split partition without shuffle.
!http://7xriy2.com1.z0.glb.clouddn.com/spark%20split%20partition%20no%20shuffer.png!
Summary: Split partition without shuffle（means repartition locally） (was: Split partition(means repartition) without shuffle)
> Split partition without shuffle（means repartition locally）
> ----------------------------------------------------------
>
> Key: SPARK-19513
> URL: https://issues.apache.org/jira/browse/SPARK-19513
> Project: Spark
> Issue Type: New Feature
> Components: Spark Core
> Affects Versions: 3.0.0
> Reporter: leocook
>
> provide a method， that can split partition without shuffle， just repartition locally.
> Sample: there is a partition with to much data, it will be OOM to spark task,
> if we increase the num of partition, we usually call repartition，but it will shuffle.
> Now I want to split partitions locally, just make less data in each partition。
> !http://7xriy2.com1.z0.glb.clouddn.com/spark%20split%20partition%20no%20shuffer.png!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org