You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Stephan Ewen (JIRA)" <ji...@apache.org> on 2014/11/13 12:57:33 UTC

[jira] [Commented] (FLINK-1237) Add custom partitioners for shuffle steps

    [ https://issues.apache.org/jira/browse/FLINK-1237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209655#comment-14209655 ] 

Stephan Ewen commented on FLINK-1237:
-------------------------------------

I'll implement that as a special global property that has the partitioner attached.

> Add custom partitioners for shuffle steps
> -----------------------------------------
>
>                 Key: FLINK-1237
>                 URL: https://issues.apache.org/jira/browse/FLINK-1237
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: Till Rohrmann
>            Assignee: Stephan Ewen
>
> It would be helpful to be able to specify a custom partitioner for shuffle steps. This would users allow to control how their data is distributed across the cluster. Especially in situations where the number of data records is equal to the number of slots it might be a possibility to avoid skewed data distributions due to the default hashing function.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)