[jira] [Commented] (FLINK-2834) Global round-robin for temporary directories

["Stephan Ewen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14948881#comment-14948881 ] 

Stephan Ewen commented on FLINK-2834:
-------------------------------------

+1, I think that is important for larger setups

> Global round-robin for temporary directories
> --------------------------------------------
>
>                 Key: FLINK-2834
>                 URL: https://issues.apache.org/jira/browse/FLINK-2834
>             Project: Flink
>          Issue Type: Improvement
>          Components: Distributed Runtime
>    Affects Versions: 0.10
>            Reporter: Greg Hogan
>            Assignee: Greg Hogan
>            Priority: Minor
>
> Currently each {{FileIOChannel}} performs a local round-robin among temporary directories. Initially every writer will be concurrently spilling to the same first directory, then the same second directory, etc. Even when evenly dispersed this effect will naturally occur due to the bunching effect.
> One solution is to use a global counter when selecting the temporary path.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)