[jira] [Commented] (FLINK-1003) Spread out scheduling strategy

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-1003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15337744#comment-15337744 ] 

ASF GitHub Bot commented on FLINK-1003:
---------------------------------------

GitHub user gyfora opened a pull request:

    https://github.com/apache/flink/pull/2129

    [FLINK-1003] [WIP] Spread out scheduling of tasks

    This is a working progress PR with the core functionality implemented but no tests yet.
    
    As this is a highly critical part of the system I would like to get some initial feedback before proceeding to write / change a huge amount of tests :)
    
    About the functionality:
    
    This is an adaptation of https://github.com/apache/flink/pull/60 to the current flink scheduler. Instead of preferring local instances when scheduling new task slots the new scheduling strategy allows users to balance the load on the different task managers.
    
    Every time a new task needs to be scheduled the scheduler considers all instances that satisfy the scheduling constraints (has available nodes + locality constraints) and picks the one with the smallest load. Load is calculated by the percentage of task slots occupied in a given task manager.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/gyfora/flink scheduling

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/2129.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2129
    
----
commit f895fd71f392482cf0a50e32dc637f7885995c4e
Author: Gyula Fora <gy...@apache.org>
Date:   2016-06-18T10:19:13Z

    [FLINK-1003] Spread out scheduling of tasks

----


> Spread out scheduling strategy
> ------------------------------
>
>                 Key: FLINK-1003
>                 URL: https://issues.apache.org/jira/browse/FLINK-1003
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: Till Rohrmann
>            Assignee: Gyula Fora
>
> Currently the Flink scheduler tries to fill one instance completely before the tasks are deployed to another instance. This is a good behaviour in multi-user and multi-job scenarios but it wastes resources if one wants to use the complete cluster. Therefore, another scheduling strategy where the load among the different instances is kept balanced might be useful. This spread out strategy will deploy the tasks such that the overall work is equally distributed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)