[jira] [Updated] (KAFKA-1792) change behavior of --generate to produce assignment config with fair replica distribution and minimal number of reassignments

["Dmitry Pekar (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmitry Pekar updated KAFKA-1792:
--------------------------------
    Attachment: KAFKA-1792_2014-12-03_19:24:56.patch

> change behavior of --generate to produce assignment config with fair replica distribution and minimal number of reassignments
> -----------------------------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-1792
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1792
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: tools
>            Reporter: Dmitry Pekar
>            Assignee: Dmitry Pekar
>             Fix For: 0.8.3
>
>         Attachments: KAFKA-1792.patch, KAFKA-1792_2014-12-03_19:24:56.patch
>
>
> Current implementation produces fair replica distribution between specified list of brokers. Unfortunately, it doesn't take
> into account current replica assignment.
> So if we have, for instance, 3 brokers id=[0..2] and are going to add fourth broker id=3, 
> generate will create an assignment config which will redistribute replicas fairly across brokers [0..3] 
> in the same way as those partitions were created from scratch. It will not take into consideration current replica 
> assignment and accordingly will not try to minimize number of replica moves between brokers.
> As proposed by [~charmalloc] this should be improved. New output of improved --generate algorithm should suite following requirements:
> - fairness of replica distribution - every broker will have R or R+1 replicas assigned;
> - minimum of reassignments - number of replica moves between brokers will be minimal;
> Example.
> Consider following replica distribution per brokers [0..3] (we just added brokers 2 and 3):
> - broker - 0, 1, 2, 3 
> - replicas - 7, 6, 0, 0
> The new algorithm will produce following assignment:
> - broker - 0, 1, 2, 3 
> - replicas - 4, 3, 3, 3
> - moves - -3, -3, +3, +3
> It will be fair and number of moves will be 6, which is minimal for specified initial distribution.
> The scope of this issue is:
> - design an algorithm matching the above requirements;
> - implement this algorithm and unit tests;
> - test it manually using different initial assignments;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)