Partition assigner that takes current assignment into account

[Kanak Biscuitwala <ka...@hotmail.com>]

Hi,



I wanted to highlight an alternative algorithm that we have written to generate JSON compatible with kafka-reassign-partitions.sh. It's novel for the following reasons:



- Like the existing algorithm, it's rack-aware, balances partitions assigned to hosts, and number of leader replicas on each host


- Unlike the existing algorithm, it takes the current assignment into account to minimize data movement


- Because of the above property, it works well for decommissioning or replacing nodes, as well as adding new nodes, and taking only a subset of partitions away from existing nodes.



Take a look: https://github.com/SiftScience/kafka-assigner