[jira] [Commented] (KAFKA-347) change number of partitions of a topic online

["Maxime Brugidou (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13553873#comment-13553873 ] 

Maxime Brugidou commented on KAFKA-347:
---------------------------------------

It's sort of out of scope but I think that using some sort of consistent hashing for partition assignment could really help here. The process could be that if you activate "automatic partition assignment", you don't need to send the ReassignPartition admin command to manage partitions but you over-partition and spread partitions for each topics over brokers using consistent hashing. This would:

- Minimize the number of partitions moved when you add/remove brokers
- Partitions would stick to a broker when some partitions are added/deleted
- Could be run on a regular basis by brokers or other means, and would be perfectly consistent

This would greatly improve operations
                
> change number of partitions of a topic online
> ---------------------------------------------
>
>                 Key: KAFKA-347
>                 URL: https://issues.apache.org/jira/browse/KAFKA-347
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: core
>    Affects Versions: 0.8
>            Reporter: Jun Rao
>              Labels: features
>
> We will need an admin tool to change the number of partitions of a topic online.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira