You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Jason Gustafson (JIRA)" <ji...@apache.org> on 2016/09/23 22:49:20 UTC
[jira] [Updated] (KAFKA-2796) add support for reassignment
partition to specified logdir
[ https://issues.apache.org/jira/browse/KAFKA-2796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Gustafson updated KAFKA-2796:
-----------------------------------
Fix Version/s: (was: 0.10.1.0)
0.10.2.0
> add support for reassignment partition to specified logdir
> ----------------------------------------------------------
>
> Key: KAFKA-2796
> URL: https://issues.apache.org/jira/browse/KAFKA-2796
> Project: Kafka
> Issue Type: Improvement
> Components: clients, controller, core, log
> Reporter: Yonghui Yang
> Assignee: Yonghui Yang
> Labels: features
> Fix For: 0.10.2.0
>
>
> Currently when creating a log, the directory is chosen by calculating the number of partitions
> in each directory and then choosing the data directory with the fewest partitions.
> However, the sizes of different TopicParitions are very different, which lead to usage vary greatly between different logDirs. And usually each logDir corresponds to a disk, so the disk usage between different disks is very imbalance .
> The possible solution is to reassign partitions in high-usage logDirs to low-usage logDirs. I change the format of /admin/reassign_partitions，add replicaDirs field. At reassigning Partitions, when broker’s LogManager.createLog() is invoked , if replicaDir is specified , the specified logDir will be chosen, otherwise the logDir with the fewest partitions will be chosen.
> the old /admin/reassign_partitions:
> {"version":1,
> "partitions":
> [
> {
> "topic" : "Foo",
> "partition": 1,
> "replicas": [1, 2, 3]
> }
> ]
> }
> the new /admin/reassign_partitions:
> {"version":1,
> "partitions":
> [
> {
> "topic" : "Foo",
> "partition": 1,
> "replicas": [1, 2, 3],
> "replicaDirs": {"1":"/data1/kafka_data", "3":"/data10/kakfa_data" }
> }
> ]
> }
> This feature has been developed.
> PR: https://github.com/apache/kafka/pull/484
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)