You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Manikumar Reddy (JIRA)" <ji...@apache.org> on 2015/06/14 08:16:00 UTC
[jira] [Commented] (KAFKA-2267) kafka-reassign-partitions.sh
--generate doesn't work when reducing replication factor
[ https://issues.apache.org/jira/browse/KAFKA-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584951#comment-14584951 ]
Manikumar Reddy commented on KAFKA-2267:
----------------------------------------
AFAIK, kafka-reassign-partitions.sh --generate option does not support decommissioning_brokers and reducing replication factor. We need to manually create the JSON file.
https://kafka.apache.org/documentation.html#basic_ops_decommissioning_brokers
> kafka-reassign-partitions.sh --generate doesn't work when reducing replication factor
> -------------------------------------------------------------------------------------
>
> Key: KAFKA-2267
> URL: https://issues.apache.org/jira/browse/KAFKA-2267
> Project: Kafka
> Issue Type: Bug
> Components: tools
> Affects Versions: 0.8.2.1
> Environment: Centos 6.5, scala_version 2.10.4
> Reporter: Robin Yamaguchi
> Priority: Minor
>
> Using kafka-reassign-partitions.sh --generate errors when attempting to reduce the replication factor of a topic.
> {code}
> $ /usr/local/kafka/bin/kafka-topics.sh --describe --zookeeper server01:2181,server02:2181,server03:2181 --topic OpsTesting
> Topic:OpsTesting PartitionCount:1 ReplicationFactor:3 Configs:
> Topic: OpsTesting Partition: 0 Leader: 2 Replicas: 2,0,1 Isr: 2,0,1
> $ more topics-to-move.json
> {"topics": [{"topic": "OpsTesting"}],
> "version":1
> }
> $ /usr/local/kafka/bin/kafka-reassign-partitions.sh --zookeeper --describe --zookeeper server01:2181,server02:2181,server03:2181 --topics-to-move-json-file topics-to-move.json --broker-list "1,2" --generate
> Partitions reassignment failed due to replication factor: 3 larger than available brokers: 2
> kafka.admin.AdminOperationException: replication factor: 3 larger than available brokers: 2
> at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
> at kafka.admin.ReassignPartitionsCommand$$anonfun$generateAssignment$1.apply(ReassignPartitionsCommand.scala:97)
> at kafka.admin.ReassignPartitionsCommand$$anonfun$generateAssignment$1.apply(ReassignPartitionsCommand.scala:96)
> at scala.collection.immutable.Map$Map1.foreach(Map.scala:109)
> at kafka.admin.ReassignPartitionsCommand$.generateAssignment(ReassignPartitionsCommand.scala:96)
> at kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:45)
> at kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
> {code}
> Yet I can manually create a json file, and kafka-reassign-partitions.sh --execute will work:
> {code}
> $ more part.json
> {"version":1,"partitions":[{"topic":"OpsTesting","partition":0,"replicas":[0,1]}]}
> $ /usr/local/kafka/bin/kafka-reassign-partitions.sh --zookeeper server01:2181,server02:2181,server03:2181 --reassignment-json-file part.json --execute
> Current partition replica assignment
> {"version":1,"partitions":[{"topic":"OpsTesting","partition":0,"replicas":[2,0,1]}]}
> Save this to use as the --reassignment-json-file option during rollback
> Successfully started reassignment of partitions {"version":1,"partitions":[{"topic":"OpsTesting","partition":0,"replicas":[0,1]}]}
> {code}
> kafka-reassign-partitions.sh --verify also works as expected after the --execute:
> {code}
> $ /usr/local/kafka/bin/kafka-topics.sh --describe --zookeeper server01:2181,server02:2181,server03:2181 --topic OpsTesting
> Topic:OpsTesting PartitionCount:1 ReplicationFactor:2 Configs:
> Topic: OpsTesting Partition: 0 Leader: 0 Replicas: 0,1 Isr: 0,1
> $ /usr/local/kafka/bin/kafka-reassign-partitions.sh --zookeeper server01:2181,server02:2181,server03:2181 --reassignment-json-file part.json --verify
> Status of partition reassignment:
> Reassignment of partition [OpsTesting,0] completed successfully
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)