[jira] [Created] (KAFKA-2267) kafka-reassign-partitions.sh --generate doesn't work when reducing replication factor

["Robin Yamaguchi (JIRA)" <ji...@apache.org>]

Robin Yamaguchi created KAFKA-2267:
--------------------------------------

             Summary: kafka-reassign-partitions.sh --generate doesn't work when reducing replication factor
                 Key: KAFKA-2267
                 URL: https://issues.apache.org/jira/browse/KAFKA-2267
             Project: Kafka
          Issue Type: Bug
          Components: tools
    Affects Versions: 0.8.2.1
         Environment: Centos 6.5, scala_version 2.10.4
            Reporter: Robin Yamaguchi
            Priority: Minor


Using kafka-reassign-partitions.sh --generate errors when attempting to reduce the replication factor of a topic.

{code}
$ /usr/local/kafka/bin/kafka-topics.sh --describe --zookeeper server01:2181,server02:2181,server03:2181 --topic OpsTesting

Topic:OpsTesting	PartitionCount:1	ReplicationFactor:3	Configs:
	Topic: OpsTesting	Partition: 0	Leader: 2	Replicas: 2,0,1	Isr: 2,0,1

$ more topics-to-move.json

{"topics": [{"topic": "OpsTesting"}],
 "version":1
}

$ /usr/local/kafka/bin/kafka-reassign-partitions.sh --zookeeper --describe --zookeeper server01:2181,server02:2181,server03:2181 --topics-to-move-json-file topics-to-move.json --broker-list "1,2" --generate


Partitions reassignment failed due to replication factor: 3 larger than available brokers: 2
kafka.admin.AdminOperationException: replication factor: 3 larger than available brokers: 2
	at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
	at kafka.admin.ReassignPartitionsCommand$$anonfun$generateAssignment$1.apply(ReassignPartitionsCommand.scala:97)
	at kafka.admin.ReassignPartitionsCommand$$anonfun$generateAssignment$1.apply(ReassignPartitionsCommand.scala:96)
	at scala.collection.immutable.Map$Map1.foreach(Map.scala:109)
	at kafka.admin.ReassignPartitionsCommand$.generateAssignment(ReassignPartitionsCommand.scala:96)
	at kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:45)
	at kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
{code}

Yet I can manually create a json file, and kafka-reassign-partitions.sh --execute will work:

{code}
$ more part.json

{"version":1,"partitions":[{"topic":"OpsTesting","partition":0,"replicas":[0,1]}]}

$ /usr/local/kafka/bin/kafka-reassign-partitions.sh --zookeeper server01:2181,server02:2181,server03:2181 --reassignment-json-file part.json --execute
Current partition replica assignment

{"version":1,"partitions":[{"topic":"OpsTesting","partition":0,"replicas":[2,0,1]}]}

Save this to use as the --reassignment-json-file option during rollback
Successfully started reassignment of partitions {"version":1,"partitions":[{"topic":"OpsTesting","partition":0,"replicas":[0,1]}]}
{code}

kafka-reassign-partitions.sh --verify also works as expected after the --execute:

{code}
$ /usr/local/kafka/bin/kafka-topics.sh --describe --zookeeper server01:2181,server02:2181,server03:2181 --topic OpsTesting
Topic:OpsTesting	PartitionCount:1	ReplicationFactor:2	Configs:
	Topic: OpsTesting	Partition: 0	Leader: 0	Replicas: 0,1	Isr: 0,1


$ /usr/local/kafka/bin/kafka-reassign-partitions.sh --zookeeper server01:2181,server02:2181,server03:2181 --reassignment-json-file part.json --verify
Status of partition reassignment:
Reassignment of partition [OpsTesting,0] completed successfully
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)