You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Robin Yamaguchi <ro...@tune.com> on 2015/06/11 01:05:25 UTC
Changing replication factor for an existing topic
Greetings,
With Kafka 0.8.2.1, is there a way to either raise or lower the replication
factor of an existing topic? Searching through this list, there was
mention that the tool has yet to be developed. It doesn't seem possible
still looking at the replication tools wiki, nor with yahoo's kafka
manager. Thought I give this list another shot.
Thank you,
Robin
Re: Changing replication factor for an existing topic
Posted by Robin Yamaguchi <ro...@tune.com>.
Thank you for the replies. I was able to lower the replication factor by
using the kafka-reassign-partitions.sh tool. What initially threw me off
was kafka-reassign-partitions.sh --generate was giving this error:
Partitions reassignment failed due to replication factor: 3 larger than
available brokers: 2
kafka.admin.AdminOperationException: replication factor: 3 larger than
available brokers: 2
at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
at
kafka.admin.ReassignPartitionsCommand$$anonfun$generateAssignment$1.apply(ReassignPartitionsCommand.scala:97)
at
kafka.admin.ReassignPartitionsCommand$$anonfun$generateAssignment$1.apply(ReassignPartitionsCommand.scala:96)
at scala.collection.immutable.Map$Map1.foreach(Map.scala:109)
at
kafka.admin.ReassignPartitionsCommand$.generateAssignment(ReassignPartitionsCommand.scala:96)
at
kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:45)
at
kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
However running kafka-reassign-partitions.sh --execute against a manually
created json file worked as expected. I documented my experiences in a
JIRA ticket here: https://issues.apache.org/jira/browse/KAFKA-2267
On Wed, Jun 10, 2015 at 4:31 PM, James Cheng <jc...@tivo.com> wrote:
> AirBNB's kafkat tool has a "set-replication-factor" option. I've never
> tried it myself.
>
> https://github.com/airbnb/kafkat
>
> -James
>
> > On Jun 10, 2015, at 4:20 PM, Aditya Auradkar
> <aa...@linkedin.com.INVALID> wrote:
> >
> > The replica list that you specify can be used to increment/decrement the
> replication factor.
> >
> http://kafka.apache.org/documentation.html#basic_ops_increase_replication_factor
> >
> > Aditya
> >
> > ________________________________________
> > From: Robin Yamaguchi [robin@tune.com]
> > Sent: Wednesday, June 10, 2015 4:05 PM
> > To: users@kafka.apache.org
> > Subject: Changing replication factor for an existing topic
> >
> > Greetings,
> >
> > With Kafka 0.8.2.1, is there a way to either raise or lower the
> replication
> > factor of an existing topic? Searching through this list, there was
> > mention that the tool has yet to be developed. It doesn't seem possible
> > still looking at the replication tools wiki, nor with yahoo's kafka
> > manager. Thought I give this list another shot.
> >
> > Thank you,
> > Robin
>
>
Re: Changing replication factor for an existing topic
Posted by James Cheng <jc...@tivo.com>.
AirBNB's kafkat tool has a "set-replication-factor" option. I've never tried it myself.
https://github.com/airbnb/kafkat
-James
> On Jun 10, 2015, at 4:20 PM, Aditya Auradkar <aa...@linkedin.com.INVALID> wrote:
>
> The replica list that you specify can be used to increment/decrement the replication factor.
> http://kafka.apache.org/documentation.html#basic_ops_increase_replication_factor
>
> Aditya
>
> ________________________________________
> From: Robin Yamaguchi [robin@tune.com]
> Sent: Wednesday, June 10, 2015 4:05 PM
> To: users@kafka.apache.org
> Subject: Changing replication factor for an existing topic
>
> Greetings,
>
> With Kafka 0.8.2.1, is there a way to either raise or lower the replication
> factor of an existing topic? Searching through this list, there was
> mention that the tool has yet to be developed. It doesn't seem possible
> still looking at the replication tools wiki, nor with yahoo's kafka
> manager. Thought I give this list another shot.
>
> Thank you,
> Robin
RE: Changing replication factor for an existing topic
Posted by Aditya Auradkar <aa...@linkedin.com.INVALID>.
The replica list that you specify can be used to increment/decrement the replication factor.
http://kafka.apache.org/documentation.html#basic_ops_increase_replication_factor
Aditya
________________________________________
From: Robin Yamaguchi [robin@tune.com]
Sent: Wednesday, June 10, 2015 4:05 PM
To: users@kafka.apache.org
Subject: Changing replication factor for an existing topic
Greetings,
With Kafka 0.8.2.1, is there a way to either raise or lower the replication
factor of an existing topic? Searching through this list, there was
mention that the tool has yet to be developed. It doesn't seem possible
still looking at the replication tools wiki, nor with yahoo's kafka
manager. Thought I give this list another shot.
Thank you,
Robin