You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Dillian Murphey <cr...@gmail.com> on 2015/06/11 02:07:23 UTC
Increased replication factor. Replication didn't happen!
Ran this:
$KAFKA_HOME/bin/kafka-reassign-partitions.sh
But Kafka did not actually do the replication. Topic description shows the
right numbers, but it just didn't replicate.
What's wrong, and how do I trigger the replication to occur??
I'm running 0.8.2.0
thanks
Re: Increased replication factor. Replication didn't happen!
Posted by Joel Koshy <jj...@gmail.com>.
This may help:
http://kafka.apache.org/documentation.html#basic_ops_increase_replication_factor
On Thu, Jun 11, 2015 at 11:20:05AM +0800, Shady Xu wrote:
> Right now, Kafka topics do not support changing replication factor or
> partition number after creation. The kafka-reassign-partitions.sh tool can
> only reassign existent partitions.
>
> 2015-06-11 9:31 GMT+08:00 Gwen Shapira <gs...@cloudera.com>:
>
> > What do the logs show?
> >
> > On Wed, Jun 10, 2015 at 5:07 PM, Dillian Murphey
> > <cr...@gmail.com> wrote:
> > > Ran this:
> > >
> > > $KAFKA_HOME/bin/kafka-reassign-partitions.sh
> > >
> > > But Kafka did not actually do the replication. Topic description shows
> > the
> > > right numbers, but it just didn't replicate.
> > >
> > > What's wrong, and how do I trigger the replication to occur??
> > >
> > > I'm running 0.8.2.0
> > >
> > > thanks
> >
Re: Increased replication factor. Replication didn't happen!
Posted by Dillian Murphey <cr...@gmail.com>.
Thank you Adam, that helps me know what to look for.
On Thu, Jun 11, 2015 at 12:49 PM, Adam Dubiel <du...@gmail.com> wrote:
> I just tried it out on my 0.8.2 cluster and it worked just fine - the ISR
> grew, replica factor changed and data was physically moved to new brokers.
> Was there not output/no logs? I see things like
>
> INFO Created log for partition [topicName,7] in /opt/kafka/.... with
> properties {.. some json}
>
> in server.log on new brokers.
>
> 2015-06-11 21:02 GMT+02:00 Dillian Murphey <cr...@gmail.com>:
>
> > Oh, hmm. There is even documentation on it:
> >
> >
> >
> http://kafka.apache.org/documentation.html#basic_ops_increase_replication_factor
> >
> > Having a difficult time sifting through the logs. Is this not a common
> > operation? Do users normally just delete the topic and create a new
> one??
> >
> >
> >
> > On Wed, Jun 10, 2015 at 8:20 PM, Shady Xu <sh...@gmail.com> wrote:
> >
> > > Right now, Kafka topics do not support changing replication factor or
> > > partition number after creation. The kafka-reassign-partitions.sh tool
> > can
> > > only reassign existent partitions.
> > >
> > > 2015-06-11 9:31 GMT+08:00 Gwen Shapira <gs...@cloudera.com>:
> > >
> > > > What do the logs show?
> > > >
> > > > On Wed, Jun 10, 2015 at 5:07 PM, Dillian Murphey
> > > > <cr...@gmail.com> wrote:
> > > > > Ran this:
> > > > >
> > > > > $KAFKA_HOME/bin/kafka-reassign-partitions.sh
> > > > >
> > > > > But Kafka did not actually do the replication. Topic description
> > shows
> > > > the
> > > > > right numbers, but it just didn't replicate.
> > > > >
> > > > > What's wrong, and how do I trigger the replication to occur??
> > > > >
> > > > > I'm running 0.8.2.0
> > > > >
> > > > > thanks
> > > >
> > >
> >
>
Re: Increased replication factor. Replication didn't happen!
Posted by Adam Dubiel <du...@gmail.com>.
I just tried it out on my 0.8.2 cluster and it worked just fine - the ISR
grew, replica factor changed and data was physically moved to new brokers.
Was there not output/no logs? I see things like
INFO Created log for partition [topicName,7] in /opt/kafka/.... with
properties {.. some json}
in server.log on new brokers.
2015-06-11 21:02 GMT+02:00 Dillian Murphey <cr...@gmail.com>:
> Oh, hmm. There is even documentation on it:
>
>
> http://kafka.apache.org/documentation.html#basic_ops_increase_replication_factor
>
> Having a difficult time sifting through the logs. Is this not a common
> operation? Do users normally just delete the topic and create a new one??
>
>
>
> On Wed, Jun 10, 2015 at 8:20 PM, Shady Xu <sh...@gmail.com> wrote:
>
> > Right now, Kafka topics do not support changing replication factor or
> > partition number after creation. The kafka-reassign-partitions.sh tool
> can
> > only reassign existent partitions.
> >
> > 2015-06-11 9:31 GMT+08:00 Gwen Shapira <gs...@cloudera.com>:
> >
> > > What do the logs show?
> > >
> > > On Wed, Jun 10, 2015 at 5:07 PM, Dillian Murphey
> > > <cr...@gmail.com> wrote:
> > > > Ran this:
> > > >
> > > > $KAFKA_HOME/bin/kafka-reassign-partitions.sh
> > > >
> > > > But Kafka did not actually do the replication. Topic description
> shows
> > > the
> > > > right numbers, but it just didn't replicate.
> > > >
> > > > What's wrong, and how do I trigger the replication to occur??
> > > >
> > > > I'm running 0.8.2.0
> > > >
> > > > thanks
> > >
> >
>
Re: Increased replication factor. Replication didn't happen!
Posted by Dillian Murphey <cr...@gmail.com>.
Oh, hmm. There is even documentation on it:
http://kafka.apache.org/documentation.html#basic_ops_increase_replication_factor
Having a difficult time sifting through the logs. Is this not a common
operation? Do users normally just delete the topic and create a new one??
On Wed, Jun 10, 2015 at 8:20 PM, Shady Xu <sh...@gmail.com> wrote:
> Right now, Kafka topics do not support changing replication factor or
> partition number after creation. The kafka-reassign-partitions.sh tool can
> only reassign existent partitions.
>
> 2015-06-11 9:31 GMT+08:00 Gwen Shapira <gs...@cloudera.com>:
>
> > What do the logs show?
> >
> > On Wed, Jun 10, 2015 at 5:07 PM, Dillian Murphey
> > <cr...@gmail.com> wrote:
> > > Ran this:
> > >
> > > $KAFKA_HOME/bin/kafka-reassign-partitions.sh
> > >
> > > But Kafka did not actually do the replication. Topic description shows
> > the
> > > right numbers, but it just didn't replicate.
> > >
> > > What's wrong, and how do I trigger the replication to occur??
> > >
> > > I'm running 0.8.2.0
> > >
> > > thanks
> >
>
Re: Increased replication factor. Replication didn't happen!
Posted by Shady Xu <sh...@gmail.com>.
Right now, Kafka topics do not support changing replication factor or
partition number after creation. The kafka-reassign-partitions.sh tool can
only reassign existent partitions.
2015-06-11 9:31 GMT+08:00 Gwen Shapira <gs...@cloudera.com>:
> What do the logs show?
>
> On Wed, Jun 10, 2015 at 5:07 PM, Dillian Murphey
> <cr...@gmail.com> wrote:
> > Ran this:
> >
> > $KAFKA_HOME/bin/kafka-reassign-partitions.sh
> >
> > But Kafka did not actually do the replication. Topic description shows
> the
> > right numbers, but it just didn't replicate.
> >
> > What's wrong, and how do I trigger the replication to occur??
> >
> > I'm running 0.8.2.0
> >
> > thanks
>
Re: Increased replication factor. Replication didn't happen!
Posted by Gwen Shapira <gs...@cloudera.com>.
What do the logs show?
On Wed, Jun 10, 2015 at 5:07 PM, Dillian Murphey
<cr...@gmail.com> wrote:
> Ran this:
>
> $KAFKA_HOME/bin/kafka-reassign-partitions.sh
>
> But Kafka did not actually do the replication. Topic description shows the
> right numbers, but it just didn't replicate.
>
> What's wrong, and how do I trigger the replication to occur??
>
> I'm running 0.8.2.0
>
> thanks