You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Bob Jervis <bj...@visibletechnologies.com> on 2013/02/11 18:32:49 UTC
Changing replication factors in 0.8.
In testing our 0.8 cluster, we started by just using the sample server.properties file that ships with 0.8 and tweaking it. The replication factor property did not have an exemplar in the file, so we didn't include it. Naturally, the cluster did not do replication.
After sending some data through the tests, we then added replication factor=2 to the config and restarted the cluster. After a half-hour it did not appear to replicate the topic data we had generated during the first test. We had to erase the topics entirely and start from scratch. Replication then happened and when we took down a broker we saw the gaps in the topic data get filled in after restarting it.
In the future, what is the correct way to get a Kafka cluster to change the replication factor on an existing topic? Will we have to flush all data from the cluster for a given topic, or is there a recommended way to convert existing data?
Bob Jervis | Senior Architect
[cid:image001.png@01CE083A.C1C14FC0]<http://www.visibletechnologies.com/>
Seattle | Boston | New York | London
Phone: 425.957.6075 | Fax: 781.404.5711
Follow Visibly Intelligent Blog<http://www.visibletechnologies.com/blog/>
[cid:image002.png@01CE083A.C1C14FC0]<http://twitter.com/visible>[cid:image003.png@01CE083A.C1C14FC0]<http://www.facebook.com/Visible.Technologies> [cid:image004.png@01CE083A.C1C14FC0] <http://www.linkedin.com/company/visible-technologies>
Re: Changing replication factors in 0.8.
Posted by Jun Rao <ju...@gmail.com>.
Bob,
We do plan to support changing replication factors online in the future.
This will be a post 0.8 feature though.
Thanks,
Jun
On Mon, Feb 11, 2013 at 9:32 AM, Bob Jervis <bjervis@visibletechnologies.com
> wrote:
> In testing our 0.8 cluster, we started by just using the sample
> server.properties file that ships with 0.8 and tweaking it. The
> replication factor property did not have an exemplar in the file, so we
> didn’t include it. Naturally, the cluster did not do replication.****
>
> ** **
>
> After sending some data through the tests, we then added replication
> factor=2 to the config and restarted the cluster. After a half-hour it did
> not appear to replicate the topic data we had generated during the first
> test. We had to erase the topics entirely and start from scratch.
> Replication then happened and when we took down a broker we saw the gaps in
> the topic data get filled in after restarting it.****
>
> ** **
>
> In the future, what is the correct way to get a Kafka cluster to change
> the replication factor on an existing topic? Will we have to flush all
> data from the cluster for a given topic, or is there a recommended way to
> convert existing data?****
>
> ** **
>
> *Bob Jervis | Senior Architect*
>
>
> *[image: Description: Description: Visible-sm]*<http://www.visibletechnologies.com/>
> **
>
> Seattle *| *Boston* | *New York *|* London****
>
> *Phone:* 425.957.6075* | Fax:* 781.404.5711 ****
>
> ** **
>
> *Follow Visibly Intelligent Blog<http://www.visibletechnologies.com/blog/>
> *
>
> ** **
>
> [image: Description: Description: LinkedIn_Logo60px[1]]<http://twitter.com/visible>[image:
> Description: Description: facebook]<http://www.facebook.com/Visible.Technologies>
> [image: Description: Description: in]<http://www.linkedin.com/company/visible-technologies>
> ****
>
> ** **
>