You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Chitra Raveendran <ch...@fluturasolutions.com> on 2013/05/17 07:21:08 UTC
What happens if one broker goes down
Hi
I was just doing some benchmarking with a 3node cluster. If one broker goes
down , what happens? Does the producer go down ? that's what happened in my
case.
Is the data lost? Or does it get distributed amongst the other brokers?
--
Chitra Raveendran
Data Scientist
*Flutura | Decision Sciences & Analytics*
mail : chitra.raveendran@fluturasoutions.com
Re: What happens if one broker goes down
Posted by Neha Narkhede <ne...@gmail.com>.
0.7.2 does not support replication. So when a broker goes down, there can
be some data loss. If you are ok with duplicates, you can configure the
producer side retries to be higher.
Thanks,
Neha
On May 19, 2013 11:32 PM, "Chitra Raveendran" <
chitra.raveendran@fluturasolutions.com> wrote:
> HI
>
> When my broker went down, my producer just stopped reading the file giving
> an exception saying "connection refused to kafka broker".
> I tried overcoming the exception by giving a try and catch ! BUT I'M SEEING
> DATA LOSS. What am I doing wrong ??? Please help
>
> For reference :
> I'm using kafka 0.7.2 since my consumer is a storm spout and i read that
> storm is most compatible with kafka 0.7.2.
> IT DOESNT SUPPORT REPLICATION RIGHT?
>
>
> On Fri, May 17, 2013 at 6:56 PM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
>
> > You can read the high level design of kafka replication here
> > http://www.slideshare.net/junrao/kafka-replication-apachecon2013
> >
> > Generally if your replication factor is more than 1 you shouldn't see
> data
> > loss in your test. When a broker fails, the producer will get an
> exception
> > and it will retry.
> >
> > Thanks,
> > Neha
> > On May 16, 2013 10:21 PM, "Chitra Raveendran" <
> > chitra.raveendran@fluturasolutions.com> wrote:
> >
> > > Hi
> > >
> > > I was just doing some benchmarking with a 3node cluster. If one broker
> > goes
> > > down , what happens? Does the producer go down ? that's what happened
> in
> > my
> > > case.
> > > Is the data lost? Or does it get distributed amongst the other brokers?
> > >
> > > --
> > > Chitra Raveendran
> > > Data Scientist
> > > *Flutura | Decision Sciences & Analytics*
> > > mail : chitra.raveendran@fluturasoutions.com
> > >
> >
>
>
>
> --
> Chitra Raveendran
> Data Scientist
> *Flutura Business Solutions Pvt. Ltd*
> Tel : +918197563660
> email : chitra.raveendran@fluturasoutions.com
>
Re: What happens if one broker goes down
Posted by Chitra Raveendran <ch...@fluturasolutions.com>.
HI
When my broker went down, my producer just stopped reading the file giving
an exception saying "connection refused to kafka broker".
I tried overcoming the exception by giving a try and catch ! BUT I'M SEEING
DATA LOSS. What am I doing wrong ??? Please help
For reference :
I'm using kafka 0.7.2 since my consumer is a storm spout and i read that
storm is most compatible with kafka 0.7.2.
IT DOESNT SUPPORT REPLICATION RIGHT?
On Fri, May 17, 2013 at 6:56 PM, Neha Narkhede <ne...@gmail.com>wrote:
> You can read the high level design of kafka replication here
> http://www.slideshare.net/junrao/kafka-replication-apachecon2013
>
> Generally if your replication factor is more than 1 you shouldn't see data
> loss in your test. When a broker fails, the producer will get an exception
> and it will retry.
>
> Thanks,
> Neha
> On May 16, 2013 10:21 PM, "Chitra Raveendran" <
> chitra.raveendran@fluturasolutions.com> wrote:
>
> > Hi
> >
> > I was just doing some benchmarking with a 3node cluster. If one broker
> goes
> > down , what happens? Does the producer go down ? that's what happened in
> my
> > case.
> > Is the data lost? Or does it get distributed amongst the other brokers?
> >
> > --
> > Chitra Raveendran
> > Data Scientist
> > *Flutura | Decision Sciences & Analytics*
> > mail : chitra.raveendran@fluturasoutions.com
> >
>
--
Chitra Raveendran
Data Scientist
*Flutura Business Solutions Pvt. Ltd*
Tel : +918197563660
email : chitra.raveendran@fluturasoutions.com
Re: What happens if one broker goes down
Posted by Neha Narkhede <ne...@gmail.com>.
You can read the high level design of kafka replication here
http://www.slideshare.net/junrao/kafka-replication-apachecon2013
Generally if your replication factor is more than 1 you shouldn't see data
loss in your test. When a broker fails, the producer will get an exception
and it will retry.
Thanks,
Neha
On May 16, 2013 10:21 PM, "Chitra Raveendran" <
chitra.raveendran@fluturasolutions.com> wrote:
> Hi
>
> I was just doing some benchmarking with a 3node cluster. If one broker goes
> down , what happens? Does the producer go down ? that's what happened in my
> case.
> Is the data lost? Or does it get distributed amongst the other brokers?
>
> --
> Chitra Raveendran
> Data Scientist
> *Flutura | Decision Sciences & Analytics*
> mail : chitra.raveendran@fluturasoutions.com
>