You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "Luo, Chao" <ch...@here.com> on 2016/07/18 15:59:20 UTC
Kafka Fault Tolerance Test
Dear Kafka fans,
I have a concern of testing Kafka fault tolerance. Or may I did not configure it right.
I have two kafka servers and one zookeeper, which are running on three different AWS EC2 instances. I created a topic with one partition and two replica. First, the two kafka servers were running and everything was perfect. After I shut down the leader for my topic, the producer and the consumer just stopped working. Why did Kafka system stop working? I did not see any fault tolerance here. However, when I restarted the leader (I just shut down ), the producer and consumer started to work again. Here is my topic description:
Topic:fast-messages PartitionCount:1 ReplicationFactor:2 Configs:
Topic: fast-messages Partition: 0 Leader: 1 Replicas: 1,0 Isr: 1,0
Any comments or suggestions are highly appreciated!
Thanks!
Chao
Re: Kafka Fault Tolerance Test
Posted by Tom Crayford <tc...@heroku.com>.
Also which version of Kafka are you using?
On Mon, Jul 18, 2016 at 7:16 PM, Guozhang Wang <wa...@gmail.com> wrote:
> This is un-expected. Any error logs / exceptions did you see from the
> clients when they can no longer send / fetch from brokers?
>
> Guozhang
>
> On Mon, Jul 18, 2016 at 8:59 AM, Luo, Chao <ch...@here.com> wrote:
>
> > Dear Kafka fans,
> >
> > I have a concern of testing Kafka fault tolerance. Or may I did not
> > configure it right.
> >
> > I have two kafka servers and one zookeeper, which are running on three
> > different AWS EC2 instances. I created a topic with one partition and two
> > replica. First, the two kafka servers were running and everything was
> > perfect. After I shut down the leader for my topic, the producer and the
> > consumer just stopped working. Why did Kafka system stop working? I did
> not
> > see any fault tolerance here. However, when I restarted the leader (I
> just
> > shut down ), the producer and consumer started to work again. Here is my
> > topic description:
> >
> > Topic:fast-messages PartitionCount:1 ReplicationFactor:2
> > Configs:
> > Topic: fast-messages Partition: 0 Leader: 1 Replicas:
> > 1,0 Isr: 1,0
> >
> >
> >
> > Any comments or suggestions are highly appreciated!
> >
> > Thanks!
> > Chao
> >
>
>
>
> --
> -- Guozhang
>
RE: Kafka Fault Tolerance Test
Posted by "Luo, Chao" <ch...@here.com>.
Hi Guozhang,
Thanks for your reply. Now the system is working fine. I did not use the right topic for testing, which only has one replica. Appreciated!!
Best,
Chao
-----Original Message-----
From: Guozhang Wang [mailto:wangguoz@gmail.com]
Sent: Monday, July 18, 2016 1:17 PM
To: users@kafka.apache.org
Subject: Re: Kafka Fault Tolerance Test
This is un-expected. Any error logs / exceptions did you see from the clients when they can no longer send / fetch from brokers?
Guozhang
On Mon, Jul 18, 2016 at 8:59 AM, Luo, Chao <ch...@here.com> wrote:
> Dear Kafka fans,
>
> I have a concern of testing Kafka fault tolerance. Or may I did not
> configure it right.
>
> I have two kafka servers and one zookeeper, which are running on three
> different AWS EC2 instances. I created a topic with one partition and
> two replica. First, the two kafka servers were running and everything
> was perfect. After I shut down the leader for my topic, the producer
> and the consumer just stopped working. Why did Kafka system stop
> working? I did not see any fault tolerance here. However, when I
> restarted the leader (I just shut down ), the producer and consumer
> started to work again. Here is my topic description:
>
> Topic:fast-messages PartitionCount:1 ReplicationFactor:2
> Configs:
> Topic: fast-messages Partition: 0 Leader: 1 Replicas:
> 1,0 Isr: 1,0
>
>
>
> Any comments or suggestions are highly appreciated!
>
> Thanks!
> Chao
>
--
-- Guozhang
Re: Kafka Fault Tolerance Test
Posted by Guozhang Wang <wa...@gmail.com>.
This is un-expected. Any error logs / exceptions did you see from the
clients when they can no longer send / fetch from brokers?
Guozhang
On Mon, Jul 18, 2016 at 8:59 AM, Luo, Chao <ch...@here.com> wrote:
> Dear Kafka fans,
>
> I have a concern of testing Kafka fault tolerance. Or may I did not
> configure it right.
>
> I have two kafka servers and one zookeeper, which are running on three
> different AWS EC2 instances. I created a topic with one partition and two
> replica. First, the two kafka servers were running and everything was
> perfect. After I shut down the leader for my topic, the producer and the
> consumer just stopped working. Why did Kafka system stop working? I did not
> see any fault tolerance here. However, when I restarted the leader (I just
> shut down ), the producer and consumer started to work again. Here is my
> topic description:
>
> Topic:fast-messages PartitionCount:1 ReplicationFactor:2
> Configs:
> Topic: fast-messages Partition: 0 Leader: 1 Replicas:
> 1,0 Isr: 1,0
>
>
>
> Any comments or suggestions are highly appreciated!
>
> Thanks!
> Chao
>
--
-- Guozhang