You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "Shaikh, Mazhar A (Mazhar)" <ma...@in.verizon.com> on 2015/11/25 12:26:13 UTC
Kafka broker goes down when consumer is stopped.
Hi Team,
In my test setup, Kafka broker goes down when consumer is stopped.
Below are the version & logs.
Request you help to solve this issue.
Kafka Version : kafka_2.10-0.8.2.1
C++ application using librdkafka_2.10-0.8.2.1 library for consumer & producer.
server.log:
[2015-11-25 05:10:12,798] INFO Closing socket connection to /192.168.108.140. (kafka.network.Processor)
[2015-11-25 05:10:12,798] INFO Closing socket connection to /192.168.108.140. (kafka.network.Processor)
[2015-11-25 05:10:12,798] ERROR Closing socket for /192.168.108.140 because of error (kafka.network.Processor)
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:251)
at sun.nio.ch.IOUtil.read(IOUtil.java:224)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:254)
at kafka.utils.Utils$.read(Utils.scala:380)
at kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
at kafka.network.Processor.read(SocketServer.scala:444)
at kafka.network.Processor.run(SocketServer.scala:340)
at java.lang.Thread.run(Thread.java:701)
[2015-11-25 05:33:58,456] INFO Verifying properties (kafka.utils.VerifiableProperties)
# bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic topic1
Topic:topic1 PartitionCount:16 ReplicationFactor:1 Configs:
Topic: topic1 Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: topic1 Partition: 1 Leader: 0 Replicas: 0 Isr: 0
Topic: topic1 Partition: 2 Leader: 0 Replicas: 0 Isr: 0
Topic: topic1 Partition: 3 Leader: 0 Replicas: 0 Isr: 0
Topic: topic1 Partition: 4 Leader: 0 Replicas: 0 Isr: 0
Topic: topic1 Partition: 5 Leader: 0 Replicas: 0 Isr: 0
Topic: topic1 Partition: 6 Leader: 0 Replicas: 0 Isr: 0
Topic: topic1 Partition: 7 Leader: 0 Replicas: 0 Isr: 0
Topic: topic1 Partition: 8 Leader: 0 Replicas: 0 Isr: 0
Topic: topic1 Partition: 9 Leader: 0 Replicas: 0 Isr: 0
Topic: topic1 Partition: 10 Leader: 0 Replicas: 0 Isr: 0
Topic: topic1 Partition: 11 Leader: 0 Replicas: 0 Isr: 0
Topic: topic1 Partition: 12 Leader: 0 Replicas: 0 Isr: 0
Topic: topic1 Partition: 13 Leader: 0 Replicas: 0 Isr: 0
Topic: topic1 Partition: 14 Leader: 0 Replicas: 0 Isr: 0
Topic: topic1 Partition: 15 Leader: 0 Replicas: 0 Isr: 0
Regards,
Mazhar Shaikh.
Re: Kafka broker goes down when consumer is stopped.
Posted by Gwen Shapira <gw...@confluent.io>.
It looks like you have a single broker (with id = 0 ) and that topic1 has a
single replica and the broker is alive and well.
The socket error is our bug (shouldn't be an error) and doesn't indicate
that the broker is down.
On Wed, Nov 25, 2015 at 3:26 AM, Shaikh, Mazhar A (Mazhar) <
mazhar.shaikh@in.verizon.com> wrote:
> Hi Team,
>
> In my test setup, Kafka broker goes down when consumer is stopped.
>
> Below are the version & logs.
>
> Request you help to solve this issue.
>
>
> Kafka Version : kafka_2.10-0.8.2.1
>
> C++ application using librdkafka_2.10-0.8.2.1 library for consumer &
> producer.
>
>
>
> server.log:
>
>
>
> [2015-11-25 05:10:12,798] INFO Closing socket connection to /
> 192.168.108.140. (kafka.network.Processor)
>
> [2015-11-25 05:10:12,798] INFO Closing socket connection to /
> 192.168.108.140. (kafka.network.Processor)
>
> [2015-11-25 05:10:12,798] ERROR Closing socket for /192.168.108.140
> because of error (kafka.network.Processor)
>
> java.io.IOException: Connection reset by peer
>
> at sun.nio.ch.FileDispatcher.read0(Native Method)
>
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
>
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:251)
>
> at sun.nio.ch.IOUtil.read(IOUtil.java:224)
>
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:254)
>
> at kafka.utils.Utils$.read(Utils.scala:380)
>
> at
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
>
> at kafka.network.Processor.read(SocketServer.scala:444)
>
> at kafka.network.Processor.run(SocketServer.scala:340)
>
> at java.lang.Thread.run(Thread.java:701)
>
> [2015-11-25 05:33:58,456] INFO Verifying properties
> (kafka.utils.VerifiableProperties)
>
>
> # bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic topic1
> Topic:topic1 PartitionCount:16 ReplicationFactor:1 Configs:
> Topic: topic1 Partition: 0 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 1 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 2 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 3 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 4 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 5 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 6 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 7 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 8 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 9 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 10 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 11 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 12 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 13 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 14 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 15 Leader: 0 Replicas: 0
> Isr: 0
>
> Regards,
> Mazhar Shaikh.
>
Re: Kafka broker goes down when consumer is stopped.
Posted by Gaurav Agarwal <ga...@gmail.com>.
Did u check with ps -ef Kafka whether Kafka broker is running or not
On Nov 25, 2015 4:56 PM, "Shaikh, Mazhar A (Mazhar)" <
mazhar.shaikh@in.verizon.com> wrote:
> Hi Team,
>
> In my test setup, Kafka broker goes down when consumer is stopped.
>
> Below are the version & logs.
>
> Request you help to solve this issue.
>
>
> Kafka Version : kafka_2.10-0.8.2.1
>
> C++ application using librdkafka_2.10-0.8.2.1 library for consumer &
> producer.
>
>
>
> server.log:
>
>
>
> [2015-11-25 05:10:12,798] INFO Closing socket connection to /
> 192.168.108.140. (kafka.network.Processor)
>
> [2015-11-25 05:10:12,798] INFO Closing socket connection to /
> 192.168.108.140. (kafka.network.Processor)
>
> [2015-11-25 05:10:12,798] ERROR Closing socket for /192.168.108.140
> because of error (kafka.network.Processor)
>
> java.io.IOException: Connection reset by peer
>
> at sun.nio.ch.FileDispatcher.read0(Native Method)
>
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
>
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:251)
>
> at sun.nio.ch.IOUtil.read(IOUtil.java:224)
>
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:254)
>
> at kafka.utils.Utils$.read(Utils.scala:380)
>
> at
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
>
> at kafka.network.Processor.read(SocketServer.scala:444)
>
> at kafka.network.Processor.run(SocketServer.scala:340)
>
> at java.lang.Thread.run(Thread.java:701)
>
> [2015-11-25 05:33:58,456] INFO Verifying properties
> (kafka.utils.VerifiableProperties)
>
>
> # bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic topic1
> Topic:topic1 PartitionCount:16 ReplicationFactor:1 Configs:
> Topic: topic1 Partition: 0 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 1 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 2 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 3 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 4 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 5 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 6 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 7 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 8 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 9 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 10 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 11 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 12 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 13 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 14 Leader: 0 Replicas: 0
> Isr: 0
> Topic: topic1 Partition: 15 Leader: 0 Replicas: 0
> Isr: 0
>
> Regards,
> Mazhar Shaikh.
>