You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Alper Akture <al...@goldenratstudios.com> on 2016/08/06 02:45:04 UTC
Testing broker failover
I'm using 0.10.0.0 and testing some failover scenarios. For dev, i have
single kafka node and a zookeeper instance. While sending events to a
topic, I shutdown the broker to see if my failover handling works. However,
I don't see any indication that the send failed, but I do see the
connection refused errors logged at debug. What is the standard way to
detect a message send failure, and handle it for offline processing later?
Here's the debug output I see:
19:20:00.906 [kafka-producer-network-thread | producer-1] DEBUG
org.apache.kafka.clients.NetworkClient - Initialize connection to node -1
for sending metadata request
19:20:00.906 [kafka-producer-network-thread | producer-1] DEBUG
org.apache.kafka.clients.NetworkClient - Initiating connection to node -1
at localhost:9092.
19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
org.apache.kafka.common.network.Selector - Connection with localhost/
127.0.0.1 disconnected
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_66]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
~[?:1.8.0_66]
at
org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:51)
~[kafka-clients-0.10.0.0.jar:?]
at
org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:73)
~[kafka-clients-0.10.0.0.jar:?]
at
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:309)
[kafka-clients-0.10.0.0.jar:?]
at org.apache.kafka.common.network.Selector.poll(Selector.java:283)
[kafka-clients-0.10.0.0.jar:?]
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:260)
[kafka-clients-0.10.0.0.jar:?]
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:229)
[kafka-clients-0.10.0.0.jar:?]
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:134)
[kafka-clients-0.10.0.0.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_66]
19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
org.apache.kafka.clients.NetworkClient - Node -1 disconnected.
19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
org.apache.kafka.clients.NetworkClient - Give up sending metadata request
since no node is available
Re: Testing broker failover
Posted by Alper Akture <al...@goldenratstudios.com>.
Got, it thanks, exactly what I had just noticed and sent a reply as you
were replying... thanks Alex.
On Mon, Aug 8, 2016 at 11:35 AM, Alex Loddengaard <al...@confluent.io> wrote:
> Hi Alper,
>
> Thanks for sharing. I was particularly interested in seeing what *acks* was
> set to. Since you haven't set it, its value is the default, *1*.
>
> To handle errors, you need to use the send() method that takes a callback,
> and build an appropriate callback to handle errors. Take a look here for an
> example:
>
> http://kafka.apache.org/0100/javadoc/org/apache/kafka/clients/producer/
> KafkaProducer.html#send(org.apache.kafka.clients.producer.
> ProducerRecord,%20org.apache.kafka.clients.producer.Callback)
>
> Let us know if you have follow-up questions.
>
> Alex
>
> On Mon, Aug 8, 2016 at 11:24 AM, Alper Akture <al...@goldenratstudios.com>
> wrote:
>
> > Thanks Alex... using producer props:
> >
> > {timeout.ms=500, max.block.ms=500, request.timeout.ms=500,
> > bootstrap.servers=localhost:9092,
> > serializer.class=kafka.serializer.StringEncoder,
> > value.serializer=org.apache.kafka.common.serialization.StringSerializer,
> > metadata.fetch.timeout.ms=500,
> > key.serializer=org.apache.kafka.common.serialization.StringSerializer}
> >
> >
> >
> >
> > On Mon, Aug 8, 2016 at 9:21 AM, Alex Loddengaard <al...@confluent.io>
> > wrote:
> >
> > > Hi Alper, can you share your producer config -- the Properties object?
> We
> > > need to learn more to help you understand the behavior you're
> observing.
> > >
> > > Thanks,
> > >
> > > Alex
> > >
> > > On Fri, Aug 5, 2016 at 7:45 PM, Alper Akture <
> alper@goldenratstudios.com
> > >
> > > wrote:
> > >
> > > > I'm using 0.10.0.0 and testing some failover scenarios. For dev, i
> have
> > > > single kafka node and a zookeeper instance. While sending events to a
> > > > topic, I shutdown the broker to see if my failover handling works.
> > > However,
> > > > I don't see any indication that the send failed, but I do see the
> > > > connection refused errors logged at debug. What is the standard way
> to
> > > > detect a message send failure, and handle it for offline processing
> > > later?
> > > >
> > > > Here's the debug output I see:
> > > >
> > > > 19:20:00.906 [kafka-producer-network-thread | producer-1] DEBUG
> > > > org.apache.kafka.clients.NetworkClient - Initialize connection to
> node
> > > -1
> > > > for sending metadata request
> > > > 19:20:00.906 [kafka-producer-network-thread | producer-1] DEBUG
> > > > org.apache.kafka.clients.NetworkClient - Initiating connection to
> node
> > > -1
> > > > at localhost:9092.
> > > > 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
> > > > org.apache.kafka.common.network.Selector - Connection with
> localhost/
> > > > 127.0.0.1 disconnected
> > > > java.net.ConnectException: Connection refused
> > > > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > > ~[?:1.8.0_66]
> > > > at sun.nio.ch.SocketChannelImpl.finishConnect(
> > > SocketChannelImpl.java:717)
> > > > ~[?:1.8.0_66]
> > > > at
> > > > org.apache.kafka.common.network.PlaintextTransportLayer.
> finishConnect(
> > > > PlaintextTransportLayer.java:51)
> > > > ~[kafka-clients-0.10.0.0.jar:?]
> > > > at
> > > > org.apache.kafka.common.network.KafkaChannel.
> > finishConnect(KafkaChannel.
> > > > java:73)
> > > > ~[kafka-clients-0.10.0.0.jar:?]
> > > > at
> > > > org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.
> > > > java:309)
> > > > [kafka-clients-0.10.0.0.jar:?]
> > > > at org.apache.kafka.common.network.Selector.poll(Selector.java:283)
> > > > [kafka-clients-0.10.0.0.jar:?]
> > > > at org.apache.kafka.clients.NetworkClient.poll(
> NetworkClient.java:260)
> > > > [kafka-clients-0.10.0.0.jar:?]
> > > > at org.apache.kafka.clients.producer.internals.Sender.run(
> > > Sender.java:229)
> > > > [kafka-clients-0.10.0.0.jar:?]
> > > > at org.apache.kafka.clients.producer.internals.Sender.run(
> > > Sender.java:134)
> > > > [kafka-clients-0.10.0.0.jar:?]
> > > > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_66]
> > > > 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
> > > > org.apache.kafka.clients.NetworkClient - Node -1 disconnected.
> > > > 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
> > > > org.apache.kafka.clients.NetworkClient - Give up sending metadata
> > > request
> > > > since no node is available
> > > >
> > >
> >
>
Re: Testing broker failover
Posted by Alex Loddengaard <al...@confluent.io>.
Hi Alper,
Thanks for sharing. I was particularly interested in seeing what *acks* was
set to. Since you haven't set it, its value is the default, *1*.
To handle errors, you need to use the send() method that takes a callback,
and build an appropriate callback to handle errors. Take a look here for an
example:
http://kafka.apache.org/0100/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html#send(org.apache.kafka.clients.producer.ProducerRecord,%20org.apache.kafka.clients.producer.Callback)
Let us know if you have follow-up questions.
Alex
On Mon, Aug 8, 2016 at 11:24 AM, Alper Akture <al...@goldenratstudios.com>
wrote:
> Thanks Alex... using producer props:
>
> {timeout.ms=500, max.block.ms=500, request.timeout.ms=500,
> bootstrap.servers=localhost:9092,
> serializer.class=kafka.serializer.StringEncoder,
> value.serializer=org.apache.kafka.common.serialization.StringSerializer,
> metadata.fetch.timeout.ms=500,
> key.serializer=org.apache.kafka.common.serialization.StringSerializer}
>
>
>
>
> On Mon, Aug 8, 2016 at 9:21 AM, Alex Loddengaard <al...@confluent.io>
> wrote:
>
> > Hi Alper, can you share your producer config -- the Properties object? We
> > need to learn more to help you understand the behavior you're observing.
> >
> > Thanks,
> >
> > Alex
> >
> > On Fri, Aug 5, 2016 at 7:45 PM, Alper Akture <alper@goldenratstudios.com
> >
> > wrote:
> >
> > > I'm using 0.10.0.0 and testing some failover scenarios. For dev, i have
> > > single kafka node and a zookeeper instance. While sending events to a
> > > topic, I shutdown the broker to see if my failover handling works.
> > However,
> > > I don't see any indication that the send failed, but I do see the
> > > connection refused errors logged at debug. What is the standard way to
> > > detect a message send failure, and handle it for offline processing
> > later?
> > >
> > > Here's the debug output I see:
> > >
> > > 19:20:00.906 [kafka-producer-network-thread | producer-1] DEBUG
> > > org.apache.kafka.clients.NetworkClient - Initialize connection to node
> > -1
> > > for sending metadata request
> > > 19:20:00.906 [kafka-producer-network-thread | producer-1] DEBUG
> > > org.apache.kafka.clients.NetworkClient - Initiating connection to node
> > -1
> > > at localhost:9092.
> > > 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
> > > org.apache.kafka.common.network.Selector - Connection with localhost/
> > > 127.0.0.1 disconnected
> > > java.net.ConnectException: Connection refused
> > > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> > ~[?:1.8.0_66]
> > > at sun.nio.ch.SocketChannelImpl.finishConnect(
> > SocketChannelImpl.java:717)
> > > ~[?:1.8.0_66]
> > > at
> > > org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(
> > > PlaintextTransportLayer.java:51)
> > > ~[kafka-clients-0.10.0.0.jar:?]
> > > at
> > > org.apache.kafka.common.network.KafkaChannel.
> finishConnect(KafkaChannel.
> > > java:73)
> > > ~[kafka-clients-0.10.0.0.jar:?]
> > > at
> > > org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.
> > > java:309)
> > > [kafka-clients-0.10.0.0.jar:?]
> > > at org.apache.kafka.common.network.Selector.poll(Selector.java:283)
> > > [kafka-clients-0.10.0.0.jar:?]
> > > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:260)
> > > [kafka-clients-0.10.0.0.jar:?]
> > > at org.apache.kafka.clients.producer.internals.Sender.run(
> > Sender.java:229)
> > > [kafka-clients-0.10.0.0.jar:?]
> > > at org.apache.kafka.clients.producer.internals.Sender.run(
> > Sender.java:134)
> > > [kafka-clients-0.10.0.0.jar:?]
> > > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_66]
> > > 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
> > > org.apache.kafka.clients.NetworkClient - Node -1 disconnected.
> > > 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
> > > org.apache.kafka.clients.NetworkClient - Give up sending metadata
> > request
> > > since no node is available
> > >
> >
>
Re: Testing broker failover
Posted by Alper Akture <al...@goldenratstudios.com>.
(continued from previous email)
Hit send too soon... but I also notice that if I use the producer send
method that takes a callback, in the onCompletion method of the callback, I
do see an exception is sent in the callback:
org.apache.kafka.common.errors.TimeoutException: Batch containing 2
record(s) expired due to timeout while requesting metadata from brokers for
BACCARAT_ROUND_END-0
On Mon, Aug 8, 2016 at 11:34 AM, Alper Akture <al...@goldenratstudios.com>
wrote:
> I also notice that if I use the send method that takes a callback: public
> Future<RecordMetadata> send(ProducerRecord<K, V> record, Callback
> callback) {
> call
> that in the onCompletionalso all
>
> On Mon, Aug 8, 2016 at 11:24 AM, Alper Akture <al...@goldenratstudios.com>
> wrote:
>
>> Thanks Alex... using producer props:
>>
>> {timeout.ms=500, max.block.ms=500, request.timeout.ms=500,
>> bootstrap.servers=localhost:9092, serializer.class=kafka.serializer.StringEncoder,
>> value.serializer=org.apache.kafka.common.serialization.StringSerializer,
>> metadata.fetch.timeout.ms=500, key.serializer=org.apache.kafk
>> a.common.serialization.StringSerializer}
>>
>>
>>
>>
>> On Mon, Aug 8, 2016 at 9:21 AM, Alex Loddengaard <al...@confluent.io>
>> wrote:
>>
>>> Hi Alper, can you share your producer config -- the Properties object? We
>>> need to learn more to help you understand the behavior you're observing.
>>>
>>> Thanks,
>>>
>>> Alex
>>>
>>> On Fri, Aug 5, 2016 at 7:45 PM, Alper Akture <alper@goldenratstudios.com
>>> >
>>> wrote:
>>>
>>> > I'm using 0.10.0.0 and testing some failover scenarios. For dev, i have
>>> > single kafka node and a zookeeper instance. While sending events to a
>>> > topic, I shutdown the broker to see if my failover handling works.
>>> However,
>>> > I don't see any indication that the send failed, but I do see the
>>> > connection refused errors logged at debug. What is the standard way to
>>> > detect a message send failure, and handle it for offline processing
>>> later?
>>> >
>>> > Here's the debug output I see:
>>> >
>>> > 19:20:00.906 [kafka-producer-network-thread | producer-1] DEBUG
>>> > org.apache.kafka.clients.NetworkClient - Initialize connection to
>>> node -1
>>> > for sending metadata request
>>> > 19:20:00.906 [kafka-producer-network-thread | producer-1] DEBUG
>>> > org.apache.kafka.clients.NetworkClient - Initiating connection to
>>> node -1
>>> > at localhost:9092.
>>> > 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
>>> > org.apache.kafka.common.network.Selector - Connection with localhost/
>>> > 127.0.0.1 disconnected
>>> > java.net.ConnectException: Connection refused
>>> > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>>> ~[?:1.8.0_66]
>>> > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl
>>> .java:717)
>>> > ~[?:1.8.0_66]
>>> > at
>>> > org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(
>>> > PlaintextTransportLayer.java:51)
>>> > ~[kafka-clients-0.10.0.0.jar:?]
>>> > at
>>> > org.apache.kafka.common.network.KafkaChannel.finishConnect(K
>>> afkaChannel.
>>> > java:73)
>>> > ~[kafka-clients-0.10.0.0.jar:?]
>>> > at
>>> > org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.
>>> > java:309)
>>> > [kafka-clients-0.10.0.0.jar:?]
>>> > at org.apache.kafka.common.network.Selector.poll(Selector.java:283)
>>> > [kafka-clients-0.10.0.0.jar:?]
>>> > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:260)
>>> > [kafka-clients-0.10.0.0.jar:?]
>>> > at org.apache.kafka.clients.producer.internals.Sender.run(Sende
>>> r.java:229)
>>> > [kafka-clients-0.10.0.0.jar:?]
>>> > at org.apache.kafka.clients.producer.internals.Sender.run(Sende
>>> r.java:134)
>>> > [kafka-clients-0.10.0.0.jar:?]
>>> > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_66]
>>> > 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
>>> > org.apache.kafka.clients.NetworkClient - Node -1 disconnected.
>>> > 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
>>> > org.apache.kafka.clients.NetworkClient - Give up sending metadata
>>> request
>>> > since no node is available
>>> >
>>>
>>
>>
>
Re: Testing broker failover
Posted by Alper Akture <al...@goldenratstudios.com>.
I also notice that if I use the send method that takes a callback:
public Future<RecordMetadata>
send(ProducerRecord<K, V> record, Callback callback) {
call
that in the onCompletionalso all
On Mon, Aug 8, 2016 at 11:24 AM, Alper Akture <al...@goldenratstudios.com>
wrote:
> Thanks Alex... using producer props:
>
> {timeout.ms=500, max.block.ms=500, request.timeout.ms=500,
> bootstrap.servers=localhost:9092, serializer.class=kafka.serializer.StringEncoder,
> value.serializer=org.apache.kafka.common.serialization.StringSerializer,
> metadata.fetch.timeout.ms=500, key.serializer=org.apache.
> kafka.common.serialization.StringSerializer}
>
>
>
>
> On Mon, Aug 8, 2016 at 9:21 AM, Alex Loddengaard <al...@confluent.io>
> wrote:
>
>> Hi Alper, can you share your producer config -- the Properties object? We
>> need to learn more to help you understand the behavior you're observing.
>>
>> Thanks,
>>
>> Alex
>>
>> On Fri, Aug 5, 2016 at 7:45 PM, Alper Akture <al...@goldenratstudios.com>
>> wrote:
>>
>> > I'm using 0.10.0.0 and testing some failover scenarios. For dev, i have
>> > single kafka node and a zookeeper instance. While sending events to a
>> > topic, I shutdown the broker to see if my failover handling works.
>> However,
>> > I don't see any indication that the send failed, but I do see the
>> > connection refused errors logged at debug. What is the standard way to
>> > detect a message send failure, and handle it for offline processing
>> later?
>> >
>> > Here's the debug output I see:
>> >
>> > 19:20:00.906 [kafka-producer-network-thread | producer-1] DEBUG
>> > org.apache.kafka.clients.NetworkClient - Initialize connection to node
>> -1
>> > for sending metadata request
>> > 19:20:00.906 [kafka-producer-network-thread | producer-1] DEBUG
>> > org.apache.kafka.clients.NetworkClient - Initiating connection to node
>> -1
>> > at localhost:9092.
>> > 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
>> > org.apache.kafka.common.network.Selector - Connection with localhost/
>> > 127.0.0.1 disconnected
>> > java.net.ConnectException: Connection refused
>> > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>> ~[?:1.8.0_66]
>> > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl
>> .java:717)
>> > ~[?:1.8.0_66]
>> > at
>> > org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(
>> > PlaintextTransportLayer.java:51)
>> > ~[kafka-clients-0.10.0.0.jar:?]
>> > at
>> > org.apache.kafka.common.network.KafkaChannel.finishConnect(
>> KafkaChannel.
>> > java:73)
>> > ~[kafka-clients-0.10.0.0.jar:?]
>> > at
>> > org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.
>> > java:309)
>> > [kafka-clients-0.10.0.0.jar:?]
>> > at org.apache.kafka.common.network.Selector.poll(Selector.java:283)
>> > [kafka-clients-0.10.0.0.jar:?]
>> > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:260)
>> > [kafka-clients-0.10.0.0.jar:?]
>> > at org.apache.kafka.clients.producer.internals.Sender.run(Sende
>> r.java:229)
>> > [kafka-clients-0.10.0.0.jar:?]
>> > at org.apache.kafka.clients.producer.internals.Sender.run(Sende
>> r.java:134)
>> > [kafka-clients-0.10.0.0.jar:?]
>> > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_66]
>> > 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
>> > org.apache.kafka.clients.NetworkClient - Node -1 disconnected.
>> > 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
>> > org.apache.kafka.clients.NetworkClient - Give up sending metadata
>> request
>> > since no node is available
>> >
>>
>
>
Re: Testing broker failover
Posted by Alper Akture <al...@goldenratstudios.com>.
Thanks Alex... using producer props:
{timeout.ms=500, max.block.ms=500, request.timeout.ms=500,
bootstrap.servers=localhost:9092,
serializer.class=kafka.serializer.StringEncoder,
value.serializer=org.apache.kafka.common.serialization.StringSerializer,
metadata.fetch.timeout.ms=500,
key.serializer=org.apache.kafka.common.serialization.StringSerializer}
On Mon, Aug 8, 2016 at 9:21 AM, Alex Loddengaard <al...@confluent.io> wrote:
> Hi Alper, can you share your producer config -- the Properties object? We
> need to learn more to help you understand the behavior you're observing.
>
> Thanks,
>
> Alex
>
> On Fri, Aug 5, 2016 at 7:45 PM, Alper Akture <al...@goldenratstudios.com>
> wrote:
>
> > I'm using 0.10.0.0 and testing some failover scenarios. For dev, i have
> > single kafka node and a zookeeper instance. While sending events to a
> > topic, I shutdown the broker to see if my failover handling works.
> However,
> > I don't see any indication that the send failed, but I do see the
> > connection refused errors logged at debug. What is the standard way to
> > detect a message send failure, and handle it for offline processing
> later?
> >
> > Here's the debug output I see:
> >
> > 19:20:00.906 [kafka-producer-network-thread | producer-1] DEBUG
> > org.apache.kafka.clients.NetworkClient - Initialize connection to node
> -1
> > for sending metadata request
> > 19:20:00.906 [kafka-producer-network-thread | producer-1] DEBUG
> > org.apache.kafka.clients.NetworkClient - Initiating connection to node
> -1
> > at localhost:9092.
> > 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
> > org.apache.kafka.common.network.Selector - Connection with localhost/
> > 127.0.0.1 disconnected
> > java.net.ConnectException: Connection refused
> > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> ~[?:1.8.0_66]
> > at sun.nio.ch.SocketChannelImpl.finishConnect(
> SocketChannelImpl.java:717)
> > ~[?:1.8.0_66]
> > at
> > org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(
> > PlaintextTransportLayer.java:51)
> > ~[kafka-clients-0.10.0.0.jar:?]
> > at
> > org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.
> > java:73)
> > ~[kafka-clients-0.10.0.0.jar:?]
> > at
> > org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.
> > java:309)
> > [kafka-clients-0.10.0.0.jar:?]
> > at org.apache.kafka.common.network.Selector.poll(Selector.java:283)
> > [kafka-clients-0.10.0.0.jar:?]
> > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:260)
> > [kafka-clients-0.10.0.0.jar:?]
> > at org.apache.kafka.clients.producer.internals.Sender.run(
> Sender.java:229)
> > [kafka-clients-0.10.0.0.jar:?]
> > at org.apache.kafka.clients.producer.internals.Sender.run(
> Sender.java:134)
> > [kafka-clients-0.10.0.0.jar:?]
> > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_66]
> > 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
> > org.apache.kafka.clients.NetworkClient - Node -1 disconnected.
> > 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
> > org.apache.kafka.clients.NetworkClient - Give up sending metadata
> request
> > since no node is available
> >
>
Re: Testing broker failover
Posted by Alex Loddengaard <al...@confluent.io>.
Hi Alper, can you share your producer config -- the Properties object? We
need to learn more to help you understand the behavior you're observing.
Thanks,
Alex
On Fri, Aug 5, 2016 at 7:45 PM, Alper Akture <al...@goldenratstudios.com>
wrote:
> I'm using 0.10.0.0 and testing some failover scenarios. For dev, i have
> single kafka node and a zookeeper instance. While sending events to a
> topic, I shutdown the broker to see if my failover handling works. However,
> I don't see any indication that the send failed, but I do see the
> connection refused errors logged at debug. What is the standard way to
> detect a message send failure, and handle it for offline processing later?
>
> Here's the debug output I see:
>
> 19:20:00.906 [kafka-producer-network-thread | producer-1] DEBUG
> org.apache.kafka.clients.NetworkClient - Initialize connection to node -1
> for sending metadata request
> 19:20:00.906 [kafka-producer-network-thread | producer-1] DEBUG
> org.apache.kafka.clients.NetworkClient - Initiating connection to node -1
> at localhost:9092.
> 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
> org.apache.kafka.common.network.Selector - Connection with localhost/
> 127.0.0.1 disconnected
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_66]
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> ~[?:1.8.0_66]
> at
> org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(
> PlaintextTransportLayer.java:51)
> ~[kafka-clients-0.10.0.0.jar:?]
> at
> org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.
> java:73)
> ~[kafka-clients-0.10.0.0.jar:?]
> at
> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.
> java:309)
> [kafka-clients-0.10.0.0.jar:?]
> at org.apache.kafka.common.network.Selector.poll(Selector.java:283)
> [kafka-clients-0.10.0.0.jar:?]
> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:260)
> [kafka-clients-0.10.0.0.jar:?]
> at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:229)
> [kafka-clients-0.10.0.0.jar:?]
> at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:134)
> [kafka-clients-0.10.0.0.jar:?]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_66]
> 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
> org.apache.kafka.clients.NetworkClient - Node -1 disconnected.
> 19:20:00.907 [kafka-producer-network-thread | producer-1] DEBUG
> org.apache.kafka.clients.NetworkClient - Give up sending metadata request
> since no node is available
>