Detecting lost connection in high level consumer

[harikiran <ha...@gmail.com>]

Hi

I am using the 0811 Kafka High level consumer and I have configured "
consumer.timeout.ms" to a value that is not -1, say 5000ms.

I create the consumer iterator and invoke hasNext() method on it.

Irrespective of whether kafka broker was shutdown or there was no message
written to kafka, I see a ConsumerTimeOut exception after 5000ms.

My goal is to detect lost connection and reconnect but I cannot figure out
a way.

Any kind of help is appreciated.

Thanks
Hari

Re: Detecting lost connection in high level consumer

[Guozhang Wang <wa...@gmail.com>]

Hi Hari,

For high level consumer the fetching logic is handled by a background
fetcher thread and is hidden from user, for either case of 1) broker down
or 2) no message is available the fetcher thread will keep retrying while
the user thread will wait on the fetcher thread to put some data into the
buffer until timeout. So in a sentence the high-level consumer design is to
not let users worry about connect / reconnect issues.

Guozhang

On Mon, Jan 19, 2015 at 1:26 AM, harikiran <ha...@gmail.com> wrote:

> Hi
>
> I am using the 0811 Kafka High level consumer and I have configured "
> consumer.timeout.ms" to a value that is not -1, say 5000ms.
>
> I create the consumer iterator and invoke hasNext() method on it.
>
> Irrespective of whether kafka broker was shutdown or there was no message
> written to kafka, I see a ConsumerTimeOut exception after 5000ms.
>
> My goal is to detect lost connection and reconnect but I cannot figure out
> a way.
>
> Any kind of help is appreciated.
>
> Thanks
> Hari
>



-- 
-- Guozhang

Re: Detecting lost connection in high level consumer

[harikiran <ha...@gmail.com>]

Thanks Guozhang and Jun for your replies.

On Wed, Feb 4, 2015 at 4:06 PM, harikiran <ha...@gmail.com> wrote:

> Hey! Did anyone get a chance to read this?
>
> Thanks!!
>
> On Mon, Jan 19, 2015 at 1:26 AM harikiran <ha...@gmail.com> wrote:
>
>> Hi
>>
>> I am using the 0811 Kafka High level consumer and I have configured "
>> consumer.timeout.ms" to a value that is not -1, say 5000ms.
>>
>> I create the consumer iterator and invoke hasNext() method on it.
>>
>> Irrespective of whether kafka broker was shutdown or there was no message
>> written to kafka, I see a ConsumerTimeOut exception after 5000ms.
>>
>> My goal is to detect lost connection and reconnect but I cannot figure
>> out a way.
>>
>> Any kind of help is appreciated.
>>
>> Thanks
>> Hari
>>
>

Re: Detecting lost connection in high level consumer

[harikiran <ha...@gmail.com>]

Hey! Did anyone get a chance to read this?

Thanks!!
On Mon, Jan 19, 2015 at 1:26 AM harikiran <ha...@gmail.com> wrote:

> Hi
>
> I am using the 0811 Kafka High level consumer and I have configured "
> consumer.timeout.ms" to a value that is not -1, say 5000ms.
>
> I create the consumer iterator and invoke hasNext() method on it.
>
> Irrespective of whether kafka broker was shutdown or there was no message
> written to kafka, I see a ConsumerTimeOut exception after 5000ms.
>
> My goal is to detect lost connection and reconnect but I cannot figure out
> a way.
>
> Any kind of help is appreciated.
>
> Thanks
> Hari
>