You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Joseph Lawson <jl...@roomkey.com> on 2013/10/29 01:21:58 UTC

Recovering High Level Consumer from exceptions

Hi everyone,

how should a high level consumer implementation handle consumer exceptions.  Using the high level consumer wiki example, the primary thread calls an example.run() and then waits for a thread.sleep and then shuts down.  If each ConsumerTest is a thread, what is the best way for the parent thread to handle consumer exceptions such as a consumer timeout or a rebalance max retries exception?

I've run into cases where my high level consumer just keeps spinning while my consumer threads are crashed.  Any suggestions as to how I should detect the exceptions from the consumer threads in the high level consumer logic?

Thanks!

-Joe Lawson

Re: Recovering High Level Consumer from exceptions

Posted by Joseph Lawson <jl...@roomkey.com>.

Thanks. i ended up catching any exception in the consumer thread and retrying the iteration unless i specifically toggled it to dump and quit on error.

Sent from my Droid Charge on Verizon 4G LTE Jun Rao wrote:
Normally, consumers don't timeout unless you have configured it. You just
need to make sure you don't kill the consumer thread if it hits an
application error.

Thanks,

Jun

On Mon, Oct 28, 2013 at 5:21 PM, Joseph Lawson <jl...@roomkey.com> wrote:

> Hi everyone,
>
> how should a high level consumer implementation handle consumer
> exceptions.  Using the high level consumer wiki example, the primary thread
> calls an example.run() and then waits for a thread.sleep and then shuts
> down.  If each ConsumerTest is a thread, what is the best way for the
> parent thread to handle consumer exceptions such as a consumer timeout or a
> rebalance max retries exception?
>
> I've run into cases where my high level consumer just keeps spinning while
> my consumer threads are crashed.  Any suggestions as to how I should detect
> the exceptions from the consumer threads in the high level consumer logic?
>
> Thanks!
>
> -Joe Lawson
>

Re: Recovering High Level Consumer from exceptions

Posted by Jun Rao <ju...@gmail.com>.

Normally, consumers don't timeout unless you have configured it. You just
need to make sure you don't kill the consumer thread if it hits an
application error.

Thanks,

Jun


On Mon, Oct 28, 2013 at 5:21 PM, Joseph Lawson <jl...@roomkey.com> wrote:

> Hi everyone,
>
> how should a high level consumer implementation handle consumer
> exceptions.  Using the high level consumer wiki example, the primary thread
> calls an example.run() and then waits for a thread.sleep and then shuts
> down.  If each ConsumerTest is a thread, what is the best way for the
> parent thread to handle consumer exceptions such as a consumer timeout or a
> rebalance max retries exception?
>
> I've run into cases where my high level consumer just keeps spinning while
> my consumer threads are crashed.  Any suggestions as to how I should detect
> the exceptions from the consumer threads in the high level consumer logic?
>
> Thanks!
>
> -Joe Lawson
>