How are rebalance failures raised to consumers?

[Tom Amon <ta...@gmail.com>]

The pattern for creating and operating consumers that we use is to create
the consumer connector, create the streams and then consume each stream by
waiting on the iterator.

If a rebalance occurs and fails, how is the error raised to the consumer?
Will I get an exception while waiting on the iterator? Is it swallowed and
the consumer is dead?

Thanks.

Re: How are rebalance failures raised to consumers?

[Neha Narkhede <ne...@gmail.com>]

What Jun described is correct, currently the consumer asynchronously fails
rebalance. But you can monitor the consumer to detect when a rebalance
operation fails since that will manifest as lag on the consumer. Please see
this<https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Myconsumerseemstohavestopped,why?>to
learn about consumer lag monitoring.

Thanks,
Neha


On Wed, Mar 26, 2014 at 9:06 PM, Jun Rao <ju...@gmail.com> wrote:

> Currently, there is no callback on rebalance failure. The consumer will
> retry failed rebalances. If all retries fail, we just log the error.
>
> Thanks,
>
> Jun
>
>
> On Wed, Mar 26, 2014 at 5:01 PM, Tom Amon <ta...@gmail.com> wrote:
>
> > The pattern for creating and operating consumers that we use is to create
> > the consumer connector, create the streams and then consume each stream
> by
> > waiting on the iterator.
> >
> > If a rebalance occurs and fails, how is the error raised to the consumer?
> > Will I get an exception while waiting on the iterator? Is it swallowed
> and
> > the consumer is dead?
> >
> > Thanks.
> >
>

Re: How are rebalance failures raised to consumers?

[Jun Rao <ju...@gmail.com>]

Currently, there is no callback on rebalance failure. The consumer will
retry failed rebalances. If all retries fail, we just log the error.

Thanks,

Jun


On Wed, Mar 26, 2014 at 5:01 PM, Tom Amon <ta...@gmail.com> wrote:

> The pattern for creating and operating consumers that we use is to create
> the consumer connector, create the streams and then consume each stream by
> waiting on the iterator.
>
> If a rebalance occurs and fails, how is the error raised to the consumer?
> Will I get an exception while waiting on the iterator? Is it swallowed and
> the consumer is dead?
>
> Thanks.
>