How to reduce time for consumer to take over?

[Christian Schneider <ch...@die-schneider.net>]

I have the following case:

1. I have 1 consumer that starts processing a message, then crashes
(Java-VM shuts down)
2. I start a second consumer that should process the message instead.

It seems that it takes about 60 seconds for the second consumer to take
over the processing.
Can this be speeded up? I use this in a test and would like to make that
test faster.

Christian

-- 
-- 
Christian Schneider
http://www.liquid-reality.de
<https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e46&URL=http%3a%2f%2fwww.liquid-reality.de>

Computer Scientist
http://www.adobe.com

Re: How to reduce time for consumer to take over?

[Stevo Slavić <ss...@gmail.com>]

Hello Christian,

Assuming that you are using new Java kafka-clients with Kafka's consumer
group management, you'd have to tune consumers and broker.

See
- Broker config https://kafka.apache.org/documentation/#brokerconfigs
-- group.initial.rebalance.delay.ms
-- group.max.session.timeout.ms
-- group.min.session.timeout.ms
- New consumer config
https://kafka.apache.org/documentation/#newconsumerconfigs
-- heartbeat.interval.ms
-- session.timeout.ms

Notice, like always, there's a tradeoff involved.

Kind regards,
Stevo Slavic.


On Mon, Oct 9, 2017 at 2:29 PM, Christian Schneider <chris@die-schneider.net
> wrote:

> I have the following case:
>
> 1. I have 1 consumer that starts processing a message, then crashes
> (Java-VM shuts down)
> 2. I start a second consumer that should process the message instead.
>
> It seems that it takes about 60 seconds for the second consumer to take
> over the processing.
> Can this be speeded up? I use this in a test and would like to make that
> test faster.
>
> Christian
>
> --
> --
> Christian Schneider
> http://www.liquid-reality.de
> <https://owa.talend.com/owa/redir.aspx?C=3aa4083e0c744ae1ba52bd062c5a7e
> 46&URL=http%3a%2f%2fwww.liquid-reality.de>
>
> Computer Scientist
> http://www.adobe.com
>