Multiple instances of HL Consumer

["Panda, Samaresh" <Sa...@washpost.com>]

I've a HL consumer receiving messages using four threads (four partitions). This is a stand-alone Java client. For fail-safe reasons, I want to run another instance of the exact same Java client in a different box.

Here are my questions:

> Can I keep the same consumer group name or it must be different for the 2nd instance?
> If same consumer group, will the 2nd client receive same set of messages again?
> In general what's the best practice to designing fail-safe clients?

Thanks
Sam

Re: Multiple instances of HL Consumer

[Sharninder Khera <sh...@gmail.com>]

You can have the same consumer id and Kafka will balance partitions across the two instances automatically. When one of them dies the partitions are rebalanced and assigned to the remaining alive consumers. 



    _____________________________
From: Panda, Samaresh <sa...@washpost.com>
Sent: Friday, June 5, 2015 7:42 pm
Subject: Multiple instances of HL Consumer
To:  <us...@kafka.apache.org>


I've a HL consumer receiving messages using four threads (four partitions). This is a stand-alone Java client. For fail-safe reasons, I want to run another instance of the exact same Java client in a different box.

Here are my questions:

> Can I keep the same consumer group name or it must be different for the 2nd instance?
> If same consumer group, will the 2nd client receive same set of messages again?
> In general what's the best practice to designing fail-safe clients?

Thanks
Sam