Duplicate messages when consumers are restarted

[Jananee S <ja...@gmail.com>]

We are using kafka 0.8.2.1 and are noticing a weird behavior when the high
level consumers are restarted.

We have two consumers (C1 and C2, processes on different machines) in a
consumer group. The topic has two partitions. We use kafka as offsets
storage and have auto commit disabled.

Here is the situation

1. C1 is processing a message 'M1'.
2. C1 and C2 are restarted in order before C1 could finish processing the
message and do a commit.
3. C2 and then C1 both get the same message 'M1'.


I came across some old messages in the forum that talked about this. But
the situation there was the messages were compressed. In our case, the
messages are not compressed. Is this a bug in consumer re-balancing?


Thanks,
Jananee