Consumers doesn't always poll first messages

[Péricé Robin <pe...@gmail.com>]

Hello everybody,

I'm testing the new 0.9.0.1 API and I try to make a basic example working.

*Java code* :

*Consumer *: http://pastebin.com/YtvW0sz5
*Producer *: http://pastebin.com/anQay9YE
*Test* : http://pastebin.com/nniYLsHL


*Kafka configuration* :

*Zookeeper propertie*s : http://pastebin.com/KC5yZdNx
*Kafka properties* : http://pastebin.com/Psy4uAYL

But when I try to run my test and restart Kafka to see what happen. The
Consumer doesn't always consume first messages. Sometimes it consume
messages at offset 0 or 574 or 1292 ... The behavior of the test seems to
be very random.

Anybody have an idea on that issue ?

Best Regards,

Robin

Re: Consumers doesn't always poll first messages

[Jan Omar <ja...@gmail.com>]

Hi Robin,

Why would you expect it to start from the first message?

You're comitting the read offsets automatically every second. The offset is persisted, next time you consume again, it will start at the persisted offset again.

 consumerProperties.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "true");
 consumerProperties.put(ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG, "1000");

Regards

Jan

> On 2 Mar 2016, at 15:14, Péricé Robin <pe...@gmail.com> wrote:
> 
> Hello everybody,
> 
> I'm testing the new 0.9.0.1 API and I try to make a basic example working.
> 
> *Java code* :
> 
> *Consumer *: http://pastebin.com/YtvW0sz5
> *Producer *: http://pastebin.com/anQay9YE
> *Test* : http://pastebin.com/nniYLsHL
> 
> 
> *Kafka configuration* :
> 
> *Zookeeper propertie*s : http://pastebin.com/KC5yZdNx
> *Kafka properties* : http://pastebin.com/Psy4uAYL
> 
> But when I try to run my test and restart Kafka to see what happen. The
> Consumer doesn't always consume first messages. Sometimes it consume
> messages at offset 0 or 574 or 1292 ... The behavior of the test seems to
> be very random.
> 
> Anybody have an idea on that issue ?
> 
> Best Regards,
> 
> Robin