Kafka - offset preservation

[Pawel Huszcza <ph...@mandaracapital.com>]

Hello,

I tried every different property I can think of - I have set ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG = true; ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG = 1000, but I have also tried with many different values
And still every time I start my consumer - I am receiving all the messages ever sent to the given topic - it is every time the same consumer group; ConsumerConfig.AUTO_OFFSET_RESET_CONFIG = "latest" but I also tried with "earliest" (just for luck). What is the proper configuration to get it to work in such a way, that every time I start my consumer - it will process only the new messages not all of them?

Kind Regards,
Pawel Jan Huszcza

Re: Kafka - offset preservation

[R Krishna <kr...@gmail.com>]

Which version is this? I am a new user myself, are you using the same
consumer group name? In V0.9, once connected, broker stores consumer
offsets per consumer group, and continues from where it left off, the
earliest/latest are for the first time only.

On Wed, Jul 6, 2016 at 6:19 AM, Pawel Huszcza <ph...@mandaracapital.com>
wrote:

> Hello,
>
> I tried every different property I can think of - I have set
> ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG = true;
> ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG = 1000, but I have also tried
> with many different values
> And still every time I start my consumer - I am receiving all the messages
> ever sent to the given topic - it is every time the same consumer group;
> ConsumerConfig.AUTO_OFFSET_RESET_CONFIG = "latest" but I also tried with
> "earliest" (just for luck). What is the proper configuration to get it to
> work in such a way, that every time I start my consumer - it will process
> only the new messages not all of them?
>
> Kind Regards,
> Pawel Jan Huszcza
>



-- 
Radha Krishna, Proddaturi
253-234-5657