maximum lag between current offset, and last commit

[Clark Sims <cl...@gmail.com>]

I liked this article:
https://blog.newrelic.com/engineering/kafka-consumer-config-auto-commit-data-loss/
I particularly like the illustration in the article:
http://newrelic-wpengine.netdna-ssl.com/wp-content/uploads/offset_explained2.jpg

What is the maximum difference between the current offset in a
partition, and the last offset committed?

When current offset = committed offset + maximum allowed, is this when
Kafka broker starts re-sending messages to the consumer?

Can I configure the maximum allowed offset, to control the re-sending behavior?

Thanks,
Clark

Re: maximum lag between current offset, and last commit

[Clark Sims <cl...@gmail.com>]

Perhaps my understanding is off.
https://docs.confluent.io/current/installation/configuration/broker-configs.html
Perhaps this is the parameter I am looking for?
offsets.retention.check.interval.ms
Perhaps the re-send behavior is controlled by time, and not the
maximum distance between the current offset, and last confirmed
offset.
Regards,
Clark

On Wed, Jan 8, 2020 at 11:55 AM Clark Sims <cl...@gmail.com> wrote:
>
> I liked this article:
> https://blog.newrelic.com/engineering/kafka-consumer-config-auto-commit-data-loss/
> I particularly like the illustration in the article:
> http://newrelic-wpengine.netdna-ssl.com/wp-content/uploads/offset_explained2.jpg
>
> What is the maximum difference between the current offset in a
> partition, and the last offset committed?
>
> When current offset = committed offset + maximum allowed, is this when
> Kafka broker starts re-sending messages to the consumer?
>
> Can I configure the maximum allowed offset, to control the re-sending behavior?
>
> Thanks,
> Clark