You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by David Luu <ma...@gmail.com> on 2016/02/01 20:51:06 UTC

Get messages between offsets? Or determine # of messages between offsets?

What's the best way to consume messages from a starting offset to an ending
offset? It seems the low level consumer (API),
kafka-simple-consumer-shell.sh, and kafkacat offer an option to consume
from a starting offset, but there's no option to tell it to stop consuming
after a given offset. One has to manually determine/track that (by figuring
out the current offset after fetching messages until the end offset is
reached/surpassed).

Alternatively, is there a way, given a starting and ending offset, to
determine how many messages exist in between? Since
kafka-simple-consumer-shell.sh and kafkacat have option to consume up to N
messages, so it could stop properly if one can figure out # messages
between offsets.

Has anyone done work in this area or can provide tips on what to do?