Offset backup and restore in Kafka v0.8.2

[Kyle Banker <ky...@gmail.com>]

We currently checkpoint partition offsets from ZK into an external
database, and we have a process for restoring those offsets to ZK in case
we ever need to replay data from Kafka.

Since offsets are now stored on the broker, the ConsumerOffsetChecker no
longer works. Is there a new way (tool) to dump and restore offsets now
that offsets are stored on the brokers themselves?

Re: Offset backup and restore in Kafka v0.8.2

[Mayuresh Gharat <gh...@gmail.com>]

There is a patch for KAFKA-1013 that has a tool for importing and exporting
offsets to and from kafka. Using that tool you can write
topic/partion/offset to a file and import it back to kafka.

Thanks,

Mayuresh

On Thu, Jan 8, 2015 at 1:21 PM, Kyle Banker <ky...@gmail.com> wrote:

> We currently checkpoint partition offsets from ZK into an external
> database, and we have a process for restoring those offsets to ZK in case
> we ever need to replay data from Kafka.
>
> Since offsets are now stored on the broker, the ConsumerOffsetChecker no
> longer works. Is there a new way (tool) to dump and restore offsets now
> that offsets are stored on the brokers themselves?
>



-- 
-Regards,
Mayuresh R. Gharat
(862) 250-7125