New Offset Management API Question

[Bhavesh Mistry <mi...@gmail.com>]

Hi Joel Koshy,

I have following questions:

1) Does Offset Commit/Fetch API works with Simple Consumer ?

Use Case:  Camus dual commit offset on hdfs as well as to Kafka offset
topic.  It will be only committed once when FILE Is committed to HDFS (Not
like auto commit)

2) With MM, can you selectively MM offset topic.

Use Case:
Let's suppose you want to build Active Consumer Group in DC1 and passive
Consumer Group (Not yet started in DC 2).  Can you MM single offset topic
and when DC1 consumer Group goes down, DC2 (with manual or  automated
custom logic) start same consumer group with last committed offset.  Is
this possible ?


Thanks,
Bhavesh

Re: New Offset Management API Question

[Joel Koshy <jj...@gmail.com>]

> 1) Does Offset Commit/Fetch API works with Simple Consumer ?

Yes - in 0.8.2.1. There is an example given at
https://cwiki.apache.org/confluence/display/KAFKA/Committing+and+fetching+consumer+offsets+in+Kafka

> 2) With MM, can you selectively MM offset topic.
> 
> Use Case:
> Let's suppose you want to build Active Consumer Group in DC1 and passive
> Consumer Group (Not yet started in DC 2).  Can you MM single offset topic
> and when DC1 consumer Group goes down, DC2 (with manual or  automated
> custom logic) start same consumer group with last committed offset.  Is
> this possible ?

Not sure I follow - the passive consumer would just need to start up
with the same group right? Then it would pick up the last committed
offset in DC1 for that group.

Thanks,

Joel