You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Gomathivinayagam Muthuvinayagam <sa...@gmail.com> on 2015/04/28 13:38:16 UTC
Could you answer the following kafka stackoverflow question?
I have just posted the following question in stackoverflow. Could you
answer the following questions?
I would like to use Kafka high level consumer API, and at the same time I
would like to disable auto commit of offsets. I tried to achieve this
through the following steps.
1) auto.commit.enable = false2) offsets.storage = kafka3)
dual.commit.enabled = false
I created a offset manager, which periodically creates offsetcommit request
to kafka and commits the offset.
Still I have the following questions
1) Does high level consumer API automatically fetches offset from kafka
storage and initializes itself with that offset? Or should I use simple
consumer API to achieve this?
2) Does kafka based storage for offsets is repicated across all brokers? Or
it is maintained on only one broker?
http://stackoverflow.com/questions/29909179/kafka-offsetcommit-request-with-high-level-consumer-api
Thanks & Regards,
Re: Could you answer the following kafka stackoverflow question?
Posted by Gomathivinayagam Muthuvinayagam <sa...@gmail.com>.
If I use high level consumer API, and also offset storage as kafka, does
high level consumer API takes care of everything?
Thanks & Regards,
On Tue, Apr 28, 2015 at 7:47 AM, Manoj Khangaonkar <kh...@gmail.com>
wrote:
> Use the Simple Consumer API if you want to control which offset in a
> partition the client should read messages from.
>
> Use the High level consumer, if you don'nt care about offsets.
>
> Offsets are per partition and stored on zookeeper.
>
> regards
>
> On Tue, Apr 28, 2015 at 4:38 AM, Gomathivinayagam Muthuvinayagam <
> sankarmail@gmail.com> wrote:
>
> > I have just posted the following question in stackoverflow. Could you
> > answer the following questions?
> >
> > I would like to use Kafka high level consumer API, and at the same time I
> > would like to disable auto commit of offsets. I tried to achieve this
> > through the following steps.
> >
> > 1) auto.commit.enable = false2) offsets.storage = kafka3)
> > dual.commit.enabled = false
> >
> > I created a offset manager, which periodically creates offsetcommit
> request
> > to kafka and commits the offset.
> >
> > Still I have the following questions
> >
> > 1) Does high level consumer API automatically fetches offset from kafka
> > storage and initializes itself with that offset? Or should I use simple
> > consumer API to achieve this?
> >
> > 2) Does kafka based storage for offsets is repicated across all brokers?
> Or
> > it is maintained on only one broker?
> >
> >
> >
> http://stackoverflow.com/questions/29909179/kafka-offsetcommit-request-with-high-level-consumer-api
> > Thanks & Regards,
> >
>
>
>
> --
> http://khangaonkar.blogspot.com/
>
Re: Could you answer the following kafka stackoverflow question?
Posted by Manoj Khangaonkar <kh...@gmail.com>.
Use the Simple Consumer API if you want to control which offset in a
partition the client should read messages from.
Use the High level consumer, if you don'nt care about offsets.
Offsets are per partition and stored on zookeeper.
regards
On Tue, Apr 28, 2015 at 4:38 AM, Gomathivinayagam Muthuvinayagam <
sankarmail@gmail.com> wrote:
> I have just posted the following question in stackoverflow. Could you
> answer the following questions?
>
> I would like to use Kafka high level consumer API, and at the same time I
> would like to disable auto commit of offsets. I tried to achieve this
> through the following steps.
>
> 1) auto.commit.enable = false2) offsets.storage = kafka3)
> dual.commit.enabled = false
>
> I created a offset manager, which periodically creates offsetcommit request
> to kafka and commits the offset.
>
> Still I have the following questions
>
> 1) Does high level consumer API automatically fetches offset from kafka
> storage and initializes itself with that offset? Or should I use simple
> consumer API to achieve this?
>
> 2) Does kafka based storage for offsets is repicated across all brokers? Or
> it is maintained on only one broker?
>
>
> http://stackoverflow.com/questions/29909179/kafka-offsetcommit-request-with-high-level-consumer-api
> Thanks & Regards,
>
--
http://khangaonkar.blogspot.com/