You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Pranay Agarwal <ag...@gmail.com> on 2015/02/18 01:22:43 UTC
Kafka High Level Consumer
Hi,
I am trying to read kafka consumer using high level kafka Consumer API. I
had to restart the consumers for some reason but I kept the same group id.
It seems the consumers have started consuming from the beginning (0 offset)
instead from the point they had already consumed.
What am I doing wrong here? How do I make sure the consumer start only
from the point they had left before?
Thanks
-Pranay
Re: Kafka High Level Consumer
Posted by Pranay Agarwal <ag...@gmail.com>.
Great news. Thanks a lot Joe
On Wed, Feb 25, 2015 at 11:46 AM, Joseph Lawson <jl...@roomkey.com> wrote:
> Doh that was probably my bad Pranay! A misinterpretation of some old
> consumer code. btw, jruby-kafka is now at 1.1.1 with proper support for
> deleting the offset, setting the auto_offset_reset and whitelist/blacklist
> topics. It's packed up in a nice gem file that includes all Kafka and
> log4j pre-requisites too.
>
> It's pretty feature complete for Kafka 0.8.1.1.
>
> Hurray! Thanks to everyone that submitted PRs to make it better.
>
> -Joe Lawson
>
> ________________________________________
> From: Pranay Agarwal <ag...@gmail.com>
> Sent: Wednesday, February 25, 2015 1:45 AM
> To: users@kafka.apache.org
> Subject: Re: Kafka High Level Consumer
>
> Thanks Jun. It seems it was an issue with jruby client I was using. Now,
> they fixed it.
>
> -Pranay
>
> On Mon, Feb 23, 2015 at 4:57 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > Did you enable auto offset commit?
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Feb 17, 2015 at 4:22 PM, Pranay Agarwal <
> agarwalpranaya@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I am trying to read kafka consumer using high level kafka Consumer
> API. I
> > > had to restart the consumers for some reason but I kept the same group
> > id.
> > > It seems the consumers have started consuming from the beginning (0
> > offset)
> > > instead from the point they had already consumed.
> > >
> > > What am I doing wrong here? How do I make sure the consumer start only
> > > from the point they had left before?
> > >
> > > Thanks
> > > -Pranay
> > >
> >
>
Re: Kafka High Level Consumer
Posted by Joseph Lawson <jl...@roomkey.com>.
Doh that was probably my bad Pranay! A misinterpretation of some old consumer code. btw, jruby-kafka is now at 1.1.1 with proper support for deleting the offset, setting the auto_offset_reset and whitelist/blacklist topics. It's packed up in a nice gem file that includes all Kafka and log4j pre-requisites too.
It's pretty feature complete for Kafka 0.8.1.1.
Hurray! Thanks to everyone that submitted PRs to make it better.
-Joe Lawson
________________________________________
From: Pranay Agarwal <ag...@gmail.com>
Sent: Wednesday, February 25, 2015 1:45 AM
To: users@kafka.apache.org
Subject: Re: Kafka High Level Consumer
Thanks Jun. It seems it was an issue with jruby client I was using. Now,
they fixed it.
-Pranay
On Mon, Feb 23, 2015 at 4:57 PM, Jun Rao <ju...@confluent.io> wrote:
> Did you enable auto offset commit?
>
> Thanks,
>
> Jun
>
> On Tue, Feb 17, 2015 at 4:22 PM, Pranay Agarwal <ag...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I am trying to read kafka consumer using high level kafka Consumer API. I
> > had to restart the consumers for some reason but I kept the same group
> id.
> > It seems the consumers have started consuming from the beginning (0
> offset)
> > instead from the point they had already consumed.
> >
> > What am I doing wrong here? How do I make sure the consumer start only
> > from the point they had left before?
> >
> > Thanks
> > -Pranay
> >
>
Re: Kafka High Level Consumer
Posted by Pranay Agarwal <ag...@gmail.com>.
Thanks Jun. It seems it was an issue with jruby client I was using. Now,
they fixed it.
-Pranay
On Mon, Feb 23, 2015 at 4:57 PM, Jun Rao <ju...@confluent.io> wrote:
> Did you enable auto offset commit?
>
> Thanks,
>
> Jun
>
> On Tue, Feb 17, 2015 at 4:22 PM, Pranay Agarwal <ag...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I am trying to read kafka consumer using high level kafka Consumer API. I
> > had to restart the consumers for some reason but I kept the same group
> id.
> > It seems the consumers have started consuming from the beginning (0
> offset)
> > instead from the point they had already consumed.
> >
> > What am I doing wrong here? How do I make sure the consumer start only
> > from the point they had left before?
> >
> > Thanks
> > -Pranay
> >
>
Re: Kafka High Level Consumer
Posted by Jun Rao <ju...@confluent.io>.
Did you enable auto offset commit?
Thanks,
Jun
On Tue, Feb 17, 2015 at 4:22 PM, Pranay Agarwal <ag...@gmail.com>
wrote:
> Hi,
>
> I am trying to read kafka consumer using high level kafka Consumer API. I
> had to restart the consumers for some reason but I kept the same group id.
> It seems the consumers have started consuming from the beginning (0 offset)
> instead from the point they had already consumed.
>
> What am I doing wrong here? How do I make sure the consumer start only
> from the point they had left before?
>
> Thanks
> -Pranay
>