You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Franco Giacosa <fg...@gmail.com> on 2016/01/25 15:07:27 UTC
re-consuming last offset
Hi,
I am facing the following issue:
When I start my consumer I get that the offset for one of the partitions is
going to be reset to the last committed offset
14:55:28.788 [pool-1-thread-1] DEBUG o.a.k.c.consumer.internals.Fetcher -
Resetting offset for partition t1-4 to the committed offset 205
Which is correct, since 205 was the last message that I process and
manually commit (with commitSync()) before closing the consumer the last
time.
The problem is that now I start the consumer and the first message that it
brings is the one in offset 205 instead of the next one, so I reprocess the
message, is there a way to avoid this situation? I thought that committed
offsets were never returned).
Thanks,
Franco
Re: re-consuming last offset
Posted by Jason Gustafson <ja...@confluent.io>.
It might be a little unintuitive, but the committed position should be the
offset of the next message to consume.
-Jason
On Mon, Jan 25, 2016 at 1:26 PM, Franco Giacosa <fg...@gmail.com> wrote:
> When doing poll() when there is no current position on the consumer, the
> position returned is the one of the last offset then? (I though that it
> will return that position + 1 because it was already commited)
>
>
>
>
>
>
>
>
> 2016-01-25 15:07 GMT+01:00 Franco Giacosa <fg...@gmail.com>:
>
> > Hi,
> >
> > I am facing the following issue:
> >
> > When I start my consumer I get that the offset for one of the partitions
> > is going to be reset to the last committed offset
> >
> > 14:55:28.788 [pool-1-thread-1] DEBUG o.a.k.c.consumer.internals.Fetcher -
> > Resetting offset for partition t1-4 to the committed offset 205
> >
> > Which is correct, since 205 was the last message that I process and
> > manually commit (with commitSync()) before closing the consumer the last
> > time.
> >
> >
> > The problem is that now I start the consumer and the first message that
> it
> > brings is the one in offset 205 instead of the next one, so I reprocess
> the
> > message, is there a way to avoid this situation? I thought that committed
> > offsets were never returned).
> >
> > Thanks,
> > Franco
> >
>
Re: re-consuming last offset
Posted by Franco Giacosa <fg...@gmail.com>.
When doing poll() when there is no current position on the consumer, the
position returned is the one of the last offset then? (I though that it
will return that position + 1 because it was already commited)
2016-01-25 15:07 GMT+01:00 Franco Giacosa <fg...@gmail.com>:
> Hi,
>
> I am facing the following issue:
>
> When I start my consumer I get that the offset for one of the partitions
> is going to be reset to the last committed offset
>
> 14:55:28.788 [pool-1-thread-1] DEBUG o.a.k.c.consumer.internals.Fetcher -
> Resetting offset for partition t1-4 to the committed offset 205
>
> Which is correct, since 205 was the last message that I process and
> manually commit (with commitSync()) before closing the consumer the last
> time.
>
>
> The problem is that now I start the consumer and the first message that it
> brings is the one in offset 205 instead of the next one, so I reprocess the
> message, is there a way to avoid this situation? I thought that committed
> offsets were never returned).
>
> Thanks,
> Franco
>