Commit offsets only work for subscribe(), not assign()

["hsy541@gmail.com" <hs...@gmail.com>]

In the new API, the explicit commit offset method call only works for
subscribe consumer, not the assign consumer, correct?

Best,
Siyuan

Re: Commit offsets only work for subscribe(), not assign()

[Guozhang Wang <wa...@gmail.com>]

Siyuan,

When you say "I didn't see anything from the time the process was killed"
did you mean your consumer did not get any data after it is restarted, or
that the processed messages before killing the consumer does not show up
again?

Guozhang


On Mon, Nov 23, 2015 at 8:13 AM, hsy541@gmail.com <hs...@gmail.com> wrote:

> Hey Jason,
>
> The test I did is very simple, I was using manual assignment with it's own
> groupid and clientid. I first started a process to consume data, then
> produce some data, then kill the process, continue produce more data and
> start the process again, I didn't see anything from the time the process
> was killed.  Do I have set "auto.offset.reset" as "none"?  Thank
>
> On Fri, Nov 20, 2015 at 3:56 PM, Jason Gustafson <ja...@confluent.io>
> wrote:
>
> > I suppose I should have added one qualification to that. The commit API
> > will not work for a consumer using manual assignment if its groupId is
> > shared with another consumer using automatic assignment (with
> subscribe()).
> > When a consumer group is active, Kafka only allows commits from members
> of
> > that group.
> >
> > -Jason
> >
> > On Fri, Nov 20, 2015 at 3:41 PM, Jason Gustafson <ja...@confluent.io>
> > wrote:
> >
> > > Hey Siyuan,
> > >
> > > The commit API should work the same regardless whether subscribe() or
> > > assign() was used. Does this not appear to be working?
> > >
> > > Thanks,
> > > Jason
> > >
> > > On Wed, Nov 18, 2015 at 4:40 PM, hsy541@gmail.com <hs...@gmail.com>
> > > wrote:
> > >
> > >> In the new API, the explicit commit offset method call only works for
> > >> subscribe consumer, not the assign consumer, correct?
> > >>
> > >> Best,
> > >> Siyuan
> > >>
> > >
> > >
> >
>



-- 
-- Guozhang

Re: Commit offsets only work for subscribe(), not assign()

["hsy541@gmail.com" <hs...@gmail.com>]

Hey Jason,

The test I did is very simple, I was using manual assignment with it's own
groupid and clientid. I first started a process to consume data, then
produce some data, then kill the process, continue produce more data and
start the process again, I didn't see anything from the time the process
was killed.  Do I have set "auto.offset.reset" as "none"?  Thank

On Fri, Nov 20, 2015 at 3:56 PM, Jason Gustafson <ja...@confluent.io> wrote:

> I suppose I should have added one qualification to that. The commit API
> will not work for a consumer using manual assignment if its groupId is
> shared with another consumer using automatic assignment (with subscribe()).
> When a consumer group is active, Kafka only allows commits from members of
> that group.
>
> -Jason
>
> On Fri, Nov 20, 2015 at 3:41 PM, Jason Gustafson <ja...@confluent.io>
> wrote:
>
> > Hey Siyuan,
> >
> > The commit API should work the same regardless whether subscribe() or
> > assign() was used. Does this not appear to be working?
> >
> > Thanks,
> > Jason
> >
> > On Wed, Nov 18, 2015 at 4:40 PM, hsy541@gmail.com <hs...@gmail.com>
> > wrote:
> >
> >> In the new API, the explicit commit offset method call only works for
> >> subscribe consumer, not the assign consumer, correct?
> >>
> >> Best,
> >> Siyuan
> >>
> >
> >
>

Re: Commit offsets only work for subscribe(), not assign()

[Jason Gustafson <ja...@confluent.io>]

I suppose I should have added one qualification to that. The commit API
will not work for a consumer using manual assignment if its groupId is
shared with another consumer using automatic assignment (with subscribe()).
When a consumer group is active, Kafka only allows commits from members of
that group.

-Jason

On Fri, Nov 20, 2015 at 3:41 PM, Jason Gustafson <ja...@confluent.io> wrote:

> Hey Siyuan,
>
> The commit API should work the same regardless whether subscribe() or
> assign() was used. Does this not appear to be working?
>
> Thanks,
> Jason
>
> On Wed, Nov 18, 2015 at 4:40 PM, hsy541@gmail.com <hs...@gmail.com>
> wrote:
>
>> In the new API, the explicit commit offset method call only works for
>> subscribe consumer, not the assign consumer, correct?
>>
>> Best,
>> Siyuan
>>
>
>

Re: Commit offsets only work for subscribe(), not assign()

[Jason Gustafson <ja...@confluent.io>]

Hey Siyuan,

The commit API should work the same regardless whether subscribe() or
assign() was used. Does this not appear to be working?

Thanks,
Jason

On Wed, Nov 18, 2015 at 4:40 PM, hsy541@gmail.com <hs...@gmail.com> wrote:

> In the new API, the explicit commit offset method call only works for
> subscribe consumer, not the assign consumer, correct?
>
> Best,
> Siyuan
>