You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Bhavesh Mistry <mi...@gmail.com> on 2014/10/28 19:44:18 UTC
High Level Consumer and Close with Auto Commit On
Hi Kafka Team,
What is expected behavior when you close *ConsumerConnector* and auto
commit is on ? Basically, when auto commit interval is set to 5 seconds
and shutdown is called (before 5 seconds elapses) does ConsumerConnector
commit the offset of message consumed by (next()) method or consumer will
get duplicate messages when it comes online after restart ?
ConsumerConnector.shutdown();
Thanks,
Bhavesh
Re: High Level Consumer and Close with Auto Commit On
Posted by Bhavesh Mistry <mi...@gmail.com>.
HI Gwen,
Thanks for info appreciate. Sometime, I have observed duplicated events...
so that is why I ask.
Thanks,
Bhavesh
On Tue, Oct 28, 2014 at 11:51 AM, Gwen Shapira <gs...@cloudera.com>
wrote:
> High level consumer commits before shutting down.
>
> If you'll look at ZookeeperConsumerConnector.scala (currently the only
> implementation of ConsumerConnector) you'll see shutdown() includes
> the following:
>
> if (config.autoCommitEnable)
> commitOffsets()
>
> Gwen
>
> On Tue, Oct 28, 2014 at 11:44 AM, Bhavesh Mistry
> <mi...@gmail.com> wrote:
> > Hi Kafka Team,
> >
> > What is expected behavior when you close *ConsumerConnector* and auto
> > commit is on ? Basically, when auto commit interval is set to 5 seconds
> > and shutdown is called (before 5 seconds elapses) does ConsumerConnector
> > commit the offset of message consumed by (next()) method or consumer will
> > get duplicate messages when it comes online after restart ?
> >
> > ConsumerConnector.shutdown();
> >
> > Thanks,
> >
> > Bhavesh
>
Re: High Level Consumer and Close with Auto Commit On
Posted by Gwen Shapira <gs...@cloudera.com>.
High level consumer commits before shutting down.
If you'll look at ZookeeperConsumerConnector.scala (currently the only
implementation of ConsumerConnector) you'll see shutdown() includes
the following:
if (config.autoCommitEnable)
commitOffsets()
Gwen
On Tue, Oct 28, 2014 at 11:44 AM, Bhavesh Mistry
<mi...@gmail.com> wrote:
> Hi Kafka Team,
>
> What is expected behavior when you close *ConsumerConnector* and auto
> commit is on ? Basically, when auto commit interval is set to 5 seconds
> and shutdown is called (before 5 seconds elapses) does ConsumerConnector
> commit the offset of message consumed by (next()) method or consumer will
> get duplicate messages when it comes online after restart ?
>
> ConsumerConnector.shutdown();
>
> Thanks,
>
> Bhavesh
Re: High Level Consumer and Close with Auto Commit On
Posted by Gwen Shapira <gs...@cloudera.com>.
High level consumer commits before shutting down.
If you'll look at ZookeeperConsumerConnector.scala (currently the only
implementation of ConsumerConnector) you'll see shutdown() includes
the following:
if (config.autoCommitEnable)
commitOffsets()
Gwen
On Tue, Oct 28, 2014 at 11:44 AM, Bhavesh Mistry
<mi...@gmail.com> wrote:
> Hi Kafka Team,
>
> What is expected behavior when you close *ConsumerConnector* and auto
> commit is on ? Basically, when auto commit interval is set to 5 seconds
> and shutdown is called (before 5 seconds elapses) does ConsumerConnector
> commit the offset of message consumed by (next()) method or consumer will
> get duplicate messages when it comes online after restart ?
>
> ConsumerConnector.shutdown();
>
> Thanks,
>
> Bhavesh