You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Sanket Maru <sa...@ecinity.com> on 2013/12/09 13:30:00 UTC
Consumer Events Problem
I am working on a small project and discovered that our consumer hasn't
been executed for over a month now.
How can i check the unprocessed events ? From which date the events are
available and what is the retention policy on the producer side ?
Thanks,
Sanket Maru
Re: Consumer Events Problem
Posted by Philip O'Toole <ph...@loggly.com>.
OK, I am only familiar with 0.72.
Philip
On Mon, Dec 9, 2013 at 4:54 AM, Sanket Maru <sa...@ecinity.com> wrote:
> I am using kafka 0.8.0
>
>
> On Mon, Dec 9, 2013 at 6:09 PM, Philip O'Toole <ph...@loggly.com> wrote:
>
> > What version are you running?
> >
> > Philip
> >
> >
> > On Mon, Dec 9, 2013 at 4:30 AM, Sanket Maru <sa...@ecinity.com> wrote:
> >
> > > I am working on a small project and discovered that our consumer hasn't
> > > been executed for over a month now.
> > >
> > > How can i check the unprocessed events ? From which date the events are
> > > available and what is the retention policy on the producer side ?
> > >
> > > Thanks,
> > > Sanket Maru
> > >
> >
>
Re: Consumer Events Problem
Posted by Sanket Maru <sa...@ecinity.com>.
I am using kafka 0.8.0
On Mon, Dec 9, 2013 at 6:09 PM, Philip O'Toole <ph...@loggly.com> wrote:
> What version are you running?
>
> Philip
>
>
> On Mon, Dec 9, 2013 at 4:30 AM, Sanket Maru <sa...@ecinity.com> wrote:
>
> > I am working on a small project and discovered that our consumer hasn't
> > been executed for over a month now.
> >
> > How can i check the unprocessed events ? From which date the events are
> > available and what is the retention policy on the producer side ?
> >
> > Thanks,
> > Sanket Maru
> >
>
Re: Consumer Events Problem
Posted by Philip O'Toole <ph...@loggly.com>.
What version are you running?
Philip
On Mon, Dec 9, 2013 at 4:30 AM, Sanket Maru <sa...@ecinity.com> wrote:
> I am working on a small project and discovered that our consumer hasn't
> been executed for over a month now.
>
> How can i check the unprocessed events ? From which date the events are
> available and what is the retention policy on the producer side ?
>
> Thanks,
> Sanket Maru
>
Re: Consumer Events Problem
Posted by Jun Rao <ju...@gmail.com>.
Yes, you can use the ConsumerOffsetChecker tool.
Thanks,
Jun
On Mon, Dec 9, 2013 at 9:40 PM, Sanket Maru <sa...@ecinity.com> wrote:
> For mine topic the offset isn't increasing which means the consumer has
> stopped. I wanted to get the count(#) of events that are still remaining to
> be processed. Is that possible ?
>
>
>
> On Mon, Dec 9, 2013 at 9:44 PM, Jun Rao <ju...@gmail.com> wrote:
>
> > By default, each topic is kept on the broker for 7 days. Older data,
> > whether consumed or not, will be deleted.
> >
> > To check # of unconsumed messages, you can either use the
> > ConsumerOffsetChecker tool (see
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Myconsumerseemstohavestopped%2Cwhy%3F
> > )
> > or use the maxlag jmx in the consumer application (see
> > http://kafka.apache.org/documentation.html#consumerconfigs)
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Mon, Dec 9, 2013 at 4:30 AM, Sanket Maru <sa...@ecinity.com> wrote:
> >
> > > I am working on a small project and discovered that our consumer hasn't
> > > been executed for over a month now.
> > >
> > > How can i check the unprocessed events ? From which date the events are
> > > available and what is the retention policy on the producer side ?
> > >
> > > Thanks,
> > > Sanket Maru
> > >
> >
>
Re: Consumer Events Problem
Posted by Sanket Maru <sa...@ecinity.com>.
For mine topic the offset isn't increasing which means the consumer has
stopped. I wanted to get the count(#) of events that are still remaining to
be processed. Is that possible ?
On Mon, Dec 9, 2013 at 9:44 PM, Jun Rao <ju...@gmail.com> wrote:
> By default, each topic is kept on the broker for 7 days. Older data,
> whether consumed or not, will be deleted.
>
> To check # of unconsumed messages, you can either use the
> ConsumerOffsetChecker tool (see
>
> https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Myconsumerseemstohavestopped%2Cwhy%3F
> )
> or use the maxlag jmx in the consumer application (see
> http://kafka.apache.org/documentation.html#consumerconfigs)
>
> Thanks,
>
> Jun
>
>
> On Mon, Dec 9, 2013 at 4:30 AM, Sanket Maru <sa...@ecinity.com> wrote:
>
> > I am working on a small project and discovered that our consumer hasn't
> > been executed for over a month now.
> >
> > How can i check the unprocessed events ? From which date the events are
> > available and what is the retention policy on the producer side ?
> >
> > Thanks,
> > Sanket Maru
> >
>
Re: Consumer Events Problem
Posted by Jun Rao <ju...@gmail.com>.
By default, each topic is kept on the broker for 7 days. Older data,
whether consumed or not, will be deleted.
To check # of unconsumed messages, you can either use the
ConsumerOffsetChecker tool (see
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Myconsumerseemstohavestopped%2Cwhy%3F)
or use the maxlag jmx in the consumer application (see
http://kafka.apache.org/documentation.html#consumerconfigs)
Thanks,
Jun
On Mon, Dec 9, 2013 at 4:30 AM, Sanket Maru <sa...@ecinity.com> wrote:
> I am working on a small project and discovered that our consumer hasn't
> been executed for over a month now.
>
> How can i check the unprocessed events ? From which date the events are
> available and what is the retention policy on the producer side ?
>
> Thanks,
> Sanket Maru
>