You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by SuoNayi <su...@163.com> on 2012/02/07 16:37:33 UTC
Is there any web console to monitor what's the status about broker?
Hi all,
It's natural that I want to see how many messages in my queue
or how many consumers are active on my broker.
I want to know about how many partitions as well.
It seems that no related document is available in the official website or I miss something?
SuoNayi
Re: Reply:Re: Is there any web console to monitor what's the status
about broker?
Posted by Patricio Echagüe <pa...@gmail.com>.
Its enabled by default.
Sent from my Android
On Feb 7, 2012 8:01 AM, "SuoNayi" <su...@163.com> wrote:
> Is jmx disabled as default?
>
> If so how to enable it ?
>
> Or there is any construction I can follow?
>
>
> At 2012-02-07 23:46:03,"Patricio Echagüe" <pa...@gmail.com> wrote:
> >you can use jmx via jconsole or ganglia. For the ganglia integration I
> use
> >jmxtrans.
> >
> >Sent from my Android
> >On Feb 7, 2012 7:38 AM, "SuoNayi" <su...@163.com> wrote:
> >
> >> Hi all,
> >>
> >> It's natural that I want to see how many messages in my queue
> >>
> >> or how many consumers are active on my broker.
> >>
> >> I want to know about how many partitions as well.
> >>
> >> It seems that no related document is available in the official website
> or
> >> I miss something?
> >>
> >>
> >>
> >> SuoNayi
>
Reply:Re: Is there any web console to monitor what's the status
about broker?
Posted by SuoNayi <su...@163.com>.
Is jmx disabled as default?
If so how to enable it ?
Or there is any construction I can follow?
At 2012-02-07 23:46:03,"Patricio Echagüe" <pa...@gmail.com> wrote:
>you can use jmx via jconsole or ganglia. For the ganglia integration I use
>jmxtrans.
>
>Sent from my Android
>On Feb 7, 2012 7:38 AM, "SuoNayi" <su...@163.com> wrote:
>
>> Hi all,
>>
>> It's natural that I want to see how many messages in my queue
>>
>> or how many consumers are active on my broker.
>>
>> I want to know about how many partitions as well.
>>
>> It seems that no related document is available in the official website or
>> I miss something?
>>
>>
>>
>> SuoNayi
Re: Is there any web console to monitor what's the status about broker?
Posted by Patricio Echagüe <pa...@gmail.com>.
you can use jmx via jconsole or ganglia. For the ganglia integration I use
jmxtrans.
Sent from my Android
On Feb 7, 2012 7:38 AM, "SuoNayi" <su...@163.com> wrote:
> Hi all,
>
> It's natural that I want to see how many messages in my queue
>
> or how many consumers are active on my broker.
>
> I want to know about how many partitions as well.
>
> It seems that no related document is available in the official website or
> I miss something?
>
>
>
> SuoNayi
Re: Is there any web console to monitor what's the status about broker?
Posted by Evan Chan <ev...@ooyala.com>.
I've created KAFKA-266. Not sure what ticket type it should be, I created
it under "New Features", but not sure if it should have been "Wish list".
On Tue, Feb 7, 2012 at 11:20 AM, Felix GV <fe...@mate1inc.com> wrote:
> The scala version of Play! is really cool!
>
> And it seems like a fitting choice considering that the core Kafka code
> base is in scala :)
>
> --
> Felix
>
>
>
> On Tue, Feb 7, 2012 at 1:01 PM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
>
> > Evan,
> >
> > That is a good idea. Would you mind filing a JIRA so that the
> > community can help discussing viable solutions there ?
> >
> > Thanks,
> > Neha
> >
> > On Tue, Feb 7, 2012 at 9:47 AM, Evan Chan <ev...@ooyala.com> wrote:
> > > Is there a trick to getting jconsole to work with Kafka? It just
> hangs
> > > and doesn't return any info.
> > >
> > > BTW I was thinking of creating a web console for Kafka. One that would
> > > aggregate the info from both JMX and ZK and present it visually, so one
> > can
> > > easily see the partitions, consumers, and offsets or at least get a
> sense
> > > of how big the queue is for each consumer.
> > >
> > > I'd like the project to be supported by the community though. Either
> > JRuby
> > > or Scala (with Play! or something like that).
> > >
> > > Any thoughts?
> > >
> > > On Tue, Feb 7, 2012 at 9:29 AM, Jun Rao <ju...@gmail.com> wrote:
> > >
> > >> SuoNayi,
> > >>
> > >> Currently, we have jmx that reports #messages (per topic) consumed by
> a
> > >> consumer (and soon to have #messages produced as well). We don't have
> > >> anything that measures #messages in the queue. This is partly due to
> > that
> > >> in Kafka, messages are not immediately deleted after consumption and
> are
> > >> only deleted after certain amount of time has passed. We do have a
> tool
> > >> ConsumerOffsetChecker that reports the lag of the consumer in bytes.
> > >> Finally, we don't have anything that exposes #partitions. This is
> > probably
> > >> useful to have. Could you open a jira for that?
> > >>
> > >> Thanks,
> > >>
> > >> Jun
> > >>
> > >> 2012/2/7 SuoNayi <su...@163.com>
> > >>
> > >> > Hi all,
> > >> >
> > >> > It's natural that I want to see how many messages in my queue
> > >> >
> > >> > or how many consumers are active on my broker.
> > >> >
> > >> > I want to know about how many partitions as well.
> > >> >
> > >> > It seems that no related document is available in the official
> > website or
> > >> > I miss something?
> > >> >
> > >> >
> > >> >
> > >> > SuoNayi
> > >>
> > >
> > >
> > >
> > > --
> > > --
> > > *Evan Chan*
> > > Senior Software Engineer |
> > > ev@ooyala.com | (650) 996-4600
> > > www.ooyala.com | blog <http://www.ooyala.com/blog> |
> > > @ooyala<http://www.twitter.com/ooyala>
> >
>
--
--
*Evan Chan*
Senior Software Engineer |
ev@ooyala.com | (650) 996-4600
www.ooyala.com | blog <http://www.ooyala.com/blog> |
@ooyala<http://www.twitter.com/ooyala>
Re: Is there any web console to monitor what's the status about broker?
Posted by Felix GV <fe...@mate1inc.com>.
The scala version of Play! is really cool!
And it seems like a fitting choice considering that the core Kafka code
base is in scala :)
--
Felix
On Tue, Feb 7, 2012 at 1:01 PM, Neha Narkhede <ne...@gmail.com>wrote:
> Evan,
>
> That is a good idea. Would you mind filing a JIRA so that the
> community can help discussing viable solutions there ?
>
> Thanks,
> Neha
>
> On Tue, Feb 7, 2012 at 9:47 AM, Evan Chan <ev...@ooyala.com> wrote:
> > Is there a trick to getting jconsole to work with Kafka? It just hangs
> > and doesn't return any info.
> >
> > BTW I was thinking of creating a web console for Kafka. One that would
> > aggregate the info from both JMX and ZK and present it visually, so one
> can
> > easily see the partitions, consumers, and offsets or at least get a sense
> > of how big the queue is for each consumer.
> >
> > I'd like the project to be supported by the community though. Either
> JRuby
> > or Scala (with Play! or something like that).
> >
> > Any thoughts?
> >
> > On Tue, Feb 7, 2012 at 9:29 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> >> SuoNayi,
> >>
> >> Currently, we have jmx that reports #messages (per topic) consumed by a
> >> consumer (and soon to have #messages produced as well). We don't have
> >> anything that measures #messages in the queue. This is partly due to
> that
> >> in Kafka, messages are not immediately deleted after consumption and are
> >> only deleted after certain amount of time has passed. We do have a tool
> >> ConsumerOffsetChecker that reports the lag of the consumer in bytes.
> >> Finally, we don't have anything that exposes #partitions. This is
> probably
> >> useful to have. Could you open a jira for that?
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >> 2012/2/7 SuoNayi <su...@163.com>
> >>
> >> > Hi all,
> >> >
> >> > It's natural that I want to see how many messages in my queue
> >> >
> >> > or how many consumers are active on my broker.
> >> >
> >> > I want to know about how many partitions as well.
> >> >
> >> > It seems that no related document is available in the official
> website or
> >> > I miss something?
> >> >
> >> >
> >> >
> >> > SuoNayi
> >>
> >
> >
> >
> > --
> > --
> > *Evan Chan*
> > Senior Software Engineer |
> > ev@ooyala.com | (650) 996-4600
> > www.ooyala.com | blog <http://www.ooyala.com/blog> |
> > @ooyala<http://www.twitter.com/ooyala>
>
Re: Is there any web console to monitor what's the status about broker?
Posted by Neha Narkhede <ne...@gmail.com>.
Evan,
That is a good idea. Would you mind filing a JIRA so that the
community can help discussing viable solutions there ?
Thanks,
Neha
On Tue, Feb 7, 2012 at 9:47 AM, Evan Chan <ev...@ooyala.com> wrote:
> Is there a trick to getting jconsole to work with Kafka? It just hangs
> and doesn't return any info.
>
> BTW I was thinking of creating a web console for Kafka. One that would
> aggregate the info from both JMX and ZK and present it visually, so one can
> easily see the partitions, consumers, and offsets or at least get a sense
> of how big the queue is for each consumer.
>
> I'd like the project to be supported by the community though. Either JRuby
> or Scala (with Play! or something like that).
>
> Any thoughts?
>
> On Tue, Feb 7, 2012 at 9:29 AM, Jun Rao <ju...@gmail.com> wrote:
>
>> SuoNayi,
>>
>> Currently, we have jmx that reports #messages (per topic) consumed by a
>> consumer (and soon to have #messages produced as well). We don't have
>> anything that measures #messages in the queue. This is partly due to that
>> in Kafka, messages are not immediately deleted after consumption and are
>> only deleted after certain amount of time has passed. We do have a tool
>> ConsumerOffsetChecker that reports the lag of the consumer in bytes.
>> Finally, we don't have anything that exposes #partitions. This is probably
>> useful to have. Could you open a jira for that?
>>
>> Thanks,
>>
>> Jun
>>
>> 2012/2/7 SuoNayi <su...@163.com>
>>
>> > Hi all,
>> >
>> > It's natural that I want to see how many messages in my queue
>> >
>> > or how many consumers are active on my broker.
>> >
>> > I want to know about how many partitions as well.
>> >
>> > It seems that no related document is available in the official website or
>> > I miss something?
>> >
>> >
>> >
>> > SuoNayi
>>
>
>
>
> --
> --
> *Evan Chan*
> Senior Software Engineer |
> ev@ooyala.com | (650) 996-4600
> www.ooyala.com | blog <http://www.ooyala.com/blog> |
> @ooyala<http://www.twitter.com/ooyala>
Re: Is there any web console to monitor what's the status about broker?
Posted by Evan Chan <ev...@ooyala.com>.
Is there a trick to getting jconsole to work with Kafka? It just hangs
and doesn't return any info.
BTW I was thinking of creating a web console for Kafka. One that would
aggregate the info from both JMX and ZK and present it visually, so one can
easily see the partitions, consumers, and offsets or at least get a sense
of how big the queue is for each consumer.
I'd like the project to be supported by the community though. Either JRuby
or Scala (with Play! or something like that).
Any thoughts?
On Tue, Feb 7, 2012 at 9:29 AM, Jun Rao <ju...@gmail.com> wrote:
> SuoNayi,
>
> Currently, we have jmx that reports #messages (per topic) consumed by a
> consumer (and soon to have #messages produced as well). We don't have
> anything that measures #messages in the queue. This is partly due to that
> in Kafka, messages are not immediately deleted after consumption and are
> only deleted after certain amount of time has passed. We do have a tool
> ConsumerOffsetChecker that reports the lag of the consumer in bytes.
> Finally, we don't have anything that exposes #partitions. This is probably
> useful to have. Could you open a jira for that?
>
> Thanks,
>
> Jun
>
> 2012/2/7 SuoNayi <su...@163.com>
>
> > Hi all,
> >
> > It's natural that I want to see how many messages in my queue
> >
> > or how many consumers are active on my broker.
> >
> > I want to know about how many partitions as well.
> >
> > It seems that no related document is available in the official website or
> > I miss something?
> >
> >
> >
> > SuoNayi
>
--
--
*Evan Chan*
Senior Software Engineer |
ev@ooyala.com | (650) 996-4600
www.ooyala.com | blog <http://www.ooyala.com/blog> |
@ooyala<http://www.twitter.com/ooyala>
Re: Is there any web console to monitor what's the status about broker?
Posted by Jun Rao <ju...@gmail.com>.
SuoNayi,
Currently, we have jmx that reports #messages (per topic) consumed by a
consumer (and soon to have #messages produced as well). We don't have
anything that measures #messages in the queue. This is partly due to that
in Kafka, messages are not immediately deleted after consumption and are
only deleted after certain amount of time has passed. We do have a tool
ConsumerOffsetChecker that reports the lag of the consumer in bytes.
Finally, we don't have anything that exposes #partitions. This is probably
useful to have. Could you open a jira for that?
Thanks,
Jun
2012/2/7 SuoNayi <su...@163.com>
> Hi all,
>
> It's natural that I want to see how many messages in my queue
>
> or how many consumers are active on my broker.
>
> I want to know about how many partitions as well.
>
> It seems that no related document is available in the official website or
> I miss something?
>
>
>
> SuoNayi