You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Jay Kreps <ja...@gmail.com> on 2015/02/10 17:07:58 UTC
Re: Lack of JMX LogCleaner and LogCleanerManager metrics
Probably you need to enable the log cleaner for those to show up? We
disable it by default and so I think those metrics never get created.
-Jay
On Tue, Feb 10, 2015 at 3:33 AM, otis@sematext.com <st...@gmail.com>
wrote:
> Hello,
>
> I have a problem with some JMX metrics. In Kafka source code I see
> LogCleaner (has metrics: 'max-buffer-utilization-percent',
> 'cleaner-recopy-percent', 'max-clean-time-secs') and LogCleanerManager (has
> metric: 'max-dirty-percent')
>
> But I don't see this beans&metrics in jconsole when I attach to broker
> process.
> Any ideas why? Maybe this beans live only short time during cleanup that's
> why I don't see them in jconsole?
>
> I use Kafka monitoring tool which show me other metrics, but not these
> 'Log clean' metrics.
>
> Below are some SPM graphs showing the state of my system.
> Here's the 'Topic Bytes/Messages' graph:
> https://apps.sematext.com/spm-reports/s/omV6bYPGAG
Re: Lack of JMX LogCleaner and LogCleanerManager metrics
Posted by Gwen Shapira <gs...@cloudera.com>.
I'm counting this as three committer +1 :)
Opened:
KAFKA-1944 - Rename LogCleaner and related classes to LogCompactor
<https://issues.apache.org/jira/browse/KAFKA-1944>
On Tue, Feb 10, 2015 at 4:31 PM, Guozhang Wang <wa...@gmail.com> wrote:
> +1
>
> On Tue, Feb 10, 2015 at 2:23 PM, Joel Koshy <jj...@gmail.com> wrote:
>
> > +1
> >
> > On Tue, Feb 10, 2015 at 01:32:13PM -0800, Jay Kreps wrote:
> > > I agree that would be a better name. We could rename it if everyone
> likes
> > > Compactor better.
> > >
> > > -Jay
> > >
> > > On Tue, Feb 10, 2015 at 9:33 AM, Gwen Shapira <gs...@cloudera.com>
> > wrote:
> > >
> > > > btw. the name LogCleaner is seriously misleading. Its more of a log
> > > > compacter.
> > > > Deleting old logs happens elsewhere from what I've seen.
> > > >
> > > > Gwen
> > > >
> > > > On Tue, Feb 10, 2015 at 8:07 AM, Jay Kreps <ja...@gmail.com>
> > wrote:
> > > >
> > > > > Probably you need to enable the log cleaner for those to show up?
> We
> > > > > disable it by default and so I think those metrics never get
> created.
> > > > >
> > > > > -Jay
> > > > >
> > > > > On Tue, Feb 10, 2015 at 3:33 AM, otis@sematext.com <
> > > > st.comm.conn@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I have a problem with some JMX metrics. In Kafka source code I
> see
> > > > > > LogCleaner (has metrics: 'max-buffer-utilization-percent',
> > > > > > 'cleaner-recopy-percent', 'max-clean-time-secs') and
> > LogCleanerManager
> > > > > (has
> > > > > > metric: 'max-dirty-percent')
> > > > > >
> > > > > > But I don't see this beans&metrics in jconsole when I attach to
> > broker
> > > > > > process.
> > > > > > Any ideas why? Maybe this beans live only short time during
> cleanup
> > > > > that's
> > > > > > why I don't see them in jconsole?
> > > > > >
> > > > > > I use Kafka monitoring tool which show me other metrics, but not
> > these
> > > > > > 'Log clean' metrics.
> > > > > >
> > > > > > Below are some SPM graphs showing the state of my system.
> > > > > > Here's the 'Topic Bytes/Messages' graph:
> > > > > > https://apps.sematext.com/spm-reports/s/omV6bYPGAG
> > > > >
> > > >
> >
> >
>
>
> --
> -- Guozhang
>
Re: Lack of JMX LogCleaner and LogCleanerManager metrics
Posted by Guozhang Wang <wa...@gmail.com>.
+1
On Tue, Feb 10, 2015 at 2:23 PM, Joel Koshy <jj...@gmail.com> wrote:
> +1
>
> On Tue, Feb 10, 2015 at 01:32:13PM -0800, Jay Kreps wrote:
> > I agree that would be a better name. We could rename it if everyone likes
> > Compactor better.
> >
> > -Jay
> >
> > On Tue, Feb 10, 2015 at 9:33 AM, Gwen Shapira <gs...@cloudera.com>
> wrote:
> >
> > > btw. the name LogCleaner is seriously misleading. Its more of a log
> > > compacter.
> > > Deleting old logs happens elsewhere from what I've seen.
> > >
> > > Gwen
> > >
> > > On Tue, Feb 10, 2015 at 8:07 AM, Jay Kreps <ja...@gmail.com>
> wrote:
> > >
> > > > Probably you need to enable the log cleaner for those to show up? We
> > > > disable it by default and so I think those metrics never get created.
> > > >
> > > > -Jay
> > > >
> > > > On Tue, Feb 10, 2015 at 3:33 AM, otis@sematext.com <
> > > st.comm.conn@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > I have a problem with some JMX metrics. In Kafka source code I see
> > > > > LogCleaner (has metrics: 'max-buffer-utilization-percent',
> > > > > 'cleaner-recopy-percent', 'max-clean-time-secs') and
> LogCleanerManager
> > > > (has
> > > > > metric: 'max-dirty-percent')
> > > > >
> > > > > But I don't see this beans&metrics in jconsole when I attach to
> broker
> > > > > process.
> > > > > Any ideas why? Maybe this beans live only short time during cleanup
> > > > that's
> > > > > why I don't see them in jconsole?
> > > > >
> > > > > I use Kafka monitoring tool which show me other metrics, but not
> these
> > > > > 'Log clean' metrics.
> > > > >
> > > > > Below are some SPM graphs showing the state of my system.
> > > > > Here's the 'Topic Bytes/Messages' graph:
> > > > > https://apps.sematext.com/spm-reports/s/omV6bYPGAG
> > > >
> > >
>
>
--
-- Guozhang
Re: Lack of JMX LogCleaner and LogCleanerManager metrics
Posted by Joel Koshy <jj...@gmail.com>.
+1
On Tue, Feb 10, 2015 at 01:32:13PM -0800, Jay Kreps wrote:
> I agree that would be a better name. We could rename it if everyone likes
> Compactor better.
>
> -Jay
>
> On Tue, Feb 10, 2015 at 9:33 AM, Gwen Shapira <gs...@cloudera.com> wrote:
>
> > btw. the name LogCleaner is seriously misleading. Its more of a log
> > compacter.
> > Deleting old logs happens elsewhere from what I've seen.
> >
> > Gwen
> >
> > On Tue, Feb 10, 2015 at 8:07 AM, Jay Kreps <ja...@gmail.com> wrote:
> >
> > > Probably you need to enable the log cleaner for those to show up? We
> > > disable it by default and so I think those metrics never get created.
> > >
> > > -Jay
> > >
> > > On Tue, Feb 10, 2015 at 3:33 AM, otis@sematext.com <
> > st.comm.conn@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I have a problem with some JMX metrics. In Kafka source code I see
> > > > LogCleaner (has metrics: 'max-buffer-utilization-percent',
> > > > 'cleaner-recopy-percent', 'max-clean-time-secs') and LogCleanerManager
> > > (has
> > > > metric: 'max-dirty-percent')
> > > >
> > > > But I don't see this beans&metrics in jconsole when I attach to broker
> > > > process.
> > > > Any ideas why? Maybe this beans live only short time during cleanup
> > > that's
> > > > why I don't see them in jconsole?
> > > >
> > > > I use Kafka monitoring tool which show me other metrics, but not these
> > > > 'Log clean' metrics.
> > > >
> > > > Below are some SPM graphs showing the state of my system.
> > > > Here's the 'Topic Bytes/Messages' graph:
> > > > https://apps.sematext.com/spm-reports/s/omV6bYPGAG
> > >
> >
Re: Lack of JMX LogCleaner and LogCleanerManager metrics
Posted by Jay Kreps <ja...@gmail.com>.
I agree that would be a better name. We could rename it if everyone likes
Compactor better.
-Jay
On Tue, Feb 10, 2015 at 9:33 AM, Gwen Shapira <gs...@cloudera.com> wrote:
> btw. the name LogCleaner is seriously misleading. Its more of a log
> compacter.
> Deleting old logs happens elsewhere from what I've seen.
>
> Gwen
>
> On Tue, Feb 10, 2015 at 8:07 AM, Jay Kreps <ja...@gmail.com> wrote:
>
> > Probably you need to enable the log cleaner for those to show up? We
> > disable it by default and so I think those metrics never get created.
> >
> > -Jay
> >
> > On Tue, Feb 10, 2015 at 3:33 AM, otis@sematext.com <
> st.comm.conn@gmail.com
> > >
> > wrote:
> >
> > > Hello,
> > >
> > > I have a problem with some JMX metrics. In Kafka source code I see
> > > LogCleaner (has metrics: 'max-buffer-utilization-percent',
> > > 'cleaner-recopy-percent', 'max-clean-time-secs') and LogCleanerManager
> > (has
> > > metric: 'max-dirty-percent')
> > >
> > > But I don't see this beans&metrics in jconsole when I attach to broker
> > > process.
> > > Any ideas why? Maybe this beans live only short time during cleanup
> > that's
> > > why I don't see them in jconsole?
> > >
> > > I use Kafka monitoring tool which show me other metrics, but not these
> > > 'Log clean' metrics.
> > >
> > > Below are some SPM graphs showing the state of my system.
> > > Here's the 'Topic Bytes/Messages' graph:
> > > https://apps.sematext.com/spm-reports/s/omV6bYPGAG
> >
>
Re: Lack of JMX LogCleaner and LogCleanerManager metrics
Posted by Gwen Shapira <gs...@cloudera.com>.
btw. the name LogCleaner is seriously misleading. Its more of a log
compacter.
Deleting old logs happens elsewhere from what I've seen.
Gwen
On Tue, Feb 10, 2015 at 8:07 AM, Jay Kreps <ja...@gmail.com> wrote:
> Probably you need to enable the log cleaner for those to show up? We
> disable it by default and so I think those metrics never get created.
>
> -Jay
>
> On Tue, Feb 10, 2015 at 3:33 AM, otis@sematext.com <st.comm.conn@gmail.com
> >
> wrote:
>
> > Hello,
> >
> > I have a problem with some JMX metrics. In Kafka source code I see
> > LogCleaner (has metrics: 'max-buffer-utilization-percent',
> > 'cleaner-recopy-percent', 'max-clean-time-secs') and LogCleanerManager
> (has
> > metric: 'max-dirty-percent')
> >
> > But I don't see this beans&metrics in jconsole when I attach to broker
> > process.
> > Any ideas why? Maybe this beans live only short time during cleanup
> that's
> > why I don't see them in jconsole?
> >
> > I use Kafka monitoring tool which show me other metrics, but not these
> > 'Log clean' metrics.
> >
> > Below are some SPM graphs showing the state of my system.
> > Here's the 'Topic Bytes/Messages' graph:
> > https://apps.sematext.com/spm-reports/s/omV6bYPGAG
>