You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by Miguel Pereira <mi...@gmail.com> on 2012/07/13 21:44:08 UTC
Re: metric gathering
Hey guys,
were would I enable a jmx port for accumulo?
the -D options i want to use are
-Dcom.sun.management.jmxremote.port=1105
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
but i don't know were to put them, accumulo-env.sh ?
I plan to use jmxtrans to feed metrics to ganglia & maybe graphite
Cheers,
Miguel
On Fri, Jun 29, 2012 at 6:52 PM, Eric Newton <er...@gmail.com> wrote:
> Edit accumulo-metrics.xml, turn the false settings to true, restart,
> and then look at the exposed JMX beans with jconsole. You may need to
> expose a jmx port with various "-D" options, none of which I remember
> how to do.
>
> I don't know much about ganglia. I've used it a little, but
> displaying 100's of graphs on a web page is almost as bad as the the
> lack of visualization and historical data in the accumulo monitor. I
> don't know how to configure it.
>
> I didn't find it difficult to modify tcollector, the performance
> collector for OpenTSDB, to collect metrics for the Accumulo tservers.
>
> If you do manage it, please post instructions!
>
> -Eric
>
> On Fri, Jun 29, 2012 at 5:07 PM, Miguel Pereira
> <mi...@gmail.com> wrote:
> > I was hoping the accumulo conf dir would have a hadoop-metrics.properties
> > file similar to hbase were you just drop these lines in..
> >
> > hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> > hbase.period=10
> > hbase.servers=master.yourcompany.com:8649
> >
> > jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> > jvm.period=10
> > jvm.servers=master.yourcompany.com:8649
> >
> > rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> > rpc.period=10
> > rpc.servers=master.yourcompany.com:8649
> >
> >
> > I did spot the accumulo-metrics.xml and the metrics page in the doc
> > directory. Anyone have a few examples of how to use JMX here, I feel
> like i
> > am really close.
> >
> > cheers and have a great weekend,
> > Miguel
> >
> >
> > On Fri, Jun 29, 2012 at 2:00 PM, Eric Newton <er...@gmail.com>
> wrote:
> >
> >> I do most of my monitoring (and first-line trouble-shooting) via the
> >> Accumulo monitor page.
> >>
> >> I know most teams use XML to import data into their monitoring systems
> >> (based on the complaints I received when I changed the XML).
> >>
> >> I did port OpenTSDB to Accumulo; and it reads data via JMX.
> >>
> >> I used to work for Zenoss, so I'm partial to it, too.
> >>
> >> -Eric
> >>
> >> On Fri, Jun 29, 2012 at 11:55 AM, Miguel Pereira
> >> <mi...@gmail.com> wrote:
> >> > Thanks Eric!
> >> >
> >> > Are you guys currently using ganglia or is most of your monitoring
> done
> >> > through jmx?
> >> >
> >> > -Miguel
> >> >
> >> > On Fri, Jun 29, 2012 at 11:09 AM, Eric Newton <er...@gmail.com>
> >> wrote:
> >> >
> >> >> You have three options:
> >> >>
> >> >> 1) You can turn on JMX monitoring.
> >> >> 2) You can use org.apache.accumulo.server.test.GetMasterStats to
> print
> >> >> the data the monitor uses
> >> >> 3) You can pull the xml from the monitor page
> >> >>
> >> >> -Eric
> >> >>
> >> >> On Fri, Jun 29, 2012 at 11:03 AM, Miguel Pereira
> >> >> <mi...@gmail.com> wrote:
> >> >> > Hi all,
> >> >> >
> >> >> > Is there an easy way to access Accumulo metrics? I noticed
> >> >> > org.apache.accumulo.server.monitor
> >> >> > is used for the many of the metrics on the monitoring page. Is
> there
> >> any
> >> >> > additional api to support performance gathering? Also what do you
> guys
> >> >> think
> >> >> > would be important to measure that could support the dev team? My
> >> goal is
> >> >> > to use ganglia with accumulo.
> >> >> >
> >> >> > Thanks,
> >> >> > Miguel Pereira
> >> >>
> >>
>
Re: metric gathering
Posted by Miguel Pereira <mi...@gmail.com>.
Thanks for the help Eric,
I successfully queried accumulo metrics with jmxtrans and passed them to
ganglia.
Going to try it with an actual cluster before posting instructions..
Cheers,
Miguel
On Fri, Jul 13, 2012 at 4:07 PM, Eric Newton <er...@gmail.com> wrote:
> You can put them in your accumulo-env.sh. Be sure to enable the
> metrics in the accumulo-metrix.xml file, too.
>
> I did this when I ported OpenTSDB to accumulo. It worked great.
>
> -Eric
>
> On Fri, Jul 13, 2012 at 3:44 PM, Miguel Pereira
> <mi...@gmail.com> wrote:
> > Hey guys,
> >
> > were would I enable a jmx port for accumulo?
> > the -D options i want to use are
> >
> > -Dcom.sun.management.jmxremote.port=1105
> > -Dcom.sun.management.jmxremote.authenticate=false
> > -Dcom.sun.management.jmxremote.ssl=false
> >
> > but i don't know were to put them, accumulo-env.sh ?
> > I plan to use jmxtrans to feed metrics to ganglia & maybe graphite
> >
> > Cheers,
> > Miguel
> >
> > On Fri, Jun 29, 2012 at 6:52 PM, Eric Newton <er...@gmail.com>
> wrote:
> >
> >> Edit accumulo-metrics.xml, turn the false settings to true, restart,
> >> and then look at the exposed JMX beans with jconsole. You may need to
> >> expose a jmx port with various "-D" options, none of which I remember
> >> how to do.
> >>
> >> I don't know much about ganglia. I've used it a little, but
> >> displaying 100's of graphs on a web page is almost as bad as the the
> >> lack of visualization and historical data in the accumulo monitor. I
> >> don't know how to configure it.
> >>
> >> I didn't find it difficult to modify tcollector, the performance
> >> collector for OpenTSDB, to collect metrics for the Accumulo tservers.
> >>
> >> If you do manage it, please post instructions!
> >>
> >> -Eric
> >>
> >> On Fri, Jun 29, 2012 at 5:07 PM, Miguel Pereira
> >> <mi...@gmail.com> wrote:
> >> > I was hoping the accumulo conf dir would have a
> hadoop-metrics.properties
> >> > file similar to hbase were you just drop these lines in..
> >> >
> >> > hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >> > hbase.period=10
> >> > hbase.servers=master.yourcompany.com:8649
> >> >
> >> > jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >> > jvm.period=10
> >> > jvm.servers=master.yourcompany.com:8649
> >> >
> >> > rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >> > rpc.period=10
> >> > rpc.servers=master.yourcompany.com:8649
> >> >
> >> >
> >> > I did spot the accumulo-metrics.xml and the metrics page in the doc
> >> > directory. Anyone have a few examples of how to use JMX here, I feel
> >> like i
> >> > am really close.
> >> >
> >> > cheers and have a great weekend,
> >> > Miguel
> >> >
> >> >
> >> > On Fri, Jun 29, 2012 at 2:00 PM, Eric Newton <er...@gmail.com>
> >> wrote:
> >> >
> >> >> I do most of my monitoring (and first-line trouble-shooting) via the
> >> >> Accumulo monitor page.
> >> >>
> >> >> I know most teams use XML to import data into their monitoring
> systems
> >> >> (based on the complaints I received when I changed the XML).
> >> >>
> >> >> I did port OpenTSDB to Accumulo; and it reads data via JMX.
> >> >>
> >> >> I used to work for Zenoss, so I'm partial to it, too.
> >> >>
> >> >> -Eric
> >> >>
> >> >> On Fri, Jun 29, 2012 at 11:55 AM, Miguel Pereira
> >> >> <mi...@gmail.com> wrote:
> >> >> > Thanks Eric!
> >> >> >
> >> >> > Are you guys currently using ganglia or is most of your monitoring
> >> done
> >> >> > through jmx?
> >> >> >
> >> >> > -Miguel
> >> >> >
> >> >> > On Fri, Jun 29, 2012 at 11:09 AM, Eric Newton <
> eric.newton@gmail.com>
> >> >> wrote:
> >> >> >
> >> >> >> You have three options:
> >> >> >>
> >> >> >> 1) You can turn on JMX monitoring.
> >> >> >> 2) You can use org.apache.accumulo.server.test.GetMasterStats to
> >> print
> >> >> >> the data the monitor uses
> >> >> >> 3) You can pull the xml from the monitor page
> >> >> >>
> >> >> >> -Eric
> >> >> >>
> >> >> >> On Fri, Jun 29, 2012 at 11:03 AM, Miguel Pereira
> >> >> >> <mi...@gmail.com> wrote:
> >> >> >> > Hi all,
> >> >> >> >
> >> >> >> > Is there an easy way to access Accumulo metrics? I noticed
> >> >> >> > org.apache.accumulo.server.monitor
> >> >> >> > is used for the many of the metrics on the monitoring page. Is
> >> there
> >> >> any
> >> >> >> > additional api to support performance gathering? Also what do
> you
> >> guys
> >> >> >> think
> >> >> >> > would be important to measure that could support the dev team?
> My
> >> >> goal is
> >> >> >> > to use ganglia with accumulo.
> >> >> >> >
> >> >> >> > Thanks,
> >> >> >> > Miguel Pereira
> >> >> >>
> >> >>
> >>
>
Re: metric gathering
Posted by Eric Newton <er...@gmail.com>.
You can put them in your accumulo-env.sh. Be sure to enable the
metrics in the accumulo-metrix.xml file, too.
I did this when I ported OpenTSDB to accumulo. It worked great.
-Eric
On Fri, Jul 13, 2012 at 3:44 PM, Miguel Pereira
<mi...@gmail.com> wrote:
> Hey guys,
>
> were would I enable a jmx port for accumulo?
> the -D options i want to use are
>
> -Dcom.sun.management.jmxremote.port=1105
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false
>
> but i don't know were to put them, accumulo-env.sh ?
> I plan to use jmxtrans to feed metrics to ganglia & maybe graphite
>
> Cheers,
> Miguel
>
> On Fri, Jun 29, 2012 at 6:52 PM, Eric Newton <er...@gmail.com> wrote:
>
>> Edit accumulo-metrics.xml, turn the false settings to true, restart,
>> and then look at the exposed JMX beans with jconsole. You may need to
>> expose a jmx port with various "-D" options, none of which I remember
>> how to do.
>>
>> I don't know much about ganglia. I've used it a little, but
>> displaying 100's of graphs on a web page is almost as bad as the the
>> lack of visualization and historical data in the accumulo monitor. I
>> don't know how to configure it.
>>
>> I didn't find it difficult to modify tcollector, the performance
>> collector for OpenTSDB, to collect metrics for the Accumulo tservers.
>>
>> If you do manage it, please post instructions!
>>
>> -Eric
>>
>> On Fri, Jun 29, 2012 at 5:07 PM, Miguel Pereira
>> <mi...@gmail.com> wrote:
>> > I was hoping the accumulo conf dir would have a hadoop-metrics.properties
>> > file similar to hbase were you just drop these lines in..
>> >
>> > hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>> > hbase.period=10
>> > hbase.servers=master.yourcompany.com:8649
>> >
>> > jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>> > jvm.period=10
>> > jvm.servers=master.yourcompany.com:8649
>> >
>> > rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>> > rpc.period=10
>> > rpc.servers=master.yourcompany.com:8649
>> >
>> >
>> > I did spot the accumulo-metrics.xml and the metrics page in the doc
>> > directory. Anyone have a few examples of how to use JMX here, I feel
>> like i
>> > am really close.
>> >
>> > cheers and have a great weekend,
>> > Miguel
>> >
>> >
>> > On Fri, Jun 29, 2012 at 2:00 PM, Eric Newton <er...@gmail.com>
>> wrote:
>> >
>> >> I do most of my monitoring (and first-line trouble-shooting) via the
>> >> Accumulo monitor page.
>> >>
>> >> I know most teams use XML to import data into their monitoring systems
>> >> (based on the complaints I received when I changed the XML).
>> >>
>> >> I did port OpenTSDB to Accumulo; and it reads data via JMX.
>> >>
>> >> I used to work for Zenoss, so I'm partial to it, too.
>> >>
>> >> -Eric
>> >>
>> >> On Fri, Jun 29, 2012 at 11:55 AM, Miguel Pereira
>> >> <mi...@gmail.com> wrote:
>> >> > Thanks Eric!
>> >> >
>> >> > Are you guys currently using ganglia or is most of your monitoring
>> done
>> >> > through jmx?
>> >> >
>> >> > -Miguel
>> >> >
>> >> > On Fri, Jun 29, 2012 at 11:09 AM, Eric Newton <er...@gmail.com>
>> >> wrote:
>> >> >
>> >> >> You have three options:
>> >> >>
>> >> >> 1) You can turn on JMX monitoring.
>> >> >> 2) You can use org.apache.accumulo.server.test.GetMasterStats to
>> print
>> >> >> the data the monitor uses
>> >> >> 3) You can pull the xml from the monitor page
>> >> >>
>> >> >> -Eric
>> >> >>
>> >> >> On Fri, Jun 29, 2012 at 11:03 AM, Miguel Pereira
>> >> >> <mi...@gmail.com> wrote:
>> >> >> > Hi all,
>> >> >> >
>> >> >> > Is there an easy way to access Accumulo metrics? I noticed
>> >> >> > org.apache.accumulo.server.monitor
>> >> >> > is used for the many of the metrics on the monitoring page. Is
>> there
>> >> any
>> >> >> > additional api to support performance gathering? Also what do you
>> guys
>> >> >> think
>> >> >> > would be important to measure that could support the dev team? My
>> >> goal is
>> >> >> > to use ganglia with accumulo.
>> >> >> >
>> >> >> > Thanks,
>> >> >> > Miguel Pereira
>> >> >>
>> >>
>>