You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Lars George (JIRA)" <ji...@apache.org> on 2010/01/26 15:26:34 UTC
[jira] Created: (HBASE-2168) [jmxtoolkit] Add JMXToolkit contrib
with Cacti graphing and Nagios checking
[jmxtoolkit] Add JMXToolkit contrib with Cacti graphing and Nagios checking
---------------------------------------------------------------------------
Key: HBASE-2168
URL: https://issues.apache.org/jira/browse/HBASE-2168
Project: Hadoop HBase
Issue Type: New Feature
Affects Versions: 0.20.4, 0.21.0
Reporter: Lars George
Assignee: Lars George
Fix For: 0.20.4, 0.21.0
Add a new contrib that allows to quickly build a package that enables easy graphing of JMX exported values.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2168) [jmxtoolkit] Add JMXToolkit contrib
with Cacti graphing and Nagios checking
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12805232#action_12805232 ]
Jean-Daniel Cryans commented on HBASE-2168:
-------------------------------------------
Lars,
This is a huge patch! Could give an overview of what this adds, which files does what, etc? Thx!
> [jmxtoolkit] Add JMXToolkit contrib with Cacti graphing and Nagios checking
> ---------------------------------------------------------------------------
>
> Key: HBASE-2168
> URL: https://issues.apache.org/jira/browse/HBASE-2168
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.4, 0.21.0
> Reporter: Lars George
> Assignee: Lars George
> Fix For: 0.20.4, 0.21.0
>
> Attachments: HBASE-2168.patch
>
>
> Add a new contrib that allows to quickly build a package that enables easy graphing of JMX exported values.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2168) [jmxtoolkit] Add JMXToolkit contrib
with Cacti graphing and Nagios checking
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830178#action_12830178 ]
stack commented on HBASE-2168:
------------------------------
The ganglia support comes from hadoop. In hbase we just emit metrics.
There isn't much to this jmxtoolkit. We might entertain adding it to core. How would it work? You'd add a target to build.xml and it'd generate the cacti drop-in jar. How would we know if its broken or not?
On the other hand, it looks nice where it is now, no? We already have an article on monitoring hbase. What if we added a section on nagios/cacti and pointed out to jmxtoolkit over in github?
(Lars, no vote was taken on contrib discussion -- just discussion to see what all are thinking on the topic of contribs).
> [jmxtoolkit] Add JMXToolkit contrib with Cacti graphing and Nagios checking
> ---------------------------------------------------------------------------
>
> Key: HBASE-2168
> URL: https://issues.apache.org/jira/browse/HBASE-2168
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.4, 0.21.0
> Reporter: Lars George
> Assignee: Lars George
> Fix For: 0.20.4, 0.21.0
>
> Attachments: HBASE-2168.patch
>
>
> Add a new contrib that allows to quickly build a package that enables easy graphing of JMX exported values.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2168) [jmxtoolkit] Add JMXToolkit contrib
with Cacti graphing and Nagios checking
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830104#action_12830104 ]
Lars George commented on HBASE-2168:
------------------------------------
Given the recent discussing about no contrib I assume this gets a -1 as it is somewhat not core and we have the supplied Ganglia support already. Nagios monitoring makes totally sense but can be added later on so no need to add this now. Am I reasoning that right?
http://github.com/larsgeorge/jmxtoolkit
> [jmxtoolkit] Add JMXToolkit contrib with Cacti graphing and Nagios checking
> ---------------------------------------------------------------------------
>
> Key: HBASE-2168
> URL: https://issues.apache.org/jira/browse/HBASE-2168
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.4, 0.21.0
> Reporter: Lars George
> Assignee: Lars George
> Fix For: 0.20.4, 0.21.0
>
> Attachments: HBASE-2168.patch
>
>
> Add a new contrib that allows to quickly build a package that enables easy graphing of JMX exported values.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2168) [jmxtoolkit] Add JMXToolkit contrib
with Cacti graphing and Nagios checking
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars George updated HBASE-2168:
-------------------------------
Attachment: HBASE-2168.patch
> [jmxtoolkit] Add JMXToolkit contrib with Cacti graphing and Nagios checking
> ---------------------------------------------------------------------------
>
> Key: HBASE-2168
> URL: https://issues.apache.org/jira/browse/HBASE-2168
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.4, 0.21.0
> Reporter: Lars George
> Assignee: Lars George
> Fix For: 0.20.4, 0.21.0
>
> Attachments: HBASE-2168.patch
>
>
> Add a new contrib that allows to quickly build a package that enables easy graphing of JMX exported values.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2168) [jmxtoolkit] Add JMXToolkit contrib
with Cacti graphing and Nagios checking
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830158#action_12830158 ]
Lars George commented on HBASE-2168:
------------------------------------
Indeed Ed, just until recently contrib was pretty much the right place. But then a vote came up to toss out every contrib and either make it part of core or force it to get into their own habitat. That seems to be the status-quo and in that light I thought I move what I submitted at least into my github repo. It will not even compile yet as the build.xml includes the build-contrib.xml from HBase. But I will fix it and make it so that it compiles at least. It still has the same purpose, provide a complete HBase oriented package and I will make sure I maintain it for that reason (standard checks and RPC graphs are missing still etc. anyways).
Hope that make sense.
> [jmxtoolkit] Add JMXToolkit contrib with Cacti graphing and Nagios checking
> ---------------------------------------------------------------------------
>
> Key: HBASE-2168
> URL: https://issues.apache.org/jira/browse/HBASE-2168
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.4, 0.21.0
> Reporter: Lars George
> Assignee: Lars George
> Fix For: 0.20.4, 0.21.0
>
> Attachments: HBASE-2168.patch
>
>
> Add a new contrib that allows to quickly build a package that enables easy graphing of JMX exported values.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2168) [jmxtoolkit] Add JMXToolkit contrib
with Cacti graphing and Nagios checking
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12805040#action_12805040 ]
Lars George commented on HBASE-2168:
------------------------------------
Please have a look at what I have so far and share your thoughts/ideas/scorn etc. with me.
> [jmxtoolkit] Add JMXToolkit contrib with Cacti graphing and Nagios checking
> ---------------------------------------------------------------------------
>
> Key: HBASE-2168
> URL: https://issues.apache.org/jira/browse/HBASE-2168
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.4, 0.21.0
> Reporter: Lars George
> Assignee: Lars George
> Fix For: 0.20.4, 0.21.0
>
> Attachments: HBASE-2168.patch
>
>
> Add a new contrib that allows to quickly build a package that enables easy graphing of JMX exported values.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2168) [jmxtoolkit] Add JMXToolkit contrib
with Cacti graphing and Nagios checking
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12805454#action_12805454 ]
Lars George commented on HBASE-2168:
------------------------------------
OK, the patch adds a new contrib package all complete with its own build.xml. Please read the INSTALL.txt and README.txt included to see what it does in detail. It adds a single class that acts as a swiss-army-knife JMX tool. It can scan servers to see what attributes and operations are exposed, query them subsequently and check the values to be in a certain range. It is based on what Ed Capriolo has implemented (http://www.jointhegrid.com/hadoop/) but simplifies the usage. Ed came from a generic perspective while I came from a HBase one, so I moved settings to an external properties file instead of having classes for each version. I adjusted his set of graph templates to match the new layout. The build.xml allows to build a jar matching the current release but also has the ability to build a custom jar for various reasons.
The INSTALL.txt explains how to do this all and how to use the resulting jar with Cacti and Nagios. Goal of the contrib is to enable our users to quickly set up graphing with more elaborate combined graphs (as opposed to graphs single values with Ganglia for example) and also monitoring with Nagios or similar tools to get notifications when something is awry.
> [jmxtoolkit] Add JMXToolkit contrib with Cacti graphing and Nagios checking
> ---------------------------------------------------------------------------
>
> Key: HBASE-2168
> URL: https://issues.apache.org/jira/browse/HBASE-2168
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.4, 0.21.0
> Reporter: Lars George
> Assignee: Lars George
> Fix For: 0.20.4, 0.21.0
>
> Attachments: HBASE-2168.patch
>
>
> Add a new contrib that allows to quickly build a package that enables easy graphing of JMX exported values.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2168) [jmxtoolkit] Add JMXToolkit contrib
with Cacti graphing and Nagios checking
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830143#action_12830143 ]
Edward Capriolo commented on HBASE-2168:
----------------------------------------
Lars,
It seems you are at the spot where I was a while back
http://issues.apache.org/jira/browse/HADOOP-4594. (pretty much the spot that inspired hadoop-cacti-jtg)
In a nutshell, hadoop+hbase has built in ganglia support. It works in that in does push metrics to a ganglia cluster, but it has its detractions. For me these are, often changing attributes and object names, missing vivid documentation of what counters mean, and being forced into a monitoring system (ganglia).
On one hand it makes a lot of sense to call cacti/nagios scripts "contrib" and take the stance that they are not easily unit testable and thus do not belong in hadoop core. Of course, you can reverse the argument "just because you can patch in ganglia support, does that mean that only an NMS that can be patched in (pushed based nms) should be officially supported"
Thus we have hadoop-cacti-jtg, because ganglia does not meet everyone's needs but it is the only NMS that seems to be able to make its way in. Every other NMS seems to be classified as unneeded contrib.
> [jmxtoolkit] Add JMXToolkit contrib with Cacti graphing and Nagios checking
> ---------------------------------------------------------------------------
>
> Key: HBASE-2168
> URL: https://issues.apache.org/jira/browse/HBASE-2168
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.4, 0.21.0
> Reporter: Lars George
> Assignee: Lars George
> Fix For: 0.20.4, 0.21.0
>
> Attachments: HBASE-2168.patch
>
>
> Add a new contrib that allows to quickly build a package that enables easy graphing of JMX exported values.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2168) [jmxtoolkit] Add JMXToolkit contrib
with Cacti graphing and Nagios checking
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830147#action_12830147 ]
Edward Capriolo commented on HBASE-2168:
----------------------------------------
For example I have noticed things like this.
http://www.matejunkie.com/hadoop-dfs-check-plugin-for-nagios/
This is a fine script, but this person took the approach of using sudo, shell scripts that parse the output of dfsadmin, awk, etc etc. If we had reasonable scripts included in the distribution that work with nagios (the most widely used NMS BTW), we would solve a practical problem for most people.
> [jmxtoolkit] Add JMXToolkit contrib with Cacti graphing and Nagios checking
> ---------------------------------------------------------------------------
>
> Key: HBASE-2168
> URL: https://issues.apache.org/jira/browse/HBASE-2168
> Project: Hadoop HBase
> Issue Type: New Feature
> Affects Versions: 0.20.4, 0.21.0
> Reporter: Lars George
> Assignee: Lars George
> Fix For: 0.20.4, 0.21.0
>
> Attachments: HBASE-2168.patch
>
>
> Add a new contrib that allows to quickly build a package that enables easy graphing of JMX exported values.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.