You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Gary Helmling (JIRA)" <ji...@apache.org> on 2009/07/30 02:39:14 UTC
[jira] Created: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Add support for exporting HBase metrics via JMX
-----------------------------------------------
Key: HBASE-1722
URL: https://issues.apache.org/jira/browse/HBASE-1722
Project: Hadoop HBase
Issue Type: Improvement
Components: ipc, master, regionserver
Affects Versions: 0.19.3
Reporter: Gary Helmling
Priority: Minor
Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
This patch adds JMX MBean support for reporting metrics collected in 3 areas:
* org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
* org.apache.hadoop.hbase.master.metrics.MasterMetrics
* org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751302#action_12751302 ]
stack commented on HBASE-1722:
------------------------------
@Gary:
I reviewed patch for TRUNK. Its great. I like the addition to monitoring docs particularly. Before applying, do you want to add in JMX environment variables to hbase-env.sh and include them in bin/hbase.sh rather than have users copy and paste out of the monitoring documentation? (Default, the JMX env variables can be null).
Otherwise, was going to apply to TRUNK and branch so would be in 0.20.1.
Thanks.
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.20.1
>
> Attachments: jmx_metrics-0.19.patch, jmx_support-trunk.patch, metrics_docs_0.19.patch, metrics_docs_trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-1722.
--------------------------
Resolution: Fixed
Fix Version/s: 0.19.4
Hadoop Flags: [Reviewed]
Thanks for the nice patch Gary. Any chance of one for hbase TRUNK with maybe a bit of documentation on how to set things up so metrics show in something like jconsole? Only issue was missing Apache copyright. In future, just watch line lengths. Hadoop project is 80 characters generally.
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.19.4
>
> Attachments: jmx_metrics-0.19.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754390#action_12754390 ]
Gary Helmling commented on HBASE-1722:
--------------------------------------
Excellent! Thanks, stack!
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: hbase-bin.patch, jmx.patch, jmx_metrics-0.19.patch, jmx_support-trunk.patch, metrics_docs_0.19.patch, metrics_docs_trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Helmling updated HBASE-1722:
---------------------------------
Attachment: hbase-bin.patch
Here's a patch for the bin/hbase and conf/hbase-env.sh scripts and conf/hadoop-metrics.properties, adding placeholder configurations to ease the process of enabling JMX monitoring. If the conditional checking in bin/hbase for a "start" action is too weird though, feel free to skip these changes. The config is documented in the metrics.xml patch. This just provides a bit more guidance for enabling.
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.20.1
>
> Attachments: hbase-bin.patch, jmx_metrics-0.19.patch, jmx_support-trunk.patch, metrics_docs_0.19.patch, metrics_docs_trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Helmling updated HBASE-1722:
---------------------------------
Attachment: metrics_docs_trunk.patch
Here's a patch for metrics.xml in trunk.
This is my first time using Forrest, so let me know if anything seems off or missing.
The last bit about editing bin/hbase is kind of hacky, but is necessary since we currently don't split out opts for master vs. regionserver vs. shell, etc.
I'll attach a separate version for 0.19 -- the only real difference is the domain where the MBeans show up.
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: jmx_metrics-0.19.patch, jmx_support-trunk.patch, metrics_docs_trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack reopened HBASE-1722:
--------------------------
Reopen till we do commit against TRUNK
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: jmx_metrics-0.19.patch, jmx_support-trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1722:
-------------------------
Fix Version/s: (was: 0.21.0)
0.20.1
Backport to 0.20 branch.
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Assignee: Gary Helmling
> Priority: Minor
> Fix For: 0.20.1
>
> Attachments: hbase-bin.patch, jmx.patch, jmx_metrics-0.19.patch, jmx_support-trunk.patch, metrics_docs_0.19.patch, metrics_docs_trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737300#action_12737300 ]
Gary Helmling commented on HBASE-1722:
--------------------------------------
Ok, I'll do a write up in metrics.xml instead and post patches for that against 0.19 and trunk. Thanks for the quick review!
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: jmx_metrics-0.19.patch, jmx_support-trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737248#action_12737248 ]
stack commented on HBASE-1722:
------------------------------
Patch looks good.
On docs, add your comment here, http://hadoop.apache.org/hbase/docs/r0.19.3/metrics.html, rather than to wiki. If its checked in as part of source, doc gets packaged with the version it targets. To find the page to edit, its at src/docs/src/documentation/content/xdocs/metrics.xml in the repository.
Thanks Gary.
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: jmx_metrics-0.19.patch, jmx_support-trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1722:
-------------------------
Fix Version/s: (was: 0.19.4)
0.21.0
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: jmx_metrics-0.19.patch, jmx_support-trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1722:
-------------------------
Attachment: jmx.patch
Roll up of Gary's classes -- Its what I committed to TRUNK.
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: hbase-bin.patch, jmx.patch, jmx_metrics-0.19.patch, jmx_support-trunk.patch, metrics_docs_0.19.patch, metrics_docs_trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12762446#action_12762446 ]
stack commented on HBASE-1722:
------------------------------
I backported this patch and the metrics jmx doc. to 0.20.1.
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Assignee: Gary Helmling
> Priority: Minor
> Fix For: 0.20.1
>
> Attachments: hbase-bin.patch, jmx.patch, jmx_metrics-0.19.patch, jmx_support-trunk.patch, metrics_docs_0.19.patch, metrics_docs_trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1722:
-------------------------
Resolution: Fixed
Assignee: Gary Helmling
Status: Resolved (was: Patch Available)
Committed to TRUNK. Thanks for the nice patch Gary.
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Assignee: Gary Helmling
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: hbase-bin.patch, jmx.patch, jmx_metrics-0.19.patch, jmx_support-trunk.patch, metrics_docs_0.19.patch, metrics_docs_trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1722:
-------------------------
Fix Version/s: (was: 0.20.1)
0.21.0
Moving to TRUNK
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: hbase-bin.patch, jmx.patch, jmx_metrics-0.19.patch, jmx_support-trunk.patch, metrics_docs_0.19.patch, metrics_docs_trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Helmling updated HBASE-1722:
---------------------------------
Attachment: jmx_metrics-0.19.patch
This patch provide JMX support against the current 0.19 branch, adding a JMX MBean interface and implementing class for each of the 3 metrics areas in the issue.
Please let me know of any problems, code style issues, etc.
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Attachments: jmx_metrics-0.19.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Helmling updated HBASE-1722:
---------------------------------
Attachment: jmx_support-trunk.patch
Here's a patch against current trunk. This one was generated with git, but I was able to test apply cleanly with patch -p1.
This one was even easier thanks to MetricsDynamicMBeanBase in hadoop 0.20! I was able to drop the need for separate interfaces and pass through accessors for each metric.
I'll put up some docs on enable JMX monitoring on the wiki. For remote monitoring this may require some tweaks to hbase-env.sh and the hbase shell script -- like separating out option vars for master vs. regionservers.
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.19.4
>
> Attachments: jmx_metrics-0.19.patch, jmx_support-trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753420#action_12753420 ]
stack commented on HBASE-1722:
------------------------------
Thanks for the new patch Gary. I took a look at it. I like the addition of HBASE_REGIONSERVER_OPTIONS, etc., only I think itneeds more work; it works for setting jmx options per daemon but would be sweet if could set jvm mx and ms per daemon (and for zookeeper) as per HBASE-1687. Its a little tricky because we have that HBASE_HEAPSIZE thingy.
But no biggie. You have done the work documenting how to make jmx run with your additions to the metrics page. Let me commit. Folks who want to jmx can just read your metrics additions. 'll commit to TRUNK. I won't add to branch, unless you insist. Lets fix the above in a new issue or as part of HBASE-1687.
When we add suggested jmx options to hbase-env.sh, maybe default should be no default? Its a bit easier than pointing to a file with user login and password for jconsole?
Good stuff Gary.
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.20.1
>
> Attachments: hbase-bin.patch, jmx_metrics-0.19.patch, jmx_support-trunk.patch, metrics_docs_0.19.patch, metrics_docs_trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754383#action_12754383 ]
stack commented on HBASE-1722:
------------------------------
Update: I committed patch added by Ryan to hbase-1687. His patch was almost exactly the content of hbase-bin.patch above. My reasons for omitting hbase-bin.patch didn't hold up under scrutiny. The HBASE_HEAPSIZE appears before the HBASE_*_OPTS added by his patch (and by hbase-bin.patch) and because the last -Xmx/-Xms wins, adding -Xmx/-Xms that override HBASE_HEAPSIZE works.
On commit of HBASE-1687, I amended the metrics.xml documentation to remove mention of HBASE_*_OPTS addition now its no longer needed.
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: hbase-bin.patch, jmx.patch, jmx_metrics-0.19.patch, jmx_support-trunk.patch, metrics_docs_0.19.patch, metrics_docs_trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Helmling updated HBASE-1722:
---------------------------------
Attachment: metrics_docs_0.19.patch
Here is a patch for metrics.xml against 0.19. The only change is the domain for where the MBeans appear (hadoop.dfs) -- this is controlled by the Hadoop MBeanUtil class.
Again if anything's missing or additional information would be useful, let me know.
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: jmx_metrics-0.19.patch, jmx_support-trunk.patch, metrics_docs_0.19.patch, metrics_docs_trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1722) Add support for exporting HBase
metrics via JMX
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1722:
-------------------------
Fix Version/s: (was: 0.21.0)
0.20.1
Status: Patch Available (was: Reopened)
> Add support for exporting HBase metrics via JMX
> -----------------------------------------------
>
> Key: HBASE-1722
> URL: https://issues.apache.org/jira/browse/HBASE-1722
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: ipc, master, regionserver
> Affects Versions: 0.19.3
> Reporter: Gary Helmling
> Priority: Minor
> Fix For: 0.20.1
>
> Attachments: jmx_metrics-0.19.patch, jmx_support-trunk.patch, metrics_docs_0.19.patch, metrics_docs_trunk.patch
>
>
> Current HBase metrics collection and reporting (for the 0.19 releases at least) doesn't seem to export any of the metrics as MBeans via JMX.
> This patch adds JMX MBean support for reporting metrics collected in 3 areas:
> * org.apache.hadoop.hbase.ipc.HBaseRpcMetrics
> * org.apache.hadoop.hbase.master.metrics.MasterMetrics
> * org.apache.hadoop.hbase.regionserver.metrics.RegionServerMetrics
> This patch is only against the 0.19 branch, as the metrics API seems to have changed a bit in Hadoop 0.20, including adding support for dynamic MBeans. I'll add a patch reworked for trunk shortly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.