You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Luke Lu (JIRA)" <ji...@apache.org> on 2011/02/15 04:09:57 UTC
[jira] Created: (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Expose JMX with something like JMXProxyServlet
-----------------------------------------------
Key: HADOOP-7144
URL: https://issues.apache.org/jira/browse/HADOOP-7144
Project: Hadoop Common
Issue Type: New Feature
Reporter: Luke Lu
Fix For: 0.23.0
Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated HADOOP-7144:
----------------------------------------
Attachment: jmx.json
HADOOP-7411-trunk-V1.patch
HADOOP-7411-0.20.20X-V1.patch
This is version 1 of the patch. It includes javadocs, tests, and the servlet.
The servlet is placed under /jmx and returns data in a JSON format. Example output is attached too.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039600#comment-13039600 ]
Hadoop QA commented on HADOOP-7144:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12480460/jmx.json
against trunk revision 1127811.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/526//console
This message is automatically generated.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13036861#comment-13036861 ]
Robert Joseph Evans commented on HADOOP-7144:
---------------------------------------------
Just a quick note, I read through the code for the JMXProxy and it is a plain text format, not JSON or XML.
It only supports GET but that can be be modified with a ?set&attr=<KEY>&val=<VALUE> to set an attribute; ?get&attr=<KEY> to get a specific attribute; or ?qry=<QUERY> to output all attributes who's names match the query using the mBeanServer.queryNames method. the default is ?qry=*:* if nothing else if given.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-trunk-alpha.patch
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated HADOOP-7144:
----------------------------------------
Attachment: HADOOP-7411-trunk-alpha.patch
I have attached an initial patch pulling in JMXProxyServlet.java from Tomcat 7.0.14.
I pulled over two classes and part of a third.
org.apache.tomcat.util.ExceptionUtils - only the package name changed.
org.apache.tomcat.util.modeler.Registry - two methods from this class were moved into JMXProxyServlet getType and convertValue and modified to use mBeanServer from JMXProxyServlet instead of the MBeanServer from Registry
org.apache.catalina.manager.JMXProxyServlet - pacakge name changed. Registry was removed and replaced with a direct call to get MBeanServer, and two private methods.
There are no tests and no documentation changes right now (Which is why I marked it alpha). I wanted feedback on it first.
1) is the package OK org.apache.hadoop.jmx?
2) is the URL OK /jmx?
3) where would be an appropriate place to document this?
4) is this even the right thing to do? I am not sure how standard the format is. I need to dig into it a bit more to really understand it, because all I have done is a simple port.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-trunk-alpha.patch
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated HADOOP-7144:
----------------------------------------
Status: Patch Available (was: Open)
The following are the results of test-patch on the security branch.
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 4 new or modified tests.
[exec]
[exec] -1 javadoc. The javadoc tool appears to have generated 1 warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] -1 Eclipse classpath. The patch causes the Eclipse classpath to differ from the contents of the lib directories.
The eclipse path is bogus, and so is the findbugs. I don't know why they keep complaining but they do.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-0.20.20X-V2.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-V2.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13037077#comment-13037077 ]
Robert Joseph Evans commented on HADOOP-7144:
---------------------------------------------
Ok because we are going to go with JSON, and not use JMXProxyServlet more or less unchanged, I think I will take the core of it, but I will probably rename it so there is no confusion with the Tomcat one.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-trunk-alpha.patch
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048936#comment-13048936 ]
Luke Lu commented on HADOOP-7144:
---------------------------------
bq. Should the access be per mbean property?
The initial customers for this jira are ops querying metrics mbeans, so the per mbean size was not an issue. I agree that for things like liveNodes from the NameNodeInfo mbean, returning all the attributes of the mbean is too wasteful for mere access of lighter-weight attributes in the same mbean.
Let's open a separate jira to add the get=attribute-or-pattern mechanism.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.20.205.0, 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-0.20.20X-V2.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-V2.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13036963#comment-13036963 ]
Luke Lu commented on HADOOP-7144:
---------------------------------
bq. is the package OK org.apache.hadoop.jmx? is the URL OK /jmx?
lgtm.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-trunk-alpha.patch
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039758#comment-13039758 ]
Robert Joseph Evans commented on HADOOP-7144:
---------------------------------------------
Hey Luke, what will it take to get HADOOP-7330 into 0.20.20X? I looked at the patch and it looks straight forward? It would be nice to have the JSON output consistent between releases.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Tanping Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048976#comment-13048976 ]
Tanping Wang commented on HADOOP-7144:
--------------------------------------
Yes, we want to be able to query each individual property of a mbean using JMXJsonServerlet. Something like
http://hostname/jmx?qry=mbeanName:property
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.20.205.0, 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-0.20.20X-V2.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-V2.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12994898#comment-12994898 ]
Luke Lu commented on HADOOP-7144:
---------------------------------
jmx is not just metrics, as the name implies, it's a management interface that can handle metrics, events and other meta info (like software version etc.) and we're still not sure whether we should expose the write/update interface (for changing configs dynamically etc.)
I'd propose that we deprecate/remove the old /metrics servlet (which needs to be ported to the new metrics system anyway) once /jmx is in place.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Labels: jmx
> Fix For: 0.23.0
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044274#comment-13044274 ]
Hudson commented on HADOOP-7144:
--------------------------------
Integrated in Hadoop-Common-trunk #709 (See [https://builds.apache.org/hudson/job/Hadoop-Common-trunk/709/])
HADOOP-7144. Expose JMX metrics via JSON servlet.
Contributed by Robert Joseph Evans
cdouglas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1131289
Files :
* /hadoop/common/trunk/src/java/org/apache/hadoop/http/HttpServer.java
* /hadoop/common/trunk/CHANGES.txt
* /hadoop/common/trunk/src/java/org/apache/hadoop/jmx
* /hadoop/common/trunk/src/test/core/org/apache/hadoop/jmx/TestJMXJsonServlet.java
* /hadoop/common/trunk/src/java/org/apache/hadoop/jmx/JMXJsonServlet.java
* /hadoop/common/trunk/src/java/org/apache/hadoop/jmx/package-info.java
* /hadoop/common/trunk/src/test/core/org/apache/hadoop/jmx
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.20.205.0, 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-0.20.20X-V2.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-V2.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12994904#comment-12994904 ]
Luke Lu commented on HADOOP-7144:
---------------------------------
bq. Did .21 ship with /metrics? If not, there is no need to deprecate it. We can just remove it.
Looks like /metrics is in .21/.22, so we'll have to deprecate it in 0.23+.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Labels: jmx
> Fix For: 0.23.0
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041907#comment-13041907 ]
Luke Lu commented on HADOOP-7144:
---------------------------------
LOG.error is always called before handleThrowable. It looked reasonable (it's not swallowing all throwables and log the errors) to me at the time, as we should never let jetty handle the exceptions/errors as stack traces would be display as part of the result, which is a security no no.
OTOH, I think the logging should be handled by handleThrowable calls for better readability and DRYness.
Looking closely though, I found follow correctness issues:
# content type should be set to "application/json; charset=utf8"
# need to response.setStatus(response.SC_INTERNAL_SERVER_ERROR) for all error cases
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-7144:
----------------------------------
Fix Version/s: 0.20.204.0
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.20.204.0, 0.20.205.0, 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-0.20.20X-V2.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-V2.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-7144:
----------------------------------
Resolution: Fixed
Fix Version/s: 0.20.205.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
+1
I committed this to trunk and the 20-security branch. Thanks, Robert!
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.20.205.0, 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-0.20.20X-V2.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-V2.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Tanping Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048978#comment-13048978 ]
Tanping Wang commented on HADOOP-7144:
--------------------------------------
Suppose we can query each property of a mbean, can we return the JSON string exactly like what the mbean property function returns? This way it is consistent between querying through JMXJsonServlet and JMX itself. No additional parsing will be needed between both situations, it will just be a plug-and-play when switching back and forth.
For example,
{code}
NameNode#getLiveNodes()
{code}
returns a Json string like,
{noformat}
"LiveNodes" : "{\"l.yahoo.com\":{\"usedSpace\":49152,\"lastContact\":2,\"adminState\":\"In Service\"},\"4.yahoo.com\":{\"usedSpace\":49152,\"lastContact\":0,\"adminState\":\"Decommissioned\"}}"
{noformat}
It would be helpful if the exact Json string can be returned when querying via JMXJsonServlet. If possible, can we eliminate the bean name and modelerType from the returned string like how JMXJsonServerlet is currently returning, i.e.
{noformat}
{
"beans" : [ {
"name" : "Hadoop:service=NameNode,name=NameNodeInfo",
"modelerType" : "org.apache.hadoop.hdfs.server.namenode.FSNamesystem",
"Threads" : 34,
....
{noformat}
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.20.205.0, 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-0.20.20X-V2.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-V2.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans reassigned HADOOP-7144:
-------------------------------------------
Assignee: Robert Joseph Evans (was: Luke Lu)
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-trunk-alpha.patch
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Tanping Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049459#comment-13049459 ]
Tanping Wang commented on HADOOP-7144:
--------------------------------------
created HADOOP-7392 to implement querying per mbean property.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.20.205.0, 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-0.20.20X-V2.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-V2.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039724#comment-13039724 ]
Robert Joseph Evans commented on HADOOP-7144:
---------------------------------------------
Ahh, Really Jenkins tried to use the JSON as a patch? Oh well the following are the manual results for trunk
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 4 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec]
[exec] +1 system test framework. The patch passed system test framework compile.
[exec]
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13036962#comment-13036962 ]
Luke Lu commented on HADOOP-7144:
---------------------------------
bq. is the package OK org.apache.hadoop.jmx? is the URL OK /jmx?
bq. where would be an appropriate place to document this?
package-info.java for javadoc would be a start.
bq. Is this even the right thing to do? I am not sure how standard the format is. I need to dig into it a bit more to really understand it, because all I have done is a simple port.
We need read-only access and have it return JSON, something like this:
{noformat}
[
{
"bean": "Hadoop:service=...",
"attrs": {
"attr1": "value1",
...
"attrn": "valuen"
}
},
...
]
{noformat}
We also need a way to return a pre-configured set of attributes.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-trunk-alpha.patch
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039372#comment-13039372 ]
Luke Lu commented on HADOOP-7144:
---------------------------------
+1, lgtm, let's see hudson/jenkins' +1. Thanks for the json output (looks like it's 203?) as well, I spotted a bug (HADOOP-7330) the metrics system JMX code that I thought I fixed but didn't go into 203.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042311#comment-13042311 ]
Hadoop QA commented on HADOOP-7144:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12481100/HADOOP-7411-trunk-V2.patch
against trunk revision 1129989.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/555//testReport/
Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/555//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/555//console
This message is automatically generated.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-0.20.20X-V2.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-V2.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12994656#comment-12994656 ]
Todd Lipcon commented on HADOOP-7144:
-------------------------------------
Seems pretty reasonable. Though the proliferation of metrics endpoints is a little strange - a lot of this stuff (but not all) is also available via the hadoop metrics system at /metrics.
Is there some guiding principle we should follow when deciding whether to add things via JMX vs via Metrics? I always tend towards Metrics since you can use a jmx metrics context to expose them to JMX consumers, whereas the reverse is not true.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Labels: jmx
> Fix For: 0.23.0
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039773#comment-13039773 ]
Luke Lu commented on HADOOP-7144:
---------------------------------
bq. what will it take to get HADOOP-7330 into 0.20.20X?
+1 on the jira and we'll nag one of the committers to push the changes to these branches :)
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12994948#comment-12994948 ]
Todd Lipcon commented on HADOOP-7144:
-------------------------------------
Yea, I'm aware JMX is a superset in functionality compared to Metrics, in that it's not read-only.
But, metrics is a superset to JMX in that it already has a bunch of different pluggable output libraries - eg Ganglia. Plus we already have metrics integration for lots of things that haven't been JMX-ified yet.
So I'd prefer to keep exposing data via metrics, and then use a JMX context to also publish to JMX. Then existing users of /metrics, the Ganglia context, etc, won't be broken. Of course adding new read/write fields to JMX makes perfect sense and wouldn't be possible with metrics.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Labels: jmx
> Fix For: 0.23.0
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12994902#comment-12994902 ]
Luke Lu commented on HADOOP-7144:
---------------------------------
bq. we're still not sure whether we should expose the write/update interface (for changing configs dynamically etc.)
... as part of the servlet interface. We already use JMX directly to change metrics config etc.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Labels: jmx
> Fix For: 0.23.0
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-7144:
----------------------------------
Fix Version/s: (was: 0.20.205.0)
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.20.204.0, 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-0.20.20X-V2.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-V2.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041866#comment-13041866 ]
Robert Joseph Evans commented on HADOOP-7144:
---------------------------------------------
It is there because that is what tomcat is doing inside their JMXProxy Servlet which I modified, and didn't take the time to understand all of the possible exceptions that could be thrown in each of the cases to clean it up.
Part of the reason for ignoring the exceptions is because JMX can execute arbitrary code inside the JMXBeans, which could throw all kinds of exceptions. Ideally we would like the code to be robust so that if some beans are causing issues that we can still output data for the rest of the beans. But I will look at what it takes to clean up the exception handling.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048883#comment-13048883 ]
Suresh Srinivas commented on HADOOP-7144:
-----------------------------------------
Is the JMX provided in this jira exposes the document equivalent to the MBean or an MBeam method?
If it is entire MBean, then this can cause issues with Namenode where live node list, dead node list, decom node list etc are returned in one huge json document. Should the access be per mbean property?
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.20.205.0, 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-0.20.20X-V2.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-V2.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044140#comment-13044140 ]
Hudson commented on HADOOP-7144:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #639 (See [https://builds.apache.org/hudson/job/Hadoop-Common-trunk-Commit/639/])
HADOOP-7144. Expose JMX metrics via JSON servlet.
Contributed by Robert Joseph Evans
cdouglas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1131289
Files :
* /hadoop/common/trunk/src/java/org/apache/hadoop/http/HttpServer.java
* /hadoop/common/trunk/CHANGES.txt
* /hadoop/common/trunk/src/java/org/apache/hadoop/jmx
* /hadoop/common/trunk/src/test/core/org/apache/hadoop/jmx/TestJMXJsonServlet.java
* /hadoop/common/trunk/src/java/org/apache/hadoop/jmx/JMXJsonServlet.java
* /hadoop/common/trunk/src/java/org/apache/hadoop/jmx/package-info.java
* /hadoop/common/trunk/src/test/core/org/apache/hadoop/jmx
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.20.205.0, 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-0.20.20X-V2.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-V2.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12994959#comment-12994959 ]
Luke Lu commented on HADOOP-7144:
---------------------------------
All metrics are published via JMX via HADOOP-6728. Metrics doesn't support status, event and a lot meta information that are not time-series, which is a good thing by design. I'm not proposing deprecating metrics system per se (in fact HADOOP-6728 enhanced it in a major way), but the /metrics servlet.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Labels: jmx
> Fix For: 0.23.0
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039385#comment-13039385 ]
Luke Lu commented on HADOOP-7144:
---------------------------------
bq. The JSON is from a task tracker on 20 security trunk, with the name of the box replaced with foo
Thanks for the precise answer. I meant whether the output is from 0.20.20x.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041841#comment-13041841 ]
Owen O'Malley commented on HADOOP-7144:
---------------------------------------
I don't think the exception handling is appropriate. Why do you need to catch them at all?
You certainly don't want to silently swallow Errors or mix their handling with IOExceptions.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated HADOOP-7144:
----------------------------------------
Status: Patch Available (was: Open)
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated HADOOP-7144:
----------------------------------------
Status: Open (was: Patch Available)
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042315#comment-13042315 ]
Luke Lu commented on HADOOP-7144:
---------------------------------
+1. It's good enough for commit, even though IMO, it's better to catch and log Throwable and setStatus 500 for security reasons (it's usually recommended to disallow stack traces in responses in production mode, although it's not a big deal in this particular case). BTW, if you submit the trunk patch last and make patch available, hudson/jenkins will pickup the right patch.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-0.20.20X-V2.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-V2.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated HADOOP-7144:
----------------------------------------
Attachment: HADOOP-7411-trunk-V2.patch
HADOOP-7411-0.20.20X-V2.patch
I believe that I have addressed all of the comments. The code no longer catches a Throwable, but enumerates the errors that can occur at each location, and handles them appropriately. IOExceptions are passed up to doGet which will return an INTERNAL SERVER ERROR. If the query is malformed then a BAD REQUEST is returned. In no case is the json being used to return the error.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-0.20.20X-V2.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-V2.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042352#comment-13042352 ]
Luke Lu commented on HADOOP-7144:
---------------------------------
Another improvement would be to setStatus 404 if the queryNames result is empty.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-0.20.20X-V2.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-V2.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Lu reassigned HADOOP-7144:
-------------------------------
Assignee: Luke Lu
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: jmx
> Fix For: 0.23.0
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13039380#comment-13039380 ]
Robert Joseph Evans commented on HADOOP-7144:
---------------------------------------------
The JSON is from a task tracker on 20 security trunk, with the name of the box replaced with foo
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security
Revision: 1127672
It is targeted for 0.20.205
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Assignee: Robert Joseph Evans
> Labels: jmx
> Fix For: 0.23.0
>
> Attachments: HADOOP-7411-0.20.20X-V1.patch, HADOOP-7411-trunk-V1.patch, HADOOP-7411-trunk-alpha.patch, jmx.json
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7144) Expose JMX with something like
JMXProxyServlet
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12994899#comment-12994899 ]
Allen Wittenauer commented on HADOOP-7144:
------------------------------------------
Did .21 ship with /metrics? If not, there is no need to deprecate it. We can just remove it.
> Expose JMX with something like JMXProxyServlet
> -----------------------------------------------
>
> Key: HADOOP-7144
> URL: https://issues.apache.org/jira/browse/HADOOP-7144
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Luke Lu
> Labels: jmx
> Fix For: 0.23.0
>
>
> Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.
> We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira