You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Jason Lowe (JIRA)" <ji...@apache.org> on 2012/06/22 00:49:43 UTC
[jira] [Created] (MAPREDUCE-4361) Fix detailed metrics for
protobuf-based RPC on 0.23
Jason Lowe created MAPREDUCE-4361:
-------------------------------------
Summary: Fix detailed metrics for protobuf-based RPC on 0.23
Key: MAPREDUCE-4361
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4361
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Jason Lowe
RPC detailed metrics for any protobuf-based RPC ports are always zero. ProtoOverHadoopRpcEngine needs the same detailed metric logic as in WritableRpcEngine. This is effectively the same change as in HADOOP-8085 except tailored for branch-0.23 which didn't take the full protobuf branch changes that went into branch-2 and trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (MAPREDUCE-4361) Fix detailed metrics for
protobuf-based RPC on 0.23
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy closed MAPREDUCE-4361.
------------------------------------
> Fix detailed metrics for protobuf-based RPC on 0.23
> ---------------------------------------------------
>
> Key: MAPREDUCE-4361
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4361
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 0.23.3, 2.0.2-alpha
>
> Attachments: MAPREDUCE-4361.patch
>
>
> RPC detailed metrics for any protobuf-based RPC ports are always zero. ProtoOverHadoopRpcEngine needs the same detailed metric logic as in WritableRpcEngine. This is effectively the same change as in HADOOP-8085 except tailored for branch-0.23 which didn't take the full protobuf branch changes that went into branch-2 and trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4361) Fix detailed metrics for
protobuf-based RPC on 0.23
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401356#comment-13401356 ]
Hudson commented on MAPREDUCE-4361:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #298 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/298/])
MAPREDUCE-4361. Fix detailed metrics for protobuf-based RPC on 0.23 (Jason Lowe via tgraves) (Revision 1353580)
Result = SUCCESS
tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1353580
Files :
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/ProtoOverHadoopRpcEngine.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java
> Fix detailed metrics for protobuf-based RPC on 0.23
> ---------------------------------------------------
>
> Key: MAPREDUCE-4361
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4361
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-4361.patch
>
>
> RPC detailed metrics for any protobuf-based RPC ports are always zero. ProtoOverHadoopRpcEngine needs the same detailed metric logic as in WritableRpcEngine. This is effectively the same change as in HADOOP-8085 except tailored for branch-0.23 which didn't take the full protobuf branch changes that went into branch-2 and trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4361) Fix detailed metrics for
protobuf-based RPC on 0.23
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-4361:
----------------------------------
Attachment: MAPREDUCE-4361.patch
Patch that ports the relevant portions of HADOOP-8085 to branch-0.23. I manually tested this by bringing up a single-node cluster and visiting the ResourceManager JMX page. The detailed metrics for YARN interfaces like {{NodeHeartbeatNumOps}} were non-zero, whereas they were zero before applying this patch.
I also manually ran the unit tests in yarn-common, and they all passed.
> Fix detailed metrics for protobuf-based RPC on 0.23
> ---------------------------------------------------
>
> Key: MAPREDUCE-4361
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4361
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4361.patch
>
>
> RPC detailed metrics for any protobuf-based RPC ports are always zero. ProtoOverHadoopRpcEngine needs the same detailed metric logic as in WritableRpcEngine. This is effectively the same change as in HADOOP-8085 except tailored for branch-0.23 which didn't take the full protobuf branch changes that went into branch-2 and trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-4361) Fix detailed metrics for
protobuf-based RPC on 0.23
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe reassigned MAPREDUCE-4361:
-------------------------------------
Assignee: Jason Lowe
> Fix detailed metrics for protobuf-based RPC on 0.23
> ---------------------------------------------------
>
> Key: MAPREDUCE-4361
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4361
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Jason Lowe
> Assignee: Jason Lowe
>
> RPC detailed metrics for any protobuf-based RPC ports are always zero. ProtoOverHadoopRpcEngine needs the same detailed metric logic as in WritableRpcEngine. This is effectively the same change as in HADOOP-8085 except tailored for branch-0.23 which didn't take the full protobuf branch changes that went into branch-2 and trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4361) Fix detailed metrics for
protobuf-based RPC on 0.23
Posted by "Thomas Graves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Graves updated MAPREDUCE-4361:
-------------------------------------
Resolution: Fixed
Fix Version/s: 0.23.3
Status: Resolved (was: Patch Available)
I committed this to branch 0.23, the change is not applicable to trunk or branch-2 since it already works there.
> Fix detailed metrics for protobuf-based RPC on 0.23
> ---------------------------------------------------
>
> Key: MAPREDUCE-4361
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4361
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-4361.patch
>
>
> RPC detailed metrics for any protobuf-based RPC ports are always zero. ProtoOverHadoopRpcEngine needs the same detailed metric logic as in WritableRpcEngine. This is effectively the same change as in HADOOP-8085 except tailored for branch-0.23 which didn't take the full protobuf branch changes that went into branch-2 and trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4361) Fix detailed metrics for
protobuf-based RPC on 0.23
Posted by "Thomas Graves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400507#comment-13400507 ]
Thomas Graves commented on MAPREDUCE-4361:
------------------------------------------
+1 lgtm. Thanks Jason!
> Fix detailed metrics for protobuf-based RPC on 0.23
> ---------------------------------------------------
>
> Key: MAPREDUCE-4361
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4361
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4361.patch
>
>
> RPC detailed metrics for any protobuf-based RPC ports are always zero. ProtoOverHadoopRpcEngine needs the same detailed metric logic as in WritableRpcEngine. This is effectively the same change as in HADOOP-8085 except tailored for branch-0.23 which didn't take the full protobuf branch changes that went into branch-2 and trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4361) Fix detailed metrics for
protobuf-based RPC on 0.23
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399592#comment-13399592 ]
Jason Lowe commented on MAPREDUCE-4361:
---------------------------------------
test-patch doesn't know how to test patches that aren't based on trunk. Manually ran test-patch on branch-0.23 and got these results:
{panel}
-1 overall.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 18 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version ) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
{panel}
The javadoc warnings are unrelated to the patch (mostly involving issues in Rumen and Hadoop Extras).
> Fix detailed metrics for protobuf-based RPC on 0.23
> ---------------------------------------------------
>
> Key: MAPREDUCE-4361
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4361
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4361.patch
>
>
> RPC detailed metrics for any protobuf-based RPC ports are always zero. ProtoOverHadoopRpcEngine needs the same detailed metric logic as in WritableRpcEngine. This is effectively the same change as in HADOOP-8085 except tailored for branch-0.23 which didn't take the full protobuf branch changes that went into branch-2 and trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4361) Fix detailed metrics for
protobuf-based RPC on 0.23
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399575#comment-13399575 ]
Hadoop QA commented on MAPREDUCE-4361:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533103/MAPREDUCE-4361.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2500//console
This message is automatically generated.
> Fix detailed metrics for protobuf-based RPC on 0.23
> ---------------------------------------------------
>
> Key: MAPREDUCE-4361
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4361
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4361.patch
>
>
> RPC detailed metrics for any protobuf-based RPC ports are always zero. ProtoOverHadoopRpcEngine needs the same detailed metric logic as in WritableRpcEngine. This is effectively the same change as in HADOOP-8085 except tailored for branch-0.23 which didn't take the full protobuf branch changes that went into branch-2 and trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4361) Fix detailed metrics for
protobuf-based RPC on 0.23
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-4361:
----------------------------------
Target Version/s: 0.23.3
Status: Patch Available (was: Open)
> Fix detailed metrics for protobuf-based RPC on 0.23
> ---------------------------------------------------
>
> Key: MAPREDUCE-4361
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4361
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4361.patch
>
>
> RPC detailed metrics for any protobuf-based RPC ports are always zero. ProtoOverHadoopRpcEngine needs the same detailed metric logic as in WritableRpcEngine. This is effectively the same change as in HADOOP-8085 except tailored for branch-0.23 which didn't take the full protobuf branch changes that went into branch-2 and trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira