You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Todd Lipcon (Created) (JIRA)" <ji...@apache.org> on 2011/10/25 21:14:32 UTC
[jira] [Created] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Reduce log level on MR2 IPC construction, etc
---------------------------------------------
Key: MAPREDUCE-3265
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mrv2
Affects Versions: 0.23.0
Reporter: Todd Lipcon
Priority: Minor
Currently MR's IPC logging is very verbose. For example, I see a lot of:
11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Arun C Murthy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155656#comment-13155656 ]
Arun C Murthy commented on MAPREDUCE-3265:
------------------------------------------
With the patch, the output looks so:
{noformat}
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
11/11/23 03:05:12 WARN conf.Configuration: mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
Running on 5 nodes to sort from hdfs://hrt12n01.cc1.ygridcore.net:8020/user/hrt_1/in into hdfs://nn.foobar.com:8020/user/hrt_1/out with 9 reduces.
Job started: Wed Nov 23 03:05:12 UTC 2011
11/11/23 03:05:12 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 379 for hrt_1 on xx.xx.xx.xx:8020
11/11/23 03:05:12 INFO security.TokenCache: Got dt for hdfs://nn.foobar.com:8020:8020;uri=xx.xx.xx.xx:8020;t.service=xx.xx.xx.xx:8020
11/11/23 03:05:12 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 380 for hrt_1 on xx.xx.xx.xx:8020
11/11/23 03:05:12 INFO security.TokenCache: Got dt for hdfs://nn.foobar.com:8020:8020;uri=xx.xx.xx.xx:8020;t.service=xx.xx.xx.xx:8020
11/11/23 03:05:12 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
11/11/23 03:05:15 INFO input.FileInputFormat: Total input paths to process : 5
11/11/23 03:05:17 INFO mapreduce.JobSubmitter: number of splits:5
11/11/23 03:05:18 INFO mapred.ResourceMgrDelegate: Submitted application application_1322016908667_0002 to ResourceManager at rm.foobar.com/yy.yy.yy.yy:8040
11/11/23 03:05:18 INFO mapreduce.Job: Running job: job_1322016908667_0002
11/11/23 03:05:19 INFO mapreduce.Job: map 0% reduce 0%
11/11/23 03:05:29 INFO mapred.ClientServiceDelegate: Job tracking-url: rm.foobar.com:8088/proxy/application_1322016908667_0002/
11/11/23 03:05:40 INFO mapreduce.Job: map 100% reduce 0%
{noformat}
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155667#comment-13155667 ]
Hadoop QA commented on MAPREDUCE-3265:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12504844/MAPREDUCE-3265.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 12 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1339//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1339//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-examples.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1339//console
This message is automatically generated.
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Mahadev konar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155654#comment-13155654 ]
Mahadev konar commented on MAPREDUCE-3265:
------------------------------------------
+1 looks good to me.
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156689#comment-13156689 ]
Hudson commented on MAPREDUCE-3265:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #103 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/103/])
Merge -c 1205628 from trunk to branch-0.23 to fix MAPREDUCE-3265.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1205629
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLog.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnProtoRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/YarnRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ApplicationTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch, MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156667#comment-13156667 ]
Hudson commented on MAPREDUCE-3265:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #86 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/86/])
Merge -c 1205628 from trunk to branch-0.23 to fix MAPREDUCE-3265.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1205629
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLog.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnProtoRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/YarnRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ApplicationTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch, MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Arun C Murthy (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy reassigned MAPREDUCE-3265:
----------------------------------------
Assignee: Arun C Murthy
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156702#comment-13156702 ]
Hudson commented on MAPREDUCE-3265:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #907 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/907/])
MAPREDUCE-3265. Removed debug logs during job submission to LOG.debug to cut down noise.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1205628
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLog.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnProtoRPC.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnRPC.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/YarnRPC.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ApplicationTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch, MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3265:
-------------------------------------
Attachment: MAPREDUCE-3265.patch
Just changed a log stmt:
{noformat}
- LOG.info("Job tracking-url: " + application.getTrackingUrl());
+ LOG.info("The url to track the job: " + application.getTrackingUrl());
{noformat}
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch, MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156407#comment-13156407 ]
Hudson commented on MAPREDUCE-3265:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1392 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1392/])
MAPREDUCE-3265. Removed debug logs during job submission to LOG.debug to cut down noise.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1205628
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLog.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnProtoRPC.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnRPC.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/YarnRPC.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ApplicationTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch, MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3265:
-------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this.
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch, MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156675#comment-13156675 ]
Hudson commented on MAPREDUCE-3265:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #873 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/873/])
MAPREDUCE-3265. Removed debug logs during job submission to LOG.debug to cut down noise.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1205628
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLog.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnProtoRPC.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnRPC.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/YarnRPC.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ApplicationTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch, MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156362#comment-13156362 ]
Hudson commented on MAPREDUCE-3265:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1342 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1342/])
MAPREDUCE-3265. Removed debug logs during job submission to LOG.debug to cut down noise.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1205628
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLog.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnProtoRPC.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnRPC.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/YarnRPC.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ApplicationTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch, MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3265:
-------------------------------------
Attachment: MAPREDUCE-3265.patch
Changed a bunch of LOG.info statements. No tests since it's only logging changes.
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Attachments: MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3265:
-------------------------------------
Fix Version/s: 0.23.1
Status: Patch Available (was: Open)
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156404#comment-13156404 ]
Hudson commented on MAPREDUCE-3265:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #211 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/211/])
Merge -c 1205628 from trunk to branch-0.23 to fix MAPREDUCE-3265.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1205629
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLog.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnProtoRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/YarnRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ApplicationTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch, MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156390#comment-13156390 ]
Hudson commented on MAPREDUCE-3265:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #208 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/208/])
Merge -c 1205628 from trunk to branch-0.23 to fix MAPREDUCE-3265.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1205629
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLog.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnProtoRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/YarnRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ApplicationTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch, MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156397#comment-13156397 ]
Hudson commented on MAPREDUCE-3265:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1319 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1319/])
MAPREDUCE-3265. Removed debug logs during job submission to LOG.debug to cut down noise.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1205628
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLog.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnProtoRPC.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnRPC.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/YarnRPC.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ApplicationTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientTokenSelector.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch, MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3265) Reduce log level on MR2 IPC
construction, etc
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156346#comment-13156346 ]
Hudson commented on MAPREDUCE-3265:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #222 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/222/])
Merge -c 1205628 from trunk to branch-0.23 to fix MAPREDUCE-3265.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1205629
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLog.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientServiceDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ResourceMgrDelegate.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnProtoRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/HadoopYarnRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/ipc/YarnRPC.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ApplicationTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/client/ClientTokenSelector.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch, MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at: c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira