You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ramya Sunil (Created) (JIRA)" <ji...@apache.org> on 2011/10/28 21:15:32 UTC
[jira] [Created] (MAPREDUCE-3307) Improve logging on the console
during job execution
Improve logging on the console during job execution
---------------------------------------------------
Key: MAPREDUCE-3307
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3307
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Fix For: 0.23.0
There is a lot of redundant information being printed on the console and a not so intuitive flow of events. We should improve the logging on console during job execution. More details in the next comment.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3307) Improve logging on the console
during job execution
Posted by "Arun C Murthy (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy resolved MAPREDUCE-3307.
--------------------------------------
Resolution: Duplicate
Duplicate of MAPREDUCE-3265.
> Improve logging on the console during job execution
> ---------------------------------------------------
>
> Key: MAPREDUCE-3307
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3307
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
>
> There is a lot of redundant information being printed on the console and a not so intuitive flow of events. We should improve the logging on console during job execution. More details in the next comment.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3307) Improve logging on the console
during job execution
Posted by "Ramya Sunil (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138671#comment-13138671 ]
Ramya Sunil commented on MAPREDUCE-3307:
----------------------------------------
The redundant information is highlighted below:
{color:red}
11/10/28 02:41:52 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
{color}
11/10/28 02:41:52 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at <hostname>/<ip:port>
{color:red}
11/10/28 02:41:52 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
{color}
11/10/28 02:41:52 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at <hostname>/<ip:port>
Running 100 maps.
Job started: Fri Oct 28 02:41:52 UTC 2011
{color:red}
11/10/28 02:41:52 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
11/10/28 02:41:52 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager <hostname>/<ip:port>
11/10/28 02:41:52 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
11/10/28 02:41:52 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at <hostname>/<ip:port>
{color}
11/10/28 02:41:53 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token <id> for <user> on <ip:port>
11/10/28 02:41:53 INFO security.TokenCache: Got dt for hdfs://<host:port>;uri=<ip:port>;t.service=<ip:port>
{color:red}
11/10/28 02:41:53 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token <id> for <user> on <ip:port>
11/10/28 02:41:53 INFO security.TokenCache: Got dt for hdfs://<host:port>;uri=<ip:port>;t.service=<ip:port>
{color}
11/10/28 02:41:53 INFO mapreduce.JobSubmitter: number of splits:100
11/10/28 02:41:53 INFO mapred.YARNRunner: AppMaster capability = memory: 2048,
11/10/28 02:41:53 INFO mapred.YARNRunner: Command to launch container for ApplicationMaster is : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.mapreduce.container.log.dir=<LOG_DIR> -Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1536m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1><LOG_DIR>/stdout 2><LOG_DIR>/stderr
11/10/28 02:41:53 INFO mapred.ResourceMgrDelegate: Submitted application <appID> to ResourceManager
11/10/28 02:41:53 INFO mapred.ClientCache: Connecting to HistoryServer at: <host:port>
{color:red}
11/10/28 02:41:53 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
{color}
11/10/28 02:41:53 INFO mapred.ClientCache: Connected to HistoryServer at: <host:port>
{color:red}
11/10/28 02:41:53 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
{color}
11/10/28 02:41:53 INFO mapreduce.Job: Running job: job_ID
11/10/28 02:41:54 INFO mapreduce.Job: map 0% reduce 0%
11/10/28 02:42:05 INFO mapred.ClientServiceDelegate: Tracking Url of JOB is <url>
11/10/28 02:42:05 INFO mapred.ClientServiceDelegate: Connecting to <host:port>
{color:red}
11/10/28 02:42:05 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
11/10/28 02:42:05 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
{color}
11/10/28 02:42:05 INFO security.ApplicationTokenSelector: Looking for a token with service <ip:port>
11/10/28 02:42:05 INFO security.ApplicationTokenSelector: Token kind is YARN_APPLICATION_TOKEN and the token's service name is <ip:port>
11/10/28 02:43:44 INFO mapreduce.Job: map 2% reduce 0%
11/10/28 02:43:51 INFO mapreduce.Job: map 3% reduce 0%
11/10/28 02:43:52 INFO mapreduce.Job: map 5% reduce 0%
11/10/28 02:43:56 INFO mapreduce.Job: map 10% reduce 0%
Also,
Printing, "Running job: job_ID" and then trying to connect to AM is misleading. This should be fixed too.
> Improve logging on the console during job execution
> ---------------------------------------------------
>
> Key: MAPREDUCE-3307
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3307
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Fix For: 0.23.0
>
>
> There is a lot of redundant information being printed on the console and a not so intuitive flow of events. We should improve the logging on console during job execution. More details in the next comment.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3307) Improve logging on the console
during job execution
Posted by "Arun C Murthy (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy reassigned MAPREDUCE-3307:
----------------------------------------
Assignee: Arun C Murthy
> Improve logging on the console during job execution
> ---------------------------------------------------
>
> Key: MAPREDUCE-3307
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3307
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
>
> There is a lot of redundant information being printed on the console and a not so intuitive flow of events. We should improve the logging on console during job execution. More details in the next comment.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira