You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Jason Lowe (Commented) (JIRA)" <ji...@apache.org> on 2012/01/27 22:52:09 UTC
[jira] [Commented] (MAPREDUCE-3744) Unable to retrieve application
logs via "yarn logs" or "mapred job -logs"
[ https://issues.apache.org/jira/browse/MAPREDUCE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13195124#comment-13195124 ]
Jason Lowe commented on MAPREDUCE-3744:
---------------------------------------
Correction, the mapred failure about the historyserver is caused because mapreduce.jobhistory.adddress was accidentally left out of the cluster configuration. After adding it back in, it now fails like this:
{code}
$ mapred job -logs job_1327700131747_0002
2012-01-27 15:44:11,643 WARN conf.Configuration (Configuration.java:set(639)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2012-01-27 15:44:12,770 INFO mapred.ClientServiceDelegate (ClientServiceDelegate.java:getProxy(246)) - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
java.io.FileNotFoundException: File does not exist: /tmp/logs/someuser/logs/application_1327700131747_0002/host.name.removed.com_45454
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:798)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:761)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:733)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:389)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:219)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:42586)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:439)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:862)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1608)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1603)
{code}
So both fail trying to get logs out of /tmp. Interestingly the paths both are trying to access are significantly different, and the only actual directory directly underneath /tmp/ in HDFS is /tmp/hadoop-yarn/.
> Unable to retrieve application logs via "yarn logs" or "mapred job -logs"
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-3744
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3744
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Jason Lowe
> Priority: Blocker
>
> Trying to retrieve application logs via the "yarn logs" shell command results in an error similar to this:
> Exception in thread "main" java.io.FileNotFoundException: File /tmp/logs/application_1327694122989_0001 does not exist.
> at org.apache.hadoop.fs.Hdfs$DirListingIterator.<init>(Hdfs.java:226)
> at org.apache.hadoop.fs.Hdfs$DirListingIterator.<init>(Hdfs.java:217)
> at org.apache.hadoop.fs.Hdfs$2.<init>(Hdfs.java:192)
> at org.apache.hadoop.fs.Hdfs.listStatusIterator(Hdfs.java:192)
> at org.apache.hadoop.fs.FileContext$20.next(FileContext.java:1371)
> at org.apache.hadoop.fs.FileContext$20.next(FileContext.java:1)
> at org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:2319)
> at org.apache.hadoop.fs.FileContext.listStatus(FileContext.java:1373)
> at org.apache.hadoop.yarn.logaggregation.LogDumper.dumpAllContainersLogs(LogDumper.java:191)
> at org.apache.hadoop.yarn.logaggregation.LogDumper.run(LogDumper.java:107)
> at org.apache.hadoop.yarn.logaggregation.LogDumper.main(LogDumper.java:226)
> Trying to grab the logs via the "mapred jobs -logs" command results in this error:
> 2012-01-27 14:05:52,040 INFO mapred.ClientServiceDelegate (ClientServiceDelegate.java:getProxy(246)) - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
> 2012-01-27 14:05:52,041 WARN mapred.ClientServiceDelegate (ClientServiceDelegate.java:checkAndGetHSProxy(257)) - Job History Server is not configured.
> Unable to get log information for job: job_1327694122989_0001
> Even though the historyserver process is running.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira