You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Kihwal Lee (JIRA)" <ji...@apache.org> on 2013/07/22 23:52:51 UTC

[jira] [Commented] (YARN-949) Failed log aggregation can leave a file open.

    [ https://issues.apache.org/jira/browse/YARN-949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715737#comment-13715737 ] 

Kihwal Lee commented on YARN-949:
---------------------------------

{noformat}
2013-07-20 14:55:23,901 [LogAggregationService #64] WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl:
Aggregation did not complete for application application_xxx_xxx
2013-07-20 14:55:23,902 [LogAggregationService #64] ERROR org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread 
Thread[LogAggregationService #64,5,main] threw an Exception.
java.lang.NullPointerException
        at org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogValue.write(AggregatedLogFormat.java:187)
        at org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogWriter.append(AggregatedLogFormat.java:264)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl.
uploadLogsForContainer(AppLogAggregatorImpl.java:128)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl.
doAppLogAggregation(AppLogAggregatorImpl.java:164)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl.run(AppLogAggregatorImpl.java:138)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService$2.run(LogAggregationService.java:361)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)
{noformat}


It looks like the loop calling uploadLogsForContainer() should be inside a try block and closeWriter() should be moved to the finally block.
                
> Failed log aggregation can leave a file open.
> ---------------------------------------------
>
>                 Key: YARN-949
>                 URL: https://issues.apache.org/jira/browse/YARN-949
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.1.0-beta, 0.23.9
>            Reporter: Kihwal Lee
>
> If log aggregation fails on a node manager, the output file in hdfs can be left open.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira