[jira] Commented: (MAPREDUCE-927) Cleanup of task-logs should happen in TaskTracker instead of the Child

["Todd Lipcon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972898#action_12972898 ] 

Todd Lipcon commented on MAPREDUCE-927:
---------------------------------------

With this patch I've seen occasional NPEs in jvmFinished - it seems initalContext.task is ending up null. Have you guys seen this as well? My current theory is that it's because jvmToRunningTask is accessed without synchronization in JvmRunner.runChild, so if that HashMap.get() overlaps with a rehashing due to a concurrent put, it will return a false null. My reasoning is that when I've seen this happen it's within a couple ms of another task being spawned or removed.

Also, what's the status of this for 22? Should it be considered a blocker?

> Cleanup of task-logs should happen in TaskTracker instead of the Child
> ----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-927
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-927
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: security, tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.22.0
>
>         Attachments: patch-927-1.txt, patch-927-2.txt, patch-927-3.txt, patch-927-4.txt, patch-927-5-dist.txt, patch-927.txt
>
>
> Task logs' cleanup is being done in Child now. This is undesirable atleast for two reasons: 1) failures while cleaning up will affect the user's tasks, and 2) the task's wall time will get affected due to operations that TT actually should own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.