[jira] [Commented] (MAPREDUCE-5989) Add DeletionService in AM

["Jason Lowe (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070246#comment-14070246 ] 

Jason Lowe commented on MAPREDUCE-5989:
---------------------------------------

Is this the same kind of DeletionService that the NM currently uses?  If so I'm unclear on the tangible benefits of this, since all that service does is potentially postpone deletions.  And as for the staging directory cleanup, implementing a deletion service is not needed to fix that issue.  Actually I believe it's already fixed by MAPREDUCE-5476 by having it delete the staging directory after unregistering so we know no other AM attempts will try to be launched after removing the staging directory.

If you could walk through an example scenario where the deletion service is used and how it's useful that would help me understand why adding such a service would be helpful.

> Add DeletionService in AM
> -------------------------
>
>                 Key: MAPREDUCE-5989
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5989
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: applicationmaster
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>
> In AM, for graceful cleanup, I propose addition of a DeletionService which will do the following :
> 1. Cleanup of failed tasks (temporary data need not occupy space till NM's Deletion Service is invoked)
> 2. Staging directory deletion (During AM shutdown, its better to place staging dir cleanup in Deletion Service: Refer to MAPREDUCE-4841 )



--
This message was sent by Atlassian JIRA
(v6.2#6252)