[jira] Updated: (HADOOP-3150) Move task file promotion into the task

["Devaraj Das (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Devaraj Das updated HADOOP-3150:
--------------------------------

    Status: Open  (was: Patch Available)

Some comments on the last patch:
1. Remove the check for tip.isComplete (it is not necessary) for cleanup task in JIP.failedTask
2. Make sure launchedCleanup is set to false if the cleanup task is failed or killed from UI or commandLine or because of lost TT
3. Make sure the cleanupTip.progress is same any other tip progress.
4. Change version number for the JobSubmissionProtocol and InterTrackerProtocol because of JobStatus and TaskStatus changes
5. Change the name of setCleanupJobTask in Task.java to setCleanupTask, because it confusing with TIP.setCleanupTask
6. You should have a error code for the case where you fail to report commitPending to TT or ask for commitapproval from TT.  for e.g., error 
code 67and 68,  just like if ping fails, then System.exit(65) is called.
7. In JobQueueTaskScheduler and LimitTasksPerJobScheduler, move the iteration over jobs for cleanup tasks to the top of assignTasks, i.e. before 
calculating the loads.
8. In JobStatus, call the old constructor from new constructor by passing 0.0f for cleanupProgress.
9. The kill for commit pending tasks that should be killed should be handled in getTasksToKill
10. JIP.updateTaskStatus should update the progress of the job only when it gets the statuses of regular tasks
11. In TaskTrackerStatus, countMapTasks/countReduceTasks should count COMMIT_PENDING tasks too

> Move task file promotion into the task
> --------------------------------------
>
>                 Key: HADOOP-3150
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3150
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.19.0
>
>         Attachments: 3150.patch, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt
>
>
> We need to move the task file promotion from the JobTracker to the Task and move it down into the output format.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.