[GitHub] johnyangk opened a new pull request #59: [NEMO-50] Carefully retry tasks in the scheduler

[GitBox <gi...@apache.org>]

johnyangk opened a new pull request #59: [NEMO-50] Carefully retry tasks in the scheduler
URL: https://github.com/apache/incubator-nemo/pull/59
 
 
   JIRA: [NEMO-50: Carefully retry tasks in the scheduler](https://issues.apache.org/jira/projects/NEMO/issues/NEMO-##)
   
   **Major changes:**
   - doSchedule(): The main scheduling entry point. Can also be used for retrying tasks 
   - recursivelyGetParentTasksForLostBlocks(): To retry tasks that need to be retried (not more, not less)
   - Clearly differentiates restarts due to task failures from restarts due to executor removal
   - Reorganizes states for proper FT
     - TaskState: READY, EXECUTING, ON_HOLD, COMPLETE, SHOULD_RETRY, FAILED
     - StageState: SCHEDULABLE, COMPLETE 
   
   **Minor changes to note:**
   - Nice logs for tracking job progress
   - Refactoring and added comments to make it easier to understand when and why schedulerRunner.onExecutorSlotAvailable() and doSchedule() are called
   
   **Tests for the changes:**
   - TaskRestartTest: Interleaves failure events with task completion events 
   
   **Other comments:**
   - INPUT_READ_FAILURE will be handled in https://issues.apache.org/jira/browse/NEMO-54
   
   resolves [NEMO-50](https://issues.apache.org/jira/projects/NEMO/issues/NEMO-50)
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services