[jira] [Commented] (REEF-1447) Validate Task close failure => FailedEvaluator Event (task has not yet finished)

["Andrew Chung (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/REEF-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15342379#comment-15342379 ] 

Andrew Chung commented on REEF-1447:
------------------------------------

Upon offline discussion, we have reached a conclusion that an {{Exception}} in {{TaskCloseHandler}} should trigger a {{FailedEvaluator}} instead of a {{FailedTask}} event, the reason being the following:
Currently when the {{TaskCloseHandler}} throws an {{Exception}}, it sets the {{Exception}} field in the {{Task}} and triggers a {{FailedTask}} heartbeat to the Driver. However, given that thread abort is voluntary and the Task itself is run on a separate thread, the Task will finish and set the result of the Task. When the Task sets the result and attempts to trigger a heartbeat back to the Driver signaling the completion of the Task, it will notice that the Task {{Exception}} field is already filled and throw an uncaught {{RuntimeException}}, triggering an Evaluator failure.

Another scenario is after the {{TaskCloseHandler}} fails and triggers a {{FailedTask}} event, the Driver can submit a new Task before the old Task thread is finished. This will also trigger an Evaluator failure because the Evaluator currently does not support running multiple Tasks.

Given that both edge cases lead to an Evaluator failure, we should trigger the Evaluator failure early rather than wait for uncertain behavior. This will need to change on the Java side as well.

> Validate Task close failure => FailedEvaluator Event (task has not yet finished)
> --------------------------------------------------------------------------------
>
>                 Key: REEF-1447
>                 URL: https://issues.apache.org/jira/browse/REEF-1447
>             Project: REEF
>          Issue Type: Sub-task
>          Components: REEF.NET, REEF.NET Driver
>            Reporter: Andrew Chung
>            Assignee: Andrew Chung
>              Labels: FT
>
> We need to validate that an {{Exception}} in {{TaskCloseHandler}} triggers a {{FailedEvaluator}} event on the Driver, if the Task has not yet completed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)