[jira] [Commented] (TEZ-781) Add unit test for fault tolerance (input failure causes re-run of previous task which also has attempt failures)

["Bikas Saha (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13890094#comment-13890094 ] 

Bikas Saha commented on TEZ-781:
--------------------------------

bq. v2-task0-attempt-0-input-1-inputattempt-0 fails. This will trigger rerun of v1-task1.
This is not correct. v2-task0-attempt-0-input-1-inputattempt-1 failure  will trigger rerun of v1-task1. The version number corresponds to the version of the producer task. The rerun of v1-task1 will be version 2 (0 failed by itself, 1 was failed by v2-task0). That will enable v2-task0 to pass.

> Add unit test for fault tolerance (input failure causes re-run of previous task which also has attempt failures)
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: TEZ-781
>                 URL: https://issues.apache.org/jira/browse/TEZ-781
>             Project: Apache Tez
>          Issue Type: Improvement
>    Affects Versions: 0.3.0
>            Reporter: Tassapol Athiapinya
>            Assignee: Tassapol Athiapinya
>             Fix For: 0.3.0
>
>         Attachments: TEZ-781.2.patch, TEZ-781.patch
>
>
> Add a unit test for tez fault tolerance.
> DAG:
>  v1-task0    v1-task1
>  |             \ /     |
> v2-task0    v2-task1
> Set maximum failed attempt (tez.am.max.task.attempts) to 2.
> v1-task1-attempt-0 fails. Attempt 1 succeeds.
> v2-task0-attempt-0-input-1-inputattempt-0 fails. This will trigger rerun of v1-task1.
> v1-task1-attempt-0 fails. Because tez.am.max.task.attempts does reset when doing rerun, this is not above maximum failed attempt. Attempt 1 should kick in and succeed. 
> In the end, v2-task0 should succeed after v1-task1 rerun.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)