[jira] [Updated] (TEZ-4172) Let tasks be killed after too many overall attempts

["László Bodor (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

László Bodor updated TEZ-4172:
------------------------------
    Fix Version/s: 0.9.3
                   0.10.1

> Let tasks be killed after too many overall attempts
> ---------------------------------------------------
>
>                 Key: TEZ-4172
>                 URL: https://issues.apache.org/jira/browse/TEZ-4172
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: László Bodor
>            Assignee: László Bodor
>            Priority: Major
>             Fix For: 0.10.1, 0.9.3
>
>         Attachments: TEZ-4172.01.patch, TEZ-4172.02.patch
>
>
> Currently, TaskImpl doesn't consider failing a task if there are too many overall attempts. In case of LLAP, the number of preempted task attempts -> overall task attempts [can grow in a linkedhashmap|https://github.com/apache/tez/blob/master/tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/TaskImpl.java#L127].
> In an edge case, where an upstream application (Hive LLAP) cannot cope with a problematic query, this can also lead to OOM in the AM, due the very high number of TaskAttemptImpl objects.
> It would be beneficial to have the chance to limit the overall number of task attempts, regardless of they have been failed or killed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)