You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "László Bodor (Jira)" <ji...@apache.org> on 2021/10/01 10:02:00 UTC
[jira] [Commented] (TEZ-4338) Tez should consider node information
for computing failure fraction - upstream(mapper) problems
[ https://issues.apache.org/jira/browse/TEZ-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423215#comment-17423215 ]
László Bodor commented on TEZ-4338:
-----------------------------------
cc: [~rbalamohan], I'm doing this separately from TEZ-4139, as this is more urgent and a smaller change
> Tez should consider node information for computing failure fraction - upstream(mapper) problems
> -----------------------------------------------------------------------------------------------
>
> Key: TEZ-4338
> URL: https://issues.apache.org/jira/browse/TEZ-4338
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
>
> Finally, I decided to split TEZ-4139 into 2 different tasks, because handling upstream problems doesn't need lots of changes to the tez event itself (downstream host).
> So, from TEZ-4139, this ticket is intended to handle downstream failures as:
> collect all reported upstream mapper task attempts for a vertex, and if it's beyond a certain amount for the same source(map) host, blame mapper task immediately => blame mapper task attempt as soon as possible if read error is likely because of upstream node failure (somewhat similar goal to TEZ-3910, but currently TEZ-3910 is about to give the power of failure downstream task completely to AM)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)