[jira] [Commented] (MAPREDUCE-7169) Speculative attempts should not run on the same node

["Ahmed Hussein (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-7169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17445997#comment-17445997 ] 

Ahmed Hussein commented on MAPREDUCE-7169:
------------------------------------------

bq. Why is denying racks and hosts should be enabled separately? Can you please elaborate? Currently we try to avoid launching on same rack as old attempt if there are no containers on diff rack then we try choosing node other than old attempt node.

Hi [~BilwaST], I second  [~jeagles] about separating the rack and the hosts.
Considering the block placement policy on Hadoop, launching the speculative attempt on a different rack every-time will be a double edge sword. on same rack will be a double edge sword.
leaving it to be configurable will give much more flexibility to work along with block placements. Otherwise, the speculative changes will be "All or nothing: Always run on a different rack or disable it and speculate on the same node ".

> Speculative attempts should not run on the same node
> ----------------------------------------------------
>
>                 Key: MAPREDUCE-7169
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7169
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: yarn
>    Affects Versions: 2.7.2
>            Reporter: Lee chen
>            Assignee: Bilwa S T
>            Priority: Major
>         Attachments: MAPREDUCE-7169-001.patch, MAPREDUCE-7169-002.patch, MAPREDUCE-7169-003.patch, MAPREDUCE-7169.004.patch, MAPREDUCE-7169.005.patch, MAPREDUCE-7169.006.patch, MAPREDUCE-7169.007.patch, image-2018-12-03-09-54-07-859.png
>
>
>           I found in all versions of yarn, Speculative Execution may set the speculative task to the node of  original task.What i have read is only it will try to have one more task attempt. haven't seen any place mentioning not on same node.It is unreasonable.If the node have some problems lead to tasks execution will be very slow. and then placement the speculative  task to same node cannot help the  problematic task.
>          In our cluster （version 2.7.2，2700 nodes），this phenomenon appear almost everyday.
>  !image-2018-12-03-09-54-07-859.png! 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org