You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Bibin A Chundatt (JIRA)" <ji...@apache.org> on 2015/11/17 06:17:11 UTC
[jira] [Commented] (YARN-4362) Too many preemption activity when
nodelabels are non exclusive
[ https://issues.apache.org/jira/browse/YARN-4362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008072#comment-15008072 ]
Bibin A Chundatt commented on YARN-4362:
----------------------------------------
Attached logs and xml.
Looks like the guaranteed resource for partition 2 for queue default will be always zero. So any container assigned to partition 2 will get preempted from ProportionalCapacityPreemptionPolicy even when no other application is running.
We should restrict assigning to partition 2.
Thoughts?
> Too many preemption activity when nodelabels are non exclusive
> --------------------------------------------------------------
>
> Key: YARN-4362
> URL: https://issues.apache.org/jira/browse/YARN-4362
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Bibin A Chundatt
> Assignee: Varun Saxena
> Priority: Critical
> Attachments: Preemptedpartition.log, ProportionalDefaultQueue.log, ProportionalPolicy.log, capacity-scheduler.xml
>
>
> Steps to reproduce
> ===============
> 1.Configure HA cluster with 6 nodes and 3 partition(1,2,3) all non exclusive
> *Partition configuration is as follows*
> 1,2 NM's mapped with Label 1
> NM 3 to label 2
> 4,5 NM's mapped to Label 3
> NM 6 in DEFAULT partition
> In capacity scheduler the queue are linked only to 1,3 partition.
> The NM 3 with label 2 is a backup node for any partition whenever required will change the label.
> Submit and application/job with 200 containers to default queue.
> All containers that gets assigned to partition 2 gets preempted
> The application/map task execution is taking more time since 30-40 task gets assigned to partition 2 then gets preempted and all of them needs to be relaunched.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)