[jira] [Commented] (YARN-4945) [Umbrella] Capacity Scheduler Preemption Within a queue

["Sunil G (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15454965#comment-15454965 ] 

Sunil G commented on YARN-4945:
-------------------------------

Thanks [~leftnoteasy]
bq.What is intraQueuePreemptionCost? And what is AppPriorityComparator?
Yes, there are not used. I will do a clean up in next patch to remove unnecessary code snippets.

bq.For the queue-level ideal allocation is it enough to trust result from previous policies?
If we are planning for normalization (I want to introduce in next patch after doing some more tests with SLS and real time), then we need to add up or subtract resources which are preempted from previous run. And this is needed to done per queue. Hence I kept the code. I can look into this and see how much more i can reuse as possible.

bq.if (tq.intraQueuePreemptionCalculationDone == true), is this always false?
intraQueuePreemptionCalculationDone will be set as false during init. We iterate over partitions, and there are chances that we may have multiple labels in a queue. Since I am iterating over partitions and then look into queues where this partition has some capacity, we may have same queue across partition. I was calculating resource demand like partition -> queue level. To avoid duplicate calculation, i use this variable. 

bq.Should we lock the queue inside for (LeafQueue leafQUeue : queues)?
Make sense for me. I will handle this case.

bq.return value of getResourceDemandFromAppsPerQueue is not consumed by anyone.
I have cleaned up the code.

bq.TempAppPerQueue is not used by anyone
 I am moving some common logic computation and some storage to here. I will have this in next patch.

Thanks for sharing the flow. I also planned to do similar approach what you suggested, but I am doing some tasks together. I feel I can rearrange the code to fit to this code flow. However I feel deductPreemptableResourcesBasedSelectedCandidates could be done while we loop over each containers of an app also (we can deduct from resourceToObtain, but do not add same container again. I handled this scenario already). 


> [Umbrella] Capacity Scheduler Preemption Within a queue
> -------------------------------------------------------
>
>                 Key: YARN-4945
>                 URL: https://issues.apache.org/jira/browse/YARN-4945
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Wangda Tan
>         Attachments: Intra-Queue Preemption Use Cases.pdf, IntraQueuepreemption-CapacityScheduler (Design).pdf, YARN-2009-wip.2.patch, YARN-2009-wip.patch
>
>
> This is umbrella ticket to track efforts of preemption within a queue to support features like:
> YARN-2009. YARN-2113. YARN-4781.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org