You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Karthik Kambatla (JIRA)" <ji...@apache.org> on 2016/03/10 19:37:40 UTC
[jira] [Commented] (YARN-4120) FSAppAttempt.getResourceUsage()
should not take preemptedResource into account
[ https://issues.apache.org/jira/browse/YARN-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189695#comment-15189695 ]
Karthik Kambatla commented on YARN-4120:
----------------------------------------
[~xinxianyin] - splitting out getResourceUsage and getNetResourceUsage makes sense, but can we wait until YARN-4752 is done?
> FSAppAttempt.getResourceUsage() should not take preemptedResource into account
> ------------------------------------------------------------------------------
>
> Key: YARN-4120
> URL: https://issues.apache.org/jira/browse/YARN-4120
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: fairscheduler
> Reporter: Xianyin Xin
>
> When compute resource usage for Schedulables, the following code is envolved,
> {{FSAppAttempt.getResourceUsage}},
> {code}
> public Resource getResourceUsage() {
> return Resources.subtract(getCurrentConsumption(), getPreemptedResources());
> }
> {code}
> and this value is aggregated to FSLeafQueues and FSParentQueues. In my opinion, taking {{preemptedResource}} into account here is not reasonable, there are two main reasons,
> # it is something in future, i.e., even though these resources are marked as preempted, it is currently used by app, and these resources will be subtracted from {{currentCosumption}} once the preemption is finished. it's not reasonable to make arrange for it ahead of time.
> # there's another problem here, consider following case,
> {code}
> root
> / \
> queue1 queue2
> / \
> queue1.3, queue1.4
> {code}
> suppose queue1.3 need resource and it can preempt resources from queue1.4, the preemption happens in the interior of queue1. But when compute resource usage of queue1, {{queue1.resourceUsage = it's_current_resource_usage - preemption}} according to the current code, which is unfair to queue2 when doing resource allocating.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)