[jira] [Commented] (YARN-2073) Fair Scheduler: Add a utilization threshold to prevent preempting resources when cluster is free

["Andrey Stepachev (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14027644#comment-14027644 ] 

Andrey Stepachev commented on YARN-2073:
----------------------------------------

I read code and don't cleanly understand why inequality uses Available resources instead of Allocated?
With current code preemption performed only when cluster utilised under 20percent.
Suppose we have occupied 8GB of 10GB. And default preemption threshold == 0.8f.
>From description I thought that preemption should occur, but it doesn't.
Code below gives us: 
{code}
0.8 < availableMb / maxMem
0.8 < (10-8) / 10
0.8 < 0.2 
false
{code}
{code}
      return (preemptionUtilizationThreshold < Math.max(
          (float) rootMetrics.getAvailableMB() / clusterResource.getMemory(),
          (float) rootMetrics.getAvailableVirtualCores() /
              clusterResource.getVirtualCores()));
    }
{code}

Is that was an intention for this parameter (and it should be documented for non native speakers to be 
more clear, say: preemption will not performed if utilisation is above this parameter) or here was a mistake?

> Fair Scheduler: Add a utilization threshold to prevent preempting resources when cluster is free
> ------------------------------------------------------------------------------------------------
>
>                 Key: YARN-2073
>                 URL: https://issues.apache.org/jira/browse/YARN-2073
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 2.4.0
>            Reporter: Karthik Kambatla
>            Assignee: Karthik Kambatla
>            Priority: Critical
>             Fix For: 2.5.0
>
>         Attachments: yarn-2073-0.patch, yarn-2073-1.patch, yarn-2073-2.patch, yarn-2073-3.patch, yarn-2073-4.patch
>
>
> Preemption should kick in only when the currently available slots don't match the request. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)