You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hadoop.apache.org by Benson Qiu <be...@salesforce.com> on 2017/11/13 22:52:36 UTC

What does JobPriority mean?

I'm having trouble finding documentation for JobPriority
(mapreduce.job.priority).

> "Changes the priority of the job. Allowed priority values are VERY_HIGH,
HIGH, NORMAL, LOW, VERY_LOW"

I am using YARN CapacityScheduler, with preemption disabled. What exactly
does "low priority" or "high priority" mean in terms of how containers are
allocated to jobs?

Thanks,
Benson

Re: What does JobPriority mean?

Posted by Benson Qiu <be...@salesforce.com>.

Thanks, Sunil! I found your JIRA (YARN-1963
<https://issues.apache.org/jira/browse/YARN-1963>) that has a really great
design doc.

On Mon, Nov 13, 2017 at 6:04 PM, Sunil G <su...@apache.org> wrote:

> Hi Benson,
>
> Prior to 2.8 releases, YARN did not support priorities for its
> applications. Currently user can specify priority (higher integer value
> means higher priority) to its applications so that high priority apps could
> get resources faster from scheduler (priority is applicable within a leaf
> queue).
>
> In case of MR, we kept the same labels though. And each of this label is
> mapped to an integer like VERY_HIGH : 5;  HIGH : 4; NORMAL : 3; LOW : 2;
> VERY_LOW : 1; DEFAULT : 0.
>
> Apart from this labels, one can also specify an integer value to
> mapreduce.job.priority property.
> This means that mapreduce.job.priority = HIGH & mapreduce.job.priority  =
> 7 are valid.
>
> - Sunil
>
>
> On Tue, Nov 14, 2017 at 4:22 AM Benson Qiu <be...@salesforce.com>
> wrote:
>
>> I'm having trouble finding documentation for JobPriority
>> (mapreduce.job.priority).
>>
>> > "Changes the priority of the job. Allowed priority values are
>> VERY_HIGH, HIGH, NORMAL, LOW, VERY_LOW"
>>
>> I am using YARN CapacityScheduler, with preemption disabled. What exactly
>> does "low priority" or "high priority" mean in terms of how containers are
>> allocated to jobs?
>>
>> Thanks,
>> Benson
>>
>

Re: What does JobPriority mean?

Posted by Sunil G <su...@apache.org>.

Hi Benson,

Prior to 2.8 releases, YARN did not support priorities for its
applications. Currently user can specify priority (higher integer value
means higher priority) to its applications so that high priority apps could
get resources faster from scheduler (priority is applicable within a leaf
queue).

In case of MR, we kept the same labels though. And each of this label is
mapped to an integer like VERY_HIGH : 5;  HIGH : 4; NORMAL : 3; LOW : 2;
VERY_LOW : 1; DEFAULT : 0.

Apart from this labels, one can also specify an integer value to
mapreduce.job.priority property.
This means that mapreduce.job.priority = HIGH & mapreduce.job.priority  = 7
are valid.

- Sunil

On Tue, Nov 14, 2017 at 4:22 AM Benson Qiu <be...@salesforce.com>
wrote:

> I'm having trouble finding documentation for JobPriority
> (mapreduce.job.priority).
>
> > "Changes the priority of the job. Allowed priority values are VERY_HIGH,
> HIGH, NORMAL, LOW, VERY_LOW"
>
> I am using YARN CapacityScheduler, with preemption disabled. What exactly
> does "low priority" or "high priority" mean in terms of how containers are
> allocated to jobs?
>
> Thanks,
> Benson
>