You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Lin Ma <li...@gmail.com> on 2013/01/28 04:02:28 UTC

hadoop queue

Hi guys,

It is mentioned below for Hadoop queue (
http://blog.cloudera.com/blog/2008/11/job-scheduling-in-hadoop/), for the
comments "FIFO scheduling with priorities is used", does it mean for
resources (mapper/reducer slots) allocation inside each queue, two factors
((1) job submit order, i.e. FIFO, and (2) individual job priorities are
also considered)? So it means even if inside one queue, the resource
allocation is not fair between jobs?

*However, within each queue, FIFO scheduling with priorities is used,
except for one aspect – you can place a limit on percent of running tasks
per user, so that users share a cluster equally. In other words, the
capacity scheduler tries to simulate a separate FIFO/priority cluster for
each user and each organization, rather than performing fair sharing
between all jobs. *

regards,
Lin