[jira] [Commented] (YARN-2882) Introducing container types

["Carlo Curino (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14219763#comment-14219763 ] 

Carlo Curino commented on YARN-2882:
------------------------------------

To help understand this notion, think of containers types as a priority. The *guaranteed-start* containers, have higher priority, and are never over-booked (i.e., when they show up in the NM they are started instantaneously). By contrast the *queueable* containers are sent to the NM, and will be started only when there is room in the node. Also if *guaranteed-start* containers show up in a node that was completely utilized running *queueable* containers, the *queueable* containers are preempted/killed, to guarantee the start of the higher priority containers. 

_(The rest of the comment below is covered in other sub-JIRAs of YARN-2877, adding here some hints to the ideas for context)_
By having an explicit notion of container types, the AM can control when to use one type vs the other. For example, one can use *queueable* containers for tasks that are not yet on the critical path, and/or for short-running tasks (higher chance to complete).  One important use of *queueable* containers is to allow us to boost utilization of the nodes (having a queue of work, minimize the times in which the NM is idle). 

> Introducing container types
> ---------------------------
>
>                 Key: YARN-2882
>                 URL: https://issues.apache.org/jira/browse/YARN-2882
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Konstantinos Karanasos
>
> This JIRA introduces the notion of container types.
> We propose two initial types of containers: guaranteed-start and queueable containers.
> Guaranteed-start are the existing containers, which are allocated by the central RM and are instantaneously started, once allocated.
> Queueable is a new type of container, which allows containers to be queued in the NM, thus their execution may be arbitrarily delayed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)