[jira] [Commented] (YARN-5501) Container Pooling in YARN

["Atri Sharma (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15414714#comment-15414714 ] 

Atri Sharma commented on YARN-5501:
-----------------------------------

Few Comments:
1) What would be the overhead of maintaining those containers when idle?
2) Would this not make sense only for short lived containers? If so, should we allow this behavior to be configurable?
3) Would the Capacity Scheduler be able to change the resource allocation for these pre allocated Containers? If the Containers are VMs, would that not require a VM restart ( I might be missing something here).

Regards,

Atri

> Container Pooling in YARN
> -------------------------
>
>                 Key: YARN-5501
>                 URL: https://issues.apache.org/jira/browse/YARN-5501
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Arun Suresh
>
> This JIRA proposes a method for reducing the container launch latency in YARN. It introduces a notion of pooling *Unattached Pre-Initialized Containers*.
> Proposal in brief:
> * Have a *Pre-Initialized Container Factory* service within the NM to create these unattached containers.
> * The NM would then advertise these containers as special resource types (this should be possible via YARN-3926).
> * When a start container request is received by the node manager for launching a container requesting this specific type of resource, it will take one of these unattached pre-initialized containers from the pool, and use it to service the container request.
> * Once the request is complete, the pre-initialized container would be released and ready to serve another request.
> This capability would help reduce container launch latencies and thereby allow for development of more interactive applications on YARN.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org