[jira] [Commented] (YARN-5636) Support reserving resources on certain nodes for certain applications

["Tao Jie (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15501286#comment-15501286 ] 

Tao Jie commented on YARN-5636:
-------------------------------

We did some trial like this:
# Add fields to ResourceRequest, ContainerRequest:
#* String reservedLabel,  to recognize those containers that resources reserved to (actually is slider app name)
#* boolean useReserved, tell the RM that whether this container needs to use reserved resource (true when it is a restarted container)
# When completing container on node, it will mark that resource as *reserved  resource* if this container has a reservedLabel
# When assigning resource to container on node, it will check available resource on node respect to reserved resource.
# Also reserved resource would release when it expires.
[~Naganarasimha], [~sunilg], [~leftnoteasy], Like to hear your thoughts.  

> Support reserving resources on certain nodes for certain applications
> ---------------------------------------------------------------------
>
>                 Key: YARN-5636
>                 URL: https://issues.apache.org/jira/browse/YARN-5636
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: scheduler
>            Reporter: Tao Jie
>
> We have met such circumstance:
> We are trying to run storm&kafka on yarn by Slider, and Storm&Kafka writes data to local disk on node. If some containers or the application fails, we expect that those containers would restart on the same node as they run before, otherwise data written on local would lost.
> For slider, it will trying to ensure restarted container on same nodes as before. However for yarn, resource may be assigned to other applications when former long-running application is down.
> As a result we'd better to have a mechanism that reserve some resource for certain long-running applications on certain nodes for a period of time. Does it make sense?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org