[jira] [Comment Edited] (YARN-4692) [Umbrella] Simplified and first-class support for services in YARN

["Karthik Kambatla (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-4692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15244843#comment-15244843 ] 

Karthik Kambatla edited comment on YARN-4692 at 4/17/16 8:39 PM:
-----------------------------------------------------------------

(Discussed this offline with [~vinodkv]. Posting here so others are in the loop and provide their thoughts.)

Thanks for the quite comprehensive survey and putting the doc together, Vinod.

I would like for us to define what exactly we mean by services. I see three kinds:
# Long-running applications like streaming applications that run as a single user
# Services like HBase that run as a single user, but serve other users. These services might want to use Yarn for deployment and fault-tolerance, but not necessarily for sharing resources among its users or accounting their resource usage.
# Services like Impala/LLAP/Drill that serve other users and would like to use Yarn for sharing/accounting resources among them. 

The document covers several requirements of the first two cases. Can we consider the requirements of the third case as well - e.g. requesting resources on behalf of other users, delegating resources, latency of allocation etc.?


was (Author: kasha):
(Discussed this offline with [~vinodkv]. Posting here so others are in the loop and provide their thoughts.)

Thanks for the quite comprehensive survey and putting the doc together, Vinod.

I would like for us to define what exactly we mean by services. I see three kinds:
# Long-running applications like streaming applications that run as a single user
# Services like HBase that run as a single user, but serve other users. These services might want to use Yarn for deployment and fault-tolerance, but not necessarily for sharing resources among its users or accounting their resource usage.
# Services like Impala/LLAP/Drill that serve other users and would like to use Yarn for sharing/accounting resources among them. 

The document covers several requirements of the first two cases. I would like for us to understand the requirements of the third case as well - e.g. requesting resources on behalf of other users, delegating resources, latency of allocation etc. 

> [Umbrella] Simplified and first-class support for services in YARN
> ------------------------------------------------------------------
>
>                 Key: YARN-4692
>                 URL: https://issues.apache.org/jira/browse/YARN-4692
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>         Attachments: YARN-First-Class-And-Simplified-Support-For-Services-v0.pdf
>
>
> YARN-896 focused on getting the ball rolling on the support for services (long running applications) on YARN.
> I’d like propose the next stage of this effort: _Simplified and first-class support for services in YARN_.
> The chief rationale for filing a separate new JIRA is threefold:
>  - Do a fresh survey of all the things that are already implemented in the project
>  - Weave a comprehensive story around what we further need and attempt to rally the community around a concrete end-goal, and
>  - Additionally focus on functionality that YARN-896 and friends left for higher layers to take care of and see how much of that is better integrated into the YARN platform itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)