[jira] [Created] (YARN-4793) [Umbrella] Simplified API layer for services and beyond

["Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>]

Vinod Kumar Vavilapalli created YARN-4793:
---------------------------------------------

             Summary: [Umbrella] Simplified API layer for services and beyond
                 Key: YARN-4793
                 URL: https://issues.apache.org/jira/browse/YARN-4793
             Project: Hadoop YARN
          Issue Type: Bug
            Reporter: Vinod Kumar Vavilapalli
            Assignee: Vinod Kumar Vavilapalli


[See overview doc at YARN-4692, modifying and copy-pasting some of the relevant pieces and sub-section 3.3.2 to track the specific sub-item.]

Bringing a new service on YARN today is not a simple experience. The APIs of existing frameworks are either too low­ level (native YARN), require writing new code (for frameworks with programmatic APIs ) or writing a complex spec (for declarative frameworks).

In addition to building critical building blocks inside YARN (as part of other efforts at YARN-4692), we should also look to simplifying the user facing story for building services. Experience of projects like Slider building real-­life services like HBase, Storm, Accumulo, Solr etc gives us some very good learnings on how simplified APIs for building services will look like.

To this end, we should look at a new simple-services API layer backed by REST interfaces. The REST layer can act as a single point of entry for creation and lifecycle management of YARN services. Services here can range from simple single-­component apps to the most complex, multi­-component applications needing special orchestration needs.

We should also look at making this a unified REST based entry point for other important features like resource­-profile management (YARN-3926), package-definitions' lifecycle­-management and service­-discovery (YARN-913 / YARN-4757). We also need to flesh out its relation to our present much ­lower level REST APIs (YARN-1695) in YARN for application-­submission and management.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)