You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Gour Saha (JIRA)" <ji...@apache.org> on 2016/06/04 02:15:59 UTC
[jira] [Updated] (YARN-4793) [Umbrella] Simplified API layer for
services and beyond
[ https://issues.apache.org/jira/browse/YARN-4793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gour Saha updated YARN-4793:
----------------------------
Attachment: 20160603-YARN-Simplified-V1-API-Examples.adoc
20160603-YARN-Simplified-V1-API-Layer-For-Services.yaml
20160603-YARN-Simplified-V1-API-Layer-For-Services.pdf
Attached is the first pdf draft (20160603) of the higher order API specification for YARN services.
An HTML version is also available here -
http://home.apache.org/~gourksaha/YARN-4793/20160603-YARN-Simplified-V1-API-Layer-For-Services.html
Thanks to [~vinodkv], [~sidharta-s], [~vvasudev], [~billie.rinaldi] and [~kkaranasos] for contributing to and/or performing extensive reviews of the specification.
The source of the specification is in Swagger format and JSON examples are in asciidoc format (both uploaded to this JIRA). I used the [swagger2markup-cli|https://github.com/Swagger2Markup/swagger2markup-cli] utility to auto generate the pdf and html.
Few points to keep in mind while reviewing the document -
- Application name is a unique identifier for a specific user. It is more user-friendly and easy to remember than the YARN application identifier.
- Few of the advanced features referred to in the spec is still in the works like – Placement Policy (YARN-4902), Cluster/Docker networking (YARN-4007)
- Readiness check for containers and service needs architectural proof of concept
I have an early prototype of the REST API Service, implemented against this API spec. It is built on top of Apache Slider and uses slider-core as a dependent module. If there is consensus on the spec, I can submit a patch.
> [Umbrella] Simplified API layer for services and beyond
> -------------------------------------------------------
>
> Key: YARN-4793
> URL: https://issues.apache.org/jira/browse/YARN-4793
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Gour Saha
> Attachments: 20160603-YARN-Simplified-V1-API-Examples.adoc, 20160603-YARN-Simplified-V1-API-Layer-For-Services.pdf, 20160603-YARN-Simplified-V1-API-Layer-For-Services.yaml
>
>
> [See overview doc at YARN-4692, modifying and copy-pasting some of the relevant pieces and sub-section 3.3.2 to track the specific sub-item.]
> Bringing a new service on YARN today is not a simple experience. The APIs of existing frameworks are either too low level (native YARN), require writing new code (for frameworks with programmatic APIs ) or writing a complex spec (for declarative frameworks).
> In addition to building critical building blocks inside YARN (as part of other efforts at YARN-4692), we should also look to simplifying the user facing story for building services. Experience of projects like Slider building real-life services like HBase, Storm, Accumulo, Solr etc gives us some very good learnings on how simplified APIs for building services will look like.
> To this end, we should look at a new simple-services API layer backed by REST interfaces. The REST layer can act as a single point of entry for creation and lifecycle management of YARN services. Services here can range from simple single-component apps to the most complex, multi-component applications needing special orchestration needs.
> We should also look at making this a unified REST based entry point for other important features like resource-profile management (YARN-3926), package-definitions' lifecycle-management and service-discovery (YARN-913 / YARN-4757). We also need to flesh out its relation to our present much lower level REST APIs (YARN-1695) in YARN for application-submission and management.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org