[jira] [Commented] (YARN-3030) set up ATS writer with basic request serving structure and lifecycle

["Sangjin Lee (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279510#comment-14279510 ] 

Sangjin Lee commented on YARN-3030:
-----------------------------------

Jotting down my initial thoughts (I wish one could create subtasks of subtasks):

We need to satisfy 3 use cases with this: *the current per-node aggregator, RM's aggregator, and the (future) per-app aggregator*.

The baseline idea is to create a *logical* per-app aggregator as a service (CompositeService). The per-node aggregator can then be thought of as a thin container/manager/router of per-app aggregators which will come and go with applications. This will give us ease of development and maximal isolation between apps. Furthermore, it would help us supporting the per-app aggregator more easily.

However, since we want to serve RM's aggregator as well, it makes sense to create a (abstract?) base aggregator service that is common between RM (not app-specific) and per-app aggregators. The RM one could be a very thin extension of the base. The per-app aggregator would add app-based logic (mostly lifecycle management).

These are the pieces for this JIRA:
- set up this class hierarchy
- work out the timeline client API (both sync and async)
- implement the lifecycle of the base aggregator service
- implement the timeline client RPC server end (can be no-op for now)
- work out some batching-related logic

We would still need to work out the backing storage interface and serving reads, etc. but they are captured in other tickets.

Thoughts? Feedback?

> set up ATS writer with basic request serving structure and lifecycle
> --------------------------------------------------------------------
>
>                 Key: YARN-3030
>                 URL: https://issues.apache.org/jira/browse/YARN-3030
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Sangjin Lee
>
> Per design in YARN-2928, create an ATS writer as a service, and implement the basic service structure including the lifecycle management.
> Also, as part of this JIRA, we should come up with the ATS client API for sending requests to this ATS writer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)