[jira] [Commented] (YARN-3981) support timeline clients not associated with an application

["Li Lu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-3981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524485#comment-15524485 ] 

Li Lu commented on YARN-3981:
-----------------------------

Continuing this thread. We may want to limit the active connections to the HBase backend, thus having collectors for clients on each node manager may be a little bit too aggressive. For normal use cases, a small subset of NMs running collectors would be enough? 

Another issue is that the RM also needs to discover available client collectors in the system. It can then dispatch those collectors to clients. The collector service discovery code for now couples app level collectors with their application id. We need to expand them to a more general collector_id, collector_data mapping to support collectors for clients. I'm opening a new JIRA to trace this. 

> support timeline clients not associated with an application
> -----------------------------------------------------------
>
>                 Key: YARN-3981
>                 URL: https://issues.apache.org/jira/browse/YARN-3981
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Rohith Sharma K S
>              Labels: YARN-5355
>
> In the current v.2 design, all timeline writes must belong in a flow/application context (cluster + user + flow + flow run + application).
> But there are use cases that require writing data outside the context of an application. One such example is a higher level client (e.g. tez client or hive/oozie/cascading client) writing flow-level data that spans multiple applications. We need to find a way to support them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org