[jira] [Updated] (YARN-5265) Make HBase configuration for the timeline service configurable

["Joep Rottinghuis (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-5265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joep Rottinghuis updated YARN-5265:
-----------------------------------
    Attachment: ATS v2 cluster deployment v1.png

Attaching diagram that [~vrushalic] created. This helps in visualizing the challenge. The cluster on which the nodemanagers run and/or connect to by default do not necessarily have to be the same as the HBase cluster that ATS needs to write to.

There are two configurations involved. The Yarn configuration for the source cluster, that is the cluster where the timelineservice client run on (as part of the nodemanager, resourcemanager, or reader service). This configuration will have most of the Yarn ATS properties.
The second configuration is the one that identifies the HBase cluster to which the timelineservice client has to write to (or read from). These settings may not be the same as the default cluster that tasks on the cluster connect to.

It isn't good enough to require the ATS HBase cluster configs to be on the classpath of the nodemanagers, resource manager etc.

> Make HBase configuration for the timeline service configurable
> --------------------------------------------------------------
>
>                 Key: YARN-5265
>                 URL: https://issues.apache.org/jira/browse/YARN-5265
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Joep Rottinghuis
>            Assignee: Joep Rottinghuis
>         Attachments: ATS v2 cluster deployment v1.png
>
>
> Currently we create "default" HBase configurations, this works as long as the user places the appropriate configuration on the classpath.
> This works fine for a standalone Hadoop cluster.
> However, if a user wants to monitor an HBase cluster and has a separate ATS HBase cluster, then it can become tricky to create the right classpath for the nodemanagers and still have tasks have their separate configs.
> It will be much easier to add a yarn configuration to let cluster admins configure which HBase to connect to to write ATS metrics to.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org