[jira] [Commented] (YARN-5281) Explore supporting a simpler back-end implementation for ATS v2

["Joep Rottinghuis (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-5281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15342385#comment-15342385 ] 

Joep Rottinghuis commented on YARN-5281:
----------------------------------------

Capturing [~varun_saxena]'s response on the email thread:
{noformat}
Thanks Karthik for sharing your views.

With regards to merging, it would help to have clear documentation on how to setup and use ATS.
--> We do have documentation on this. You and others who are interested can check out YARN-5174 which is the latest documentation related JIRA for ATSv2.

Slightly unrelated to the merge, do we plan to support any other simpler backend for users to try out, in addition to HBase? LevelDB?
--> We do have a File System based implementation but it is strictly for test purposes (as we write data into a local file). It does not support all the features of Timeline Service v.2 as well.
Regarding LevelDB, Timeline Service v.2 has distributed writers and Level DB writes data (log files or SSTable files) to local file system. This means there will be no easy way to have a LevelDB based implementation because we would not know where to read the data from, especially while fetching flow level information.
We can however, potentially change the Local File System based implementation to a HDFS based implementation and have it as an alternate for non-production use, if there is a potential need for it, based on community feedback. This however, would have to be further discussed with the team.

Regards,
Varun Saxena.
{noformat}

> Explore supporting a simpler back-end implementation for ATS v2
> ---------------------------------------------------------------
>
>                 Key: YARN-5281
>                 URL: https://issues.apache.org/jira/browse/YARN-5281
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Joep Rottinghuis
>
> During the merge discussion [~kasha] raised the question whether we would support simpler backend for users to try out, in addition to the HBase implementation.
> The understanding is that this would not be meant to scale, but it could simplify initial adoption and early usage.
> I'm filing this jira to gather the merits and challenges of such approach in one place.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org