[jira] [Comment Edited] (YARN-2838) Issues with TimeLineServer (Application History)

["Zhijie Shen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14206671#comment-14206671 ] 

Zhijie Shen edited comment on YARN-2838 at 11/12/14 12:44 AM:
--------------------------------------------------------------

[~Naganarasimha], sorry for not responding you immediately as being busy on finalizing 2.6. A quick scan through your issue document. Here's my clarification:

1. While the entry point of the this sub-module is still called ApplicationHistoryServer, it is actually generalized to be TimelineServer right now (definitely we need to refactor the code at some point). The baseline service provided the the timeline server is to allow the cluster and its apps to store their history information, metrics and so on by complying with the defined timeline data model. Later on, users and admins can query this information to do the analysis.

2. Application history (or we prefer to call it generic history service) is now a built-in service in the timeline server to record the generic history information of YARN apps. It was on a separate store (on FS), but after YARN-2033, it has been moved to the timeline store too, as a payload. We replace the old storage layer, but keep the existing interfaces (web UI, services, CLI) not changed to be the analog of what RM provides for running apps. We still didn't integrate TimelineClient and AHSClient, the latter of which is RPC interface of getting generic history information via RPC interface. APPLICATION_HISTORY_ENABLED is the only remaining old config to control whether we also want to pull the app info from the generic history service inside the timeline server. You may want to take a look at YARN-2033 to get more context about the change. Moreover, as a number of limitation of the old history store, we're no longer going to support it.

3. The document is definitely staled. I'll file separate document Jira, however, it's too late for 2.6. Let's target 2.7 for an up-to-date document about timeline service and its built-in generic history service (YARN-2854). Does it sound good?


was (Author: zjshen):
[~Naganarasimha], sorry for not responding you immediately as being busy on finalizing 2.6. A quick scan through your issue document. Here's my clarification:

1. While the entry point of the this sub-module is still called ApplicationHistoryServer, it is actually generalized to be TimelineServer right now (definitely we need to refactor the code at some point). The baseline service provided the the timeline server is to allow the cluster and its apps to store their history information, metrics and so on by complying with the defined timeline data model. Later on, users and admins can query this information to do the analysis.

2. Application history (or we prefer to call it generic history service) is now a built-in service in the timeline server to record the generic history information of YARN apps. It was on a separate store (on FS), but after YARN-2033, it has been moved to the timeline store too, as a payload. We replace the old storage layer, but keep the existing interfaces (web UI, services, CLI) not changed to be the analog of what RM provides for running apps. We still didn't integrate TimelineClient and AHSClient, the latter of which is RPC interface of getting generic history information via RPC interface. APPLICATION_HISTORY_ENABLED is the only remaining old config to control whether we also want to pull the app info from the generic history service inside the timeline server. You may want to take a look at YARN-2033 to get more context about the change. Moreover, as a number of limitation of the old history store, we're no longer going to support it.

3. The document is definitely staled. I'll file separate document Jira, however, it's too late for 2.6. Let's target 2.7 for an up-to-date document about timeline service and its built-in generic history service. Does it sound good?

> Issues with TimeLineServer (Application History)
> ------------------------------------------------
>
>                 Key: YARN-2838
>                 URL: https://issues.apache.org/jira/browse/YARN-2838
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: timelineserver
>    Affects Versions: 2.5.1
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
>         Attachments: IssuesInTimelineServer.pdf
>
>
> Few issues in usage of Timeline server for generic application history access



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)