You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Varun Saxena (JIRA)" <ji...@apache.org> on 2016/03/01 10:51:18 UTC
[jira] [Commented] (YARN-4700) ATS storage has one extra record
each time the RM got restarted
[ https://issues.apache.org/jira/browse/YARN-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173510#comment-15173510 ]
Varun Saxena commented on YARN-4700:
------------------------------------
Thanks [~Naganarasimha] for the patch.
I have nothing further to add other than what Vrushali said about changes in main code. I have same comments.
Looked at the test failures.
For TestHBaseStorageFlowActivity,
FlowActivityRowKey constructor is used while parsing row key so I don't think we should be changing that.
I think we can just change the timestamps of the app events and as Vrushali suggested, keep all the timestamps within one day. So that we can test that different apps on a single day generate one flow for that day. Currently 4 flow activity entries are coming due to app event timestamps generating 4 different top of the day timestamps.
For the other test case failure i.e. in TestTimelineReaderWebServicesHBaseStorage, you will have to change daterange queries because those REST queries are based on current timestamp.
> ATS storage has one extra record each time the RM got restarted
> ---------------------------------------------------------------
>
> Key: YARN-4700
> URL: https://issues.apache.org/jira/browse/YARN-4700
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Li Lu
> Assignee: Naganarasimha G R
> Labels: yarn-2928-1st-milestone
> Attachments: YARN-4700-YARN-2928.v1.001.patch, YARN-4700-YARN-2928.wip.patch
>
>
> When testing the new web UI for ATS v2, I noticed that we're creating one extra record for each finished application (but still hold in the RM state store) each time the RM got restarted. It's quite possible that we add the cluster start timestamp into the default cluster id, thus each time we're creating a new record for one application (cluster id is a part of the row key). We need to fix this behavior, probably by having a better default cluster id.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)