[jira] [Commented] (YARN-3862) Decide which contents to retrieve and send back in response in TimelineReader

["Sangjin Lee (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14986444#comment-14986444 ] 

Sangjin Lee commented on YARN-3862:
-----------------------------------

I just kicked off a jenkins run on the latest patch.

Sorry [~varun_saxena] it took me a while to get around to looking at the patch. The overall approach seems pretty reasonable to me. I'll need to go over the patch in some detail, however.

One point I'd like to make is regarding {{TimelineEntityReader.constructFilterListBasedOnFields()}}. I see it using {{EntityColumnFamily}} and {{EntityColumnPrefix}}. I don't think that's quite right. In terms of the class hierarchy {{TimelineEntityReader}} sits below {{GenericEntityReader}} (which deals with the generic entity table). As such, it should be agnostic to the actual specific tables. The {{TimelineEntityReader.constructFilterListBasedOnFields()}} method should contain only the most generic implementation (which may well be returning null). Any logic that deals with the entity columns should belong in {{GenericEntityReader}}.

This also points to an issue with {{ApplicationEntityReader}}. Its {{constructFilterListBasedOnFields()}} method needs to be implemented in terms of application column family and application column prefix. So it needs to be properly overridden in that class.

> Decide which contents to retrieve and send back in response in TimelineReader
> -----------------------------------------------------------------------------
>
>                 Key: YARN-3862
>                 URL: https://issues.apache.org/jira/browse/YARN-3862
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-3862-YARN-2928.wip.01.patch, YARN-3862-YARN-2928.wip.02.patch
>
>
> Currently, we will retrieve all the contents of the field if that field is specified in the query API. In case of configs and metrics, this can become a lot of data even though the user doesn't need it. So we need to provide a way to query only a set of configs or metrics.
> As a comma spearated list of configs/metrics to be returned will be quite cumbersome to specify, we have to support either of the following options :
> # Prefix match
> # Regex
> # Group the configs/metrics and query that group.
> We also need a facility to specify a metric time window to return metrics in a that window. This may be useful in plotting graphs 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)