[jira] [Commented] (YARN-6179) Support for collapse filter for flow entities with in the boundary of date range.

["Rohith Sharma K S (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-6179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862458#comment-15862458 ] 

Rohith Sharma K S commented on YARN-6179:
-----------------------------------------

As discussed on weekly call, copying [~sjlee0] [comment|https://issues.apache.org/jira/browse/YARN-6027?focusedCommentId=15861739&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15861739] from YARN-6027

{code}
Absolutely. Thanks for the reminder.
The idea of supporting the "collapse" here is to combine flow activities from different dates for the same flow and the user so that they are returned as a single flow activity record by the reader REST server. While we recognized that this may make rendering certain UI easier on the UI implementation standpoint, we saw a few problems with that as well.
First, this goes against the design intent on flow activities as "daily activities" of flows as a fair amount of thought went into designing that table to represent daily activities. We're not saying since the schema is this way the reader has to return the data that way too to satisfy the schema. We tried to design the schema to model truly useful concepts and that way we could accomplish things like filter pushdowns. In that sense, we need to see if we can support various use cases still based on that intent and design.
Another point was that if we were to introduce something like collapse we would ideally need to do this in a consistent manner across many/most endpoints. And that definitely means a large scope of introducing collapse if we do. So we agreed that we would recognize that as a separate discussion and still move forward with implementing pagination.
Let me know if I missed anything major.
{code}

> Support for collapse filter for flow entities with in the boundary of date range.
> ---------------------------------------------------------------------------------
>
>                 Key: YARN-6179
>                 URL: https://issues.apache.org/jira/browse/YARN-6179
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Rohith Sharma K S
>
> This is continuation discussion from YARN-6027 for supporting collapse flag in /flows API. The existing API returns flow entities per day basis. Lets say, if user X runs SleepJob yesterday and today, then /flows will return 2 flow entities with flow runs per daily basis. This basically because, (UseCase-1) flows are tracked by daily basis.
> (UseCase-2) Some users are not interested in duplicate flow entities being sent in response rather they expect single flow entity as response. The ability to collapse with a given time range need to supported from timeline reader. This will adds up flexible API filters that can be used by users. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org