[jira] [Commented] (YARN-1530) [Umbrella] Store, manage and serve per-framework application-timeline data

["Lohit Vijayarenu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13868242#comment-13868242 ] 

Lohit Vijayarenu commented on YARN-1530:
----------------------------------------

We might have also think about data transfer rate for REST endpoint from all AMs/Containers if this is hosted by ResourceManager. One idea could be to have REST endpoint be library which any AM can inherit. When AM initializes this library can init REST endpoint which then can push events to pluggable storage (HDFS/Kafka ...). This might be similar to how AM writes history events to HDFS today. This should give good scalability without changing much from API perspective. 

> [Umbrella] Store, manage and serve per-framework application-timeline data
> --------------------------------------------------------------------------
>
>                 Key: YARN-1530
>                 URL: https://issues.apache.org/jira/browse/YARN-1530
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Vinod Kumar Vavilapalli
>         Attachments: application timeline design-20140108.pdf
>
>
> This is a sibling JIRA for YARN-321.
> Today, each application/framework has to do store, and serve per-framework data all by itself as YARN doesn't have a common solution. This JIRA attempts to solve the storage, management and serving of per-framework data from various applications, both running and finished. The aim is to change YARN to collect and store data in a generic manner with plugin points for frameworks to do their own thing w.r.t interpretation and serving.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)