You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Jakob Homan (JIRA)" <ji...@apache.org> on 2014/06/11 19:59:02 UTC

[jira] [Commented] (SAMZA-286) Improve AM dashboard

    [ https://issues.apache.org/jira/browse/SAMZA-286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028122#comment-14028122 ] 

Jakob Homan commented on SAMZA-286:
-----------------------------------

My idea is that the dashboard should consume the metrics generated by the container and present them graphically, gaugy, etc. in a useful way out of the box.  While Samza does publish lots of metrics, it currently requires some other system (JMX direct connection, JMX via something like Nagios, consuming the snapshots via Kafka) in order to access them.  This severely hobbles the out-of-box experience as all new users have to gone on is the static dashboard and following along with various logs.

We should extend the AM dashboard to consume the metrics generated from the containers, store them locally within the AM, and present them as graphs with lots of jquery/d3/whatever goodness.  This functionality can be pluggable and dependent on either the existing jmx reporter or a new DashboardJMX reporter (or such).  

It's fine to store these metrics within the AM and have them go away at the end of each job run.  These are meant to be analogous to MR job pages rather than capturing performance across multiple instances of a particular job.  Cross-job-run metrics should continue to be stored in external reporters, such as nagios-type systems. 

These proxied-through-the-AM metrics should also be made available over REST so that users can build applications on top of them, such as dashboards over entire Samza DAGs or all the jobs running on a particular YARN cluster.

> Improve AM dashboard
> --------------------
>
>                 Key: SAMZA-286
>                 URL: https://issues.apache.org/jira/browse/SAMZA-286
>             Project: Samza
>          Issue Type: Improvement
>          Components: yarn
>            Reporter: Jakob Homan
>
> Currently the webpage provided by the AM consists of effectively just links to the log directories of running containers and a table of the job's configuration.  We should make it more interactive and useful.



--
This message was sent by Atlassian JIRA
(v6.2#6252)