[jira] [Commented] (KAFKA-266) Kafka web console design

["Jay Kreps (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202937#comment-13202937 ] 

Jay Kreps commented on KAFKA-266:
---------------------------------

I think this would be fantastic to have. We don't have such a thing at LinkedIn, we just have hooks into our monitoring system, but I think this would be better.

We do have an application that does a data audit. This basically proves that all data that is sent is received by all interested consumers. See KAFKA-260 for a detailed description and screenshot. We had planned to fold this functionality into the open source project. Since both of these are kind of monitoring/health functionality are pretty closely related, maybe it would make sense to combine them.

The existing app is a java app that implements runs a few jetty servlets to implement a JSON api. The UI just runs off this api and is implemented in hmtl/css/js. The charts are dygraph and it uses jquery for javascript helpers. I am not married to this setup (the original code was written by someone else and just refactored by me), so I would be okay moving it into another mvc framework if there is something much better out there as long as it isn't too complex.
                
> Kafka web console design
> ------------------------
>
>                 Key: KAFKA-266
>                 URL: https://issues.apache.org/jira/browse/KAFKA-266
>             Project: Kafka
>          Issue Type: New Feature
>          Components: contrib
>            Reporter: Evan Chan
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> This issue is created to track a community-contributed Kafka Web UI.
> Here is an initial list of goals:
> - Be able to easily see which brokers are up
> - Be able to see lists of topics, connected producers, consumer groups, connected consumers
> - Be able to see, for each consumer/partition, its offset, and more importantly, # of bytes unconsumed (== largest offset for partition - current offset)
> - (Wish list) have a graphical view of the offsets
> - (Wish list) be able to clean up consumer state, such as stale claimed partitions
> List of challenges/questions:
> - Which framework?  Play! for Scala?
> - Is all the data available from JMX and ZK?  Hopefully, watching the files on the filesystem can be avoided....
> - How to handle large numbers of topics, partitions, consumers, etc. efficiently

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira