Showing key cluster stats in the Web UI

[Nick Chammas <ni...@gmail.com>]

Someone correct me if this is wrong, but I believe 2 very important things
to know about your cluster are:

   1. How many cores does your cluster have available.
   2. How much memory does your cluster have available. (Perhaps this could
   be divided into total/in-use/free or something.)

Is this information easily available on the Web UI? Would it make sense to
add it in there in the environment overview page?

Continuing on that note, is it not also important to know what level of
parallelism your stages are running at? As in, how many concurrent tasks
are running for a given stage? If that is much lower than the number of
cores you have available, for example, that may be something obvious to
look into.

If so, showing the number of tasks running concurrently would be another
useful thing to add to the UI for the Stage detail page.

Does this make sense?

Nick




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Showing-key-cluster-stats-in-the-Web-UI-tp7150.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.