[GitHub] [spark] tgravescs commented on a change in pull request #29247: [SPARK-32446][SHS] Add new executor metrics summary REST APIs

[GitBox <gi...@apache.org>]

tgravescs commented on a change in pull request #29247:
URL: https://github.com/apache/spark/pull/29247#discussion_r523029232



##########
File path: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala
##########
@@ -52,6 +53,25 @@ private[v1] class AbstractApplicationResource extends BaseAppResource {
   @Path("executors")
   def executorList(): Seq[ExecutorSummary] = withUI(_.store.executorList(true))
 
+  @GET
+  @Path("executorSummary")

Review comment:
       should be executorMetricsSummary or perhaps this should be executormetricsDistribution or Quantiles or something.  The metrics themselves are in the /executors path in ExecutorSummary, correct?

##########
File path: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala
##########
@@ -366,6 +366,18 @@ private[spark] class AppStatusStore(
     Some(computedQuantiles)
   }
 
+  /**
+   * Calculates a summary of the executor metrics for executors, returning the
+   * requested quantiles for the recorded metrics.
+   */
+  def executorMetricSummary(activeOnly: Boolean,
+                            unsortedQuantiles: Array[Double])

Review comment:
       indentation si off

##########
File path: docs/monitoring.md
##########
@@ -518,6 +518,15 @@ can be identified by their `[attempt-id]`. In the API listed below, when running
     <td><code>/applications/[app-id]/allexecutors</code></td>
     <td>A list of all(active and dead) executors for the given application.</td>
   </tr>
+  <tr>
+    <td><code>/applications/[app-id]/executorSummary</code></td>
+    <td>
+      Summary peak memory metrics of executors.

Review comment:
       this is misleading the peak memory metrics are in the executors rest api already right?
   This seems just to add in the ability to get them as quantiles? 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org