You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/08/01 00:37:16 UTC

[GitHub] [spark] HeartSaVioR commented on a change in pull request #29269: [SPARK-31792][SS][DOC][FOLLOW-UP] Rephrase the description for some operations

HeartSaVioR commented on a change in pull request #29269:
URL: https://github.com/apache/spark/pull/29269#discussion_r463897312



##########
File path: docs/web-ui.md
##########
@@ -426,11 +426,11 @@ queries. Currently, it contains the following metrics.
 * **Batch Duration.** The process duration of each batch. 
 * **Operation Duration.** The amount of time taken to perform various operations in milliseconds.
 The tracked operations are listed as follows.
-    * addBatch: Adds result data of the current batch to the sink.
-    * getBatch: Gets a new batch of data to process.
-    * latestOffset: Gets the latest offsets for sources. 
-    * queryPlanning: Generates the execution plan.
-    * walCommit: Writes the offsets to the metadata log.
+    * addBatch: Time taken to read the micro-batch's input data from the sources, process it, and write the batch's output to the sink. This should take the bulk of the micro-batch's time.
+    * getBatch: Time taken to prepare the logical query to read the input of the current micro-batch from the sources.
+    * getOffset: Time taken to query the maximum available offset for this source.

Review comment:
       What I meant about commenting latestOffset vs getOffset is, we are using "both" as of now, which looks to be OK to consolidate into one. I prefer `latestOffset`, given it will be picked up at least most of built-in data source.
   
   If you want to "just" change doc in this PR, you'd like to change this like `* latestOffset & getOffset: ...`.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org