[GitHub] [spark] Ngone51 commented on a change in pull request #33116: [SPARK-35259][SHUFFLE] Rename ExternalBlockHandler Timer variables to remove incorrect millis suffix

[GitBox <gi...@apache.org>]

Ngone51 commented on a change in pull request #33116:
URL: https://github.com/apache/spark/pull/33116#discussion_r662860573



##########
File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalBlockHandler.java
##########
@@ -323,10 +323,13 @@ private void checkAuth(TransportClient client, String appId) {
 
     public ShuffleMetrics() {
       allMetrics = new HashMap<>();
-      allMetrics.put("openBlockRequestLatencyMillis", openBlockRequestLatencyMillis);
-      allMetrics.put("registerExecutorRequestLatencyMillis", registerExecutorRequestLatencyMillis);
-      allMetrics.put("fetchMergedBlocksMetaLatencyMillis", fetchMergedBlocksMetaLatencyMillis);
-      allMetrics.put("finalizeShuffleMergeLatencyMillis", finalizeShuffleMergeLatencyMillis);
+      // Note that for the latency metrics, the default unit is actually nanos, not millis.
+      // The variables have been renamed, but to preserve backwards compatibility, the metric
+      // names remain unchanged. See SPARK-35259 for more details.
+      allMetrics.put("openBlockRequestLatencyMillis", openBlockRequestLatency);

Review comment:
       @dongjoon-hyun It requires a `Metric` here rather than a long value.
   
   The `Timer` doesn't seem to provide any APIs to get the milliseconds. The only way I see now is to implements Spark's own `Timer` by extending Dropwizard's one.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org