You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by "maobaolong (via GitHub)" <gi...@apache.org> on 2024/10/11 02:23:45 UTC

[PR] [MINOR] improvement(server): Record more grpc process time and total metrics [incubator-uniffle]

maobaolong opened a new pull request, #2167:
URL: https://github.com/apache/incubator-uniffle/pull/2167

   <!--
   1. Title: [#<issue>] <type>(<scope>): <subject>
      Examples:
        - "[#123] feat(operator): support xxx"
        - "[#233] fix: check null before access result in xxx"
        - "[MINOR] refactor: fix typo in variable name"
        - "[MINOR] docs: fix typo in README"
        - "[#255] test: fix flaky test NameOfTheTest"
      Reference: https://www.conventionalcommits.org/en/v1.0.0/
   2. Contributor guidelines:
      https://github.com/apache/incubator-uniffle/blob/master/CONTRIBUTING.md
   3. If the PR is unfinished, please mark this PR as draft.
   -->
   
   ### What changes were proposed in this pull request?
   
   Record more grpc process time and total metrics
   
   ### Why are the changes needed?
   
   Especially, the reportShuffleResult and getShuffleResultForMultiPart would be the most cost time call, this PR could measure  the performance of these methods.
   
   ### Does this PR introduce _any_ user-facing change?
   
   Introduced new metrics.
   - grpc_get_shuffle_result_for_multi_part_process_latency
   - grpc_report_shuffle_result_process_latency
   - grpc_get_shuffle_result_for_multi_part_total
   
   ### How was this patch tested?
   
   Locally.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] improvement(server): Record more grpc process time and total metrics [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong closed pull request #2167: [MINOR] improvement(server): Record more grpc process time and total metrics
URL: https://github.com/apache/incubator-uniffle/pull/2167


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] improvement(server): Record more grpc process time and total metrics [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on PR #2167:
URL: https://github.com/apache/incubator-uniffle/pull/2167#issuecomment-2408412366

   @jerqi Thanks for your review!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] improvement(server): Record more grpc process time and total metrics [incubator-uniffle]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #2167:
URL: https://github.com/apache/incubator-uniffle/pull/2167#issuecomment-2406477394

   ## Test Results
    2 926 files  ±0   2 926 suites  ±0   5h 59m 36s :stopwatch: + 3m 50s
    1 041 tests ±0   1 038 :white_check_mark:  -  1   2 :zzz: ±0   1 :x: + 1 
   13 003 runs  ±0  12 958 :white_check_mark:  - 15  30 :zzz: ±0  15 :x: +15 
   
   For more details on these failures, see [this check](https://github.com/apache/incubator-uniffle/runs/31388836772).
   
   Results for commit 449fe334. ± Comparison against base commit be23f613.
   
   [test-results]:data:application/gzip;base64,H4sIAPeXCGcC/03MTQ6DIBAF4KsY1l0gAyq9TMPfJKQqDcLK9O4FW6nL982btxP0s9vIvWOSDbeObNmna7Y5quTDWqUXY6VyT7XRU96f8bFlYw6D6W9P/6p/DVD5uZYauBhDLEKLxLweq0Ap/GJbZVJMzb6rcP60VXHCddWEZfGpBMK5RAfARxRGoFVaglaIeihEpTbaOSmsseT9Aaar3iEVAQAA
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] improvement(server): Record more grpc process time and total metrics [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on PR #2167:
URL: https://github.com/apache/incubator-uniffle/pull/2167#issuecomment-2406550871

   @jerqi Thanks for your review, PTAL


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] improvement(server): Record more grpc process time and total metrics [incubator-uniffle]

Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on code in PR #2167:
URL: https://github.com/apache/incubator-uniffle/pull/2167#discussion_r1796365501


##########
server/src/test/java/org/apache/uniffle/server/ShuffleServerMetricsTest.java:
##########
@@ -240,7 +242,9 @@ public void testNettyMetrics() throws Exception {
     ObjectMapper mapper = new ObjectMapper();
     JsonNode actualObj = mapper.readTree(content);
     assertEquals(2, actualObj.size());
-    assertEquals(68, actualObj.get("metrics").size());
+    // the original metrics have 69 items before this commit, but is would grow up
+    // when the new metric is added, we needn't correct this test while adding new metrics
+    assertTrue(actualObj.get("metrics").size() > 50);

Review Comment:
   Prefer not. This is also a check for metrics.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] improvement(server): Record more grpc process time and total metrics [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on code in PR #2167:
URL: https://github.com/apache/incubator-uniffle/pull/2167#discussion_r1796418935


##########
server/src/test/java/org/apache/uniffle/server/ShuffleServerMetricsTest.java:
##########
@@ -240,7 +242,9 @@ public void testNettyMetrics() throws Exception {
     ObjectMapper mapper = new ObjectMapper();
     JsonNode actualObj = mapper.readTree(content);
     assertEquals(2, actualObj.size());
-    assertEquals(68, actualObj.get("metrics").size());
+    // the original metrics have 69 items before this commit, but is would grow up
+    // when the new metric is added, we needn't correct this test while adding new metrics
+    assertTrue(actualObj.get("metrics").size() > 50);

Review Comment:
   Ok, revert it back.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] improvement(server): Record more grpc process time and total metrics [incubator-uniffle]

Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi merged PR #2167:
URL: https://github.com/apache/incubator-uniffle/pull/2167


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org