You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by "kaijchen (via GitHub)" <gi...@apache.org> on 2023/02/07 09:25:16 UTC

[GitHub] [incubator-uniffle] kaijchen opened a new pull request, #558: [Followup #249] Cleanup code and unify interfaces

kaijchen opened a new pull request, #558:
URL: https://github.com/apache/incubator-uniffle/pull/558

   ### What changes were proposed in this pull request?
   
   Followup #249. Cleanup code and unify interfaces.
   
   ### Why are the changes needed?
   
   Cleanup code and unify interfaces.
   
   ### Does this PR introduce _any_ user-facing change?
   
   No.
   
   ### How was this patch tested?
   
   CI.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org

[GitHub] [incubator-uniffle] kaijchen commented on pull request #558: [Followup #249] refactor: cleanup code and unify interfaces

Posted by "kaijchen (via GitHub)" <gi...@apache.org>.
kaijchen commented on PR #558:
URL: https://github.com/apache/incubator-uniffle/pull/558#issuecomment-1422118888

   Thanks @advancedxy for the review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org

[GitHub] [incubator-uniffle] kaijchen merged pull request #558: [Followup #249] refactor: cleanup code and unify interfaces

Posted by "kaijchen (via GitHub)" <gi...@apache.org>.
kaijchen merged PR #558:
URL: https://github.com/apache/incubator-uniffle/pull/558


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org

[GitHub] [incubator-uniffle] advancedxy commented on a diff in pull request #558: [Followup #249] Cleanup code and unify interfaces

Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on code in PR #558:
URL: https://github.com/apache/incubator-uniffle/pull/558#discussion_r1099680871


##########
server/src/main/java/org/apache/uniffle/server/ShuffleFlushManager.java:
##########
@@ -361,8 +362,9 @@ private void addPendingEventsInternal(PendingShuffleFlushEvent event) {
     }
   }
 
-  public void removeResourcesOfShuffleId(String appId, int shuffleId) {
-    Optional.ofNullable(committedBlockIds.get(appId)).ifPresent(x -> x.remove(shuffleId));
+  public void removeResourcesOfShuffleId(String appId, Collection<Integer> shuffleIds) {

Review Comment:
   > This is aligned with removeBufferByShuffleId
   
   Then should `removeBufferByShuffleId` called `removeBufferByShuffleIds` instead?
   
   WDYT @kaijchen @zuston 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org

[GitHub] [incubator-uniffle] kaijchen commented on a diff in pull request #558: [Followup #249] Cleanup code and unify interfaces

Posted by "kaijchen (via GitHub)" <gi...@apache.org>.
kaijchen commented on code in PR #558:
URL: https://github.com/apache/incubator-uniffle/pull/558#discussion_r1099578320


##########
server/src/main/java/org/apache/uniffle/server/ShuffleFlushManager.java:
##########
@@ -361,8 +362,9 @@ private void addPendingEventsInternal(PendingShuffleFlushEvent event) {
     }
   }
 
-  public void removeResourcesOfShuffleId(String appId, int shuffleId) {
-    Optional.ofNullable(committedBlockIds.get(appId)).ifPresent(x -> x.remove(shuffleId));
+  public void removeResourcesOfShuffleId(String appId, Collection<Integer> shuffleIds) {

Review Comment:
   > IMO, this should be called `removeResourceOfShuffleIds`?
   
   This is aligned with `removeBufferByShuffleId`.
   
   > And another singular method `removeResourceOfShuffleId` should be added?
   
   There is no usage elsewhere now. We can add it when needed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org

[GitHub] [incubator-uniffle] advancedxy commented on a diff in pull request #558: [Followup #249] Cleanup code and unify interfaces

Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on code in PR #558:
URL: https://github.com/apache/incubator-uniffle/pull/558#discussion_r1099726120


##########
server/src/main/java/org/apache/uniffle/server/ShuffleFlushManager.java:
##########
@@ -361,8 +362,9 @@ private void addPendingEventsInternal(PendingShuffleFlushEvent event) {
     }
   }
 
-  public void removeResourcesOfShuffleId(String appId, int shuffleId) {
-    Optional.ofNullable(committedBlockIds.get(appId)).ifPresent(x -> x.remove(shuffleId));
+  public void removeResourcesOfShuffleId(String appId, Collection<Integer> shuffleIds) {

Review Comment:
   Good point.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org

[GitHub] [incubator-uniffle] codecov-commenter commented on pull request #558: [Followup #249] Cleanup code and unify interfaces

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #558:
URL: https://github.com/apache/incubator-uniffle/pull/558#issuecomment-1420493204

   # [Codecov](https://codecov.io/gh/apache/incubator-uniffle/pull/558?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#558](https://codecov.io/gh/apache/incubator-uniffle/pull/558?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7e71582) into [master](https://codecov.io/gh/apache/incubator-uniffle/commit/5cba4e77618ebb0d7dd057ca6e23513c33d8a918?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5cba4e7) will **decrease** coverage by `0.01%`.
   > The diff coverage is `100.00%`.
   
   ```diff
   @@             Coverage Diff              @@
   ##             master     #558      +/-   ##
   ============================================
   - Coverage     60.87%   60.86%   -0.01%     
   + Complexity     1796     1793       -3     
   ============================================
     Files           213      213              
     Lines         12372    12371       -1     
     Branches       1050     1049       -1     
   ============================================
   - Hits           7531     7530       -1     
     Misses         4434     4434              
     Partials        407      407              
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-uniffle/pull/558?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...org/apache/uniffle/server/ShuffleFlushManager.java](https://codecov.io/gh/apache/incubator-uniffle/pull/558?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS91bmlmZmxlL3NlcnZlci9TaHVmZmxlRmx1c2hNYW5hZ2VyLmphdmE=) | `84.45% <100.00%> (+0.08%)` | :arrow_up: |
   | [.../org/apache/uniffle/server/ShuffleTaskManager.java](https://codecov.io/gh/apache/incubator-uniffle/pull/558?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS91bmlmZmxlL3NlcnZlci9TaHVmZmxlVGFza01hbmFnZXIuamF2YQ==) | `76.62% <100.00%> (-0.14%)` | :arrow_down: |
   | [...he/uniffle/server/buffer/ShuffleBufferManager.java](https://codecov.io/gh/apache/incubator-uniffle/pull/558?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS91bmlmZmxlL3NlcnZlci9idWZmZXIvU2h1ZmZsZUJ1ZmZlck1hbmFnZXIuamF2YQ==) | `83.98% <100.00%> (ø)` | |
   
   :mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org

[GitHub] [incubator-uniffle] advancedxy commented on a diff in pull request #558: [Followup #249] Cleanup code and unify interfaces

Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on code in PR #558:
URL: https://github.com/apache/incubator-uniffle/pull/558#discussion_r1099574096


##########
server/src/main/java/org/apache/uniffle/server/ShuffleFlushManager.java:
##########
@@ -361,8 +362,9 @@ private void addPendingEventsInternal(PendingShuffleFlushEvent event) {
     }
   }
 
-  public void removeResourcesOfShuffleId(String appId, int shuffleId) {
-    Optional.ofNullable(committedBlockIds.get(appId)).ifPresent(x -> x.remove(shuffleId));
+  public void removeResourcesOfShuffleId(String appId, Collection<Integer> shuffleIds) {

Review Comment:
   IMO, this should be called `removeResourceOfShuffleIds`? And another singular method `removeResourceOfShuffleId` should be added?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org

[GitHub] [incubator-uniffle] kaijchen commented on a diff in pull request #558: [Followup #249] Cleanup code and unify interfaces

Posted by "kaijchen (via GitHub)" <gi...@apache.org>.
kaijchen commented on code in PR #558:
URL: https://github.com/apache/incubator-uniffle/pull/558#discussion_r1099682424


##########
server/src/main/java/org/apache/uniffle/server/ShuffleFlushManager.java:
##########
@@ -361,8 +362,9 @@ private void addPendingEventsInternal(PendingShuffleFlushEvent event) {
     }
   }
 
-  public void removeResourcesOfShuffleId(String appId, int shuffleId) {
-    Optional.ofNullable(committedBlockIds.get(appId)).ifPresent(x -> x.remove(shuffleId));
+  public void removeResourcesOfShuffleId(String appId, Collection<Integer> shuffleIds) {

Review Comment:
   Let's say if we need to add a singular method, we can just overload `removeBufferByShuffleId`.
   So `removeBufferByShuffleId` is fine.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org