You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@celeborn.apache.org by GitBox <gi...@apache.org> on 2022/11/28 03:53:56 UTC

[GitHub] [incubator-celeborn] waitinfuture opened a new pull request, #1015: [CELEBORN-70][FOLLOWUP] Add epoch for each commitFiles request. Addre…

waitinfuture opened a new pull request, #1015:
URL: https://github.com/apache/incubator-celeborn/pull/1015

   …ss comments.
   
   # [BUG]/[FEATURE] title
   
   ### What changes were proposed in this pull request?
   
   
   ### Why are the changes needed?
   
   <!--
   Please clarify why the changes are needed. For instance,
     1. If you propose a new API, clarify the use case for a new API.
     2. If you fix a bug, you can clarify why it is a bug.
   -->
   
   ### What are the items that need reviewer attention?
   
   
   ### Related issues.
   
   
   ### Related pull requests.
   
   
   ### How was this patch tested?
   
   
   /cc @related-reviewer
   
   /assign @main-reviewer
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-celeborn] waitinfuture commented on pull request #1015: [CELEBORN-70][FOLLOWUP] Add epoch for each commitFiles request.

Posted by GitBox <gi...@apache.org>.
waitinfuture commented on PR #1015:
URL: https://github.com/apache/incubator-celeborn/pull/1015#issuecomment-1328531310

   cc @AngersZhuuuu I have refined some logic, plz have a look.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-celeborn] waitinfuture commented on pull request #1015: [CELEBORN-70][FOLLOWUP] Add epoch for each commitFiles request.

Posted by GitBox <gi...@apache.org>.
waitinfuture commented on PR #1015:
URL: https://github.com/apache/incubator-celeborn/pull/1015#issuecomment-1328520904

   cc @AngersZhuuuu I have refined some logic, plz have a look


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-celeborn] AngersZhuuuu commented on a diff in pull request #1015: [CELEBORN-70][FOLLOWUP] Add epoch for each commitFiles request.

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on code in PR #1015:
URL: https://github.com/apache/incubator-celeborn/pull/1015#discussion_r1033108579


##########
worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Controller.scala:
##########
@@ -318,8 +318,8 @@ private[deploy] class Controller(
       mapAttempts: Array[Int],
       epoch: Long): Unit = {
     // return null if shuffleKey does not exist
-    if (!partitionLocationInfo.containsShuffle(shuffleKey) && !shuffleCommitInfos.containsKey(
-        shuffleKey) && !shuffleCommitInfos.get(shuffleKey).containsKey(epoch)) {
+    if (!partitionLocationInfo.containsShuffle(shuffleKey) && (!shuffleCommitInfos.containsKey(
+        shuffleKey) || !shuffleCommitInfos.get(shuffleKey).containsKey(epoch))) {

Review Comment:
   We'd better add a comment for this logic.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-celeborn] AngersZhuuuu merged pull request #1015: [CELEBORN-70][FOLLOWUP] Add epoch for each commitFiles request.

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu merged PR #1015:
URL: https://github.com/apache/incubator-celeborn/pull/1015


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org