You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@celeborn.apache.org by re...@apache.org on 2024/07/16 01:08:05 UTC

(celeborn) branch main updated: [CELEBORN-1507] Prevent invalid Filegroups from being used

This is an automated email from the ASF dual-hosted git repository.

rexxiong pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git


The following commit(s) were added to refs/heads/main by this push:
     new 82a3abf10 [CELEBORN-1507] Prevent invalid Filegroups from being used
82a3abf10 is described below

commit 82a3abf10e9a9c25307b3b2eb2859d5893cd2115
Author: zhangzhao.08 <zh...@bytedance.com>
AuthorDate: Tue Jul 16 09:07:54 2024 +0800

    [CELEBORN-1507] Prevent invalid Filegroups from being used
    
    ### What changes were proposed in this pull request?
    
    Prevent invalid Filegroups from being used.
    
    ### Why are the changes needed?
    
    #2531 introduced a new feature regarding Filegroups. For multiple tasks belonging to the same shuffleKey, only one of them will actively fetch, and the others will use the cached results in memory. If the Task that actively fetches the Filegroups is killed for some reason (such as another attempt succeeded), it will cause other tasks to obtain invalid Filegroups, eventually leading to stage retry.
    
    ![image](https://github.com/user-attachments/assets/7bb258bc-3c9b-457d-a583-2c60888af0e2)
    
    ### Does this PR introduce _any_ user-facing change?
    NO
    
    ### How was this patch tested?
    Internal test.
    
    Closes #2625 from zhaostu4/invalid_cache.
    
    Lead-authored-by: zhangzhao.08 <zh...@bytedance.com>
    Co-authored-by: ZHAO ZHAO <44...@users.noreply.github.com>
    Signed-off-by: Shuang <lv...@alibaba-inc.com>
---
 .../java/org/apache/celeborn/client/ShuffleClientImpl.java     | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java b/client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java
index e8770c537..53978d733 100644
--- a/client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java
+++ b/client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java
@@ -1669,7 +1669,15 @@ public class ShuffleClientImpl extends ShuffleClient {
   public ReduceFileGroups updateFileGroup(int shuffleId, int partitionId)
       throws CelebornIOException {
     Tuple2<ReduceFileGroups, String> fileGroupTuple =
-        reduceFileGroupsMap.computeIfAbsent(shuffleId, (id) -> loadFileGroupInternal(shuffleId));
+        reduceFileGroupsMap.compute(
+            shuffleId,
+            (id, existsTuple) -> {
+              if (existsTuple == null || existsTuple._1 == null) {
+                return loadFileGroupInternal(shuffleId);
+              } else {
+                return existsTuple;
+              }
+            });
     if (fileGroupTuple._1 == null) {
       throw new CelebornIOException(
           loadFileGroupException(shuffleId, partitionId, (fileGroupTuple._2)));