You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@celeborn.apache.org by ni...@apache.org on 2025/12/11 05:22:12 UTC

(celeborn) branch branch-0.6 updated: [CELEBORN-2212] Optimize the sorting efficiency of memoryWriters when evicting the largest memory file

This is an automated email from the ASF dual-hosted git repository.

nicholasjiang pushed a commit to branch branch-0.6
in repository https://gitbox.apache.org/repos/asf/celeborn.git


The following commit(s) were added to refs/heads/branch-0.6 by this push:
     new 76903f92f [CELEBORN-2212] Optimize the sorting efficiency of memoryWriters when evicting the largest memory file
76903f92f is described below

commit 76903f92f268276084d3b39a36faa4138f4df3d9
Author: xxx <95...@qq.com>
AuthorDate: Thu Dec 11 13:21:47 2025 +0800

    [CELEBORN-2212] Optimize the sorting efficiency of memoryWriters when evicting the largest memory file
    
    ### What changes were proposed in this pull request?
    
    Optimize the sorting efficiency of memoryWriters when evicting the largest memory file.
    
    ### Why are the changes needed?
    
    Optimize the sorting efficiency of memoryWriters when evicting the largest memory file.
    
    ### Does this PR resolve a correctness bug?
    
    No.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    CI.
    
    Closes #3549 from xy2953396112/CELEBORN-2212.
    
    Authored-by: xxx <95...@qq.com>
    Signed-off-by: SteNicholas <pr...@163.com>
    (cherry picked from commit 77a8921648c77a6fd073a09e07aa6f41a98153c1)
    Signed-off-by: SteNicholas <pr...@163.com>
---
 .../apache/celeborn/service/deploy/worker/memory/MemoryManager.java  | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/worker/src/main/java/org/apache/celeborn/service/deploy/worker/memory/MemoryManager.java b/worker/src/main/java/org/apache/celeborn/service/deploy/worker/memory/MemoryManager.java
index da81c92a3..a11b1b543 100644
--- a/worker/src/main/java/org/apache/celeborn/service/deploy/worker/memory/MemoryManager.java
+++ b/worker/src/main/java/org/apache/celeborn/service/deploy/worker/memory/MemoryManager.java
@@ -257,8 +257,9 @@ public class MemoryManager {
                 logger.info("Start evicting {} memory file infos", memoryWriters.size());
                 // always evict the largest memory file info first
                 memoryWriters.sort(
-                    Comparator.comparingLong(o -> o.getMemoryFileInfo().getFileLength()));
-                Collections.reverse(memoryWriters);
+                    Comparator.comparingLong(
+                            (PartitionDataWriter o) -> o.getMemoryFileInfo().getFileLength())
+                        .reversed());
                 for (PartitionDataWriter writer : memoryWriters) {
                   // this branch means that there is no memory pressure
                   if (!shouldEvict(aggressiveEvictModeEnabled, evictRatio)) {