You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@uniffle.apache.org by ro...@apache.org on 2025/09/18 09:55:17 UTC

(uniffle) branch branch-0.10.bak created (now 3cab5a8a0)

This is an automated email from the ASF dual-hosted git repository.

roryqi pushed a change to branch branch-0.10.bak
in repository https://gitbox.apache.org/repos/asf/uniffle.git


      at 3cab5a8a0 [#2307] license: Add left jars with Apache license (#2310)

This branch includes the following new commits:

     new 334a3fb71 Fix ConcurrentModifiedException while use TestAppender
     new 7ca24bb38 [#2282] fix(spark3): Invoke with object self in DelegationRssShuffleManager (#2283)
     new 562fa7304 [#2290] docker: Fix the build of Centos image (#2291)
     new d2faa9566 [#2292] fix(server): Potential hang when HadoopShuffleWriteHandler initialization failure in PooledHadoopShuffleWriteHandler (#2293)
     new 7d07a9d1f [#2298] fix(server): Correct the metric of grpc_get_local_shuffle_data_process_latency (#2299)
     new 77fbd8582 [#2188] fix(client): Fixup timeoutMs for executeTasks in ShuffleWriteClientImpl#sendAppHeartbeat (#2189)
     new b10a4bc82 [#2298] fix(server): followup to correct the metric of grpc_get_local_shuffle_index_process_latency (#2302)
     new 23ff0ea10 [#2304] improvement: Update the year of NOTICE (#2313)
     new 9aaead8c3 [#2306] improvement: Remove repeated notice information (#2311)
     new 401488705 [MINOR] docs: Fix the license format of remote_merge_guide.md (#2314)
     new 3cab5a8a0 [#2307] license: Add left jars with Apache license (#2310)

The 11 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.



(uniffle) 06/11: [#2188] fix(client): Fixup timeoutMs for executeTasks in ShuffleWriteClientImpl#sendAppHeartbeat (#2189)

Posted by ro...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

roryqi pushed a commit to branch branch-0.10.bak
in repository https://gitbox.apache.org/repos/asf/uniffle.git

commit 77fbd8582a780e10c663e82d9890d53449285ffe
Author: xumanbu <ja...@vipshop.com>
AuthorDate: Thu Dec 19 12:00:56 2024 +0800

    [#2188] fix(client): Fixup timeoutMs for executeTasks in ShuffleWriteClientImpl#sendAppHeartbeat (#2189)
    
    ### What changes were proposed in this pull request?
    
     Fixup timeoutMs for executeTasks in ShuffleWriteClientImpl#sendAppHeartbeat
    
    ### Why are the changes needed?
    
    The `timeoutMs` in ShuffleWriteClientImpl#sendAppHeartbeat method. The timeoutMs for per RPC is similar to the timeoutMs of ThreadUtils#executeTasks` for all servers RPC execute, and the logic here is flawed.
    
    Resetting the `timeoutMs` for `ThreadUtils.executeTask`.
    
    Fix: #2188
    
    ### Does this PR introduce _any_ user-facing change?
    No.
    
    ### How was this patch tested?
    
    no need
---
 .../java/org/apache/uniffle/client/impl/ShuffleWriteClientImpl.java     | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/client/src/main/java/org/apache/uniffle/client/impl/ShuffleWriteClientImpl.java b/client/src/main/java/org/apache/uniffle/client/impl/ShuffleWriteClientImpl.java
index b78f06bd3..006eb1819 100644
--- a/client/src/main/java/org/apache/uniffle/client/impl/ShuffleWriteClientImpl.java
+++ b/client/src/main/java/org/apache/uniffle/client/impl/ShuffleWriteClientImpl.java
@@ -934,7 +934,7 @@ public class ShuffleWriteClientImpl implements ShuffleWriteClient {
           }
           return null;
         },
-        timeoutMs,
+        timeoutMs * allShuffleServers.size() / heartBeatThreadNum,
         "send heartbeat to shuffle server");
     if (coordinatorClient != null) {
       coordinatorClient.scheduleAtFixedRateToSendAppHeartBeat(request);


(uniffle) 09/11: [#2306] improvement: Remove repeated notice information (#2311)

Posted by ro...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

roryqi pushed a commit to branch branch-0.10.bak
in repository https://gitbox.apache.org/repos/asf/uniffle.git

commit 9aaead8c3cbb2e63384bb0d4c883da677176cb0b
Author: roryqi <ro...@apache.org>
AuthorDate: Mon Dec 30 14:11:00 2024 +0800

    [#2306] improvement: Remove repeated notice information (#2311)
    
    ### What changes were proposed in this pull request?
    
    Remove repeated notice information
    
    ### Why are the changes needed?
    
    Fix: #2306
    
    
    ### Does this PR introduce _any_ user-facing change?
    No.
    
    ### How was this patch tested?
    No need.
---
 NOTICE-binary | 23 +----------------------
 1 file changed, 1 insertion(+), 22 deletions(-)

diff --git a/NOTICE-binary b/NOTICE-binary
index 80eec6c2d..c7ffeecb0 100644
--- a/NOTICE-binary
+++ b/NOTICE-binary
@@ -201,28 +201,7 @@ src/test/org/apache/commons/codec/language/DoubleMetaphoneTest.java
 contains test data from http://aspell.net/test/orig/batch0.tab.
 Copyright (C) 2002 Kevin Atkinson (kevina@gnu.org)
 
-# Jackson JSON processor
-
-Jackson is a high-performance, Free/Open Source JSON processing library.
-It was originally written by Tatu Saloranta (tatu.saloranta@iki.fi), and has
-been in development since 2007.
-It is currently developed by a community of developers, as well as supported
-commercially by FasterXML.com.
-
-## Licensing
-
-Jackson core and extension components may be licensed under different licenses.
-To find the details that apply to this artifact see the accompanying LICENSE file.
-For more information, including possible other licensing options, contact
-FasterXML.com (http://fasterxml.com).
-
-## Credits
-
-A list of contributors may be found from CREDITS file, which is included
-in some artifacts (usually source distributions); but is always available
-from the source code management (SCM) system project uses.
-
-# Notice for Jersey 
+# Notice for Jersey
 This content is produced and maintained by the Eclipse Jersey project.
 
 *  Project home: https://projects.eclipse.org/projects/ee4j.jersey


(uniffle) 03/11: [#2290] docker: Fix the build of Centos image (#2291)

Posted by ro...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

roryqi pushed a commit to branch branch-0.10.bak
in repository https://gitbox.apache.org/repos/asf/uniffle.git

commit 562fa7304ff9f5647474119c83ece8e9294141f5
Author: roryqi <ro...@apache.org>
AuthorDate: Mon Dec 16 13:33:44 2024 +0800

    [#2290] docker: Fix the build of Centos image (#2291)
    
    ### What changes were proposed in this pull request?
    
     Fix the build of Centos image
    
    ### Why are the changes needed?
    Fix #2290
    
    ### Does this PR introduce _any_ user-facing change?
    No.
    
    ### How was this patch tested?
    Tested locally.
---
 deploy/kubernetes/docker/base/centos/Dockerfile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/deploy/kubernetes/docker/base/centos/Dockerfile b/deploy/kubernetes/docker/base/centos/Dockerfile
index 4d9a5346f..275e8aadb 100644
--- a/deploy/kubernetes/docker/base/centos/Dockerfile
+++ b/deploy/kubernetes/docker/base/centos/Dockerfile
@@ -25,6 +25,6 @@ RUN sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-* && \
     yum clean all && \
     yum makecache
 
-RUN yum install lzo zlib zlib-devel lzop lsof netcat dnsutils less procps iputils-ping -y && \
+RUN yum install lzo zlib zlib-devel lzop lsof netcat dnsutils less procps iputils-ping which -y && \
     yum clean all && \
     rm -rf /var/cache/yum


(uniffle) 08/11: [#2304] improvement: Update the year of NOTICE (#2313)

Posted by ro...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

roryqi pushed a commit to branch branch-0.10.bak
in repository https://gitbox.apache.org/repos/asf/uniffle.git

commit 23ff0ea10fedf4ccce1852ae54223fd6790d4cd0
Author: roryqi <ro...@apache.org>
AuthorDate: Fri Dec 27 20:55:05 2024 +0800

    [#2304] improvement: Update the year of NOTICE (#2313)
    
    ### What changes were proposed in this pull request?
    
    Update the year of NOTICE
    
    ### Why are the changes needed?
    
    Fix: #2304
    
    ### Does this PR introduce _any_ user-facing change?
    No.
    
    ### How was this patch tested?
    No need.
---
 NOTICE | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/NOTICE b/NOTICE
index 0b491637f..2cfb9fb7c 100644
--- a/NOTICE
+++ b/NOTICE
@@ -1,5 +1,5 @@
 Apache Uniffle (incubating)
-Copyright 2022 The Apache Software Foundation.
+Copyright 2022 and onwards The Apache Software Foundation.
 
 This product includes software developed at
 The Apache Software Foundation (https://www.apache.org/).


(uniffle) 01/11: Fix ConcurrentModifiedException while use TestAppender

Posted by ro...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

roryqi pushed a commit to branch branch-0.10.bak
in repository https://gitbox.apache.org/repos/asf/uniffle.git

commit 334a3fb7164ebd8fb0ebd1332a1f11f0897956be
Author: baoloongmao <ba...@tencent.com>
AuthorDate: Tue Dec 10 19:07:28 2024 +0800

    Fix ConcurrentModifiedException while use TestAppender
---
 .../test/java/org/apache/uniffle/common/log/TestLoggerExtension.java    | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/common/src/test/java/org/apache/uniffle/common/log/TestLoggerExtension.java b/common/src/test/java/org/apache/uniffle/common/log/TestLoggerExtension.java
index df0e815eb..ac3f968e3 100644
--- a/common/src/test/java/org/apache/uniffle/common/log/TestLoggerExtension.java
+++ b/common/src/test/java/org/apache/uniffle/common/log/TestLoggerExtension.java
@@ -134,7 +134,7 @@ public class TestLoggerExtension implements BeforeEachCallback, AfterEachCallbac
     }
 
     @Override
-    public void append(LogEvent event) {
+    public synchronized void append(LogEvent event) {
       events.add(event);
     }
   }


(uniffle) 05/11: [#2298] fix(server): Correct the metric of grpc_get_local_shuffle_data_process_latency (#2299)

Posted by ro...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

roryqi pushed a commit to branch branch-0.10.bak
in repository https://gitbox.apache.org/repos/asf/uniffle.git

commit 7d07a9d1f7dc62cbd68108629be4a80b80d583ed
Author: Junfan Zhang <zu...@apache.org>
AuthorDate: Wed Dec 18 11:53:49 2024 +0800

    [#2298] fix(server): Correct the metric of grpc_get_local_shuffle_data_process_latency (#2299)
    
    ### What changes were proposed in this pull request?
    
    Correct the metric of grpc_get_local_shuffle_data_process_latency
    
    ### Why are the changes needed?
    
    fix #2298
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Needn't
    
    
    Co-authored-by: Junfan Zhang <zh...@qiyi.com>
---
 .../org/apache/uniffle/server/ShuffleServerGrpcService.java  | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/server/src/main/java/org/apache/uniffle/server/ShuffleServerGrpcService.java b/server/src/main/java/org/apache/uniffle/server/ShuffleServerGrpcService.java
index 79206492a..d0b0213cd 100644
--- a/server/src/main/java/org/apache/uniffle/server/ShuffleServerGrpcService.java
+++ b/server/src/main/java/org/apache/uniffle/server/ShuffleServerGrpcService.java
@@ -1170,6 +1170,12 @@ public class ShuffleServerGrpcService extends ShuffleServerImplBase {
                       offset,
                       length,
                       storageId);
+          reply =
+              GetLocalShuffleDataResponse.newBuilder()
+                  .setStatus(status.toProto())
+                  .setRetMsg(msg)
+                  .setData(UnsafeByteOperations.unsafeWrap(sdr.getData()))
+                  .build();
           long readTime = System.currentTimeMillis() - start;
           ShuffleServerMetrics.counterTotalReadTime.inc(readTime);
           ShuffleServerMetrics.counterTotalReadDataSize.inc(sdr.getDataLength());
@@ -1183,12 +1189,6 @@ public class ShuffleServerGrpcService extends ShuffleServerImplBase {
               "Successfully getShuffleData cost {} ms for shuffle data with {}",
               readTime,
               requestInfo);
-          reply =
-              GetLocalShuffleDataResponse.newBuilder()
-                  .setStatus(status.toProto())
-                  .setRetMsg(msg)
-                  .setData(UnsafeByteOperations.unsafeWrap(sdr.getData()))
-                  .build();
         } catch (Exception e) {
           status = StatusCode.INTERNAL_ERROR;
           msg = "Error happened when get shuffle data for " + requestInfo + ", " + e.getMessage();


(uniffle) 02/11: [#2282] fix(spark3): Invoke with object self in DelegationRssShuffleManager (#2283)

Posted by ro...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

roryqi pushed a commit to branch branch-0.10.bak
in repository https://gitbox.apache.org/repos/asf/uniffle.git

commit 7ca24bb388964ab302a5bb5dab1a1b6f121b52f3
Author: Junfan Zhang <zu...@apache.org>
AuthorDate: Wed Dec 11 09:30:19 2024 +0800

    [#2282] fix(spark3): Invoke with object self in DelegationRssShuffleManager (#2283)
    
    Co-authored-by: Junfan Zhang <zh...@qiyi.com>
    
    ### What changes were proposed in this pull request?
    
    Fix the bug of none object self when invoking the reflection method.
    
    ### Why are the changes needed?
    
    Fix: #2282
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Existing tests
---
 .../main/java/org/apache/spark/shuffle/DelegationRssShuffleManager.java | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/client-spark/spark3/src/main/java/org/apache/spark/shuffle/DelegationRssShuffleManager.java b/client-spark/spark3/src/main/java/org/apache/spark/shuffle/DelegationRssShuffleManager.java
index aaa2c1092..dfdf587c3 100644
--- a/client-spark/spark3/src/main/java/org/apache/spark/shuffle/DelegationRssShuffleManager.java
+++ b/client-spark/spark3/src/main/java/org/apache/spark/shuffle/DelegationRssShuffleManager.java
@@ -247,6 +247,7 @@ public class DelegationRssShuffleManager implements ShuffleManager {
                       TaskContext.class,
                       ShuffleReadMetricsReporter.class)
                   .invoke(
+                      delegate,
                       handle,
                       startMapIndex,
                       endMapIndex,
@@ -285,6 +286,7 @@ public class DelegationRssShuffleManager implements ShuffleManager {
                       TaskContext.class,
                       ShuffleReadMetricsReporter.class)
                   .invoke(
+                      delegate,
                       handle,
                       startMapIndex,
                       endMapIndex,


(uniffle) 04/11: [#2292] fix(server): Potential hang when HadoopShuffleWriteHandler initialization failure in PooledHadoopShuffleWriteHandler (#2293)

Posted by ro...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

roryqi pushed a commit to branch branch-0.10.bak
in repository https://gitbox.apache.org/repos/asf/uniffle.git

commit d2faa95668c9e97f4892a7855074840975af96de
Author: Junfan Zhang <zu...@apache.org>
AuthorDate: Tue Dec 17 11:25:13 2024 +0800

    [#2292] fix(server): Potential hang when HadoopShuffleWriteHandler initialization failure in PooledHadoopShuffleWriteHandler (#2293)
    
    ### What changes were proposed in this pull request?
    
    Correct the initialization flag to fix potential hang in PooledHadoopShuffleWriteHandler
    
    ### Why are the changes needed?
    
    fix: #2292
    
    ### Does this PR introduce _any_ user-facing change?
    <!--
    (Please list the user-facing changes introduced by your change, including
      1. Change in user-facing APIs.
      2. Addition or removal of property keys.)
    -->
    No.
    
    ### How was this patch tested?
    
    Unit tests
    
    
    ---------
    
    Co-authored-by: Junfan Zhang <zh...@qiyi.com>
---
 .../impl/PooledHadoopShuffleWriteHandler.java      | 12 ++++---
 .../impl/PooledHadoopShuffleWriteHandlerTest.java  | 39 ++++++++++++++++++++++
 2 files changed, 46 insertions(+), 5 deletions(-)

diff --git a/storage/src/main/java/org/apache/uniffle/storage/handler/impl/PooledHadoopShuffleWriteHandler.java b/storage/src/main/java/org/apache/uniffle/storage/handler/impl/PooledHadoopShuffleWriteHandler.java
index f46ebfdb2..f801516bf 100644
--- a/storage/src/main/java/org/apache/uniffle/storage/handler/impl/PooledHadoopShuffleWriteHandler.java
+++ b/storage/src/main/java/org/apache/uniffle/storage/handler/impl/PooledHadoopShuffleWriteHandler.java
@@ -19,6 +19,7 @@ package org.apache.uniffle.storage.handler.impl;
 
 import java.util.Collection;
 import java.util.concurrent.LinkedBlockingDeque;
+import java.util.concurrent.atomic.AtomicInteger;
 import java.util.function.Function;
 
 import com.google.common.annotations.VisibleForTesting;
@@ -49,7 +50,7 @@ public class PooledHadoopShuffleWriteHandler implements ShuffleWriteHandler {
   private final int maxConcurrency;
   private final String basePath;
   private Function<Integer, ShuffleWriteHandler> createWriterFunc;
-  private volatile int initializedHandlerCnt = 0;
+  private AtomicInteger initializedHandlerCntRef = new AtomicInteger(0);
 
   // Only for tests
   @VisibleForTesting
@@ -109,10 +110,11 @@ public class PooledHadoopShuffleWriteHandler implements ShuffleWriteHandler {
 
   @Override
   public void write(Collection<ShufflePartitionedBlock> shuffleBlocks) throws Exception {
-    if (queue.isEmpty() && initializedHandlerCnt < maxConcurrency) {
+    if (queue.isEmpty() && initializedHandlerCntRef.get() < maxConcurrency) {
       synchronized (this) {
-        if (initializedHandlerCnt < maxConcurrency) {
-          queue.add(createWriterFunc.apply(initializedHandlerCnt++));
+        if (initializedHandlerCntRef.get() < maxConcurrency) {
+          queue.add(createWriterFunc.apply(initializedHandlerCntRef.get()));
+          initializedHandlerCntRef.addAndGet(1);
         }
       }
     }
@@ -132,6 +134,6 @@ public class PooledHadoopShuffleWriteHandler implements ShuffleWriteHandler {
 
   @VisibleForTesting
   protected int getInitializedHandlerCnt() {
-    return initializedHandlerCnt;
+    return initializedHandlerCntRef.get();
   }
 }
diff --git a/storage/src/test/java/org/apache/uniffle/storage/handler/impl/PooledHadoopShuffleWriteHandlerTest.java b/storage/src/test/java/org/apache/uniffle/storage/handler/impl/PooledHadoopShuffleWriteHandlerTest.java
index 32e8786c8..460741db4 100644
--- a/storage/src/test/java/org/apache/uniffle/storage/handler/impl/PooledHadoopShuffleWriteHandlerTest.java
+++ b/storage/src/test/java/org/apache/uniffle/storage/handler/impl/PooledHadoopShuffleWriteHandlerTest.java
@@ -34,6 +34,7 @@ import org.apache.uniffle.common.ShufflePartitionedBlock;
 import org.apache.uniffle.storage.handler.api.ShuffleWriteHandler;
 
 import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.fail;
 
 public class PooledHadoopShuffleWriteHandlerTest {
 
@@ -41,6 +42,7 @@ public class PooledHadoopShuffleWriteHandlerTest {
     private List<Integer> invokedList;
     private int index;
     private Runnable execution;
+    private boolean markInitializationFail = false;
 
     FakedShuffleWriteHandler(List<Integer> invokedList, int index, Runnable runnable) {
       this.invokedList = invokedList;
@@ -48,6 +50,12 @@ public class PooledHadoopShuffleWriteHandlerTest {
       this.execution = runnable;
     }
 
+    FakedShuffleWriteHandler(boolean isMarkInitializationFail) {
+      if (isMarkInitializationFail) {
+        throw new RuntimeException("Fail to init");
+      }
+    }
+
     FakedShuffleWriteHandler(
         List<Integer> initializedList, List<Integer> invokedList, int index, Runnable runnable) {
       initializedList.add(index);
@@ -63,6 +71,37 @@ public class PooledHadoopShuffleWriteHandlerTest {
     }
   }
 
+  @Test
+  public void initializationFailureTest() throws Exception {
+    int maxConcurrency = 2;
+    LinkedBlockingDeque<ShuffleWriteHandler> deque = new LinkedBlockingDeque<>(maxConcurrency);
+
+    PooledHadoopShuffleWriteHandler handler =
+        new PooledHadoopShuffleWriteHandler(
+            deque, maxConcurrency, index -> new FakedShuffleWriteHandler(true));
+
+    // to check the initialization
+    for (int i = 0; i < maxConcurrency; i++) {
+      try {
+        handler.write(Collections.emptyList());
+        fail();
+      } catch (Exception e) {
+        // ignore
+      }
+    }
+
+    // after initialization, the next writing will still fail due to the previous initialization
+    // fail.
+    for (int i = 0; i < maxConcurrency; i++) {
+      try {
+        handler.write(Collections.emptyList());
+        fail();
+      } catch (Exception e) {
+        // ignore
+      }
+    }
+  }
+
   @Test
   public void lazyInitializeWriterHandlerTest() throws Exception {
     int maxConcurrency = 5;


(uniffle) 07/11: [#2298] fix(server): followup to correct the metric of grpc_get_local_shuffle_index_process_latency (#2302)

Posted by ro...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

roryqi pushed a commit to branch branch-0.10.bak
in repository https://gitbox.apache.org/repos/asf/uniffle.git

commit b10a4bc8212c006c1289a38c713a4e76cea71f9b
Author: Junfan Zhang <zu...@apache.org>
AuthorDate: Tue Dec 24 11:54:45 2024 +0800

    [#2298] fix(server): followup to correct the metric of grpc_get_local_shuffle_index_process_latency (#2302)
    
    ### What changes were proposed in this pull request?
    
    followup to correct the metric of grpc_get_local_shuffle_index
    
    ### Why are the changes needed?
    
    Fix: #2298
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Co-authored-by: Junfan Zhang <zh...@qiyi.com>
---
 .../org/apache/uniffle/server/ShuffleServerGrpcService.java | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/server/src/main/java/org/apache/uniffle/server/ShuffleServerGrpcService.java b/server/src/main/java/org/apache/uniffle/server/ShuffleServerGrpcService.java
index d0b0213cd..34b586b2b 100644
--- a/server/src/main/java/org/apache/uniffle/server/ShuffleServerGrpcService.java
+++ b/server/src/main/java/org/apache/uniffle/server/ShuffleServerGrpcService.java
@@ -1294,6 +1294,12 @@ public class ShuffleServerGrpcService extends ShuffleServerImplBase {
           ShuffleServerMetrics.gaugeReadLocalIndexFileBufferSize.inc(assumedFileSize);
           GetLocalShuffleIndexResponse.Builder builder =
               GetLocalShuffleIndexResponse.newBuilder().setStatus(status.toProto()).setRetMsg(msg);
+          builder.setIndexData(UnsafeByteOperations.unsafeWrap(data));
+          builder.setDataFileLen(shuffleIndexResult.getDataFileLen());
+          builder.addAllStorageIds(
+              Arrays.stream(shuffleIndexResult.getStorageIds())
+                  .boxed()
+                  .collect(Collectors.toList()));
           long readTime = System.currentTimeMillis() - start;
           shuffleServer
               .getGrpcMetrics()
@@ -1303,13 +1309,6 @@ public class ShuffleServerGrpcService extends ShuffleServerImplBase {
               readTime,
               data.remaining(),
               requestInfo);
-
-          builder.setIndexData(UnsafeByteOperations.unsafeWrap(data));
-          builder.setDataFileLen(shuffleIndexResult.getDataFileLen());
-          builder.addAllStorageIds(
-              Arrays.stream(shuffleIndexResult.getStorageIds())
-                  .boxed()
-                  .collect(Collectors.toList()));
           auditContext.withReturnValue("len=" + shuffleIndexResult.getDataFileLen());
           reply = builder.build();
         } catch (FileNotFoundException indexFileNotFoundException) {


(uniffle) 11/11: [#2307] license: Add left jars with Apache license (#2310)

Posted by ro...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

roryqi pushed a commit to branch branch-0.10.bak
in repository https://gitbox.apache.org/repos/asf/uniffle.git

commit 3cab5a8a063490b411b527fe3df2342442373338
Author: roryqi <ro...@apache.org>
AuthorDate: Tue Dec 31 18:53:00 2024 +0800

    [#2307] license: Add left jars with Apache license (#2310)
    
    ### What changes were proposed in this pull request?
    
    Add left jars with Apache license
    
    ### Why are the changes needed?
    Fix #2307
    
    ### Does this PR introduce _any_ user-facing change?
    No.
    
    ### How was this patch tested?
    No need.
---
 LICENSE-binary | 102 +++++++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 74 insertions(+), 28 deletions(-)

diff --git a/LICENSE-binary b/LICENSE-binary
index b56a9e3bf..6a98acd40 100644
--- a/LICENSE-binary
+++ b/LICENSE-binary
@@ -205,51 +205,92 @@
 This project bundles some components that are also licensed under the Apache
 License Version 2.0:
 
-org.apache.commons:commons-lang3
-org.eclipse.jetty:jetty-server
-org.eclipse.jetty:jetty-servlet
-org.eclipse.jetty:jetty-util
-org.eclipse.jetty:jetty-http
-org.eclipse.jetty:jetty-io
-org.eclipse.jetty:jetty-security
-commons-cli:commons-cli
-org.apache.httpcomponents:httpclient
-net.jpountz.lz4:lz4
-com.google.guava:guava
-com.google.guava:listenablefuture
-com.google.guava:failureaccess
 com.fasterxml.jackson.core:jackson-annotations
 com.fasterxml.jackson.core:jackson-core
 com.fasterxml.jackson.core:jackson-databind
 com.fasterxml.jackson.module:jackson-module-scala_2.12
-info.picocli:picocli
+com.google.android:annotations
+com.google.api.grpc:proto-google-common-protos
+com.google.code.findbugs:jsr305
+com.google.code.gson:gson
 com.google.errorprone:error_prone_annotations
-io.netty:netty-all
-org.xerial.snappy:snappy-java
+com.google.guava:failureaccess
+com.google.guava:guava
+com.google.guava:listenablefuture
+com.google.j2objc:j2objc-annotations
+commons-cli:commons-cli
+commons-codec:commons-codec
+commons-collections:commons-collections
+commons-logging:commons-logging
+info.picocli:picocli
+io.grpc:grpc-api
+io.grpc:grpc-context
+io.grpc:grpc-core
 io.grpc:grpc-netty-shaded
 io.grpc:grpc-protobuf
-io.grpc:grpc-stub
-io.grpc:grpc-core
-io.grpc:grpc-context
-io.grpc:grpc-api
 io.grpc:grpc-protobuf-lite
+io.grpc:grpc-stub
+io.grpc:grpc-util
+io.netty:netty-all
+io.netty:netty-buffer
+io.netty:netty-codec
+io.netty:netty-codec-dns
+io.netty:netty-codec-haproxy
+io.netty:netty-codec-http
+io.netty:netty-codec-http2
+io.netty:netty-codec-memcache
+io.netty:netty-codec-mqtt
+io.netty:netty-codec-redis
+io.netty:netty-codec-smtp
+io.netty:netty-codec-socks
+io.netty:netty-codec-stomp
+io.netty:netty-codec-xml
+io.netty:netty-common
+io.netty:netty-handler
+io.netty:netty-handler-proxy
+io.netty:netty-handler-ssl-ocsp
+io.netty:netty-resolver
+io.netty:netty-resolver-dns
+io.netty:netty-resolver-dns-classes-macos
+io.netty:netty-resolver-dns-native-macos
+io.netty:netty-transport
+io.netty:netty-transport-classes-epoll
+io.netty:netty-transport-classes-kqueue
+io.netty:netty-transport-native-epoll
+io.netty:netty-transport-native-kqueue
+io.netty:netty-transport-native-unix-common
+io.netty:netty-transport-rxtx
+io.netty:netty-transport-sctp
+io.netty:netty-transport-udt
+io.perfmark:perfmark-api
 io.prometheus:simpleclient
+io.prometheus:simpleclient_common
 io.prometheus:simpleclient_hotspot
 io.prometheus:simpleclient_httpserver
 io.prometheus:simpleclient_jetty
 io.prometheus:simpleclient_pushgateway
 io.prometheus:simpleclient_servlet
+jakarta.validation:jakarta.validation-api
+javax.inject:javax.inject
+net.jpountz.lz4:lz4
+org.apache.commons:commons-lang3
+org.apache.httpcomponents:httpclient
+org.apache.httpcomponents:httpcore
+org.apache.logging.log4j:log4j-api
+org.apache.logging.log4j:log4j-core
+org.apache.logging.log4j:log4j-slf4j-impl
+org.eclipse.jetty:jetty-client
+org.eclipse.jetty:jetty-http
+org.eclipse.jetty:jetty-io
+org.eclipse.jetty:jetty-proxy
+org.eclipse.jetty:jetty-security
+org.eclipse.jetty:jetty-server
+org.eclipse.jetty:jetty-servlet
+org.eclipse.jetty:jetty-util
 org.roaringbitmap:RoaringBitmap
 org.roaringbitmap:shims
-javax.inject:javax.inject
-com.google.code.gson:gson
-com.google.j2objc:j2objc-annotations
-com.google.api.grpc:proto-google-common-protos
-org.javassist:javassist
-com.google.code.findbugs:jsr305
-jakarta.validation:jakarta.validation-api
+org.xerial.snappy:snappy-java
 org.yaml:snakeyaml
-org.eclipse.jetty:jetty-proxy
 
 -----------------------------------------------------------------------------------
 This product bundles various third-party components under other open source licenses.
@@ -303,3 +344,8 @@ org.apache.hbase.thirdparty:hbase-shaded-jersey
 The ISC License
 ---------------
 rimraf
+
+The Apache Software License, Version 2.0, LGPL License or MPL License
+--------------------------------------------------------------------
+org.javassist:javassist
+


(uniffle) 10/11: [MINOR] docs: Fix the license format of remote_merge_guide.md (#2314)

Posted by ro...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

roryqi pushed a commit to branch branch-0.10.bak
in repository https://gitbox.apache.org/repos/asf/uniffle.git

commit 401488705de569d07f316a8135b308f6d0586a25
Author: xianjingfeng <xi...@gmail.com>
AuthorDate: Mon Dec 30 22:02:10 2024 +0800

    [MINOR] docs: Fix the license format of remote_merge_guide.md (#2314)
    
    ### What changes were proposed in this pull request?
    Fix the license format of remote_merge_guide.md
    
    ### Why are the changes needed?
    If we open https://github.com/apache/incubator-uniffle/blob/master/docs/remote_merge_guide.md, we can see the following warning.
    
    Error in user YAML: (<unknown>): could not find expected ':' while scanning a simple key at line 6 column 1
    
    ### Does this PR introduce any user-facing change?
    No.
    
    ### How was this patch tested?
    Unnecessary
---
 docs/remote_merge_guide.md | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)

diff --git a/docs/remote_merge_guide.md b/docs/remote_merge_guide.md
index 3016625dd..0aaa08e7f 100644
--- a/docs/remote_merge_guide.md
+++ b/docs/remote_merge_guide.md
@@ -4,20 +4,20 @@ displayTitle: Uniffle Remote Merge Guide
 title: Uniffle Remote Merge Guide
 description: Uniffle Remote Merge Guide
 license: |
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements.  See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to You under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License.  You may obtain a copy of the License at
-
-      http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
 ---
 # remote merge guide