You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@celeborn.apache.org by GitBox <gi...@apache.org> on 2022/11/23 11:07:02 UTC
[GitHub] [incubator-celeborn] zhongqiangczq opened a new pull request, #1001: [CELEBORN-11] ShuffleClient supports MapPartition shuffle write
zhongqiangczq opened a new pull request, #1001:
URL: https://github.com/apache/incubator-celeborn/pull/1001
# [BUG]/[FEATURE] title
### What changes were proposed in this pull request?
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
### What are the items that need reviewer attention?
### Related issues.
### Related pull requests.
### How was this patch tested?
/cc @related-reviewer
@RexXiong @FMX @waitinfuture
/assign @main-reviewer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] zhongqiangczq closed pull request #1001: [CELEBORN-11] ShuffleClient supports MapPartition shuffle write
Posted by GitBox <gi...@apache.org>.
zhongqiangczq closed pull request #1001: [CELEBORN-11] ShuffleClient supports MapPartition shuffle write
URL: https://github.com/apache/incubator-celeborn/pull/1001
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] leesf commented on a diff in pull request #1001: [CELEBORN-11] ShuffleClient supports MapPartition shuffle write
Posted by GitBox <gi...@apache.org>.
leesf commented on code in PR #1001:
URL: https://github.com/apache/incubator-celeborn/pull/1001#discussion_r1032854857
##########
client/src/main/java/org/apache/celeborn/client/ShuffleClient.java:
##########
@@ -172,4 +175,15 @@ public abstract RssInputStream readPartition(
public abstract boolean unregisterShuffle(String applicationId, int shuffleId, boolean isDriver);
public abstract void shutDown();
+
+ public abstract int pushDataByteBuf(
Review Comment:
please add some description to the method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] leesf commented on a diff in pull request #1001: [CELEBORN-11] ShuffleClient supports MapPartition shuffle write
Posted by GitBox <gi...@apache.org>.
leesf commented on code in PR #1001:
URL: https://github.com/apache/incubator-celeborn/pull/1001#discussion_r1033590855
##########
client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java:
##########
@@ -20,15 +20,14 @@
import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.*;
-import java.util.concurrent.Callable;
-import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.ExecutorService;
-import java.util.concurrent.TimeUnit;
+import java.util.concurrent.*;
Review Comment:
the optimized is determined by your idea import config, you should check it and keep align with the codebase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] zhongqiangczq commented on a diff in pull request #1001: [CELEBORN-11] ShuffleClient supports MapPartition shuffle write
Posted by GitBox <gi...@apache.org>.
zhongqiangczq commented on code in PR #1001:
URL: https://github.com/apache/incubator-celeborn/pull/1001#discussion_r1033057354
##########
client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java:
##########
@@ -20,15 +20,14 @@
import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.*;
-import java.util.concurrent.Callable;
-import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.ExecutorService;
-import java.util.concurrent.TimeUnit;
+import java.util.concurrent.*;
Review Comment:
it's optimized automaticly by code format
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] leesf commented on a diff in pull request #1001: [CELEBORN-11] ShuffleClient supports MapPartition shuffle write
Posted by GitBox <gi...@apache.org>.
leesf commented on code in PR #1001:
URL: https://github.com/apache/incubator-celeborn/pull/1001#discussion_r1032854842
##########
client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java:
##########
@@ -20,15 +20,14 @@
import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.*;
-import java.util.concurrent.Callable;
-import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.ExecutorService;
-import java.util.concurrent.TimeUnit;
+import java.util.concurrent.*;
Review Comment:
useless changes
##########
client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java:
##########
@@ -47,8 +46,7 @@
import org.apache.celeborn.common.network.client.RpcResponseCallback;
import org.apache.celeborn.common.network.client.TransportClient;
import org.apache.celeborn.common.network.client.TransportClientFactory;
-import org.apache.celeborn.common.network.protocol.PushData;
-import org.apache.celeborn.common.network.protocol.PushMergedData;
+import org.apache.celeborn.common.network.protocol.*;
Review Comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] leesf commented on a diff in pull request #1001: [CELEBORN-11] ShuffleClient supports MapPartition shuffle write
Posted by GitBox <gi...@apache.org>.
leesf commented on code in PR #1001:
URL: https://github.com/apache/incubator-celeborn/pull/1001#discussion_r1032854857
##########
client/src/main/java/org/apache/celeborn/client/ShuffleClient.java:
##########
@@ -172,4 +175,15 @@ public abstract RssInputStream readPartition(
public abstract boolean unregisterShuffle(String applicationId, int shuffleId, boolean isDriver);
public abstract void shutDown();
+
+ public abstract int pushDataByteBuf(
Review Comment:
it would be great if you would add some description to the method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] zhongqiangczq commented on a diff in pull request #1001: [CELEBORN-11] ShuffleClient supports MapPartition shuffle write
Posted by GitBox <gi...@apache.org>.
zhongqiangczq commented on code in PR #1001:
URL: https://github.com/apache/incubator-celeborn/pull/1001#discussion_r1033079270
##########
client/src/main/java/org/apache/celeborn/client/ShuffleClient.java:
##########
@@ -172,4 +175,15 @@ public abstract RssInputStream readPartition(
public abstract boolean unregisterShuffle(String applicationId, int shuffleId, boolean isDriver);
public abstract void shutDown();
+
+ public abstract int pushDataByteBuf(
Review Comment:
Tks. I have added descriptions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org