You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@celeborn.apache.org by "jiang13021 (via GitHub)" <gi...@apache.org> on 2024/10/24 08:16:13 UTC

[PR] [CELEBORN-1670] Avoid swallowing InterruptedException in ShuffleClientImpl [celeborn]

jiang13021 opened a new pull request, #2849:
URL: https://github.com/apache/celeborn/pull/2849

   <!--
   Thanks for sending a pull request!  Here are some tips for you:
     - Make sure the PR title start w/ a JIRA ticket, e.g. '[CELEBORN-XXXX] Your PR title ...'.
     - Be sure to keep the PR description updated to reflect all changes.
     - Please write your PR title to summarize what this PR proposes.
     - If possible, provide a concise example to reproduce the issue for a faster review.
   -->
   
   ### What changes were proposed in this pull request?
   As title.
   
   
   ### Why are the changes needed?
   In the ShuffleClientImpl, methods such as pushData and pushMergedData might encounter interruptions during message transmission via the TransportClient. However, the InterruptedException may be ignored, as it is handled as a standard exception. As a result, the ShuffleClientImpl continues its operation(retry or revive) even when an InterruptedException occurs.
   
   
   ### Does this PR introduce _any_ user-facing change?
   No
   
   
   ### How was this patch tested?
   Add UT: org.apache.celeborn.client.ShuffleClientSuiteJ#testPushDataAndInterrupted
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [CELEBORN-1670] Avoid swallowing InterruptedException in ShuffleClientImpl [celeborn]

Posted by "RexXiong (via GitHub)" <gi...@apache.org>.
RexXiong commented on code in PR #2849:
URL: https://github.com/apache/celeborn/pull/2849#discussion_r1825343269


##########
client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java:
##########
@@ -354,8 +354,12 @@ private void submitRetryPushData(
             batchId,
             newLoc,
             e);
-        pushDataRpcResponseCallback.onFailure(
-            new CelebornIOException(StatusCode.PUSH_DATA_CREATE_CONNECTION_FAIL_PRIMARY, e));
+        if (e instanceof InterruptedException) {
+          Thread.currentThread().interrupt();

Review Comment:
   > Contributor Author
   
   yes, better keep that behavior If there is no particular reason.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [CELEBORN-1670] Avoid swallowing InterruptedException in ShuffleClientImpl [celeborn]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #2849:
URL: https://github.com/apache/celeborn/pull/2849#issuecomment-2497262888

   This PR is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 10 days.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [CELEBORN-1670] Avoid swallowing InterruptedException in ShuffleClientImpl [celeborn]

Posted by "turboFei (via GitHub)" <gi...@apache.org>.
turboFei commented on PR #2849:
URL: https://github.com/apache/celeborn/pull/2849#issuecomment-2523821795

   reopen this PR, cc @jiang13021 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [CELEBORN-1670] Avoid swallowing InterruptedException in ShuffleClientImpl [celeborn]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #2849:
URL: https://github.com/apache/celeborn/pull/2849#issuecomment-2519620525

   This issue was closed because it has been staled for 10 days with no activity.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [CELEBORN-1670] Avoid swallowing InterruptedException in ShuffleClientImpl [celeborn]

Posted by "RexXiong (via GitHub)" <gi...@apache.org>.
RexXiong closed pull request #2849: [CELEBORN-1670] Avoid swallowing InterruptedException in ShuffleClientImpl
URL: https://github.com/apache/celeborn/pull/2849


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [CELEBORN-1670] Avoid swallowing InterruptedException in ShuffleClientImpl [celeborn]

Posted by "RexXiong (via GitHub)" <gi...@apache.org>.
RexXiong commented on PR #2849:
URL: https://github.com/apache/celeborn/pull/2849#issuecomment-2544471826

   Merge to main(V0.6.0)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [CELEBORN-1670] Avoid swallowing InterruptedException in ShuffleClientImpl [celeborn]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #2849: [CELEBORN-1670] Avoid swallowing InterruptedException in ShuffleClientImpl
URL: https://github.com/apache/celeborn/pull/2849


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [CELEBORN-1670] Avoid swallowing InterruptedException in ShuffleClientImpl [celeborn]

Posted by "jiang13021 (via GitHub)" <gi...@apache.org>.
jiang13021 commented on PR #2849:
URL: https://github.com/apache/celeborn/pull/2849#issuecomment-2440634459

   Could you please review this? ping @waitinfuture @RexXiong @FMX @SteNicholas 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [CELEBORN-1670] Avoid swallowing InterruptedException in ShuffleClientImpl [celeborn]

Posted by "jiang13021 (via GitHub)" <gi...@apache.org>.
jiang13021 commented on code in PR #2849:
URL: https://github.com/apache/celeborn/pull/2849#discussion_r1824353646


##########
client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java:
##########
@@ -354,8 +354,12 @@ private void submitRetryPushData(
             batchId,
             newLoc,
             e);
-        pushDataRpcResponseCallback.onFailure(
-            new CelebornIOException(StatusCode.PUSH_DATA_CREATE_CONNECTION_FAIL_PRIMARY, e));
+        if (e instanceof InterruptedException) {
+          Thread.currentThread().interrupt();

Review Comment:
   Are you suggesting that we can invoke callback.onFailure(InterruptedException e) to interrupt the current thread? That seems reasonable. I believe that ignoring InterruptedException is unsafe, especially when we catch it without any specific handling.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [CELEBORN-1670] Avoid swallowing InterruptedException in ShuffleClientImpl [celeborn]

Posted by "RexXiong (via GitHub)" <gi...@apache.org>.
RexXiong commented on code in PR #2849:
URL: https://github.com/apache/celeborn/pull/2849#discussion_r1824204229


##########
client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java:
##########
@@ -354,8 +354,12 @@ private void submitRetryPushData(
             batchId,
             newLoc,
             e);
-        pushDataRpcResponseCallback.onFailure(
-            new CelebornIOException(StatusCode.PUSH_DATA_CREATE_CONNECTION_FAIL_PRIMARY, e));
+        if (e instanceof InterruptedException) {
+          Thread.currentThread().interrupt();

Review Comment:
   Could we maintain only the thread's interrupt state while still invoking callback.onFailure? And I've noticed some inconsistencies in how InterruptedException is handled in the Celeborn client.  In some cases it throws an exception, while in other cases it is simply ignored.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org