[GitHub] [spark] otterc commented on a diff in pull request #41071: [SPARK-43391][CORE] Idle connection should be kept when closeIdleConnection is disabled

["otterc (via GitHub)" <gi...@apache.org>]

otterc commented on code in PR #41071:
URL: https://github.com/apache/spark/pull/41071#discussion_r1194381617


##########
common/network-common/src/main/java/org/apache/spark/network/server/TransportChannelHandler.java:
##########
@@ -163,14 +163,11 @@ public void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exc
         if (e.state() == IdleState.ALL_IDLE && isActuallyOverdue) {
           if (responseHandler.hasOutstandingRequests()) {
             String address = getRemoteAddress(ctx.channel());
-            logger.error("Connection to {} has been quiet for {} ms while there are outstanding " +
-              "requests. Assuming connection is dead; please adjust" +
-              " spark.{}.io.connectionTimeout if this is wrong.",
-              address, requestTimeoutNs / 1000 / 1000, transportContext.getConf().getModuleName());
-            client.timeOut();
-            ctx.close();

Review Comment:
   
   I believe this change alters the semantics of the system. Previously, when a client had pending requests but had not received any response from the server within the last requestTimeoutNs, we inferred that the channel was inactive and closed it. The client would then initiate a new fetch request. If we do not close the channel, what would happen? Would the client wait indefinitely for the shuffle response?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org