You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@celeborn.apache.org by "SteNicholas (via GitHub)" <gi...@apache.org> on 2026/02/06 02:57:37 UTC

[PR] [CELEBORN-2258] Bump Netty version from 4.1.118.Final to 4.2.9.Final [celeborn]

SteNicholas opened a new pull request, #3596:
URL: https://github.com/apache/celeborn/pull/3596

   ### What changes were proposed in this pull request?
   
   Bump Netty version from 4.1.118.Final to 4.2.9.Final.
   
   ### Why are the changes needed?
   
   The Netty 4.2.9.Final version has been released, which netty version is 4.1.118.Final at present.
   
   Backport:
   
   - https://github.com/apache/spark/pull/34881
   - https://github.com/apache/spark/pull/53499
   
   ### Does this PR resolve a correctness bug?
   
   No.
   
   ### Does this PR introduce _any_ user-facing change?
   
   No.
   
   ### How was this patch tested?
   
   CI.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [CELEBORN-2258] Bump Netty version from 4.1.118.Final to 4.2.10.Final [celeborn]

Posted by "eolivelli (via GitHub)" <gi...@apache.org>.
eolivelli commented on code in PR #3596:
URL: https://github.com/apache/celeborn/pull/3596#discussion_r2794191933


##########
common/src/test/java/org/apache/celeborn/common/network/TransportClientFactorySuiteJ.java:
##########
@@ -241,6 +241,7 @@ public void unlimitedConnectionAndCreationTimeouts() throws IOException, Interru
       TransportServer server = ctx.createServer();
       int unreachablePort = server.getPort();
       JavaUtils.closeQuietly(server);
+      Thread.sleep(1000);

Review Comment:
   why do we need this sleep ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [CELEBORN-2258] Bump Netty version from 4.1.118.Final to 4.2.10.Final [celeborn]

Posted by "SteNicholas (via GitHub)" <gi...@apache.org>.
SteNicholas commented on PR #3596:
URL: https://github.com/apache/celeborn/pull/3596#issuecomment-3906931990

   Ping @dongjoon-hyun, @pan3793.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [CELEBORN-2258] Bump Netty version from 4.1.118.Final to 4.2.10.Final [celeborn]

Posted by "SteNicholas (via GitHub)" <gi...@apache.org>.
SteNicholas commented on code in PR #3596:
URL: https://github.com/apache/celeborn/pull/3596#discussion_r2811060014


##########
common/src/main/java/org/apache/celeborn/common/network/util/TransportConf.java:
##########
@@ -59,12 +59,12 @@ public boolean preferDirectBufs() {
 
   /** Connect timeout in milliseconds. Default 10 secs. */
   public int connectTimeoutMs() {
-    return celebornConf.networkIoConnectTimeoutMs(module);
+    return Math.max(celebornConf.networkIoConnectTimeoutMs(module), 0);

Review Comment:
   Backport: https://github.com/apache/spark/pull/41785. The `TransportClientFactorySuiteJ#unlimitedConnectionAndCreationTimeouts` throws the following exception without this change:
   ```
   java.io.IOException: Failed to connect to /172.16.240.150:49777
   
     at org.apache.celeborn.common.network.client.TransportClientFactory.internalCreateClient(TransportClientFactory.java:311)
     at org.apache.celeborn.common.network.client.TransportClientFactory.createClient(TransportClientFactory.java:254)
     at org.apache.celeborn.common.network.client.TransportClientFactory.retryCreateClient(TransportClientFactory.java:161)
     at org.apache.celeborn.common.network.client.TransportClientFactory.createClient(TransportClientFactory.java:149)
     at org.apache.celeborn.common.network.client.TransportClientFactory.createClient(TransportClientFactory.java:261)
     at org.apache.celeborn.common.network.TransportClientFactorySuiteJ.unlimitedConnectionAndCreationTimeouts(TransportClientFactorySuiteJ.java:232)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:498)
     at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
     at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
     at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
     at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
     at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
     at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
     at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
     at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
     at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
     at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
     at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
     at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
     at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
     at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
     at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
     at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
     at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
     at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
     at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
     at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
     at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
     at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
     at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
     at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
     at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)
   Caused by: java.lang.IllegalArgumentException: connectTimeoutMillis : -1 (expected: >= 0)
     at io.netty.util.internal.ObjectUtil.checkPositiveOrZero(ObjectUtil.java:156)
     at io.netty.channel.DefaultChannelConfig.setConnectTimeoutMillis(DefaultChannelConfig.java:216)
     at io.netty.channel.kqueue.KQueueChannelConfig.setConnectTimeoutMillis(KQueueChannelConfig.java:129)
     at io.netty.channel.kqueue.KQueueSocketChannelConfig.setConnectTimeoutMillis(KQueueSocketChannelConfig.java:339)
     at io.netty.channel.kqueue.KQueueSocketChannelConfig.setConnectTimeoutMillis(KQueueSocketChannelConfig.java:41)
     at io.netty.channel.DefaultChannelConfig.setOption(DefaultChannelConfig.java:173)
     at io.netty.channel.kqueue.KQueueChannelConfig.setOption(KQueueChannelConfig.java:98)
     at io.netty.channel.kqueue.KQueueSocketChannelConfig.setOption(KQueueSocketChannelConfig.java:127)
     at io.netty.bootstrap.AbstractBootstrap.setChannelOption(AbstractBootstrap.java:488)
     at io.netty.bootstrap.AbstractBootstrap.setChannelOptions(AbstractBootstrap.java:480)
     at io.netty.bootstrap.Bootstrap.init(Bootstrap.java:272)
     at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:328)
     at io.netty.bootstrap.Bootstrap.doResolveAndConnect(Bootstrap.java:164)
     at io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:148)
     at org.apache.celeborn.common.network.client.TransportClientFactory.internalCreateClient(TransportClientFactory.java:304)
     ... 34 more
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [CELEBORN-2258] Bump Netty version from 4.1.118.Final to 4.2.10.Final [celeborn]

Posted by "SteNicholas (via GitHub)" <gi...@apache.org>.
SteNicholas commented on code in PR #3596:
URL: https://github.com/apache/celeborn/pull/3596#discussion_r2811060014


##########
common/src/main/java/org/apache/celeborn/common/network/util/TransportConf.java:
##########
@@ -59,12 +59,12 @@ public boolean preferDirectBufs() {
 
   /** Connect timeout in milliseconds. Default 10 secs. */
   public int connectTimeoutMs() {
-    return celebornConf.networkIoConnectTimeoutMs(module);
+    return Math.max(celebornConf.networkIoConnectTimeoutMs(module), 0);

Review Comment:
   Backport: https://github.com/apache/spark/pull/41785.
   The `TransportClientFactorySuiteJ#unlimitedConnectionAndCreationTimeouts` throws the following exception without this change:
   ```
   java.io.IOException: Failed to connect to /172.16.240.150:49777
   
     at org.apache.celeborn.common.network.client.TransportClientFactory.internalCreateClient(TransportClientFactory.java:311)
     at org.apache.celeborn.common.network.client.TransportClientFactory.createClient(TransportClientFactory.java:254)
     at org.apache.celeborn.common.network.client.TransportClientFactory.retryCreateClient(TransportClientFactory.java:161)
     at org.apache.celeborn.common.network.client.TransportClientFactory.createClient(TransportClientFactory.java:149)
     at org.apache.celeborn.common.network.client.TransportClientFactory.createClient(TransportClientFactory.java:261)
     at org.apache.celeborn.common.network.TransportClientFactorySuiteJ.unlimitedConnectionAndCreationTimeouts(TransportClientFactorySuiteJ.java:232)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:498)
     at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
     at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
     at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
     at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
     at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
     at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
     at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
     at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
     at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
     at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
     at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
     at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
     at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
     at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
     at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
     at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
     at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
     at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
     at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
     at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
     at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
     at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
     at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
     at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
     at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)
   Caused by: java.lang.IllegalArgumentException: connectTimeoutMillis : -1 (expected: >= 0)
     at io.netty.util.internal.ObjectUtil.checkPositiveOrZero(ObjectUtil.java:156)
     at io.netty.channel.DefaultChannelConfig.setConnectTimeoutMillis(DefaultChannelConfig.java:216)
     at io.netty.channel.kqueue.KQueueChannelConfig.setConnectTimeoutMillis(KQueueChannelConfig.java:129)
     at io.netty.channel.kqueue.KQueueSocketChannelConfig.setConnectTimeoutMillis(KQueueSocketChannelConfig.java:339)
     at io.netty.channel.kqueue.KQueueSocketChannelConfig.setConnectTimeoutMillis(KQueueSocketChannelConfig.java:41)
     at io.netty.channel.DefaultChannelConfig.setOption(DefaultChannelConfig.java:173)
     at io.netty.channel.kqueue.KQueueChannelConfig.setOption(KQueueChannelConfig.java:98)
     at io.netty.channel.kqueue.KQueueSocketChannelConfig.setOption(KQueueSocketChannelConfig.java:127)
     at io.netty.bootstrap.AbstractBootstrap.setChannelOption(AbstractBootstrap.java:488)
     at io.netty.bootstrap.AbstractBootstrap.setChannelOptions(AbstractBootstrap.java:480)
     at io.netty.bootstrap.Bootstrap.init(Bootstrap.java:272)
     at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:328)
     at io.netty.bootstrap.Bootstrap.doResolveAndConnect(Bootstrap.java:164)
     at io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:148)
     at org.apache.celeborn.common.network.client.TransportClientFactory.internalCreateClient(TransportClientFactory.java:304)
     ... 34 more
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [CELEBORN-2258] Bump Netty version from 4.1.118.Final to 4.2.10.Final [celeborn]

Posted by "SteNicholas (via GitHub)" <gi...@apache.org>.
SteNicholas commented on code in PR #3596:
URL: https://github.com/apache/celeborn/pull/3596#discussion_r2796505328


##########
common/src/test/java/org/apache/celeborn/common/network/TransportClientFactorySuiteJ.java:
##########
@@ -241,6 +241,7 @@ public void unlimitedConnectionAndCreationTimeouts() throws IOException, Interru
       TransportServer server = ctx.createServer();
       int unreachablePort = server.getPort();
       JavaUtils.closeQuietly(server);
+      Thread.sleep(1000);

Review Comment:
   @eolivelli, you could refer to https://github.com/apache/spark/pull/52552/changes#r2427655105 to verify the reason why add `sleep`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org