You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@celeborn.apache.org by "mridulm (via GitHub)" <gi...@apache.org> on 2024/07/25 19:52:45 UTC

[PR] [CELEBORN-1524] Support IPv6 hostnames for Apache Ratis [celeborn]

mridulm opened a new pull request, #2646:
URL: https://github.com/apache/celeborn/pull/2646

   ### What changes were proposed in this pull request?
   
   Workaround a Apache Ratis bug in Celeborn until a new Ratis release with the fix is released which we can use.
   
   
   ### Why are the changes needed?
   
   [RATIS-2131](https://issues.apache.org/jira/browse/RATIS-2131) has been fixed, and will be available in 3.2.0 - until it is released, this will work around the issue.
   
   
   ### Does this PR introduce _any_ user-facing change?
   
   Fixes a bug
   
   
   ### How was this patch tested?
   
   Manual testing in IPv6 env with hostnames for ratis config.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [CELEBORN-1524] Support IPv6 hostnames for Apache Ratis [celeborn]

Posted by "mridulm (via GitHub)" <gi...@apache.org>.
mridulm commented on PR #2646:
URL: https://github.com/apache/celeborn/pull/2646#issuecomment-2253164372

   Thanks @pan3793, @RexXiong !


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [CELEBORN-1524] Support IPv6 hostnames for Apache Ratis [celeborn]

Posted by "RexXiong (via GitHub)" <gi...@apache.org>.
RexXiong commented on code in PR #2646:
URL: https://github.com/apache/celeborn/pull/2646#discussion_r1694418109


##########
master/src/main/scala/org/apache/celeborn/service/deploy/master/clustermeta/ha/MasterNode.scala:
##########
@@ -41,15 +41,9 @@ case class MasterNode(
 
   def rpcEndpoint: String = rpcHost + ":" + rpcPort
 
-  def rpcIpEndpoint: String = rpcAddr.getAddress.getHostAddress + ":" + rpcPort
-
   def internalRpcEndpoint: String = rpcHost + ":" + internalRpcPort
 
-  def internalRpcIpEndpoint: String = rpcAddr.getAddress.getHostAddress + ":" + rpcPort
-
   lazy val ratisAddr = MasterNode.createSocketAddr(ratisHost, ratisPort)
-
-  lazy val rpcAddr = MasterNode.createSocketAddr(rpcHost, rpcPort)

Review Comment:
   Thanks for cleaning unused code!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [CELEBORN-1524] Support IPv6 hostnames for Apache Ratis [celeborn]

Posted by "RexXiong (via GitHub)" <gi...@apache.org>.
RexXiong closed pull request #2646: [CELEBORN-1524] Support IPv6 hostnames for Apache Ratis
URL: https://github.com/apache/celeborn/pull/2646


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [CELEBORN-1524] Support IPv6 hostnames for Apache Ratis [celeborn]

Posted by "mridulm (via GitHub)" <gi...@apache.org>.
mridulm commented on code in PR #2646:
URL: https://github.com/apache/celeborn/pull/2646#discussion_r1692041430


##########
master/src/main/scala/org/apache/celeborn/service/deploy/master/clustermeta/ha/MasterNode.scala:
##########
@@ -41,15 +41,9 @@ case class MasterNode(
 
   def rpcEndpoint: String = rpcHost + ":" + rpcPort
 
-  def rpcIpEndpoint: String = rpcAddr.getAddress.getHostAddress + ":" + rpcPort
-
   def internalRpcEndpoint: String = rpcHost + ":" + internalRpcPort
 
-  def internalRpcIpEndpoint: String = rpcAddr.getAddress.getHostAddress + ":" + rpcPort
-
   lazy val ratisAddr = MasterNode.createSocketAddr(ratisHost, ratisPort)
-
-  lazy val rpcAddr = MasterNode.createSocketAddr(rpcHost, rpcPort)

Review Comment:
   These are not used, and I cleaned them up as well.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [CELEBORN-1524] Support IPv6 hostnames for Apache Ratis [celeborn]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #2646:
URL: https://github.com/apache/celeborn/pull/2646#discussion_r1692446156


##########
master/src/main/scala/org/apache/celeborn/service/deploy/master/clustermeta/ha/MasterNode.scala:
##########
@@ -41,15 +41,9 @@ case class MasterNode(
 
   def rpcEndpoint: String = rpcHost + ":" + rpcPort
 
-  def rpcIpEndpoint: String = rpcAddr.getAddress.getHostAddress + ":" + rpcPort
-
   def internalRpcEndpoint: String = rpcHost + ":" + internalRpcPort
 
-  def internalRpcIpEndpoint: String = rpcAddr.getAddress.getHostAddress + ":" + rpcPort
-
   lazy val ratisAddr = MasterNode.createSocketAddr(ratisHost, ratisPort)
-
-  lazy val rpcAddr = MasterNode.createSocketAddr(rpcHost, rpcPort)

Review Comment:
   @RexXiong this is added in https://github.com/apache/celeborn/commit/993d3f2587b1c9cd8acd1a593a186d8044487c14, seems was never used



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [CELEBORN-1524] Support IPv6 hostnames for Apache Ratis [celeborn]

Posted by "mridulm (via GitHub)" <gi...@apache.org>.
mridulm commented on PR #2646:
URL: https://github.com/apache/celeborn/pull/2646#issuecomment-2251464677

   The test failure is unrelated to this PR:
   ```
   DeviceMonitorSuite:
   - init
   - register/unregister/notify/report
   0worker-device-checker-1
   24/07/25 20:09:13,757 ERROR [worker-disk-checker] LocalDeviceMonitor: /mnt high_disk_usage error, notify observers
   24/07/25 20:09:13,982 ERROR [ScalaTest-main-running-DeviceMonitorSuite] Utils: TimeoutException in thread ScalaTest-main-running-DeviceMonitorSuite, error message: none
   1worker-device-checker-3
   24/07/25 20:09:14,984 ERROR [ScalaTest-main-running-DeviceMonitorSuite] Utils: TimeoutException in thread ScalaTest-main-running-DeviceMonitorSuite, error message: none
   2worker-device-checker-4
   24/07/25 20:09:15,985 ERROR [ScalaTest-main-running-DeviceMonitorSuite] Utils: TimeoutException in thread ScalaTest-main-running-DeviceMonitorSuite, error message: none
   - tryWithTimeoutAndCallback
   - monitor non-critical error metrics *** FAILED ***
     java.lang.AssertionError: expected:<2> but was:<1>
     at org.junit.Assert.fail(Assert.java:89)
     at org.junit.Assert.failNotEquals(Assert.java:835)
     at org.junit.Assert.assertEquals(Assert.java:120)
     at org.junit.Assert.assertEquals(Assert.java:146)
     at org.apache.celeborn.service.deploy.worker.storage.DeviceMonitorSuite.$anonfun$new$22(DeviceMonitorSuite.scala:405)
     at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
     at org.mockito.MockitoEnhancer.withObject(MockitoAPI.scala:632)
     at org.mockito.MockitoEnhancer.withObject$(MockitoAPI.scala:620)
     at org.mockito.MockitoSugar$.withObject(MockitoSugar.scala:8)
     at org.mockito.MockitoEnhancer.withObjectMocked(MockitoAPI.scala:607)
     ...
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [CELEBORN-1524] Support IPv6 hostnames for Apache Ratis [celeborn]

Posted by "mridulm (via GitHub)" <gi...@apache.org>.
mridulm commented on PR #2646:
URL: https://github.com/apache/celeborn/pull/2646#issuecomment-2251301922

   +CC @RexXiong, @waitinfuture 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org