You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@geode.apache.org by "Nabarun Nag (Jira)" <ji...@apache.org> on 2021/09/27 22:01:00 UTC
[jira] [Created] (GEODE-9644) ClusterConfigLocatorRestartDUnitTest
> serverRestartsAfterLocatorReconnects FAILED
Nabarun Nag created GEODE-9644:
----------------------------------
Summary: ClusterConfigLocatorRestartDUnitTest > serverRestartsAfterLocatorReconnects FAILED
Key: GEODE-9644
URL: https://issues.apache.org/jira/browse/GEODE-9644
Project: Geode
Issue Type: Bug
Components: membership, messaging
Reporter: Nabarun Nag
There is sequence of cascading exceptions that occur in the VMs and we need a more detailed investigation: Possible culprit may be the bind address in use exeception:
[*http://files.apachegeode-ci.info/builds/apache-develop-mass-test-run/1.15.0-build.0513/test-results/distributedTest/1632566050/*]
{noformat}
org.apache.geode.management.internal.configuration.ClusterConfigLocatorRestartDUnitTest > serverRestartsAfterLocatorReconnects FAILED
org.apache.geode.test.dunit.RMIException: While invoking org.apache.geode.management.internal.configuration.ClusterConfigLocatorRestartDUnitTest$$Lambda$163/1450009752.run in VM 0 running on Host heavy-lifter-2c03c48d-8a0c-58ae-bad6-31f64bb5400a.c.apachegeode-ci.internal with 5 VMs
at org.apache.geode.test.dunit.VM.executeMethodOnObject(VM.java:631)
at org.apache.geode.test.dunit.VM.invoke(VM.java:448)
at org.apache.geode.test.junit.rules.VMProvider.invoke(VMProvider.java:94)
at org.apache.geode.management.internal.configuration.ClusterConfigLocatorRestartDUnitTest.waitForLocatorToReconnect(ClusterConfigLocatorRestartDUnitTest.java:225)
at org.apache.geode.management.internal.configuration.ClusterConfigLocatorRestartDUnitTest.serverRestartsAfterLocatorReconnects(ClusterConfigLocatorRestartDUnitTest.java:90)
Caused by:
org.awaitility.core.ConditionTimeoutException: Condition with org.apache.geode.management.internal.configuration.ClusterConfigLocatorRestartDUnitTest was not fulfilled within 5 minutes.
at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:166)
at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
at org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:939)
at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:908)
at org.apache.geode.management.internal.configuration.ClusterConfigLocatorRestartDUnitTest.lambda$waitForLocatorToReconnect$f182e747$2(ClusterConfigLocatorRestartDUnitTest.java:226)
8300 tests completed, 1 failed, 413 skipped{noformat}
VM2 mentions that cluster membership has failed.
{noformat}
[vm2] [info 2021/09/25 09:36:44.487 UTC server-2 <DisconnectThread> tid=0x153] cluster membership failed due to
[vm2] org.apache.geode.distributed.internal.membership.api.MemberDisconnectedException: for testing
[vm2] at org.apache.geode.distributed.internal.membership.gms.GMSMembership$ManagerImpl.forceDisconnect(GMSMembership.java:1787)
[vm2] at org.apache.geode.distributed.internal.membership.api.MembershipManagerHelper.crashDistributedSystem(MembershipManagerHelper.java:139)
[vm2] at org.apache.geode.test.junit.rules.MemberStarterRule.forceDisconnectMember(MemberStarterRule.java:568)
[vm2] at org.apache.geode.test.dunit.rules.MemberVM.lambda$forceDisconnect$bb17a952$1(MemberVM.java:90)
[vm2] at org.apache.geode.test.dunit.internal.IdentifiableRunnable.run(IdentifiableRunnable.java:41)
[vm2] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[vm2] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[vm2] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[vm2] at java.lang.reflect.Method.invoke(Method.java:498)
[vm2] at org.apache.geode.test.dunit.internal.MethodInvoker.executeObject(MethodInvoker.java:123)
[vm2] at org.apache.geode.test.dunit.internal.RemoteDUnitVM.executeMethodOnObject(RemoteDUnitVM.java:78)
[vm2] at sun.reflect.GeneratedMethodAccessor55.invoke(Unknown Source)
[vm2] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[vm2] at java.lang.reflect.Method.invoke(Method.java:498)
[vm2] at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
[vm2] at sun.rmi.transport.Transport$1.run(Transport.java:200)
[vm2] at sun.rmi.transport.Transport$1.run(Transport.java:197)
[vm2] at java.security.AccessController.doPrivileged(Native Method)
[vm2] at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
[vm2] at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
[vm2] at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
[vm2] at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
[vm2] at java.security.AccessController.doPrivileged(Native Method)
[vm2] at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
[vm2] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[vm2] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[vm2] at java.lang.Thread.run(Thread.java:748){noformat}
VM0 hits address already in use.
{noformat}
[vm0] [info 2021/09/25 09:36:53.762 UTC locator-0 <Location services restart thread> tid=0x60] attempt to restart location services terminated
[vm0] java.net.BindException: Failed to create server socket on 0.0.0.0/0.0.0.0[43091]
[vm0] at org.apache.geode.distributed.internal.tcpserver.ClusterSocketCreatorImpl.createServerSocket(ClusterSocketCreatorImpl.java:75)
[vm0] at org.apache.geode.internal.net.SCClusterSocketCreator.createServerSocket(SCClusterSocketCreator.java:55)
[vm0] at org.apache.geode.distributed.internal.tcpserver.ClusterSocketCreatorImpl.createServerSocket(ClusterSocketCreatorImpl.java:54)
[vm0] at org.apache.geode.distributed.internal.tcpserver.TcpServer.initializeServerSocket(TcpServer.java:196)
[vm0] at org.apache.geode.distributed.internal.tcpserver.TcpServer.startServerThread(TcpServer.java:183)
[vm0] at org.apache.geode.distributed.internal.tcpserver.TcpServer.restarting(TcpServer.java:161)
[vm0] at org.apache.geode.distributed.internal.membership.gms.locator.MembershipLocatorImpl.restarting(MembershipLocatorImpl.java:144)
[vm0] at org.apache.geode.distributed.internal.InternalLocator.restartWithoutSystem(InternalLocator.java:1200)
[vm0] at org.apache.geode.distributed.internal.InternalLocator.attemptReconnect(InternalLocator.java:1144)
[vm0] at org.apache.geode.distributed.internal.InternalLocator.lambda$launchRestartThread$4(InternalLocator.java:1077)
[vm0] at java.lang.Thread.run(Thread.java:748)
[vm0] Caused by: java.net.BindException: Address already in use (Bind failed)
[vm0] at java.net.PlainSocketImpl.socketBind(Native Method)
[vm0] at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
[vm0] at java.net.ServerSocket.bind(ServerSocket.java:390)
[vm0] at org.apache.geode.distributed.internal.tcpserver.ClusterSocketCreatorImpl.createServerSocket(ClusterSocketCreatorImpl.java:72)
[vm0] ... 10 more{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)