You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@geode.apache.org by "Dan Smith (JIRA)" <ji...@apache.org> on 2018/10/18 20:02:00 UTC
[jira] [Commented] (GEODE-5816) ClusterStartupRule fails to launch
JMX manager (port already in use)
[ https://issues.apache.org/jira/browse/GEODE-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655815#comment-16655815 ]
Dan Smith commented on GEODE-5816:
----------------------------------
Looking in the logs for one of these runs, it looks like it actually picked the same port for the http service and the JMX port. That is bound to be a conflict:
{noformat}
[vm0] http-service-port=25305
[vm0] jmx-manager=true
[vm0] jmx-manager-hostname-for-clients=localhost
[vm0] jmx-manager-port=25305
{noformat}
These means the port supplier fix in GEODE-5622 is not actually working, since this test looks like it failed after those changes. We are still giving out the same port to two different services.
> ClusterStartupRule fails to launch JMX manager (port already in use)
> --------------------------------------------------------------------
>
> Key: GEODE-5816
> URL: https://issues.apache.org/jira/browse/GEODE-5816
> Project: Geode
> Issue Type: Bug
> Components: tests
> Reporter: Bill
> Priority: Major
> Labels: swat
>
> We see these related failures on a couple tests in RegionMembershipMBeanOverHttpDUnitTest from our recent mass-test-run.
> From looking at the stack traces though, we surmise that the problem occurs in the ClusterStartupRule _before_ the actual tests run. Since it occurs before the tests run, we think the problem lies outside the RegionMembershipMBeanOverHttpDUnitTest class.
> {noformat}
> 4 failures (99.600% success rate)org.apache.geode.management.internal.cli.commands.RegionMembershipMBeanOverHttpDUnitTest
> | .testAddRmNewMemberWithReplicatedRegionsAndSubregions: 1 failures (99.900% success rate)
> | | Failed build 24 at https://concourse.apachegeode-ci.info/teams/main/pipelines/mass-test-run/jobs/DistributedTest/builds/24
> | .testMultiplePartitionedRegions: 3 failures (99.700% success rate)
> | | Failed build 982 at https://concourse.apachegeode-ci.info/teams/main/pipelines/mass-test-run/jobs/DistributedTest/builds/982
> | | Failed build 256 at https://concourse.apachegeode-ci.info/teams/main/pipelines/mass-test-run/jobs/DistributedTest/builds/256
> {noformat}
> Here's a stack trace:
> {noformat}
> java.lang.AssertionError: Suspicious strings were written to the log during this run.
> Fix the strings or use IgnoredException.addIgnoredException to ignore.
> -----------------------------------------------------------------------
> Found suspect string in log4j at line 358
> [error 2018/10/01 06:28:30.869 UTC <RMI TCP Connection(1)-172.17.0.20> tid=0x20] Jmx manager could not be started because java.rmi.server.ExportException: Port already in use: 25305; nested exception is:
> java.net.BindException: Failed to create server socket on ffd50f3577c5/172.17.0.20[25305]
> org.apache.geode.management.ManagementException: java.rmi.server.ExportException: Port already in use: 25305; nested exception is:
> java.net.BindException: Failed to create server socket on ffd50f3577c5/172.17.0.20[25305]
> at org.apache.geode.management.internal.ManagementAgent.startAgent(ManagementAgent.java:162)
> at org.apache.geode.management.internal.SystemManagementService.startManager(SystemManagementService.java:435)
> at org.apache.geode.management.internal.beans.ManagementAdapter.handleCacheCreation(ManagementAdapter.java:173)
> at org.apache.geode.management.internal.beans.ManagementListener.handleEvent(ManagementListener.java:118)
> at org.apache.geode.distributed.internal.InternalDistributedSystem.notifyResourceEventListeners(InternalDistributedSystem.java:2201)
> at org.apache.geode.distributed.internal.InternalDistributedSystem.handleResourceEvent(InternalDistributedSystem.java:591)
> at org.apache.geode.internal.cache.GemFireCacheImpl.initialize(GemFireCacheImpl.java:1218)
> at org.apache.geode.internal.cache.GemFireCacheImpl.basicCreate(GemFireCacheImpl.java:793)
> at org.apache.geode.internal.cache.GemFireCacheImpl.create(GemFireCacheImpl.java:779)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:177)
> at org.apache.geode.cache.CacheFactory.create(CacheFactory.java:224)
> at org.apache.geode.distributed.internal.InternalLocator.startCache(InternalLocator.java:662)
> at org.apache.geode.distributed.internal.InternalLocator.startDistributedSystem(InternalLocator.java:649)
> at org.apache.geode.distributed.internal.InternalLocator.startLocator(InternalLocator.java:311)
> at org.apache.geode.distributed.Locator.startLocator(Locator.java:253)
> at org.apache.geode.distributed.Locator.startLocatorAndDS(Locator.java:140)
> at org.apache.geode.test.junit.rules.LocatorStarterRule.startLocator(LocatorStarterRule.java:87)
> at org.apache.geode.test.junit.rules.LocatorStarterRule.before(LocatorStarterRule.java:68)
> at org.apache.geode.test.dunit.rules.ClusterStartupRule.lambda$startLocatorVM$22d9b8a8$1(ClusterStartupRule.java:206)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at hydra.MethExecutor.executeObject(MethExecutor.java:244)
> at org.apache.geode.test.dunit.standalone.RemoteDUnitVM.executeMethodOnObject(RemoteDUnitVM.java:70)
> at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.rmi.server.ExportException: Port already in use: 25305; nested exception is:
> java.net.BindException: Failed to create server socket on ffd50f3577c5/172.17.0.20[25305]
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:346)
> at sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:254)
> at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
> at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
> at sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:236)
> at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:213)
> at sun.rmi.registry.RegistryImpl.<init>(RegistryImpl.java:173)
> at sun.rmi.registry.RegistryImpl.<init>(RegistryImpl.java:144)
> at java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:239)
> at org.apache.geode.management.internal.ManagementAgent.configureAndStart(ManagementAgent.java:456)
> at org.apache.geode.management.internal.ManagementAgent.startAgent(ManagementAgent.java:160)
> ... 40 more
> Caused by: java.net.BindException: Failed to create server socket on ffd50f3577c5/172.17.0.20[25305]
> at org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:758)
> at org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:715)
> at org.apache.geode.management.internal.ManagementAgent$GemFireRMIServerSocketFactory.createServerSocket(ManagementAgent.java:615)
> at sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666)
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:335)
> ... 50 more
> Caused by: java.net.BindException: Address already in use (Bind failed)
> at java.net.PlainSocketImpl.socketBind(Native Method)
> at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
> at java.net.ServerSocket.bind(ServerSocket.java:375)
> at org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:754)
> ... 54 more
> -----------------------------------------------------------------------
> Found suspect string in log4j at line 431
> [error 2018/10/01 06:28:30.872 UTC <RMI TCP Connection(1)-172.17.0.20> tid=0x20] org.apache.geode.management.ManagementException: java.rmi.server.ExportException: Port already in use: 25305; nested exception is:
> java.net.BindException: Failed to create server socket on ffd50f3577c5/172.17.0.20[25305]
> at org.junit.Assert.fail(Assert.java:88)
> at org.apache.geode.test.dunit.standalone.DUnitLauncher.closeAndCheckForSuspects(DUnitLauncher.java:411)
> at org.apache.geode.test.dunit.rules.ClusterStartupRule.after(ClusterStartupRule.java:172)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:50)
> at org.apache.geode.test.junit.rules.DescribedExternalResource$1.evaluate(DescribedExternalResource.java:40)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106)
> at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
> at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
> at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66)
> at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
> at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
> at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
> at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
> at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:117)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
> at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:155)
> at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:137)
> at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
> at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
> at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> The three stack traces are almost identical.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)