You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@geode.apache.org by "Bruce Schuchardt (JIRA)" <ji...@apache.org> on 2018/02/14 19:44:00 UTC

[jira] [Assigned] (GEODE-4675) CI failure (suspect strings): DistributedSystemDisconnectedException: This connection to a distributed system has been disconnected reported as fatal log message during shutdown

     [ https://issues.apache.org/jira/browse/GEODE-4675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bruce Schuchardt reassigned GEODE-4675:
---------------------------------------

    Assignee: Darrel Schneider

> CI failure (suspect strings): DistributedSystemDisconnectedException: This connection to a distributed system has been disconnected reported as fatal log message during shutdown
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: GEODE-4675
>                 URL: https://issues.apache.org/jira/browse/GEODE-4675
>             Project: Geode
>          Issue Type: Bug
>          Components: regions
>    Affects Versions: 1.5.0
>            Reporter: Shelley Lynn Hughes-Godfrey
>            Assignee: Darrel Schneider
>            Priority: Major
>
> This failure occurred during CI on geode:
> https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/DistributedTest/builds/140
> {noformat}
> org.apache.geode.internal.cache.wan.concurrent.ConcurrentParallelGatewaySenderOffHeapDUnitTest > testPartitionedParallelPropagationHA FAILED
>     java.lang.AssertionError: Suspicious strings were written to the log during this run.
>     Fix the strings or use IgnoredException.addIgnoredException to ignore.
>     -----------------------------------------------------------------------
>     Found suspect string in log4j at line 9339
>     [fatal 2018/02/13 21:12:48.099 UTC <DM-MemberEventInvoker> tid=891] Unexpected exception:
>     org.apache.geode.distributed.DistributedSystemDisconnectedException: This connection to a distributed system has been disconnected.
>     	at org.apache.geode.distributed.internal.InternalDistributedSystem.checkConnected(InternalDistributedSystem.java:911)
>     	at org.apache.geode.distributed.internal.InternalDistributedSystem.getDistributionManager(InternalDistributedSystem.java:1499)
>     	at org.apache.geode.internal.cache.AbstractRegion.getDistributionManager(AbstractRegion.java:1757)
>     	at org.apache.geode.distributed.internal.DistributionAdvisor.getDistributionManager(DistributionAdvisor.java:380)
>     	at org.apache.geode.distributed.internal.DistributionAdvisor.notifyListenersMemberRemoved(DistributionAdvisor.java:1225)
>     	at org.apache.geode.distributed.internal.DistributionAdvisor.basicRemoveId(DistributionAdvisor.java:897)
>     	at org.apache.geode.distributed.internal.DistributionAdvisor.doRemoveId(DistributionAdvisor.java:964)
>     	at org.apache.geode.distributed.internal.DistributionAdvisor.removeId(DistributionAdvisor.java:926)
>     	at org.apache.geode.internal.cache.CacheDistributionAdvisor.removeId(CacheDistributionAdvisor.java:1183)
>     	at org.apache.geode.internal.cache.partitioned.RegionAdvisor.removeId(RegionAdvisor.java:391)
>     	at org.apache.geode.distributed.internal.DistributionAdvisor$1.memberDeparted(DistributionAdvisor.java:232)
>     	at org.apache.geode.distributed.internal.ClusterDistributionManager$MemberDepartedEvent.handleEvent(ClusterDistributionManager.java:4198)
>     	at org.apache.geode.distributed.internal.ClusterDistributionManager$MemberEvent.handleEvent(ClusterDistributionManager.java:4127)
>     	at org.apache.geode.distributed.internal.ClusterDistributionManager$MemberEvent.handleEvent(ClusterDistributionManager.java:4116)
>     	at org.apache.geode.distributed.internal.ClusterDistributionManager.handleMemberEvent(ClusterDistributionManager.java:2218)
>     	at org.apache.geode.distributed.internal.ClusterDistributionManager.access$900(ClusterDistributionManager.java:109)
>     	at org.apache.geode.distributed.internal.ClusterDistributionManager$MemberEventInvoker.run(ClusterDistributionManager.java:2250)
>     	at java.lang.Thread.run(Thread.java:748)
>     -----------------------------------------------------------------------
> {noformat}
> According to the logs, this looks like it occurs during shutdown ... 
> {noformat}
> [vm1] [info 2018/02/13 21:12:48.075 UTC <RMI TCP Connection(5)-172.17.0.2> tid=30] Stopping membership services
> [vm0] [info 2018/02/13 21:12:48.077 UTC <Geode Failure Detection Server thread 0> tid=398] GMSHealthMonitor server thread exiting
> [vm0] [info 2018/02/13 21:12:48.078 UTC <RMI TCP Connection(5)-172.17.0.2> tid=30] GMSHealthMonitor serverSocketExecutor is terminated
> [vm3] [info 2018/02/13 21:12:48.079 UTC <unicast receiver,b85562d33221-49229> tid=896] received leave request from 172.17.0.2<ec><v0>:32771 for 172.17.0.2(180)<ec><v0>:32771
> [vm0] [info 2018/02/13 21:12:48.084 UTC <RMI TCP Connection(5)-172.17.0.2> tid=30] DistributionManager stopped in 121ms.
> [vm0] [info 2018/02/13 21:12:48.086 UTC <RMI TCP Connection(5)-172.17.0.2> tid=30] Marking DistributionManager 172.17.0.2(176)<ec><v0>:32770 as closed.
> [vm1] [info 2018/02/13 21:12:48.087 UTC <RMI TCP Connection(5)-172.17.0.2> tid=30] GMSHealthMonitor server socket is closed in stopServices().
> [vm0] [info 2018/02/13 21:12:48.089 UTC <RMI TCP Connection(5)-172.17.0.2> tid=30] Got result: null
> [vm0]  from org.apache.geode.internal.cache.wan.WANTestBase$$Lambda$45/1443408061.run with 0 args on object: org.apache.geode.internal.cache.wan.WANTestBase$$Lambda$45/1443408061@13fa96af (took 139 ms)
> [vm1] [info 2018/02/13 21:12:48.090 UTC <Geode Failure Detection Server thread 0> tid=402] GMSHealthMonitor server thread exiting
> [vm1] [info 2018/02/13 21:12:48.090 UTC <RMI TCP Connection(5)-172.17.0.2> tid=30] GMSHealthMonitor serverSocketExecutor is terminated
> [vm1] [info 2018/02/13 21:12:48.101 UTC <RMI TCP Connection(5)-172.17.0.2> tid=30] DistributionManager stopped in 135ms.
> [vm1] [info 2018/02/13 21:12:48.102 UTC <RMI TCP Connection(5)-172.17.0.2> tid=30] Marking DistributionManager 172.17.0.2(180)<ec><v0>:32771 as closed.
> [vm3] [info 2018/02/13 21:12:48.122 UTC <RMI TCP Connection(2)-172.17.0.2> tid=30] Shutting down DistributionManager 172.17.0.2(189)<v2>:32773.
> [vm1] [info 2018/02/13 21:12:48.115 UTC <RMI TCP Connection(5)-172.17.0.2> tid=30] Got result: null
> [vm1]  from org.apache.geode.internal.cache.wan.WANTestBase$$Lambda$45/1846294803.run with 0 args on object: org.apache.geode.internal.cache.wan.WANTestBase$$Lambda$45/1846294803@76a45c45 (took 161 ms)
> [vm3] [fatal 2018/02/13 21:12:48.099 UTC <DM-MemberEventInvoker> tid=891] Unexpected exception:
> [vm3] org.apache.geode.distributed.DistributedSystemDisconnectedException: This connection to a distributed system has been disconnected.
> [vm3]   at org.apache.geode.distributed.internal.InternalDistributedSystem.checkConnected(InternalDistributedSystem.java:911)
> [vm3]   at org.apache.geode.distributed.internal.InternalDistributedSystem.getDistributionManager(InternalDistributedSystem.java:1499)
> [vm3]   at org.apache.geode.internal.cache.AbstractRegion.getDistributionManager(AbstractRegion.java:1757)
> [vm3]   at org.apache.geode.distributed.internal.DistributionAdvisor.getDistributionManager(DistributionAdvisor.java:380)
> [vm3]   at org.apache.geode.distributed.internal.DistributionAdvisor.notifyListenersMemberRemoved(DistributionAdvisor.java:1225)
> [vm3]   at org.apache.geode.distributed.internal.DistributionAdvisor.basicRemoveId(DistributionAdvisor.java:897)
> [vm3]   at org.apache.geode.distributed.internal.DistributionAdvisor.doRemoveId(DistributionAdvisor.java:964)
> [vm3]   at org.apache.geode.distributed.internal.DistributionAdvisor.removeId(DistributionAdvisor.java:926)
> [vm3]   at org.apache.geode.internal.cache.CacheDistributionAdvisor.removeId(CacheDistributionAdvisor.java:1183)
> [vm3]   at org.apache.geode.internal.cache.partitioned.RegionAdvisor.removeId(RegionAdvisor.java:391)
> [vm3]   at org.apache.geode.distributed.internal.DistributionAdvisor$1.memberDeparted(DistributionAdvisor.java:232)
> [vm3]   at org.apache.geode.distributed.internal.ClusterDistributionManager$MemberDepartedEvent.handleEvent(ClusterDistributionManager.java:4198)
> [vm3]   at org.apache.geode.distributed.internal.ClusterDistributionManager$MemberEvent.handleEvent(ClusterDistributionManager.java:4127)
> [vm3]   at org.apache.geode.distributed.internal.ClusterDistributionManager$MemberEvent.handleEvent(ClusterDistributionManager.java:4116)
> [vm3]   at org.apache.geode.distributed.internal.ClusterDistributionManager.handleMemberEvent(ClusterDistributionManager.java:2218)
> [vm3]   at org.apache.geode.distributed.internal.ClusterDistributionManager.access$900(ClusterDistributionManager.java:109)
> [vm3]   at org.apache.geode.distributed.internal.ClusterDistributionManager$MemberEventInvoker.run(ClusterDistributionManager.java:2250)
> [vm3]   at java.lang.Thread.run(Thread.java:748)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)