You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@geode.apache.org by "Barry Oglesby (JIRA)" <ji...@apache.org> on 2018/03/09 17:47:00 UTC
[jira] [Commented] (GEODE-4815)
ParallelGatewaySenderOperation_2_DUnitTest
testParallelGatewaySender_SingleNode_UserPR_Destroy_SimultaneousPut_RecreateRegion
failed with suspect string
[ https://issues.apache.org/jira/browse/GEODE-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393270#comment-16393270 ]
Barry Oglesby commented on GEODE-4815:
--------------------------------------
This issue is happening during shutdown of the test. \{{WANTestBase preTearDown}} calls \{{cleanupVM}} in all members. This method closes the cache. In this test, the two locators shutdown first followed by the normal members.
Below are the relevant log messages. The most interesting ones are in vm2.
cleanupVM received:
{noformat}
[vm0] [info 2018/03/06 23:47:36.017 UTC <RMI TCP Connection(4)-172.17.0.5> tid=30] Received method: org.apache.geode.internal.cache.wan.WANTestBase$$Lambda$45/264636554.run with 0 args on object: org.apache.geode.internal.cache.wan.WANTestBase$$Lambda$45/264636554@192f6325
[vm1] [info 2018/03/06 23:47:36.017 UTC <RMI TCP Connection(3)-172.17.0.5> tid=30] Received method: org.apache.geode.internal.cache.wan.WANTestBase$$Lambda$46/1058934172.run with 0 args on object: org.apache.geode.internal.cache.wan.WANTestBase$$Lambda$46/1058934172@1aa836bf
[vm2] [info 2018/03/06 23:47:36.025 UTC <RMI TCP Connection(3)-172.17.0.5> tid=578] Received method: org.apache.geode.internal.cache.wan.WANTestBase$$Lambda$65/1655036390.run with 0 args on object: org.apache.geode.internal.cache.wan.WANTestBase$$Lambda$65/1655036390@19342e48
[vm4] [info 2018/03/06 23:47:36.166 UTC <RMI TCP Connection(3)-172.17.0.5> tid=511] Received method: org.apache.geode.internal.cache.wan.WANTestBase$$Lambda$75/1665157343.run with 0 args on object: org.apache.geode.internal.cache.wan.WANTestBase$$Lambda$75/1665157343@6c35c26e
{noformat}
DMs begin shutting down:
{noformat}
[vm0] [info 2018/03/06 23:47:36.024 UTC <RMI TCP Connection(4)-172.17.0.5> tid=30] Shutting down DistributionManager 172.17.0.5(177)<ec><v0>:32770.
[vm1] [info 2018/03/06 23:47:36.060 UTC <RMI TCP Connection(3)-172.17.0.5> tid=30] Shutting down DistributionManager 172.17.0.5(181)<ec><v0>:32771.
[vm2] [info 2018/03/06 23:47:36.372 UTC <RMI TCP Connection(3)-172.17.0.5> tid=578] Shutting down DistributionManager 172.17.0.5(185)<v1>:32773.
[vm4] [info 2018/03/06 23:47:37.390 UTC <RMI TCP Connection(3)-172.17.0.5> tid=511] Shutting down DistributionManager 172.17.0.5(417)<v1>:32772.
{noformat}
locators complete shutdown:
{noformat}
[vm1] [info 2018/03/06 23:47:36.237 UTC <RMI TCP Connection(3)-172.17.0.5> tid=30] DistributionManager stopped in 177ms.
[vm0] [info 2018/03/06 23:47:36.288 UTC <RMI TCP Connection(4)-172.17.0.5> tid=30] DistributionManager stopped in 263ms.
{noformat}
vm2 does suspect checking and logs the fatal:
{noformat}
[vm2] [info 2018/03/06 23:47:36.330 UTC <P2P message reader for 172.17.0.5(181)<ec><v0>:32771 shared unordered uid=10 port=41496> tid=727] Performing final check for suspect member 172.17.0.5(181)<ec><v0>:32771 reason=member unexpectedly shut down shared, unordered connection
[vm2] [info 2018/03/06 23:47:36.333 UTC <P2P message reader for 172.17.0.5(181)<ec><v0>:32771 shared unordered uid=10 port=41496> tid=727] Final check failed - requesting removal of suspect member 172.17.0.5(181)<ec><v0>:32771
[vm2] [info 2018/03/06 23:47:36.333 UTC <P2P message reader for 172.17.0.5(181)<ec><v0>:32771 shared unordered uid=10 port=41496> tid=727] This member is becoming the membership coordinator with address 172.17.0.5(185)<v1>:32773
[vm2] [info 2018/03/06 23:47:36.333 UTC <P2P message reader for 172.17.0.5(181)<ec><v0>:32771 shared unordered uid=10 port=41496> tid=727] ViewCreator starting on:172.17.0.5(185)<v1>:32773
[vm2] [info 2018/03/06 23:47:36.334 UTC <Geode Membership View Creator> tid=770] View Creator thread is starting
[vm2] [info 2018/03/06 23:47:36.344 UTC <Geode Membership View Creator> tid=770] 172.17.0.5(181)<ec><v0>:32771 had a weight of 15
[vm2] [warn 2018/03/06 23:47:36.345 UTC <Geode Membership View Creator> tid=770] total weight lost in this view change is 15 of 25. Quorum has been lost!
[vm2] [fatal 2018/03/06 23:47:36.347 UTC <Geode Membership View Creator> tid=770] Possible loss of quorum due to the loss of 1 cache processes: [172.17.0.5(181)<ec><v0>:32771]
{noformat}
normal members complete shutdown:
{noformat}
[vm2] [info 2018/03/06 23:47:36.485 UTC <RMI TCP Connection(3)-172.17.0.5> tid=578] DistributionManager stopped in 113ms.
[vm4] [info 2018/03/06 23:47:37.501 UTC <RMI TCP Connection(3)-172.17.0.5> tid=511] DistributionManager stopped in 111ms.
{noformat}
cleanupVM completed:
{noformat}
[vm0] [info 2018/03/06 23:47:36.292 UTC <RMI TCP Connection(4)-172.17.0.5> tid=30] Got result: null
[vm1] [info 2018/03/06 23:47:36.365 UTC <RMI TCP Connection(3)-172.17.0.5> tid=30] Got result: null
[vm2] [info 2018/03/06 23:47:36.486 UTC <RMI TCP Connection(3)-172.17.0.5> tid=578] Got result: null
[vm4] [info 2018/03/06 23:47:37.503 UTC <RMI TCP Connection(3)-172.17.0.5> tid=511] Got result: null
{noformat}
> ParallelGatewaySenderOperation_2_DUnitTest testParallelGatewaySender_SingleNode_UserPR_Destroy_SimultaneousPut_RecreateRegion failed with suspect string
> --------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: GEODE-4815
> URL: https://issues.apache.org/jira/browse/GEODE-4815
> Project: Geode
> Issue Type: Improvement
> Components: tests, wan
> Reporter: Barry Oglesby
> Priority: Major
>
> ParallelGatewaySenderOperation_2_DUnitTest testParallelGatewaySender_SingleNode_UserPR_Destroy_SimultaneousPut_RecreateRegion failed with suspect string
> {noformat}
> Found suspect string in log4j at line 2444
> [fatal 2018/03/06 23:47:36.347 UTC <Geode Membership View Creator> tid=770] Possible loss of quorum due to the loss of 1 cache processes: [172.17.0.5(181)<ec><v0>:32771]
> at org.junit.Assert.fail(Assert.java:88)
> at org.apache.geode.test.dunit.standalone.DUnitLauncher.closeAndCheckForSuspects(DUnitLauncher.java:403)
> at org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase.cleanupAllVms(JUnit4DistributedTestCase.java:561)
> at org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase.doTearDownDistributedTestCase(JUnit4DistributedTestCase.java:508)
> at org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase.tearDownDistributedTestCase(JUnit4DistributedTestCase.java:495)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)