You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@geode.apache.org by "Bruce Schuchardt (JIRA)" <ji...@apache.org> on 2019/03/13 18:21:00 UTC
[jira] [Assigned] (GEODE-6522) If a member is deposed as membership
coordinator it never becomes one again
[ https://issues.apache.org/jira/browse/GEODE-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bruce Schuchardt reassigned GEODE-6522:
---------------------------------------
Assignee: Bruce Schuchardt
> If a member is deposed as membership coordinator it never becomes one again
> ---------------------------------------------------------------------------
>
> Key: GEODE-6522
> URL: https://issues.apache.org/jira/browse/GEODE-6522
> Project: Geode
> Issue Type: Bug
> Components: membership
> Reporter: Bruce Schuchardt
> Assignee: Bruce Schuchardt
> Priority: Major
>
> The fix for GEODE-870 introduced a field in GmsJoinLeave that causes the view creator to shut down if another member becomes coordinator. Unfortunately this new field is never reset after it's used, preventing the deposed node from ever becoming (and staying) coordinator. This causes a lot of churn in the node and can cause it to fail to be able to shut down.
> {noformat}
> [info 2019/03/13 08:38:24.959 PDT <Geode Membership View Creator> tid=0x147] finished waiting for responses to view preparation
> [info 2019/03/13 08:38:24.959 PDT <Geode Membership View Creator> tid=0x147] received new view: View[turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014|15] members: [turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002, turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead}, turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015, turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019, turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023] shutdown: [turtle(locatorgemfire_2_3_host1_17698:17698:locator)<ec><v0>:41000, turtle(locatorgemfire_2_4_host1_17715:17715:locator)<ec><v1>:41003, turtle(locatorgemfire_2_2_host1_17676:17676:locator)<ec><v1>:41005]
> old view is: View[turtle(locatorgemfire_2_3_host1_17698:17698:locator)<ec><v0>:41000|3] members: [turtle(locatorgemfire_2_3_host1_17698:17698:locator)<ec><v0>:41000, turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002, turtle(locatorgemfire_2_4_host1_17715:17715:locator)<ec><v1>:41003, turtle(locatorgemfire_2_2_host1_17676:17676:locator)<ec><v1>:41005, turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead}, turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015, turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019, turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023]
> [info 2019/03/13 08:38:24.974 PDT <Geode Membership View Creator> tid=0x147] Failure detection is now watching turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015; suspects are {turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002=View[turtle(locatorgemfire_2_3_host1_17698:17698:locator)<ec><v0>:41000|3] members: [turtle(locatorgemfire_2_3_host1_17698:17698:locator)<ec><v0>:41000, turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002, turtle(locatorgemfire_2_4_host1_17715:17715:locator)<ec><v1>:41003, turtle(locatorgemfire_2_2_host1_17676:17676:locator)<ec><v1>:41005, turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead}, turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015, turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019, turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023]}
> [info 2019/03/13 08:38:24.981 PDT <Geode Membership View Creator> tid=0x147] sending new view View[turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014|15] members: [turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002, turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead}, turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015, turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019, turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023] shutdown: [turtle(locatorgemfire_2_3_host1_17698:17698:locator)<ec><v0>:41000, turtle(locatorgemfire_2_4_host1_17715:17715:locator)<ec><v1>:41003, turtle(locatorgemfire_2_2_host1_17676:17676:locator)<ec><v1>:41005]
> [info 2019/03/13 08:38:24.981 PDT <Geode Membership View Creator> tid=0x147] BRUCE: setting shutdown flag in view creator
> java.lang.Exception: stack trace
> at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.setShutdownFlag(GMSJoinLeave.java:2247)
> at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.prepareAndSendView(GMSJoinLeave.java:2713)
> at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.sendInitialView(GMSJoinLeave.java:2220)
> at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.run(GMSJoinLeave.java:2299)
> [info 2019/03/13 08:38:24.982 PDT <Geode Membership View Creator> tid=0x147] View Creator thread is exiting
> [info 2019/03/13 08:38:24.982 PDT <Geode Membership View Creator> tid=0x147] BRUCE: setting shutdown flag in view creator
> java.lang.Exception: stack trace
> at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.setShutdownFlag(GMSJoinLeave.java:2247)
> at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.run(GMSJoinLeave.java:2379)
> [info 2019/03/13 08:38:26.416 PDT <vm_4_thr_8_bridge_2_1_host1_17023> tid=0x150] GemFireCache[id = 66144348; isClosing = true; isShutDownAll = false; created = Wed Mar 13 08:34:41 PDT 2019; server = false; copyOnRead = false; lockLease = 120; lockTimeout = 60]: Now closing.
> ...
> [info 2019/03/13 08:38:27.495 PDT <Geode Membership View Creator> tid=0x161] View Creator thread is starting
> [info 2019/03/13 08:38:27.507 PDT <Geode Membership View Creator> tid=0x161] preparing new view View[turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014|21] members: [turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002, turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead}, turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015, turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023] shutdown: [turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019]
> [info 2019/03/13 08:38:27.508 PDT <Geode Membership View Creator> tid=0x161] finished waiting for responses to view preparation
> [info 2019/03/13 08:38:27.508 PDT <Geode Membership View Creator> tid=0x161] received new view: View[turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014|21] members: [turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002, turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead}, turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015, turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023] shutdown: [turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019]
> old view is: View[turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014|15] members: [turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002, turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead}, turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015, turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019, turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023] shutdown: [turtle(locatorgemfire_2_3_host1_17698:17698:locator)<ec><v0>:41000, turtle(locatorgemfire_2_4_host1_17715:17715:locator)<ec><v1>:41003, turtle(locatorgemfire_2_2_host1_17676:17676:locator)<ec><v1>:41005]
> [info 2019/03/13 08:38:27.566 PDT <Geode Membership View Creator> tid=0x161] sending new view View[turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014|21] members: [turtle(locatorgemfire_2_1_host1_17653:17653:locator)<ec><v1>:41002, turtle(bridgegemfire_2_1_host1_17023:17023)<ec><v2>:41014{lead}, turtle(bridgegemfire_2_4_host1_17100:17100)<ec><v2>:41015, turtle(bridgegemfire_2_3_host1_17064:17064)<ec><v3>:41023] shutdown: [turtle(bridgegemfire_2_2_host1_17048:17048)<ec><v3>:41019]
> [info 2019/03/13 08:38:27.567 PDT <Geode Membership View Creator> tid=0x161] BRUCE: setting shutdown flag in view creator
> java.lang.Exception: stack trace
> at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.setShutdownFlag(GMSJoinLeave.java:2247)
> at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.prepareAndSendView(GMSJoinLeave.java:2713)
> at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.sendInitialView(GMSJoinLeave.java:2220)
> at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.run(GMSJoinLeave.java:2299)
> [info 2019/03/13 08:38:27.567 PDT <Geode Membership View Creator> tid=0x161] View Creator thread is exiting
> {noformat}
> etc.
> The field should be reset to false after it's used so that the next View Creator thread won't exit after sending its first view.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)