You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@geode.apache.org by GitBox <gi...@apache.org> on 2022/04/15 23:53:20 UTC
[GitHub] [geode] pivotal-eshu opened a new pull request, #7596: GEODE-10242: Generate tail key after all child regions become primary
pivotal-eshu opened a new pull request, #7596:
URL: https://github.com/apache/geode/pull/7596
* When an event occurred on a bucket just become primary, it will not
generate tail key for the event until all the colocated child buckets
become primary as well. So that tail key will always be generated on the
same server.
<!-- Thank you for submitting a contribution to Apache Geode. -->
<!-- In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
-->
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?
- [ ] Has your PR been rebased against the latest commit within the target branch (typically `develop`)?
- [ ] Is your initial contribution a single, squashed commit?
- [ ] Does `gradlew build` run cleanly?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
<!-- Note:
Please ensure that once the PR is submitted, check Concourse for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to dev@geode.apache.org.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-eshu commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-eshu commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r855677359
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegion.java:
##########
@@ -629,6 +642,76 @@ public void handleWANEvent(EntryEventImpl event) {
}
}
+ boolean needToWaitForColocatedBucketsBecomePrimary() {
+ if (hasChildRegion()) {
+ synchronized (this) {
+ return notPrimary;
+ }
+ }
+ return false;
+ }
+
+ boolean hasChildRegion() {
+ return ColocationHelper.getFirstColocatedNonShadowChildRegions(partitionedRegion).size() > 0;
+ }
+
+ void waitForAllChildColocatedBucketsBecomePrimary() {
+ synchronized (allChildBucketsBecomePrimaryLock) {
+ if (!alreadyInWaitForAllChildBucketsToBecomePrimary) {
+ alreadyInWaitForAllChildBucketsToBecomePrimary = true;
+ executeCheckIfAllChildBucketsBecomePrimary();
+ }
+ while (!allChildBucketsBecomePrimary && getBucketAdvisor().isPrimary()) {
+ // if no longer primary, no need to wait as the operation should fail
+ // with PrimaryBucketException later.
+ try {
+ allChildBucketsBecomePrimaryLock.wait(10);
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ getCache().getCancelCriterion().checkCancelInProgress(e);
+ }
+ }
+ }
+ }
+
+ void executeCheckIfAllChildBucketsBecomePrimary() {
+ if (waitForAllChildBucketsToBecomePrimaryExecutor == null) {
+ waitForAllChildBucketsToBecomePrimaryExecutor =
+ LoggingExecutors.newSingleThreadExecutor("CheckPrimaryForColocation", true);
+ }
+ waitForAllChildBucketsToBecomePrimaryExecutor
+ .execute(this::checkIfAllChildBucketsBecomePrimary);
+ }
+
+ void checkIfAllChildBucketsBecomePrimary() {
+ try {
+ while (!getBucketAdvisor().checkIfAllColocatedChildBucketsBecomePrimary()) {
+ if (!getBucketAdvisor().isPrimary()) {
+ // This parent bucket is no longer a primary, no need to wait.
+ return;
+ }
+ }
+ synchronized (allChildBucketsBecomePrimaryLock) {
+ allChildBucketsBecomePrimary = true;
+ notPrimary = false;
Review Comment:
NotPrimary is intended to used for early return (if there is no colocated regions for this bucket), we only need to check once once notPrimary is set to false. I think to use same variable may cause confusion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-eshu commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-eshu commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r857829063
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketAdvisor.java:
##########
@@ -2766,23 +2766,23 @@ public boolean isShadowBucketDestroyed(String shadowBucketPath) {
boolean checkIfAllColocatedChildBucketsBecomePrimary() {
List<PartitionedRegion> colocatedChildPRs = getColocateNonShadowChildRegions();
- if (colocatedChildPRs.size() > 0) {
- for (PartitionedRegion partitionedRegion : colocatedChildPRs) {
- Bucket bucket = partitionedRegion.getRegionAdvisor().getBucket(getBucket().getId());
- if (bucket != null) {
- BucketAdvisor bucketAdvisor = bucket.getBucketAdvisor();
- if (!bucketAdvisor.checkIfAllColocatedChildBucketsBecomePrimary()) {
+ if (colocatedChildPRs.isEmpty()) {
+ return checkIfBecomesPrimary();
Review Comment:
This is intended for the leaf colocated child bucket (we need to wait for it to become primary as well). It will fail the test case if you simply change it to return true. (This is to ensure all child buckets to become primary)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-eshu commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-eshu commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r855679936
##########
geode-core/src/main/java/org/apache/geode/internal/cache/ColocationHelper.java:
##########
@@ -377,6 +378,26 @@ public static Map<String, LocalDataSet> getColocatedLocalDataSetsForBuckets(
*/
public static List<PartitionedRegion> getColocatedChildRegions(
PartitionedRegion partitionedRegion) {
+ List<PartitionedRegion> colocatedChildRegions =
+ getColocatedChildRegions(partitionedRegion, true, false);
+
+ // Fix for 44484 - Make the list of colocated child regions
+ // is always in the same order on all nodes.
+ Collections.sort(colocatedChildRegions, (o1, o2) -> {
+ if (o1.isShadowPR() == o2.isShadowPR()) {
+ return o1.getFullPath().compareTo(o2.getFullPath());
+ }
+ if (o1.isShadowPR()) {
+ return 1;
+ }
+ return -1;
+ });
+ return colocatedChildRegions;
+ }
+
+ @NotNull
+ private static List<PartitionedRegion> getColocatedChildRegions(
Review Comment:
The newly added methods which only track child regions without wanQueue. They do not need to be sorted in current use case. When these method usage changes we can move the sorting there, but for now there is no need.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] DonalEvans commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
DonalEvans commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r857800483
##########
geode-core/src/test/java/org/apache/geode/internal/cache/BucketRegionTest.java:
##########
@@ -800,11 +808,11 @@ public void handleWANEventSetNotPrimaryIfWasPrimaryAndNoLongerAPrimary() {
verify(bucketRegion).setNotPrimaryIfNecessary();
assertThat(bucketRegion.notPrimary).isTrue();
- assertThat(bucketRegion.allChildBucketsBecomePrimary).isFalse();
+ assertThat(bucketRegion.allColocatedBucketsBecomePrimary).isFalse();
}
@Test
- public void handleWANEventDoesNotSetNotPrimaryIfWasNotPrimary() {
+ public void handleWANEventDoesNotSetNotPrimaryIfWas0NotPrimary() {
Review Comment:
Typo here, I think. Should not have a "0" in the test name.
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketAdvisor.java:
##########
@@ -2766,23 +2766,23 @@ public boolean isShadowBucketDestroyed(String shadowBucketPath) {
boolean checkIfAllColocatedChildBucketsBecomePrimary() {
List<PartitionedRegion> colocatedChildPRs = getColocateNonShadowChildRegions();
- if (colocatedChildPRs.size() > 0) {
- for (PartitionedRegion partitionedRegion : colocatedChildPRs) {
- Bucket bucket = partitionedRegion.getRegionAdvisor().getBucket(getBucket().getId());
- if (bucket != null) {
- BucketAdvisor bucketAdvisor = bucket.getBucketAdvisor();
- if (!bucketAdvisor.checkIfAllColocatedChildBucketsBecomePrimary()) {
+ if (colocatedChildPRs.isEmpty()) {
+ return checkIfBecomesPrimary();
Review Comment:
I think that this can just be `return true;` since if there are no colocated child PRs, then it doesn't make sense to ever return false here.
##########
geode-core/src/test/java/org/apache/geode/internal/cache/BucketRegionTest.java:
##########
@@ -715,4 +722,178 @@ public void txHandleWANEventDoesNotCallHandleWANEventIfParallelWanNotEnabled() {
verify(bucketRegion, never()).handleWANEvent(event);
}
+
+ @Test
+ public void needToWaitForColocatedBucketsBecomePrimaryReturnsTrueIfHasChildRegionAndWasNotPrimary() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ doReturn(true).when(bucketRegion).hasChildRegion();
+
+ assertThat(bucketRegion.needToWaitForColocatedBucketsBecomePrimary()).isTrue();
+ }
+
+ @Test
+ public void needToWaitForColocatedBucketsBecomePrimaryReturnsFalseIfHasChildRegionAndWasPrimary() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ doReturn(true).when(bucketRegion).hasChildRegion();
+ bucketRegion.notPrimary = false;
+
+ assertThat(bucketRegion.needToWaitForColocatedBucketsBecomePrimary()).isFalse();
+ }
+
+ @Test
+ public void needToWaitForColocatedBucketsBecomePrimaryReturnsFalseIfNoChildRegion() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ doReturn(false).when(bucketRegion).hasChildRegion();
+
+ assertThat(bucketRegion.needToWaitForColocatedBucketsBecomePrimary()).isFalse();
+ }
+
+ @Test
+ public void handleWANEventDoesNotWaitForAllChildColocatedBucketsBecomePrimaryIfNoNeedToWait() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ when(bucketAdvisor.isPrimary()).thenReturn(true);
+ doReturn(false).when(bucketRegion).needToWaitForColocatedBucketsBecomePrimary();
+
+ when(partitionedRegion.getTotalNumberOfBuckets()).thenReturn(4);
+ doReturn(0).when(bucketRegion).getId();
+
+ bucketRegion.handleWANEvent(event);
+
+ verify(bucketRegion, never()).waitForAllChildColocatedBucketsBecomePrimary();
+ verify(event).setTailKey(4L);
+ }
+
+ @Test
+ public void handleWANEventWaitsForAllChildColocatedBucketsBecomePrimaryIfWasNotPrimary() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ when(bucketAdvisor.isPrimary()).thenReturn(true);
+ doReturn(true).when(bucketRegion).hasChildRegion();
+ doNothing().when(bucketRegion).waitForAllChildColocatedBucketsBecomePrimary();
+ when(partitionedRegion.getTotalNumberOfBuckets()).thenReturn(4);
+ doReturn(0).when(bucketRegion).getId();
+
+ bucketRegion.handleWANEvent(event);
+
+ verify(bucketRegion).waitForAllChildColocatedBucketsBecomePrimary();
+ verify(event).setTailKey(4L);
+ }
+
+ @Test
+ public void handleWANEventSetNotPrimaryIfWasPrimaryAndNoLongerAPrimary() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ when(bucketAdvisor.isPrimary()).thenReturn(false);
+ bucketRegion.notPrimary = false;
+
+ bucketRegion.handleWANEvent(event);
+
+ verify(bucketRegion).setNotPrimaryIfNecessary();
+ assertThat(bucketRegion.notPrimary).isTrue();
+ assertThat(bucketRegion.allChildBucketsBecomePrimary).isFalse();
+ }
+
+ @Test
+ public void handleWANEventDoesNotSetNotPrimaryIfWasNotPrimary() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ when(bucketAdvisor.isPrimary()).thenReturn(false);
+ bucketRegion.notPrimary = true;
+
+ bucketRegion.handleWANEvent(event);
+
+ verify(bucketRegion, never()).setNotPrimaryIfNecessary();
+ }
+
+ @Test
+ public void onlyOneThreadWillExecuteCheckIfAllChildBucketsBecomePrimary() throws Exception {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ doNothing().when(bucketRegion).executeCheckIfAllChildBucketsBecomePrimary();
+
+ Future<?> future = executor.submit(bucketRegion::waitForAllChildColocatedBucketsBecomePrimary);
+ Future<?> future2 = executor.submit(bucketRegion::waitForAllChildColocatedBucketsBecomePrimary);
+
+ future.get();
+ future2.get();
Review Comment:
None that I can think of, sorry. Since this test isn't actually testing what it says its testing, it should probably be renamed or reworked though. Maybe remove the async parts and just make it "multipleCallsToWaitForAllColocatedBucketsFromLeaderBecomePrimaryWillOnlyInvokeExecuteCheckIfAllChildBucketsFromLeaderBecomePrimaryOnce()" although that's a bit long.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-eshu commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-eshu commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r855677627
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegion.java:
##########
@@ -604,17 +611,23 @@ public void handleWANEvent(EntryEventImpl event) {
if (!(this instanceof AbstractBucketRegionQueue)) {
if (getBucketAdvisor().isPrimary()) {
+ if (notPrimary && needToWaitForColocatedBucketsBecomePrimary()) {
Review Comment:
It is later accessed in needToWaitForColocatedBucketsBecomePrimary() with synchronized on notPrimary. I see your point, the notPrimary may need to be volatile. Was intend to avoid synchronization here to avoid regular cases without colocated regions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-jbarrett commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-jbarrett commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r857935699
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketAdvisor.java:
##########
@@ -2762,4 +2763,44 @@ void markShadowBucketAsDestroyed(String shadowBucketPath) {
public boolean isShadowBucketDestroyed(String shadowBucketPath) {
return destroyedShadowBuckets.getOrDefault(shadowBucketPath, false);
}
+
+ boolean checkIfAllColocatedChildBucketsBecomePrimary() {
+ List<PartitionedRegion> colocatedChildPRs = getColocateNonShadowChildRegions();
+ if (colocatedChildPRs.size() > 0) {
+ for (PartitionedRegion partitionedRegion : colocatedChildPRs) {
+ Bucket bucket = partitionedRegion.getRegionAdvisor().getBucket(getBucket().getId());
+ if (bucket != null) {
+ BucketAdvisor bucketAdvisor = bucket.getBucketAdvisor();
+ if (!bucketAdvisor.checkIfAllColocatedChildBucketsBecomePrimary()) {
+ return false;
+ } else {
+ if (!checkIfBecomesPrimary()) {
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+ }
+ return checkIfBecomesPrimary();
+ }
+
+ @NotNull
+ List<PartitionedRegion> getColocateNonShadowChildRegions() {
+ return ColocationHelper.getColocatedNonShadowChildRegions(pRegion);
+ }
+
+ private boolean checkIfBecomesPrimary() {
+ synchronized (this) {
+ try {
+ if (!isPrimary()) {
+ wait(10);
Review Comment:
What is the ramifications if it becomes primary in 11ms? It seems if there is no adverse ramifications then adding a pause serves no purpose.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-jbarrett commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-jbarrett commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r857941403
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegion.java:
##########
@@ -629,6 +642,153 @@ public void handleWANEvent(EntryEventImpl event) {
}
}
+ boolean needToWaitForColocatedBucketsBecomePrimary() {
+ if (hasChildRegion() || getBucketAdvisor().hasParent()) {
+ synchronized (allColocatedBucketsBecomePrimaryLock) {
+ return notPrimary;
+ }
+ }
+ synchronized (allColocatedBucketsBecomePrimaryLock) {
+ // If there are no colocated regions for this bucket region,
+ notPrimary = false;
+ }
+ return false;
+ }
+
+ boolean hasChildRegion() {
+ return ColocationHelper.getFirstColocatedNonShadowChildRegions(partitionedRegion).size() > 0;
Review Comment:
```java
!ColocationHelper.getFirstColocatedNonShadowChildRegions(partitionedRegion).isEmpty()
```
Calling `size()` can be costly depending on the implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-eshu closed pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-eshu closed pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
URL: https://github.com/apache/geode/pull/7596
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-eshu commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-eshu commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r855676114
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegion.java:
##########
@@ -235,6 +237,12 @@ AtomicLong5 getEventSeqNum() {
return eventSeqNum;
}
+ boolean notPrimary = true;
+ volatile boolean allChildBucketsBecomePrimary = false;
+ private Object allChildBucketsBecomePrimaryLock = new Object();
Review Comment:
non private members are accessed in the unit test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-eshu commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-eshu commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r858112810
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketAdvisor.java:
##########
@@ -2762,4 +2763,44 @@ void markShadowBucketAsDestroyed(String shadowBucketPath) {
public boolean isShadowBucketDestroyed(String shadowBucketPath) {
return destroyedShadowBuckets.getOrDefault(shadowBucketPath, false);
}
+
+ boolean checkIfAllColocatedChildBucketsBecomePrimary() {
+ List<PartitionedRegion> colocatedChildPRs = getColocateNonShadowChildRegions();
+ if (colocatedChildPRs.size() > 0) {
+ for (PartitionedRegion partitionedRegion : colocatedChildPRs) {
+ Bucket bucket = partitionedRegion.getRegionAdvisor().getBucket(getBucket().getId());
+ if (bucket != null) {
+ BucketAdvisor bucketAdvisor = bucket.getBucketAdvisor();
+ if (!bucketAdvisor.checkIfAllColocatedChildBucketsBecomePrimary()) {
+ return false;
+ } else {
+ if (!checkIfBecomesPrimary()) {
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+ }
+ return checkIfBecomesPrimary();
+ }
+
+ @NotNull
+ List<PartitionedRegion> getColocateNonShadowChildRegions() {
+ return ColocationHelper.getColocatedNonShadowChildRegions(pRegion);
+ }
+
+ private boolean checkIfBecomesPrimary() {
+ synchronized (this) {
+ try {
+ if (!isPrimary()) {
+ wait(10);
Review Comment:
The getBucketAdvisor().checkIfAllColocatedChildBucketsBecomePrimary() will retry again, if it becomes primary in 11ms and will pass in the retry.
The reason added the pause is for a scenario that a child region is destroyed - while the leader is wait for bucketAdvisor state change, and never receive any if region is destroyed. However, the retry (after the pause) should detect this in the profile change and no longer wait for this child bucket when retrying.
I do think I may need to increase the pause to like 30 seconds or 60 seconds instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-eshu commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-eshu commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r855680249
##########
geode-core/src/test/java/org/apache/geode/internal/cache/BucketRegionTest.java:
##########
@@ -715,4 +722,178 @@ public void txHandleWANEventDoesNotCallHandleWANEventIfParallelWanNotEnabled() {
verify(bucketRegion, never()).handleWANEvent(event);
}
+
+ @Test
+ public void needToWaitForColocatedBucketsBecomePrimaryReturnsTrueIfHasChildRegionAndWasNotPrimary() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ doReturn(true).when(bucketRegion).hasChildRegion();
+
+ assertThat(bucketRegion.needToWaitForColocatedBucketsBecomePrimary()).isTrue();
+ }
+
+ @Test
+ public void needToWaitForColocatedBucketsBecomePrimaryReturnsFalseIfHasChildRegionAndWasPrimary() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ doReturn(true).when(bucketRegion).hasChildRegion();
+ bucketRegion.notPrimary = false;
+
+ assertThat(bucketRegion.needToWaitForColocatedBucketsBecomePrimary()).isFalse();
+ }
+
+ @Test
+ public void needToWaitForColocatedBucketsBecomePrimaryReturnsFalseIfNoChildRegion() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ doReturn(false).when(bucketRegion).hasChildRegion();
+
+ assertThat(bucketRegion.needToWaitForColocatedBucketsBecomePrimary()).isFalse();
+ }
+
+ @Test
+ public void handleWANEventDoesNotWaitForAllChildColocatedBucketsBecomePrimaryIfNoNeedToWait() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ when(bucketAdvisor.isPrimary()).thenReturn(true);
+ doReturn(false).when(bucketRegion).needToWaitForColocatedBucketsBecomePrimary();
+
+ when(partitionedRegion.getTotalNumberOfBuckets()).thenReturn(4);
+ doReturn(0).when(bucketRegion).getId();
+
+ bucketRegion.handleWANEvent(event);
+
+ verify(bucketRegion, never()).waitForAllChildColocatedBucketsBecomePrimary();
+ verify(event).setTailKey(4L);
+ }
+
+ @Test
+ public void handleWANEventWaitsForAllChildColocatedBucketsBecomePrimaryIfWasNotPrimary() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ when(bucketAdvisor.isPrimary()).thenReturn(true);
+ doReturn(true).when(bucketRegion).hasChildRegion();
+ doNothing().when(bucketRegion).waitForAllChildColocatedBucketsBecomePrimary();
+ when(partitionedRegion.getTotalNumberOfBuckets()).thenReturn(4);
+ doReturn(0).when(bucketRegion).getId();
+
+ bucketRegion.handleWANEvent(event);
+
+ verify(bucketRegion).waitForAllChildColocatedBucketsBecomePrimary();
+ verify(event).setTailKey(4L);
+ }
+
+ @Test
+ public void handleWANEventSetNotPrimaryIfWasPrimaryAndNoLongerAPrimary() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ when(bucketAdvisor.isPrimary()).thenReturn(false);
+ bucketRegion.notPrimary = false;
+
+ bucketRegion.handleWANEvent(event);
+
+ verify(bucketRegion).setNotPrimaryIfNecessary();
+ assertThat(bucketRegion.notPrimary).isTrue();
+ assertThat(bucketRegion.allChildBucketsBecomePrimary).isFalse();
+ }
+
+ @Test
+ public void handleWANEventDoesNotSetNotPrimaryIfWasNotPrimary() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ when(bucketAdvisor.isPrimary()).thenReturn(false);
+ bucketRegion.notPrimary = true;
+
+ bucketRegion.handleWANEvent(event);
+
+ verify(bucketRegion, never()).setNotPrimaryIfNecessary();
+ }
+
+ @Test
+ public void onlyOneThreadWillExecuteCheckIfAllChildBucketsBecomePrimary() throws Exception {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ doNothing().when(bucketRegion).executeCheckIfAllChildBucketsBecomePrimary();
+
+ Future<?> future = executor.submit(bucketRegion::waitForAllChildColocatedBucketsBecomePrimary);
+ Future<?> future2 = executor.submit(bucketRegion::waitForAllChildColocatedBucketsBecomePrimary);
+
+ future.get();
+ future2.get();
Review Comment:
Not sure if we definitely can hit the race issue in the unit test. Is there any suggestion for this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-jbarrett commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-jbarrett commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r857944312
##########
geode-core/src/test/java/org/apache/geode/internal/cache/BucketRegionTest.java:
##########
@@ -61,8 +67,12 @@
import org.apache.geode.internal.cache.tier.sockets.ConnectionListener;
import org.apache.geode.internal.statistics.StatisticsClock;
import org.apache.geode.test.fake.Fakes;
+import org.apache.geode.test.junit.rules.ExecutorServiceRule;
public class BucketRegionTest {
+ @Rule
+ public ExecutorServiceRule executor = new ExecutorServiceRule();
Review Comment:
Is there a way to eliminate the addition of this rule so we can use JUnit 5.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] DonalEvans commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
DonalEvans commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r854578032
##########
geode-core/src/main/java/org/apache/geode/internal/cache/ColocationHelper.java:
##########
@@ -377,6 +378,26 @@ public static Map<String, LocalDataSet> getColocatedLocalDataSetsForBuckets(
*/
public static List<PartitionedRegion> getColocatedChildRegions(
PartitionedRegion partitionedRegion) {
+ List<PartitionedRegion> colocatedChildRegions =
+ getColocatedChildRegions(partitionedRegion, true, false);
+
+ // Fix for 44484 - Make the list of colocated child regions
+ // is always in the same order on all nodes.
+ Collections.sort(colocatedChildRegions, (o1, o2) -> {
Review Comment:
Since `colocatedChildRegions` is a list, this can be simplified to:
```
colocatedChildRegions.sort((o1, o2) -> {
```
##########
geode-core/src/main/java/org/apache/geode/internal/cache/ColocationHelper.java:
##########
@@ -377,6 +378,26 @@ public static Map<String, LocalDataSet> getColocatedLocalDataSetsForBuckets(
*/
public static List<PartitionedRegion> getColocatedChildRegions(
PartitionedRegion partitionedRegion) {
+ List<PartitionedRegion> colocatedChildRegions =
+ getColocatedChildRegions(partitionedRegion, true, false);
+
+ // Fix for 44484 - Make the list of colocated child regions
+ // is always in the same order on all nodes.
+ Collections.sort(colocatedChildRegions, (o1, o2) -> {
+ if (o1.isShadowPR() == o2.isShadowPR()) {
+ return o1.getFullPath().compareTo(o2.getFullPath());
+ }
+ if (o1.isShadowPR()) {
+ return 1;
+ }
+ return -1;
+ });
+ return colocatedChildRegions;
+ }
+
+ @NotNull
+ private static List<PartitionedRegion> getColocatedChildRegions(
Review Comment:
This method does not sort the child regions, so the returned list when calling `getColocatedChildRegions(partitionedRegion)` will not be the same as calling `getColocatedChildRegions(partitionedRegion, true, false)`. Would it be better to move the sorting into this method, since the one-argument version of the method calls this one anyway?
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegion.java:
##########
@@ -629,6 +642,76 @@ public void handleWANEvent(EntryEventImpl event) {
}
}
+ boolean needToWaitForColocatedBucketsBecomePrimary() {
+ if (hasChildRegion()) {
+ synchronized (this) {
+ return notPrimary;
+ }
+ }
+ return false;
+ }
+
+ boolean hasChildRegion() {
+ return ColocationHelper.getFirstColocatedNonShadowChildRegions(partitionedRegion).size() > 0;
+ }
+
+ void waitForAllChildColocatedBucketsBecomePrimary() {
+ synchronized (allChildBucketsBecomePrimaryLock) {
+ if (!alreadyInWaitForAllChildBucketsToBecomePrimary) {
+ alreadyInWaitForAllChildBucketsToBecomePrimary = true;
+ executeCheckIfAllChildBucketsBecomePrimary();
+ }
+ while (!allChildBucketsBecomePrimary && getBucketAdvisor().isPrimary()) {
+ // if no longer primary, no need to wait as the operation should fail
+ // with PrimaryBucketException later.
+ try {
+ allChildBucketsBecomePrimaryLock.wait(10);
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ getCache().getCancelCriterion().checkCancelInProgress(e);
+ }
+ }
+ }
+ }
+
+ void executeCheckIfAllChildBucketsBecomePrimary() {
+ if (waitForAllChildBucketsToBecomePrimaryExecutor == null) {
+ waitForAllChildBucketsToBecomePrimaryExecutor =
+ LoggingExecutors.newSingleThreadExecutor("CheckPrimaryForColocation", true);
+ }
+ waitForAllChildBucketsToBecomePrimaryExecutor
+ .execute(this::checkIfAllChildBucketsBecomePrimary);
+ }
+
+ void checkIfAllChildBucketsBecomePrimary() {
+ try {
+ while (!getBucketAdvisor().checkIfAllColocatedChildBucketsBecomePrimary()) {
+ if (!getBucketAdvisor().isPrimary()) {
+ // This parent bucket is no longer a primary, no need to wait.
+ return;
+ }
+ }
+ synchronized (allChildBucketsBecomePrimaryLock) {
+ allChildBucketsBecomePrimary = true;
+ notPrimary = false;
Review Comment:
We seem to always set these two fields together, and they're always the opposite value of eachother. Are both needed?
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegion.java:
##########
@@ -235,6 +237,12 @@ AtomicLong5 getEventSeqNum() {
return eventSeqNum;
}
+ boolean notPrimary = true;
+ volatile boolean allChildBucketsBecomePrimary = false;
+ private Object allChildBucketsBecomePrimaryLock = new Object();
Review Comment:
This should be `final`
##########
geode-core/src/test/java/org/apache/geode/internal/cache/BucketRegionTest.java:
##########
@@ -715,4 +722,178 @@ public void txHandleWANEventDoesNotCallHandleWANEventIfParallelWanNotEnabled() {
verify(bucketRegion, never()).handleWANEvent(event);
}
+
+ @Test
+ public void needToWaitForColocatedBucketsBecomePrimaryReturnsTrueIfHasChildRegionAndWasNotPrimary() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ doReturn(true).when(bucketRegion).hasChildRegion();
+
+ assertThat(bucketRegion.needToWaitForColocatedBucketsBecomePrimary()).isTrue();
+ }
+
+ @Test
+ public void needToWaitForColocatedBucketsBecomePrimaryReturnsFalseIfHasChildRegionAndWasPrimary() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ doReturn(true).when(bucketRegion).hasChildRegion();
+ bucketRegion.notPrimary = false;
+
+ assertThat(bucketRegion.needToWaitForColocatedBucketsBecomePrimary()).isFalse();
+ }
+
+ @Test
+ public void needToWaitForColocatedBucketsBecomePrimaryReturnsFalseIfNoChildRegion() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ doReturn(false).when(bucketRegion).hasChildRegion();
+
+ assertThat(bucketRegion.needToWaitForColocatedBucketsBecomePrimary()).isFalse();
+ }
+
+ @Test
+ public void handleWANEventDoesNotWaitForAllChildColocatedBucketsBecomePrimaryIfNoNeedToWait() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ when(bucketAdvisor.isPrimary()).thenReturn(true);
+ doReturn(false).when(bucketRegion).needToWaitForColocatedBucketsBecomePrimary();
+
+ when(partitionedRegion.getTotalNumberOfBuckets()).thenReturn(4);
+ doReturn(0).when(bucketRegion).getId();
+
+ bucketRegion.handleWANEvent(event);
+
+ verify(bucketRegion, never()).waitForAllChildColocatedBucketsBecomePrimary();
+ verify(event).setTailKey(4L);
+ }
+
+ @Test
+ public void handleWANEventWaitsForAllChildColocatedBucketsBecomePrimaryIfWasNotPrimary() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ when(bucketAdvisor.isPrimary()).thenReturn(true);
+ doReturn(true).when(bucketRegion).hasChildRegion();
+ doNothing().when(bucketRegion).waitForAllChildColocatedBucketsBecomePrimary();
+ when(partitionedRegion.getTotalNumberOfBuckets()).thenReturn(4);
+ doReturn(0).when(bucketRegion).getId();
+
+ bucketRegion.handleWANEvent(event);
+
+ verify(bucketRegion).waitForAllChildColocatedBucketsBecomePrimary();
+ verify(event).setTailKey(4L);
+ }
+
+ @Test
+ public void handleWANEventSetNotPrimaryIfWasPrimaryAndNoLongerAPrimary() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ when(bucketAdvisor.isPrimary()).thenReturn(false);
+ bucketRegion.notPrimary = false;
+
+ bucketRegion.handleWANEvent(event);
+
+ verify(bucketRegion).setNotPrimaryIfNecessary();
+ assertThat(bucketRegion.notPrimary).isTrue();
+ assertThat(bucketRegion.allChildBucketsBecomePrimary).isFalse();
+ }
+
+ @Test
+ public void handleWANEventDoesNotSetNotPrimaryIfWasNotPrimary() {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ when(bucketAdvisor.isPrimary()).thenReturn(false);
+ bucketRegion.notPrimary = true;
+
+ bucketRegion.handleWANEvent(event);
+
+ verify(bucketRegion, never()).setNotPrimaryIfNecessary();
+ }
+
+ @Test
+ public void onlyOneThreadWillExecuteCheckIfAllChildBucketsBecomePrimary() throws Exception {
+ BucketRegion bucketRegion =
+ spy(new BucketRegion(regionName, regionAttributes, partitionedRegion,
+ cache, internalRegionArgs, disabledClock()));
+ doNothing().when(bucketRegion).executeCheckIfAllChildBucketsBecomePrimary();
+
+ Future<?> future = executor.submit(bucketRegion::waitForAllChildColocatedBucketsBecomePrimary);
+ Future<?> future2 = executor.submit(bucketRegion::waitForAllChildColocatedBucketsBecomePrimary);
+
+ future.get();
+ future2.get();
Review Comment:
If I remove the synchronization from `waitForAllChildColocatedBucketsBecomePrimary()`, this test still passes, even when run 1000 times, so I think that without something to ensure that both threads are trying to execute the synchronized block at the same time, this test can't guarantee that it's testing correct synchronization, just that successive calls to `bucketRegion.waitForAllChildColocatedBucketsBecomePrimary()` do not result in `executeCheckIfAllChildBucketsBecomePrimary` being called twice.
##########
geode-core/src/test/java/org/apache/geode/internal/cache/BucketAdvisorTest.java:
##########
@@ -317,4 +372,163 @@ public void testGetAllHostingMembersReturnsMemberWhenBucketAdvisorHasTwoProfiles
assertThat(bucketAdvisor.adviseInitialized().size()).isEqualTo(1);
}
+
+ @Test
+ void checkIfAllColocatedChildBucketsBecomePrimaryRetrunsTrueIfAllChildBucketsArePrimary() {
Review Comment:
Typo here, should be "PrimaryReturnsTrue"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-jbarrett commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-jbarrett commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r857938638
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegion.java:
##########
@@ -604,17 +611,23 @@ public void handleWANEvent(EntryEventImpl event) {
if (!(this instanceof AbstractBucketRegionQueue)) {
if (getBucketAdvisor().isPrimary()) {
+ if (notPrimary && needToWaitForColocatedBucketsBecomePrimary()) {
Review Comment:
Access should be symmetrical, when it isn't it raises an eyebrow, so even making it volatile still begs why synchronized in some cases and not others.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-eshu commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-eshu commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r858112810
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketAdvisor.java:
##########
@@ -2762,4 +2763,44 @@ void markShadowBucketAsDestroyed(String shadowBucketPath) {
public boolean isShadowBucketDestroyed(String shadowBucketPath) {
return destroyedShadowBuckets.getOrDefault(shadowBucketPath, false);
}
+
+ boolean checkIfAllColocatedChildBucketsBecomePrimary() {
+ List<PartitionedRegion> colocatedChildPRs = getColocateNonShadowChildRegions();
+ if (colocatedChildPRs.size() > 0) {
+ for (PartitionedRegion partitionedRegion : colocatedChildPRs) {
+ Bucket bucket = partitionedRegion.getRegionAdvisor().getBucket(getBucket().getId());
+ if (bucket != null) {
+ BucketAdvisor bucketAdvisor = bucket.getBucketAdvisor();
+ if (!bucketAdvisor.checkIfAllColocatedChildBucketsBecomePrimary()) {
+ return false;
+ } else {
+ if (!checkIfBecomesPrimary()) {
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+ }
+ return checkIfBecomesPrimary();
+ }
+
+ @NotNull
+ List<PartitionedRegion> getColocateNonShadowChildRegions() {
+ return ColocationHelper.getColocatedNonShadowChildRegions(pRegion);
+ }
+
+ private boolean checkIfBecomesPrimary() {
+ synchronized (this) {
+ try {
+ if (!isPrimary()) {
+ wait(10);
Review Comment:
The getBucketAdvisor().checkIfAllColocatedChildBucketsBecomePrimary() will retry again, if it becomes primary in 11ms and will pass in the retry.
The reason added the pause is for a scenario that a child region is destroyed - while the leader is wait for bucketAdvisor state change, and never receive any if region is destroyed. However, the retry (after the pause) should detect this in the profile change and no longer wait for this child bucket when retrying.
I do think I may need to increase the wait/pause to like 30 seconds or 60 seconds instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-eshu commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-eshu commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r855677627
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegion.java:
##########
@@ -604,17 +611,23 @@ public void handleWANEvent(EntryEventImpl event) {
if (!(this instanceof AbstractBucketRegionQueue)) {
if (getBucketAdvisor().isPrimary()) {
+ if (notPrimary && needToWaitForColocatedBucketsBecomePrimary()) {
Review Comment:
It is later accessed in needToWaitForColocatedBucketsBecomePrimary() with synchronized on notPrimary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] DonalEvans commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
DonalEvans commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r857833762
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketAdvisor.java:
##########
@@ -2766,23 +2766,23 @@ public boolean isShadowBucketDestroyed(String shadowBucketPath) {
boolean checkIfAllColocatedChildBucketsBecomePrimary() {
List<PartitionedRegion> colocatedChildPRs = getColocateNonShadowChildRegions();
- if (colocatedChildPRs.size() > 0) {
- for (PartitionedRegion partitionedRegion : colocatedChildPRs) {
- Bucket bucket = partitionedRegion.getRegionAdvisor().getBucket(getBucket().getId());
- if (bucket != null) {
- BucketAdvisor bucketAdvisor = bucket.getBucketAdvisor();
- if (!bucketAdvisor.checkIfAllColocatedChildBucketsBecomePrimary()) {
+ if (colocatedChildPRs.isEmpty()) {
+ return checkIfBecomesPrimary();
Review Comment:
Ah, I see, thanks for the explanation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] upthewaterspout commented on pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
upthewaterspout commented on PR #7596:
URL: https://github.com/apache/geode/pull/7596#issuecomment-1110288344
@pivotal-eshu and I discussed an alternative to this fix that involved making all colocated BucketAdvisors use the same read/write lock (instead of many, as we do currently). This would ensure that we could not still have operations in progress on the old primary for some regions as well as in progress on the new primary for other regions.
I feel like something along those lines is a cleaner fix - something that doesn't involve introducing additional state with additional locking.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-jbarrett commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-jbarrett commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r851668162
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegion.java:
##########
@@ -604,17 +611,23 @@ public void handleWANEvent(EntryEventImpl event) {
if (!(this instanceof AbstractBucketRegionQueue)) {
if (getBucketAdvisor().isPrimary()) {
+ if (notPrimary && needToWaitForColocatedBucketsBecomePrimary()) {
Review Comment:
`nonPrimary` appears to not be synchronized before access here.
##########
geode-core/src/test/java/org/apache/geode/internal/cache/BucketAdvisorTest.java:
##########
@@ -30,11 +29,18 @@
import static org.mockito.Mockito.when;
import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
import java.util.List;
import java.util.Map;
-import org.junit.Test;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
Review Comment:
Thanks for upgrading to JUnit5.
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegion.java:
##########
@@ -235,6 +237,12 @@ AtomicLong5 getEventSeqNum() {
return eventSeqNum;
}
+ boolean notPrimary = true;
+ volatile boolean allChildBucketsBecomePrimary = false;
+ private Object allChildBucketsBecomePrimaryLock = new Object();
+ volatile boolean alreadyInWaitForAllChildBucketsToBecomePrimary = false;
Review Comment:
All access to `alreadyInWaitForAllChildBucketsToBecomePrimary` is done under synchronization so `volatile` is redundant.
##########
geode-core/src/test/java/org/apache/geode/internal/cache/BucketAdvisorTest.java:
##########
@@ -153,12 +207,13 @@ public void volunteerForPrimaryIgnoresMissingPrimaryElector() {
mock(BucketAdvisor.VolunteeringDelegate.class);
advisorSpy.setVolunteeringDelegate(volunteeringDelegate);
advisorSpy.initializePrimaryElector(missingElectorId);
- assertEquals(missingElectorId, advisorSpy.getPrimaryElector());
+ Assertions.assertEquals(missingElectorId, advisorSpy.getPrimaryElector());
Review Comment:
replace with `AssertJ`
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegion.java:
##########
@@ -235,6 +237,12 @@ AtomicLong5 getEventSeqNum() {
return eventSeqNum;
}
+ boolean notPrimary = true;
Review Comment:
This new member is mutated and accessed under different synchronization objects.
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegion.java:
##########
@@ -235,6 +237,12 @@ AtomicLong5 getEventSeqNum() {
return eventSeqNum;
}
+ boolean notPrimary = true;
+ volatile boolean allChildBucketsBecomePrimary = false;
Review Comment:
All access to `allChildBucketsBecomePrimary` is done under synchronization so `volatile` is redundant.
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegion.java:
##########
@@ -235,6 +237,12 @@ AtomicLong5 getEventSeqNum() {
return eventSeqNum;
}
+ boolean notPrimary = true;
+ volatile boolean allChildBucketsBecomePrimary = false;
+ private Object allChildBucketsBecomePrimaryLock = new Object();
Review Comment:
Why are some of these members private and others not?
##########
geode-core/src/main/java/org/apache/geode/internal/cache/ColocationHelper.java:
##########
@@ -377,6 +378,26 @@ public static Map<String, LocalDataSet> getColocatedLocalDataSetsForBuckets(
*/
public static List<PartitionedRegion> getColocatedChildRegions(
PartitionedRegion partitionedRegion) {
+ List<PartitionedRegion> colocatedChildRegions =
+ getColocatedChildRegions(partitionedRegion, true, false);
+
+ // Fix for 44484 - Make the list of colocated child regions
Review Comment:
Please remove bug ticket numbers in comments.
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketAdvisor.java:
##########
@@ -2762,4 +2763,44 @@ void markShadowBucketAsDestroyed(String shadowBucketPath) {
public boolean isShadowBucketDestroyed(String shadowBucketPath) {
return destroyedShadowBuckets.getOrDefault(shadowBucketPath, false);
}
+
+ boolean checkIfAllColocatedChildBucketsBecomePrimary() {
+ List<PartitionedRegion> colocatedChildPRs = getColocateNonShadowChildRegions();
+ if (colocatedChildPRs.size() > 0) {
+ for (PartitionedRegion partitionedRegion : colocatedChildPRs) {
+ Bucket bucket = partitionedRegion.getRegionAdvisor().getBucket(getBucket().getId());
+ if (bucket != null) {
+ BucketAdvisor bucketAdvisor = bucket.getBucketAdvisor();
+ if (!bucketAdvisor.checkIfAllColocatedChildBucketsBecomePrimary()) {
+ return false;
+ } else {
+ if (!checkIfBecomesPrimary()) {
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+ }
+ return checkIfBecomesPrimary();
+ }
+
+ @NotNull
+ List<PartitionedRegion> getColocateNonShadowChildRegions() {
+ return ColocationHelper.getColocatedNonShadowChildRegions(pRegion);
+ }
+
+ private boolean checkIfBecomesPrimary() {
+ synchronized (this) {
+ try {
+ if (!isPrimary()) {
+ wait(10);
Review Comment:
Why the arbitrary wait 10ms?
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketAdvisor.java:
##########
@@ -2762,4 +2763,44 @@ void markShadowBucketAsDestroyed(String shadowBucketPath) {
public boolean isShadowBucketDestroyed(String shadowBucketPath) {
return destroyedShadowBuckets.getOrDefault(shadowBucketPath, false);
}
+
+ boolean checkIfAllColocatedChildBucketsBecomePrimary() {
+ List<PartitionedRegion> colocatedChildPRs = getColocateNonShadowChildRegions();
+ if (colocatedChildPRs.size() > 0) {
Review Comment:
Early exit.
```java
if (colocatedChildPRs.isEmpty()) {
return true;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] demery-pivotal commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
demery-pivotal commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r857986132
##########
geode-core/src/test/java/org/apache/geode/internal/cache/BucketRegionTest.java:
##########
@@ -61,8 +67,12 @@
import org.apache.geode.internal.cache.tier.sockets.ConnectionListener;
import org.apache.geode.internal.statistics.StatisticsClock;
import org.apache.geode.test.fake.Fakes;
+import org.apache.geode.test.junit.rules.ExecutorServiceRule;
public class BucketRegionTest {
+ @Rule
+ public ExecutorServiceRule executor = new ExecutorServiceRule();
Review Comment:
It's possible that this rule can work with JUnit 5. Try these two steps:
- Add `testImplementation('org.junit.jupiter:junit-jupiter-migrationsupport')` to geode-core's build.gradle file.
- Add `@EnableRuleMigrationSupport` to the test class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-eshu commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-eshu commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r855675613
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketAdvisor.java:
##########
@@ -2762,4 +2763,44 @@ void markShadowBucketAsDestroyed(String shadowBucketPath) {
public boolean isShadowBucketDestroyed(String shadowBucketPath) {
return destroyedShadowBuckets.getOrDefault(shadowBucketPath, false);
}
+
+ boolean checkIfAllColocatedChildBucketsBecomePrimary() {
+ List<PartitionedRegion> colocatedChildPRs = getColocateNonShadowChildRegions();
+ if (colocatedChildPRs.size() > 0) {
+ for (PartitionedRegion partitionedRegion : colocatedChildPRs) {
+ Bucket bucket = partitionedRegion.getRegionAdvisor().getBucket(getBucket().getId());
+ if (bucket != null) {
+ BucketAdvisor bucketAdvisor = bucket.getBucketAdvisor();
+ if (!bucketAdvisor.checkIfAllColocatedChildBucketsBecomePrimary()) {
+ return false;
+ } else {
+ if (!checkIfBecomesPrimary()) {
+ return false;
+ }
+ }
+ }
+ }
+ return true;
+ }
+ return checkIfBecomesPrimary();
+ }
+
+ @NotNull
+ List<PartitionedRegion> getColocateNonShadowChildRegions() {
+ return ColocationHelper.getColocatedNonShadowChildRegions(pRegion);
+ }
+
+ private boolean checkIfBecomesPrimary() {
+ synchronized (this) {
+ try {
+ if (!isPrimary()) {
+ wait(10);
Review Comment:
When we waiting for child buckets movePrimary to finish, we will allow sometime to see if it becomes primary. It can be argued how long it should wait.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode] pivotal-eshu commented on a diff in pull request #7596: GEODE-10242: Generate tail key after all child regions become primary
Posted by GitBox <gi...@apache.org>.
pivotal-eshu commented on code in PR #7596:
URL: https://github.com/apache/geode/pull/7596#discussion_r855677627
##########
geode-core/src/main/java/org/apache/geode/internal/cache/BucketRegion.java:
##########
@@ -604,17 +611,23 @@ public void handleWANEvent(EntryEventImpl event) {
if (!(this instanceof AbstractBucketRegionQueue)) {
if (getBucketAdvisor().isPrimary()) {
+ if (notPrimary && needToWaitForColocatedBucketsBecomePrimary()) {
Review Comment:
It is later accessed in needToWaitForColocatedBucketsBecomePrimary() with synchronized on notPrimary. I see your point, the notPrimary may need to be volatile.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@geode.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org