You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/03/28 03:27:12 UTC
[GitHub] [pulsar] wangjialing218 opened a new pull request #14903: fix MetadataStore#put may get unexcepted exception
wangjialing218 opened a new pull request #14903:
URL: https://github.com/apache/pulsar/pull/14903
### Motivation
call MetadataStore#put without version by `MetadataStore.put(path, data, Optional.empty())` concurrently with same node may get `BadVersionException`
At first the node is not exist, put operation will fail and fall back to create node with `MetadataStore.put(path, data, -1)`.
`version=-1` means create the node, and if the node was just created by other thread, it will thow `BadVersionException`.
### Modifications
Since original call does not set `version`, `BadVersionException` is not excepted for caller.
We could just overwrite the value in this case.
### Documentation
Check the box below or label this PR directly (if you have committer privilege).
- [x] `no-need-doc`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] wangjialing218 commented on pull request #14903: [metadata] fix MetadataStore#put may get unexcepted exception
Posted by GitBox <gi...@apache.org>.
wangjialing218 commented on pull request #14903:
URL: https://github.com/apache/pulsar/pull/14903#issuecomment-1085315007
@RobertIndie tests added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] wangjialing218 closed pull request #14903: [metadata] fix MetadataStore#put may get unexcepted exception
Posted by GitBox <gi...@apache.org>.
wangjialing218 closed pull request #14903:
URL: https://github.com/apache/pulsar/pull/14903
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] wangjialing218 commented on pull request #14903: [metadata] fix MetadataStore#put may get unexcepted exception
Posted by GitBox <gi...@apache.org>.
wangjialing218 commented on pull request #14903:
URL: https://github.com/apache/pulsar/pull/14903#issuecomment-1085291153
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Jason918 commented on a change in pull request #14903: [metadata] fix MetadataStore#put may get unexcepted exception
Posted by GitBox <gi...@apache.org>.
Jason918 commented on a change in pull request #14903:
URL: https://github.com/apache/pulsar/pull/14903#discussion_r840410789
##########
File path: pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/ZKMetadataStore.java
##########
@@ -386,7 +386,13 @@ private void internalStorePut(OpPut opPut) {
put(opPut.getPath(), opPut.getData(), Optional.of(-1L)).thenAccept(
s -> future.complete(s))
.exceptionally(ex -> {
- future.completeExceptionally(MetadataStoreException.wrap(ex.getCause()));
+ if (ex.getCause() instanceof BadVersionException) {
+ // The z-node exist now, let's overwrite it
+ internalStorePut(opPut);
Review comment:
`future` is not completed here.
##########
File path: pulsar-metadata/src/test/java/org/apache/pulsar/metadata/MetadataStoreTest.java
##########
@@ -83,6 +84,30 @@ public void emptyStoreTest(String provider, Supplier<String> urlSupplier) throws
}
}
+ @Test(dataProvider = "impl")
+ public void concurrentPutTest(String provider, Supplier<String> urlSupplier) throws Exception {
+ if (!provider.equals("ZooKeeper")) {
Review comment:
This case make sense to other impl too.
##########
File path: pulsar-metadata/src/test/java/org/apache/pulsar/metadata/MetadataStoreTest.java
##########
@@ -83,6 +84,30 @@ public void emptyStoreTest(String provider, Supplier<String> urlSupplier) throws
}
}
+ @Test(dataProvider = "impl")
+ public void concurrentPutTest(String provider, Supplier<String> urlSupplier) throws Exception {
+ if (!provider.equals("ZooKeeper")) {
+ return;
+ }
+ @Cleanup
+ MetadataStore store = MetadataStoreFactory.create(urlSupplier.get(), MetadataStoreConfig.builder().build());
+
+ String data = "data";
+ String path = "/non-existing-key";
+ int concurrent = 50;
+ List<CompletableFuture<Stat>> resultList = new ArrayList<>();
+ while (concurrent > 0) {
Review comment:
This will very likely introduces a flaky test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaozhangmin commented on a change in pull request #14903: fix MetadataStore#put may get unexcepted exception
Posted by GitBox <gi...@apache.org>.
gaozhangmin commented on a change in pull request #14903:
URL: https://github.com/apache/pulsar/pull/14903#discussion_r836073675
##########
File path: pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/ZKMetadataStore.java
##########
@@ -386,7 +386,13 @@ private void internalStorePut(OpPut opPut) {
put(opPut.getPath(), opPut.getData(), Optional.of(-1L)).thenAccept(
s -> future.complete(s))
.exceptionally(ex -> {
- future.completeExceptionally(MetadataStoreException.wrap(ex.getCause()));
+ if (ex instanceof BadVersionException) {
Review comment:
BadVersion exception happened with concurrent write to same ZNODE. I think it's ok when some writer failed with BadVersionException because of competing failed. @wangjialing218
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] wangjialing218 commented on pull request #14903: fix MetadataStore#put may get unexcepted exception
Posted by GitBox <gi...@apache.org>.
wangjialing218 commented on pull request #14903:
URL: https://github.com/apache/pulsar/pull/14903#issuecomment-1080196223
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] wangjialing218 commented on a change in pull request #14903: fix MetadataStore#put may get unexcepted exception
Posted by GitBox <gi...@apache.org>.
wangjialing218 commented on a change in pull request #14903:
URL: https://github.com/apache/pulsar/pull/14903#discussion_r836099445
##########
File path: pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/ZKMetadataStore.java
##########
@@ -386,7 +386,13 @@ private void internalStorePut(OpPut opPut) {
put(opPut.getPath(), opPut.getData(), Optional.of(-1L)).thenAccept(
s -> future.complete(s))
.exceptionally(ex -> {
- future.completeExceptionally(MetadataStoreException.wrap(ex.getCause()));
+ if (ex instanceof BadVersionException) {
Review comment:
The code here is handling the put request without `expectedVersion`, that means the put operation should succeed in case of concurrent write. we should not throw BadVersionException here, which means the expected version doesn't match the actual version of the data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org