You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/10/13 16:33:16 UTC
[GitHub] [pulsar] poorbarcode opened a new pull request, #17861: [fix][test] flaky AdminApi2Test.cleanup
poorbarcode opened a new pull request, #17861:
URL: https://github.com/apache/pulsar/pull/17861
Fixes: #16980
Fixes: #17530
- https://github.com/apache/pulsar/issues/16980
- https://github.com/apache/pulsar/issues/17530
### Motivation
When runs `AdminApi2Test.testTopicPoliciesWithMultiBroker` or `BrokerServiceLookupTest.testModularLoadManagerSplitBundle`, you can see some logs like this:
```
2022-09-27T22:21:13,414 - WARN - [metadata-store-12286-1:LeaderElectionImpl@310] - Leadership released for /loadbalance/leader
2022-09-27T22:21:13,418 - INFO - [metadata-store-12319-1:LeaderElectionImpl@183] - Acquired leadership on /loadbalance/leader
2022-09-27T22:21:13,418 - INFO - [metadata-store-12319-1:PulsarService@1067] - This broker was elected leader
2022-09-27T22:21:13,418 - WARN - [metadata-store-12319-1:LeaderElectionImpl@310] - Leadership released for /loadbalance/leader
2022-09-27T22:21:13,422 - INFO - [metadata-store-12286-1:LeaderElectionImpl@183] - Acquired leadership on /loadbalance/leader
2022-09-27T22:21:13,422 - INFO - [metadata-store-12286-1:PulsarService@1067] - This broker was elected leader
2022-09-27T22:21:13,423 - WARN - [metadata-store-12286-1:LeaderElectionImpl@310] - Leadership released for /loadbalance/leader
2022-09-27T22:21:13,428 - INFO - [metadata-store-12319-1:LeaderElectionImpl@183] - Acquired leadership on /loadbalance/leader
2022-09-27T22:21:13,428 - INFO - [metadata-store-12319-1:PulsarService@1067] - This broker was elected leader
2022-09-27T22:21:13,428 - WARN - [metadata-store-12319-1:LeaderElectionImpl@310] - Leadership released for /loadbalance/leader
2022-09-27T22:21:13,433 - INFO - [metadata-store-12286-1:LeaderElectionImpl@183] - Acquired leadership on /loadbalance/leader
2022-09-27T22:21:13,433 - INFO - [metadata-store-12286-1:PulsarService@1067] - This broker was elected leader
2022-09-27T22:21:13,433 - WARN - [metadata-store-12286-1:LeaderElectionImpl@310] - Leadership released for /loadbalance/leader
2022-09-27T22:21:13,434 - INFO - [TestNG-method=testModularLoadManagerSplitBundle-1:OwnershipCache@193] - Trying to acquire ownership of pulsar/test/localhost:62802/0x00000000_0xffffffff
2022-09-27T22:21:13,438 - INFO - [metadata-store-12319-1:LeaderElectionImpl@183] - Acquired leadership on /loadbalance/leader
2022-09-27T22:21:13,438 - INFO - [metadata-store-12319-1:PulsarService@1067] - This broker was elected leader
2022-09-27T22:21:13,438 - WARN - [metadata-store-12319-1:LeaderElectionImpl@310] - Leadership released for /loadbalance/leader
....
```
#### Location problem
These test uses `setup()` and `PulsarService pulsar_new = startBroker(conf_new)` to create two brokers, but the both brokers use the same `zkc`, so both brokers will share the `session` of `ZK`, which will lead to the possibility that the two brokers will constantly elect for the leader. see code:
https://github.com/apache/pulsar/blob/b89c1451551a6bbe681465726906a2e61c9d8a69/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/coordination/impl/LeaderElectionImpl.java#L143-L147
#### Impact
- `BrokerServiceLookupTest`
```
constantly elect for the leader makes instability of bundle assign
error occurs
```
- `AdminApi2Test.cleanup`
```
close PulsarService
close LeaderElectionService
delete zk-node `/rebalance/leader`, but this node has concurrently deleted by elect-thread
error occurs
```
### Modifications
- Make two brokers use different `session`
### Documentation
- [x] `doc-not-needed`
### Matching PR in forked repository
PR in forked repository:
- https://github.com/poorbarcode/pulsar/pull/15
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #17861: [fix][test] flaky AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on PR #17861:
URL: https://github.com/apache/pulsar/pull/17861#issuecomment-1278107975
/pulsarbot rerun-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode closed pull request #17861: [fix][test] flaky AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
poorbarcode closed pull request #17861: [fix][test] flaky AdminApi2Test.cleanup
URL: https://github.com/apache/pulsar/pull/17861
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #17861: [fix][test] flaky AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on PR #17861:
URL: https://github.com/apache/pulsar/pull/17861#issuecomment-1274160277
Can this PR merge? (^_^)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #17861: [fix][flaky-test]AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on PR #17861:
URL: https://github.com/apache/pulsar/pull/17861#issuecomment-1261995892
Hi @lhotari @nicoloboschi
Could you take a look? Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode closed pull request #17861: [fix][flaky-test]AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
poorbarcode closed pull request #17861: [fix][flaky-test]AdminApi2Test.cleanup
URL: https://github.com/apache/pulsar/pull/17861
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- merged pull request #17861: [fix][test] flaky AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
Technoboy- merged PR #17861:
URL: https://github.com/apache/pulsar/pull/17861
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #17861: [fix][test] AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on PR #17861:
URL: https://github.com/apache/pulsar/pull/17861#issuecomment-1274045950
Hi @tisonkun
Got it! Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codecov-commenter commented on pull request #17861: [fix][test] flaky AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #17861:
URL: https://github.com/apache/pulsar/pull/17861#issuecomment-1275607059
# [Codecov](https://codecov.io/gh/apache/pulsar/pull/17861?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> :exclamation: No coverage uploaded for pull request base (`master@882fcfb`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#section-missing-base-commit).
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## master #17861 +/- ##
=========================================
Coverage ? 70.92%
Complexity ? 437
=========================================
Files ? 26
Lines ? 2246
Branches ? 245
=========================================
Hits ? 1593
Misses ? 480
Partials ? 173
```
| Flag | Coverage Δ | |
|---|---|---|
| unittests | `70.92% <0.00%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode closed pull request #17861: [fix][flaky-test]AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
poorbarcode closed pull request #17861: [fix][flaky-test]AdminApi2Test.cleanup
URL: https://github.com/apache/pulsar/pull/17861
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #17861: [fix][test] flaky AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on PR #17861:
URL: https://github.com/apache/pulsar/pull/17861#issuecomment-1276212776
/pulsarbot rerun-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on a diff in pull request #17861: [fix][test] flaky AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on code in PR #17861:
URL: https://github.com/apache/pulsar/pull/17861#discussion_r992925720
##########
pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApi2Test.java:
##########
@@ -147,6 +152,20 @@ protected ServiceConfiguration getDefaultConf() {
return conf;
}
+ @Override
+ protected MetadataStoreExtended createConfigurationMetadataStore() throws MetadataStoreException {
+ // use MockZooKeeperSession to provide a unique session id for each instance
+ return new ZKMetadataStore(MockZooKeeperSession.newInstance(mockZooKeeperGlobal), MetadataStoreConfig.builder()
Review Comment:
Good idea, already changed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #17861: [fix][test] flaky AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on PR #17861:
URL: https://github.com/apache/pulsar/pull/17861#issuecomment-1275683816
/pulsarbot rerun-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode closed pull request #17861: [fix][flaky-test]AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
poorbarcode closed pull request #17861: [fix][flaky-test]AdminApi2Test.cleanup
URL: https://github.com/apache/pulsar/pull/17861
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] heesung-sn commented on pull request #17861: [fix][flaky-test]BrokerServiceLookupTest.testModularLoadManagerSplitBundle
Posted by GitBox <gi...@apache.org>.
heesung-sn commented on PR #17861:
URL: https://github.com/apache/pulsar/pull/17861#issuecomment-1259844340
LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #17861: [fix][flaky-test]AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on PR #17861:
URL: https://github.com/apache/pulsar/pull/17861#issuecomment-1272723559
rebase master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun commented on pull request #17861: fix [flaky-test]AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
tisonkun commented on PR #17861:
URL: https://github.com/apache/pulsar/pull/17861#issuecomment-1274044414
@poorbarcode you may change the title to `[fix][test] flaky AdminApi2Test.cleanup`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #17861: [fix][flaky-test]AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on PR #17861:
URL: https://github.com/apache/pulsar/pull/17861#issuecomment-1274041550
rebase master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nicoloboschi commented on a diff in pull request #17861: [fix][test] flaky AdminApi2Test.cleanup
Posted by GitBox <gi...@apache.org>.
nicoloboschi commented on code in PR #17861:
URL: https://github.com/apache/pulsar/pull/17861#discussion_r991915473
##########
pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApi2Test.java:
##########
@@ -147,6 +152,20 @@ protected ServiceConfiguration getDefaultConf() {
return conf;
}
+ @Override
+ protected MetadataStoreExtended createConfigurationMetadataStore() throws MetadataStoreException {
+ // use MockZooKeeperSession to provide a unique session id for each instance
+ return new ZKMetadataStore(MockZooKeeperSession.newInstance(mockZooKeeperGlobal), MetadataStoreConfig.builder()
Review Comment:
it shouldn't be better to make this the default behavior in `MockedPulsarServiceBaseTest ` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org