You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/12/08 13:17:14 UTC
[GitHub] [pulsar] liangyepianzhou opened a new pull request, #18823: [fix][broker] Fix delete system topic clean topic policy
liangyepianzhou opened a new pull request, #18823:
URL: https://github.com/apache/pulsar/pull/18823
### Motivation
If users set topic policy for system topic, then delete this system topic, the topic policy should be deleted.
### Modification
Only change_events topic do not need to clear topic policies.
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
*(Please pick either of the following options)*
This change is a trivial rework / code cleanup without any test coverage.
*(or)*
This change is already covered by existing tests, such as *(please describe tests)*.
*(or)*
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end deployment with large payloads (10MB)*
- *Extended integration test for recovery after broker failure*
### Does this pull request potentially affect one of the following parts:
*If the box was checked, please highlight the changes*
- [ ] Dependencies (add or upgrade a dependency)
- [ ] The public API
- [ ] The schema
- [ ] The default values of configurations
- [ ] The threading model
- [ ] The binary protocol
- [ ] The REST endpoints
- [ ] The admin CLI options
- [ ] Anything that affects deployment
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] `doc` <!-- Your PR contains doc changes. Please attach the local preview screenshots (run `sh start.sh` at `pulsar/site2/website`) to your PR description, or else your PR might not get merged. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
### Matching PR in forked repository
PR in forked repository: https://github.com/liangyepianzhou/pulsar/pull/16
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] congbobo184 merged pull request #18823: [fix][broker] Fix delete system topic clean topic policy
Posted by GitBox <gi...@apache.org>.
congbobo184 merged PR #18823:
URL: https://github.com/apache/pulsar/pull/18823
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] congbobo184 commented on a diff in pull request #18823: [fix][broker] Fix delete system topic clean topic policy
Posted by GitBox <gi...@apache.org>.
congbobo184 commented on code in PR #18823:
URL: https://github.com/apache/pulsar/pull/18823#discussion_r1043995153
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java:
##########
@@ -224,18 +225,28 @@ protected CompletableFuture<Void> internalDeleteNamespaceAsync(boolean force) {
boolean hasNonSystemTopic = false;
List<String> allSystemTopics = new ArrayList<>();
List<String> allPartitionedSystemTopics = new ArrayList<>();
+ List<String> topicPolicy = new ArrayList<>();
+ List<String> partitionedTopicPolicy = new ArrayList<>();
for (String topic : allTopics) {
if (!pulsar().getBrokerService().isSystemTopic(TopicName.get(topic))) {
hasNonSystemTopic = true;
- allUserCreatedTopics.add(topic);
+ if (SystemTopicNames.isTopicPoliciesSystemTopic(topic)) {
+ topicPolicy.add(topic);
Review Comment:
if topic is not systemTopic, it also not the TopicPoliciesSystemTopic right?
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java:
##########
@@ -1221,8 +1221,7 @@ private CompletableFuture<Void> delete(boolean failIfHasSubscriptions,
deleteTopicAuthenticationFuture.thenCompose(ignore -> deleteSchema())
.thenCompose(ignore -> {
- if (!this.getBrokerService().getPulsar().getBrokerService()
- .isSystemTopic(TopicName.get(topic))) {
+ if (!SystemTopicNames.isTopicPoliciesSystemTopic(topic)) {
Review Comment:
topic policy system topic also can delete it topic policy, why we don't clean up the topic policy of the topic policy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on a diff in pull request #18823: [fix][broker] Fix delete system topic clean topic policy
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on code in PR #18823:
URL: https://github.com/apache/pulsar/pull/18823#discussion_r1044120978
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java:
##########
@@ -224,20 +225,30 @@ protected CompletableFuture<Void> internalDeleteNamespaceAsync(boolean force) {
boolean hasNonSystemTopic = false;
List<String> allSystemTopics = new ArrayList<>();
List<String> allPartitionedSystemTopics = new ArrayList<>();
+ List<String> topicPolicy = new ArrayList<>();
Review Comment:
why not named `topicPolicySystemTopic` or `changeEventTopic`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codecov-commenter commented on pull request #18823: [fix][broker] Fix delete system topic clean topic policy
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #18823:
URL: https://github.com/apache/pulsar/pull/18823#issuecomment-1343887377
# [Codecov](https://codecov.io/gh/apache/pulsar/pull/18823?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#18823](https://codecov.io/gh/apache/pulsar/pull/18823?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (82d5817) into [master](https://codecov.io/gh/apache/pulsar/commit/da87e40aca848c0cb1ede7ba56605bdcd5f96137?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (da87e40) will **decrease** coverage by `9.30%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/pulsar/pull/18823?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #18823 +/- ##
============================================
- Coverage 46.42% 37.12% -9.31%
+ Complexity 10437 1971 -8466
============================================
Files 703 209 -494
Lines 68816 14425 -54391
Branches 7377 1574 -5803
============================================
- Hits 31950 5355 -26595
+ Misses 33252 8484 -24768
+ Partials 3614 586 -3028
```
| Flag | Coverage Δ | |
|---|---|---|
| unittests | `37.12% <ø> (-9.31%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pulsar/pull/18823?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...pache/pulsar/broker/admin/impl/NamespacesBase.java](https://codecov.io/gh/apache/pulsar/pull/18823/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9hZG1pbi9pbXBsL05hbWVzcGFjZXNCYXNlLmphdmE=) | | |
| [...ache/pulsar/broker/loadbalance/LinuxInfoUtils.java](https://codecov.io/gh/apache/pulsar/pull/18823/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9sb2FkYmFsYW5jZS9MaW51eEluZm9VdGlscy5qYXZh) | | |
| [...ker/loadbalance/impl/LinuxBrokerHostUsageImpl.java](https://codecov.io/gh/apache/pulsar/pull/18823/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9sb2FkYmFsYW5jZS9pbXBsL0xpbnV4QnJva2VySG9zdFVzYWdlSW1wbC5qYXZh) | | |
| [...ker/resourcegroup/ResourceQuotaCalculatorImpl.java](https://codecov.io/gh/apache/pulsar/pull/18823/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9yZXNvdXJjZWdyb3VwL1Jlc291cmNlUXVvdGFDYWxjdWxhdG9ySW1wbC5qYXZh) | | |
| [.../service/SystemTopicBasedTopicPoliciesService.java](https://codecov.io/gh/apache/pulsar/pull/18823/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL1N5c3RlbVRvcGljQmFzZWRUb3BpY1BvbGljaWVzU2VydmljZS5qYXZh) | | |
| [...sar/broker/service/persistent/PersistentTopic.java](https://codecov.io/gh/apache/pulsar/pull/18823/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL3BlcnNpc3RlbnQvUGVyc2lzdGVudFRvcGljLmphdmE=) | | |
| [...n/java/org/apache/pulsar/broker/PulsarService.java](https://codecov.io/gh/apache/pulsar/pull/18823/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9QdWxzYXJTZXJ2aWNlLmphdmE=) | | |
| [...he/pulsar/broker/admin/v1/NonPersistentTopics.java](https://codecov.io/gh/apache/pulsar/pull/18823/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9hZG1pbi92MS9Ob25QZXJzaXN0ZW50VG9waWNzLmphdmE=) | | |
| [...ar/broker/loadbalance/impl/BundleSplitterTask.java](https://codecov.io/gh/apache/pulsar/pull/18823/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9sb2FkYmFsYW5jZS9pbXBsL0J1bmRsZVNwbGl0dGVyVGFzay5qYXZh) | | |
| [...ache/pulsar/broker/service/PublishRateLimiter.java](https://codecov.io/gh/apache/pulsar/pull/18823/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL1B1Ymxpc2hSYXRlTGltaXRlci5qYXZh) | | |
| ... and [486 more](https://codecov.io/gh/apache/pulsar/pull/18823/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] congbobo184 commented on a diff in pull request #18823: [fix][broker] Fix delete system topic clean topic policy
Posted by GitBox <gi...@apache.org>.
congbobo184 commented on code in PR #18823:
URL: https://github.com/apache/pulsar/pull/18823#discussion_r1044035006
##########
pulsar-broker/src/test/java/org/apache/pulsar/broker/transaction/TransactionTest.java:
##########
@@ -1616,4 +1616,33 @@ public void testGetTxnState() throws Exception {
Transaction abortingTxn = transaction;
Awaitility.await().until(() -> abortingTxn.getState() == Transaction.State.ABORTING);
}
+ @Test
+ public void testDeleteTopicPolicyWhenDeleteSystemTopic() throws Exception {
Review Comment:
please move this test to [NamespacesTest.java](https://github.com/apache/pulsar/pull/18823/files#diff-24788d54292c43e74fe60920b6edd12c0526b6cf5808aec9ac5a134c554db31a)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on a diff in pull request #18823: [fix][broker] Fix delete system topic clean topic policy
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on code in PR #18823:
URL: https://github.com/apache/pulsar/pull/18823#discussion_r1044000930
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java:
##########
@@ -1221,8 +1221,7 @@ private CompletableFuture<Void> delete(boolean failIfHasSubscriptions,
deleteTopicAuthenticationFuture.thenCompose(ignore -> deleteSchema())
.thenCompose(ignore -> {
- if (!this.getBrokerService().getPulsar().getBrokerService()
- .isSystemTopic(TopicName.get(topic))) {
+ if (!SystemTopicNames.isTopicPoliciesSystemTopic(topic)) {
Review Comment:
The topic policy topic will send a message to itself and then report `topic is fenced`.
And delete topic policy topic does not need to delete its policy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- commented on pull request #18823: [fix][broker] Fix delete system topic clean topic policy
Posted by "Technoboy- (via GitHub)" <gi...@apache.org>.
Technoboy- commented on PR #18823:
URL: https://github.com/apache/pulsar/pull/18823#issuecomment-1471812963
Cherry-picked by #19835
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on a diff in pull request #18823: [fix][broker] Fix delete system topic clean topic policy
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on code in PR #18823:
URL: https://github.com/apache/pulsar/pull/18823#discussion_r1044075839
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java:
##########
@@ -224,18 +225,28 @@ protected CompletableFuture<Void> internalDeleteNamespaceAsync(boolean force) {
boolean hasNonSystemTopic = false;
List<String> allSystemTopics = new ArrayList<>();
List<String> allPartitionedSystemTopics = new ArrayList<>();
+ List<String> topicPolicy = new ArrayList<>();
+ List<String> partitionedTopicPolicy = new ArrayList<>();
for (String topic : allTopics) {
if (!pulsar().getBrokerService().isSystemTopic(TopicName.get(topic))) {
hasNonSystemTopic = true;
- allUserCreatedTopics.add(topic);
+ if (SystemTopicNames.isTopicPoliciesSystemTopic(topic)) {
+ topicPolicy.add(topic);
Review Comment:
Right.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #18823: [fix][broker] Fix delete system topic clean topic policy
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on PR #18823:
URL: https://github.com/apache/pulsar/pull/18823#issuecomment-1343898401
Link to
- https://github.com/apache/pulsar/pull/18307
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org