You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/12/16 02:52:36 UTC
[GitHub] [pulsar] thetumbled opened a new pull request, #18949: [fix] [broker] do not filter system topic while shedding.
thetumbled opened a new pull request, #18949:
URL: https://github.com/apache/pulsar/pull/18949
<!-- Either this PR fixes an issue, -->
See this PR for detailed discussion
https://github.com/apache/pulsar/pull/18936
### Motivation
Currently, topics/bundles in `pulsar/system` will be filter while doing shedding, which is introduced by mistake by pr https://github.com/apache/pulsar/pull/15252.
But we need to unload topics/bundles in `pulsar/system` for load balancing.
### Modifications
do not filter topics/bundles in `pulsar/system`.
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
This change is already covered by existing tests, such as *(please describe tests)*.
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] `doc` <!-- Your PR contains doc changes. Please attach the local preview screenshots (run `sh start.sh` at `pulsar/site2/website`) to your PR description, or else your PR might not get merged. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
### Matching PR in forked repository
PR in forked repository: <!-- ENTER URL HERE -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] thetumbled commented on a diff in pull request #18949: [fix] [broker] do not filter system topic while shedding.
Posted by GitBox <gi...@apache.org>.
thetumbled commented on code in PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#discussion_r1051295947
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java:
##########
@@ -1440,6 +1440,17 @@ public static boolean isSystemServiceNamespace(String namespace) {
|| HEARTBEAT_NAMESPACE_PATTERN_V2.matcher(namespace).matches();
}
+ /**
+ * used for filtering bundles in special namespace.
+ * @param namespace
+ * @return True if namespace is HEARTBEAT_NAMESPACE or SLA_NAMESPACE
+ */
+ public static boolean filterNamespaceForShedding(String namespace) {
+ return SLA_NAMESPACE_PATTERN.matcher(namespace).matches()
+ || HEARTBEAT_NAMESPACE_PATTERN.matcher(namespace).matches()
+ || HEARTBEAT_NAMESPACE_PATTERN_V2.matcher(namespace).matches();
+ }
+
Review Comment:
ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #18949: [fix] [broker] do not filter system topic while shedding.
Posted by github-actions.
github-actions[bot] commented on PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#issuecomment-1407254058
The pr had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] thetumbled commented on pull request #18949: [fix] [broker] do not filter system topic while shedding.
Posted by GitBox <gi...@apache.org>.
thetumbled commented on PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#issuecomment-1366313125
PTAL, thanks. @codelipenghui @eolivelli
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #18949: [fix] [broker] do not filter system topic while shedding.
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#issuecomment-1492786918
The pr had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] congbobo184 closed pull request #18949: [fix] [broker] do not filter system topic while shedding.
Posted by "congbobo184 (via GitHub)" <gi...@apache.org>.
congbobo184 closed pull request #18949: [fix] [broker] do not filter system topic while shedding.
URL: https://github.com/apache/pulsar/pull/18949
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] thetumbled commented on pull request #18949: [fix] [broker] do not filter system topic while shedding.
Posted by GitBox <gi...@apache.org>.
thetumbled commented on PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#issuecomment-1354116027
PTAL, thanks. @congbobo184
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on a diff in pull request #18949: [fix] [broker] do not filter system topic while shedding.
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on code in PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#discussion_r1051276223
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java:
##########
@@ -1440,6 +1440,17 @@ public static boolean isSystemServiceNamespace(String namespace) {
|| HEARTBEAT_NAMESPACE_PATTERN_V2.matcher(namespace).matches();
}
+ /**
+ * used for filtering bundles in special namespace.
+ * @param namespace
+ * @return True if namespace is HEARTBEAT_NAMESPACE or SLA_NAMESPACE
+ */
+ public static boolean filterNamespaceForShedding(String namespace) {
+ return SLA_NAMESPACE_PATTERN.matcher(namespace).matches()
+ || HEARTBEAT_NAMESPACE_PATTERN.matcher(namespace).matches()
+ || HEARTBEAT_NAMESPACE_PATTERN_V2.matcher(namespace).matches();
+ }
+
Review Comment:
If it is only used by `LoadData.java`, can we move to the internal of the `LoadData.java`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] congbobo184 closed pull request #18949: [fix] [broker] do not filter system topic while shedding.
Posted by "congbobo184 (via GitHub)" <gi...@apache.org>.
congbobo184 closed pull request #18949: [fix] [broker] do not filter system topic while shedding.
URL: https://github.com/apache/pulsar/pull/18949
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codecov-commenter commented on pull request #18949: [fix] [broker] do not filter system topic while shedding.
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#issuecomment-1355016885
# [Codecov](https://codecov.io/gh/apache/pulsar/pull/18949?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#18949](https://codecov.io/gh/apache/pulsar/pull/18949?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (b30ae90) into [master](https://codecov.io/gh/apache/pulsar/commit/3180a4aa04d518fa401a781d646545221c4d1fa6?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3180a4a) will **increase** coverage by `0.38%`.
> The diff coverage is `52.63%`.
[](https://codecov.io/gh/apache/pulsar/pull/18949?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #18949 +/- ##
============================================
+ Coverage 46.17% 46.55% +0.38%
+ Complexity 10359 8984 -1375
============================================
Files 703 597 -106
Lines 68845 56863 -11982
Branches 7382 5904 -1478
============================================
- Hits 31788 26472 -5316
+ Misses 33448 27541 -5907
+ Partials 3609 2850 -759
```
| Flag | Coverage Δ | |
|---|---|---|
| unittests | `46.55% <52.63%> (+0.38%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pulsar/pull/18949?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../main/java/org/apache/pulsar/PulsarStandalone.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL1B1bHNhclN0YW5kYWxvbmUuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...broker/intercept/ManagedLedgerInterceptorImpl.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9pbnRlcmNlcHQvTWFuYWdlZExlZGdlckludGVyY2VwdG9ySW1wbC5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...org/apache/pulsar/broker/loadbalance/LoadData.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9sb2FkYmFsYW5jZS9Mb2FkRGF0YS5qYXZh) | `58.33% <0.00%> (ø)` | |
| [...ache/pulsar/broker/namespace/NamespaceService.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9uYW1lc3BhY2UvTmFtZXNwYWNlU2VydmljZS5qYXZh) | `58.11% <0.00%> (+0.03%)` | :arrow_up: |
| [...va/org/apache/pulsar/broker/service/ServerCnx.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL1NlcnZlckNueC5qYXZh) | `49.14% <ø> (+1.71%)` | :arrow_up: |
| [...er/service/persistent/GeoPersistentReplicator.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL3BlcnNpc3RlbnQvR2VvUGVyc2lzdGVudFJlcGxpY2F0b3IuamF2YQ==) | `4.00% <0.00%> (-0.71%)` | :arrow_down: |
| [...va/org/apache/pulsar/client/impl/ConsumerImpl.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL0NvbnN1bWVySW1wbC5qYXZh) | `15.09% <0.00%> (-0.04%)` | :arrow_down: |
| [...he/pulsar/client/impl/MultiTopicsConsumerImpl.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL011bHRpVG9waWNzQ29uc3VtZXJJbXBsLmphdmE=) | `22.78% <0.00%> (-0.09%)` | :arrow_down: |
| [...va/org/apache/pulsar/client/impl/ProducerImpl.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1Byb2R1Y2VySW1wbC5qYXZh) | `15.66% <0.00%> (-1.34%)` | :arrow_down: |
| [...nsaction/pendingack/impl/PendingAckHandleImpl.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci90cmFuc2FjdGlvbi9wZW5kaW5nYWNrL2ltcGwvUGVuZGluZ0Fja0hhbmRsZUltcGwuamF2YQ==) | `51.73% <50.00%> (-0.11%)` | :arrow_down: |
| ... and [162 more](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] thetumbled commented on pull request #18949: [fix] [broker] do not filter system topic while shedding.
Posted by "thetumbled (via GitHub)" <gi...@apache.org>.
thetumbled commented on PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#issuecomment-1447673157
PTAL, thanks. @codelipenghui @eolivelli
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] congbobo184 merged pull request #18949: [fix] [broker] do not filter system topic while shedding.
Posted by "congbobo184 (via GitHub)" <gi...@apache.org>.
congbobo184 merged PR #18949:
URL: https://github.com/apache/pulsar/pull/18949
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on a diff in pull request #18949: [fix] [broker] do not filter system topic while shedding.
Posted by GitBox <gi...@apache.org>.
eolivelli commented on code in PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#discussion_r1051893596
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/LoadData.java:
##########
@@ -64,11 +66,22 @@ public Map<String, BundleData> getBundleData() {
public Map<String, BundleData> getBundleDataForLoadShedding() {
return bundleData.entrySet().stream()
- .filter(e -> !NamespaceService.isSystemServiceNamespace(
+ .filter(e -> !filterNamespaceForShedding(
NamespaceBundle.getBundleNamespace(e.getKey())))
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
}
+ /**
+ * used for filtering bundles in special namespace.
+ * @param namespace Namespace
+ * @return True if namespace is HEARTBEAT_NAMESPACE or SLA_NAMESPACE
+ */
+ public static boolean filterNamespaceForShedding(String namespace) {
Review Comment:
we should keep this logic near to NamespaceService.isSystemServiceNamespace
otherwise if we add other special namespaces it will be easy to forget
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] thetumbled commented on a diff in pull request #18949: [fix] [broker] do not filter system topic while shedding.
Posted by GitBox <gi...@apache.org>.
thetumbled commented on code in PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#discussion_r1051896748
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/LoadData.java:
##########
@@ -64,11 +66,22 @@ public Map<String, BundleData> getBundleData() {
public Map<String, BundleData> getBundleDataForLoadShedding() {
return bundleData.entrySet().stream()
- .filter(e -> !NamespaceService.isSystemServiceNamespace(
+ .filter(e -> !filterNamespaceForShedding(
NamespaceBundle.getBundleNamespace(e.getKey())))
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
}
+ /**
+ * used for filtering bundles in special namespace.
+ * @param namespace Namespace
+ * @return True if namespace is HEARTBEAT_NAMESPACE or SLA_NAMESPACE
+ */
+ public static boolean filterNamespaceForShedding(String namespace) {
Review Comment:
it did. Could i move it back to `NamespaceService`? @codelipenghui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org