You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/12/16 02:52:36 UTC

[GitHub] [pulsar] thetumbled opened a new pull request, #18949: [fix] [broker] do not filter system topic while shedding.

thetumbled opened a new pull request, #18949:
URL: https://github.com/apache/pulsar/pull/18949

   
   <!-- Either this PR fixes an issue, -->
   
   See this PR for detailed discussion
   https://github.com/apache/pulsar/pull/18936
   
   
   ### Motivation
   
   Currently, topics/bundles in `pulsar/system` will be filter while doing shedding, which is introduced by mistake by pr https://github.com/apache/pulsar/pull/15252.
   But we need to unload topics/bundles in `pulsar/system` for load balancing. 
   
   ### Modifications
   
   do not filter topics/bundles in `pulsar/system`. 
   
   ### Verifying this change
   
   - [ ] Make sure that the change passes the CI checks.
   
   This change is already covered by existing tests, such as *(please describe tests)*.
   
   
   ### Documentation
   
   <!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
   
   - [ ] `doc` <!-- Your PR contains doc changes. Please attach the local preview screenshots (run `sh start.sh` at `pulsar/site2/website`) to your PR description, or else your PR might not get merged. -->
   - [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
   - [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
   - [ ] `doc-complete` <!-- Docs have been already added -->
   
   ### Matching PR in forked repository
   
   PR in forked repository: <!-- ENTER URL HERE -->
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] thetumbled commented on a diff in pull request #18949: [fix] [broker] do not filter system topic while shedding.

Posted by GitBox <gi...@apache.org>.
thetumbled commented on code in PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#discussion_r1051295947


##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java:
##########
@@ -1440,6 +1440,17 @@ public static boolean isSystemServiceNamespace(String namespace) {
                 || HEARTBEAT_NAMESPACE_PATTERN_V2.matcher(namespace).matches();
     }
 
+    /**
+     * used for filtering bundles in special namespace.
+     * @param namespace
+     * @return True if namespace is HEARTBEAT_NAMESPACE or SLA_NAMESPACE
+     */
+    public static boolean filterNamespaceForShedding(String namespace) {
+        return SLA_NAMESPACE_PATTERN.matcher(namespace).matches()
+                || HEARTBEAT_NAMESPACE_PATTERN.matcher(namespace).matches()
+                || HEARTBEAT_NAMESPACE_PATTERN_V2.matcher(namespace).matches();
+    }
+

Review Comment:
   ok. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] github-actions[bot] commented on pull request #18949: [fix] [broker] do not filter system topic while shedding.

Posted by github-actions.
github-actions[bot] commented on PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#issuecomment-1407254058

   The pr had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] thetumbled commented on pull request #18949: [fix] [broker] do not filter system topic while shedding.

Posted by GitBox <gi...@apache.org>.
thetumbled commented on PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#issuecomment-1366313125

   PTAL, thanks. @codelipenghui @eolivelli 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] github-actions[bot] commented on pull request #18949: [fix] [broker] do not filter system topic while shedding.

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#issuecomment-1492786918

   The pr had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] congbobo184 closed pull request #18949: [fix] [broker] do not filter system topic while shedding.

Posted by "congbobo184 (via GitHub)" <gi...@apache.org>.
congbobo184 closed pull request #18949: [fix] [broker] do not filter system topic while shedding.
URL: https://github.com/apache/pulsar/pull/18949


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] thetumbled commented on pull request #18949: [fix] [broker] do not filter system topic while shedding.

Posted by GitBox <gi...@apache.org>.
thetumbled commented on PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#issuecomment-1354116027

   PTAL, thanks. @congbobo184 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] codelipenghui commented on a diff in pull request #18949: [fix] [broker] do not filter system topic while shedding.

Posted by GitBox <gi...@apache.org>.
codelipenghui commented on code in PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#discussion_r1051276223


##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java:
##########
@@ -1440,6 +1440,17 @@ public static boolean isSystemServiceNamespace(String namespace) {
                 || HEARTBEAT_NAMESPACE_PATTERN_V2.matcher(namespace).matches();
     }
 
+    /**
+     * used for filtering bundles in special namespace.
+     * @param namespace
+     * @return True if namespace is HEARTBEAT_NAMESPACE or SLA_NAMESPACE
+     */
+    public static boolean filterNamespaceForShedding(String namespace) {
+        return SLA_NAMESPACE_PATTERN.matcher(namespace).matches()
+                || HEARTBEAT_NAMESPACE_PATTERN.matcher(namespace).matches()
+                || HEARTBEAT_NAMESPACE_PATTERN_V2.matcher(namespace).matches();
+    }
+

Review Comment:
   If it is only used by `LoadData.java`, can we move to the internal of the `LoadData.java`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] congbobo184 closed pull request #18949: [fix] [broker] do not filter system topic while shedding.

Posted by "congbobo184 (via GitHub)" <gi...@apache.org>.
congbobo184 closed pull request #18949: [fix] [broker] do not filter system topic while shedding.
URL: https://github.com/apache/pulsar/pull/18949


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] codecov-commenter commented on pull request #18949: [fix] [broker] do not filter system topic while shedding.

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#issuecomment-1355016885

   # [Codecov](https://codecov.io/gh/apache/pulsar/pull/18949?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#18949](https://codecov.io/gh/apache/pulsar/pull/18949?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (b30ae90) into [master](https://codecov.io/gh/apache/pulsar/commit/3180a4aa04d518fa401a781d646545221c4d1fa6?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3180a4a) will **increase** coverage by `0.38%`.
   > The diff coverage is `52.63%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pulsar/pull/18949/graphs/tree.svg?width=650&height=150&src=pr&token=acYqCpsK9J&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pulsar/pull/18949?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #18949      +/-   ##
   ============================================
   + Coverage     46.17%   46.55%   +0.38%     
   + Complexity    10359     8984    -1375     
   ============================================
     Files           703      597     -106     
     Lines         68845    56863   -11982     
     Branches       7382     5904    -1478     
   ============================================
   - Hits          31788    26472    -5316     
   + Misses        33448    27541    -5907     
   + Partials       3609     2850     -759     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `46.55% <52.63%> (+0.38%)` | :arrow_up: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pulsar/pull/18949?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [.../main/java/org/apache/pulsar/PulsarStandalone.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL1B1bHNhclN0YW5kYWxvbmUuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...broker/intercept/ManagedLedgerInterceptorImpl.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9pbnRlcmNlcHQvTWFuYWdlZExlZGdlckludGVyY2VwdG9ySW1wbC5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...org/apache/pulsar/broker/loadbalance/LoadData.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9sb2FkYmFsYW5jZS9Mb2FkRGF0YS5qYXZh) | `58.33% <0.00%> (ø)` | |
   | [...ache/pulsar/broker/namespace/NamespaceService.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9uYW1lc3BhY2UvTmFtZXNwYWNlU2VydmljZS5qYXZh) | `58.11% <0.00%> (+0.03%)` | :arrow_up: |
   | [...va/org/apache/pulsar/broker/service/ServerCnx.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL1NlcnZlckNueC5qYXZh) | `49.14% <ø> (+1.71%)` | :arrow_up: |
   | [...er/service/persistent/GeoPersistentReplicator.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL3BlcnNpc3RlbnQvR2VvUGVyc2lzdGVudFJlcGxpY2F0b3IuamF2YQ==) | `4.00% <0.00%> (-0.71%)` | :arrow_down: |
   | [...va/org/apache/pulsar/client/impl/ConsumerImpl.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL0NvbnN1bWVySW1wbC5qYXZh) | `15.09% <0.00%> (-0.04%)` | :arrow_down: |
   | [...he/pulsar/client/impl/MultiTopicsConsumerImpl.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL011bHRpVG9waWNzQ29uc3VtZXJJbXBsLmphdmE=) | `22.78% <0.00%> (-0.09%)` | :arrow_down: |
   | [...va/org/apache/pulsar/client/impl/ProducerImpl.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2NsaWVudC9pbXBsL1Byb2R1Y2VySW1wbC5qYXZh) | `15.66% <0.00%> (-1.34%)` | :arrow_down: |
   | [...nsaction/pendingack/impl/PendingAckHandleImpl.java](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci90cmFuc2FjdGlvbi9wZW5kaW5nYWNrL2ltcGwvUGVuZGluZ0Fja0hhbmRsZUltcGwuamF2YQ==) | `51.73% <50.00%> (-0.11%)` | :arrow_down: |
   | ... and [162 more](https://codecov.io/gh/apache/pulsar/pull/18949/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] thetumbled commented on pull request #18949: [fix] [broker] do not filter system topic while shedding.

Posted by "thetumbled (via GitHub)" <gi...@apache.org>.
thetumbled commented on PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#issuecomment-1447673157

   PTAL, thanks. @codelipenghui @eolivelli


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] congbobo184 merged pull request #18949: [fix] [broker] do not filter system topic while shedding.

Posted by "congbobo184 (via GitHub)" <gi...@apache.org>.
congbobo184 merged PR #18949:
URL: https://github.com/apache/pulsar/pull/18949


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] eolivelli commented on a diff in pull request #18949: [fix] [broker] do not filter system topic while shedding.

Posted by GitBox <gi...@apache.org>.
eolivelli commented on code in PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#discussion_r1051893596


##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/LoadData.java:
##########
@@ -64,11 +66,22 @@ public Map<String, BundleData> getBundleData() {
 
     public Map<String, BundleData> getBundleDataForLoadShedding() {
         return bundleData.entrySet().stream()
-                .filter(e -> !NamespaceService.isSystemServiceNamespace(
+                .filter(e -> !filterNamespaceForShedding(
                         NamespaceBundle.getBundleNamespace(e.getKey())))
                 .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
     }
 
+    /**
+     * used for filtering bundles in special namespace.
+     * @param namespace Namespace
+     * @return True if namespace is HEARTBEAT_NAMESPACE or SLA_NAMESPACE
+     */
+    public static boolean filterNamespaceForShedding(String namespace) {

Review Comment:
   we should keep this logic near to NamespaceService.isSystemServiceNamespace
   otherwise if we add other special namespaces it will be easy to forget



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] thetumbled commented on a diff in pull request #18949: [fix] [broker] do not filter system topic while shedding.

Posted by GitBox <gi...@apache.org>.
thetumbled commented on code in PR #18949:
URL: https://github.com/apache/pulsar/pull/18949#discussion_r1051896748


##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/LoadData.java:
##########
@@ -64,11 +66,22 @@ public Map<String, BundleData> getBundleData() {
 
     public Map<String, BundleData> getBundleDataForLoadShedding() {
         return bundleData.entrySet().stream()
-                .filter(e -> !NamespaceService.isSystemServiceNamespace(
+                .filter(e -> !filterNamespaceForShedding(
                         NamespaceBundle.getBundleNamespace(e.getKey())))
                 .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
     }
 
+    /**
+     * used for filtering bundles in special namespace.
+     * @param namespace Namespace
+     * @return True if namespace is HEARTBEAT_NAMESPACE or SLA_NAMESPACE
+     */
+    public static boolean filterNamespaceForShedding(String namespace) {

Review Comment:
   it did. Could i move it back to `NamespaceService`? @codelipenghui 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org