You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/04/06 05:22:49 UTC

[GitHub] [pulsar] Demogorgon314 opened a new issue, #15037: PIP-151: Use the system topic to store the bundle load data

Demogorgon314 opened a new issue, #15037:
URL: https://github.com/apache/pulsar/issues/15037

   ## Motivation
   
   Currently, Pulsar LoadManager is using Zookeeper to store the bundle load data, when we have many bundles, this might put a lot of pressure on Zookeeper.
   
   ## Goal
   
   This PIP proposes storing the load manager's bundle load data to a system topic and using TableView to read it.
   
   
   
   ## Implementation
   
   Since the bundle load data is stats data, it no needs a strong consistent guarantee. So we can use the system topic to store the load data and use TableView to read it.
   
   
   ### System topic client
   
   
   
   Add a new SystemTopicClient calls `LoadBalanceBundleDataSystemTopicClient` , the topic name is `persistent://pulsar/system/__load_balance_bundle_data`, and the key is the bundle name, value is BundleData.
   
   ```java
   public class LoadBalanceBundleDataSystemTopicClient extends SystemTopicClientBase<LoadBalanceBundleDataEvent> {
   // ...
   }
   ```
   
   
   
   Add new Event calls `LoadBalanceBundleDataEvent`
   
   ```
   @Data
   public class LoadBalanceBundleDataEvent {
   
       private String bundle;
   
       private BundleData bundleData;
   }
   
   ```
   
   
   ### ModularLoadManagerImpl
   
   
   Add a TableView in `ModularLoadManagerImpl` to replace the bundle cache
   
   ```
   private TableView<String, BundleData> bundlesTableView;
   ```
   
   
   ## Compatibility
   
   This feature can have both backward and forward compatibility since the bundle data is stats data.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] PIP-151: Use the system topic to store the bundle load data [pulsar]

Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on issue #15037:
URL: https://github.com/apache/pulsar/issues/15037#issuecomment-6377580084

   Thanks @Demogorgon314 for the proposal. PIP-151 was discussed on dev@ (https://lists.apache.org/thread/ww2khbc109mwlchk2wyo2mpmonp3f09k) but never went to a vote. Its goal was later covered by PIP-192, the extensible load manager (#16691, pip/pip-192.md), which keeps bundle load data in the `loadbalancer-top-bundles-load-data` system topic instead of the metadata store. I'm closing this as not planned, superseded by PIP-192. Anyone is welcome to restart it for the modular load manager as a pip/ document following pip/README.md.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] github-actions[bot] commented on issue #15037: PIP-151: Use the system topic to store the bundle load data

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #15037:
URL: https://github.com/apache/pulsar/issues/15037#issuecomment-1152835564

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] github-actions[bot] commented on issue #15037: PIP-151: Use the system topic to store the bundle load data

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #15037:
URL: https://github.com/apache/pulsar/issues/15037#issuecomment-1124458716

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] PIP-151: Use the system topic to store the bundle load data [pulsar]

Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat closed issue #15037: PIP-151: Use the system topic to store the bundle load data
URL: https://github.com/apache/pulsar/issues/15037


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org