[GitHub] [pulsar] merajblueshift commented on issue #3716: BookKeeper rack-aware placement policies

[GitBox <gi...@apache.org>]

merajblueshift commented on issue #3716:
URL: https://github.com/apache/pulsar/issues/3716#issuecomment-649437938


   @sijie 
   
   I am able to use Script-based policy on Bookie by setting the script path in bookkeeper.conf.
   
   However in the broker, while initializing the EnsemblePlacementPolicy it still puts every bookie in default-rack. When I checked the code I saw the following behavior
   https://github.com/apache/pulsar/blob/24cc4bbb0a0b488ef23ede3ba5e914b96d188a5a/pulsar-broker/src/main/java/org/apache/pulsar/broker/BookKeeperClientFactoryImpl.java
   
   ```
   if (conf.isBookkeeperClientRackawarePolicyEnabled() || conf.isBookkeeperClientRegionawarePolicyEnabled()) {
               if (conf.isBookkeeperClientRegionawarePolicyEnabled()) {
                   bkConf.setEnsemblePlacementPolicy(RegionAwareEnsemblePlacementPolicy.class);
               } else {
                   bkConf.setEnsemblePlacementPolicy(RackawareEnsemblePlacementPolicy.class);
               }
               bkConf.setProperty(RackawareEnsemblePlacementPolicy.REPP_DNS_RESOLVER_CLASS,
                       ZkBookieRackAffinityMapping.class.getName());
   
               ZooKeeperCache zkc = new ZooKeeperCache(zkClient) {
               };
               if (!rackawarePolicyZkCache.compareAndSet(null, zkc)) {
                   zkc.stop();
               }
   ```
   
   I see that the REPP_DNS_RESOLVER_CLASS is set to ZkBookieRackAffinityMapping. 
   
   Can you guys provide an option through the broker config to override this class? I will actually like to set it to 
   org.apache.bookkeeper.net.ScriptBasedMapping
   In order to use the above class and additional config parameter will be required asking for the script path.
   
   So overall the following two parameters need to be exposed via broker config
   **reppDnsResolverClass** The default can be set to ZkBookieRackAffinityMapping.
    **networkTopologyScriptFileName** This will be used if the above parameter value is **org.apache.bookkeeper.net.ScriptBasedMapping**
   
   These changes will make it very easy for anyone to use custom rack awareness script rather than setting the values via REST API. Doing this via REST API is a bit difficult in an environment where bookies are scaled up and down.
             


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org