You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by zh...@apache.org on 2018/12/13 04:37:29 UTC

[pulsar] branch master updated: Add bookkeeperClientRegionawarePolicyEnabled and bookkeeperClientReor… (#3171)

This is an automated email from the ASF dual-hosted git repository.

zhaijia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git


The following commit(s) were added to refs/heads/master by this push:
     new 24cc4bb  Add bookkeeperClientRegionawarePolicyEnabled and bookkeeperClientReor… (#3171)
24cc4bb is described below

commit 24cc4bbb0a0b488ef23ede3ba5e914b96d188a5a
Author: Christophe Bornet <cb...@hotmail.com>
AuthorDate: Thu Dec 13 05:37:25 2018 +0100

    Add bookkeeperClientRegionawarePolicyEnabled and bookkeeperClientReor… (#3171)
    
    ## Motivation
    Fix #3119. This allows to configure region-aware policy and read-reordering so that brokers first read on bookies of their own region.
    
    ## Modifications
    1. Added parameters:
    ```
    // Enable region-aware bookie selection policy. BK will chose bookies from
    // different regions and racks when forming a new bookie ensemble
    // If enabled, the value of bookkeeperClientRackawarePolicyEnabled is ignored
    bookkeeperClientRegionawarePolicyEnabled=false
    
    // Enable/disable reordering read sequence on reading entries.
    bookkeeperClientReorderReadSequenceEnabled=false
    ```
    
    2. Fixed bug in ZkBookieRackAffinityMapping: the value set to racksWithHost by deserialize() was overriden by the affectation in setConf(). The fix just moves the hostname workaround in setConf().
    
    ## Result
    Users can enable bookkeeperClientRegionawarePolicyEnabled and bookkeeperClientReorderReadSequenceEnabled to make brokers read on bookies of their own region
---
 conf/broker.conf                                   |  8 ++++
 conf/standalone.conf                               |  8 ++++
 deployment/terraform-ansible/templates/broker.conf |  8 ++++
 .../apache/pulsar/broker/ServiceConfiguration.java |  5 +++
 .../pulsar/broker/BookKeeperClientFactoryImpl.java | 10 ++++-
 .../org/apache/pulsar/PulsarBrokerStarterTest.java |  4 ++
 .../configurations/pulsar_broker_test.conf         |  2 +
 pulsar-client-cpp/test-conf/standalone-ssl.conf    |  8 ++++
 pulsar-client-cpp/test-conf/standalone.conf        |  8 ++++
 pulsar-client-cpp/tests/authentication.conf        |  8 ++++
 pulsar-client-cpp/tests/standalone.conf            |  8 ++++
 .../zookeeper/ZkBookieRackAffinityMapping.java     | 48 +++++++++++-----------
 .../zookeeper/ZkBookieRackAffinityMappingTest.java |  6 +--
 site/_data/config/broker.yaml                      |  6 +++
 site/_data/config/standalone.yaml                  |  6 +++
 site2/docs/reference-configuration.md              |  4 ++
 16 files changed, 118 insertions(+), 29 deletions(-)

diff --git a/conf/broker.conf b/conf/broker.conf
index 71f14a2..85bbb26 100644
--- a/conf/broker.conf
+++ b/conf/broker.conf
@@ -351,6 +351,14 @@ bookkeeperClientHealthCheckQuarantineTimeInSeconds=1800
 # forming a new bookie ensemble
 bookkeeperClientRackawarePolicyEnabled=true
 
+# Enable region-aware bookie selection policy. BK will chose bookies from
+# different regions and racks when forming a new bookie ensemble
+# If enabled, the value of bookkeeperClientRackawarePolicyEnabled is ignored
+bookkeeperClientRegionawarePolicyEnabled=false
+
+# Enable/disable reordering read sequence on reading entries.
+bookkeeperClientReorderReadSequenceEnabled=false
+
 # Enable bookie isolation by specifying a list of bookie groups to choose from. Any bookie
 # outside the specified groups will not be used by the broker
 bookkeeperClientIsolationGroups=
diff --git a/conf/standalone.conf b/conf/standalone.conf
index cc8f564..7085054 100644
--- a/conf/standalone.conf
+++ b/conf/standalone.conf
@@ -245,6 +245,14 @@ bookkeeperClientHealthCheckQuarantineTimeInSeconds=1800
 # forming a new bookie ensemble
 bookkeeperClientRackawarePolicyEnabled=true
 
+# Enable region-aware bookie selection policy. BK will chose bookies from
+# different regions and racks when forming a new bookie ensemble.
+# If enabled, the value of bookkeeperClientRackawarePolicyEnabled is ignored
+bookkeeperClientRegionawarePolicyEnabled=false
+
+# Enable/disable reordering read sequence on reading entries.
+bookkeeperClientReorderReadSequenceEnabled=false
+
 # Enable bookie isolation by specifying a list of bookie groups to choose from. Any bookie
 # outside the specified groups will not be used by the broker
 bookkeeperClientIsolationGroups=
diff --git a/deployment/terraform-ansible/templates/broker.conf b/deployment/terraform-ansible/templates/broker.conf
index a4fad1c..22f74be 100644
--- a/deployment/terraform-ansible/templates/broker.conf
+++ b/deployment/terraform-ansible/templates/broker.conf
@@ -280,6 +280,14 @@ bookkeeperClientHealthCheckQuarantineTimeInSeconds=1800
 # forming a new bookie ensemble
 bookkeeperClientRackawarePolicyEnabled=true
 
+# Enable region-aware bookie selection policy. BK will chose bookies from
+# different regions and racks when forming a new bookie ensemble
+# If enabled, the value of bookkeeperClientRackawarePolicyEnabled is ignored
+bookkeeperClientRegionawarePolicyEnabled=false
+
+# Enable/disable reordering read sequence on reading entries.
+bookkeeperClientReorderReadSequenceEnabled=false
+
 # Enable bookie isolation by specifying a list of bookie groups to choose from. Any bookie
 # outside the specified groups will not be used by the broker
 bookkeeperClientIsolationGroups=
diff --git a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
index bb8822f..6e7b168 100644
--- a/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
+++ b/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
@@ -310,6 +310,11 @@ public class ServiceConfiguration implements PulsarConfiguration {
     // Enable rack-aware bookie selection policy. BK will chose bookies from
     // different racks when forming a new bookie ensemble
     private boolean bookkeeperClientRackawarePolicyEnabled = true;
+    // Enable region-aware bookie selection policy. BK will chose bookies from
+    // different regions and racks when forming a new bookie ensemble
+    private boolean bookkeeperClientRegionawarePolicyEnabled = false;
+    // Enable/disable reordering read sequence on reading entries.
+    private boolean bookkeeperClientReorderReadSequenceEnabled = false;
     // Enable bookie isolation by specifying a list of bookie groups to choose
     // from. Any bookie outside the specified groups will not be used by the
     // broker
diff --git a/pulsar-broker/src/main/java/org/apache/pulsar/broker/BookKeeperClientFactoryImpl.java b/pulsar-broker/src/main/java/org/apache/pulsar/broker/BookKeeperClientFactoryImpl.java
index 2231f97..61f4feb 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/BookKeeperClientFactoryImpl.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/BookKeeperClientFactoryImpl.java
@@ -25,6 +25,7 @@ import java.util.concurrent.atomic.AtomicReference;
 import org.apache.bookkeeper.client.BKException;
 import org.apache.bookkeeper.client.BookKeeper;
 import org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicy;
+import org.apache.bookkeeper.client.RegionAwareEnsemblePlacementPolicy;
 import org.apache.bookkeeper.conf.ClientConfiguration;
 import org.apache.bookkeeper.meta.HierarchicalLedgerManagerFactory;
 import org.apache.pulsar.zookeeper.ZkBookieRackAffinityMapping;
@@ -64,8 +65,12 @@ public class BookKeeperClientFactoryImpl implements BookKeeperClientFactory {
                     TimeUnit.SECONDS);
         }
 
-        if (conf.isBookkeeperClientRackawarePolicyEnabled()) {
-            bkConf.setEnsemblePlacementPolicy(RackawareEnsemblePlacementPolicy.class);
+        if (conf.isBookkeeperClientRackawarePolicyEnabled() || conf.isBookkeeperClientRegionawarePolicyEnabled()) {
+            if (conf.isBookkeeperClientRegionawarePolicyEnabled()) {
+                bkConf.setEnsemblePlacementPolicy(RegionAwareEnsemblePlacementPolicy.class);
+            } else {
+                bkConf.setEnsemblePlacementPolicy(RackawareEnsemblePlacementPolicy.class);
+            }
             bkConf.setProperty(RackawareEnsemblePlacementPolicy.REPP_DNS_RESOLVER_CLASS,
                     ZkBookieRackAffinityMapping.class.getName());
 
@@ -77,6 +82,7 @@ public class BookKeeperClientFactoryImpl implements BookKeeperClientFactory {
 
             bkConf.setProperty(ZooKeeperCache.ZK_CACHE_INSTANCE, this.rackawarePolicyZkCache.get());
         }
+        bkConf.setReorderReadSequenceEnabled(conf.isBookkeeperClientReorderReadSequenceEnabled());
 
         if (conf.getBookkeeperClientIsolationGroups() != null && !conf.getBookkeeperClientIsolationGroups().isEmpty()) {
             bkConf.setEnsemblePlacementPolicy(ZkIsolatedBookieEnsemblePlacementPolicy.class);
diff --git a/pulsar-broker/src/test/java/org/apache/pulsar/PulsarBrokerStarterTest.java b/pulsar-broker/src/test/java/org/apache/pulsar/PulsarBrokerStarterTest.java
index 9cce6db..5ccea59 100644
--- a/pulsar-broker/src/test/java/org/apache/pulsar/PulsarBrokerStarterTest.java
+++ b/pulsar-broker/src/test/java/org/apache/pulsar/PulsarBrokerStarterTest.java
@@ -59,6 +59,8 @@ public class PulsarBrokerStarterTest {
         printWriter.println("bookkeeperClientHealthCheckEnabled=true");
         printWriter.println("bookkeeperClientHealthCheckErrorThresholdPerInterval=5");
         printWriter.println("bookkeeperClientRackawarePolicyEnabled=true");
+        printWriter.println("bookkeeperClientRegionawarePolicyEnabled=false");
+        printWriter.println("bookkeeperClientReorderReadSequenceEnabled=false");
         printWriter.println("bookkeeperClientIsolationGroups=group1,group2");
         printWriter.println("backlogQuotaDefaultLimitGB=18");
         printWriter.println("clusterName=usc");
@@ -120,6 +122,8 @@ public class PulsarBrokerStarterTest {
         Assert.assertEquals(serviceConfig.isBookkeeperClientHealthCheckEnabled(), true);
         Assert.assertEquals(serviceConfig.getBookkeeperClientHealthCheckErrorThresholdPerInterval(), 5);
         Assert.assertEquals(serviceConfig.isBookkeeperClientRackawarePolicyEnabled(), true);
+        Assert.assertEquals(serviceConfig.isBookkeeperClientRegionawarePolicyEnabled(), false);
+        Assert.assertEquals(serviceConfig.isBookkeeperClientReorderReadSequenceEnabled(), false);
         Assert.assertEquals(serviceConfig.getBookkeeperClientIsolationGroups(), "group1,group2");
         Assert.assertEquals(serviceConfig.getBookkeeperClientSpeculativeReadTimeoutInMillis(), 3000);
         Assert.assertEquals(serviceConfig.getBookkeeperClientTimeoutInSeconds(), 12345);
diff --git a/pulsar-broker/src/test/resources/configurations/pulsar_broker_test.conf b/pulsar-broker/src/test/resources/configurations/pulsar_broker_test.conf
index 8034185..c55955f 100644
--- a/pulsar-broker/src/test/resources/configurations/pulsar_broker_test.conf
+++ b/pulsar-broker/src/test/resources/configurations/pulsar_broker_test.conf
@@ -56,6 +56,8 @@ bookkeeperClientHealthCheckIntervalSeconds=60
 bookkeeperClientHealthCheckErrorThresholdPerInterval=5
 bookkeeperClientHealthCheckQuarantineTimeInSeconds=1800
 bookkeeperClientRackawarePolicyEnabled=true
+bookkeeperClientRegionawarePolicyEnabled=false
+bookkeeperClientReorderReadSequenceEnabled=false
 bookkeeperClientIsolationGroups="test_group"
 managedLedgerDefaultEnsembleSize=3
 managedLedgerDefaultWriteQuorum=2
diff --git a/pulsar-client-cpp/test-conf/standalone-ssl.conf b/pulsar-client-cpp/test-conf/standalone-ssl.conf
index c7e6d3c..426ba43 100644
--- a/pulsar-client-cpp/test-conf/standalone-ssl.conf
+++ b/pulsar-client-cpp/test-conf/standalone-ssl.conf
@@ -142,6 +142,14 @@ bookkeeperClientHealthCheckQuarantineTimeInSeconds=1800
 # forming a new bookie ensemble
 bookkeeperClientRackawarePolicyEnabled=true
 
+# Enable region-aware bookie selection policy. BK will chose bookies from
+# different regions and racks when forming a new bookie ensemble
+# If enabled, the value of bookkeeperClientRackawarePolicyEnabled is ignored
+bookkeeperClientRegionawarePolicyEnabled=false
+
+# Enable/disable reordering read sequence on reading entries.
+bookkeeperClientReorderReadSequenceEnabled=false
+
 # Enable bookie isolation by specifying a list of bookie groups to choose from. Any bookie
 # outside the specified groups will not be used by the broker
 bookkeeperClientIsolationGroups=
diff --git a/pulsar-client-cpp/test-conf/standalone.conf b/pulsar-client-cpp/test-conf/standalone.conf
index 630de70..6f799c1 100644
--- a/pulsar-client-cpp/test-conf/standalone.conf
+++ b/pulsar-client-cpp/test-conf/standalone.conf
@@ -127,6 +127,14 @@ bookkeeperClientHealthCheckQuarantineTimeInSeconds=1800
 # forming a new bookie ensemble 
 bookkeeperClientRackawarePolicyEnabled=true
 
+# Enable region-aware bookie selection policy. BK will chose bookies from
+# different regions and racks when forming a new bookie ensemble
+# If enabled, the value of bookkeeperClientRackawarePolicyEnabled is ignored
+bookkeeperClientRegionawarePolicyEnabled=false
+
+# Enable/disable reordering read sequence on reading entries.
+bookkeeperClientReorderReadSequenceEnabled=false
+
 # Enable bookie isolation by specifying a list of bookie groups to choose from. Any bookie 
 # outside the specified groups will not be used by the broker 
 bookkeeperClientIsolationGroups=
diff --git a/pulsar-client-cpp/tests/authentication.conf b/pulsar-client-cpp/tests/authentication.conf
index 6db26da..25d7e30 100644
--- a/pulsar-client-cpp/tests/authentication.conf
+++ b/pulsar-client-cpp/tests/authentication.conf
@@ -138,6 +138,14 @@ bookkeeperClientHealthCheckQuarantineTimeInSeconds=1800
 # forming a new bookie ensemble
 bookkeeperClientRackawarePolicyEnabled=true
 
+# Enable region-aware bookie selection policy. BK will chose bookies from
+# different regions and racks when forming a new bookie ensemble
+# If enabled, the value of bookkeeperClientRackawarePolicyEnabled is ignored
+bookkeeperClientRegionawarePolicyEnabled=false
+
+# Enable/disable reordering read sequence on reading entries.
+bookkeeperClientReorderReadSequenceEnabled=false
+
 # Enable bookie isolation by specifying a list of bookie groups to choose from. Any bookie
 # outside the specified groups will not be used by the broker
 bookkeeperClientIsolationGroups=
diff --git a/pulsar-client-cpp/tests/standalone.conf b/pulsar-client-cpp/tests/standalone.conf
index 2fb8c0b..8a01642 100644
--- a/pulsar-client-cpp/tests/standalone.conf
+++ b/pulsar-client-cpp/tests/standalone.conf
@@ -130,6 +130,14 @@ bookkeeperClientHealthCheckQuarantineTimeInSeconds=1800
 # forming a new bookie ensemble
 bookkeeperClientRackawarePolicyEnabled=true
 
+# Enable region-aware bookie selection policy. BK will chose bookies from
+# different regions and racks when forming a new bookie ensemble
+# If enabled, the value of bookkeeperClientRackawarePolicyEnabled is ignored
+bookkeeperClientRegionawarePolicyEnabled=false
+
+# Enable/disable reordering read sequence on reading entries.
+bookkeeperClientReorderReadSequenceEnabled=false
+
 # Enable bookie isolation by specifying a list of bookie groups to choose from. Any bookie
 # outside the specified groups will not be used by the broker
 bookkeeperClientIsolationGroups=
diff --git a/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java b/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java
index ff50d8b..0b40157 100644
--- a/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java
+++ b/pulsar-zookeeper-utils/src/main/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMapping.java
@@ -64,12 +64,31 @@ public class ZkBookieRackAffinityMapping extends AbstractDNSToSwitchMapping
         bookieMappingCache = getAndSetZkCache(conf);
 
         try {
-            racksWithHost = bookieMappingCache.get(BOOKIE_INFO_ROOT_PATH).orElse(new BookiesRackConfiguration());
+            BookiesRackConfiguration racks = bookieMappingCache.get(BOOKIE_INFO_ROOT_PATH).orElse(new BookiesRackConfiguration());
+            updateRacksWithHost(racks);
         } catch (Exception e) {
             throw new RuntimeException(e);
         }
     }
 
+    private void updateRacksWithHost(BookiesRackConfiguration racks) {
+        // In config z-node, the bookies are added in the `ip:port` notation, while BK will ask
+        // for just the IP/hostname when trying to get the rack for a bookie.
+        // To work around this issue, we insert in the map the bookie ip/hostname with same rack-info
+        BookiesRackConfiguration newRacksWithHost = new BookiesRackConfiguration();
+        racks.forEach((group, bookies) ->
+                bookies.forEach((addr, bi) -> {
+                    try {
+                        BookieSocketAddress bsa = new BookieSocketAddress(addr);
+                        newRacksWithHost.updateBookie(group, bsa.getHostName(), bi);
+                    } catch (UnknownHostException e) {
+                        throw new RuntimeException(e);
+                    }
+                })
+        );
+        racksWithHost = newRacksWithHost;
+    }
+
     private ZooKeeperDataCache<BookiesRackConfiguration> getAndSetZkCache(Configuration conf) {
         ZooKeeperCache zkCache = null;
         if (conf.getProperty(ZooKeeperCache.ZK_CACHE_INSTANCE) != null) {
@@ -95,15 +114,13 @@ public class ZkBookieRackAffinityMapping extends AbstractDNSToSwitchMapping
         }
         ZooKeeperDataCache<BookiesRackConfiguration> zkDataCache = getZkBookieRackMappingCache(
                 zkCache);
-        if (zkDataCache != null) {
-            zkDataCache.registerListener(this);
-        }
+        zkDataCache.registerListener(this);
         return zkDataCache;
     }
 
     private ZooKeeperDataCache<BookiesRackConfiguration> getZkBookieRackMappingCache(
             ZooKeeperCache zkCache) {
-        ZooKeeperDataCache<BookiesRackConfiguration> zkDataCache = new ZooKeeperDataCache<BookiesRackConfiguration>(
+        return new ZooKeeperDataCache<BookiesRackConfiguration>(
                 zkCache) {
 
             @Override
@@ -114,33 +131,16 @@ public class ZkBookieRackAffinityMapping extends AbstractDNSToSwitchMapping
                     LOG.debug("Loading the bookie mappings with bookie info data: {}", new String(content));
                 }
                 BookiesRackConfiguration racks = jsonMapper.readValue(content, BookiesRackConfiguration.class);
-
-                // In config z-node, the bookies are added in the `ip:port` notation, while BK will ask
-                // for just the IP/hostname when trying to get the rack for a bookie.
-                // To work around this issue, we also insert in the map the bookie ip/hostname with same rack-info
-                BookiesRackConfiguration racksWithHost = new BookiesRackConfiguration();
-                racks.forEach((group, bookies) -> {
-                    bookies.forEach((addr, bi) -> {
-                        try {
-                            BookieSocketAddress bsa = new BookieSocketAddress(addr);
-                            racksWithHost.updateBookie(group, bsa.getHostName(), bi);
-                        } catch (UnknownHostException e) {
-                            throw new RuntimeException(e);
-                        }
-                    });
-                });
-
-                ZkBookieRackAffinityMapping.this.racksWithHost = racksWithHost;
+                updateRacksWithHost(racks);
                 return racks;
             }
 
         };
-        return zkDataCache;
     }
 
     @Override
     public List<String> resolve(List<String> bookieAddressList) {
-        List<String> racks = new ArrayList<String>(bookieAddressList.size());
+        List<String> racks = new ArrayList<>(bookieAddressList.size());
         for (String bookieAddress : bookieAddressList) {
             racks.add(getRack(bookieAddress));
         }
diff --git a/pulsar-zookeeper-utils/src/test/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMappingTest.java b/pulsar-zookeeper-utils/src/test/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMappingTest.java
index 258e818..60e7c06 100644
--- a/pulsar-zookeeper-utils/src/test/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMappingTest.java
+++ b/pulsar-zookeeper-utils/src/test/java/org/apache/pulsar/zookeeper/ZkBookieRackAffinityMappingTest.java
@@ -83,7 +83,7 @@ public class ZkBookieRackAffinityMappingTest {
         });
         mapping1.setConf(bkClientConf1);
         List<String> racks1 = mapping1
-                .resolve(Lists.newArrayList(BOOKIE1.toString(), BOOKIE2.toString(), BOOKIE3.toString()));
+                .resolve(Lists.newArrayList(BOOKIE1.getHostName(), BOOKIE2.getHostName(), BOOKIE3.getHostName()));
         assertEquals(racks1.get(0), "/rack0");
         assertEquals(racks1.get(1), "/rack1");
         assertEquals(racks1.get(2), NetworkTopology.DEFAULT_RACK);
@@ -96,7 +96,7 @@ public class ZkBookieRackAffinityMappingTest {
         bkClientConf2.setZkTimeout(1000);
         mapping2.setConf(bkClientConf2);
         List<String> racks2 = mapping2
-                .resolve(Lists.newArrayList(BOOKIE1.toString(), BOOKIE2.toString(), BOOKIE3.toString()));
+                .resolve(Lists.newArrayList(BOOKIE1.getHostName(), BOOKIE2.getHostName(), BOOKIE3.getHostName()));
         assertEquals(racks2.get(0), "/rack0");
         assertEquals(racks2.get(1), "/rack1");
         assertEquals(racks2.get(2), NetworkTopology.DEFAULT_RACK);
@@ -156,7 +156,7 @@ public class ZkBookieRackAffinityMappingTest {
         });
         mapping.setConf(bkClientConf);
         List<String> racks = mapping
-                .resolve(Lists.newArrayList(BOOKIE1.toString(), BOOKIE2.toString(), BOOKIE3.toString()));
+                .resolve(Lists.newArrayList(BOOKIE1.getHostName(), BOOKIE2.getHostName(), BOOKIE3.getHostName()));
         assertEquals(racks.get(0), "/rack0");
         assertEquals(racks.get(1), "/rack1");
         assertEquals(racks.get(2), NetworkTopology.DEFAULT_RACK);
diff --git a/site/_data/config/broker.yaml b/site/_data/config/broker.yaml
index fb82b49..31285be 100644
--- a/site/_data/config/broker.yaml
+++ b/site/_data/config/broker.yaml
@@ -186,6 +186,12 @@ configs:
 - name: bookkeeperClientRackawarePolicyEnabled
   default: 'true'
   description: Enable rack-aware bookie selection policy. BK will chose bookies from different racks when forming a new bookie ensemble
+- name: bookkeeperClientRegionawarePolicyEnabled
+  default: 'false'
+  description: Enable region-aware bookie selection policy. BK will chose bookies from different regions and racks when forming a new bookie ensemble. If enabled, the value of bookkeeperClientRackawarePolicyEnabled is ignored
+- name: bookkeeperClientReorderReadSequenceEnabled
+  default: 'false'
+  description: Enable/disable reordering read sequence on reading entries.
 - name: bookkeeperClientIsolationGroups
   default: ''
   description: Enable bookie isolation by specifying a list of bookie groups to choose from. Any bookie outside the specified groups will not be used by the broker
diff --git a/site/_data/config/standalone.yaml b/site/_data/config/standalone.yaml
index 22bcceb..d6f4c59 100644
--- a/site/_data/config/standalone.yaml
+++ b/site/_data/config/standalone.yaml
@@ -131,6 +131,12 @@ configs:
   description: If bookies have more than the allowed number of failures within the time interval specified by [`bookkeeperClientHealthCheckIntervalSeconds`](#)
 - name: bookkeeperClientRackawarePolicyEnabled
   default: 'true'
+- name: bookkeeperClientRegionawarePolicyEnabled
+  default: 'false'
+  description: Enable region-aware bookie selection policy. BK will chose bookies from different regions and racks when forming a new bookie ensemble. If enabled, the value of bookkeeperClientRackawarePolicyEnabled is ignored
+- name: bookkeeperClientReorderReadSequenceEnabled
+  default: 'false'
+  description: Enable/disable reordering read sequence on reading entries.
 - name: bookkeeperClientIsolationGroups
   default: ''
 - name: managedLedgerDefaultEnsembleSize
diff --git a/site2/docs/reference-configuration.md b/site2/docs/reference-configuration.md
index 2fec49e..3c9edaf 100644
--- a/site2/docs/reference-configuration.md
+++ b/site2/docs/reference-configuration.md
@@ -165,6 +165,8 @@ Pulsar brokers are responsible for handling incoming messages from producers, di
 |bookkeeperClientHealthCheckErrorThresholdPerInterval||5|
 |bookkeeperClientHealthCheckQuarantineTimeInSeconds ||1800|
 |bookkeeperClientRackawarePolicyEnabled|  Enable rack-aware bookie selection policy. BK will chose bookies from different racks when forming a new bookie ensemble  |true|
+|bookkeeperClientRegionawarePolicyEnabled|  Enable region-aware bookie selection policy. BK will chose bookies from different regions and racks when forming a new bookie ensemble. If enabled, the value of bookkeeperClientRackawarePolicyEnabled is ignored  |false|
+|bookkeeperClientReorderReadSequenceEnabled|  Enable/disable reordering read sequence on reading entries.  |false|
 |bookkeeperClientIsolationGroups| Enable bookie isolation by specifying a list of bookie groups to choose from. Any bookie outside the specified groups will not be used by the broker  ||
 |managedLedgerDefaultEnsembleSize|  Number of bookies to use when creating a ledger |2|
 |managedLedgerDefaultWriteQuorum| Number of copies to store for each message  |2|
@@ -337,6 +339,8 @@ The [`pulsar-client`](reference-cli-tools.md#pulsar-client) CLI tool can be used
 |bookkeeperClientHealthCheckErrorThresholdPerInterval|  Error threshold for health checks.  |5|
 |bookkeeperClientHealthCheckQuarantineTimeInSeconds|  If bookies have more than the allowed number of failures within the time interval specified by bookkeeperClientHealthCheckIntervalSeconds |1800|
 |bookkeeperClientRackawarePolicyEnabled|    |true|
+|bookkeeperClientRegionawarePolicyEnabled|    |false|
+|bookkeeperClientReorderReadSequenceEnabled|    |false|
 |bookkeeperClientIsolationGroups|||   
 |managedLedgerDefaultEnsembleSize|    |1|
 |managedLedgerDefaultWriteQuorum|   |1|