[GitHub] sijie closed pull request #1325: Update missing configuration settings & missing shell commands

[GitBox <gi...@apache.org>]

sijie closed pull request #1325: Update missing configuration settings & missing shell commands
URL: https://github.com/apache/bookkeeper/pull/1325
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/conf/bk_server.conf b/conf/bk_server.conf
index 8016a7867..96ddcbca7 100755
--- a/conf/bk_server.conf
+++ b/conf/bk_server.conf
@@ -349,10 +349,28 @@ ledgerDirectories=/tmp/bk-data
 # Directories to store index files. If not specified, will use ledgerDirectories to store.
 # indexDirectories=/tmp/bk-data
 
-# Minimum safe Usable size to be available in index directory for bookie to create
+# Minimum safe usable size to be available in index directory for bookie to create
 # Index File while replaying journal at the time of bookie Start in Readonly Mode (in bytes)
 # minUsableSizeForIndexFileCreation=1073741824
 
+# Minimum safe usable size to be available in ledger directory for bookie to create
+# entry log files (in bytes).
+#
+# This parameter allows creating entry log files when there are enough disk spaces, even when
+# the bookie is running at readonly mode because of the disk usage is exceeding `diskUsageThreshold`.
+# Because compaction, journal replays can still write data to disks when a bookie is readonly.
+# 
+# Default value is 1.2 * `logSizeLimit`.
+#
+# minUsableSizeForEntryLogCreation=
+
+# Minimum safe usable size to be available in ledger directory for bookie to accept
+# high priority writes even it is in readonly mode.
+#
+# If not set, it is the value of `minUsableSizeForEntryLogCreation`
+#
+# minUsableSizeForHighPriorityWrites=
+
 # When entryLogPerLedgerEnabled is enabled, checkpoint doesn't happens
 # when a new active entrylog is created / previous one is rolled over.
 # Instead SyncThread checkpoints periodically with 'flushInterval' delay
diff --git a/site/_data/cli/shell.yaml b/site/_data/cli/shell.yaml
index 6edc2e665..092c8b126 100644
--- a/site/_data/cli/shell.yaml
+++ b/site/_data/cli/shell.yaml
@@ -6,7 +6,7 @@ commands:
     description: Enable autorecovery of underreplicated ledgers
   - flag: -disable
     description: Disable autorecovery of underreplicated ledgers
-- name: bookieFormat
+- name: bookieformat
   description: Format the current server contents.
   options:
   - flag: -nonInteractive
@@ -15,6 +15,13 @@ commands:
     description: If [nonInteractive] is specified, then whether to force delete the old data without prompt..?
   - flag: -deleteCookie
     description: Delete its cookie on zookeeper
+- name: initbookie
+  description: |
+    Initialize new bookie, by making sure that the journalDir, ledgerDirs and
+    indexDirs are empty and there is no registered Bookie with this BookieId.
+
+    If there is data present in current bookie server, the init operation will fail. If you want to format
+    the bookie server, use `bookieformat`.
 - name: bookieinfo
   description: Retrieve bookie info such as free and total disk space.
 - name: bookiesanity
@@ -79,13 +86,30 @@ commands:
     description: Bookie Id of missing replica
   - flag: -excludingmissingreplica N
     description: Bookie Id of missing replica to ignore
+  - flag: -printmissingreplica
+    description: Whether to print missingreplicas list?
 - name: metaformat
-  description: Format Bookkeeper metadata in Zookeeper.
+  description: |
+    Format Bookkeeper metadata in Zookeeper. This command is deprecated since 4.7.0,
+    in favor of using `initnewcluster` for initializing a new cluster and `nukeexistingcluster` for nuking an existing cluster.
   options:
   - flag: -nonInteractive
     description: Whether to confirm if old data exists..?
   - flag: -force
     description: If [nonInteractive] is specified, then whether to force delete the old data without prompt.
+- name: initnewcluster
+  description: |
+    Initializes a new bookkeeper cluster. If initnewcluster fails then try nuking
+    existing cluster by running nukeexistingcluster before running initnewcluster again
+- name: nukeexistingcluster
+  description: Nuke bookkeeper cluster by deleting metadata
+  options:
+  - flag: -zkledgersrootpath
+    description: zookeeper ledgers rootpath
+  - flag: -instanceid
+    description: instance id
+  - flag: -force
+    description: If instanceid is not specified, then whether to force nuke the metadata without validating instanceid
 - name: lostbookierecoverydelay
   description: Setter and Getter for LostBookieRecoveryDelay value (in seconds) in Zookeeper.
   options:
@@ -154,3 +178,13 @@ commands:
     description: Print status of the ledger updation (default false)
   - flag: -printprogress N
     description: Print messages on every configured seconds if verbose turned on (default 10 secs)
+- name: whoisauditor
+  description: Print the node which holds the auditor lock
+- name: whatisinstanceid
+  description: Print the instanceid of the cluster
+- name: convert-to-db-storage
+  description: Convert bookie indexes from InterleavedStorage to DbLedgerStorage format
+- name: convert-to-interleaved-storage
+  description: Convert bookie indexes from DbLedgerStorage to InterleavedStorage format
+- name: rebuild-db-ledger-locations-index
+  description: Rebuild DbLedgerStorage locations index
diff --git a/site/_data/config/bk_server.yaml b/site/_data/config/bk_server.yaml
index ce45f2028..fd3ab7ad1 100644
--- a/site/_data/config/bk_server.yaml
+++ b/site/_data/config/bk_server.yaml
@@ -270,6 +270,17 @@ groups:
   - param: minUsableSizeForIndexFileCreation
     description: Minimum safe usable size to be available in index directory for bookie to create index file while replaying journal at the time of bookie start in readonly mode (in bytes)
     default: 1073741824
+  - param: minUsableSizeForEntryLogCreation
+    description: |
+      Minimum safe usable size to be available in ledger directory for bookie to create entry log files (in bytes).
+      This parameter allows creating entry log files when there are enough disk spaces, even when
+      the bookie is running at readonly mode because of the disk usage is exceeding `diskUsageThreshold`.
+      Because compaction, journal replays can still write data to disks when a bookie is readonly.
+    default: 1.2 * `logSizeLimit`
+  - param: minUsableSizeForHighPriorityWrites
+    description: |
+      Minimum safe usable size to be available in ledger directory for bookie to accept high priority writes even it is in readonly mode.
+    default: 1.2 * `logSizeLimit`
   - param: flushInterval
     description: When entryLogPerLedgerEnabled is enabled, checkpoint doesn't happens when a new active entrylog is created / previous one is rolled over. Instead SyncThread checkpoints periodically with 'flushInterval' delay (in milliseconds) in between executions. Checkpoint flushes both ledger entryLogs and ledger index pages to disk.  Flushing entrylog and index files will introduce much random disk I/O. If separating journal dir and ledger dirs each on different devices, flushing would not affect performance. But if putting journal dir and ledger dirs on same device, performance degrade significantly on too frequent flushing. You can consider increment flush interval to get better performance, but you need to pay more time on bookie server restart after failure. This config is used only when entryLogPerLedgerEnabled is enabled.
     default: 10000


 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services