You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@couchdb.apache.org by ko...@apache.org on 2019/12/19 01:48:34 UTC
[couchdb-documentation] branch ioq-docs created (now 0f5686e)
This is an automated email from the ASF dual-hosted git repository.
kocolosk pushed a change to branch ioq-docs
in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git.
at 0f5686e Document the IO Queue subsystem
This branch includes the following new commits:
new d671b2c WIP
new 0f5686e Document the IO Queue subsystem
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[couchdb-documentation] 01/02: WIP
Posted by ko...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kocolosk pushed a commit to branch ioq-docs
in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git
commit d671b2ca5df9d6c356433d70e269e5454e4b048c
Author: Adam Kocoloski <ko...@apache.org>
AuthorDate: Fri Sep 13 09:19:45 2019 -0400
WIP
---
src/config/ioq.rst | 0
1 file changed, 0 insertions(+), 0 deletions(-)
diff --git a/src/config/ioq.rst b/src/config/ioq.rst
new file mode 100644
index 0000000..e69de29
[couchdb-documentation] 02/02: Document the IO Queue subsystem
Posted by ko...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kocolosk pushed a commit to branch ioq-docs
in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git
commit 0f5686e38926a8530a5a052fdcbdc683910acadf
Author: Adam Kocoloski <ko...@apache.org>
AuthorDate: Wed Dec 18 20:47:57 2019 -0500
Document the IO Queue subsystem
---
src/config/index.rst | 1 +
src/config/ioq.rst | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 110 insertions(+)
diff --git a/src/config/index.rst b/src/config/index.rst
index eebdc17..5f15796 100644
--- a/src/config/index.rst
+++ b/src/config/index.rst
@@ -26,6 +26,7 @@ Configuration
http
auth
compaction
+ ioq
logging
replicator
query-servers
diff --git a/src/config/ioq.rst b/src/config/ioq.rst
index e69de29..bcf902b 100644
--- a/src/config/ioq.rst
+++ b/src/config/ioq.rst
@@ -0,0 +1,109 @@
+.. Licensed under the Apache License, Version 2.0 (the "License"); you may not
+.. use this file except in compliance with the License. You may obtain a copy of
+.. the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+.. License for the specific language governing permissions and limitations under
+.. the License.
+
+.. default-domain:: config
+.. highlight:: ini
+
+.. _config/ioq:
+
+========
+IO Queue
+========
+
+CouchDB has an internal subsystem that can prioritize IO associated with certain
+classes of operations. This subsystem can be configured to limit the resources
+devoted to background operations like internal replication and compaction
+according to the settings described below.
+
+.. config:section:: ioq :: IO Queue Configuration
+
+ .. config:option:: concurrency :: Number of in-flight IO requests
+
+ Specifies the maximum number of concurrent in-flight IO requests that
+ the queueing system will submit::
+
+ [ioq]
+ concurrency = 10
+
+ .. config:option:: ratio :: Preference for selecting background over interactive IO
+
+ The fraction of the time that a background IO request will be selected
+ over an interactive IO request when both queues are non-empty::
+
+ [ioq]
+ ratio = 0.01
+
+.. config:section:: ioq.bypass :: Bypass selected IO classes by setting these to true
+
+ System administrators can choose to submit specific classes of IO directly
+ to the underlying file descriptor or OS process, bypassing the queues
+ altogether. Installing a bypass can yield higher throughput and lower
+ latency, but relinquishes some control over prioritization. The following
+ classes are recognized:
+
+ .. config:option:: os_process
+
+ Messages on their way to an external process (e.g., ``couchjs``).
+
+ .. config:option:: read
+
+ Disk IO fulfilling interactive read requests.
+
+ .. config:option:: write
+
+ Disk IO required to update a database.
+
+ .. config:option:: view_update
+
+ Disk IO required to update views and other secondary indexes.
+
+ .. config:option:: shard_sync
+
+ Disk IO issued by the background replication processes that fix any
+ inconsistencies between shard copies.
+
+ .. config:option:: compaction
+
+ Disk IO issued by compaction jobs.
+
+ Without any configuration CouchDB will enqueue all classes of IO. The
+ default.ini configuration file that ships with CouchDB activates a bypass
+ for each of the interactive IO classes and only background IO goes into the
+ queueing system::
+
+ [ioq.bypass]
+ os_process = true
+ read = true
+ write = true
+ view_update = true
+ shard_sync = false
+ compaction = false
+
+Recommendations
+===============
+
+The default configuration protects against excessive IO from background
+operations like compaction disrupting the latency of interactive operations,
+while maximizing the overall IO throughput devoted to those interactive
+requests. There are certain situations where this configuration could be
+sub-optimal:
+
+* An administrator may want to devote a larger portion of the overall IO
+ bandwidth to compaction in order to stay ahead of the incoming write load. In
+ this it may be necessary to disable the bypass for ``write`` (to help with
+ database compaction) and/or ``view_update`` (to help with view index compaction)
+ and then increase the ``ratio`` to give compaction a higher priority.
+
+* A server with a large number of views that do not need to be comlpetely
+ up-to-date may benefit from removing the bypass on ``view_update`` in order to
+ optimize the latency for regular document read and write operations, and build
+ the views during quieter periods.