You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@couchdb.apache.org by ko...@apache.org on 2019/12/19 02:06:49 UTC
[couchdb-documentation] branch ioq-docs updated (0f5686e -> 8ae145e)
This is an automated email from the ASF dual-hosted git repository.
kocolosk pushed a change to branch ioq-docs
in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git.
discard 0f5686e Document the IO Queue subsystem
discard d671b2c WIP
add 0fcf72a Documentation for background indexing (#439)
add 3af64ec Smoosh documentation (#438)
add 5c3404d RFC for document storage (#403)
add 1ce0b67 Add latest RFC draft for FDB revision storage (#397)
add e33ac3c RFC for CouchDB background jobs (#409)
add 2cf5d9d RFC-011 : Opentracing support
add d611484 Merge pull request #440 from cloudant/011-opentracing-support
add f6b5f74 Correct misleading commentary on ERL_MAX_PORTS
add 76feee1 Revert "Correct misleading commentary on ERL_MAX_PORTS"
add 443b135 Fix typo (#446)
add 4fa6ebb Remove 417 response codes
add 8aeeaa7 Remove delayed commits (#445)
add 982f335 Merge branch 'master' into patch-1
add d85235b Merge pull request #447 from vmatyusGitHub/patch-1
add 91d4a1b Add sudo to apt-key, closes #450 (#451)
add afb929c RFC on implementation of _changes feed in FoundationDB (#401)
add ab354b1 Added some missing status codes (#452)
add 40b2ac5 Document source_proxy and target_proxy replication parameters (#454)
add de522a6 Add Kubernetes install instructions
add 1be80ec Update docker tags
add fec7c50 Merge pull request #455 from apache/kubernetes
add 051acd9 Update _scheduler/docs and _scheduler/jobs's "info" field (#456)
add c34e62d Add autoupdate field to design docs API
add 985dadb Merge pull request #453 from apache/autoupdate
add b982eef [RFC] Introduce CouchDB node types (#457)
add f9bb181 Update POST for view (#459)
add 6874283 All docs post all params (#462)
new 8ae145e Document the IO Queue subsystem
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (0f5686e)
\
N -- N -- N refs/heads/ioq-docs (8ae145e)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.gitignore | 2 +
rfcs/001-fdb-revision-metadata-model.md | 215 ++++++++++++++
rfcs/003-fdb-seq-index.md | 244 +++++++++++++++
rfcs/004-document-storage.md | 251 ++++++++++++++++
rfcs/007-background-jobs.md | 347 ++++++++++++++++++++++
rfcs/011-opentracing.md | 236 +++++++++++++++
rfcs/013-node-types.md | 143 +++++++++
src/api/database/bulk-api.rst | 60 ++--
src/api/database/common.rst | 6 -
src/api/database/compact.rst | 16 +-
src/api/database/find.rst | 1 +
src/api/ddoc/common.rst | 2 +
src/api/ddoc/views.rst | 11 +-
src/api/document/attachments.rst | 7 +-
src/api/document/common.rst | 10 +-
src/api/local.rst | 34 +--
src/api/server/common.rst | 90 +++++-
src/api/server/configuration.rst | 1 -
src/best-practices/documents.rst | 4 +-
src/config/compaction.rst | 201 ++++++-------
src/config/couchdb.rst | 28 --
src/config/index.rst | 1 +
src/config/indexbuilds.rst | 61 ++++
src/config/services.rst | 7 -
src/install/docker.rst | 12 +-
src/install/index.rst | 1 +
src/{cluster/index.rst => install/kubernetes.rst} | 34 ++-
src/install/unix.rst | 2 +-
src/json-structure.rst | 7 +-
src/maintenance/compaction.rst | 158 +++++++++-
src/maintenance/performance.rst | 10 -
src/replication/protocol.rst | 7 +-
src/replication/replicator.rst | 29 +-
33 files changed, 1934 insertions(+), 304 deletions(-)
create mode 100644 rfcs/001-fdb-revision-metadata-model.md
create mode 100644 rfcs/003-fdb-seq-index.md
create mode 100644 rfcs/004-document-storage.md
create mode 100644 rfcs/007-background-jobs.md
create mode 100644 rfcs/011-opentracing.md
create mode 100644 rfcs/013-node-types.md
create mode 100644 src/config/indexbuilds.rst
copy src/{cluster/index.rst => install/kubernetes.rst} (50%)
[couchdb-documentation] 01/01: Document the IO Queue subsystem
Posted by ko...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kocolosk pushed a commit to branch ioq-docs
in repository https://gitbox.apache.org/repos/asf/couchdb-documentation.git
commit 8ae145e86cb676dffbe41f4fd897c5c2a2537114
Author: Adam Kocoloski <ko...@apache.org>
AuthorDate: Fri Sep 13 09:19:45 2019 -0400
Document the IO Queue subsystem
---
src/config/index.rst | 1 +
src/config/ioq.rst | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 110 insertions(+)
diff --git a/src/config/index.rst b/src/config/index.rst
index 9ab1ace..fab14ce 100644
--- a/src/config/index.rst
+++ b/src/config/index.rst
@@ -27,6 +27,7 @@ Configuration
auth
compaction
indexbuilds
+ ioq
logging
replicator
query-servers
diff --git a/src/config/ioq.rst b/src/config/ioq.rst
new file mode 100644
index 0000000..bcf902b
--- /dev/null
+++ b/src/config/ioq.rst
@@ -0,0 +1,109 @@
+.. Licensed under the Apache License, Version 2.0 (the "License"); you may not
+.. use this file except in compliance with the License. You may obtain a copy of
+.. the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+.. License for the specific language governing permissions and limitations under
+.. the License.
+
+.. default-domain:: config
+.. highlight:: ini
+
+.. _config/ioq:
+
+========
+IO Queue
+========
+
+CouchDB has an internal subsystem that can prioritize IO associated with certain
+classes of operations. This subsystem can be configured to limit the resources
+devoted to background operations like internal replication and compaction
+according to the settings described below.
+
+.. config:section:: ioq :: IO Queue Configuration
+
+ .. config:option:: concurrency :: Number of in-flight IO requests
+
+ Specifies the maximum number of concurrent in-flight IO requests that
+ the queueing system will submit::
+
+ [ioq]
+ concurrency = 10
+
+ .. config:option:: ratio :: Preference for selecting background over interactive IO
+
+ The fraction of the time that a background IO request will be selected
+ over an interactive IO request when both queues are non-empty::
+
+ [ioq]
+ ratio = 0.01
+
+.. config:section:: ioq.bypass :: Bypass selected IO classes by setting these to true
+
+ System administrators can choose to submit specific classes of IO directly
+ to the underlying file descriptor or OS process, bypassing the queues
+ altogether. Installing a bypass can yield higher throughput and lower
+ latency, but relinquishes some control over prioritization. The following
+ classes are recognized:
+
+ .. config:option:: os_process
+
+ Messages on their way to an external process (e.g., ``couchjs``).
+
+ .. config:option:: read
+
+ Disk IO fulfilling interactive read requests.
+
+ .. config:option:: write
+
+ Disk IO required to update a database.
+
+ .. config:option:: view_update
+
+ Disk IO required to update views and other secondary indexes.
+
+ .. config:option:: shard_sync
+
+ Disk IO issued by the background replication processes that fix any
+ inconsistencies between shard copies.
+
+ .. config:option:: compaction
+
+ Disk IO issued by compaction jobs.
+
+ Without any configuration CouchDB will enqueue all classes of IO. The
+ default.ini configuration file that ships with CouchDB activates a bypass
+ for each of the interactive IO classes and only background IO goes into the
+ queueing system::
+
+ [ioq.bypass]
+ os_process = true
+ read = true
+ write = true
+ view_update = true
+ shard_sync = false
+ compaction = false
+
+Recommendations
+===============
+
+The default configuration protects against excessive IO from background
+operations like compaction disrupting the latency of interactive operations,
+while maximizing the overall IO throughput devoted to those interactive
+requests. There are certain situations where this configuration could be
+sub-optimal:
+
+* An administrator may want to devote a larger portion of the overall IO
+ bandwidth to compaction in order to stay ahead of the incoming write load. In
+ this it may be necessary to disable the bypass for ``write`` (to help with
+ database compaction) and/or ``view_update`` (to help with view index compaction)
+ and then increase the ``ratio`` to give compaction a higher priority.
+
+* A server with a large number of views that do not need to be comlpetely
+ up-to-date may benefit from removing the bypass on ``view_update`` in order to
+ optimize the latency for regular document read and write operations, and build
+ the views during quieter periods.