You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@couchdb.apache.org by kx...@apache.org on 2013/06/19 20:08:04 UTC
[2/2] git commit: updated refs/heads/1781-reorganize-and-improve-docs
to 8555312
Refactor compaction's reference. Init administrators guide.
Merge various compaction-related articles into single compaction.rst;
Port wiki's article about compactions to the administrators guide.
The administrators guide is aims to provide useful information about CouchDB
like how to secure it, support it and fix possible problems.
Project: http://git-wip-us.apache.org/repos/asf/couchdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/couchdb/commit/8555312a
Tree: http://git-wip-us.apache.org/repos/asf/couchdb/tree/8555312a
Diff: http://git-wip-us.apache.org/repos/asf/couchdb/diff/8555312a
Branch: refs/heads/1781-reorganize-and-improve-docs
Commit: 8555312a7cfc692434f6e8b009a0d831f0fef5a5
Parents: 394d02e
Author: Alexander Shorin <kx...@apache.org>
Authored: Wed Jun 19 22:07:16 2013 +0400
Committer: Alexander Shorin <kx...@apache.org>
Committed: Wed Jun 19 22:07:16 2013 +0400
----------------------------------------------------------------------
share/doc/build/Makefile.am | 12 +-
share/doc/src/admin/index.rst | 23 +++
share/doc/src/admin/maintenance.rst | 204 ++++++++++++++++++++++
share/doc/src/config/compaction.rst | 62 ++++++-
share/doc/src/config/couchdb.rst | 11 +-
share/doc/src/config/daemons.rst | 3 +-
share/doc/src/config/database-compaction.rst | 43 -----
share/doc/src/config/index.rst | 4 +-
share/doc/src/config/view-compaction.rst | 31 ----
share/doc/src/index.rst | 1 +
10 files changed, 298 insertions(+), 96 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/couchdb/blob/8555312a/share/doc/build/Makefile.am
----------------------------------------------------------------------
diff --git a/share/doc/build/Makefile.am b/share/doc/build/Makefile.am
index b93520c..1ef9777 100644
--- a/share/doc/build/Makefile.am
+++ b/share/doc/build/Makefile.am
@@ -45,6 +45,8 @@ html_files = \
html/_images/futon-editeddoc.png \
html/_images/futon-overview.png \
html/_images/futon-replform.png \
+ html/_sources/admin/index.txt \
+ html/_sources/admin/maintenance.txt \
html/_sources/api/authn.txt \
html/_sources/api/configuration.txt \
html/_sources/api/database.txt \
@@ -60,7 +62,6 @@ html_files = \
html/_sources/config/couch-httpd-auth.txt \
html/_sources/config/couchdb.txt \
html/_sources/config/daemons.txt \
- html/_sources/config/database-compaction.txt \
html/_sources/config/httpd.txt \
html/_sources/config/index.txt \
html/_sources/config/log.txt \
@@ -75,7 +76,6 @@ html_files = \
html/_sources/config/uuids.txt \
html/_sources/config/vendor.txt \
html/_sources/config/vhosts.txt \
- html/_sources/config/view-compaction.txt \
html/_sources/config/intro.txt \
html/_sources/config/proxying.txt \
html/_sources/api-basics.txt \
@@ -112,6 +112,8 @@ html_files = \
html/_static/up-pressed.png \
html/_static/up.png \
html/_static/websupport.js \
+ html/admin/index.html \
+ html/admin/maintenance.html \
html/api/authn.html \
html/api/configuration.html \
html/api/database.html \
@@ -127,7 +129,6 @@ html_files = \
html/config/couch-httpd-auth.html \
html/config/couchdb.html \
html/config/daemons.html \
- html/config/database-compaction.html \
html/config/httpd.html \
html/config/index.html \
html/config/log.html \
@@ -142,7 +143,6 @@ html_files = \
html/config/uuids.html \
html/config/vendor.html \
html/config/vhosts.html \
- html/config/view-compaction.html \
html/config/intro.html \
html/config/proxying.html \
html/api-basics.html \
@@ -177,6 +177,8 @@ image_files = \
../images/logo.png
src_files = \
+ ../src/admin/index.rst \
+ ../src/admin/maintenance.rst \
../src/api/authn.rst \
../src/api/configuration.rst \
../src/api/database.rst \
@@ -192,7 +194,6 @@ src_files = \
../src/config/couch-httpd-auth.rst \
../src/config/couchdb.rst \
../src/config/daemons.rst \
- ../src/config/database-compaction.rst \
../src/config/httpd.rst \
../src/config/index.rst \
../src/config/log.rst \
@@ -207,7 +208,6 @@ src_files = \
../src/config/uuids.rst \
../src/config/vendor.rst \
../src/config/vhosts.rst \
- ../src/config/view-compaction.rst \
../src/config/intro.rst \
../src/config/proxying.rst \
../src/api-basics.rst \
http://git-wip-us.apache.org/repos/asf/couchdb/blob/8555312a/share/doc/src/admin/index.rst
----------------------------------------------------------------------
diff --git a/share/doc/src/admin/index.rst b/share/doc/src/admin/index.rst
new file mode 100644
index 0000000..a633973
--- /dev/null
+++ b/share/doc/src/admin/index.rst
@@ -0,0 +1,23 @@
+.. Licensed under the Apache License, Version 2.0 (the "License")you may not
+.. use this file except in compliance with the License. You may obtain a copy of
+.. the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+.. License for the specific language governing permissions and limitations under
+.. the License.
+
+.. _admin:
+
+=============================
+CouchDB Administrator's Guide
+=============================
+
+
+.. toctree::
+ :maxdepth: 2
+
+ maintenance
http://git-wip-us.apache.org/repos/asf/couchdb/blob/8555312a/share/doc/src/admin/maintenance.rst
----------------------------------------------------------------------
diff --git a/share/doc/src/admin/maintenance.rst b/share/doc/src/admin/maintenance.rst
new file mode 100644
index 0000000..3569053
--- /dev/null
+++ b/share/doc/src/admin/maintenance.rst
@@ -0,0 +1,204 @@
+.. Licensed under the Apache License, Version 2.0 (the "License")you may not
+.. use this file except in compliance with the License. You may obtain a copy of
+.. the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+.. License for the specific language governing permissions and limitations under
+.. the License.
+
+.. _admin/maintenance:
+
+==================
+Server Maintenance
+==================
+
+.. _compact:
+
+Compaction
+==========
+
+The `compaction` operation is the way to reduce disk space usage by removing
+unused and old data from database or view index files. This operation is a very
+similar to the `vacuum` available for other database management systems.
+
+During compaction of the `target` CouchDB creates new file with the ``.compact``
+extension and transfers only actual data into. Because of this, CouchDB checks
+first for the available disk space - it should be *twice greater* than the
+compacted file's data.
+
+When all actual data successful transferred to the `compacted` file CouchDB
+*removes* the `target` replacing him with it.
+
+
+.. _compact/db:
+
+Database Compaction
+-------------------
+
+Database compaction compresses the database file by removing unused file
+sections created during updates. Old documents revisions are replaced with
+small amount of metadata called `tombstone` which are used for conflicts
+resolution during replication. The number of stored revisions
+(and their `tombstones`) can be configured by using the
+:ref:`_revs_limit <api/db/revs_limit>` URL endpoint.
+
+Compaction is manually triggered operation per database and runs as a background
+task. To start it for specific database there is need to send HTTP `POST`
+request to the :ref:`_compact <api/db/compact>` sub-resource of the target
+database::
+
+ curl -H "Content-Type: application/json" -X POST http://localhost:5984/my_db/_compact
+
+On success, HTTP status `202 Accepted` is returned immediately:
+
+.. code-block:: http
+
+ HTTP/1.1 202 Accepted
+ Cache-Control: must-revalidate
+ Content-Length: 12
+ Content-Type: text/plain; charset=utf-8
+ Date: Wed, 19 Jun 2013 09:43:52 GMT
+ Server: CouchDB/1.3.0 (Erlang OTP/R15B03)
+
+.. code-block:: javascript
+
+ {"ok":true}
+
+Although the request body is not used you must still specify
+``Content-Type: application/json`` header for the request. If you don't, you
+will be aware about with HTTP status `415 Unsupported Media Type` response:
+
+.. code-block:: http
+
+ HTTP/1.1 415 Unsupported Media Type
+ Cache-Control: must-revalidate
+ Content-Length: 78
+ Content-Type: text/plain; charset=utf-8
+ Date: Wed, 19 Jun 2013 09:43:44 GMT
+ Server: CouchDB/1.3.0 (Erlang OTP/R15B03)
+
+.. code-block:: javascript
+
+ {"error":"bad_content_type","reason":"Content-Type must be application/json"}
+
+
+When the compaction is successful started and running it is possible to get
+information about it via :ref:`database information resource <api/db>`::
+
+ curl http://localhost:5984/my_db
+
+.. code-block:: http
+
+ HTTP/1.1 200 OK
+ Cache-Control: must-revalidate
+ Content-Length: 246
+ Content-Type: application/json
+ Date: Wed, 19 Jun 2013 16:51:20 GMT
+ Server: CouchDB/1.3.0 (Erlang OTP/R15B03)
+
+.. code-block:: javascript
+
+ {
+ "committed_update_seq": 76215,
+ "compact_running": true,
+ "data_size": 3787996,
+ "db_name": "my_db",
+ "disk_format_version": 6,
+ "disk_size": 17703025,
+ "doc_count": 5091,
+ "doc_del_count": 0,
+ "instance_start_time": "1371660751878859",
+ "purge_seq": 0,
+ "update_seq": 76215
+ }
+
+
+Note that ``compaction_running`` field is ``true`` indicating that compaction
+is actually running. To track the compaction progress you may query the
+:ref:`_active_tasks <api/misc/active_tasks>` resource::
+
+ curl http://localhost:5984/my_db
+
+.. code-block:: http
+
+ HTTP/1.1 200 OK
+ Cache-Control: must-revalidate
+ Content-Length: 175
+ Content-Type: application/json
+ Date: Wed, 19 Jun 2013 16:27:23 GMT
+ Server: CouchDB/1.3.0 (Erlang OTP/R15B03)
+
+.. code-block:: javascript
+
+ [
+ {
+ "changes_done": 44461,
+ "database": "my_db",
+ "pid": "<0.218.0>",
+ "progress": 58,
+ "started_on": 1371659228,
+ "total_changes": 76215,
+ "type": "database_compaction",
+ "updated_on": 1371659241
+ }
+ ]
+
+
+.. _compact/views:
+
+Views Compaction
+----------------
+
+`Views` are also need compaction like databases, unlike databases views
+are compacted by groups per `design document`. To start their compaction there
+is need to send HTTP `POST` request to the :ref:`_compact <api/db/compact/ddoc>`
+database resource with mention of the target design document::
+
+ curl -H "Content-Type: application/json" -X POST http://localhost:5984/dbname/_compact/designname
+
+.. code-block:: javascript
+
+ {"ok":true}
+
+This compacts the view index from the current version of the specified design
+document. The HTTP response code is `202 Accepted`
+(like :ref:`compaction for databases <compact/db>`) and a compaction background
+task will be created.
+
+
+.. _compact/views/cleanup:
+
+Views cleanup
+^^^^^^^^^^^^^
+
+View indexes on disk are named after their `MD5` hash of the view definition.
+When you change a view, old indexes remain on disk. To clean up all outdated
+view indexes (files named after the MD5 representation of views, that does not
+exist anymore) you can trigger a :ref:`view cleanup <api/db/view_cleanup>`::
+
+ curl -H "Content-Type: application/json" -X POST http://localhost:5984/dbname/_view_cleanup
+
+.. code-block:: javascript
+
+ {"ok":true}
+
+
+.. _compact/auto:
+
+Automatic Compaction
+--------------------
+
+While both :ref:`database <compact/db>` and :ref:`views <compact/views>`
+compactions are required be manually triggered, it is also possible to configure
+automatic compaction, so that compaction of databases and views is automatically
+triggered based on various criteria. Automatic compaction is configured in
+CouchDB's :ref:`configuration files <config/intro>`.
+
+The :ref:`compaction daemon <config/daemons/compaction_daemon>` is responsible
+for triggering the compaction. It is automatically started, but disabled by
+default. The criteria for triggering the compactions is configured in the
+:ref:`compactions <config/compactions>` section.
http://git-wip-us.apache.org/repos/asf/couchdb/blob/8555312a/share/doc/src/config/compaction.rst
----------------------------------------------------------------------
diff --git a/share/doc/src/config/compaction.rst b/share/doc/src/config/compaction.rst
index f609e0e..d6d8348 100644
--- a/share/doc/src/config/compaction.rst
+++ b/share/doc/src/config/compaction.rst
@@ -12,25 +12,60 @@
.. highlight:: ini
-.. _config/compaction:
+========================
+Compaction Configuration
+========================
-``[compaction]`` :: Compaction Daemon Rules
-===========================================
+These parameters are controls :ref:`compaction <compact>` operations.
-List of compaction rules for the
+.. _config/database_compaction:
+
+``[database_compaction]`` :: Database Compaction Options
+========================================================
+
+.. _config/database_compaction/doc_buffer_size:
+
+``doc_buffer_size`` :: Documents buffer size
+--------------------------------------------
+
+Specifies maximum copy buffer size in bytes::
+
+ [database_compaction]
+ doc_buffer_size = 524288
+
+
+.. _config/database_compaction/checkpoint_after:
+
+``checkpoint_after`` :: Checkpoint trigger
+------------------------------------------
+
+Triggers checkpoint after special amount of bytes was successfully copied to
+the compacted database::
+
+ [database_compaction]
+ checkpoint_after = 5242880
+
+
+
+.. _config/compactions:
+
+``[compactions]`` :: Compaction Daemon Rules
+============================================
+
+Defines the compaction rules for the
:ref:`compaction daemon <config/daemons/compaction_daemon>`.
The daemon compacts databases and their respective view groups when all the
condition parameters are satisfied. Configuration can be per database or
global, and it has the following format::
- [compaction]
+ [compactions]
database_name = [ {ParamName, ParamValue}, {ParamName, ParamValue}, ... ]
_default = [ {ParamName, ParamValue}, {ParamName, ParamValue}, ... ]
For example::
- [compaction]
+ [compactions]
_default = [{db_fragmentation, "70%"}, {view_fragmentation, "60%"}, {from, "23:00"}, {to, "04:00"}]
Possible parameters:
@@ -135,3 +170,18 @@ fragmentation therefore it's not worth to compact them::
min_file_size = 131072
+.. _config/view_compaction:
+
+``[view_compaction]`` :: Views Compaction Options
+=================================================
+
+.. _config/view_compaction/keyvalue_buffer_size:
+
+``keyvalue_buffer_size`` :: Key-Values buffer size
+--------------------------------------------------
+
+Specifies maximum copy buffer size in bytes used during compaction::
+
+ [view_compaction]
+ keyvalue_buffer_size = 2097152
+
http://git-wip-us.apache.org/repos/asf/couchdb/blob/8555312a/share/doc/src/config/couchdb.rst
----------------------------------------------------------------------
diff --git a/share/doc/src/config/couchdb.rst b/share/doc/src/config/couchdb.rst
index 6266bf1..f696cff 100644
--- a/share/doc/src/config/couchdb.rst
+++ b/share/doc/src/config/couchdb.rst
@@ -12,13 +12,14 @@
.. highlight:: ini
-.. _config/couchdb:
-
-``[couchdb]`` :: Base CouchDB Options
-=====================================
+====================
+Base CouchDB Options
+====================
-These options are under ``[couchdb]`` section.
+.. _config/couchdb:
+``[couchdb]``
+=============
.. _config/couchdb/attachment_stream_buffer_size:
http://git-wip-us.apache.org/repos/asf/couchdb/blob/8555312a/share/doc/src/config/daemons.rst
----------------------------------------------------------------------
diff --git a/share/doc/src/config/daemons.rst b/share/doc/src/config/daemons.rst
index 0de55e8..1ef366e 100644
--- a/share/doc/src/config/daemons.rst
+++ b/share/doc/src/config/daemons.rst
@@ -39,8 +39,7 @@ closing of the `_users` database for each request requiring authentication::
``compaction_daemon``
---------------------
-Automatic compaction daemon. Could be configured with
-:ref:`compaction rules set <config/compaction>`::
+:ref:`Automatic compaction <compact/auto>` daemon::
[daemons]
compaction_daemon={couch_compaction_daemon, start_link, []}
http://git-wip-us.apache.org/repos/asf/couchdb/blob/8555312a/share/doc/src/config/database-compaction.rst
----------------------------------------------------------------------
diff --git a/share/doc/src/config/database-compaction.rst b/share/doc/src/config/database-compaction.rst
deleted file mode 100644
index 3ccadee..0000000
--- a/share/doc/src/config/database-compaction.rst
+++ /dev/null
@@ -1,43 +0,0 @@
-.. Licensed under the Apache License, Version 2.0 (the "License")you may not
-.. use this file except in compliance with the License. You may obtain a copy of
-.. the License at
-..
-.. http://www.apache.org/licenses/LICENSE-2.0
-..
-.. Unless required by applicable law or agreed to in writing, software
-.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-.. License for the specific language governing permissions and limitations under
-.. the License.
-
-.. highlight:: ini
-
-.. _config/database_compaction:
-
-``[database_compaction]`` :: Database Compaction Options
-========================================================
-
-These options are under ``[database_compaction]`` section.
-
-.. _config/database_compaction/doc_buffer_size:
-
-``doc_buffer_size`` :: Documents buffer size
---------------------------------------------
-
-Specifies maximum copy buffer size in bytes::
-
- [database_compaction]
- doc_buffer_size = 524288
-
-
-.. _config/database_compaction/checkpoint_after:
-
-``checkpoint_after`` :: Checkpoint trigger
-------------------------------------------
-
-Triggers checkpoint after special amount of bytes was successfully copied to
-the compacted database::
-
- [database_compaction]
- checkpoint_after = 5242880
-
http://git-wip-us.apache.org/repos/asf/couchdb/blob/8555312a/share/doc/src/config/index.rst
----------------------------------------------------------------------
diff --git a/share/doc/src/config/index.rst b/share/doc/src/config/index.rst
index f43542e..a0af9bf 100644
--- a/share/doc/src/config/index.rst
+++ b/share/doc/src/config/index.rst
@@ -20,16 +20,15 @@ Configuring CouchDB
:maxdepth: 2
intro
+ compaction
proxying
admins
attachments
- compaction
cors
couch-httpd-auth
couchdb
daemons
- database-compaction
httpd
log
native-query-servers
@@ -43,7 +42,6 @@ Configuring CouchDB
uuids
vendor
vhosts
- view-compaction
http://git-wip-us.apache.org/repos/asf/couchdb/blob/8555312a/share/doc/src/config/view-compaction.rst
----------------------------------------------------------------------
diff --git a/share/doc/src/config/view-compaction.rst b/share/doc/src/config/view-compaction.rst
deleted file mode 100644
index 061c26b..0000000
--- a/share/doc/src/config/view-compaction.rst
+++ /dev/null
@@ -1,31 +0,0 @@
-.. Licensed under the Apache License, Version 2.0 (the "License")you may not
-.. use this file except in compliance with the License. You may obtain a copy of
-.. the License at
-..
-.. http://www.apache.org/licenses/LICENSE-2.0
-..
-.. Unless required by applicable law or agreed to in writing, software
-.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-.. License for the specific language governing permissions and limitations under
-.. the License.
-
-.. highlight:: ini
-
-.. _config/view_compaction:
-
-``[view_compaction]`` :: Views Compaction Options
-====================================================
-
-These options are under ``[view_compaction]`` section.
-
-.. _config/view_compaction/keyvalue_buffer_size:
-
-``keyvalue_buffer_size`` :: Key-Values buffer size
---------------------------------------------------
-
-Specifies maximum copy buffer size in bytes used during compaction::
-
- [view_compaction]
- keyvalue_buffer_size = 2097152
-
http://git-wip-us.apache.org/repos/asf/couchdb/blob/8555312a/share/doc/src/index.rst
----------------------------------------------------------------------
diff --git a/share/doc/src/index.rst b/share/doc/src/index.rst
index b51c853..0474ecf 100644
--- a/share/doc/src/index.rst
+++ b/share/doc/src/index.rst
@@ -29,6 +29,7 @@ Contents
intro
api-basics
+ admin/index
config/index
replication
replicator