[20/51] trafficserver git commit: Documentation reorganization

[jp...@apache.org]

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/architecture/ram-cache.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/architecture/ram-cache.en.rst b/doc/developer-guide/architecture/ram-cache.en.rst
new file mode 100644
index 0000000..bb6851e
--- /dev/null
+++ b/doc/developer-guide/architecture/ram-cache.en.rst
@@ -0,0 +1,171 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-cache-ram-cache:
+
+
+RAM Cache
+*********
+
+New RAM Cache Algorithm (CLFUS)
+===============================
+
+The new RAM Cache uses ideas from a number of cache replacement policies and
+algorithms, including LRU, LFU, CLOCK, GDFS and 2Q, called CLFUS (Clocked Least
+Frequently Used by Size). It avoids any patented algorithms and includes the
+following features:
+
+* Balances Recentness, Frequency and Size to maximize hit rate (not byte hit
+  rate).
+
+* Is Scan Resistant and extracts robust hit rates even when the working set does
+  not fit in the RAM Cache.
+
+* Supports compression at 3 levels: fastlz, gzip (libz), and xz (liblzma).
+  Compression can be moved to another thread.
+
+* Has very low CPU overhead, only slightly more than a basic LRU. Rather than
+  using an O(lg n) heap, it uses a probabilistic replacement policy for O(1)
+  cost with low C.
+
+* Has relatively low memory overhead of approximately 200 bytes per object in
+  memory.
+
+The rationale for emphasizing hit rate over byte hit rate is that the overhead
+of pulling more bytes from secondary storage is low compared to the cost of a
+request.
+
+The RAM Cache consists of an object hash fronting 2 LRU/CLOCK lists and a *seen*
+hash table. The first cached list contains objects in memory, while the second
+contains a history of objects which have either recently been in memory or are
+being considered for keeping in memory. The *seen* hash table is used to make
+the algorithm scan resistant.
+
+The list entries record the following information:
+
+============== ================================================================
+Value          Description
+============== ================================================================
+key            16 byte unique object identifier
+auxkeys        8 bytes worth of version number (in our system, the block in the
+               partition). When the version of an object changes old entries are
+               purged from the cache.
+hits           Number of hits within this clock period.
+size           size of the object in the cache.
+len            Length of the object, which differs from *size* because of
+               compression and padding).
+compressed_len Compressed length of the object.
+compressed     Compression type, or ``none`` if no compression. Possible types
+               are: *fastlz*, *libz*, and *liblzma*.
+uncompressible Flag indicating that content cannot be compressed (true), or that
+               it mat be compressed (false).
+copy           Whether or not this object should be copied in and copied out
+               (e.g. HTTP HDR).
+LRU link
+HASH link
+IOBufferData   Smart point to the data buffer.
+============== ================================================================
+
+The interface to the cache is *Get* and *Put* operations. Get operations check
+if an object is in the cache and are called on a read attempt. The Put operation
+decides whether or not to cache the provided object in memory. It is called
+after a read from secondary storage.
+
+Seen Hash
+=========
+
+The *Seen List* becomes active after the *Cached* and *History* lists become
+full following a cold start. The purpose is to make the cache scan resistant,
+which means that the cache state must not be affected at all by a long sequence
+Get and Put operations on objects which are seen only once. This is essential,
+and without it not only would the cache be polluted, but it could lose critical
+information about the objects that it cares about. It is therefore essential
+that the Cache and History lists are not affected by Get or Put operations on
+objects seen the first time. The Seen Hash maintains a set of 16 bit hash tags,
+and requests which do not hit in the object cache (are in the Cache List or
+History List) and do not match the hash tag result in the hash tag being updated
+but are otherwise ignored. The Seen Hash is sized to approximately the number of
+objects in the cache in order to match the number that are passed through it
+with the CLOCK rate of the Cached and History Lists.
+
+Cached List
+===========
+
+The *Cached List* contains objects actually in memory. The basic operation is
+LRU with new entries inserted into a FIFO queue and hits causing objects to be
+reinserted. The interesting bit comes when an object is being considered for
+insertion. A check is first made against the Object Hash to see if the object
+is in the Cached List or History. Hits result in updating the ``hit`` field and
+reinsertion of the object. History hits result in the ``hit`` field being
+updated and a comparison to see if this object should be kept in memory. The
+comparison is against the least recently used members of the Cache List, and
+is based on a weighted frequency::
+
+   CACHE_VALUE = hits / (size + overhead)
+
+A new object must be enough bytes worth of currently cached objects to cover
+itself. Each time an object is considered for replacement the CLOCK moves
+forward. If the History object has a greater value then it is inserted into the
+Cached List and the replaced objects are removed from memory and their list
+entries are inserted into the History List. If the History object has a lesser
+value it is reinserted into the History List. Objects considered for replacement
+(at least one) but not replaced have their ``hits`` field set to ``0`` and are
+reinserted into the Cached List. This is the CLOCK operation on the Cached List.
+
+History List
+============
+
+Each CLOCK, the least recently used entry in the History List is dequeued and
+if the ``hits`` field is not greater than ``1`` (it was hit at least once in
+the History or Cached List) it is deleted. Otherwise, the ``hits`` is set to
+``0`` and it is requeued on the History List.
+
+Compression and Decompression
+=============================
+
+Compression is performed by a background operation (currently called as part of
+Put) which maintains a pointer into the Cached List and runs toward the head
+compressing entries. Decompression occurs on demand during a Get. In the case
+of objects tagged ``copy``, the compressed version is reinserted in the LRU
+since we need to make a copy anyway. Those not tagged ``copy`` are inserted
+uncompressed in the hope that they can be reused in uncompressed form. This is
+a compile time option and may be something we want to change.
+
+There are 3 algorithms and levels of compression (speed on an Intel i7 920
+series processor using one thread):
+
+======= ================ ================== ====================================
+Method  Compression Rate Decompression Rate Notes
+======= ================ ================== ====================================
+fastlz  173 MB/sec       442 MB/sec         Basically free since disk or network
+                                            will limit first; ~53% final size.
+libz    55 MB/sec        234 MB/sec         Almost free, particularly
+                                            decompression; ~37% final size.
+liblzma 3 MB/sec         50 MB/sec          Expensive; ~27% final size.
+======= ================ ================== ====================================
+
+These are ballpark numbers, and your millage will vary enormously. JPEG, for
+example, will not compress with any of these (or at least will only do so at
+such a marginal level that the cost of compression and decompression is wholly
+unjustified), and the same is true of many other media and binary file types
+which embed some form of compression. The RAM Cache does detect compression
+level and will declare something *incompressible* if it doesn't get below 90% of
+the original size. This value is cached so that the RAM Cache will not attempt
+to compress it again (at least as long as it is in the history).
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/architecture/tiered-storage.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/architecture/tiered-storage.en.rst b/doc/developer-guide/architecture/tiered-storage.en.rst
new file mode 100644
index 0000000..faaf4be
--- /dev/null
+++ b/doc/developer-guide/architecture/tiered-storage.en.rst
@@ -0,0 +1,142 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-cache-tiered-storage:
+
+Tiered Storage
+**************
+
+Tiered storage is an attempt to allow |TS| to take advantage of physical storage
+with different properties. This design concerns only mechanism. Policies to take
+advantage of these are outside of the scope of this document. Instead we will
+presume an *oracle* which implements this policy and describe the queries that
+must be answered by the oracle and the effects of the answers.
+
+Beyond avoiding question of tier policy, the design is also intended to be
+effectively identical to current operations for the case where there is only
+one tier.
+
+The most common case for tiers is an ordered list of tiers, where higher tiers
+are presumed faster but more expensive (or more limited in capacity). This is
+not required. It might be that different tiers are differentiated by other
+properties (such as expected persistence). The design here is intended to
+handle both cases.
+
+The design presumes that if a user has multiple tiers of storage and an ordering
+for those tiers, they will usually want content stored at one tier level to also
+be stored at every other lower level as well, so that it does not have to be
+copied if evicted from a higher tier.
+
+Configuration
+=============
+
+Each :term:`storage unit` in :file:`storage.config` can be marked with a
+*quality* value which is 32 bit number. Storage units that are not marked are
+all assigned the same value which is guaranteed to be distinct from all explicit
+values. The quality value is arbitrary from the point of view of this design,
+serving as a tag rather than a numeric value. The user (via the oracle) can
+impose what ever additional meaning is useful on this value (rating, bit
+slicing, etc.).
+
+In such cases, all :term:`volumes <cache volume>` should be explicitly assigned
+a value, as the default (unmarked) value is not guaranteed to have any
+relationship to explicit values. The unmarked value is intended to be useful in
+situations where the user has no interest in tiered storage and so wants to let
+|TS| automatically handle all volumes as a single tier.
+
+Operations
+==========
+
+After a client request is received and processed, volume assignment is done. For
+each tier, the oracle would return one of four values along with a volume
+pointer:
+
+``READ``
+    The tier appears to have the object and can serve it.
+
+``WRITE``
+    The object is not in this tier and should be written to this tier if
+    possible.
+
+``RW``
+    Treat as ``READ`` if possible, but if the object turns out to not in the
+    cache treat as ``WRITE``.
+
+``NO_SALE``
+    Do not interact with this tier for this object.
+
+The :term:`volume <cache volume>` returned for the tier must be a volume with
+the corresponding tier quality value. In effect, the current style of volume
+assignment is done for each tier, by assigning one volume out of all of the
+volumes of the same quality and returning one of ``RW`` or ``WRITE``, depending
+on whether the initial volume directory lookup succeeds. Note that as with
+current volume assignment, it is presumed this can be done from in memory
+structures (no disk I/O required).
+
+If the oracle returns ``READ`` or ``RW`` for more than one tier, it must also
+return an ordering for those tiers (it may return an ordering for all tiers,
+ones that are not readable will be ignored). For each tier, in that order, a
+read of cache storage is attempted for the object. A successful read locks that
+tier as the provider of cached content. If no tier has a successful read, or no
+tier is marked ``READ`` or ``RW`` then it is a cache miss. Any tier marked
+``RW`` that fails the read test is demoted to ``WRITE``.
+
+If the object is cached, every tier that returns ``WRITE`` receives the object
+to store in the selected volume (this includes ``RW`` returns that are demoted
+to ``WRITE``). This is a cache to cache copy, not from the :term:`origin server`.
+In this case, tiers marked ``RW`` that are not tested for read will not receive
+any data and will not be further involved in the request processing.
+
+For a cache miss, all tiers marked ``WRITE`` will receive data from the origin
+server connection (if successful).
+
+This means, among other things, that if there is a tier with the object all
+other tiers that are written will get a local copy of the object, and the origin
+server will not be used. In terms of implementation, currently a cache write to
+a volume is done via the construction of an instance of :cpp:class:`CacheVC`
+which recieves the object stream. For tiered storage, the same thing is done
+for each target volume.
+
+For cache volume overrides (via :file:`hosting.config`) this same process is
+used except with only the volumes stripes contained within the specified cache
+volume.
+
+Copying
+=======
+
+It may be necessary to provide a mechanism to copy objects between tiers outside
+of a client originated transaction. In terms of implementation, this is straight
+forward using :cpp:class:`HttpTunnel` as if in a transaction, only using a
+:cpp:class:`CacheVC` instance for both the producer and consumer. The more
+difficult question is what event would trigger a possible copy. A signal could
+be provided whenever a volume directory entry is deleted, although it should be
+noted that the object in question may have already been evicted when this event
+happens.
+
+Additional Notes
+================
+
+As an example use, it would be possible to have only one cache volume that uses
+tiered storage for a particular set of domains using volume tagging.
+:file:`hosting.config` would be used to direct those domains to the selected
+cache volume. The oracle would check the URL in parallel and return ``NO_SALE``
+for the tiers in the target cache volume for other domains. For the other tier
+(that of the unmarked storage units), the oracle would return ``RW`` for the
+tier in all cases as that tier would not be queried for the target domains.
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/config-vars.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/config-vars.en.rst b/doc/developer-guide/config-vars.en.rst
new file mode 100644
index 0000000..755b74d
--- /dev/null
+++ b/doc/developer-guide/config-vars.en.rst
@@ -0,0 +1,337 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../common.defs
+
+.. Referenced source files
+
+.. |RecCore.cc| replace:: ``RecCore.cc``
+
+.. _RecCore.cc: https://github.com/apache/trafficserver/blob/master/lib/records/RecCore.cc
+
+.. |RecordsConfig.cc| replace:: ``RecordsConfig.cc``
+
+.. _RecordsConfig.cc: https://github.com/apache/trafficserver/blob/master/mgmt/RecordsConfig.cc
+
+.. |apidefs.h.in| replace:: ``apidefs.h.in``
+
+.. _apidefs.h.in: https://github.com/apache/trafficserver/blob/master/lib/ts/apidefs.h.in
+
+.. |InkAPI.cc| replace:: ``InkAPI.cc``
+
+.. _InkAPI.cc: https://github.com/apache/trafficserver/blob/master/proxy/api/InkAPI.cc
+
+.. |InkAPITest.cc| replace:: ``InkAPITest.cc``
+
+.. _InkAPITest.cc: https://github.com/apache/trafficserver/blob/master/proxy/api/InkAPITest.cc
+
+.. |ts_lua_http_config.c| replace:: ``ts_lua_http_config.c``
+
+.. _ts_lua_http_config.c: https://github.com/apache/trafficserver/blob/master/plugins/experimental/ts_lua/ts_lua_http_config.c
+
+.. |TSHttpOverridableConfig.en.rst| replace:: ``TSHttpOverridableConfig.en.rst``
+
+.. _TSHttpOverridableConfig.en.rst: https://github.com/apache/trafficserver/blob/master/doc/reference/api/TSHttpOverridableConfig.en.rst
+
+.. Referenced enumeration values
+
+.. |RECU_DYNAMIC| replace:: ``RECU_DYNAMIC``
+
+.. _RECU_DYNAMIC: recu-dynamic_
+
+
+Configuration Variable Implementation
+*************************************
+
+Adding a new configuration variable in :file:`records.config` requires a number
+of steps which are mostly documented here.
+
+Before adding a new configuration variable, please discuss it on the mailing
+list. It will commonly be the case that a better name, or a more general
+approach to the problem which solves several different issues, may be suggested.
+
+Defining the Variable
+=====================
+
+To begin, the new configuration variables must be added to |RecordsConfig.cc|_.
+This contains a long array of configuration variable records. The fields for
+each record are:
+
+type:``RecT``
+   Type of record. The valid values are:
+
+   ``RECT_NULL``
+      Undefined record.
+
+   ``RECT_CONFIG``
+      General configuration variable.
+
+   ``RECT_PROCESS``
+      Process related statistic.
+
+   ``RECT_NODE``
+      Local statistic.
+
+   ``RECT_CLUSTER``
+      Cluster statistic.
+
+   ``RECT_LOCAL``
+      Configuration variable that is explicitly not shared across a cluster.
+
+   ``RECT_PLUGIN``
+      Plugin created statistic.
+
+   In general, ``RECT_CONFIG`` should be used unless it is required that the
+   value not be shared among members of a cluster, in which case ``RECT_LOCAL``
+   should be used. If you use ``RECT_LOCAL``, you must also start the line with
+   ``LOCAL`` instead of ``CONFIG`` and the name should use ``.local.`` instead
+   of ``.config.``.
+
+name:``char const*``
+   The fully qualified name of the configuration variable. Although there
+   appears to be a hierarchial naming scheme, that's just a convention, and it
+   is not actually used by the code. Nonetheless, new variables should adhere
+   to the hierarchial scheme.
+
+value_type:``RecDataT``
+   The data type of the value. It should be one of ``RECD_INT``,
+   ``RECD_STRING``, ``RECD_FLOAT`` as appropriate.
+
+default:``char const*``
+   The default value for the variable. This is always a string regardless of
+   the *value_type*.
+
+update:``RecUpdateT``
+   Information about how the variable is updated. The valid values are:
+
+   ``RECU_NULL``
+      Behavior is unknown or unspecified.
+
+.. _recu-dynamic:
+
+   ``RECU_DYNAMIC``
+      This can be updated via command line tools.
+
+   ``RECD_RESTART_TS``
+      The :program:`traffic_server` process must be restarted for a new value to take effect.
+
+   ``RECD_RESTART_TM``
+      The :program:`traffic_manager` process must be restarted for a new value to take effect.
+
+   ``RECD_RESTART_TC``
+      The :program:`traffic_cop` process must be restarted for a new value to take effect.
+
+required:``RecordRequiredType``
+   Effectively a boolean that specifies if the record is required to be present,
+   with ``RR_NULL`` meaning not required and ``RR_REQUIRED`` indicating that it
+   is required. Given that using ``RR_REQUIRED`` would be a major
+   incompatibility, ``RR_NULL`` is generally the better choice.
+
+check:``RecCheckT``
+   Additional type checking. It is unclear if this is actually implemented. The
+   valid values are:
+
+   ``RECC_NULL``
+      No additional checking.
+
+   ``RECC_STR``
+      Verify the value is a string.
+
+   ``RECC_INT``
+      Verify the value is an integer.
+
+   ``RECC_IP``
+      Verify the value is an IP address. Unknown if this checks for IPv6.
+
+.. XXX confirm RECC_IP & IPv6 behavior
+
+pattern:``char const*``
+   This provides a regular expressions (PCRE format) for validating the value,
+   beyond the basic type validation performed by ``RecCheckT``. This can be
+   ``NULL`` if there is no regular expression to use.
+
+access:``RecAccessT``
+   Access control. The valid values are:
+
+   ``RECA_NULL``
+      The value is read / write.
+
+   ``RECA_READ_ONLY``
+      The value is read only.
+
+   ``RECA_NO_ACCESS``
+      No access to the value; only privileged level parts of ATS can access the
+      value.
+
+Variable Infrastructure
+=======================
+
+The primary effort in defining a configuration variable is handling updates,
+generally via :option:`traffic_ctl config reload`. This is handled in a generic way, as
+described in the next section, or in a :ref:`more specialized way <http-config-var-impl>`
+(built on top of the generic mechanism) for HTTP related configuration
+variables. This is only needed if the variable is marked as dynamically
+updateable (|RECU_DYNAMIC|_) although HTTP configuration variables should be
+dynamic if possible.
+
+Documentation and Defaults
+--------------------------
+
+A configuration variable should be documented in :file:`records.config`. There
+are many examples in the file already that can be used for guidance. The general
+format is to use the tag ::
+
+   .. ts:cv:`variable.name.here`
+
+The arguments to this are the same as for the configuration file. The
+documentation generator will pick out key bits and use them to decorate the
+entry. In particular if a value is present it will be removed and used as the
+default value. You can attach some additional options to the variable. These
+are:
+
+reloadable
+   The variable can be reloaded via command line on a running Traffic Server.
+
+metric
+   Specify the units for the value. This is critical for variables that use unexpected or non-obvious metrics, such as minutes instead of seconds, or disk sectors instead of bytes.
+
+deprecated
+   Mark a variable as deprecated.
+
+.. topic:: Example
+
+   \:ts\:cv\:\`custom.variable\`
+      :reloadable:
+      :metric: minutes
+      :deprecated:
+
+If you need to refer to another configuration variable in the documentation, you
+can use the form ::
+
+   :ts:cv:`the.full.name.of.the.variable`
+
+This will display the name as a link to the full definition.
+
+In general, a new configuration variable should not be present in the default
+:file:`records.config`. If it is added, such defaults should be added to the
+file ``proxy/config/records.config.default.in``. This is used to generate the
+default :file:`records.config`. Just add the variable to the file in an
+appropriate place with a proper default as this will now override whatever
+default you put in the code for new installs.
+
+Handling Updates
+----------------
+
+The simplest mechanism for handling updates is the ``REC_EstablishStaticConfigXXX``
+family of functions. This mechanism will cause the value in the indicated
+instance to be updated in place when an update to :file:`records.config` occurs.
+This is done asynchronously using atomic operations. Use of these variables must
+keep that in mind.
+
+If a variable requires additional handling when updated a callback can be
+registered which is called when the variable is updated. This is what the
+``REC_EstablishStaticConfigXXX`` calls do internally with a callback that simply
+reads the new value and writes it to storage indicated by the call parameters.
+The functions used are the ``link_XXX`` static functions in |RecCore.cc|_.
+
+To register a configuration variable callback, call ``RecRegisterConfigUpdateCb``
+with the arguments:
+
+``char const*`` *name*
+   The variable name.
+
+*callback*
+   A function with the signature ``<int (char const* name, RecDataT type, RecData data, void* cookie)>``.
+   The :arg:`name` value passed is the same as the :arg:`name` passed to the
+   registration function as is the :arg:`cookie` argument. The :arg:`type` and
+   :arg:`data` are the new value for the variable. The return value is currently
+   ignored. For future compatibility return ``REC_ERR_OKAY``.
+
+``void*`` *cookie*
+   A value passed to the *callback*. This is only for the callback, the
+   internals simply store it and pass it on.
+
+*callback* is called under lock so it should be quick and not block. If that is
+necessary a :term:`continuation` should be scheduled to handle the required
+action.
+
+.. note::
+
+   The callback occurs asynchronously. For HTTP variables as described in the
+   next section, this is handled by the more specialized HTTP update mechanisms.
+   Otherwise it is the implementor's responsibility to avoid race conditions.
+
+.. _http-config-var-impl:
+
+HTTP Configuation Values
+------------------------
+
+Variables used for HTTP processing should be declared as members of the
+``HTTPConfigParams`` structure (but see :ref:`overridable-config-vars` for
+further details) and use the specialized HTTP update mechanisms which handle
+synchronization and initialization issues.
+
+The configuration logic maintains two copies of the ``HTTPConfigParams``
+structure, the master copy and the current copy. The master copy is kept in the
+``m_master`` member of the ``HttpConfig`` singleton. The current copy is kept in
+the ConfigProcessor. The goal is to provide a (somewhat) atomic update for
+configuration variables which are loaded individually in to the master copy as
+updates are received and then bulk copied to a new instance which is then
+swapped in as the current copy. The HTTP state machine interacts with this
+mechanism to avoid race conditions.
+
+For each variable, a mapping between the variable name and the appropriate
+member in the master copy should be established between in the ``HTTPConfig::startup``
+method. The ``HttpEstablishStaticConfigXXX`` functions should be used unless
+there is a strong, explicit reason to not do so.
+
+The ``HTTPConfig::reconfigure`` method handles the current copy of the HTTP
+configuration variables. Logic should be added here to copy the value from the
+master copy to the current copy. Generally this will be a simple assignment. If
+there are dependencies between variables, those should be checked and enforced
+in this method.
+
+.. _overridable-config-vars:
+
+Overridable Variables
+---------------------
+
+HTTP related variables that are changeable per transaction are stored in the
+``OverridableHttpConfigParams`` structure, an instance of which is the ``oride``
+member of ``HTTPConfigParams`` and therefore the points in the previous section
+still apply. The only difference for that is the further ``.oride`` member specifier in the structure references.
+
+The variable is required to be accessible from the transaction API. In addition
+to any custom API functions used to access the value, the following items are
+required for generic access:
+
+#. Add a value to the ``TSOverridableConfigKey`` enumeration in |apidefs.h.in|_.
+
+#. Augment the ``TSHttpTxnConfigFind`` function to return this enumeration value
+   when given the name of the configuration variable. Be sure to count the
+   charaters very carefully.
+
+#. Augment the ``_conf_to_memberp`` function in |InkAPI.cc|_ to return a pointer
+   to the appropriate member of ``OverridableHttpConfigParams`` and set the type
+   if not a byte value.
+
+#. Update the testing logic in |InkAPITest.cc|_ by adding the string name of the
+   configuration variable to the ``SDK_Overridable_Configs`` array.
+
+#. Update the Lua plugin enumeration ``TSLuaOverridableConfigKey`` in |ts_lua_http_config.c|_.
+
+#. Update the documentation of :ref:`ts-overridable-config` in |TSHttpOverridableConfig.en.rst|_.

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/continuous-integration/index.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/continuous-integration/index.en.rst b/doc/developer-guide/continuous-integration/index.en.rst
new file mode 100644
index 0000000..2f20c7a
--- /dev/null
+++ b/doc/developer-guide/continuous-integration/index.en.rst
@@ -0,0 +1,24 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-continuous-integration:
+
+Continuous Integration
+**********************
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/contributing/index.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/contributing/index.en.rst b/doc/developer-guide/contributing/index.en.rst
new file mode 100644
index 0000000..f4e695a
--- /dev/null
+++ b/doc/developer-guide/contributing/index.en.rst
@@ -0,0 +1,24 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-contributing:
+
+Contributing to |TS|
+********************
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/debugging/core-dump-analysis.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/debugging/core-dump-analysis.en.rst b/doc/developer-guide/debugging/core-dump-analysis.en.rst
new file mode 100644
index 0000000..10da34d
--- /dev/null
+++ b/doc/developer-guide/debugging/core-dump-analysis.en.rst
@@ -0,0 +1,24 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-core-dump-analysis:
+
+Core Dump Analysis
+******************
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/debugging/debug-builds.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/debugging/debug-builds.en.rst b/doc/developer-guide/debugging/debug-builds.en.rst
new file mode 100644
index 0000000..53ad22c
--- /dev/null
+++ b/doc/developer-guide/debugging/debug-builds.en.rst
@@ -0,0 +1,43 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-debug-builds:
+
+Debug Builds
+************
+
+A debugger can set breakpoints in a plugin. Use a Traffic Server debug
+build and compile the plugin with the ``-g`` option. A debugger can also
+be used to analyze a core dump. To generate core, set the size limit of
+the core files in the :file:`records.config` file to -1 as follows:
+
+::
+
+    CONFIG :ts:cv:`proxy.config.core_limit` INT -1
+
+This is the equivalent of setting ``ulimit -c unlimited``
+
+Debugging Tips:
+~~~~~~~~~~~~~~~
+
+-  Use a Traffic Server debug version.
+
+-  Use assertions in your plugin (:c:func:`TSAssert` and :c:func:`TSReleaseAssert`).
+
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/debugging/debug-tags.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/debugging/debug-tags.en.rst b/doc/developer-guide/debugging/debug-tags.en.rst
new file mode 100644
index 0000000..fecc3f5
--- /dev/null
+++ b/doc/developer-guide/debugging/debug-tags.en.rst
@@ -0,0 +1,113 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-debug-tags:
+
+Debug Tags
+**********
+
+Use the API
+``void TSDebug (const char *tag, const char *format_str, ...)`` to add
+traces in your plugin. In this API:
+
+-  ``tag`` is the Traffic Server parameter that enables Traffic Server
+   to print out *``format_str``*
+
+-  ``...`` are variables for *``format_str``* in the standard ``printf``
+   style.
+
+Run Traffic Server with the ``-Ttag`` option. For example, if the tag is
+``my-plugin``, then the debug output goes to ``traffic.out.``\ See
+below:
+
+::
+
+       traffic_server -T"my-plugin"
+
+Set the following variables in :file:`records.config` (in the Traffic Server
+``config`` directory):
+
+::
+
+       CONFIG proxy.config.diags.debug.enabled INT 1
+       CONFIG proxy.config.diags.debug.tags STRING debug-tag-name
+
+In this case, debug output goes to ``traffic.out``.
+
+Example:
+
+.. code-block:: c
+
+       TSDebug ("my-plugin", "Starting my-plugin at %d\n", the_time);
+
+The statement ``"Starting my-plugin at <time>"`` appears whenever you
+run Traffic Server with the ``my-plugin`` tag:
+
+::
+
+       traffic_server -T"my-plugin"
+
+Other Useful Internal Debug Tags
+================================
+
+Embedded in the base Traffic Server code are many debug tags for
+internal debugging purposes. These can also be used to follow Traffic
+Server behavior for testing and analysis.
+
+The debug tag setting (``-T`` and ``proxy.config.diag.debug.tags``) is a
+anchored regular expression against which the tag for a specific debug
+message is matched. This means the value "http" matches debug emssages
+with the tag "http" but also "http\_seq" and "http\_trans". If you want
+multiple tags then use a pipe symbol to separate the tags. For example
+"http\_tproxy\|dns\|hostdb" will match any of the message tags
+"http\_tproxy", "dns", "hostdb", or "dns\_srv" (but not "http\_trans"
+nor "splitdns").
+
+Some of the useful HTTP debug tags are:
+
+-  ``http_hdrs`` - traces all incoming and outgoing HTTP headers.
+
+-  ``http_trans`` - traces actions in an HTTP transaction.
+
+-  ``http_seq`` - traces the sequence of the HTTP state machine.
+
+-  ``http_tproxy`` - transparency related HTTP events
+
+-  ``dns`` - DNS operations
+
+-  ``hostdb`` - Host name lookup
+
+-  ``iocore_net`` - Socket and low level IO (very voluminous)
+
+-  ``socket`` - socket operations
+
+-  ``ssl`` - SSL related events
+
+-  ``cache`` - Cache operations (many subtags, examine the output to
+   narrow the tag set)
+
+-  ``cache_update`` - Cache updates including writes
+
+-  ``cache_read`` - Cache read events.
+
+-  ``dir_probe`` - Cache searches.
+
+-  ``sdk`` - gives some warning concerning API usage.
+
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/debugging/index.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/debugging/index.en.rst b/doc/developer-guide/debugging/index.en.rst
new file mode 100644
index 0000000..36f5cb8
--- /dev/null
+++ b/doc/developer-guide/debugging/index.en.rst
@@ -0,0 +1,34 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-debugging:
+
+Debugging and Analysis
+**********************
+
+.. toctree::
+   :maxdepth: 2
+
+   debug-builds.en
+   using-tsassert.en
+   debug-tags.en
+   core-dump-analysis.en
+   profiling.en
+   memory-leaks.en
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/debugging/memory-leaks.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/debugging/memory-leaks.en.rst b/doc/developer-guide/debugging/memory-leaks.en.rst
new file mode 100644
index 0000000..64a8b78
--- /dev/null
+++ b/doc/developer-guide/debugging/memory-leaks.en.rst
@@ -0,0 +1,35 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-memory-leaks:
+
+Memory Leaks
+************
+
+Memory leaks in a plugin can be detected using e.g. an MRTG graph
+related to memory - you can use memory dump information. Enable
+``mem dump`` in :file:`records.config` as follows:
+
+::
+
+      CONFIG proxy.config.dump_mem_info_frequency INT <value>
+
+This causes Traffic Server to dump memory information to ``traffic.out``
+at ``<value>`` (intervals are in seconds). A zero value means that it is
+disabled.

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/debugging/profiling.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/debugging/profiling.en.rst b/doc/developer-guide/debugging/profiling.en.rst
new file mode 100644
index 0000000..e2073cf
--- /dev/null
+++ b/doc/developer-guide/debugging/profiling.en.rst
@@ -0,0 +1,24 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-profiling:
+
+Profiling
+*********
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/debugging/using-tsassert.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/debugging/using-tsassert.en.rst b/doc/developer-guide/debugging/using-tsassert.en.rst
new file mode 100644
index 0000000..346ce98
--- /dev/null
+++ b/doc/developer-guide/debugging/using-tsassert.en.rst
@@ -0,0 +1,24 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-using-tsassert:
+
+Using TSAssert
+**************
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/documentation/adding-domains.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/documentation/adding-domains.en.rst b/doc/developer-guide/documentation/adding-domains.en.rst
new file mode 100644
index 0000000..d99e868
--- /dev/null
+++ b/doc/developer-guide/documentation/adding-domains.en.rst
@@ -0,0 +1,331 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-doc-adding-domains:
+
+Creating New Domains
+********************
+
+In the event a new type of object or reference needs to be documented, and none
+of the existing markup options or domains are appropriate, it is possible to
+extend |RST| and Sphinx by adding custom domains.
+
+Each domain may be designed to accept any number of required and optional
+arguments, as well as any collection of domain options, and each option may be
+designed to support arbitrary values, restricted (enumerated) values, or to
+simply act as flags.
+
+All custom domain definitions should be located in ``doc/ext/traffic-server.py``
+and consist of, at a bare minimum, a domain class definition and a domain
+reference class definition. Sphinx domains are implemented in Python.
+
+For this section, we will use the contrived example of creating a domain which
+permits us to define and reference a set of variables which are constrained by
+the following characteristics:
+
+#. Each variable in the domain must be one of known list of data types, which
+   we will limit here to the possibilities of :literal:`integeer`,
+   :literal:`float`, :literal:`string`.
+
+#. Where the data type is not specified, we can assume it is :literal:`string`.
+
+#. Variables which are numeric in their type may have a range of permissible
+   values.
+
+#. Variables in the domain may still be present and supported in the system, but
+   are planned to be removed in some future release.
+
+#. Every variable is associated with a single URI protocol, though there is no
+   validation performed on the value used to represent the protocol name.
+
+As stated, this example is fairly contrived and would not match any particularly
+likely real-world needs, but it will allow us to demonstrate the full extent of
+custom domain definition without needless complexity, reader's suspension of
+disbelief permitting.
+
+For this chapter's purpose, we will call this domain simply *Variables*, and we
+will construct classes which allow us to document variables thusly::
+
+    .. ts:variable:: http_enabled http integer
+       :deprecated:
+
+       Enables (any postive, non-zero value) or disables (any zero or negative
+       value) processing of HTTP requests.
+
+And referencing of those variables defined with this domain via::
+
+    :ts:variable:`http_enabled`
+
+Defining the Domain
+===================
+
+Each domain is defined by a class which inherits from ``std.Target``. Several
+class attributes are expected, which determine how domain object definitions are
+processed. |TS| convention is to name each domain's class in camel case,
+beginning with :literal:`TS` to prevent any class name collisions with builtin
+Sphinx classes.
+
+.. code::
+
+    class TSVariable(std.Target):
+
+We have named the domain's defining class as *TSVariable* and inherited from the
+:literal:`std.Target` class. Given the earlier stated requirements, we need a
+domain which supports at least two required attributes (a name, of course, and
+a URI protocol with which it is associated) and a commonly defined, though
+optional, third attribute (a data type). We'll deal with the value ranges and
+deprecation status later.
+
+.. code::
+
+    class TSVariable(std.Target):
+        required_arguments = 2
+        optional_arguments = 1
+        final_argument_whitespace = False
+        has_content = True
+
+We've now specified the appropriate number of required and optional arguments,
+though not what each one happens to be or in what order the required arguments
+need be written. Additionally, we've declared that definitions using this
+domain do not permit whitespace in the final argument, but definitions can have
+a block of text content which follows them and should be associated with the
+item being defined.
+
+.. note::
+
+   Permitting whitespace in the final argument causes the final value of a valid
+   definition to *slurp* the remaining content of the definition. Normally, each
+   argument is separated by whitespace, thus ``foo bar baz`` would only be a
+   valid definition if the domain's required and optional argument counts added
+   up to exactly three. If the domain defined only two arguments as expected,
+   but sets ``final_argument_whitespace`` to *True*, then the definition would
+   be valid and the second argument in this case would be ``bar baz``.
+
+Our requirements also state support for optional value ranges, and a flag to
+indicate whether the variable is being deprecated. These can easily be
+supported through the ``option_spec``, which allows for options to be tagged
+on to a domain item, on the lines immediately following its definition.
+
+.. code::
+
+    class TSVariable(std.Target):
+        ...
+        option_spec = {
+            'deprecated' : rst.directives.flag,
+            'range' : rst.directives.unchanged
+        }
+
+For our example, ``deprecated`` is simply a boolean flag, and ``range`` will be
+an arbitrary string on which we will perform no particular transformation or
+validation (good behavior will be left up to those documenting their variables
+with this domain). The ``rst.directives`` module may be consulted for a wider
+range of predefined option types, including the ability to define your own
+types which can perform any complexity of validation you may desire to
+implement.
+
+It would be good form to also include a docstring for the class explaining the
+expected arguments in brief. With that included, our class now looks like:
+
+.. code::
+
+    class TSVariable(std.Target):
+        """
+        Description of a Traffic Server protocol variable.
+
+        Required arguments, in order, are:
+
+            URI Protocol
+            Variable name
+
+        Optional argument is the data type of the variable, with "string" the
+        the default. Possible values are: "string", "integer", and "float".
+
+        Options supported are:
+
+            :deprecated: - A simple flag option indicating whether the variable
+            is slated for removal in future releases.
+
+            :range: - A string describing the permissible range of values the
+            variable may contain.
+        """
+
+        option_spec = {
+            'deprecated' : rst.directives.flag,
+            'range' : rst.directives.unchanged
+        }
+
+        required_arguments = 2
+        optional_arguments = 1
+        final_argument_whitespace = False
+        has_content = True
+
+Every domain class must also provide a ``run`` method, which is called every
+time an item definition using the domain is encountered. This method is where
+all argument and option validations are performed, and where transformation of
+the definition into the documentation's rendered output occurs.
+
+The core responsibilities of the ``run`` method in a domain class are to
+populate the domain's data dictionary, for use by references, as well as to
+transform the item's definition into a document structure suitable for
+rendering. The default title to be used for references will be constructed in
+this method, and all arguments and options will be processed.
+
+Our variables domain might have the following ``run`` method:
+
+.. code::
+
+    def run(self):
+        var_name, var_proto = self.arguments[0:2]
+        var_type = 'string'
+
+        if (len(self.arguments) > 2):
+            var_type = self.arguments[2]
+
+        # Create a documentation node to use as the parent.
+        node = sphinx.addnodes.desc()
+        node.document = self.state.document
+        node['objtype'] = 'variable'
+
+        # Add the signature child node for permalinks.
+        title = sphinx.addnodes.desc_signature(var_name, '')
+        title['ids'].append(nodes.make_id('variable-'+var_name))
+        title['names'].append(var_name)
+        title['first'] = False
+        title['objtype'] = node['objtype']
+        self.add_name(title)
+        title.set_class('ts-variable-title')
+
+        title += sphinx.addnodes.desc_name(var_name, var_name)
+        node.append(title)
+
+        env.domaindata['ts']['variable'][var_name] = env.docname
+
+        # Create table detailing all provided domain options
+        fl = nodes.field_list()
+
+        if ('deprecated' in self.options):
+            fl.append(self.make_field('Deprecated', 'Yes'))
+
+        if ('range' in self.options):
+            fl.append(self.make_field('Value range:', self.options['range']))
+
+        # Parse any associated block content for the item's description
+        nn = nodes.compound()
+        self.state.nested_parse(self.content, self.content_offset, nn)
+
+        # Create an index node so Sphinx will list this variable and its
+        # references in the index section.
+        indexnode = sphinx.addnodes.index(entries=[])
+        indexnode['entries'].append(
+            ('single', _('%s') % var_name, nodes.make_id(var_name), '')
+        )
+
+        return [ indexnode, node, fl, nn ]
+
+Defining the Domain Reference
+=============================
+
+Domain reference definitions are quite simple in comparison to the full domain
+definition. As with the domain itself, they are defined by a single class, but
+inherit from ``XRefRole`` instead. There are no attributes necessary, and only
+a single method, ``process_link`` need be defined.
+
+For our variables domain references, the class definition is a very short one.
+|TS| convention is to name the reference class the same as the domain class, but
+with :literal:`Ref` appended to the name. Thus, the domain class ``TSVariable``
+is accompanied by a ``TSVariableRef`` reference class.
+
+.. code::
+
+    class TSVariableRef(XRefRole):
+        def process_link(self, env, ref_node, explicit_title_p, title, target):
+            return title, target
+
+The ``process_link`` method will receive several arguments, as described below,
+and should return two values: a string containing the title of the reference,
+and a hyperlink target to be used for the rendered documentation.
+
+The ``process_link`` method receives the following arguments:
+
+``self``
+    The reference instance object, as per Python method conventions.
+
+``env``
+    A dictionary object containing the environment of the documentation
+    processor in its state at the time of the reference encounter.
+
+``ref_node``
+    The node object of the reference as encountered in the documentation source.
+
+``explicit_title_p``
+    Contains the text content of the reference's explicit title overriding, if
+    present in the reference markup.
+
+``title``
+    The processed form of the reference title, which may be the result of domain
+    class transformations or an overriding of the reference title within the
+    reference itself.
+
+``target``
+    The computed target of the reference, suitable for use by Sphinx to
+    construct hyperlinks to the location of the item's definition, wherever it
+    may reside in the final rendered form of the documentation.
+
+In our reference class, we have simply returned the processed title (allowing
+the documentation to override the variable's name if desired, or defaulting to
+the domain class's representation of the variable name in all other cases) and
+the parser's computed target.
+
+It is recommended to leave the ``target`` untouched, however you may choose to
+perform any transformations you wish on the value of the ``title``, bearing in
+mind that whatever string is returned will appear verbatim in the rendered
+documentation everywhere references for this domain are used.
+
+Exporting the Domain
+====================
+
+With both the domain itself and references to it now defined, the final step is
+to register those classes as domain and reference handlers in a namespace. This
+is done for |TS| (in its ``:ts:`` namespace) quite easily by modifying the
+``TrafficServerDomain`` class, also located in ``doc/ext/traffic-server.py``.
+
+The following dictionaries defined by that class should be updated to include
+the new domain and reference. In each case, the key used when adding to the
+dictionary should be the string you wish to use in documentation markup for
+your new domain. In our example's case, we will choose ``variable`` since it
+aligns with the Python classes we've created above, and their contrived purpose.
+
+object_types
+    Used to define the actual markup string
+
+directives
+    Defines which class is used to implement a given domain.
+
+roles
+    Defines the class used to implement references to a domain.
+
+initial_data
+    Used to initialized the dictionary which tracks all encountered instances of
+    each domain. This should always be set to an empty dictionary for each
+    domain.
+
+dangling_warnings
+    May be used to provide a default warning if a reference is attempted to a
+    non-existent item for a domain.
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/documentation/building.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/documentation/building.en.rst b/doc/developer-guide/documentation/building.en.rst
new file mode 100644
index 0000000..1b1be2d
--- /dev/null
+++ b/doc/developer-guide/documentation/building.en.rst
@@ -0,0 +1,50 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-doc-building:
+
+Building the Documentation
+**************************
+
+All documentation and related files are located in the source tree under the
+``doc/`` directory. Makefiles are generated automatiically by the main configure
+script. For simplicity's sake, it is recommended that contributors new to the
+documentation make use of the included Vagrant configurations, as these will
+take care of providing all dependencies. Please refer to the
+:ref:`dev-testing-with-vagrant` chapter for complete details on using Vagrant to
+build and test the |TS| source tree.
+
+With a configured source tree, building the documentation requires only the
+invocation ``make html`` from within ``doc/``. For repeated builds while working
+on the documentation, it is advisable to clean out the built and intermediate
+files each time by running the following instead (again, from within the ``doc/``
+directory of the |TS| source tree)::
+
+    make clean && make && make html
+
+This will ensure that make doesn't inadvertantly skip the regeneration of any
+targets.
+
+To view the built documentation, you may point any browser to the directory
+``doc/docbuild/html/``. If you are building the documentation on your local
+machine, you may access the HTML documentation files directly without the need
+for a full-fledged web server, as all necessary resources (CSS, Javascript, and
+images) are referenced using relative paths and there are no server-side scripts
+necessary to render the documentation.
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/documentation/conventions.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/documentation/conventions.en.rst b/doc/developer-guide/documentation/conventions.en.rst
new file mode 100644
index 0000000..41ab704
--- /dev/null
+++ b/doc/developer-guide/documentation/conventions.en.rst
@@ -0,0 +1,216 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-doc-conventions:
+
+Conventions
+***********
+
+The conventions detailed in this chapter should be followed when modifying the
+project documentation to maintain readability and consistency.
+
+Typographic
+===========
+
+Italic
+    Used to introduce new terms. Italics used solely for emphasis should be
+    avoided. When introducing new terms, only the first use should be set in
+    italics, and its introduction should be followed with a definition or
+    description of the term.
+
+    Example::
+
+        The |ATS| object storage is based on a *cyclone buffer* architecture.
+        Cyclone buffers are most simply described as a fixed size, but
+        continually looping block of storage updated by a single writer
+        process, wherein the writer continually reclaims the oldest allocations
+        for the most recent object updates.
+
+Bold
+    Bold typesetting is to be reserved for section, table, and glossary
+    headings and should be avoided in paragraph copy.
+
+Monospace
+    Used to indicate filesystem paths, variable names, language keywords and
+    functions, and command output. Note that in the case of variables and
+    functions, whenever possible references to their documentation should be
+    used in place of simple monospace markup.
+
+    Example::
+
+        Documentation source files for |TS| are located in the ``doc/``
+        directory of the project source tree.
+
+Bracketed Monospace
+    Used to indicate, within command or source code examples, variables for
+    which the reader should substitute a value.
+
+    Example::
+
+        To examine a performance statistic of a running |TS| instance, you may
+        run the comand ``traffic_line -r <name>``, replacing ``<name>`` with
+        the statistic you wish to examine.
+
+Ellipsis
+    Used to indicate the omission of irrelevant or unimportant information. May
+    also be used to separate matter to be treated in detail elsewhere.
+
+Layout Styles
+=============
+
+Block Content
+-------------
+
+Notes
+    Use of ``.. note::`` blocks should be sparing. In most rendered forms, the
+    note block will appear quite prominently and draw the readers' eyes away
+    from the surrounding copy. It is therefore advisable that the note itself
+    provide enough context and detail to be read on its own, without a full
+    reading of the surrounding copy.
+
+Important Notes
+    The use of ``.. important::`` callout blocks should be limited only to those
+    situations in which critical information needs to be prominently displayed.
+    Suitable use would include noting that resizing a cache voiume will result
+    in |TS| resetting the cache contents to empty when the service is started.
+    This is information that may not be obvious, or safe to assume, for the
+    reader but which can significantly (and negatively) impact the use and
+    performance of |TS| in their environment. Important note blocks should not
+    be used for behavior or actions which generally do not have potential
+    negative side effects.
+
+Sidebars
+    The use of ``.. sidebar::`` blocks in |TS| documentation should be avoided,
+    and note blocks favored in their place.
+
+Code Samples
+    Content should be set within ``.. code::`` blocks whenever a full line or
+    multiple lines of source code are being included in the documentation, or
+    when example shell or network commands are being demonstrated. Blank lines
+    may be used to separate lines of code within the block for readability, and
+    all normal indentation practices should be followed. No additional markup
+    should be used within the content block.
+
+Definition Lists
+    Definition lists may be used in multiple ways, not just for the term
+    listings in the glossary section. They may be used to provide individual
+    detailed treatment for a list of function or command arguments or where
+    any series of terms need to be explained outside of the formal glossary.
+
+Ordered Lists
+    Explicityly numbered ordered lists should be avoided. |RST| provides two
+    methods of marking up ordered, numbered lists, and the automatic numbering
+    form should be used in all cases where surrounding paragraphs do not need
+    to reference individual list entries.
+
+Tables
+------
+
+Tabular content may be used in any situation where a selection of items, terms,
+options, formats, etc. are being compared or where a grouping of the same are
+being presented alongside a small number of attributes which do not require
+detailed expositions.
+
+The |TS| project documentation supplies a custom styling override which causes
+cell contents to wrap within wide tables whenever possible in cases where it is
+necessary to prevent the content from overflowing into the margin. If, however,
+the cell content cannot be wrapped because there are no breaking spaces present
+(for example, a long variable name containing only letters, periods,
+underscores, dashes, and so no, but no whitespace), the table may still require
+overflowing into the page margin. Whenever possible, please try to avoid the
+use of tables when presenting information that will lead to this, as it greatly
+hampers readibility on smaller screens, especially tablets and mobile devices.
+Alternatives such as a definition list may be better suited to the content.
+
+Tables may be marked up using any of the |RST| styles, though it is generally
+easiest to maintain those using the *simple* table markup style.
+
+Structural
+==========
+
+Common Definitions
+------------------
+
+The |TS| project documentation maintains a common definitions, abbreviations,
+and shortcut listing in the file ``doc/common.defs``. This file should be
+included by all |RST| source files after the standard project copyright notice.
+
+The file should always be included using a relative path from the current file's
+location. Any commonly or repeatedly used abbreviations, especialy those of
+product, company, or person names, should be added to the definitions file as
+useful to avoid repetitive typing and ensure accurate spellings or legal usage.
+
+Tables of Content
+-----------------
+
+Any chapters of non-trivial scope should begin with a table of contents on the
+chapter index. The ``:depth:`` option for the table of contents may be set to
+a value appropriate to the amount of content in the chapter sections. A depth of
+``2`` will generally provide a balance between usefully describing the contents
+of the chapter without overwhelming a reader scanning for topics relevant to
+their needs or interests.
+
+Sections and Headings
+---------------------
+
+Each chapter section should be located in a separate |RST| file. Each file
+should contain the standard project copyright notice first, followed by a
+unique reference marker for the section.
+
+While |RST| itself does not define a fixed ordering of section markers, |TS|
+documentation source files should use the same set of single line section
+markings, proceeding through the section levels without skipping. For
+consistency, the following section line markers should be used in order::
+
+    Top Level
+    *********
+
+    Second Level
+    ============
+
+    Third Level
+    -----------
+
+    Fourth Level
+    ~~~~~~~~~~~~
+
+Any section file which reaches or has need to exceed the fourth level style of
+section line markings is an excellent candidate for breaking into several
+smaller, and ideally more focused, |RST| source files and referenced by an
+index with its own table of contents.
+
+Footnotes and Endnotes
+----------------------
+
+Both footnotes and endnotes should be avoided. The |TS| documentation is
+intended primarily for online viewing and the positioning of footnotes in the
+rendered output is not always intuitive. In cases where a footnote might have
+been appropriate in print-oriented material for referencing an external
+resource, that reference is more ideally integrated as a standard |RST|
+reference.
+
+For more descriptive content that might have been included as a footnote, it is
+less disruptive and more useful to choose between reformullating the text to
+simply include the additional wording, or consider the use of an inline note
+block.
+
+Grammatical
+===========
+
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/documentation/index.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/documentation/index.en.rst b/doc/developer-guide/documentation/index.en.rst
new file mode 100644
index 0000000..c4d6fdf
--- /dev/null
+++ b/doc/developer-guide/documentation/index.en.rst
@@ -0,0 +1,34 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-documentation:
+
+Documentation
+*************
+
+.. toctree::
+   :maxdepth: 2
+
+   conventions.en
+   structure.en
+   building.en
+   rst-and-sphinx.en
+   ts-markup.en
+   adding-domains.en
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/documentation/rst-and-sphinx.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/documentation/rst-and-sphinx.en.rst b/doc/developer-guide/documentation/rst-and-sphinx.en.rst
new file mode 100644
index 0000000..f4c6ce8
--- /dev/null
+++ b/doc/developer-guide/documentation/rst-and-sphinx.en.rst
@@ -0,0 +1,43 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-doc-rst-and-sphinx:
+
+|RST| and Sphinx
+****************
+
+The |TS| documentation source files are written using |RST| (RST) markup. This
+is a plain-text format intended to be easily readable both in its source and
+final rendered forms. While not entirely dissimilar to other plain text markup
+formats like Markdown, RST provides additional functionality for defining
+internal links between documents, producing tables of contents, and indices.
+Additionally, RST works in hand with Sphinx for providing advanced features
+such as markup domains, of which this documenentation makes extensive use.
+
+Both |RST| and Sphinx have official documentation resources which detail their
+full capabilities and all markup options, as well as methods of extending them
+with custom markup. This section will explain the most common markup features
+used, but the following resources may be consulted for further details.
+
+- `reStructuredText Markup Specification <http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html>`_
+
+- `Sphinx Documentation Generator <http://sphinx-doc.org/>`_
+
+- `RST Primary by Sphinx <http://sphinx-doc.org/rest.html>`_
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/documentation/structure.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/documentation/structure.en.rst b/doc/developer-guide/documentation/structure.en.rst
new file mode 100644
index 0000000..45787d9
--- /dev/null
+++ b/doc/developer-guide/documentation/structure.en.rst
@@ -0,0 +1,60 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-doc-structure:
+
+Structure
+*********
+
+The |ATS| documentation is presented as a single manual, sub-divided into the
+following sections:
+
+Preface
+    The introductory section to the manual as a whole, with a brief exposition
+    on |TS| and its general role in a network and application infrastructure.
+    The typographic conventions used in the manual should be covered here to
+    ease readers' comprehension of the following sections, as well as pointers
+    to additional resources available outside of the manual.
+
+Getting Started
+    Aimed at administrators and developers new to |TS| who wish to install and
+    configure a |TS| instance in the least amount of time necessary, without
+    delving into the full featureset or the internals of the |TS| architecture.
+
+Administrator's Guide
+    Provides in-depth coverage of all |TS| features and configurations for use
+    by administrators responsible for installing, configuring, and maintaining
+    |TS| instances. Troubleshooting, performance tuning, and plugin usage should
+    all be covered in this section of the manual.
+
+Developer's Guide
+    Documentation of internal |TS| architecture as well as guides on plugin
+    development and debugging are presented in this section.
+
+Appendices
+    Glossary, common references to both administrator and developer guides (such
+    as status code tables and command line utility usage), frequently asked
+    questions, and errata should all be located in this final section of the
+    manual.
+
+Within each top-level section, a short introduction should be provided to the
+content of that section, as well as a brief note on the intended audience. A
+very high level treatment of each chapter should be presented to allow readers
+to more easily locate the most relevant chapters for their current needs.
+

http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/developer-guide/documentation/ts-markup.en.rst
----------------------------------------------------------------------
diff --git a/doc/developer-guide/documentation/ts-markup.en.rst b/doc/developer-guide/documentation/ts-markup.en.rst
new file mode 100644
index 0000000..8b3b2bd
--- /dev/null
+++ b/doc/developer-guide/documentation/ts-markup.en.rst
@@ -0,0 +1,205 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+.. include:: ../../common.defs
+
+.. _developer-doc-ts-markup:
+
+|TS| Specific Markup
+********************
+
+This section covers markup used specific to the |TS| documentation and custom
+domains provided by local extensions.
+
+Types
+=====
+
+Data types are documented using the standard ``:c:type:`` markup provided by
+Sphinx. Types provided by the C API should be documented in
+``doc/developer-guide/api/types/``
+
+Constants
+=========
+
+Functions
+=========
+
+Custom Domains
+==============
+
+The |TS| documentation provides a Sphinx extension, located at
+``doc/ext/traffic-server.py`` which defines custom domains for various purposes.
+Those domains and their usage are documented here.
+
+Configuration Variables
+-----------------------
+
+Configuration variables are documented with the ``:ts:cv:`` domain, which takes
+three required arguments (the scope, which is the literal string
+:literal:`CONFIG`, the variable name, and the data type) and a fourth optional
+argument (the default value). ::
+
+    :ts:cv: <scope> <variable name> <data type> <value>
+
+Definition
+~~~~~~~~~~
+
+Scope
+    The scope of the variable. For configuration variables, this will always be
+    the literal string :literal:`CONFIG`.
+
+Variable Name
+    The full and exact configuration variable name.
+
+Data Type
+    Indicates the data type of the variable and must be one of the following
+    literal strings:
+
+    :literal:`INT`
+        Any integer value. Values may optionally be expressed with a binary
+        order of magnitude suffix; e.g. :literal:`K` for *value* * 1024,
+        :literal:`M` for *value* * 1024^2, :literal:`G` for *value* * 1024^3,
+        or :literal:`T` for *value* * 1024^4.
+
+    :literal:`FLOAT`
+        Any floating point value.
+
+    :literal:`STRING`
+        Any alphanumeric string.
+
+Value
+    The default value of the configuration variable. It is preferable to not
+    use any order of magnitude suffix, as |TS| will rewrite its configuration
+    files under varioua circumstances, and when doing so it does not maintain
+    those suffixes.
+
+Options
+~~~~~~~
+
+The domain for configuration variables takes only a single option at this time.
+
+deprecated
+    A simple flag option which, when attached to a configuration variable, is
+    used to indicate that the variable has been deprecated and should no longer
+    be used or relied upon, as it may be removed at any time by future releases.
+
+References
+~~~~~~~~~~
+
+References to configuration variables from elsewhere in the documentation should
+be made using the standard domain reference markup::
+
+    :ts:cv:`full.variable.name`
+
+Statistics
+----------
+
+|TS| statistics are documented using the domain ``:ts:stat:``. The domain takes
+three required arguments (collection, name, data type) and an option fourth
+(an example value). ::
+
+    :ts:stat: <collection> <statistic name> <data type> <example-value>
+
+Definition
+~~~~~~~~~~
+
+Collection
+    The key name of the collection in the returned JSON data from the
+    :ref:`plugin-stats-over-http` plugin. For most statistics, this is the
+    literal sting :literal:`global`. *Required*
+
+Statistic Name
+    The exact and full name of the statistic. *Required*
+
+Data Type
+    One of the following literal string values: :literal:`integer`,
+    :literal:`float`, :literal:`boolean`, or :literal:`string`. *Required*
+
+Example Value
+    A valid example of the value which may be exported by the statistic. In
+    cases where the description of the statistic makes note of particular
+    features of the values, such as the case of string statistics which may be
+    formatted in specific ways, providing the optional example is strongly
+    recommended. *Optional*
+
+The statistics domain also supports several options which can provide even more
+metadata about the statistic. These are currently:
+
+Options
+~~~~~~~
+
+type
+    Defines the type of metric exposed by the statistic. Valid values are:
+
+    :literal:`counter`
+        Numeric values which only increment based on the accumulation or
+        occurrence of underlying events. Examples include the total number of
+        incoming connections, or the total number of bytes transferred.
+
+    :literal:`gauge`
+        Used for moment-in-time metrics, such as the current number of |TS|
+        processes running, or the current number of connections open to origin
+        servers.
+
+    :literal:`flag`
+        Indicates that values for this statistic will be an integer with each
+        value indicating a particular state. Most statistics of this type are
+        booleans, where :literal:`1` indicates a truth or an *on* state, and
+        :literal:`0` the opposite.
+
+    :literal:`derivative`
+        Statistics of this type presents values that are calculated, or derived,
+        from other staistics. They do not expose a number or state gathered
+        directly. Typical statistics of this type are representations of a
+        statistic over a given period (e.g. average origin connections per
+        second), ratio or percentage of a statistic as part of a set (e.g. the
+        percentage of total dns lookups which have failed), or any other
+        statistic whose computation depends on the value(s) of one or more
+        other statistics.
+
+unit
+    Indicates the units of measurement that should be assumed for the given
+    statistic's value.
+
+introduced
+    May be used to indicate the version of |TS| in which the statistic was first
+    available. The value of this option should be a valid, human-readable
+    version number string, e.g. :literal:`5.3.0`.
+
+deprecated
+    Used to indicate that the statistic is no longer supported and may be
+    removed in later versions. This option may be used as a simple flag without
+    any given value, or may have a value associated in which case it should be
+    a valid, human-readable version number string for the |TS| release which was
+    first to deprecate the statistic.
+
+ungathered
+    A simple flag option, without any associated values, indicating that while
+    the statistic is included in the output of plugins like :ref:`plugin-stats-over-http`
+    there is no underlying data gathered for the statistic. If a statistic is
+    thus marked, it should be assumed to be invalid or simply unimplemented.
+
+References
+~~~~~~~~~~
+
+To reference a statistic from elsewhere in the documentation, the standard
+domain reference markup should be used::
+
+    :ts:stat:`full.statistic.name`
+
+References should not include the collection name, data type, or any other
+components aside from the statistic name.