You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficserver.apache.org by jp...@apache.org on 2015/11/03 07:10:10 UTC
[34/51] trafficserver git commit: Documentation reorganization
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/security/index.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/security/index.en.rst b/doc/admin-guide/security/index.en.rst
new file mode 100644
index 0000000..3a14747
--- /dev/null
+++ b/doc/admin-guide/security/index.en.rst
@@ -0,0 +1,332 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. include:: ../../common.defs
+
+.. _admin-security:
+
+Security
+********
+
+.. toctree::
+ :maxdepth: 2
+
+.. _admin-controlling-access:
+
+Controlling Access
+==================
+
+Traffic Server can be configured to allow only certain clients to use
+the proxy cache.
+
+#. Add a line to :file:`ip_allow.config` for each IP address or
+ range of IP addresses allowed to access Traffic Server.
+
+#. Run the command :option:`traffic_ctl config reload` to apply the configuration
+ changes.
+
+.. _admin-ssl-termination:
+
+SSL Termination
+===============
+
+The Traffic Server *SSL termination* option enables you to secure
+connections in reverse proxy mode between a client and a Traffic Server
+and/or Traffic Server and an origin server.
+
+The following sections describe how to enable and configure the SSL
+termination option.
+
+- Enable and configure SSL termination for client/Traffic Server
+ connections: :ref:`admin-client-and-traffic-server-connections`
+
+- Enable and configure SSL termination for Traffic Server/origin server
+ connections: :ref:`admin-traffic-server-and-origin-server-connections`
+
+.. _client-and-traffic-server-connections:
+
+Client and Traffic Server Connections
+-------------------------------------
+
+.. XXX sanity check/second opinions on example paths used for certs/keys below
+
+The figure below illustrates communication between a client and Traffic
+Server (and between Traffic Server and an origin server) when the SSL
+termination option is enabled and configured for Client/Traffic
+Server connections only.
+
+.. figure:: ../static/images/admin/ssl_c.jpg
+ :align: center
+ :alt: Client and Traffic Server communication using SSL termination
+
+ Client and Traffic Server communication using SSL termination
+
+.. Manual list numbering below corresponds to figure markings above.
+
+The figure above depicts the following:
+
+1. The client sends an HTTPS request for content. Traffic Server receives the
+ request and performs the SSL handshake to authenticate the client (depending
+ on the authentication options configured) and determine the encryption
+ method that will be used. If the client is allowed access, then Traffic
+ Server checks its cache for the requested content.
+
+2. If the request is a cache hit and the content is fresh, then Traffic Server
+ encrypts the content and sends it to the client. The client decrypts the
+ content (using the method determined during the handshake) and displays it.
+
+3. If the request is a cache miss or cached content is stale, then Traffic
+ Server communicates with the origin server via HTTP and obtains a plain text
+ version of the content. Traffic Server saves the plain text version of the
+ content in its cache, encrypts the content, and sends it to the client. The
+ client decrypts and displays the content.
+
+To configure Traffic Server to use the SSL termination option for
+Client/Traffic Server connections, you must do the following:
+
+#. Obtain and install an SSL server certificate from a recognized
+ certificate authority. The SSL server certificate contains
+ information that enables the client to authenticate Traffic Server
+ and exchange encryption keys.
+
+#. Set the port number used for SSL communication using
+ :ts:cv:`proxy.config.http.server_ports` in :file:`records.config`.
+
+#. Set the appropriate base path for your SSL certificates and private keys
+ in :file:`records.config`. ::
+
+ CONFIG proxy.config.ssl.server.cert.path STRING "/opt/ts/etc/ssl/certs/"
+ CONFIG proxy.config.ssl.server.private_key.path STRING "/opt/ts/etc/ssl/keys/"
+
+#. Add an entry to :file:`ssl_multicert.config` for each certificate and key
+ which your Traffic Server system will be using to terminate SSL connections
+ with clients. ::
+
+ ip_dest=1.2.3.4 ssl_cert_name=example.com.pem
+ ip_dest=* ssl_cert_name=default.pem
+
+#. *Optional*: Configure the use of client certificates using the variable
+ :ts:cv:`proxy.config.ssl.client.certification_level` in :file:`records.config`.
+ If you configure Traffic Server to require client certificates, then Traffic
+ Server verifies the client certificate during the SSL handshake that
+ authenticates the client. If you configure Traffic Server to not require
+ client certificates, or if you configure certificates to be optional and the
+ connecting client does not present one, then access to Traffic Server is
+ managed through other Traffic Server options that have been set (such as
+ rules in :file:`ip_allow.config`). ::
+
+ CONFIG proxy.config.ssl.client.certification_level INT 0
+
+ This variable permits one of the following values to be set:
+
+ ===== =======================================================================
+ Value Description
+ ===== =======================================================================
+ ``0`` Client certificates not required.
+ ``1`` Client certificates optional. If present, will be used to validate.
+ ``2`` Client certficates required, and must validate based on configured CAs.
+ ===== =======================================================================
+
+#. *Optional*: Configure the use of Certification Authorities (CAs). CAs add
+ security by verifying the identity of the person requesting a certificate.
+ The list of acceptable CA signers is configured with
+ :ts:cv:`proxy.config.ssl.CA.cert.path` in :file:`records.config`. ::
+
+ CONFIG proxy.config.ssl.CA.cert.path STRING "/opt/CA/certs/private-ca.pem"
+
+#. Run the command :option:`traffic_ctl server restart` to restart Traffic Server on the
+ local node or :option:`traffic_ctl cluster restart` to restart Traffic Server on all
+ the nodes in a cluster.
+
+.. _traffic-server-and-origin-server-connections:
+
+Traffic Server and Origin Server Connections
+--------------------------------------------
+
+.. XXX sanity check/second opinions on example paths used for certs/keys below
+
+The figure below illustrates communication between Traffic Server and an
+origin server when the SSL termination option is enabled for Traffic
+Server/origin server connections.
+
+.. figure:: ../static/images/admin/ssl_os.jpg
+ :align: center
+ :alt: Traffic Server and origin server communication using SSL termination
+
+ Traffic Server and origin server communication using SSL termination
+
+.. Manual list numbering below corresponds to figure markings above.
+
+The figure above depicts the following:
+
+1. If a client request is a cache miss or is stale, then Traffic Server sends
+ an HTTPS request for the content to the origin server. The origin server
+ receives the request and performs the SSL handshake to authenticate Traffic
+ Server and determine the encryption method to be used.
+
+2. If Traffic Server is allowed access, then the origin server encrypts the
+ content and sends it to Traffic Server, where it is decrypted (using the
+ method determined during the handshake). A plain text version of the content
+ is saved in the cache, if Traffic Server deems the content cacheable.
+
+3. If SSL termination is enabled for Client/Traffic Server connections, then
+ Traffic Server re-encrypts the content and sends it to the client via HTTPS,
+ where it is decrypted and displayed. If SSL termination is not enabled for
+ Client/Traffic Server connections, then Traffic Server sends the plain text
+ version of the content to the client via HTTP.
+
+To configure Traffic Server to use the SSL termination option for Traffic Server
+and origin server connections, you must do the following:
+
+#. Ensure first that your origin server responds properly to SSL requests, and
+ configure it for client certificate validation if you intend to use that as
+ part of your access control scheme.
+
+ Refer to your origin server's documentation for details. If your origin
+ server is another Traffic Server system, then you may follow the steps
+ outlined in `Client and Traffic Server Connections`_ for configuring the
+ origin server to validate client certificates.
+
+#. *Optional*: Obtain and install an SSL client certificate from a recognized
+ certificate authority, if your origin server requires client certificate
+ validation for access control. Your client certificate must be signed by a
+ Certificate Authority recognized by your origin server.
+
+ If you are using a client certificate, you must add its location to
+ :file:`records.config` in the setting :ts:cv:`proxy.config.ssl.client.cert.path`
+ and :ts:cv:`proxy.config.ssl.client.cert.filename`. ::
+
+ CONFIG proxy.config.ssl.client.cert.path STRING "/opt/ts/etc/ssl/certs/"
+ CONFIG proxy.config.ssl.client.cert.filename STRING "client.pem"
+
+ You must also provide the paths to the private key for this certificate,
+ unless the key is contained within the same file as the certificate, using
+ :ts:cv:`proxy.config.ssl.client.private_key.path` and
+ :ts:cv:`proxy.config.ssl.client.private_key.filename`. ::
+
+ CONFIG proxy.config.ssl.client.private_key.path STRING "/opt/ts/etc/ssl/keys/"
+ CONFIG proxy.config.ssl.client.private_key.filename STRING "client.pem"
+
+#. Enable or disable, per your security policy, server SSL certificate
+ verification using :ts:cv:`proxy.config.ssl.client.verify.server` in
+ :file:`records.config`. ::
+
+ CONFIG proxy.config.ssl.client.verify.server INT 1
+
+#. Add the collection of authorized Certificate Authorities to the Traffic
+ Server configuration in :file:`records.config` using the settings
+ :ts:cv:`proxy.config.ssl.client.CA.cert.path` and
+ :ts:cv:`proxy.config.ssl.client.CA.cert.filename`. ::
+
+ CONFIG proxy.config.ssl.client.CA.cert.path STRING "/opt/ts/etc/ssl/certs/"
+ CONFIG proxy.config.ssl.client.CA.cert.filename STRING "CAs.pem"
+
+#. Run the command :option:`traffic_ctl server restart` to restart Traffic Server on the
+ local node or :option:`traffic_ctl cluster restart` to restart Traffic Server on all
+ the nodes in a cluster.
+
+.. _admin-rotating-tls-session-ticket-keys:
+
+Rotating TLS Session Ticket Keys
+================================
+
+TLS sessions can be resumed through session tickets which are encrypted with
+a session ticket key and stored on clients. For better security, the ticket keys
+can be rotated periodically, say, every 24 hours. The ticket keys are stored in
+a ticket key file as a reverse queue in 48-byte chunks.
+
+#. Generate a new ticket key and push it to the beginning of the ticket key file.
+
+#. *Optional*: Delete the last ticket key from the ticket key file.
+
+#. Touch :file:`ssl_multicert.config` to indicate that the SSL configuration is stale.
+
+#. Run the command :option:`traffic_ctl config reload` to apply the new ticket key.
+
+OCSP Stapling
+=============
+
+OCSP Stapling is an alternative approach to checking the revocation
+status of an SSL certificate using the Online Certificate Status
+Protocol.
+
+Under the original OCSP implementation, clients requested a
+certificate's revocation status directly from the Certificate
+Authority (CA) that issued the certificate. This could cause
+significant load on the CA servers since they were required to
+provide a response to every client of a given certificate in real
+time.
+
+Enabling OCSP Stapling instructs Traffic Server to retrieve and cache the
+revocation status of all configured SSL certificates, and present them to the
+client when the client requests the status. Traffic Server will automatically
+query the OCSP responder specified in the SSL certificate to gather the latest
+revocation status. Traffic Server will then cache the results for each
+configured certifcate. The location of the OCSP responder is taken from the
+Authority Information Access field of the signed certificate. For example::
+
+ Authority Information Access:
+ OCSP - URI:http://ocsp.digicert.com
+ CA Issuers - URI:http://cacerts.digicert.com/DigiCertSHA2SecureServerCA.crt
+
+Support for OCSP Stapling can be tested using the -status option of the OpenSSL client::
+
+ $ openssl s_client -connect mozillalabs.com:443 -status
+ ...
+ ======================================
+ OCSP Response Data:
+ OCSP Response Status: successful (0x0)
+ Response Type: Basic OCSP Response
+ Version: 1 (0x0)
+ ...
+
+Details of the OCSP Stapling TLS extension can be found in :rfc:`6961`.
+
+To configure Traffic Server to use OCSP Stapling, edit the following variables
+in :file:`records.config` file:
+
+* :ts:cv:`proxy.config.ssl.ocsp.enabled`
+* :ts:cv:`proxy.config.ssl.ocsp.cache_timeout`
+* :ts:cv:`proxy.config.ssl.ocsp.request_timeout`
+* :ts:cv:`proxy.config.ssl.ocsp.update_period`
+
+.. _admin-split-dns:
+
+Split DNS
+=========
+
+The *Split DNS* option enables you to configure Traffic Server to use
+multiple DNS servers, as dictated by your security requirements. For
+example, you might configure Traffic Server to use one set of DNS
+servers to resolve hostnames on your internal network, while allowing
+DNS servers outside the firewall to resolve hosts on the Internet. This
+maintains the security of your intranet, while continuing to provide
+direct access to sites outside your organization.
+
+To configure Split DNS:
+
+#. Specify the rules for performing DNS server selection based on the
+ destination domain, the destination host, or a URL regular expression.
+ These rules are located in :file:`splitdns.config`.
+
+#. Enable the *Split DNS* option by adjusting :ts:cv:`proxy.config.dns.splitDNS.enabled`
+ in :file:`records.config`. ::
+
+ CONFIG proxy.config.dns.splitDNS.enabled INT 1
+
+#. Run the command :option:`traffic_line -x` to apply the configuration changes.
+
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/storage/index.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/storage/index.en.rst b/doc/admin-guide/storage/index.en.rst
new file mode 100644
index 0000000..5ecdac2
--- /dev/null
+++ b/doc/admin-guide/storage/index.en.rst
@@ -0,0 +1,424 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. _admin-cache-storage:
+
+Cache Storage
+*************
+
+The Traffic Server cache consists of a high-speed object database called
+the :term:`object store` that indexes :term:`cache objects <cache object>`
+according to URLs and their associated headers.
+
+.. toctree::
+ :maxdepth: 2
+
+The Traffic Server Cache
+========================
+
+The Traffic Server cache consists of a high-speed object database called
+the :term:`object store`. The object store indexes
+:term:`cache objects <cache object>` according to URLs and associated headers.
+This enables Traffic Server to store, retrieve, and serve not only web pages,
+but also parts of web pages - which provides optimum bandwidth savings. Using
+sophisticated object management, the object store can cache
+:term:`alternate` versions of the same object (versions may differ because of
+dissimilar language or encoding types). It can also efficiently store very
+small and very large documents, thereby minimizing wasted space. When the
+cache is full, Traffic Server removes :term:`stale` data to ensure the most
+requested objects are kept readily available and fresh.
+
+Traffic Server is designed to tolerate total disk failures on any of the
+cache disks. If the disk fails completely, then Traffic Server marks the
+entire disk as corrupt and continues using the remaining disks. An alarm
+is then created to indicate which disk failed. If all of the cache disks
+fail, then Traffic Server goes into proxy-only mode.
+
+You can perform the following cache configuration tasks:
+
+- Change the total amount of disk space allocated to the cache; refer
+ to `Changing Cache Capacity`_.
+
+- Partition the cache by reserving cache disk space for specific
+ protocols and :term:`origin servers/domains <origin server>`; refer to
+ `Partitioning the Cache`_.
+
+- Delete all data in the cache; refer to `Clearing the Cache`_.
+
+- Override cache directives for a requested domain name, regex on a url,
+ hostname or ip, with extra filters for time, port, method of the request,
+ and more. ATS can be configured to never cache, always cache,
+ ignore no-cache directives, etc. These are configured in :file:`cache.config`.
+
+The RAM Cache
+=============
+
+Traffic Server maintains a small RAM cache of extremely popular objects.
+This RAM cache serves the most popular objects as quickly as possible
+and reduces load on disks, especially during temporary traffic peaks.
+You can configure the RAM cache size to suit your needs, as described in
+:ref:`changing-the-size-of-the-ram-cache` below.
+
+The RAM cache supports two cache eviction algorithms, a regular *LRU*
+(Least Recently Used) and the more advanced *CLFUS* (Clocked Least
+Frequently Used by Size; which balances recentness, frequency, and size
+to maximize hit rate, similar to a most frequently used algorithm).
+The default is to use *CLFUS*, and this is controlled via
+:ts:cv:`proxy.config.cache.ram_cache.algorithm`.
+
+Both the *LRU* and *CLFUS* RAM caches support a configuration to increase
+scan resistance. In a typical *LRU*, if you request all possible objects in
+sequence, you will effectively churn the cache on every request. The option
+:ts:cv:`proxy.config.cache.ram_cache.use_seen_filter` can be set to add some
+resistance against this problem.
+
+In addition, *CLFUS* also supports compressing in the RAM cache itself.
+This can be useful for content which is not compressed by itself (e.g.
+images). This should not be confused with ``Content-Encoding: gzip``, this
+feature is only present to save space internally in the RAM cache itself. As
+such, it is completely transparent to the User-Agent. The RAM cache
+compression is enabled with the option
+:ts:cv:`proxy.config.cache.ram_cache.compress`.
+
+Possible values are:
+
+======= =============================
+Value Meaning
+======= =============================
+0 No compression (*default*)
+1 *fastlz* compression
+2 *libz* compression
+3 *liblzma* compression
+======= =============================
+
+.. _changing-the-size-of-the-ram-cache:
+
+Changing the Size of the RAM Cache
+==================================
+
+Traffic Server provides a dedicated RAM cache for fast retrieval of
+popular small objects. The default RAM cache size is automatically
+calculated based on the number and size of the
+:term:`cache partitions <cache partition>` you have configured. If you've
+partitioned your cache according to protocol and/or hosts, then the size of
+the RAM cache for each partition is proportional to the size of that partition.
+
+You can increase the RAM cache size for better cache hit performance.
+However, if you increase the size of the RAM cache and observe a
+decrease in performance (such as increased latencies), then it's
+possible that the operating system requires more memory for network
+resources. In such instances, you should return the RAM cache size to
+its previous value.
+
+To change the RAM cache size:
+
+#. Stop Traffic Server.
+
+#. Set the variable :ts:cv:`proxy.config.cache.ram_cache.size`
+ to specify the size of the RAM cache. The default value of ``-1`` means
+ that the RAM cache is automatically sized at approximately 1MB per
+ gigabyte of disk.
+
+#. Restart Traffic Server. If you increase the RAM cache to a size of
+ 1GB or more, then restart with the :program:`trafficserver` command
+ (refer to :ref:`start-traffic-server`).
+
+Changing Cache Capacity
+=======================
+
+You can increase or reduce the total amount of disk space allocated to
+the cache without clearing the content. To check the size of the cache
+(in bytes), enter the command :option:`traffic_ctl metric get`
+``proxy.process.cache.bytes_total``.
+
+Increasing Cache Capacity
+-------------------------
+
+To increase the total amount of disk space allocated to the cache on
+existing disks, or to add new disks to a Traffic Server node:
+
+#. Stop Traffic Server.
+
+#. Add hardware, if necessary.
+
+#. Edit :file:`storage.config` to increase the amount of disk space allocated
+ to the cache on existing disks or describe the new hardware you are adding.
+
+#. Restart Traffic Server.
+
+Reducing Cache Capacity
+-----------------------
+
+To reduce the total amount of disk space allocated to the cache on an
+existing disk, or to remove disks from a Traffic Server node:
+
+#. Stop Traffic Server.
+
+#. Remove hardware, if necessary.
+
+#. Edit :file:`storage.config` to reduce the amount of disk space allocated
+ to the cache on existing disks or delete the reference to the hardware you're removing.
+
+#. Restart Traffic Server.
+
+.. important:: In :file:`storage.config`, a formatted or raw disk must be at least 128 MB.
+
+.. _partitioning-the-cache:
+
+Partitioning the Cache
+======================
+
+You can manage your cache space more efficiently and restrict disk usage
+by creating :term:`cache volumes <cache volume>` with different sizes for
+specific protocols. You can further configure these volumes to store data from
+specific :term:`origin servers <origin server>` and/or domains. The volume
+configuration must be the same on all nodes in a :ref:`cluster <traffic-server-cluster>`.
+
+Creating Cache Partitions for Specific Protocols
+------------------------------------------------
+
+You can create separate :term:`volumes <cache volume>` for your cache that vary
+in size to store content according to protocol. This ensures that a certain
+amount of disk space is always available for a particular protocol. Traffic
+Server currently supports only the ``http`` partition type.
+
+To partition the cache according to protocol:
+
+#. Enter a line in :file:`volume.config` for each volume you want to create. ::
+
+ volume=1 scheme=http size=50%
+ volume=2 scheme=http size=50%
+
+#. Restart Traffic Server.
+
+.. important::
+
+ Volume definitions must be the same across all nodes in a cluster.
+
+Making Changes to Partition Sizes and Protocols
+-----------------------------------------------
+
+After you've configured your cache volumes based on protocol, you can
+make changes to the configuration at any time. Before making changes,
+note the following:
+
+- You must stop Traffic Server before you change the cache volume size
+ and protocol assignment.
+
+- When you increase the size of a volume, the contents of the volume
+ are *not* deleted. However, when you reduce the size of a volume, the
+ contents of the volume *are* deleted.
+
+- When you change the volume number, the volume is deleted and then
+ recreated, even if the size and protocol type remain the same.
+
+- When you add new disks to your Traffic Server node, volume sizes
+ specified in percentages will increase proportionately.
+
+- Substantial changes to volume sizes can result in disk fragmentation,
+ which affects performance and cache hit rate. You should clear the cache
+ before making many changes to cache volume sizes (refer to `Clearing the Cache`_).
+
+Partitioning the Cache According to Origin Server or Domain
+-----------------------------------------------------------
+
+.. XXX: rewrite to remove repetitious single-v-multiple points; break out global partition note for clarify; fix up plurality
+
+After you have partitioned the cache according to size and protocol, you
+can assign the volumes you created to specific origin servers and/or
+domains. You can assign a volume to a single origin server or to
+multiple origin servers. However, if a volume is assigned to multiple
+origin servers, then there is no guarantee on the space available in the
+volumes for each origin server. Content is stored in the volumes
+according to popularity. In addition to assigning volumes to specific
+origin servers and domains, you must assign a generic volume to store
+content from all origin servers and domains that are not listed. This
+generic volume is also used if the partitions for a particular origin
+server or domain become corrupt. If you do not assign a generic volume,
+then Traffic Server will run in proxy-only mode.
+
+.. note::
+
+ You do not need to stop Traffic Server before you assign volumes
+ to particular hosts or domains. However, this type of configuration
+ is time-consuming and can cause a spike in memory usage.
+ Therefore, it's best to configure partition assignment during
+ periods of low traffic.
+
+To partition the cache according to hostname and domain:
+
+#. Configure the cache volumes according to size and protocol, as
+ described in `Creating Cache Partitions for Specific Protocols`_.
+#. Create a separate volume based on protocol for each host and domain,
+ as well as an additional generic partition to use for content that
+ does not belong to these origin servers or domains. The volumes do
+ not need to be the same size.
+#. Enter a line in the :file:`hosting.config` file to
+ allocate the volume(s) used for each origin server and/or domain.
+#. Assign a generic volume to use for content that does not belong to
+ any of the origin servers or domains listed in the file. If all
+ volumes for a particular origin server become corrupt, then Traffic
+ Server will also use the generic volume to store content for that
+ origin server as per :file:`hosting.config`.
+#. Run the command :option:`traffic_ctl config reload` to apply the configuration
+ changes.
+
+Configuring the Cache Object Size Limit
+=======================================
+
+By default, Traffic Server allows objects of any size to be cached. You
+can change the default behavior and specify a size limit for objects in
+the cache via the steps below:
+
+#. Set :ts:cv:`proxy.config.cache.max_doc_size`
+ to specify the maximum size in bytes allowed for objects in the cache.
+ A setting of ``0`` (zero) will permit cache objects to be unlimited in size.
+#. Run the command :option:`traffic_ctl config reload` to apply the configuration
+ changes.
+
+.. _clearing-the-cache:
+
+Clearing the Cache
+==================
+
+When you clear the cache, you remove all data from the entire cache -
+including data in the host database. You should clear the cache before
+performing certain cache configuration tasks such as partitioning. You
+cannot clear the cache when Traffic Server is running.
+
+To clear the cache:
+
+#. Stop Traffic Server (see :ref:`stop-traffic-server`)
+#. Enter the following command to clear the cache::
+
+ traffic_server -Cclear
+
+ The ``clear`` command deletes all data in the object store and the
+ host database. Traffic Server does not prompt you to confirm the
+ deletion.
+
+#. Restart Traffic Server (see :ref:`start-traffic-server`).
+
+Removing an Object From the Cache
+=================================
+
+Traffic Server accepts the custom HTTP request method ``PURGE`` when
+removing a specific object from cache. If the object is found in the
+cache and is successfully removed, then Traffic Server responds with a
+``200 OK`` HTTP message; otherwise, a ``404 File Not Found`` message is
+returned.
+
+.. note::
+
+ By default, the PURGE request method is only processed if received on
+ the localhost interface.
+
+In the following example, Traffic Server is running on the domain
+``example.com`` and you want to remove the image ``remove_me.jpg``
+from cache. Because by default we do not permit ``PURGE`` requests from
+any other IP, we connect to the daemon via localhost: ::
+
+ $ curl -X PURGE -H 'Host: example.com' -v "http://localhost/remove_me.jpg"
+ * About to connect() to localhost port 80 (#0)
+ * Trying 127.0.0.1... connected
+ * Connected to localhost (127.0.0.1) port 80 (#0)
+
+ > PURGE /remove_me.jpg HTTP/1.1
+ > User-Agent: curl/7.19.7
+ > Host: example.com
+ > Accept: */*
+ >
+ < HTTP/1.1 200 Ok
+ < Date: Thu, 08 Jan 2010 20:32:07 GMT
+ < Connection: keep-alive
+
+The next time Traffic Server receives a request for the removed object,
+it will contact the origin server to retrieve a new copy, which will replace
+the previously cached version in Traffic Server.
+
+This procedure only removes an object from a specific Traffic Server cache.
+Users may still see the old (removed) content if it was cached by intermediary
+caches or by the end-users' web browser.
+
+.. _inspecting-the-cache:
+
+Inspecting the Cache
+====================
+
+Traffic Server provides a Cache Inspector utility that enables you to
+view, delete, and invalidate URLs in the cache (HTTP only). The Cache
+Inspector utility is a powerful tool that's capable of deleting all
+the objects in your cache. Therefore, make sure that only authorized
+administrators are allowed to access this utility through proper use
+of the ``@src_ip`` option in :file:`remap.config` and the instructions
+detailed in :ref:`controlling-client-access-to-cache`.
+
+Accessing the Cache Inspector Utility
+-------------------------------------
+
+To access the Cache Inspector utility:
+
+#. Set :ts:cv:`proxy.config.http_ui_enabled` to ``1``.
+#. To access the cache inspector in reverse proxy mode, you must add a
+ remap rule to :file:`remap.config` to expose the URL. This should be
+ restricted to a limited set of hosts using the ``@src_ip`` option.
+ To restrict access to the network 172.28.56.0/24, use ::
+
+ map http://yourhost.com/myCI/ http://{cache} @action=allow @src_ip=172.28.56.1-172.28.56.254
+
+#. Reload the Traffic Server configuration by running :option:`traffic_ctl config reload`.
+#. Open your web browser and go to the the following URL::
+
+ http://yourhost/myCI/
+
+ You will now be presented with the Cache Inspector interface.
+
+Using the Cache Inspector Utility
+---------------------------------
+
+The Cache Inspector Utility provides several options that enable you to view and
+delete the contents of your cache.
+
+Lookup URL
+ Search for a particular URL in the cache. When Traffic Server finds the URL
+ in the cache, it will display details of the object that corresponds to the
+ URL (e.g. header length and number of alternates). The option to delete the
+ URL from the cache will be presented.
+
+Delete URL
+ Delete the object from the cache which corresponds to the given URL. Success
+ or failure will be indicated after a delete has been attempted.
+
+Regex Lookup
+ Search URLs within the cache using one or more regular expressions.
+
+Regex Delete
+ Deletes all objects from the cache which match the provided regular
+ expressions.
+
+Regex Invalidate
+ Marks any objects in the cache which match the given regular expressions as
+ stale. Traffic Server will contact the relevant origin server(s) to confirm
+ the validity and freshness of the cached object, updating the cached object
+ if necessary.
+
+.. note::
+
+ Only one administrator should delete and invalidate cache
+ entries from the Cache Inspector at any point in time. Changes made by
+ multiple administrators at the same time can lead to unpredictable
+ results.
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/transparent-proxy.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/transparent-proxy.en.rst b/doc/admin-guide/transparent-proxy.en.rst
new file mode 100644
index 0000000..7cb76f3
--- /dev/null
+++ b/doc/admin-guide/transparent-proxy.en.rst
@@ -0,0 +1,118 @@
+.. _transparent-proxy:
+
+Transparent Proxying
+********************
+
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. toctree::
+ :maxdepth: 2
+
+ transparent-proxy/build.en
+ transparent-proxy/bridge.en
+ transparent-proxy/router-inline.en
+ transparent-proxy/wccp-configuration.en
+ transparent-proxy/wccp-service-config.en
+
+Transparent Proxying is the ability of a proxy (such as ATS) to
+intercept connections between clients and servers without being visible.
+
+The general network structure that will be used in this documentation is
+shown in the following figure.
+
+.. figure:: ../static/images/admin/ats-basic-traffic.png
+ :align: center
+ :alt: ATS basic traffic flow of Transparent Proxy
+
+ ATS basic traffic flow of Transparent Proxy
+
+There must be a gateway device through which all network traffic passes
+from the client to the Internet (or external cloud). The gateway is
+responsible for effectively splicing ATS in to selected streams of that
+traffic. Each traffic stream is split in two, with ATS terminating
+both sides. That is, stream green-1, red-2, is split in to the green
+connection and the red connection. Note that ATS may or may not be on
+the gateway system, the redirected traffic can flow over other network
+infrastructure.
+
+Because ATS uses two connections, transparency can be set independently
+on the client and origin server (Internet / external cloud) side. We
+will define what is generally called "transparent proxy" as two aspects,
+*inbound transparency* and *outbound transparency*.
+
+Inbound transparency is a proxy that is transparent to connections that
+are inbound to the proxy, i.e. a connection initiated by a client which
+connects to the proxy (green-1). Similarly, outbound transparency is a
+proxy that is transparent to connections that are outbound from the
+proxy, i.e. a connection initiated by the proxy to an origin server
+(red-2).
+
+In most situations these two types of transparency are combined, but that is
+not required. Traffic Server supports transparency independently on the two
+sides.
+
+.. important::
+
+ It is critical to note that any transparency requires specialized
+ routing and cannot be done solely by configuring ATS. ATS transparency
+ also requires support from the Linux kernel and therefore currently only
+ works on sufficiently recent Linux kernels that support the following
+ features:
+
+ - TPROXY
+ - POSIX capabilities
+
+ In addition the specialized routing will require using ``iptables`` and
+ in some cases ``ebtables``.
+
+Standard build procedures should work for transparency support but if
+not consult these :ref:`more detailed instructions <building-ats-for-transparency>`.
+
+Transparency is configured per server port, not globally. This is done
+via the configuration values :ts:cv:`proxy.config.http.server_ports`.
+In addition, :ts:cv:`proxy.config.reverse_proxy.enabled` must be enabled if the
+client side is transparent. That should be fixed in a future patch.
+
+.. XXX has that been fixed?
+
+.. XXX revisit section below
+
+In the first case use the attribute character (replacing the default
+'X')
+
+**Attribute** **Transparency Style** **Reverse Proxy**
+
+``=``
+ Full transparency: either
+
+``>``
+ Inbound (client) transparency: enabled
+
+``<``
+ Outbound (origin server) transparency: either
+
+In the outbound transparent case clients must connect directly to ATS
+either through an explicit proxy mechanism or by advertising the IP
+address of the ATS server via DNS as the origin server address.
+
+Some tested scenarios --
+
+- :doc:`transparent-proxy/bridge.en`
+- :doc:`transparent-proxy/router-inline.en`
+- :doc:`transparent-proxy/wccp-configuration.en`
+
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/transparent-proxy/bridge.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/transparent-proxy/bridge.en.rst b/doc/admin-guide/transparent-proxy/bridge.en.rst
new file mode 100644
index 0000000..8435ad9
--- /dev/null
+++ b/doc/admin-guide/transparent-proxy/bridge.en.rst
@@ -0,0 +1,177 @@
+Inline on a Linux Bridge
+************************
+
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
+
+A Linux can be configured to operate in `bridge mode <http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge>`_.
+Two or more physical interfaces are assigned to the bridge. A single IP
+address is shared across the interfaces. By default any packet that
+arrives on one interface is immediately routed out another bridge
+interface.
+
+Linux packages required:
+
+- bridge-utils
+- ebtables
+
+In our example of setting up bridge mode we will use a local address of
+192.168.1.11/24 and interfaces ``eth0`` and ``eth1`` as the bridge
+interfaces (more detailed documentation is available
+`here <http://www.tldp.org/HOWTO/BRIDGE-STP-HOWTO/preparing-the-bridge.html>`_).
+You may omit the '#' character and everything after it. ::
+
+ brctl addbr br0 # create bridge device
+ brctl stp br0 off # Disable spanning tree protocol
+ brctl addif br0 eth0 # Add eth0 to bridge
+ brctl addif br0 eth1 # Add eth1 to bridge
+
+ ifconfig eth0 0 0.0.0.0 # Get rid of interface IP addresses
+ ifconfig eth1 0 0.0.0.0 # ditto # Set the bridge IP address and enable it
+ ifconfig br0 192.168.1.11 netmask 255.255.255.0 up
+
+If you have not already done so, remember to add a default route, such
+as this one for a gateway of 192.168.1.1. ::
+
+ ip route add default via 192.168.1.1
+
+At this point it is a good idea to test connectivity to verify the basic
+bridge is functional.
+
+Once the bridge is verified to work, this is the basic traffic pattern
+of interest.
+
+.. figure:: ../../static/images/admin/ats-traffic-bridge.png
+ :align: center
+ :alt: Picture of traffic flow through a bridge with ATS
+
+ Picture of traffic flow through a bridge with ATS
+
+The green arrows are packets originating from the client and the red
+arrows are packets originating from the origin server. All traffic not
+directed to the local address will pass through the bridge. We need to
+break into some of the traffic and subject it to routing so that it can
+be routed to ATS. This requires ``ebtables``. The flows we want to
+intercept are green 1 (from client to bridge) and red 1 (origin server
+to bridge).
+
+In this example we will intercept port 80 (HTTP) traffic. We will use
+the ``BROUTING`` chain because it is traversed only for packets that
+originated externally and arrived on a (forwarding enabled) interface.
+Although it looks like this will intercept all port 80 traffic it will
+only affect the two flows described above. ``-j redirect`` marks the
+packet as being diverted to the bridge and not forwarded, and the
+``DROP`` target puts the packets in the normal ``iptables`` routing so
+that we can use standard device tests on them [1]_. Although this
+example handles only port 80, other ports are the same except for the
+port value. Note also the port here is the port from the point of view
+of the clients and origin servers, not the Traffic Server server port. ::
+
+ ebtables -t broute -F # Flush the table
+ # inbound traffic
+ ebtables -t broute -A BROUTING -p IPv4 --ip-proto tcp --ip-dport 80 \
+ -j redirect --redirect-target DROP
+ # returning outbound traffic
+ ebtables -t broute -A BROUTING -p IPv4 --ip-proto tcp --ip-sport 80 \
+ -j redirect --redirect-target DROP
+
+Traffic Server operates at layer 3 so we need to use ``iptables`` to
+handle IP packets appropriately.::
+
+ iptables -t mangle -A PREROUTING -i eth1 -p tcp -m tcp --dport 80 \
+ -j TPROXY --on-ip 0.0.0.0 --on-port 8080 --tproxy-mark 1/1
+ iptables -t mangle -A PREROUTING -i eth0 -p tcp -m tcp --sport 80 \
+ -j MARK --set-mark 1/1
+
+At this point the directionality of the interfaces matters. For the
+example ``eth1`` is the inbound (client side) interface, while ``eth0``
+is the outbound (origin server side) interface. We mark both flows of
+packets so that we can use policy routing on them. For inbound packets
+we need to use ``TPROXY`` to force acceptance of packets to foreign IP
+addresses. For returning outbound packets there will be a socket open
+bound to the foreign address, we need only force it to be delivered
+locally. The value for ``--on-ip`` is 0 because the target port is
+listening and not bound to a specific address. The value for
+``--on-port`` must match the Traffic Server server port. Otherwise its
+value is arbitrary. ``--dport`` and ``--sport`` specify the port from
+the point of view of the clients and origin servers.
+
+Once the flows are marked we can force them to be delivered locally via
+the loopback interface via a policy routing table.::
+
+ ip rule add fwmark 1/1 table 1
+ ip route add local 0.0.0.0/0 dev lo table 1
+
+The marking used is arbitrary but it must be consistent between
+``iptables`` and the routing rule. The table number must be in the range
+1..253.
+
+To configure Traffic Server set the following values in
+:file:`records.config`
+
+- :ts:cv:`proxy.config.http.server_ports` *value from* ``--on-port`` (see below)
+
+- :ts:cv:`proxy.config.reverse_proxy.enabled` ``1``
+
+- :ts:cv:`proxy.config.url_remap.remap_required` ``0``
+
+You may also need to set :ts:cv:`proxy.config.cluster.ethernet_interface` to
+"br0" (the name of the bridge interface from the Bridge Commands).
+
+Additional troubleshooting
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+* Check to make sure that ``iptables`` is not filtering (blocking)
+ incoming HTTP connections.
+
+ It is frequently the case that the default tables prevent incoming HTTP. You can clear all filters with the
+ commands::
+
+ iptables -t filter --flush FORWARD
+ iptables -t filter --flush INPUT
+
+ That is a bit drastic and should only be used for testing / debugging. A
+ live system will likely need some filters in place but that is beyond
+ the scope of this document. If this fixes the problem, then your filter
+ set is too restrictive.
+
+ Note that this problem will prevent the basic bridge (without ATS) from
+ allowing HTTP traffic through.
+
+* Verify that IP packet forwarding is enabled.
+
+ You can check this with::
+
+ cat /proc/sys/net/ipv4/ip_forward
+
+ The output should be a non-zero value (usually '1'). If it is zero, you
+ can set it with::
+
+ echo '1' > /proc/sys/net/ipv4/ip_forward
+
+ This can setting can be persisted by putting it in ``/etc/sysctl.conf``: ::
+
+ net/ipv4/ip_forward=1
+
+.. rubric:: Footnotes
+
+.. [1]
+ The ``--redirect-target`` can be omitted, but then the ``iptables``
+ rules would need to use ``--physdev`` instead of just ``-i``. The
+ actual packet processing is identical.
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/transparent-proxy/build.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/transparent-proxy/build.en.rst b/doc/admin-guide/transparent-proxy/build.en.rst
new file mode 100644
index 0000000..e547a06
--- /dev/null
+++ b/doc/admin-guide/transparent-proxy/build.en.rst
@@ -0,0 +1,60 @@
+.. _building-ats-for-transparency:
+
+Building ATS for transparency
+*****************************
+
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
+In most cases, if your environment supports transparency then
+``configure`` will automatically enable it. For other environments you
+may need to twiddle the ``configure`` options.
+
+``--enable-posix-cap``
+ This enables POSIX capabilities, which are required for
+ transparency. These are compiled in by default. To check your
+ system, look for the header file ``sys/capability.h`` and the system
+ library ``libcap``. These are in the packages ``libcap`` and
+ ``libcap-devel`` or ``libcap-dev`` (depending on the Distribution)
+ contra-respectively.
+
+``--enable-tproxy[=value]``
+ Enable TPROXY support, which is the Linux kernel feature used for
+ transparency. This should be present in the base installation, there
+ is no package associated with it. \* ``auto`` Do automatic checks
+ for the the TPROXY header file (``linux/in.h``) and enable TPROXY
+ support if the ``IP_TRANSPARENT`` definition is present. This is the
+ default if this option is not specified or ``value`` is omitted. \*
+ ``no`` Do not check for TPROXY support, disable support for it. \*
+ ``force`` Do not check for TPROXY support, enable it using the $ats@
+ built in value for ``IP_TRANSPARENT``. This is useful for systems
+ that have it in the kernel for but some reason do not have the
+ appropriate system header file. \* *number* Do not check for TPROXY
+ support, use *number* as the ``IP_TRANSPARENT`` value. There are, at
+ present, no known standard distributions of Linux that support
+ TPROXY but use a value different from the built in ATS default.
+ However, a custom built kernel may do so and in that case the
+ specific value can be specified.
+
+In the default case, ATS configuration will automatically check for
+TPROXY support via the presence of the ``linux/in.h`` header file and
+compile in TPROXY support if it is available. If that fails, you may be
+able to recover by using one of the options above. Note that
+transparency may be built in by default but it is not active unless
+explicitly enabled in the ATS configuration files.
+
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/transparent-proxy/router-inline.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/transparent-proxy/router-inline.en.rst b/doc/admin-guide/transparent-proxy/router-inline.en.rst
new file mode 100644
index 0000000..0e9060a
--- /dev/null
+++ b/doc/admin-guide/transparent-proxy/router-inline.en.rst
@@ -0,0 +1,87 @@
+Inline on a Linux router
+************************
+
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+The routed set up presumes the set of clients are on distinct networks
+behind a single physical interface. For the purposes of this example
+will we presume
+
+- The clients are on network 172.28.56.0/24
+- The router connects the networks 172.28.56.0/24 and 192.168.1.0/24
+- Interface ``eth0`` is on the network 192.168.1.0/24
+- Interface ``eth1`` is on the network 172.28.56.0/24
+- The router is already configured to route traffic correctly for the
+ clients.
+
+In this example we will intercept port 80 (HTTP) traffic that traverses
+the router. The first step is to use ``iptables`` to handle IP packets
+appropriately.
+
+::
+
+ # reflow client web traffic to TPROXY
+ iptables -t mangle -A PREROUTING -i eth1 -p tcp -m tcp --dport 80 -j TPROXY \
+ --on-ip 0.0.0.0 --on-port 8080 --tproxy-mark 1/1
+ # Let locally directed traffic pass through.
+ iptables -t mangle -A PREROUTING -i eth0 --source 192.168.1.0/24 -j ACCEPT
+ iptables -t mangle -A PREROUTING -i eth0 --destination 192.168.1.0/24 -j ACCEPT
+ # Mark presumed return web traffic
+ iptables -t mangle -A PREROUTING -i eth0 -p tcp -m tcp --sport 80 -j MARK --set-mark 1/1
+
+We mark packets so that we can use policy routing on them. For inbound
+packets we use ``TPROXY`` to make it possible to accept packets sent to
+foreign IP addresses. For returning outbound packets there will be a
+socket open bound to the foreign address, we need only force it to be
+delivered locally. The value for ``--on-ip`` is 0 because the target
+port is listening and not bound to a specific address. The value for
+``--on-port`` must match the Traffic Server server port. Otherwise its
+value is arbitrary. ``--dport`` and ``--sport`` specify the port from
+the point of view of the clients and origin servers. The middle two
+lines exempt local web traffic from being marked for Traffic Server --
+these rules can be tightened or loosened as needed. They server by
+matching traffic and exiting the ``iptables`` processing via ``ACCEPT``
+before the last line is checked.
+
+Once the flows are marked we can force them to be delivered locally via
+the loopback interface via a policy routing table.
+
+::
+
+ ip rule add fwmark 1/1 table 1
+ ip route add local 0.0.0.0/0 dev lo table 1
+
+The marking used is arbitrary but it must be consistent between
+``iptables`` and the routing rule. The table number must be in the range
+1..253.
+
+To configure Traffic Server set the following values in
+:file:`records.config`
+
+``proxy.config.http.server_ports``
+ ``STRING``
+ Default: *value from* ``--on-port``
+
+``proxy.config.reverse_proxy.enabled``
+ ``INT``
+ Default: ``1``
+
+``proxy.config.url_remap.remap_required``
+ ``INT``
+ Default: ``0``
+
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/transparent-proxy/wccp-configuration.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/transparent-proxy/wccp-configuration.en.rst b/doc/admin-guide/transparent-proxy/wccp-configuration.en.rst
new file mode 100644
index 0000000..a36401c
--- /dev/null
+++ b/doc/admin-guide/transparent-proxy/wccp-configuration.en.rst
@@ -0,0 +1,156 @@
+WCCP Configuration
+******************
+
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. _wccp-configuration:
+
+.. toctree::
+ :maxdepth: 2
+
+`WCCP <http://cio.cisco.com/en/US/docs/ios/12_0t/12_0t3/feature/guide/wccp.html>`_
+is de-facto semi-standard used by routers to redirect network traffic to
+caches. It is available on most Cisco™ routers although it does not
+appear to be officially supported by Cisco. The primary benefits of WCCP
+are
+
+- If already have a router that supports WCCP inline you do not
+ have to change your network topology.
+- WCCP fails open so that if the Traffic Server machine fails, it is
+ bypassed and users continue to have Internet access.
+
+Use of WCCP only makes sense for client side transparency [1]_
+because if the clients are explicitly proxied by Traffic Server there's
+no benefit to WCCP fail open, as the clients will continue to directly
+access the unresponsive Traffic Server host. It would be better to
+adjust the routing tables on the router for explicit proxying.
+
+Because the router serves as the inline network element, Traffic Server
+must run on a separate host. This host can be located anywhere as long
+as Traffic Server is either on the same network segment or a GRE tunnel
+can be maintained between the Traffic Server host and the router.
+
+|important| This document presumes that the router is already properly
+configured to handle traffic between the clients and the origin servers.
+If you are not certain, verify it before attempting to configure Traffic
+Server with WCCP. This is also a good state to which to revert should
+the configuration go badly.
+
+Configuration overview
+======================
+
+Setting WCCP is a three step process, first configuring the router, the
+Traffic Server host, and Traffic Server.
+
+|image1| The router will **not** respond to WCCP protocol packets unless
+explicitly configured to do so. Via WCCP, the router can be made to
+perform packet interception and redirection needed by Traffic Server
+transparency. The WCCP protocol in effect acts as means of controlling a
+rough form of policy routing with positive heartbeat cutoff.
+
+The Traffic Server host system must also be configured using
+``iptables`` to accept connections on foreign addresses. This is done
+roughly the same way as the standard transparency use.
+
+Finally Traffic Server itself must be configured for transparency and
+use of WCCP. The former is again very similar to the standard use, while
+WCCP configuration is specific to WCCP and uses a separate configuration
+file, referred to by the :file:`records.config` file.
+
+The primary concern for configuration in which of three basic topologies
+are to be used.
+
+- Dedicated -- Traffic Server traffic goes over an interface that is
+ not used for client nor origin server traffic.
+- Shared -- Traffic Server traffic shares an interface with client or
+ server traffic.
+- Inside Shared -- Traffic Server and client traffic share an
+ interface.
+- Outside Shared -- Traffic Server and
+ origin server traffic share an interface.
+
+In general the dedicated topology is preferred. However, if the router
+has only two interfaces one of the shared topologies will be
+required [2]_ Click the links above for more detailed configuration
+information on a specific topology.
+
+Shared interface issues
+-----------------------
+
+A shared interface topology has additional issues compared to a
+dedicated topology that must be handled. Such a topology is required if
+the router has only two interfaces, and because of these additional
+issues it is normally only used in such cases, although nothing prevents
+it use even if the router has three or more interfaces.
+
+The basic concept for a shared interface is to use a tunnel to simulate
+the dedicated interface case. This enables the packets to be
+distinguished at layer 3. For this reason, layer 2 redirection cannot be
+used because the WCCP configuration cannot distinguish between packets
+returning from the origin server and packets returning from Traffic
+Server as they are distinguished only by layer 2 addressing [3]_.
+Fortunately the GRE tunnel used for packet forwarding and return can be
+used as the simulated interface for Traffic Server.
+
+Frequently encountered problems
+-------------------------------
+
+MTU and fragmentation
+~~~~~~~~~~~~~~~~~~~~~
+
+In most cases the basic configuration using a tunnel in any topology can
+fail due to issues with fragmentation. The socket logic is unable to
+know that its packets will eventually be put in to a tunnel which will
+by its nature have a smaller
+`MTU <http://en.wikipedia.org/wiki/Maximum_transmission_unit>`_ than the
+physical interface which it uses. This can lead to pathological behavior
+or outright failure as the packets sent are just a little too big. It is
+not possible to solve easily by changing the MTU on the physical
+interface because the tunnel interface uses that to compute its own MTU.
+
+References
+==========
+
+- `WCCP Router Configuration Commands - IOS
+ 12.2 <http://www.cisco.com/en/US/docs/ios/12_2/configfun/command/reference/frf018.html>`_
+
+
+
+
+
+
+
+
+.. [1]
+ Server side transparency should also be used, but it is not as
+ significant. In its absence, however, origin servers may see the
+ source address of connections suddenly change from the Traffic Server
+ address to client addresses, which could cause problems. Further, the
+ primary reason for not having server side transparency is to hide
+ client addresses which is defeated if the Traffic Server host fails.
+
+.. [2]
+ If your router has only one interface, it's hardly a *router*.
+
+.. [3]
+ This is not fundamentally impossible, as the packets are distinct in
+ layer
+
+.. |important| image:: ../../static/images/docbook/important.png
+.. |image1| image:: ../../static/images/docbook/important.png
+
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/transparent-proxy/wccp-service-config.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/transparent-proxy/wccp-service-config.en.rst b/doc/admin-guide/transparent-proxy/wccp-service-config.en.rst
new file mode 100644
index 0000000..7726ebe
--- /dev/null
+++ b/doc/admin-guide/transparent-proxy/wccp-service-config.en.rst
@@ -0,0 +1,75 @@
+WCCP Service Configuration
+**************************
+
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. _wccp-service-configuration:
+
+The service definition file is used by
+:ref:`traffic_wccp` and :ref:`traffic-server` directly.
+The elements in the security definition file are inspired by the `WCCP RFC (8/2012) <http://tools.ietf.org/html/draft-mclaggan-wccp-v2rev1-00>`_. There is also an older version of the RFC that shows up commonly in search results, `WCCP (4/2001) <https://tools.ietf.org/id/draft-wilson-wrec-wccp-v2-01.txt>`_, and was the RFC reference used in the original WCCP support for Traffic Server several years ago.
+
+A sample service group file is included in the source tree under cmd/traffic_wccp.
+
+Security Section
+================
+
+In the security section, you can define a password shared between the WCCP Client and the WCCP router. This password is used to encrypt the WCCP traffic. It is optional, but highly recommended.
+
+Attributes in this section
+
+* option - Must be set to MD5 if you want to encrypt your WCCP traffic
+
+* key – The same password that you set with the associated WCCP router.
+
+Services Section
+================
+
+In the services section you can define a list of service groups. Each top level entry is a separate service group.
+
+Service group attributes include
+
+* name – A name for the service. Not used in the rest of the WCCP processing.
+
+* description – A description of the service. Again, not used in the rest of the WCCP processing.
+
+* id - The security group ID. It must match the service group ID that has been defined on the associated WCCP router. This is the true service group identifier from the WCCP perspective.
+
+* type – This defines the type of service group either “STANDARD” or “DYNAMIC”. There is one standard defined service group, HTTP with the id of 0. The 4/2001 RFC indicates that id’s 0-50 are reserved for well known service groups. But more recent 8/2012 RFC indicates that values 0 through 254 are valid service id’s for dynamic services. To avoid differences with older WCCP routers, you probably want to avoid dynamic service ID’s 0 through 50.
+
+* priority – This is a value from 0 to 255. The higher number is a higher priority. Well known (STANDARD) services are set to a value of 240. If there are multiple service groups that could match a given packet, the higher priority service group is applied. RFC For example, you have service group 100 defined for packets with destination port 80, and service group 101 defined for packets with source port 1024. For a packet with destination port set to 80 and source port set to 1024, the priorities of the service groups would need to be compared to determine which service group applies.
+
+* protocol – This is IP protocol number that should match. Generally this is set to 6 (TCP) or 17 (UDP).
+
+* assignment – WCCP supports multiple WCCP clients supporting a single service group. However, the current WCCP client implementation in Traffic Server assumes there is only a single WCCP client, and so creates assignment tables that will direct all traffic to that WCCP client. The assignment type is either hash or mask, and if it is not set, it defaults to hash. If Traffic Server ever supports more than one cache, it will likely only support a balanced hash assignment. The mask/value assignment seems to be better suited to situations where the traffic needs to be more strongly controlled.
+
+* primary-hash – This is the element of the packet that is used to compute the primary key. The value options are src_ip, dst_ip, src_port, or dst_port. This entry is a list, so multiple values can be specified. In that case, all the specified packet attributes will be used to compute the hash bucket. In the current implementation, the primary hash value does not matter, since the client always generates a hash table that directs all matching traffic to it. But if multiple clients are ever supported, knowledge of the local traffic distribution could be used to pick a packet attribute that will better spread traffic over the WCCP clients.
+* alt-hash – The protocol supports a two level hash. This attribute is a list with the same value options as for primary-hash. Again, since the current Traffic Server implementation only creates assignment tables to a single client, specifying the alt-hash values does nothing.
+
+* ports – This is a list of port values. Up to 8 port values may be included in a service group definition.
+
+* port-type – This attribute can have the value of src or dst. If not specified, it defaults to dst. It indicates whether the port values should be interpreted as source ports or destination ports.
+
+* forward – This is a list. The list of the values can be GRE or L2. This advertises how the client wants to process WCCP packets. GRE means that the packets will be delivered in a GRE tunnel. This is the default. L2 means that the client is on the same network and can get traffic delivered to it from the router by L2 routing (MAC addresses).
+
+* return – The WCCP protocol allows a WCCP client to decline a packet and return it back to the router. The current WCCP client implementation never does this. The value options are the same as for the forward attribute.
+
+* routers – This is the list of router addresses the WCCP client communicates with. The WCCP protocols allows for multiple WCCP routers to be involved in a service group. The multiple router scenario has at most been lightly tested in the Traffic Server implementation.
+
+* proc-name – This attribute is only used by traffic_wccp. It is not used in the traffic_server WCCP support. This is the path to a process’ PID file. The service group is advertised to the WCCP router if the process identified in the PID file is currently operational.
+
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin/cluster-howto.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin/cluster-howto.en.rst b/doc/admin/cluster-howto.en.rst
deleted file mode 100644
index aa37530..0000000
--- a/doc/admin/cluster-howto.en.rst
+++ /dev/null
@@ -1,180 +0,0 @@
-.. _traffic-server-cluster:
-
-Traffic Server Cluster
-**********************
-
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-Traffic Server scales from a single node to multiple nodes that form a
-cluster allowing you to improve system performance and reliability.
-
-.. toctree::
- :maxdepth: 1
-
-Understanding Traffic Server Clusters
-=====================================
-
-A Traffic Server cluster consists of multiple Traffic Server nodes. The
-nodes in a cluster share configuration information and can form a single
-logical cache. Traffic Server detects the addition and deletion of nodes
-in the cluster automatically and can detect when a node is unavailable.
-Traffic Server uses its own protocol for clustering, which is multicast
-for node location and heartbeat, but unicast for all data exchange
-within the cluster. Traffic Server has two clustering modes:
-
-- Management-only mode; refer to `Management-Only Clustering`_ below.
-- Full-clustering mode; refer to `Full Clustering`_
-
-Management-Only Clustering
-==========================
-
-In management-only clustering mode, Traffic Server cluster nodes share
-configuration information. You can administer all the nodes at the same
-time. Traffic Server uses a multicast management protocol to provide a
-single system image of your Traffic Server cluster. Information about
-cluster membership, configuration, and exceptions is shared across all
-nodes, and the :program:`traffic_manager` process automatically propagates
-configuration changes to all the nodes.
-
-Full Clustering
-===============
-
-In full-clustering mode, as well as sharing configuration information, a
-Traffic Server cluster distributes its cache across its nodes into a
-single, virtual object store, rather than replicating the cache node by
-node. Traffic Server can provide an enormous aggregate cache size and
-can maximize cache hit rate by storing objects only once across the
-entire cluster.
-
-A fully clustered Traffic Server maps objects to specific nodes in the
-cluster. When a node receives a request, it checks to see if the request
-is a hit somewhere in the cluster. If the request is a hit on a
-different node, the node handling the request obtains the object from
-the hit node and serves it to the client. Traffic Server uses its own
-communication protocol to obtain an object from sibling cluster nodes.
-
-If a node fails or is shut down and removed, Traffic Server removes
-references to the missing node on all nodes in the cluster.
-
-Full clustering recommends a dedicated network interface for cluster
-communication to achieve better performance.
-
-Enabling Clustering Mode
-========================
-
-Before a node is added into a cluster, please ensure that the following
-conditions are being met:
-
-- All nodes are running the same operating system:
-
- - The same distribution, e.g.: RHEL 6.6
- - The same kernel, e.g.: 2.6.32-504.23.4.el6
- - The same architecture, e.g.: x86_64
-
-- All nodes have the same version of Traffic Server installed
-- All nodes are composed of the same hardware
-- All nodes are on the same switch or same VLAN.
-
-Traffic Server does not apply the clustering mode change to all the
-nodes in the cluster. You must change the clustering mode on each node
-individually. You may following these instructions:
-
-1. Setup the same cluster name, with :ts:cv:`proxy.config.proxy_name`, e.g. ``MyCluster``.
-
-2. Set :ts:cv:`proxy.local.cluster.type` to ``1`` to enable cluster mode. The
- following values of this configuration are valid:
-
-================= ==================================================
-Value Description
-================= ==================================================
-1 Full-Clustering mode
-2 Management-Only mode
-3 No clustering (*default*)
-================= ==================================================
-
-3. Configure :ts:cv:`proxy.config.cluster.ethernet_interface`, e.g.: ``eth0``.
- This should be replaced with the node's real interface. We recommends a
- dedicated physical interface here. Refer to :ts:cv:`proxy.local.cluster.type`
- for a full description.
-
-4. Enable configuration changes::
-
- traffic_ctl config reload
-
-5. Restart traffic server::
-
- traffic_ctl server restart
-
- The :program:`traffic_server` and :program:`traffic_manager` processes will need to
- restart after the change of :ts:cv:`proxy.local.cluster.type` and
- :ts:cv:`proxy.config.cluster.ethernet_interface` have taken place.
-
-Traffic Server will join the cluster in about 10 seconds. To verify the hosts in the
-cluster, you can run::
-
- traffic_ctl metric get proxy.process.cluster.nodes
-
-Cluster node status is also tracked in ``cluster.config`` in the configuration
-directory. This configuration is generated by the system and should not be
-edited. It contains a list of the machines that are currently members of the
-cluster, for example::
-
- # 3
- 127.1.2.3:80
- 127.1.2.4:80
- 127.1.2.5:80
-
-After successfully joining a cluster, all changes of global configurations
-performed on any node in that cluster will take effect on all nodes, removing
-the need to manually duplicate configuration changes across each node individually.
-
-Deleting Nodes from a Cluster
-=============================
-
-To delete a node from the Traffic Server cluster, return the setting
-:ts:cv:`proxy.local.cluster.type` to the default value ``3`` and reload.
-
-Common issues for Cluster setup
-===============================
-
-1. The Cluster member auto discovery is built upon multi-casting UDP, and as such
- is impossible to setup where multi-casting is not avaliable, such as AWS EC2.
-
-2. The Cluster will depend on ports 8088, 8089, and 8086. These ports must not be
- blocked by any network configurations or firewalls on the network used for
- internal cluster communication.
-
-Performance Tuning for Busy Clusters
-====================================
-
-Beginning with version 3.2.0, Apache Traffic Server can handle multiple internal
-cluster connections and the number of Cluster Threads is configurable. Each
-of the threads will keep one connection open to all peering cluster nodes.
-
-Increasing Cluster Threads
---------------------------
-
-In the cluster environment, the current performance of the cluster threads
-will consume the same cpu usage as a normal network thread. It's reasonable
-to keep roughly the same number of cluster threads as network threads. For
-example, if you are running a system with 10 network processing threads,
-you can set the number of cluster threads by modifying
-:ts:cv:`proxy.config.cluster.threads` to ``10``::
-
- traffic_ctl config set proxy.config.cluster.threads 10
-