You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficserver.apache.org by jp...@apache.org on 2015/11/03 07:10:15 UTC
[39/51] trafficserver git commit: Documentation reorganization
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/introduction.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/introduction.en.rst b/doc/admin-guide/introduction.en.rst
new file mode 100644
index 0000000..7fbed69
--- /dev/null
+++ b/doc/admin-guide/introduction.en.rst
@@ -0,0 +1,297 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. include:: ../common.defs
+
+.. _admin-introduction:
+
+Introduction
+************
+
+Global data networking has become part of everyday life: Internet users
+request billions of documents and petabytes of data, on a daily basis,
+to and from all parts of the world. Information is free, abundant, and
+accessible. Unfortunately, global data networking can also be a
+nightmare for IT professionals as they struggle with overloaded servers
+and congested networks. It can be challenging to consistently and
+reliably accommodate society’s growing data demands.
+
+Traffic Server is a high-performance web proxy cache that improves
+network efficiency and performance by caching frequently-accessed
+information at the edge of the network. This brings content physically
+closer to end users, while enabling faster delivery and reduced
+bandwidth use. Traffic Server is designed to improve content delivery
+for enterprises, Internet service providers (ISPs), backbone providers,
+and large intranets by maximizing existing and available bandwidth.
+
+Traffic Server Deployment Options
+=================================
+
+To best suit your needs, Traffic Server can be deployed in several ways:
+
+- As a web proxy cache
+- As a reverse proxy
+- In a cache hierarchy
+
+The following sections provide a summary of these Traffic Server
+deployment options. Please keep in mind that with every one of these options
+Traffic Server can be run as a *single instance*, or as a *multi-node cluster*.
+
+Traffic Server as a Web Proxy Cache
+-----------------------------------
+
+As a web proxy cache, Traffic Server receives user requests for web
+content as those requests travel to the destined web server (origin
+server). If Traffic Server contains the requested content, then it
+serves the content directly. If the requested content is not available
+from cache, then Traffic Server acts as a proxy: it obtains the content
+from the origin server on the user’s behalf and also keeps a copy to
+satisfy future requests.
+
+Traffic Server provides explicit proxy caching, in which the user’s
+client software must be configured to send requests directly to Traffic
+Server. Explicit proxy caching is described in the :ref:`explicit-proxy-caching`
+chapter.
+
+Traffic Server can also be employed as a transparent caching proxy server, in
+which the client software needs no special configuration or even knowledge of
+the proxy's existence. This setup is described in the :ref:`transparent-proxy`
+section.
+
+Traffic Server as a Reverse Proxy
+---------------------------------
+
+As a reverse proxy, Traffic Server is configured to be the origin server
+to which the user is trying to connect (typically, the origin server’s
+advertised hostname resolves to Traffic Server, which acts as the real
+origin server). The reverse proxy feature is also called server
+acceleration. Reverse proxy is described in more detail in
+:ref:`reverse-proxy-and-http-redirects`.
+
+Traffic Server in a Cache Hierarchy
+-----------------------------------
+
+Traffic Server can participate in flexible cache hierarchies, in which
+Internet requests not fulfilled from one cache are routed to other
+regional caches, thereby leveraging the contents and proximity of nearby
+caches. In a hierarchy of proxy servers, Traffic Server can act either
+as a parent or a child cache to other Traffic Server systems or to
+similar caching products.
+
+Traffic Server supports ICP (Internet Cache Protocol) peering.
+Hierarchical caching is described in more detail in :ref:`admin-hierarchical-caching`.
+
+Deployment Limitations
+----------------------
+
+There are a number of deployment options that Traffic Server does not support
+right out of the box. Such functionality may be implemented in a plugin, but
+in some cases Traffic Server's internal APIs or architectural restrictions
+won't make it easy:
+
+* Load Balancing - note that there is an experimental plugin for this: :ref:`balancer-plugin`.
+
+.. XXX needs re-work: the leadin states there's "a number" of scenarios, and then only lists one -- one's a number, but not much of a list
+
+Traffic Server Components
+=========================
+
+Traffic Server consists of several components that work together to form
+a web proxy cache you can easily monitor and configure.
+
+The Traffic Server Cache
+------------------------
+
+The Traffic Server cache consists of a high-speed object database called
+the *object store*. The object store indexes objects according to URLs and
+associated headers. Using sophisticated object management, the object
+store can cache alternate versions of the same object (perhaps in a
+different language or encoding type). It can also efficiently store very
+small and very large objects, thereby minimizing wasted space. When the
+cache is full, Traffic Server removes stale data to ensure that the most
+requested objects are readily available and fresh.
+
+Traffic Server is designed to tolerate total disk failures on any of the
+cache disks. If the disk fails completely, then Traffic Server marks the
+entire disk as corrupt and continues to use remaining disks. If all of
+the cache disks fail, then Traffic Server switches to proxy-only mode.
+You can partition the cache to reserve a certain amount of disk space
+for storing data for specific protocols and origin servers. For more
+information about the cache, see :ref:`http-proxy-caching`.
+
+The RAM Cache
+-------------
+
+Traffic Server maintains a small RAM cache that contains extremely
+popular objects. This RAM cache serves the most popular objects as fast
+as possible and reduces load on disks, especially during temporary
+traffic peaks. You can configure the RAM cache size to suit your needs.
+For detailed information, refer to :ref:`changing-the-size-of-the-ram-cache`.
+
+The Host Database
+-----------------
+
+The Traffic Server host database stores the domain name server (DNS)
+entries of origin servers to which Traffic Server connects to fulfill
+user requests. This information is used to adapt future protocol
+interactions and optimize performance. Along with other information, the
+host database tracks:
+
+- DNS information (for fast conversion of hostnames to IP addresses).
+
+- The HTTP version of each host (so advanced protocol features can be
+ used with hosts running modern servers).
+
+- Host reliability and availability information (so users will not wait
+ for servers that are not running).
+
+The DNS Resolver
+----------------
+
+Traffic Server includes a fast, asynchronous DNS resolver to streamline
+conversion of hostnames to IP addresses. Traffic Server implements the
+DNS resolver natively by directly issuing DNS command packets rather
+than relying on slower, conventional resolver libraries. Since many DNS
+queries can be issued in parallel and a fast DNS cache maintains popular
+bindings in memory, DNS traffic is reduced.
+
+Traffic Server Processes
+------------------------
+
+Traffic Server contains three processes that work together to serve
+requests and manage, control, and monitor the health of the system.
+
+- The :program:`traffic_server` process is the transaction processing engine
+ of Traffic Server. It is responsible for accepting connections,
+ processing protocol requests, and serving documents from the cache or
+ origin server.
+
+- The :program:`traffic_manager` process is the command and control facility
+ of the Traffic Server, responsible for launching, monitoring, and
+ reconfiguring the :program:`traffic_server` process. The :program:`traffic_manager`
+ process is also responsible for the proxy autoconfiguration port, the
+ statistics interface, cluster administration, and virtual IP
+ failover.
+
+ If the :program:`traffic_manager` process detects a :program:`traffic_server`
+ process failure, it instantly restarts the process but also maintains
+ a connection queue of all incoming requests. All incoming connections
+ that arrive in the several seconds before full server restart are
+ saved in the connection queue and processed in first-come,
+ first-served order. This connection queueing shields users from any
+ server restart downtime.
+
+- The :program:`traffic_cop` process monitors the health of both the
+ :program:`traffic_server` and :program:`traffic_manager` processes. The
+ :program:`traffic_cop` process periodically (several times each minute)
+ queries the :program:`traffic_server` and :program:`traffic_manager`
+ processes by issuing heartbeat requests to fetch synthetic web pages. In the
+ event of failure (if no response is received within a timeout interval or
+ if an incorrect response is received), :program:`traffic_cop` restarts the
+ :program:`traffic_manager` and :program:`traffic_server` processes.
+
+The figure below illustrates the three Traffic Server processes.
+
+.. figure:: ../static/images/admin/process.jpg
+ :align: center
+ :alt: Illustration of the three Traffic Server Processes
+
+ Illustration of the three Traffic Server Processes
+
+Administration Tools
+--------------------
+
+Traffic Server offers the following administration options:
+
+- The :program:`traffic_ctl` command-line interface is a
+ text-based interface from which you can monitor Traffic Server performance
+ and network traffic, as well as configure the Traffic Server system. From
+ Traffic Line, you can execute individual commands or script a series of
+ commands in a shell.
+
+- Various configuration files enable you to configure Traffic Server
+ through a simple file-editing and signal-handling interface. Any
+ changes you make through Traffic Line is
+ automatically made to the configuration files as well.
+
+- Finally, there is a clean C API which can be put to good use from a
+ multitude of languages. The Traffic Server Admin Client demonstrates
+ this for Perl.
+
+Traffic Analysis Options
+========================
+
+Traffic Server provides several options for network traffic analysis and
+monitoring:
+
+- :program:`traffic_ctl` enables you to collect and process
+ statistics obtained from network traffic information.
+
+- Transaction logging enables you to record information (in a log file)
+ about every request Traffic Server receives and every error it
+ detects. By analyzing the log files, you can determine how many
+ clients used the Traffic Server cache, how much information each of
+ them requested, and what pages were most popular. You can also see
+ why a particular transaction was in error and what state the Traffic
+ Server was in at a particular time. For example, you can see that
+ Traffic Server was restarted or that cluster communication timed out.
+
+ Traffic Server supports several standard log file formats, such as
+ Squid and Netscape, and its own custom format. You can analyze the
+ standard format log files with off-the-shelf analysis packages. To
+ help with log file analysis, you can separate log files so that they
+ contain information specific to protocol or hosts.
+
+Traffic analysis options are described in more detail in :ref:`monitoring-traffic`.
+
+Traffic Server logging options are described in :ref:`working-with-log-files`.
+
+Traffic Server Security Options
+===============================
+
+Traffic Server provides numerous options that enable you to establish
+secure communication between the Traffic Server system and other
+computers on the network. Using the security options, you can do the
+following:
+
+- Control client access to the Traffic Server proxy cache.
+
+- Configure Traffic Server to use multiple DNS servers to match your
+ site's security configuration. For example, Traffic Server can use
+ different DNS servers, depending on whether it needs to resolve
+ hostnames located inside or outside a firewall. This enables you to
+ keep your internal network configuration secure while continuing to
+ provide transparent access to external sites on the Internet.
+
+- Configure Traffic Server to verify that clients are authenticated
+ before they can access content from the Traffic Server cache.
+
+- Secure connections in reverse proxy mode between a client and Traffic
+ Server, and Traffic Server and the origin server, using the SSL
+ termination option.
+
+- Control access via SSL (Secure Sockets Layer).
+
+Traffic Server security options are described in more detail in
+:ref:`admin-security`.
+
+Tuning Traffic Server
+=====================
+
+Finally, this last chapter on :ref:`performance-tuning` discusses the vast
+number of options that allow administrators to optimally tune Apache Traffic
+Server for maximum performance.
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/monitoring/alarms.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/monitoring/alarms.en.rst b/doc/admin-guide/monitoring/alarms.en.rst
new file mode 100644
index 0000000..f06eb55
--- /dev/null
+++ b/doc/admin-guide/monitoring/alarms.en.rst
@@ -0,0 +1,80 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. include:: ../../common.defs
+
+.. _admin-monitoring-alarms:
+
+Traffic Manager Alarms
+**********************
+
+|TS| signals an alarm when it detects a problem. For example, the space
+allocated to event logs could be full or |TS| may not be able to write to a
+configuration file.
+
+Email Alarms
+============
+
+To configure |TS| to send an email to a specific address whenever an alarm
+occurs, follow the steps below:
+
+#. Set :ts:cv:`proxy.config.alarm_email` in :file:`records.config` to the email
+ address you want to receive alarm notifications. ::
+
+ CONFIG proxy.config.alarm_email STRING "alerts@example.com"
+
+#. Run the command :option:`traffic_ctl config reload` to apply the configuration changes.
+
+Using a Script File for Alarms
+------------------------------
+
+Alarm messages are built into Traffic Server and cannot be changed.
+However, you can write a script file to execute certain actions when an
+alarm is signaled. Traffic Server provides a sample script file named
+``example_alarm_bin.sh`` in the ``bin`` directory which can serve as the
+basis for your custom alarm scripts.
+
+Viewing Statistics from Traffic Line
+====================================
+
+You can use the Traffic Line command-line interface to view statistics
+about Traffic Server performance and web traffic. In addition to viewing
+statistics, you can also configure, stop, and restart the Traffic Server
+system. For additional information, refer to :ref:`configure-using-traffic-line`
+and :program:`traffic_line`. You can view
+specific information about a Traffic Server node or cluster by
+specifying the variable that corresponds to the statistic you want to
+see.
+
+To view a statistic, enter the following command:::
+
+ traffic_ctl metric get VARIABLE
+
+where ``variable`` is the variable representing the information you
+want to view. For a list of variables you can specify, refer to :ref:`Traffic
+Server Metrics <traffic-line-performance-statistics>`.
+
+For example, the following command displays the document hit rate for
+the Traffic Server node:::
+
+ traffic_ctl metric get proxy.node.cache_hit_ratio
+
+If the Traffic Server ``bin`` directory is not in your path, then
+prepend the Traffic Line command with ``./`` (for example:
+:option:`traffic_ctl metric get` ``VARIABLE``).
+
+
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/monitoring/error-messages.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/monitoring/error-messages.en.rst b/doc/admin-guide/monitoring/error-messages.en.rst
new file mode 100644
index 0000000..498ba65
--- /dev/null
+++ b/doc/admin-guide/monitoring/error-messages.en.rst
@@ -0,0 +1,319 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. include:: ../../common.defs
+
+.. _admin-monitoring-errors:
+
+Error Messages
+**************
+
+The following table lists messages that can appear in system log files.
+This list is not exhaustive; it simply describes common warning messages
+that can occur and which might require your attention.
+
+Fatal Process Messages
+======================
+
+``Accept port is not between 1 and 65535. Please check configuration``
+ The port specified in :file:`records.config` that accepts
+ incoming HTTP requests is not valid.
+
+``Self loop is detected in parent proxy configuration``
+ The name and port of the parent proxy match that of Traffic Server.
+ This creates a loop when Traffic Server attempts to send the request
+ to the parent proxy.
+
+Process Warnings
+================
+
+``<Logfile> error: error_number``
+ Generic logging error.
+
+``Bad cluster major version range <version1-version2> for node <IP address> connect failed``
+ Incompatible software versions causing a problem.
+
+``Connect by disallowed client <IP address>, closing``
+ The specified client is not allowed to connect to Traffic Server;
+ the client IP address is not listed in the ``ip_allow.config`` file.
+
+``Could not rename log <filename> to <rolled filename>``
+ System error when renaming log file during roll.
+
+``Did <this amount> of backup; still to do <remaining amount>``
+ Congestion is approaching.
+
+``Different clustering minor versions <version1, version2> for node <IP address> continuing``
+ Incompatible software versions are causing a problem.
+
+``Log format symbol <symbol name> not found``
+ Custom log format references a field symbol that does not exist.
+ Refer to :ref:`admin-event-logging-formats`.
+
+``Missing field for field marker``
+ Error reading a log buffer.
+
+``Unable to open log file <filename>, errno=<error number>``
+ Cannot open the log file.
+
+``Error accessing disk <disk name>``
+ Traffic Server might have a cache read problem. You might need to
+ replace the disk.
+
+``Too many errors accessing disk <disk name>: declaring disk bad``
+ Traffic Server is not using the cache disk because it encountered
+ too many errors. The disk might be corrupt and might have to be
+ replaced.
+
+``No cache disks specified in storage.config file: cache disabled``
+ The Traffic Server :file:`storage.config` file does not list any cache
+ disks; Traffic Server is running in proxy-only mode. You must add
+ the disks you want to use for the cache to :file:`storage.config`.
+
+Alarm Messages
+==============
+
+``[Rollback::Rollback] Config file is read-only: <filename>``
+ Go to the Traffic Server ``config`` directory and check the
+ indicated file permissions; change if necessary.
+
+``[Rollback::Rollback] Unable to read or write config file <filename>``
+ Go to the Traffic Server ``config`` directory and make sure the
+ indicated file exists. Check permissions and modify if necessary.
+
+``[Traffic Manager] Configuration File Update Failed: <error number>``
+ Go to the Traffic Server ``config`` directory and check the
+ indicated file permissions; change if necessary.
+
+``[Traffic Manager] Mgmt <==>Proxy conn. closed``
+ An informational message to inform you that the :program:`traffic_server`
+ process is down.
+
+``Access logging suspended - configured space allocation exhausted.``
+ The space allocated to the event log files is full; you must either
+ increase the space or delete some log files so that access logging
+ to continue. To prevent this error, consider rolling log files more
+ frequently and enabling the autodelete feature.
+
+``Access logging suspended - no more space on the logging partition.``
+ The entire partition containing the event logs is full; you must
+ delete or move some log files to enable access logging to continue.
+ To prevent this error, consider rolling log files more frequently
+ and enabling the autodelete feature.
+
+``Created zero length place holder for config file <filename>``
+ Go to the Traffic Server ``config`` directory and check the
+ indicated file. If it is indeed zero in length, then use a backup
+ copy of the configuration file.
+
+``Traffic Server could not open logfile <filename>``
+ Check permissions for the indicated file and the logging directory.
+
+``Traffic Server failed to parse line <line number> of the logging config file <filename>``
+ Check your custom log configuration file; there could be syntax
+ errors. Refer to :ref:`custom-logging-fields` for correct custom log format fields.
+
+``vip_config binary is not setuid root, manager will be unable to enable virtual ip addresses``
+ The :program:`traffic_manager` process is not able to set virtual IP
+ addresses. You must ``setuid root`` for the ``vip_config`` file in
+ the Traffic Server ``bin`` directory.
+
+.. _body-factory:
+
+HTML Messages Sent to Clients
+=============================
+
+Traffic Server returns detailed error messages to client browsers when there are
+problems with the HTTP transactions requested by the browser. These Traffic
+Server response messages correspond to standard HTTP response codes, but provide
+more information. A list of the more frequently encountered HTTP response codes
+is provided in :ref:`standard-http-response-messages`.
+
+The error messages can be customized. The actual response is generated from a template. These
+templates are stored in files which means the errors responses can be customized by modifying these
+files. The default directory for the template files is ``PREFIX/body_factory/default``
+but this can be changed by the configuration variable
+:ts:cv:`proxy.config.body_factory.template_sets_dir`. All files in this directory are added to a
+lookup table which is consulted when the error message is generated. The name used for lookup is by
+default that listed in the :ref:`following table <body-factory-error-table>`. It can be overridden by
+:ts:cv:`proxy.config.body_factory.template_base` which, if set, is a string that is prepended to the
+search name along with an underscore. For example, if the default lookup name is
+``cache#read_error`` then by default the response will be generated from the template in the file
+named ``cache#read_error``. If the template base name were set to "apache" then the lookup would
+look for a file named ``apache_cache#read_error`` in the template table. This can be used to switch
+out error message sets or, because this variable is overridable, to select an error message set
+based on data in the transaction.
+
+The text for an error message is processed as if it were a :ref:`custom logging format <custom-logging-fields>` which
+enables customization by values present in the transaction for which the error occurred.
+
+The following table lists the hard-coded Traffic Server HTTP messages,
+with corresponding HTTP response codes and customizable files.
+
+.. _body-factory-error-table:
+
+``Access Denied``
+ ``403``
+ You are not allowed to access the document at location ``URL``.
+ ``access#denied``
+
+``Cache Read Error``
+ ``500``
+ Error reading from cache; please retry request.
+ ``cache#read_error``
+
+``Connection Timed Out``
+ ``504``
+ Too much time has elapsed since the server has sent data.
+ ``timeout#inactivity``
+
+``Content Length Required``
+ ``400``
+ Could not process this request because ``Content-Length`` was not specified.
+ ``request#no_content_length``
+
+``Cycle Detected``
+ ``400``
+ Your request is prohibited because it would cause an HTTP proxy cycle.
+ ``request#cycle_detected``
+
+``Forbidden``
+ ``403``
+ ``<port number>`` is not an allowed port for SSL connections (you have made a request for a secure SSL connection to a forbidden port number).
+ ``access#ssl_forbidden``
+
+``Host Header Required``
+ ``400``
+ An attempt was made to transparently proxy your request, but this
+ attempt failed because your browser did not send an HTTP ``Host``
+ header. Manually configure your browser to use
+ ``http://<proxy name>:<proxy port>`` as the HTTP
+ proxy. Alternatively, end users can upgrade to a browser that
+ supports the HTTP ``Host`` header field.
+ ``interception#no_host``
+
+``Host Header Required``
+ ``400``
+ Because your browser did not send a ``Host`` HTTP header field, the
+ virtual host being requested could not be determined. To access the
+ website correctly, you must upgrade to a browser that supports the
+ HTTP ``Host`` header field.
+ ``request#no_host``
+
+``HTTP Version Not Supported``
+ ``505``
+ The origin server ``<server name>`` is using an unsupported version
+ of the HTTP protocol.
+ ``response#bad_version``
+
+``Invalid Content Length``
+ ``400``
+ Could not process this request because the specified ``Content-Length``
+ was invalid (less than 0)..
+ ``request#invalid_content_length``
+
+``Invalid HTTP Request``
+ ``400``
+ Could not process this ``<client request>`` HTTP method request for ``URL``.
+ ``request#syntax_error``
+
+``Invalid HTTP Response``
+ ``502``
+ The host ``<server name>`` did not return the document ``URL`` correctly.
+ ``response#bad_response``
+
+``Malformed Server Response``
+ ``502``
+ The host ``<server name>`` did not return the document ``URL`` correctly.
+ ``response#bad_response``
+
+``Malformed Server Response Status``
+ ``502``
+ The host ``<server name>`` did not return the document ``URL`` correctly.
+ ``response#bad_response``
+
+``Maximum Transaction Time exceeded``
+ ``504``
+ Too much time has elapsed while transmitting document ``URL``.
+ ``timeout#activity``
+
+``No Response Header From Server``
+ ``502``
+ The host ``<server name>`` did not return the document ``URL`` correctly.
+ ``response#bad_response``
+
+``Not Cached``
+ ``504``
+ This document was not available in the cache, and you (the client)
+ only accept cached copies.
+ ``cache#not_in_cache``
+
+``Not Found on Accelerator``
+ ``404``
+ The request for ``URL`` on host ``<server name>`` was not found.
+ Check the location and try again.
+ ``urlrouting#no_mapping``
+
+``NULL``
+ ``502``
+ The host ``<hostname>`` did not return the document ``URL`` correctly.
+ ``response#bad_response``
+
+``Proxy Authentication Required``
+ ``407``
+ Please log in with username and password.
+ ``access#proxy_auth_required``
+
+``Server Hangup``
+ ``502``
+ The server ``<hostname>`` closed the connection before the transaction was completed.
+ ``connect#hangup``
+
+``Temporarily Moved``
+ ``302``
+ The document you requested, ``URL``, has moved to a new location. The new location is ``<new URL>``.
+ ``redirect#moved_temporarily``
+
+``Transcoding Not Available``
+ ``406``
+ Unable to provide the document ``URL`` in the format requested by your browser.
+ ``transcoding#unsupported``
+
+``Tunnel Connection Failed``
+ ``502``
+ Could not connect to the server ``<hostname>``.
+ ``connect#failed_connect``
+
+``Unknown Error``
+ ``502``
+ The host ``<hostname>`` did not return the document ``URL`` correctly.
+ ``response#bad_response``
+
+``Unknown Host``
+ ``500``
+ Unable to locate the server named ``<hostname>``; the server does
+ not have a DNS entry. Perhaps there is a misspelling in the server
+ name or the server no longer exists; double-check the name and try
+ again.
+ ``connect#dns_failed``
+
+``Unsupported URL Scheme``
+ ``400``
+ Cannot perform your request for the document ``URL`` because the
+ protocol scheme is unknown.
+ ``request#scheme_unsupported``
+
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/monitoring/index.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/monitoring/index.en.rst b/doc/admin-guide/monitoring/index.en.rst
new file mode 100644
index 0000000..adf3212
--- /dev/null
+++ b/doc/admin-guide/monitoring/index.en.rst
@@ -0,0 +1,33 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. include:: ../../common.defs
+
+.. _admin-monitoring:
+
+Event and Error Monitoring
+**************************
+
+.. toctree::
+ :maxdepth: 3
+
+ logging/index.en
+ alarms.en
+ statistics/index.en
+ monitoring/index.en
+ error-messages.en
+
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/monitoring/logging/index.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/monitoring/logging/index.en.rst b/doc/admin-guide/monitoring/logging/index.en.rst
new file mode 100644
index 0000000..fe581b0
--- /dev/null
+++ b/doc/admin-guide/monitoring/logging/index.en.rst
@@ -0,0 +1,40 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. include:: ../../../common.defs
+
+.. _admin-monitoring-logging:
+
+Logging
+*******
+
+|TS| generates log files that contain information about every
+request it receives and every error it detects. This chapter will examine the
+various log features, the configuration formats and also examine the various
+pre-defined log formats that are available.
+
+.. toctree::
+ :maxdepth: 2
+
+ understanding.en
+ managing-logs.en
+ log-formats.en
+ summary-logs.en
+ log-collation.en
+ pipes.en
+ log-builder.en
+
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/monitoring/logging/log-builder.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/monitoring/logging/log-builder.en.rst b/doc/admin-guide/monitoring/logging/log-builder.en.rst
new file mode 100644
index 0000000..1b80757
--- /dev/null
+++ b/doc/admin-guide/monitoring/logging/log-builder.en.rst
@@ -0,0 +1,26 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. include:: ../../../common.defs
+
+Event Log Builder
+*****************
+
+If you need any assistance building your event log, you can try out our
+`online log builder <http://trafficserver.apache.org/logbuilder/>`_. This is a
+work in progress, so any comments, critique or suggestions are most welcome.
+
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/monitoring/logging/log-collation.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/monitoring/logging/log-collation.en.rst b/doc/admin-guide/monitoring/logging/log-collation.en.rst
new file mode 100644
index 0000000..b8c5cbe
--- /dev/null
+++ b/doc/admin-guide/monitoring/logging/log-collation.en.rst
@@ -0,0 +1,199 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. include:: ../../../common.defs
+
+Log Collation
+*************
+
+You can use the |TS| log file collation feature to collect all logged
+information in one place. Log collation enables you to analyze a set of |TS|
+clustered nodes as a whole (rather than as individual nodes) and to use a large
+disk that might only be located on one of the nodes in the cluster.
+
+|TS| collates log files by using one or more nodes as log collation servers and
+all remaining nodes as log collation clients. When a |TS| node generates a
+buffer of event log entries, it first determines if it is the collation server
+or a collation client. The collation server node writes all log buffers to its
+local disk, just as it would if log collation was not enabled. Log collation
+servers can be standalone or they can be part of a node running |TS|.
+
+The collation client nodes prepare their log buffers for transfer across the
+network and send the buffers to the log collation server. When the log
+collation server receives a log buffer from a client, it writes it to its own
+log file as if it was generated locally. For a visual representation of this,
+see the figure below.
+
+.. figure:: /static/images/admin/logcolat.jpg
+ :align: center
+ :alt: Log collation
+
+ Log collation
+
+If log clients cannot contact their log collation server, then they write their
+log buffers to their local disks, into *orphan* log files. Orphaned log files
+require manual collation.
+
+.. note::
+
+ Log collation can have an impact on network performance. Because all nodes
+ are forwarding their log data buffers to the single collation server, a
+ bottleneck can occur. In addition, collated log files contain timestamp
+ information for each entry, but entries in the files do not appear in
+ strict chronological order. You may want to sort collated log files before
+ doing analysis.
+
+.. _admin-configuring-traffic-server-to-be-a-collation-server:
+
+Server Configuration
+====================
+
+To configure a |TS| node to be a collation server, perform the following
+configuration adjustments in :file:`records.config`:
+
+#. Set :ts:cv:`proxy.local.log.collation_mode` to ``1`` to indicate this node
+ will be a server. ::
+
+ CONFIG proxy.local.log.collation_mode INT 1
+
+#. Configure the port on which the server will listen to incoming collation
+ transfers from clients, using :ts:cv:`proxy.config.log.collation_port`. If
+ omitted, this defaults to port ``8085``. ::
+
+ CONFIG proxy.config.log.collation_port INT 8085
+
+#. Configure the shared secret used by collation clients to authenticate their
+ sessions, using :ts:cv:`proxy.config.log.collation_secret`. ::
+
+ CONFIG proxy.config.log.collation_secret STRING "seekrit"
+
+#. Run the command :option:`traffic_line -x` to apply the configuration
+ changes.
+
+.. note::
+
+ If you modify the ``collation_port`` or ``secret`` after connections
+ between the collation server and collation clients have been established,
+ then you must restart Traffic Server on all nodes.
+
+.. _admin-using-a-standalone-collator:
+
+Standalone Collator
+-------------------
+
+If you do not want the log collation server to be a |TS| node,
+then you can install and configure a standalone collator (*SAC*) that will
+dedicate more of its power to collecting, processing, and writing log
+files.
+
+To install and configure a standalone collator:
+
+#. Configure your |TS| nodes as log collation clients based on the instructions
+ in :ref:`admin-monitoring-logging-collation-client`.
+
+#. Copy the :program:`traffic_sac` binary from the |TS| ``bin`` directory, and
+ place in a suitable location on the system that will act as the standalone
+ collator.
+
+#. Copy the ``libtsutil.so`` libraries from the |TS| ``lib`` directory to the
+ machine serving as the standalone collator.
+
+#. Create a directory called ``config`` in the directory that contains
+ the :program:`traffic_sac` binary.
+
+#. Create a directory called ``internal`` in the ``config`` directory
+ you created above. This directory is used internally by the standalone
+ collator to store lock files.
+
+#. Copy the :file:`records.config` file from a |TS| node configured to be a log
+ collation client to the ``config`` directory you created on the standalone
+ collator.
+
+ The :file:`records.config` file contains the log collation secret and the
+ port you specified when configuring |TS| nodes to be collation clients. The
+ collation port and secret must be the same for all collation clients and
+ servers.
+
+#. Edit :ts:cv:`proxy.config.log.logfile_dir` in :file:`records.config` to
+ specify a location on your standalone collator where the collected log files
+ should be stored. ::
+
+ CONFIG proxy.config.log.logfile_dir STRING "/var/log/trafficserver/"
+
+#. Enter the following command to start the standalone collator process::
+
+ traffic_sac -c config
+
+You will likely want to configure this program to run at server startup, as
+well as configure a service monitor in the event the process terminates
+abnormally. Please consult your operating system's documentation for how to
+achieve this.
+
+.. _admin-monitoring-logging-collation-client:
+
+Client Configuration
+====================
+
+To configure a |TS| node to be a collation client, follow the steps below. If
+you modify the ``collation_port`` or ``secret`` after connections between the
+collation clients and the collation server have been established, then you must
+restart |TS|.
+
+#. In the :file:`records.config` file, edit the following variables:
+
+ - :ts:cv:`proxy.local.log.collation_mode`: ``2`` to configure this node as
+ log collation client and sen standard formatted log entries to the
+ collation server. For XML-based formatted log entries, see
+ :file:`logs_xml.config` file.
+ - :ts:cv:`proxy.config.log.collation_host`
+ - :ts:cv:`proxy.config.log.collation_port`
+ - :ts:cv:`proxy.config.log.collation_secret`
+ - :ts:cv:`proxy.config.log.collation_host_tagged`
+ - :ts:cv:`proxy.config.log.max_space_mb_for_orphan_logs`
+
+#. Run the command :option:`traffic_line -x` to apply the configuration
+ changes.
+
+Collating Custom Logs
+=====================
+
+If you use custom event log files, then you must edit :file:`logs_xml.config`,
+in addition to configuring a collation server and collation clients.
+
+To collate custom event log files:
+
+#. On each collation client, edit :file:`logs_xml.config` and add the
+ :ref:`CollationHosts <logs-xml-logobject-collationhost>` attribute to the
+ :ref:`LogObject` specification:
+
+ .. code-block:: xml
+
+ <LogObject>
+ <Format = "squid"/>
+ <Filename = "squid"/>
+ <CollationHosts="ipaddress:port"/>
+ </LogObject>
+
+ Where ``ipaddress`` is the hostname or IP address of the collation
+ server to which all log entries (for this object) are forwarded, and
+ ``port`` is the port number for communication between the collation
+ server and collation clients.
+
+#. Run the command :option:`traffic_line -L` to restart Traffic Server on the
+ local node or :option:`traffic_line -M` to restart Traffic Server on all
+ the nodes in a cluster.
+
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/ce162a6d/doc/admin-guide/monitoring/logging/log-formats.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/monitoring/logging/log-formats.en.rst b/doc/admin-guide/monitoring/logging/log-formats.en.rst
new file mode 100644
index 0000000..643b85a
--- /dev/null
+++ b/doc/admin-guide/monitoring/logging/log-formats.en.rst
@@ -0,0 +1,1031 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. include:: ../../../common.defs
+
+.. _admin-monitoring-logging-formats:
+
+Log Formats
+***********
+
+This document provides a reference for all the different logging formats |TS|
+supports. Rather than just reading about those formats, you may also want to
+try our `online event log builder <http://trafficserver.apache.org/logbuilder/>`_
+for an interactive way of building and understanding log formats.
+
+Binary or ASCII?
+================
+
+You can configure |TS| to create event log files in either of the following:
+
+ASCII
+ These files are human-readable and can be processed using standard,
+ off-the-shelf log analysis tools. However, |TS| must perform additional
+ processing to create the files in ASCII, which mildly impacts system
+ overhead. ASCII files also tend to be larger than the equivalent binary
+ files. By default, ASCII log files have a ``.log`` filename extension.
+
+Binary
+ These files generate lower system overhead and generally occupy less space
+ on the disk than ASCII files (depending on the type of information being
+ logged). However, you must use a converter application before you can read
+ or analyze binary files via standard tools. By default, binary log files use
+ a ``.blog`` filename extension.
+
+While binary log files typically require less disk space, there are exceptions.
+
+For example, the value ``0`` (zero) requires only one byte to store in ASCII,
+but requires four bytes when stored as a binary integer. Conversely, if you
+define a custom format that logs IP addresses, then a binary log file would
+only require four bytes of storage per 32-bit address. However, the same IP
+address stored in dot notation would require around 15 characters (bytes) in an
+ASCII log file.
+
+It is wise to consider the type of data that will be logged before you select
+ASCII or binary for your log files, if your decision is being driven by storage
+space concerns. For example, you might try logging for one day using ASCII and
+then another day using binary. If the number of requests is roughly the same
+for both days, then you can calculate a rough metric that compares the two
+formats.
+
+For standard log formats, select Binary or ASCII (refer to `Using Standard
+Formats`_). For the custom log format, specify ASCII or Binary mode in the
+:ref:`LogObject` (refer to :ref:`using-custom-log-formats`). In addition to the
+ASCII and binary options, you can also write custom log entries to a UNIX-named
+pipe (a buffer in memory) with the `ASCII_PIPE File Mode`_ setting.
+
+Defining Log Objects
+====================
+
+To perform any logging at all on your |TS| nodes, you must have at least one
+:ref:`LogObject` defined in :file:`logs_xml.config`. These definitions configure
+what logs will be created, the format they will use (covered in the sections
+:ref:`admin-monitoring-logging-format-standard` and
+:ref:`admin-monitoring-logging-format-custom`), any filters which may be
+applied to events before logging them, and when or how the logs will be rolled.
+
+Log Filters
+===========
+
+:ref:`LogFilter` objects, configured in :file:`logs_xml.config` allow you to
+create filters, which may be applied to :ref:`LogObject` definitions, limiting
+the types of entries which will be included in the log output. This may be
+useful if your |TS| nodes receive many events which you have no need to log or
+analyze.
+
+.. _admin-monitoring-logging-format-standard:
+
+Standard Formats
+================
+
+The standard log formats include Squid, Netscape Common, Netscape extended, and
+Netscape Extended-2. The standard log file formats can be analyzed with a wide
+variety of off-the-shelf log-analysis packages. You should use one of the
+standard event log formats unless you need information that these formats do
+not provide.
+
+Set standard log file format options by following the steps below:
+
+#. In the :file:`records.config` file, edit the following variables
+
+#. Edit the following variables to use the Squid format:
+
+ - :ts:cv:`proxy.config.log.squid_log_enabled`
+ - :ts:cv:`proxy.config.log.squid_log_is_ascii`
+ - :ts:cv:`proxy.config.log.squid_log_name`
+ - :ts:cv:`proxy.config.log.squid_log_header`
+
+#. To use the Netscape Common format, edit the following variables:
+
+ - :ts:cv:`proxy.config.log.common_log_enabled`
+ - :ts:cv:`proxy.config.log.common_log_is_ascii`
+ - :ts:cv:`proxy.config.log.common_log_name`
+ - :ts:cv:`proxy.config.log.common_log_header`
+
+#. To use the Netscape Extended format, edit the following variables:
+
+ - :ts:cv:`proxy.config.log.extended_log_enabled`
+ - :ts:cv:`proxy.config.log.extended_log_is_ascii`
+ - :ts:cv:`proxy.config.log.extended_log_name`
+ - :ts:cv:`proxy.config.log.extended_log_header`
+
+#. To use the Netscape Extended-2 format, edit the following variables:
+
+ - :ts:cv:`proxy.config.log.extended2_log_enabled`
+ - :ts:cv:`proxy.config.log.extended2_log_is_ascii`
+ - :ts:cv:`proxy.config.log.extended2_log_name`
+ - :ts:cv:`proxy.config.log.extended2_log_header`
+
+#. Run the command :option:`traffic_line -x` to apply the configuration
+ changes.
+
+You may enable as many of the formats as you wish, keeping in mind the
+additional overhead, both in processing time to create the individual formats
+and storage space to keep them.
+
+Squid
+-----
+
+The following figure shows a sample log entry in a Squid log file.
+
+.. figure:: /static/images/admin/squid_format.jpg
+ :align: center
+ :alt: Sample log entry in squid.log
+
+ Sample log entry in squid.log
+
+====== ========= ==============================================================
+Field Symbol Description
+====== ========= ==============================================================
+1 cqtq The client request timestamp in Squid format. The time of the
+ client request in seconds since January 1, 1970 UTC (with
+ millisecond resolution).
+2 ttms The time |TS| spent processing the client request. The number
+ of milliseconds between the time the client established the
+ connection with |TS| and the time |TS| sent the last byte of
+ the response back to the client.
+3 chi The IP address of the client’s host machine.
+4 crc/pssc The cache result code; how the cache responded to the request:
+ ``HIT``, ``MISS``, and so on. Cache result codes are described
+ in :ref:`<squid-netscape-result-codes>`. The proxy response
+ status code (HTTP response status code from |TS| to client).
+5 psql The length of the |TS| response to the client in bytes,
+ including headers and content.
+6 cqhm The client request method: ``GET``, ``POST``, and so on.
+7 cauc The client request canonical URL; blanks and other characters
+ that might not be parsed by log analysis tools are replaced by
+ escape sequences. The escape sequence is a percentage sign
+ followed by the ASCII code number of the replaced character in
+ hex.
+8 caun The username of the authenticated client. A hyphen (``-``)
+ means that no authentication was required.
+9 phr/pqsn The proxy hierarchy route. The route |TS| used to retrieve the
+ object.
+10 psct The proxy response content type. The object content type taken
+ from the |TS| response header.
+====== ========= ==============================================================
+
+Netscape Common
+---------------
+
+The following figure shows a sample log entry in a Netscape Common log file.
+
+.. figure:: /static/images/admin/netscape_common_format.jpg
+ :align: center
+ :alt: Sample log entry in common.log
+
+ Sample log entry in common.log
+
+====== ========= ==============================================================
+Field Symbol Description
+====== ========= ==============================================================
+1 chi The IP address of the client's host machine.
+2 -- This hyphen (``-``) is always present in Netscape log entries.
+3 caun The authenticated client username. A hyphen (``-``) means no
+ authentication was required.
+4 cqtd The date and time of the client request, enclosed in brackets.
+5 cqtx The request line, enclosed in quotes.
+6 pssc The proxy response status code (HTTP reply code).
+7 pscl The length of the |TS| response to the client in bytes.
+====== ========= ==============================================================
+
+Netscape Extended
+-----------------
+
+The following figure shows a sample log entry in a Netscape Extended log file.
+
+.. figure:: /static/images/admin/netscape_extended_format.jpg
+ :align: center
+ :alt: Sample log entry in extended.log
+
+ Sample log entry in extended.log
+
+In addition to field 1-7 from the Netscape Common log format above, the
+Extended format also adds the following fields:
+
+====== ========= ==============================================================
+Field Symbol Description
+====== ========= ==============================================================
+8 sssc The origin server response status code.
+9 sshl The server response transfer length; the body length in the
+ origin server response to |TS|, in bytes.
+10 cqbl The client request transfer length; the body length in the
+ client request to |TS|, in bytes.
+11 pqbl The proxy request transfer length; the body length in the |TS|
+ request to the origin server.
+12 cqhl The client request header length; the header length in the
+ client request to |TS|.
+13 pshl The proxy response header length; the header length in the
+ |TS| response to the client.
+14 pqhl The proxy request header length; the header length in |TS|
+ request to the origin server.
+15 sshl The server response header length; the header length in the
+ origin server response to |TS|.
+16 tts The time |TS| spent processing the client request; the number
+ of seconds between the time that the client established the
+ connection with |TS| and the time that |TS| sent the last byte
+ of the response back to the client.
+====== ========= ==============================================================
+
+Netscape Extended2
+------------------
+
+The following figure shows a sample log entry in a Netscape Extended2 log file.
+
+.. figure:: /static/images/admin/netscape_extended2_format.jpg
+ :align: center
+ :alt: Sample log entry in extended2.log
+
+ Sample log entry in extended2.log
+
+In addition to field 1-16 from the log formats above, the Extended2 format also adds
+the following fields:
+
+====== ======== ===============================================================
+Field Symbol Description
+====== ======== ===============================================================
+17 phr The proxy hierarchy route; the route |TS| used to retrieve
+ the object.
+18 cfsc The client finish status code: ``FIN`` if the client request
+ completed successfully or ``INTR`` if the client request was
+ interrupted.
+19 pfsc The proxy finish status code: ``FIN`` if the |TS| request to
+ the origin server completed successfully or ``INTR`` if the
+ request was interrupted.
+20 crc The cache result code; how the |TS| cache responded to the
+ request: ``HIT``, ``MISS``, and so on. Cache result codes are
+ listed in :ref:`<admin-monitoring-logging-cache-result-codes>`.
+====== ======== ===============================================================
+
+.. _admin-monitoring-logging-format-custom:
+
+Custom Formats
+==============
+
+Defining a Format
+-----------------
+
+Custom logging formats in |TS| are defined by editing :file:`logs_xml.config`
+and adding new :ref:`LogFormat` entries for each format you wish to define. The
+syntax is fairly simple: every :ref:`LogFormat` element should contain at least
+two child elements (additional elements are used for features such as log
+summarization and are covered elsewhere):
+
+- A ``<Name>`` which contains an arbitrary string (using only the allowed
+ characters: ``[a-z0-9]``) naming your custom format.
+
+- A ``<Format>`` which defines the fields that will populate each entry in the
+ custom logs, as well as the order in which they appear.
+
+A very simple example format, which contains only the timestamp of when the
+event began and the canonical URL of the request, and named *myformat* would
+be written as follows:
+
+.. code-block:: xml
+
+ <LogFormat>
+ <Name = "myformat"/>
+ <Format = "%<cqtq> %<cauc>"/>
+ </LogFormat>
+
+You may include as many custom field codes as you wish. The full list of codes
+available can be found in :ref:`custom-logging-fields`. You may also include
+any literal characters in your format. For example, if we wished to separate
+the timestamp and canonical URL in our customer format above with a slash
+instead of a space, or even a slash surrounded by spaces, we could do so by
+just adding the desired characters to the format string::
+
+ %<cqtq> / %<cauc>
+
+You may define as many custom formats as you wish. To apply changes to custom
+formats, you will need to run the command :option:`traffic_line -x` after
+saving your changes to :file:`logs_xml.config`.
+
+.. _custom-logging-fields:
+
+Custom Logging Fields
+---------------------
+
+The following list describes Traffic Server custom logging fields.
+
+.. _cqh:
+
+``{HTTP header field name}cqh``
+ Logs the information in the requested field of the client request
+ HTTP header. For example, ``%<{Accept-Language}cqh>`` logs the
+ ``Accept-Language:`` field in client request headers.
+
+ .. note::
+ ecqh is the escaped version of this map
+
+.. _pqh:
+
+``{HTTP header field name}pqh``
+ Logs the information in the requested field of the proxy request
+ HTTP header. For example, ``%<{Authorization}pqh>`` logs
+ the ``Authorization:`` field in proxy request headers.
+
+ .. note::
+ epqh is the escaped version of this map
+
+.. _psh:
+
+``{HTTP header field name}psh``
+ Logs the information in the requested field of the proxy response
+ HTTP header. For example, ``%<{Retry-After}psh>`` logs the
+ ``Retry-After:`` field in proxy response headers.
+
+ .. note::
+ epsh is the escaped version of this map
+
+.. _ssh:
+
+``{HTTP header field name}ssh``
+ Logs the information in the requested field of the server response
+ HTTP header. For example, ``%<{Age}ssh>`` logs the ``Age:`` field in
+ server response headers.
+
+ .. note::
+ essh is the escaped version of this map
+
+.. _cssh:
+
+``{HTTP header field name}cssh``
+ Logs the information in the requested field of the cached server response
+ HTTP header. For example, ``%<{Age}cssh>`` logs the ``Age:`` field in
+ the cached server response headers.
+
+ .. note::
+ ecssh is the escaped version of this map
+
+.. _caun:
+
+``caun``
+ The client authenticated username; result of the RFC931/ident lookup
+ of the client username.
+
+.. _cfsc:
+
+``cfsc``
+ The client finish status code; specifies whether the client request
+ to Traffic Server was successfully completed (``FIN``) or
+ interrupted (``INTR``).
+
+.. _chi:
+
+``chi``
+ The IP address of the client's host machine.
+
+.. _chih:
+
+``chih``
+ The IP address of the client's host machine in hexadecimal.
+
+.. _chp:
+
+``chp``
+ The port number of the client's host machine.
+
+.. _cps:
+
+``cps``
+ Client Protocol Stack, the output would be the conjunction of
+ protocol names in the stack spliced with '+', such as "TLS+SPDY".
+
+.. _cqbl:
+
+``cqbl``
+ The client request transfer length; the body length in the client
+ request to Traffic Server (in bytes).
+
+.. _cqhl:
+
+``cqhl``
+ The client request header length; the header length in the client
+ request to Traffic Server.
+
+.. _cqhm:
+
+``cqhm``
+ The HTTP method in the client request to Traffic Server: ``GET``,
+ ``POST``, and so on (subset of ``cqtx``).
+
+.. _cqhv:
+
+``cqhv``
+ The client request HTTP version.
+
+.. _cqtd:
+
+``cqtd``
+ The client request timestamp. Specifies the date of the client
+ request in the format yyyy-mm-dd, where yyyy is the 4-digit year, mm
+ is the 2-digit month, and dd is the 2-digit day.
+
+.. _cqtn:
+
+``cqtn``
+ The client request timestamp; date and time of the client's request
+ (in the Netscape timestamp format).
+
+.. _cqtq:
+
+``cqtq``
+ The client request timestamp, with millisecond resolution.
+
+.. _cqts:
+
+``cqts``
+ The client-request timestamp in Squid format; the time of the client
+ request since January 1, 1970 UTC. Time is expressed in seconds,
+ with millisecond resolution.
+
+.. _cqtt:
+
+``cqtt``
+ The client request timestamp. The time of the client request in the
+ format hh:mm:ss, where hh is the two-digit hour in 24-hour format,
+ mm is the two-digit minutes value, and ss is the 2-digit seconds
+ value (for example, 16:01:19).
+
+.. _cqtx:
+
+``cqtx``
+ The full HTTP client request text, minus headers; for example, ::
+
+ GET http://www.company.com HTTP/1.0
+
+ In reverse proxy mode, Traffic Server logs the rewritten/mapped URL
+ (according to the rules in :file:`remap.config`), _not_ the
+ pristine/unmapped URL.
+
+.. _cqu:
+
+``cqu``
+ The universal resource identifier (URI) of the request from client
+ to Traffic Server (subset of ``cqtx`` ).
+
+ In reverse proxy mode, Traffic Server logs the rewritten/mapped URL
+ (according to the rules in :file:`remap.config`), _not_ the
+ pristine/unmapped URL.
+
+.. _cquc:
+
+``cquc``
+ The client request canonical URL. This differs from ``cqu`` in that
+ blanks (and other characters that might not be parsed by log
+ analysis tools) are replaced by escape sequences. The escape
+ sequence is a percentage sign followed by the ASCII code number in
+ hex.
+
+ See `cquuc`_.
+
+.. _cqup:
+
+``cqup``
+ The client request URL path; specifies the argument portion of the
+ URL (everything after the host). For example, if the URL is
+ ``http://www.company.com/images/x.gif``, then this field displays
+ ``/images/x.gif``
+
+ See `cquup`_.
+
+.. _cqus:
+
+``cqus``
+ The client request URL scheme.
+
+.. _cquuc:
+
+``cquuc``
+ The client request unmapped URL canonical. This field records a URL
+ before it is remapped (reverse proxy mode).
+
+.. _cquup:
+
+``cquup``
+ The client request unmapped URL path. This field records a URL path
+ before it is remapped (reverse proxy mode).
+
+.. _cquuh:
+
+``cquuh``
+ The client request unmapped URL host. This field records a URL's
+ host before it is remapped (reverse proxy mode).
+
+.. _crat:
+
+``crat``
+ The Retry-After time in seconds, if specified by the origin server.
+
+.. _crc:
+
+``crc``
+ The cache result code; specifies how the cache responded to the
+ request (``HIT``, ``MISS``, and so on).
+
+.. _csscl:
+
+``csscl``
+ The cached response length (in bytes) from origin server to Traffic
+ Server.
+
+.. _csshl:
+
+``csshl``
+ The cached header length in the origin server response to Traffic
+ Server (in bytes).
+
+.. _csshv:
+
+``csshv``
+ The cached server response HTTP version (1.0, 1.1, etc.).
+
+.. _csssc:
+
+``csssc``
+ The cached HTTP response status code from origin server to Traffic
+ Server.
+
+.. _cwr:
+
+``cwr``
+ The cache write result (``-``, ``WL_MISS``, ``INTR```, ``ERR`` or ``FIN``)
+
+.. _cwtr:
+
+``cwtr``
+ The cache write transform result
+
+.. _fsiz:
+
+``fsiz``
+ The size of the file (*n* bytes) as seen by the origin server.
+
+.. _pfsc:
+
+``pfsc``
+ The proxy finish status code; specifies whether the Traffic Server
+ request to the origin server was successfully completed (``FIN``),
+ interrupted (``INTR``) or timed out (``TIMEOUT``).
+
+.. _phn:
+
+``phn``
+ The hostname of the Traffic Server that generated the log entry in
+ collated log files.
+
+.. _phi:
+
+``phi``
+ The IP of the Traffic Server that generated the log entry in
+ collated log files.
+
+.. _phr:
+
+``phr``
+ The proxy hierarchy route; the route Traffic Server used to retrieve
+ the object.
+
+.. _php:
+
+``php``
+ The TCP port number that Traffic Server served this request from.
+
+.. _piid:
+
+``piid``
+ The plugin ID for the transaction. This is set for plugin driven
+ transactions via :c:func:`TSHttpConnectWithPluginId`.
+
+.. _pitag:
+
+``pitag``
+ The plugin tag for the transaction. This is set for plugin driven
+ transactions via :c:func:`TSHttpConnectWithPluginId`.
+
+.. _pqbl:
+
+``pqbl``
+ The proxy request transfer length; the body length in Traffic
+ Server's request to the origin server.
+
+.. _pqhl:
+
+``pqhl``
+ The proxy request header length; the header length in Traffic
+ Server's request to the origin server.
+
+.. _pqsi:
+
+``pqsi``
+ The proxy request server IP address (0 on cache hits and parent-ip
+ for requests to parent proxies).
+
+.. _pqsn:
+
+``pqsn``
+ The proxy request server name; the name of the server that fulfilled
+ the request.
+
+.. _pscl:
+
+``pscl``
+ The length of the Traffic Server response to the client (in bytes).
+
+.. _psct:
+
+``psct``
+ The content type of the document from server response header: (for
+ example, ``img/gif`` ).
+
+.. _pshl:
+
+``pshl``
+ The header length in Traffic Server's response to the client.
+
+.. _psql:
+
+``psql``
+ The proxy response transfer length in Squid format (includes header
+ and content length).
+
+.. _pssc:
+
+``pssc``
+ The HTTP response status code from Traffic Server to the client.
+
+.. _shi:
+
+``shi``
+ The IP address resolved from the DNS name lookup of the host in the
+ request. For hosts with multiple IP addresses, this field records
+ the IP address resolved from that particular DNS lookup.
+
+ This can be misleading for cached documents. For example: if the
+ first request was a cache miss and came from *IP1* for server
+ *S* and the second request for server *S* resolved to
+ *IP2* but came from the cache, then the log entry for the
+ second request will show *IP2*.
+
+.. _shn:
+
+``shn``
+ The hostname of the origin server.
+
+.. _sscl:
+
+``sscl``
+ The response length (in bytes) from origin server to Traffic Server.
+
+.. _sshl:
+
+``sshl``
+ The header length (in bytes) in the origin server response to Traffic Server.
+
+.. _sshv:
+
+``sshv``
+ The server response HTTP version (1.0, 1.1, etc.).
+
+.. _sssc:
+
+``sssc``
+ The HTTP response status code from origin server to Traffic Server.
+
+.. _stms:
+
+``stms``
+ The time spent accessing the origin (in milliseconds); the time is
+ measured from the time the connection with the origin is established
+ to the time the connection is closed.
+
+.. _stmsh:
+
+``stmsh``
+ Same as ``stms`` but in hexadecimal.
+
+.. _stmsf:
+
+``stmsf``
+ The time Traffic Server spends accessing the origin as a fractional
+ number of seconds. That is, the time is formated as a floating-point
+ number, instead of an integer as in ``stms``.
+
+ For example: if the time is 1500 milliseconds, then this field
+ displays 1.5 while the ``stms`` field displays 1500 and the ``sts``
+ field displays 1.
+
+.. _sts:
+
+``sts``
+ The time Traffic Server spends accessing the origin, in seconds.
+
+.. _ttms:
+
+``ttms``
+ The time Traffic Server spends processing the client request; the
+ number of milliseconds between the time the client establishes the
+ connection with Traffic Server and the time Traffic Server sends the
+ last byte of the response back to the client.
+
+.. _ttmsh:
+
+``ttmsh``
+ Same as ``ttms`` but in hexadecimal.
+
+.. _ttmsf:
+
+``ttmsf``
+ The time Traffic Server spends processing the client request as a
+ fractional number of seconds. Time is specified in millisecond
+ resolution; however, instead of formatting the output as an integer
+ (as with ``ttms``), the display is formatted as a floating-point
+ number representing a fractional number of seconds.
+
+ For example: if the time is 1500 milliseconds, then this field
+ displays 1.5 while the ``ttms`` field displays 1500 and the ``tts``
+ field displays 1.
+
+.. _tts:
+
+``tts``
+ The time Traffic Server spends processing the client request; the
+ number of seconds between the time at which the client establishes
+ the connection with Traffic Server and the time at which Traffic
+ Server sends the last byte of the response back to the client.
+
+.. _logging-format-cross-reference:
+
+Custom Field Cross-Reference
+----------------------------
+
+The following sections illustrate the correspondence between |TS| custom
+logging fields and standard logging fields for the Squid and Netscape formats.
+
+Squid
+~~~~~
+
+The following is a list of the Squid logging fields and the
+corresponding logging field symbols.
+
+============== =============
+Squid Field Symbols
+============== =============
+time `cqts`_
+elapsed `ttms`_
+client `chi`_
+action/code `crc`_/`pssc`_
+size `psql`_
+method `cqhm`_
+url `cquc`_
+ident `caun`_
+hierarchy/from `phr`_/`pqsn`_
+content `psct`_
+============== =============
+
+This is the equivalent XML configuration for the log above::
+
+ <LogFormat>
+ <Name = "squid"/>
+ <Format = "%<cqtq> %<ttms> %<chi> %<crc>/%<pssc> %<psql> %<cqhm> %<cquc>
+ %<caun> %<phr>/%<pqsn> %<psct>"/>
+ </LogFormat>
+
+.. _admin-log-formats-netscape-common:
+
+Netscape Common
+~~~~~~~~~~~~~~~
+
+The following is a list of the Netscape Common logging fields and the
+corresponding Traffic Server logging field symbols.
+
+=============== =============
+Netscape Common Field Symbols
+=============== =============
+host `chi`_
+usr `caun`_
+[time] [`cqtn`_]
+"req" "`cqtx`_"
+s1 `pssc`_
+c1 `pscl`_
+=============== =============
+
+This is the equivalent XML configuration for the log above::
+
+ <LogFormat>
+ <Name = "common"/>
+ <Format = "%<chi> - %<caun> [%<cqtn>] \"%<cqtx>\" %<pssc> %<pscl>"/>
+ </LogFormat>
+
+.. _admin-log-formats-netscape-extended:
+
+Netscape Extended
+~~~~~~~~~~~~~~~~~
+
+The following table lists the Netscape Extended logging fields and the
+corresponding Traffic Server logging field symbols.
+
+================= =============
+Netscape Extended Field Symbols
+================= =============
+host `chi`_
+usr `caun`_
+[time] [`cqtn`_]
+"req" "`cqtx`_"
+s1 `pssc`_
+c1 `pscl`_
+s2 `sssc`_
+c2 `sscl`_
+b1 `cqbl`_
+b2 `pqbl`_
+h1 `cqhl`_
+h2 `pshl`_
+h3 `pqhl`_
+h4 `sshl`_
+xt `tts`_
+================= =============
+
+This is the equivalent XML configuration for the log above::
+
+ <LogFormat>
+ <Name = "extended"/>
+ <Format = "%<chi> - %<caun> [%<cqtn>] \"%<cqtx>\" %<pssc> %<pscl>
+ %<sssc> %<sscl> %<cqbl> %<pqbl> %<cqhl> %<pshl> %<pqhl> %<sshl> %<tts>"/>
+ </LogFormat>
+
+.. _admin-log-formats-netscape-extended2:
+
+Netscape Extended-2
+~~~~~~~~~~~~~~~~~~~
+
+The following is a list of the Netscape Extended-2 logging fields and
+the corresponding Traffic Server logging field symbols.
+
+=================== =============
+Netscape Extended-2 Field Symbols
+=================== =============
+``host`` ``chi``
+``usr`` ``caun``
+``[time]`` ``[cqtn]``
+``"req"`` ``"cqtx"``
+``s1`` ``pssc``
+``c1`` ``pscl``
+``s2`` ``sssc``
+``c2`` ``sscl``
+``b1`` ``cqbl``
+``b2`` ``pqbl``
+``h1`` ``cqhl``
+``h2`` ``pshl``
+``h3`` ``pqhl``
+``h4`` ``sshl``
+``xt`` ``tts``
+``route`` ``phr``
+``pfs`` ``cfsc``
+``ss`` ``pfsc``
+``crc`` ``crc``
+=================== =============
+
+This is the equivalent XML configuration for the log above::
+
+ <LogFormat>
+ <Name = "extended2"/>
+ <Format = "%<chi> - %<caun> [%<cqtn>] \"%<cqtx>\" %<pssc> %<pscl>
+ %<sssc> %<sscl> %<cqbl> %<pqbl> %<cqhl> %<pshl> %<pqhl> %<sshl> %<tts> %<phr> %<cfsc> %<pfsc> %<crc>"/>
+ </LogFormat>
+
+.. _log-field-slicing:
+
+Log Field Slicing
+-----------------
+
+It is sometimes desirable to slice a log field to limit the length of a given
+log field's output.
+
+Log Field slicing can be specified as below::
+
+ %<field[start:end]>
+ %<{field}container[start:end]>
+
+Omitting the slice notation defaults to the entire log field.
+
+Slice notation only applies to a log field that is of type string and can not
+be applied to IPs or timestamp which are converted to string from integer.
+
+The below slice specifiers are allowed.
+
+``[start:end]``
+ Log field value from start through end-1
+``[start:]``
+ Log field value from start through the rest of the string
+``[:end]``
+ Log field value from the beginning through end-1
+``[:]``
+ Default - entire Log field
+
+Some examples below ::
+
+ '%<cqup>' // the whole characters of <cqup>.
+ '%<cqup>[:]' // the whole characters of <cqup>.
+ '%<cqup[0:30]>' // the first 30 characters of <cqup>.
+ '%<cqup[-10:]>' // the last 10 characters of <cqup>.
+ '%<cqup[:-5]>' // everything except the last 5 characters of <cqup>.
+
+.. _admin-monitoring-logging-cache-result-codes:
+
+Cache Result Codes
+==================
+
+The following table describes the cache result codes in Squid and
+Netscape log files.
+
+``TCP_HIT``
+ A valid copy of the requested object was in the cache and Traffic
+ Server sent the object to the client.
+
+``TCP_MISS``
+ The requested object was not in cache, so Traffic Server retrieved
+ the object from the origin server (or a parent proxy) and sent it to
+ the client.
+
+``TCP_REFRESH_HIT``
+ The object was in the cache, but it was stale. Traffic Server made an
+ ``if-modified-since`` request to the origin server and the
+ origin server sent a ``304`` not-modified response. Traffic
+ Server sent the cached object to the client.
+
+``TCP_REF_FAIL_HIT``
+ The object was in the cache but was stale. Traffic Server made an
+ ``if-modified-since`` request to the origin server but the server
+ did not respond. Traffic Server sent the cached object to the
+ client.
+
+``TCP_REFRESH_MISS``
+ The object was in the cache but was stale. Traffic Server made an
+ ``if-modified-since`` request to the origin server and the server
+ returned a new object. Traffic Server served the new object to the
+ client.
+
+``TCP_CLIENT_REFRESH``
+ The client issued a request with a ``no-cache`` header. Traffic
+ Server obtained the requested object from the origin server and sent
+ a copy to the client. Traffic Server deleted the previous copy of
+ the object from cache.
+
+``TCP_IMS_HIT``
+ The client issued an ``if-modified-since`` request and the object
+ was in cache and fresher than the IMS date, or an
+ ``if-modified-since`` request to the origin server revealed the
+ cached object was fresh. Traffic Server served the cached object to
+ the client.
+
+``TCP_IMS_MISS``
+ The client issued an
+ ``if-modified-since request`` and the object was either not in
+ cache or was stale in cache. Traffic Server sent an
+ ``if-modified-since request`` to the origin server and received the
+ new object. Traffic Server sent the updated object to the client.
+
+``TCP_SWAPFAIL``
+ The object was in the cache but could not be accessed. The client
+ did not receive the object.
+
+``ERR_CLIENT_ABORT``
+ The client disconnected before the complete object was sent.
+
+``ERR_CONNECT_FAIL``
+ Traffic Server could not reach the origin server.
+
+``ERR_DNS_FAIL``
+ The Domain Name Server (DNS) could not resolve the origin server
+ name, or no DNS could be reached.
+
+``ERR_INVALID_REQ``
+ The client HTTP request was invalid. (Traffic Server forwards
+ requests with unknown methods to the origin server.)
+
+``ERR_READ_TIMEOUT``
+ The origin server did not respond to Traffic Server's request within
+ the timeout interval.
+
+``ERR_PROXY_DENIED``
+ Client service was denied.
+
+``ERR_UNKNOWN``
+ The client connected, but subsequently disconnected without sending
+ a request.
+