You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficcontrol.apache.org by mi...@apache.org on 2018/10/10 18:26:03 UTC
[trafficcontrol] 05/39: No, seriously, what's with the arrow?
This is an automated email from the ASF dual-hosted git repository.
mitchell852 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 0f94d63c1402c7f44e8723751f0694097d33e951
Author: ocket8888 <oc...@gmail.com>
AuthorDate: Tue Sep 18 18:10:35 2018 -0600
No, seriously, what's with the arrow?
---
docs/source/overview/fwda.png | Bin 206 -> 0 bytes
docs/source/overview/traffic_monitor.rst | 40 ++++----------
docs/source/overview/traffic_router.rst | 89 +++++++++++++------------------
docs/source/overview/traffic_stats.rst | 31 +++++------
4 files changed, 60 insertions(+), 100 deletions(-)
diff --git a/docs/source/overview/fwda.png b/docs/source/overview/fwda.png
deleted file mode 100644
index ba63bee..0000000
Binary files a/docs/source/overview/fwda.png and /dev/null differ
diff --git a/docs/source/overview/traffic_monitor.rst b/docs/source/overview/traffic_monitor.rst
index eefb67c..05b4dcc 100644
--- a/docs/source/overview/traffic_monitor.rst
+++ b/docs/source/overview/traffic_monitor.rst
@@ -15,27 +15,17 @@
.. _tc-tm:
-.. index::
- Traffic Monitor - Overview
-
-.. |arrow| image:: fwda.png
-
+***************
Traffic Monitor
-===============
-Traffic Monitor is an HTTP service that monitors the caches in a CDN for a variety of metrics. These metrics are for use in determining the overall health of a given cache and the related delivery services. A given CDN can operate a number of Traffic Monitors, from a number of geographically diverse locations, to prevent false positives caused by network problems at a given site.
-
-Traffic Monitors operate independently, but use the state of other Traffic Monitors in conjunction with their own state, to provide a consistent view of CDN cache health to upstream applications such as Traffic Router. Health Protocol governs the cache and Delivery Service availability.
-
-Traffic Monitor provides a view into CDN health using several RESTful JSON endpoints, which are consumed by other Traffic Monitors and upstream components such as Traffic Router. Traffic Monitor is also responsible for serving the overall CDN configuration to Traffic Router, which ensures that the configuration of these two critical components remain synchronized as operational and health related changes propagate through the CDN.
+***************
+Traffic Monitor is an HTTP service that monitors the caches in a CDN for a variety of metrics. These metrics are for use in determining the overall health of a given cache and the related delivery services. A given CDN can operate a number of Traffic Monitors, from a number of geographically diverse locations, to prevent false positives caused by network problems at a given site. Traffic Monitors operate independently, but use the state of other Traffic Monitors in conjunction with their [...]
.. _astats:
-|arrow| Cache Monitoring
--------------------------
-Traffic Monitor polls all caches configured with a status of ``REPORTED`` or ``ADMIN_DOWN`` at an interval specified as a configuration parameter in Traffic Ops. If the cache is set to ``ADMIN_DOWN`` it is marked as unavailable but still polled for availability and statistics. If the cache is explicitly configured with a status of ``ONLINE`` or ``OFFLINE``, it is not polled by Traffic Monitor and presented to Traffic Router as configured, regardless of actual availability.
-
-Traffic Monitor makes HTTP requests at regular intervals to a special URL on each EDGE cache and consumes the JSON output. The special URL is a plugin running on the Apache Traffic Server (ATS) caches called astats, which is restricted to Traffic Monitor only. The astats plugin provides insight into application and system performance, such as:
+Cache Monitoring
+================
+Traffic Monitor polls all caches configured with a status of ``REPORTED`` or ``ADMIN_DOWN`` at an interval specified as a configuration parameter in Traffic Ops. If the cache is set to ``ADMIN_DOWN`` it is marked as unavailable but still polled for availability and statistics. If the cache is explicitly configured with a status of ``ONLINE`` or ``OFFLINE``, it is not polled by Traffic Monitor and presented to Traffic Router as configured, regardless of actual availability. Traffic Monito [...]
- Throughput (e.g. bytes in, bytes out, etc).
- Transactions (e.g. number of 2xx, 3xx, 4xx responses, etc).
@@ -44,23 +34,13 @@ Traffic Monitor makes HTTP requests at regular intervals to a special URL on eac
- Storage performance (e.g.: writes, reads, frags, directories, etc).
- System performance (e.g: load average, network interface throughput, etc).
-Many of the application-level statistics are available at the global or aggregate level, some at the Delivery Service (remap rule) level. Traffic Monitor uses the system level performance to determine the overall health of the cache by evaluating network throughput and load against values configured in Traffic Ops. Traffic Monitor also uses throughput and transaction statistics at the remap rule level to determine Delivery Service health.
-
-If astats is unavailable due to a network related issue or the system statistics have exceeded the configured thresholds, Traffic Monitor will mark the cache as unavailable. If the delivery service statistics exceed the configured thresholds, the delivery service is marked as unavailable, and Traffic Router will start sending clients to the overflow destinations for that delivery service, but the cache remains available to serve other content,
+Many of the application-level statistics are available at the global or aggregate level, some at the Delivery Service (remap rule) level. Traffic Monitor uses the system level performance to determine the overall health of the cache by evaluating network throughput and load against values configured in Traffic Ops. Traffic Monitor also uses throughput and transaction statistics at the remap rule level to determine Delivery Service health. If astats is unavailable due to a network related [...]
.. seealso:: For more information on ATS Statistics, see the `ATS documentation <https://docs.trafficserver.apache.org/en/latest/index.html>`_
.. _health-proto:
-|arrow| Health Protocol
------------------------
-Redundant Traffic Monitor servers operate independently from each other but take the state of other Traffic Monitors into account when asked for health state information. In the above overview of cache monitoring, the behavior of Traffic Monitor pertains only to how an individual instance detects and handles failures. The Health Protocol adds another dimension to the health state of the CDN by merging the states of all Traffic Monitors into one, and then taking the *optimistic* approach [...]
-
-Upon startup or configuration change in Traffic Ops, in addition to caches, Traffic Monitor begins polling its peer Traffic Monitors whose state is set to ``ONLINE``. Each ``ONLINE`` Traffic Monitor polls all of its peers at a configurable interval and saves the peer's state for later use. When polling its peers, Traffic Monitor asks for the raw health state from each respective peer, which is strictly that instance's view of the CDN's health. When any ``ONLINE`` Traffic Monitor is asked [...]
-
-In operation of the health protocol, Traffic Monitor takes all health states from all peers, including the locally known health state, and serves an optimistic outlook to the requesting client. This means that, for example, if three of the four Traffic Monitors see a given cache or Delivery Service as exceeding its thresholds and unavailable, it is still considered available. Only if all Traffic Monitors agree that the given object is unavailable is that state propagated to upstream comp [...]
-
-It is not uncommon for a cache to be marked unavailable by Traffic Monitor - in fact, it is business as usual for many CDNs. Should a widely requested video asset cause a single cache to get close to its interface capacity, the Health Protocol will "kicks in", and Traffic Monitor marks cache-03 as unavailable. New clients want to see the same asset, and now, Traffic Router will send these customers to another cache (say cache-01) in the same Cache Group. The load is now shared between ca [...]
-
-It is less common for a delivery service to be marked unavailable by Traffic Monitor, the delivery service thresholds are usually used for overflow situations at extreme peaks to protect other delivery services in the CDN from getting impacted.
+Health Protocol
+===============
+Redundant Traffic Monitor servers operate independently from each other but take the state of other Traffic Monitors into account when asked for health state information. In the above overview of cache monitoring, the behavior of Traffic Monitor pertains only to how an individual instance detects and handles failures. The Health Protocol adds another dimension to the health state of the CDN by merging the states of all Traffic Monitors into one, and then taking the *optimistic* approach [...]
diff --git a/docs/source/overview/traffic_router.rst b/docs/source/overview/traffic_router.rst
index d466e8a..4b310d7 100644
--- a/docs/source/overview/traffic_router.rst
+++ b/docs/source/overview/traffic_router.rst
@@ -15,13 +15,9 @@
.. _tc-tr:
-.. |arrow| image:: fwda.png
-
-.. index::
- Traffic Router - Overview
-
+**************
Traffic Router
-==============
+**************
Traffic Router's function is to send clients to the most optimal cache. 'Optimal' in this case is based on a number of factors:
* Distance between the cache and the client (not necessarily measured in meters, but quite often in layer 3 network hops). Less network distance between the client and cache yields better performance, and lower network load. Traffic Router helps clients connect to the best performing cache for their location at the lowest network cost.
@@ -34,69 +30,56 @@ Traffic routing options are often configured at the Delivery Service level.
.. _ds:
-|arrow| Delivery Service
-------------------------
- As discussed in the basic concepts section, the EDGE caches are configured as reverse proxies, and the Traffic Control CDN looks from the outside as a very large reverse proxy. Delivery Services are often referred to a reverse proxy remap rule. In most cases, a Delivery Service is a one to one mapping to a FQDN that is used as a hostname to deliver the content. Many options and settings regarding how to optimize the content delivery, which is configurable on a Delivery Service basis. So [...]
+Delivery Service
+================
+As discussed in the basic concepts section, the EDGE caches are configured as reverse proxies, and the Traffic Control CDN looks from the outside as a very large reverse proxy. Delivery Services are often referred to a reverse proxy remap rule. In most cases, a Delivery Service is a one to one mapping to a FQDN that is used as a hostname to deliver the content. Many options and settings regarding how to optimize the content delivery, which is configurable on a Delivery Service basis. Som [...]
- * Cache in RAM, cache on disk, or do not cache at all.
- * Use DNS or HTTP Content routing (see below).
- * Limits on transactions per second and bandwidth.
- * Protocol (HTTP or HTTPS).
- * Token based authentication settings.
- * Header rewrite rules.
+* Cache in RAM, cache on disk, or do not cache at all.
+* Use DNS or HTTP Content routing (see below).
+* Limits on transactions per second and bandwidth.
+* Protocol (HTTP or HTTPS).
+* Token based authentication settings.
+* Header rewrite rules.
- Since Traffic Control version 2.1 Delivery Services can optionally be linked to a :ref:`profile`, and have parameters associated with them. The first feature that uses Delivery Service parameters is the :ref:`multi-site-origin` configuration.
- Delivery Services are also for use in allowing multi-tenants to coexist in the Traffic Control CDN without interfering with each other, and to keep information about their content separated.
+Since Traffic Control version 2.1 Delivery Services can optionally be linked to a :ref:`profile`, and have parameters associated with them. The first feature that uses Delivery Service parameters is the :ref:`multi-site-origin` configuration. Delivery Services are also for use in allowing multi-tenants to coexist in the Traffic Control CDN without interfering with each other, and to keep information about their content separated.
.. _localization:
-|arrow| Localization
---------------------
- Traffic Router uses a JSON input file called the *coverage zone map* to determine what *Cache Group* is closest to the client. If the client IP address is not in this coverage zone map, it falls back to geographic mapping, using a GeoIP2 database to find the client's location, and the geographic coordinates from Traffic Ops for the Cache Group.
-
-Traffic Router is inserted into the HTTP retrieval process by making it DNS authoritative for the domain of the CDN delivery service. In the example of the reverse proxy, the client was given the ``http://www-origin-cache.cdn.com/foo/bar/fun.html`` URL. In a Traffic Control CDN, URLs start with a routing name, which is configurable per-Delivery Service, e.g. ``http://foo.mydeliveryservice.cdn.com/fun/example.html`` with the chosen routing name ``foo``.
-
-.. index::
- Content Routing
+Localization
+============
+Traffic Router uses a JSON input file called the *coverage zone map* to determine what *Cache Group* is closest to the client. If the client IP address is not in this coverage zone map, it falls back to geographic mapping, using a GeoIP2 database to find the client's location, and the geographic coordinates from Traffic Ops for the Cache Group. Traffic Router is inserted into the HTTP retrieval process by making it DNS authoritative for the domain of the CDN delivery service. In the exam [...]
.. _dns-cr:
-|arrow| DNS Content Routing
----------------------------
- For a DNS Delivery Service the client might receive a URL such as ``http://foo.dsname.cdn.com/fun/example.html``. When the Local Domain Name Server (LDNS) server is resolving this ``foo.dsname.cdn.com`` hostname to an IP address, it ends at Traffic Router because it is the authoritative DNS server for ``cdn.com`` and the domains below it, and subsequently responds with a list of IP addresses from the eligible caches based on the location of the LDNS server. When responding, Traffic Rout [...]
+DNS Content Routing
+===================
+For a DNS Delivery Service the client might receive a URL such as ``http://foo.dsname.cdn.com/fun/example.html``. When the Local Domain Name Server (LDNS) server is resolving this ``foo.dsname.cdn.com`` hostname to an IP address, it ends at Traffic Router because it is the authoritative DNS server for ``cdn.com`` and the domains below it, and subsequently responds with a list of IP addresses from the eligible caches based on the location of the LDNS server. When responding, Traffic Route [...]
.. _http-cr:
-|arrow| HTTP Content Routing
-----------------------------
- For an HTTP Delivery Service the client might receive a URL such as ``http://bar.dsname.cdn.com/fun/example.html``. The LDNS server resolves this ``bar.dsname.cdn.com`` to an IP address, but in this case Traffic Router returns its own IP address. The client opens a connection to port 80 on the Traffic Router's IP address, and sends:
-
- .. code-block:: http
-
- GET /fun/example.html HTTP/1.1
- Host: bar.dsname.cdn.com
-
- Traffic Router uses an HTTP 302 to redirect the client to the best cache. For example:
-
- .. code-block:: http
+HTTP Content Routing
+====================
+For an HTTP Delivery Service the client might receive a URL such as ``http://bar.dsname.cdn.com/fun/example.html``. The LDNS server resolves this ``bar.dsname.cdn.com`` to an IP address, but in this case Traffic Router returns its own IP address. The client opens a connection to port 80 on the Traffic Router's IP address, and sends:
- HTTP/1.1 302 Moved Temporarily
- Server: Apache-Coyote/1.1
- Location: http://atsec-nyc-02.dsname.cdn.com/fun/example.html
- Content-Length: 0
- Date: Tue, 13 Jan 2015 20:01:41 GMT
+.. code-block:: http
- The information Traffic Router can consider when selecting a cache in this case is much better:
+ GET /fun/example.html HTTP/1.1
+ Host: bar.dsname.cdn.com
- * The client's IP address (the other side of the socket).
- * The URL path the client is requesting, excluding query string.
- * All HTTP 1.1 headers.
+Traffic Router uses an HTTP ``302 FOUND`` response to redirect the client to the best cache. For example:
- The client follows the redirect and performs a DNS request for the IP address for ``atsec-nyc-02.dsname.cdn.com``, and normal HTTP steps follow, except the sending of the Host: header when connected to the cache is ``Host: atsec-nyc-02.dsname.cdn``, and the configuration of the cache includes the remap rule (e.g.``http://atsec-nyc-02.dsname.cdn http://origin.dsname.com``).
+.. code-block:: http
- Traffic Router sends all requests for the same path in a delivery service to the same cache in a cache group using consistent hashing, in this case all caches in a cache group are not carrying the same content, and there is a much larger combined cache in the cache group.
+ HTTP/1.1 302 Moved Temporarily
+ Server: Apache-Coyote/1.1
+ Location: http://atsec-nyc-02.dsname.cdn.com/fun/example.html
+ Content-Length: 0
+ Date: Tue, 13 Jan 2015 20:01:41 GMT
-In many cases DNS content routing is the best possible option, especially in cases where the client is receiving small objects from the CDN like images and web pages.
+The information Traffic Router can consider when selecting a cache in this case is much better:
-Traffic Router is redundant and horizontally scalable by adding more instances into the DNS hierarchy using NS records.
+* The client's IP address (the other side of the socket).
+* The URL path the client is requesting, excluding query string.
+* All HTTP 1.1 headers.
+The client follows the redirect and performs a DNS request for the IP address for ``atsec-nyc-02.dsname.cdn.com``, and normal HTTP steps follow, except the sending of the Host: header when connected to the cache is ``Host: atsec-nyc-02.dsname.cdn``, and the configuration of the cache includes the remap rule (e.g.``http://atsec-nyc-02.dsname.cdn http://origin.dsname.com``). Traffic Router sends all requests for the same path in a delivery service to the same cache in a cache group using c [...]
diff --git a/docs/source/overview/traffic_stats.rst b/docs/source/overview/traffic_stats.rst
index debf5a0..2242f35 100644
--- a/docs/source/overview/traffic_stats.rst
+++ b/docs/source/overview/traffic_stats.rst
@@ -14,11 +14,10 @@
..
.. _tc-ts:
-.. |arrow| image:: fwda.png
-
+*************
Traffic Stats
-=============
+*************
Traffic Stats is a program written in `Go <http://golang.org>`_ that is used to acquire and store statistics about CDNs controlled by Traffic Control. Traffic Stats mines metrics from Traffic Monitor's JSON APIs and stores the data in `InfluxDB <http://influxdb.com>`_. Data is typically stored in InfluxDB on a short-term basis (30 days or less). The data from InfluxDB is then used to drive graphs created by `Grafana <http://grafana.org>`_ - which are linked to from Traffic Ops - as well [...]
- Gathers stat data for Edge Caches and Delivery Services at a configurable interval (10 second default) from the Traffic Monitor API's and stores the data in InfluxDB
@@ -26,23 +25,21 @@ Traffic Stats is a program written in `Go <http://golang.org>`_ that is used to
Stat data is stored in three different databases:
- - ``cache_stats``: Stores data gathered from edge caches. The `measurements <https://influxdb.com/docs/v0.9/concepts/glossary.html#measurement>`_ stored by cache are: bandwidth, maxKbps, and ``client_connections`` (``ats.proxy.process.http.current_client_connections``). Cache Data is stored with `tags <https://influxdb.com/docs/v0.9/concepts/glossary.html#tag>`_ for hostname, Cache Group, and CDN. Data can be queried using tags.
-
- - ``deliveryservice_stats``: Stores data for Delivery Services. The measurements stored by delivery service are:
+- ``cache_stats``: Stores data gathered from edge caches. The `measurements <https://influxdb.com/docs/v0.9/concepts/glossary.html#measurement>`_ stored by cache are: bandwidth, maxKbps, and ``client_connections`` (``ats.proxy.process.http.current_client_connections``). Cache Data is stored with `tags <https://influxdb.com/docs/v0.9/concepts/glossary.html#tag>`_ for hostname, Cache Group, and CDN. Data can be queried using tags.
- - ``kbps``
- - ``status_4xx``
- - ``status_5xx``
- - ``tps_2xx``
- - ``tps_3xx``
- - ``tps_4xx``
- - ``tps_5xx``
- - ``tps_total``
+- ``deliveryservice_stats``: Stores data for Delivery Services. The measurements stored by delivery service are:
- Delivery Service statistics are stored with tags for Cache Group, CDN, and Delivery Service ``xml_id``.
+ - ``kbps``
+ - ``status_4xx``
+ - ``status_5xx``
+ - ``tps_2xx``
+ - ``tps_3xx``
+ - ``tps_4xx``
+ - ``tps_5xx``
+ - ``tps_total``
- - ``daily_stats``: Stores summary data for daily activities. The statistics that are currently summarized are Max Bandwidth and Bytes Served and they are stored by CDN.
+ Delivery Service statistics are stored with tags for Cache Group, CDN, and Delivery Service ``xml_id``.
-------------
+- ``daily_stats``: Stores summary data for daily activities. The statistics that are currently summarized are Max Bandwidth and Bytes Served and they are stored by CDN.
Traffic Stats does not influence overall CDN operation, but is required in order to display charts in Traffic Portal and the legacy Traffic Ops UI.