You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficcontrol.apache.org by ra...@apache.org on 2019/06/03 15:58:05 UTC
[trafficcontrol] branch master updated: TP/TO/TR Consistent Hash
with Query Parameters (#3552)
This is an automated email from the ASF dual-hosted git repository.
rawlin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
The following commit(s) were added to refs/heads/master by this push:
new 65eaf24 TP/TO/TR Consistent Hash with Query Parameters (#3552)
65eaf24 is described below
commit 65eaf243a127c485ef2754f43015430d1c6bb49c
Author: Dan Kirkwood <da...@apache.org>
AuthorDate: Mon Jun 3 09:57:59 2019 -0600
TP/TO/TR Consistent Hash with Query Parameters (#3552)
* add queryKeys to ds
* Added consistent-hashing-considered query parameters to snapshots
* Update for consistentHashQueryParams
* add consistentHashQueryParams example to CIAB
* Added consistentHashQueryParams to DS object in TR
* Added TR build instructions
* Added ConsistentHashQueryParams to TR Consistent hashing
* add api test for ds consistent hash query params
* consistentHashQueryParams now explicitly sorted
* no null in json
* Fixed consistent hashing not considering query param values
* changelog for consistent hash with query params
* querystrings now decoded before hashing
* DeliveryService.consistentHashQueryParams is now private....
* querystring now sorted before hashing
* iterator-based iteration > index-based iteration
* added other targets to gitignore
* name_empty constraint
* adds the ability to define consistent hash query params for HTTP delivery services
* stopped re-encoding parameters prior to hashing
* query params no longer case-sensitive
* changes per brennan feedback. using Set to removed dups and change ng-pattern to pattern
* check for constraint violation
* fix detection of empty query param
* Querykeys (#71)
* Added CHQP to TOAPI docs
* removed incorrect deprecation notices
* PBCHR now takes place before CHQP are considered
* Added CHQP implementation docs to TR admin section
* ds querykeys must not be split
* simplify ordering on query_keys
* sorting of query params done from db side
* log error message set by handler to debug
* revert order of sql columns
* fixed PMD violations
* disallow consistentHashQueryParams for non-http dses
* optimizations to DeliveryService.extractSignificantQueryParams (#75)
* Removed unnecessary decoding (#76)
* fix crconfig ds test
* omitempty queryparams in crconfig
* fixed TR tests (#77)
---
CHANGELOG.md | 1 +
.../admin/traffic_portal/usingtrafficportal.rst | 5 +-
docs/source/admin/traffic_router.rst | 93 ++++----
docs/source/api/cdns_name_snapshot.rst | 20 +-
docs/source/api/cdns_name_snapshot_new.rst | 20 +-
docs/source/api/deliveryservices.rst | 157 ++++++++------
docs/source/api/deliveryservices_id.rst | 68 +++---
docs/source/api/deliveryservices_id_safe.rst | 42 ++--
docs/source/api/deliveryservices_id_servers.rst | 3 +-
docs/source/api/deliveryservices_xmlid_servers.rst | 2 -
docs/source/api/servers_id_deliveryservices.rst | 24 +--
docs/source/api/users_id_deliveryservices.rst | 41 ++--
docs/source/glossary.rst | 12 +-
docs/source/overview/delivery_services.rst | 34 +++
.../cdn-in-a-box/traffic_ops/to-access.sh | 4 +-
.../deliveryservices/010-ciab.json | 8 +-
.../traffic_ops_integration_test/run.sh | 1 +
lib/go-tc/crconfig.go | 49 ++---
lib/go-tc/deliveryservices.go | 37 +++-
lib/go-tc/nullable_test.go | 1 -
.../20190513000000_add-allowed_query_keys.sql | 32 +++
.../testing/api/v14/deliveryservices_test.go | 11 +
traffic_ops/testing/api/v14/tc-fixtures.json | 3 +
traffic_ops/traffic_ops_golang/api/api.go | 23 +-
.../traffic_ops_golang/crconfig/deliveryservice.go | 84 +++++++-
.../crconfig/deliveryservice_test.go | 68 +++++-
.../deliveryservice/deliveryservices.go | 234 ++++++++++++++++++++-
...ervicesv13_test.go => deliveryservices_test.go} | 0
.../app/src/common/api/DeliveryServiceService.js | 6 +
.../app/src/common/modules/form/_form.scss | 32 +++
.../FormDeliveryServiceController.js | 19 ++
.../form.deliveryService.HTTP.tpl.html | 31 +++
.../app/src/traffic_portal_properties.json | 3 +-
traffic_router/.gitignore | 4 +
.../traffic_router/core/ds/DeliveryService.java | 74 ++++++-
.../traffic_router/core/router/TrafficRouter.java | 179 +++++++++++++---
.../core/ds/DeliveryServiceTest.java | 30 +++
.../core/hashing/ConsistentHasherTest.java | 23 ++
.../core/router/TrafficRouterTest.java | 2 +-
39 files changed, 1195 insertions(+), 285 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 9f567cf..3948912 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -29,6 +29,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
- Traffic Portal standalone Dockerfile
- In Traffic Portal, removes the need to specify line breaks using `__RETURN__` in delivery service edge/mid header rewrite rules, regex remap expressions, raw remap text and traffic router additional request/response headers.
- In Traffic Portal, provides the ability to clone delivery service assignments from one cache to another cache of the same type. Issue #2963.
+- Traffic Ops now allows each delivery service to have a set of query parameter keys to be retained for consistent hash generation by Traffic Router.
### Changed
- Traffic Router, added TLS certificate validation on certificates imported from Traffic Ops
diff --git a/docs/source/admin/traffic_portal/usingtrafficportal.rst b/docs/source/admin/traffic_portal/usingtrafficportal.rst
index 556e143..73e488f 100644
--- a/docs/source/admin/traffic_portal/usingtrafficportal.rst
+++ b/docs/source/admin/traffic_portal/usingtrafficportal.rst
@@ -191,14 +191,15 @@ This page contains a table displaying all :term:`Delivery Service`\ s visible to
- clone an existing :term:`Delivery Service`
- update an existing :term:`Delivery Service`
- delete an existing :term:`Delivery Service`
-- compare :term:`Delivery Service`\ s
+- compare :term:`Delivery Services`
- manage :term:`Delivery Service` SSL keys
- manage :term:`Delivery Service` URL signature keys
- manage :term:`Delivery Service` URI signing keys
-- view and assign :term:`Delivery Service`\ servers
+- view and assign :term:`Delivery Service` servers
- create, update and delete :term:`Delivery Service` regular expressions
- view and create :term:`Delivery Service` invalidate content jobs
- manage steering targets
+- test :ref:`pattern-based-consistenthash`
- view and manage static DNS records within a :term:`Delivery Service` subdomain
.. seealso:: :ref:`static-dns-qht`
diff --git a/docs/source/admin/traffic_router.rst b/docs/source/admin/traffic_router.rst
index 0879596..cb0b92d 100644
--- a/docs/source/admin/traffic_router.rst
+++ b/docs/source/admin/traffic_router.rst
@@ -169,6 +169,62 @@ For the most part, the configuration files and :term:`Parameter`\ s used by Traf
| | | of Tomcat | |
+----------------------------+-------------------------------------------+----------------------------------------------------------------------------------+----------------------------------------------------+
+
+.. _consistent-hashing:
+
+Consistent Hashing
+==================
+Traffic Router does special optimization for some requests to ensure that requests for specific content are consistently fetched from a small number (often exactly one, but dependent on :ref:`ds-initial-dispersion`) of :term:`cache servers` - thus ensuring it stays "fresh" in the cache. This is done by performing "consistent hashing" on request paths (when HTTP routing) or names requested for resolution (when DNS routing). To an extent, this behavior is configurable by modifying fields o [...]
+
+- HTTP, HTTP_NO_CACHE, HTTP_LIVE, HTTP_LIVE_NATNL, DNS, DNS_LIVE, and DNS_NATNL
+ These :ref:`Delivery Service Types <ds-types>` route directly to :term:`cache servers`, so consistent hashing is used to choose a :term:`cache server` to which the client will be redirected.
+
+- STEERING and CLIENT_STEERING
+ These :ref:`Delivery Service Types <ds-types>` route to "target" :term:`Delivery Services`, so consistent hashing is used to choose a "target" which will service the client request.
+
+.. seealso:: See `the Wikipedia article on consistent hashing <http://en.wikipedia.org/wiki/Consistent_hashing>`_.
+
+.. _pattern-based-consistenthash:
+
+Consistent Hashing Patterns
+---------------------------
+.. versionadded:: 4.0
+
+Regular expressions ("patterns") can be provided in the :ref:`ds-consistent-hashing-regex` field of an HTTP-:ref:`routed <ds-types>` Delivery Service to influence what parts of an HTTP request path are considered when performing consistent hashing. These patterns propagate to Traffic Router through :term:`Snapshots`.
+
+.. important:: Consistent Hashing Patterns on STEERING-:ref:`ds-types` :term:`Delivery Services` will be used for Consistent Hashing - the Consistent Hashing Pattern(s) of said :term:`Delivery Service`'s target(s) will **not** be considered. If Consistent Hashing Patterns are important to the routing of content on a STEERING-:ref:`ds-types` or CLIENT_STEERING-:ref:`ds-types` :term:`Delivery Service`, they **must** be defined *on that* :term:`Delivery Service` *itself, and* **not** *on it [...]
+
+How it Works
+""""""""""""
+The supplied :ref:`ds-consistent-hashing-regex` is applied to the request path to extract matching elements to build a new string *before* consistent hashing is done. For example, using the pattern :regexp:`/.*?(/.*?/).*?(m3u8)` and given the request paths ``/test/path/asset.m3u8`` and ``/other/path/asset.m3u8`` the resulting string used for consistent hashing will be ``/path/m3u8``
+
+.. seealso:: See Oracle's `documentation for the java.util.regex.Pattern <https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html>`_ implementation in Java.
+
+Testing Pattern-Based Consistent Hashing
+""""""""""""""""""""""""""""""""""""""""
+In order to test this feature without affecting the delivery of traffic through a CDN, there are several test tools in place.
+
+- :ref:`tr-api`
+ Several Traffic Router endpoints exist to test regular expression application against a request path, :term:`cache server` selection, and :term:`Delivery Service` selection.
+- :ref:`to-api`
+ The :ref:`to-api-consistenthash` endpoint will proxy request data through to one of the Traffic Router endpoints in order to test regular expression application against a request path, in the event that direct access to the :ref:`tr-api` is not possible and/or desired.
+- Traffic Portal
+ On the :term:`Delivery Service` creation/modification form in Traffic Portal (under :ref:`tp-services-delivery-service`), there is a :guilabel:`Test Regex` section that the user can use to validate a regular expression before saving it to a :term:`Delivery Service`.
+
+Consistent Hash Query Parameters
+--------------------------------
+Normally, when performing consistent hashing for an HTTP-:ref:`routed <ds-types>` :term:`Delivery Service`, any query parameters present in the request are ignored. That is, if a client requests ``/some/path?key=value`` consistent hashing is only performed on the string '``/some/path``'. However, query parameters that are part of uniquely identifying content can be specified by adding them to the set of :ref:`ds-consistent-hashing-qparams` of a :term:`Delivery Service`. For example, supp [...]
+
+.. note:: `Consistent Hashing Patterns`_ are applied *before* query parameters are considered - i.e. a pattern cannot match against query parameters, and need not worry about query parameters contaminating matches.
+
+.. important:: Consistent Hash Query Parameters on the *targets* of STEERING-:ref:`ds-types` :term:`Delivery Services` will be used for Consistent Hashing - the Consistent Hash Query Parameters of said :term:`Delivery Services` themselves will **not** be considered. If Consistent Hash Query Parameters are important to the routing of content on a STEERING-:ref:`ds-types` or CLIENT_STEERING-:ref:`ds-types` :term:`Delivery Service`, they **must** be defined *on that* :term:`Delivery Service [...]
+
+.. caution:: Certain query parameters are reserved by Traffic Router for its own use, and thus cannot be present in any Consistent Hash Query Parameters. These reserved parameters are:
+
+ - trred
+ - format
+ - fakeClientIPAddress
+
.. _tr-dnssec:
DNSSEC
@@ -569,40 +625,3 @@ The following is an example of the command line parameters set in :file:`/opt/tr
-XX:+UseG1GC \
-XX:+UnlockExperimentalVMOptions \
-XX:InitiatingHeapOccupancyPercent=30"
-
-.. _pattern-based-consistenthash:
-
-Pattern-Based Consistent Hashing Feature
-========================================
-
-.. versionadded:: 3.1
- Traffic Router now has the ability to influence consisting hashing using a regular expression on a per-HTTP :term:`Delivery Service` basis.
-
-Overview
---------
-Pattern-Based Consistent Hashing is a feature to modify the request path given to Traffic Router's consistent hasher for Cache selection (and :term:`Delivery Service` selection for Steering Delivery Services) using a regular expression. This new regular expression field 'Consistent Hash Regex' is applied on a per-Delivery Service basis and is given to Traffic Router via the CDN :term:`Snapshot`. The purpose of this feature is to increase cache efficiency by directing requests for the sam [...]
-
-.. Note:: Pattern-Based Consistent Hashing is only available for HTTP and Steering Delivery Services
-
-How it Works
-------------
-
-With Pattern-Based Consistent Hashing, a regular expression (Consistent Hash Regex) is applied to the request path to extract matching elements to build a new string to pass to the consistent hasher. i.e.: using the Consistent Hash Regex :regexp:`/.*?(/.*?/).*?(m3u8)` given the request paths ``/test/path/asset.m3u8`` and ``/other/path/asset.m3u8`` the resulting path to hash will be ``/path/m3u8``
-
-.. seealso:: See Oracle's `documentation for the java.util.regex.Pattern <https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html>`_ implementation in Java.
-
-HTTP
- When routing requests for an HTTP Delivery Service, Traffic Router compiles a list of eligible caches and uses consistent hashing on the request path to select one. With Pattern-Based Consistent Hashing, the request path is rebuilt prior to consistent hashing for Cache Selection.
-Steering
- For a Steering Delivery Service, Traffic Router uses consistent hashing on the request path to select a Target Delivery Service. In the case of Pattern-Based Consistent Hashing, the request path is rebuilt using the Consistent Hash Regex assigned to the Steering Delivery Service prior to consistent hashing. Then, the Target Delivery Service inherits the Consistent Hash Regex from the Steering Delivery Service and uses it for Cache selection.
-
-Testing Pattern-Based Consistent Hashing
-----------------------------------------
-In order to test this feature without affecting the delivery of traffic through a CDN, there are several test tools in place.
-
-Traffic Router API
- Several Traffic Router endpoints have been added to test regular expression application against a request path, Cache selection, and Delivery Service selection using Pattern-Based Consistent Hashing. For more information see the :ref:`Traffic Router API documentation <tr-api>`.
-Traffic Ops API
- A Traffic Ops endpoint has been added that will proxy request data through to one of the Traffic Router endpoints in order to test regular expression application against a request path. For more information see the :ref:`Traffic Ops API documentation <to-api-consistenthash>`.
-Traffic Portal Test Tool
- On the Delivery Service (HTTP and Steering) form in Traffic Portal, a "Test Regex" link has been added so that the user can validate a regular expression before saving it to a Delivery Service.
diff --git a/docs/source/api/cdns_name_snapshot.rst b/docs/source/api/cdns_name_snapshot.rst
index 6603ad8..846c6bd 100644
--- a/docs/source/api/cdns_name_snapshot.rst
+++ b/docs/source/api/cdns_name_snapshot.rst
@@ -166,6 +166,16 @@ Response Structure
.. seealso:: :ref:`anonymous_blocking-qht`
+ :consistentHashQueryParameters: A set of query parameters that Traffic Router should consider when determining a consistent hash for a given client request.
+
+ .. versionadded:: ATCv4
+ This endpoint does not, in general, obey the same versioning rules as all others. So this will appear in all API versions, but *only* if the Traffic Ops server is on version 4+
+
+ :consistentHashRegex: An optional regular expression that will ensure clients are consistently routed to a :term:`cache server` based on matches to it.
+
+ .. versionadded:: ATCv4
+ This endpoint does not, in general, obey the same versioning rules as all others. So this will appear in all API versions, but *only* if the Traffic Ops server is on version 4+
+
:coverageZoneOnly: A string containing a boolean that tells whether or not this :term:`Delivery Service` routes traffic based only on its Coverage Zone file
:deepCachingType: A string that tells when Deep Caching is used by this :term:`Delivery Service`; one of:
@@ -203,13 +213,13 @@ Response Structure
:type: The type of match performed using ``pattern`` to determine whether or not to use this :term:`Delivery Service`
HOST_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request\ [1]_
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request, or the name requested for resolution in a DNS request
HEADER_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [1]_
+ Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [#httpOnly]_
PATH_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL
+ Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL\ [#httpOnly]_
STEERING_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Service`\ s
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Services`
:missLocation: An object representing the default geographic coordinates to use for a client when lookup of their IP has failed in both the Coverage Zone file(s) and the IP-to-geographic-location database
@@ -537,4 +547,4 @@ Response Structure
}
}}
-.. [1] These only apply to HTTP-routed :term:`Delivery Service`\ s
+.. [#httpOnly] These only apply to HTTP-:ref:`routed <ds-types>` :term:`Delivery Services`
diff --git a/docs/source/api/cdns_name_snapshot_new.rst b/docs/source/api/cdns_name_snapshot_new.rst
index 486510b..352e771 100644
--- a/docs/source/api/cdns_name_snapshot_new.rst
+++ b/docs/source/api/cdns_name_snapshot_new.rst
@@ -165,6 +165,16 @@ Response Structure
.. seealso:: :ref:`anonymous_blocking-qht`
+ :consistentHashQueryParameters: A set of query parameters that Traffic Router should consider when determining a consistent hash for a given client request.
+
+ .. versionadded:: ATCv4
+ This endpoint does not, in general, obey the same versioning rules as all others. So this will appear in all API versions, but *only* if the Traffic Ops server is on version 4+
+
+ :consistentHashRegex: An optional regular expression that will ensure clients are consistently routed to a :term:`cache server` based on matches to it.
+
+ .. versionadded:: ATCv4
+ This endpoint does not, in general, obey the same versioning rules as all others. So this will appear in all API versions, but *only* if the Traffic Ops server is on version 4+
+
:coverageZoneOnly: A string containing a boolean that tells whether or not this :term:`Delivery Service` routes traffic based only on its Coverage Zone file
:deepCachingType: A string that tells when Deep Caching is used by this :term:`Delivery Service`; one of:
@@ -202,13 +212,13 @@ Response Structure
:type: The type of match performed using ``pattern`` to determine whether or not to use this :term:`Delivery Service`
HOST_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request\ [1]_
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request, or the name requested for resolution in a DNS request
HEADER_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [1]_
+ Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [#httpOnly]_
PATH_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL
+ Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL\ [#httpOnly]_
STEERING_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Service`\ s
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Services`
:missLocation: An object representing the default geographic coordinates to use for a client when lookup of their IP has failed in both the Coverage Zone file(s) and the IP-to-geographic-location database
@@ -540,4 +550,4 @@ Response Structure
}
}}
-.. [1] These only apply to HTTP-routed :term:`Delivery Service`\ s
+.. [#httpOnly] These only apply to HTTP-:ref:`routed <ds-types>` :term:`Delivery Services`
diff --git a/docs/source/api/deliveryservices.rst b/docs/source/api/deliveryservices.rst
index 87f63ae..4dfa8e2 100644
--- a/docs/source/api/deliveryservices.rst
+++ b/docs/source/api/deliveryservices.rst
@@ -21,7 +21,7 @@
``GET``
=======
-Retrieves all :term:`Delivery Service`\ s
+Retrieves all :term:`Delivery Services`
:Auth. Required: Yes
:Roles Required: None\ [1]_
@@ -56,20 +56,27 @@ Response Structure
.. deprecated:: ATCv3.0
This field has been deprecated in Traffic Control 3.x and is subject to removal in Traffic Control 4.x or later
-:ccrDnsTtl: The Time To Live (TTL) of the DNS response for A or AAAA record queries requesting the IP address of the Traffic Router - named "ccrDnsTtl" for legacy reasons
-:cdnId: The integral, unique identifier of the CDN to which the :term:`Delivery Service` belongs
-:cdnName: Name of the CDN to which the :term:`Delivery Service` belongs
-:checkPath: The path portion of the URL to check connections to this :term:`Delivery Service`'s origin server
-:consistentHashRegex: If defined, this is a regex used for the Pattern-Based Consistent Hashing feature. It is only applicable for HTTP and Steering Delivery Services
-:displayName: The display name of the :term:`Delivery Service`
-:dnsBypassCname: Domain name to overflow requests for HTTP :term:`Delivery Service`\ s - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`\ [4]_
-:dnsBypassIp: The IPv4 IP to use for bypass on a DNS :term:`Delivery Service` - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`\ [4]_
-:dnsBypassIp6: The IPv6 IP to use for bypass on a DNS :term:`Delivery Service` - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`\ [4]_
-:dnsBypassTtl: The time for which a DNS bypass of this :term:`Delivery Service`\ shall remain active\ [4]_
-:dscp: The Differentiated Services Code Point (DSCP) with which to mark traffic as it leaves the CDN and reaches clients
-:edgeHeaderRewrite: Rewrite operations to be performed on TCP headers at the Edge-tier cache level - used by the Header Rewrite Apache Trafficserver plugin
-:fqPacingRate: The Fair-Queuing Pacing Rate in Bytes per second set on the all TCP connection sockets in the :term:`Delivery Service` (see ``man tc-fc_codel`` for more information) - Linux only
-:geoLimit: The setting that determines how content is geographically limited - this is an integer on the interval [0-2] where the values have these meanings:
+:ccrDnsTtl: The Time To Live (TTL) of the DNS response for A or AAAA record queries requesting the IP address of the Traffic Router - named "ccrDnsTtl" for legacy reasons
+:cdnId: The integral, unique identifier of the CDN to which the :term:`Delivery Service` belongs
+:cdnName: Name of the CDN to which the :term:`Delivery Service` belongs
+:checkPath: The path portion of the URL to check connections to this :term:`Delivery Service`'s origin server
+:consistentHashRegex: If defined, this is a regular expression used for the Pattern-Based Consistent Hashing feature.\ [#httpOnly]_
+
+ .. versionadded:: 1.4
+
+:consistentHashQueryParams: A set (actually array due to limitations of JSON) of query parameters which will be considered by Traffic Router when using a client request to consistently find an :term:`Edge-tier cache server` to which to redirect them.\ [#httpOnly]_
+
+ .. versionadded:: 1.4
+
+:displayName: The display name of the :term:`Delivery Service`
+:dnsBypassCname: Domain name to overflow requests for HTTP :term:`Delivery Service`\ s - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`\ [4]_
+:dnsBypassIp: The IPv4 IP to use for bypass on a DNS :term:`Delivery Service` - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`\ [4]_
+:dnsBypassIp6: The IPv6 IP to use for bypass on a DNS :term:`Delivery Service` - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`\ [4]_
+:dnsBypassTtl: The time for which a DNS bypass of this :term:`Delivery Service`\ shall remain active\ [4]_
+:dscp: The Differentiated Services Code Point (DSCP) with which to mark traffic as it leaves the CDN and reaches clients
+:edgeHeaderRewrite: Rewrite operations to be performed on TCP headers at the Edge-tier cache level - used by the Header Rewrite Apache Trafficserver plugin
+:fqPacingRate: The Fair-Queuing Pacing Rate in Bytes per second set on the all TCP connection sockets in the :term:`Delivery Service` (see ``man tc-fc_codel`` for more information) - Linux only
+:geoLimit: The setting that determines how content is geographically limited - this is an integer on the interval [0-2] where the values have these meanings:
0
None - no limitations
@@ -94,8 +101,8 @@ Response Structure
:httpBypassFqdn: The HTTP destination to use for bypass on an HTTP :term:`Delivery Service` - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`
:id: An integral, unique identifier for this :term:`Delivery Service`
:infoUrl: This is a string which is expected to contain at least one URL pointing to more information about the :term:`Delivery Service`. Historically, this has been used to link relevant JIRA tickets
-:initialDispersion: The number of caches between which traffic requesting the same object will be randomly split - meaning that if 4 clients all request the same object (one after another), then if this is above 4 there is a possibility that all 4 are cache misses. For most use-cases, this should be 1\ [2]_
-:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable Edge-tier cache; if ``false`` all addresses will be IPv4, regardless of the client connection\ [2]_
+:initialDispersion: The number of caches between which traffic requesting the same object will be randomly split - meaning that if 4 clients all request the same object (one after another), then if this is above 4 there is a possibility that all 4 are cache misses. For most use-cases, this should be 1\ [#httpOnly]_
+:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable Edge-tier cache; if ``false`` all addresses will be IPv4, regardless of the client connection
:lastUpdated: The date and time at which this :term:`Delivery Service` was last updated, in a ``ctime``-like format
:logsEnabled: If ``true``, logging is enabled for this :term:`Delivery Service`, otherwise it is disabled
:longDesc: A description of the :term:`Delivery Service`
@@ -108,13 +115,13 @@ Response Structure
:type: The type of match performed using ``pattern`` to determine whether or not to use this :term:`Delivery Service`
HOST_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request\ [2]_
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request, or the name requested for resolution in a DNS request
HEADER_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [2]_
+ Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [#httpOnly]_
PATH_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL
+ Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL\ [#httpOnly]_
STEERING_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Service`\ s
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Service`
:maxDnsAnswers: The maximum number of IPs to put in responses to A/AAAA DNS record requests (0 means all available)\ [4]_
:maxOriginConnections: The maximum number of connections allowed to the origin (0 means no maximum).
@@ -133,7 +140,7 @@ Response Structure
:profileDescription: The description of the Traffic Router Profile with which this :term:`Delivery Service` is associated
:profileId: The integral, unique identifier for the Traffic Router profile with which this :term:`Delivery Service` is associated
:profileName: The name of the Traffic Router Profile with which this :term:`Delivery Service` is associated
-:protocol: The protocol which clients will use to communicate with Edge-tier :term:`cache server` s\ [2]_ - this is an integer on the interval [0-2] where the values have these meanings:
+:protocol: The protocol which clients will use to communicate with Edge-tier :term:`cache server` s\ [#httpOnly]_ - this is an integer on the interval [0-2] where the values have these meanings:
0
HTTP
@@ -186,8 +193,8 @@ Response Structure
.. warning:: This number will not be correct if keys are manually replaced using the API, as the key generation API does not increment it!
:tenantId: The integral, unique identifier of the tenant who owns this :term:`Delivery Service`
-:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [2]_
-:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [2]_
+:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
+:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
:type: The name of the routing type of this :term:`Delivery Service` e.g. "HTTP"
:typeId: The integral, unique identifier of the routing type of this :term:`Delivery Service`
:xmlId: A unique string that describes this :term:`Delivery Service` - exists for legacy reasons
@@ -207,8 +214,7 @@ Response Structure
Date: Thu, 15 Nov 2018 19:04:29 GMT
Transfer-Encoding: chunked
- { "response": [
- {
+ { "response": [{
"active": true,
"anonymousBlockingEnabled": false,
"cacheurl": null,
@@ -234,7 +240,7 @@ Response Structure
"infoUrl": null,
"initialDispersion": 1,
"ipv6RoutingEnabled": true,
- "lastUpdated": "2018-11-14 18:21:17+00",
+ "lastUpdated": "2019-05-15 14:32:05+00",
"logsEnabled": true,
"longDesc": "Apachecon North America 2018",
"longDesc1": null,
@@ -247,7 +253,6 @@ Response Structure
}
],
"maxDnsAnswers": null,
- "maxOriginConnections": 0,
"midHeaderRewrite": null,
"missLat": 42,
"missLong": -88,
@@ -257,7 +262,7 @@ Response Structure
"profileDescription": null,
"profileId": null,
"profileName": null,
- "protocol": 0,
+ "protocol": 2,
"qstringIgnore": 0,
"rangeRequestHandling": 0,
"regexRemap": null,
@@ -271,17 +276,29 @@ Response Structure
"typeId": 1,
"xmlId": "demo1",
"exampleURLs": [
- "http://video.demo1.mycdn.ciab.test"
+ "http://video.demo1.mycdn.ciab.test",
+ "https://video.demo1.mycdn.ciab.test"
],
"deepCachingType": "NEVER",
+ "fqPacingRate": null,
"signingAlgorithm": null,
- "tenant": "root"
+ "tenant": "root",
+ "trResponseHeaders": null,
+ "trRequestHeaders": null,
+ "consistentHashRegex": null,
+ "consistentHashQueryParams": [
+ "abc",
+ "pdq",
+ "xxx",
+ "zyx"
+ ],
+ "maxOriginConnections": 0
}]}
-.. [1] Users with the roles "admin" and/or "operations" will be able to see *all* :term:`Delivery Service`\ s, whereas any other user will only see the :term:`Delivery Service`\ s their Tenant is allowed to see.
-.. [2] This only applies to HTTP-routed :term:`Delivery Service`\ s
+.. [1] Users with the roles "admin" and/or "operations" will be able to see *all* :term:`Delivery Services`, whereas any other user will only see the :term:`Delivery Services` their Tenant is allowed to see.
+.. [#httpOnly] This only applies to HTTP-:ref:`routed <ds-types>` :term:`Delivery Services`
.. [3] See :ref:`ds-multi-site-origin`
-.. [4] This only applies to DNS-routed :term:`Delivery Service`\ s
+.. [4] This only applies to DNS-routed :term:`Delivery Services`
``POST``
========
@@ -300,11 +317,18 @@ Request Structure
.. deprecated:: ATCv3.0
This field has been deprecated in Traffic Control 3.x and is subject to removal in Traffic Control 4.x or later
-:ccrDnsTtl: The Time To Live (TTL) in seconds of the DNS response for A or AAAA record queries requesting the IP address of the Traffic Router - named "ccrDnsTtl" for legacy reasons
-:cdnId: The integral, unique identifier for the CDN to which this :term:`Delivery Service`\ shall be assigned
-:checkPath: The path portion of the URL which will be used to check connections to this :term:`Delivery Service`'s origin server
-:consistentHashRegex: If defined, this is a regex used for the Pattern-Based Consistent Hashing feature. It is only applicable for HTTP and Steering Delivery Services
-:deepCachingType: A string describing when to do Deep Caching for this :term:`Delivery Service`:
+:ccrDnsTtl: The Time To Live (TTL) in seconds of the DNS response for A or AAAA record queries requesting the IP address of the Traffic Router - named "ccrDnsTtl" for legacy reasons
+:cdnId: The integral, unique identifier for the CDN to which this :term:`Delivery Service`\ shall be assigned
+:checkPath: The path portion of the URL which will be used to check connections to this :term:`Delivery Service`'s origin server
+:consistentHashRegex: If defined, this is a regular expression used for the Pattern-Based Consistent Hashing feature. It is only applicable for HTTP and Steering Delivery Services
+
+ .. versionadded:: 1.4
+
+:consistentHashQueryParams: If defined, this is a set (encoded as an array, but duplicates are **not** allowed and order is not preserved) of query parameters which will be considered by Traffic Router when using a client request to consistently find an :term:`Edge-tier cache server` to which to redirect them.\ [#httpOnly]_
+
+ .. versionadded:: 1.4
+
+:deepCachingType: A string describing when to do Deep Caching for this :term:`Delivery Service`:
NEVER
Deep Caching will never be used by this :term:`Delivery Service` (default)
@@ -341,10 +365,10 @@ Request Structure
:globalMaxMbps: An optional integer that will set the maximum global bandwidth allowed on this :term:`Delivery Service`. If exceeded, traffic will be routed to ``dnsBypassIp`` (or ``dnsBypassIp6`` for IPv6 traffic) for DNS :term:`Delivery Service`\ s and to ``httpBypassFqdn`` for HTTP :term:`Delivery Service`\ s
:globalMaxTps: An optional integer that will set the maximum global transactions per second allowed on this :term:`Delivery Service`. When this is exceeded traffic will be sent to the ``dnsBpassIp`` (and/or ``dnsBypassIp6``)for DNS :term:`Delivery Service`\ s and to the ``httpBypassFqdn`` for HTTP :term:`Delivery Service`\ s
-:httpBypassFqdn: An optional Fully Qualified Domain Name (FQDN) to use for bypass on an HTTP :term:`Delivery Service` - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`\ [2]_
+:httpBypassFqdn: An optional Fully Qualified Domain Name (FQDN) to use for bypass on an HTTP :term:`Delivery Service` - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`\ [#httpOnly]_
:infoUrl: An optional string which, if present, is expected to contain at least one URL pointing to more information about the :term:`Delivery Service`. Historically, this has been used to link relevant JIRA tickets
-:initialDispersion: The number of caches between which traffic requesting the same object will be randomly split - meaning that if 4 clients all request the same object (one after another), then if this is above 4 there is a possibility that all 4 are cache misses. For most use-cases, this should be 1\ [2]_\ [6]_
-:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable Edge-tier cache; if ``false`` all addresses will be IPv4, regardless of the client connection - optional for ANY_MAP :term:`Delivery Service`\ s
+:initialDispersion: The number of caches between which traffic requesting the same object will be randomly split - meaning that if 4 clients all request the same object (one after another), then if this is above 4 there is a possibility that all 4 are cache misses. For most use-cases, this should be 1\ [#httpOnly]_\ [6]_
+:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable :term:`Edge-tier cache server`; if ``false`` all addresses will be IPv4, regardless of the client connection - optional for ANY_MAP-:ref:`ds-types` :term:`Delivery Services`
:logsEnabled: If ``true``, logging is enabled for this :term:`Delivery Service`, otherwise it is disabled
:longDesc: An optional description of the :term:`Delivery Service`
:longDesc1: An optional field used when more detailed information that that provided by ``longDesc`` is desired
@@ -354,11 +378,11 @@ Request Structure
.. versionadded:: 1.4
-:midHeaderRewrite: An optional string containing rewrite operations to be performed on TCP headers at the Edge-tier cache level - used by the Header Rewrite Apache Trafficserver plugin
-:missLat: The latitude to use when the client cannot be found in the CZF or a geographic IP lookup\ [7]_
-:missLong: The longitude to use when the client cannot be found in the CZF or a geographic IP lookup\ [7]_
-:multiSiteOrigin: ``true`` if the Multi Site Origin feature is enabled for this :term:`Delivery Service`, ``false`` otherwise\ [3]_\ [7]_
-:orgServerFqdn: The URL of the :term:`Delivery Service`'s origin server for use in retrieving content from the origin server\ [7]_
+:midHeaderRewrite: An optional string containing rewrite operations to be performed on TCP headers at the Edge-tier cache level - used by the Header Rewrite Apache Trafficserver plugin
+:missLat: The latitude to use when the client cannot be found in the CZF or a geographic IP lookup\ [7]_
+:missLong: The longitude to use when the client cannot be found in the CZF or a geographic IP lookup\ [7]_
+:multiSiteOrigin: ``true`` if the Multi Site Origin feature is enabled for this :term:`Delivery Service`, ``false`` otherwise\ [3]_\ [7]_
+:orgServerFqdn: The URL of the :term:`Delivery Service`'s origin server for use in retrieving content from the origin server\ [7]_
.. note:: Despite the field name, this must truly be a full URL - including the protocol (e.g. ``http://`` or ``https://``) - **NOT** merely the server's Fully Qualified Domain Name (FQDN)
@@ -418,8 +442,8 @@ Request Structure
.. warning:: This number will not be correct if keys are manually replaced using the API, as the key generation API does not increment it!
:tenantId: An optional, integral, unique identifier of the tenant who will own this :term:`Delivery Service`
-:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [2]_
-:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [2]_
+:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
+:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
:typeId: The integral, unique identifier for the routing type of this :term:`Delivery Service`
:xmlId: A unique string that describes this :term:`Delivery Service` - exists for legacy reasons
@@ -485,11 +509,18 @@ Response Structure
.. deprecated:: ATCv3.0
This field has been deprecated in Traffic Control 3.x and is subject to removal in Traffic Control 4.x or later
-:ccrDnsTtl: The Time To Live (TTL) of the DNS response for A or AAAA record queries requesting the IP address of the Traffic Router - named "ccrDnsTtl" for legacy reasons
-:cdnId: The integral, unique identifier of the CDN to which the :term:`Delivery Service` belongs
-:cdnName: Name of the CDN to which the :term:`Delivery Service` belongs
-:checkPath: The path portion of the URL to check connections to this :term:`Delivery Service`'s origin server
-:consistentHashRegex: If defined, this is a regex used for the Pattern-Based Consistent Hashing feature. It is only applicable for HTTP and Steering Delivery Services
+:ccrDnsTtl: The Time To Live (TTL) of the DNS response for A or AAAA record queries requesting the IP address of the Traffic Router - named "ccrDnsTtl" for legacy reasons
+:cdnId: The integral, unique identifier of the CDN to which the :term:`Delivery Service` belongs
+:cdnName: Name of the CDN to which the :term:`Delivery Service` belongs
+:checkPath: The path portion of the URL to check connections to this :term:`Delivery Service`'s origin server
+:consistentHashRegex: If defined, this is a regular expression used for the Pattern-Based Consistent Hashing feature.\ [#httpOnly]_
+
+ .. versionadded:: 1.4
+
+:consistentHashQueryParams: A set (actually array due to limitations of JSON) of query parameters which will be considered by Traffic Router when using a client request to consistently find an :term:`Edge-tier cache server` to which to redirect them.\ [#httpOnly]_
+
+ .. versionadded:: 1.4
+
:displayName: The display name of the :term:`Delivery Service`
:dnsBypassCname: Domain name to overflow requests for HTTP :term:`Delivery Service`\ s - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`\ [4]_
:dnsBypassIp: The IPv4 IP to use for bypass on a DNS :term:`Delivery Service` - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`\ [4]_
@@ -523,8 +554,8 @@ Response Structure
:httpBypassFqdn: The HTTP destination to use for bypass on an HTTP :term:`Delivery Service` - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`
:id: An integral, unique identifier for this :term:`Delivery Service`
:infoUrl: This is a string which is expected to contain at least one URL pointing to more information about the :term:`Delivery Service`. Historically, this has been used to link relevant JIRA tickets
-:initialDispersion: The number of caches between which traffic requesting the same object will be randomly split - meaning that if 4 clients all request the same object (one after another), then if this is above 4 there is a possibility that all 4 are cache misses. For most use-cases, this should be 1\ [2]_
-:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable Edge-tier cache; if ``false`` all addresses will be IPv4, regardless of the client connection\ [2]_
+:initialDispersion: The number of caches between which traffic requesting the same object will be randomly split - meaning that if 4 clients all request the same object (one after another), then if this is above 4 there is a possibility that all 4 are cache misses. For most use-cases, this should be 1\ [#httpOnly]_
+:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable :term:`Edge-tier cache server`; if ``false`` all addresses will be IPv4, regardless of the client connection
:lastUpdated: The date and time at which this :term:`Delivery Service` was last updated, in a ``ctime``-like format
:logsEnabled: If ``true``, logging is enabled for this :term:`Delivery Service`, otherwise it is disabled
:longDesc: A description of the :term:`Delivery Service`
@@ -537,13 +568,13 @@ Response Structure
:type: The type of match performed using ``pattern`` to determine whether or not to use this :term:`Delivery Service`
HOST_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request\ [2]_
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request, or the name requested for resolution in a DNS request
HEADER_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [2]_
+ Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [#httpOnly]_
PATH_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL
+ Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL\ [#httpOnly]_
STEERING_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Service`\ s
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Service`
:maxDnsAnswers: The maximum number of IPs to put in responses to A/AAAA DNS record requests (0 means all available)\ [4]_
:maxOriginConnections: The maximum number of connections allowed to the origin (0 means no maximum).
@@ -562,7 +593,7 @@ Response Structure
:profileDescription: The description of the Traffic Router Profile with which this :term:`Delivery Service` is associated
:profileId: The integral, unique identifier for the Traffic Router profile with which this :term:`Delivery Service` is associated
:profileName: The name of the Traffic Router Profile with which this :term:`Delivery Service` is associated
-:protocol: The protocol which clients will use to communicate with Edge-tier :term:`cache server` s\ [2]_ - this is an integer on the interval [0-2] where the values have these meanings:
+:protocol: The protocol which clients will use to communicate with Edge-tier :term:`cache server` s\ [#httpOnly]_ - this is an integer on the interval [0-2] where the values have these meanings:
0
HTTP
@@ -615,8 +646,8 @@ Response Structure
.. warning:: This number will not be correct if keys are manually replaced using the API, as the key generation API does not increment it!
:tenantId: The integral, unique identifier of the tenant who owns this :term:`Delivery Service`
-:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [2]_
-:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [2]_
+:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
+:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
:type: The name of the routing type of this :term:`Delivery Service` e.g. "HTTP"
:typeId: The integral, unique identifier of the routing type of this :term:`Delivery Service`
:xmlId: A unique string that describes this :term:`Delivery Service` - exists for legacy reasons
diff --git a/docs/source/api/deliveryservices_id.rst b/docs/source/api/deliveryservices_id.rst
index d35c257..8984f23 100644
--- a/docs/source/api/deliveryservices_id.rst
+++ b/docs/source/api/deliveryservices_id.rst
@@ -68,15 +68,19 @@ Response Structure
.. deprecated:: ATCv3.0
This field has been deprecated in Traffic Control 3.x and is subject to removal in Traffic Control 4.x or later
-:ccrDnsTtl: The Time To Live (TTL) of the DNS response for A or AAAA record queries requesting the IP address of the Traffic Router - named "ccrDnsTtl" for legacy reasons
-:cdnId: The integral, unique identifier of the CDN to which the :term:`Delivery Service` belongs
-:cdnName: Name of the CDN to which the :term:`Delivery Service` belongs
-:checkPath: The path portion of the URL to check connections to this :term:`Delivery Service`'s origin server
-:consistentHashRegex: If defined, this is a regex used for the Pattern-Based Consistent Hashing feature. It is only applicable for HTTP and Steering Delivery Services
+:ccrDnsTtl: The Time To Live (TTL) of the DNS response for A or AAAA record queries requesting the IP address of the Traffic Router - named "ccrDnsTtl" for legacy reasons
+:cdnId: The integral, unique identifier of the CDN to which the :term:`Delivery Service` belongs
+:cdnName: Name of the CDN to which the :term:`Delivery Service` belongs
+:checkPath: The path portion of the URL to check connections to this :term:`Delivery Service`'s origin server
+:consistentHashRegex: If defined, this is a regular expression used for the Pattern-Based Consistent Hashing feature.\ [#httpOnly]_
- .. versionadded:: 1.5
+ .. versionadded:: 1.4
-:deepCachingType: A string that describes when "Deep Caching" will be used by this :term:`Delivery Service` - one of:
+:consistentHashQueryParams: A set (actually array due to limitations of JSON) of query parameters which will be considered by Traffic Router when using a client request to consistently find an :term:`Edge-tier cache server` to which to redirect them.\ [#httpOnly]_
+
+ .. versionadded:: 1.4
+
+:deepCachingType: A string that describes when "Deep Caching" will be used by this :term:`Delivery Service` - one of:
ALWAYS
"Deep Caching" will always be used with this :term:`Delivery Service`
@@ -117,7 +121,7 @@ Response Structure
:id: An integral, unique identifier for this :term:`Delivery Service`
:infoUrl: This is a string which is expected to contain at least one URL pointing to more information about the :term:`Delivery Service`. Historically, this has been used to link relevant JIRA tickets
:initialDispersion: The number of caches between which traffic requesting the same object will be randomly split - meaning that if 4 clients all request the same object (one after another), then if this is above 4 there is a possibility that all 4 are cache misses. For most use-cases, this should be 1
-:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable Edge-tier cache; if ``false`` all addresses will be IPv4, regardless of the client connection\ [2]_
+:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable :term:`Edge-tier cache server`; if ``false`` all addresses will be IPv4, regardless of the client connection
:lastUpdated: The date and time at which this :term:`Delivery Service` was last updated, in a ``ctime``-like format
:logsEnabled: If ``true``, logging is enabled for this :term:`Delivery Service`, otherwise it is disabled
:longDesc: A description of the :term:`Delivery Service`
@@ -130,13 +134,13 @@ Response Structure
:type: The type of match performed using ``pattern`` to determine whether or not to use this :term:`Delivery Service`
HOST_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request\ [2]_
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request, or the name requested for resolution in a DNS request
HEADER_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [2]_
+ Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [#httpOnly]_
PATH_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL
+ Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL\ [#httpOnly]_
STEERING_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Service`\ s
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Services`
:maxDnsAnswers: The maximum number of IPs to put in a A/AAAA response for a DNS :term:`Delivery Service` (0 means all available)
:midHeaderRewrite: Rewrite operations to be performed on TCP headers at the Edge-tier cache level - used by the Header Rewrite Apache Trafficserver plugin
@@ -148,7 +152,7 @@ Response Structure
:profileDescription: The description of the Traffic Router Profile with which this :term:`Delivery Service` is associated
:profileId: The integral, unique identifier for the Traffic Router profile with which this :term:`Delivery Service` is associated
:profileName: The name of the Traffic Router Profile with which this :term:`Delivery Service` is associated
-:protocol: The protocol which clients will use to communicate with Edge-tier :term:`cache server` s\ [2]_ - this is an integer on the interval [0-2] where the values have these meanings:
+:protocol: The protocol which clients will use to communicate with Edge-tier :term:`cache server` s\ [#httpOnly]_ - this is an integer on the interval [0-2] where the values have these meanings:
0
HTTP
@@ -166,7 +170,7 @@ Response Structure
2
Query strings are stripped out by Edge-tier caches, and thus are neither taken into consideration for caching purposes, nor passed upstream in requests to the origin
-:rangeRequestHandling: Tells caches how to handle range requests\ [2]_ - this is an integer on the interval [0-2] where the values have these meanings:
+:rangeRequestHandling: Tells caches how to handle range requests\ [#httpOnly]_ - this is an integer on the interval [0-2] where the values have these meanings:
0
Range requests will not be cached, but range requests that request ranges of content already cached will be served from the cache
@@ -207,8 +211,8 @@ Response Structure
.. versionadded:: 1.3
:tenantId: The integral, unique identifier of the tenant who owns this :term:`Delivery Service`
-:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [2]_
-:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [2]_
+:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
+:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
:type: The name of the routing type of this :term:`Delivery Service` e.g. "HTTP"
:typeId: The integral, unique identifier of the routing type of this :term:`Delivery Service`
:xmlId: A unique string that describes this :term:`Delivery Service` - exists for legacy reasons
@@ -300,10 +304,10 @@ Response Structure
]}
-.. [1] Users with the roles "admin" and/or "operation" will be able to see *all* :term:`Delivery Service`\ s, whereas any other user will only see the :term:`Delivery Service`\ s their Tenant is allowed to see.
-.. [2] This only applies to HTTP :term:`Delivery Service`\ s
+.. [1] Users with the :term:`Roles` "admin" and/or "operation" will be able to see *all* :term:`Delivery Services`, whereas any other user will only see the :term:`Delivery Services` their :term:`Tenant` is allowed to see.
+.. [#httpOnly] This only applies to HTTP :term:`Delivery Services`
.. [3] See :ref:`ds-multi-site-origin`
-.. [4] This only applies to DNS-routed :term:`Delivery Service`\ s
+.. [4] This only applies to DNS-:ref:`routed <ds-types>` :term:`Delivery Services`
``PUT``
=======
@@ -322,14 +326,18 @@ Request Structure
.. deprecated:: ATCv3.0
This field has been deprecated in Traffic Control 3.x and is subject to removal in Traffic Control 4.x or later
-:ccrDnsTtl: The Time To Live (TTL) in seconds of the DNS response for A or AAAA record queries requesting the IP address of the Traffic Router - named "ccrDnsTtl" for legacy reasons
-:cdnId: The integral, unique identifier for the CDN to which this :term:`Delivery Service`\ shall be assigned
-:checkPath: The path portion of the URL which will be used to check connections to this :term:`Delivery Service`'s origin server
-:consistentHashRegex: If defined, this is a regex used for the Pattern-Based Consistent Hashing feature. It is only applicable for HTTP and Steering Delivery Services
+:ccrDnsTtl: The Time To Live (TTL) in seconds of the DNS response for A or AAAA record queries requesting the IP address of the Traffic Router - named "ccrDnsTtl" for legacy reasons
+:cdnId: The integral, unique identifier for the CDN to which this :term:`Delivery Service`\ shall be assigned
+:checkPath: The path portion of the URL which will be used to check connections to this :term:`Delivery Service`'s origin server
+:consistentHashRegex: If defined, this is a regular expression used for the Pattern-Based Consistent Hashing feature.\ [#httpOnly]_
+
+ .. versionadded:: 1.4
+
+:consistentHashQueryParams: A set (actually array due to limitations of JSON) of query parameters which will be considered by Traffic Router when using a client request to consistently find an :term:`Edge-tier cache server` to which to redirect them.\ [#httpOnly]_
- .. versionadded:: 1.5
+ .. versionadded:: 1.4
-:deepCachingType: A string describing when to do Deep Caching for this :term:`Delivery Service`:
+:deepCachingType: A string describing when to do Deep Caching for this :term:`Delivery Service`:
NEVER
Deep Caching will never be used by this :term:`Delivery Service` (default)
@@ -366,10 +374,10 @@ Request Structure
:globalMaxMbps: An optional integer that will set the maximum global bandwidth allowed on this :term:`Delivery Service`. If exceeded, traffic will be routed to ``dnsBypassIp`` (or ``dnsBypassIp6`` for IPv6 traffic) for DNS :term:`Delivery Service`\ s and to ``httpBypassFqdn`` for HTTP :term:`Delivery Service`\ s
:globalMaxTps: An optional integer that will set the maximum global transactions per second allowed on this :term:`Delivery Service`. When this is exceeded traffic will be sent to the ``dnsBpassIp`` (and/or ``dnsBypassIp6``)for DNS :term:`Delivery Service`\ s and to the ``httpBypassFqdn`` for HTTP :term:`Delivery Service`\ s
-:httpBypassFqdn: An optional Fully Qualified Domain Name (FQDN) to use for bypass on an HTTP :term:`Delivery Service` - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`\ [2]_
+:httpBypassFqdn: An optional Fully Qualified Domain Name (FQDN) to use for bypass on an HTTP :term:`Delivery Service` - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`\ [#httpOnly]_
:infoUrl: An optional string which, if present, is expected to contain at least one URL pointing to more information about the :term:`Delivery Service`. Historically, this has been used to link relevant JIRA tickets
-:initialDispersion: The number of caches between which traffic requesting the same object will be randomly split - meaning that if 4 clients all request the same object (one after another), then if this is above 4 there is a possibility that all 4 are cache misses. For most use-cases, this should be 1\ [2]_\ [6]_
-:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable Edge-tier cache; if ``false`` all addresses will be IPv4, regardless of the client connection - optional for ANY_MAP :term:`Delivery Service`\ s
+:initialDispersion: The number of caches between which traffic requesting the same object will be randomly split - meaning that if 4 clients all request the same object (one after another), then if this is above 4 there is a possibility that all 4 are cache misses. For most use-cases, this should be 1\ [#httpOnly]_\ [6]_
+:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable :term:`Edge-tier cache server`; if ``false`` all addresses will be IPv4, regardless of the client connection - optional for ANY_MAP-:ref:`ds-types` :term:`Delivery Services`
:logsEnabled: If ``true``, logging is enabled for this :term:`Delivery Service`, otherwise it is disabled
:longDesc: An optional description of the :term:`Delivery Service`
:longDesc1: An optional field used when more detailed information that that provided by ``longDesc`` is desired
@@ -439,8 +447,8 @@ Request Structure
.. warning:: This number will not be correct if keys are manually replaced using the API, as the key generation API does not increment it!
:tenantId: An optional, integral, unique identifier of the tenant who will own this :term:`Delivery Service`
-:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [2]_
-:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [2]_
+:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
+:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
:typeId: The integral, unique identifier for the routing type of this :term:`Delivery Service`
:xmlId: A unique string that describes this :term:`Delivery Service` - exists for legacy reasons
diff --git a/docs/source/api/deliveryservices_id_safe.rst b/docs/source/api/deliveryservices_id_safe.rst
index 11f91fb..cf0f042 100644
--- a/docs/source/api/deliveryservices_id_safe.rst
+++ b/docs/source/api/deliveryservices_id_safe.rst
@@ -77,13 +77,17 @@ Response Structure
.. deprecated:: ATCv3.0
This field has been deprecated in Traffic Control 3.x and is subject to removal in Traffic Control 4.x or later
-:ccrDnsTtl: The Time To Live (TTL) of the DNS response for A or AAAA record queries requesting the IP address of the Traffic Router - named "ccrDnsTtl" for legacy reasons
-:cdnId: The integral, unique identifier of the CDN to which the :term:`Delivery Service` belongs
-:cdnName: Name of the CDN to which the :term:`Delivery Service` belongs
-:checkPath: The path portion of the URL to check connections to this :term:`Delivery Service`'s origin server
-:consistentHashRegex: If defined, this is a regex used for the Pattern-Based Consistent Hashing feature. It is only applicable for HTTP and Steering Delivery Services
+:ccrDnsTtl: The Time To Live (TTL) of the DNS response for A or AAAA record queries requesting the IP address of the Traffic Router - named "ccrDnsTtl" for legacy reasons
+:cdnId: The integral, unique identifier of the CDN to which the :term:`Delivery Service` belongs
+:cdnName: Name of the CDN to which the :term:`Delivery Service` belongs
+:checkPath: The path portion of the URL to check connections to this :term:`Delivery Service`'s origin server
+:consistentHashRegex: If defined, this is a regular expression used for the Pattern-Based Consistent Hashing feature.\ [#httpOnly]_
- .. versionadded:: 1.5
+ .. versionadded:: 1.4
+
+:consistentHashQueryParams: A set (actually array due to limitations of JSON) of query parameters which will be considered by Traffic Router when using a client request to consistently find an :term:`Edge-tier cache server` to which to redirect them.\ [#httpOnly]_
+
+ .. versionadded:: 1.4
:deepCachingType: A string that describes when "Deep Caching" will be used by this :term:`Delivery Service` - one of:
@@ -126,7 +130,7 @@ Response Structure
:id: An integral, unique identifier for this :term:`Delivery Service`
:infoUrl: This is a string which is expected to contain at least one URL pointing to more information about the :term:`Delivery Service`. Historically, this has been used to link relevant JIRA tickets
:initialDispersion: The number of caches between which traffic requesting the same object will be randomly split - meaning that if 4 clients all request the same object (one after another), then if this is above 4 there is a possibility that all 4 are cache misses. For most use-cases, this should be 1
-:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable Edge-tier cache; if ``false`` all addresses will be IPv4, regardless of the client connection\ [2]_
+:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable :term:`Edge-tier cache server`; if ``false`` all addresses will be IPv4, regardless of the client connection
:lastUpdated: The date and time at which this :term:`Delivery Service` was last updated, in a ``ctime``-like format
:logsEnabled: If ``true``, logging is enabled for this :term:`Delivery Service`, otherwise it is disabled
:longDesc: A description of the :term:`Delivery Service`
@@ -139,13 +143,13 @@ Response Structure
:type: The type of match performed using ``pattern`` to determine whether or not to use this :term:`Delivery Service`
HOST_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request\ [2]_
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request, or the name requested for resolution in a DNS request
HEADER_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [2]_
+ Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [#httpOnly]_
PATH_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL
+ Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL\ [#httpOnly]_
STEERING_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Service`\ s
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Services`
:maxDnsAnswers: The maximum number of IPs to put in a A/AAAA response for a DNS :term:`Delivery Service` (0 means all available)\ [4]_
:midHeaderRewrite: Rewrite operations to be performed on TCP headers at the Edge-tier cache level - used by the Header Rewrite Apache Trafficserver plugin
@@ -157,7 +161,7 @@ Response Structure
:profileDescription: The description of the Traffic Router Profile with which this :term:`Delivery Service` is associated
:profileId: The integral, unique identifier for the Traffic Router profile with which this :term:`Delivery Service` is associated
:profileName: The name of the Traffic Router Profile with which this :term:`Delivery Service` is associated
-:protocol: The protocol which clients will use to communicate with Edge-tier :term:`cache server`\ s\ [2]_ - this is an integer on the interval [0-2] where the values have these meanings:
+:protocol: The protocol which clients will use to communicate with Edge-tier :term:`cache server`\ s\ [#httpOnly]_ - this is an integer on the interval [0-2] where the values have these meanings:
0
HTTP
@@ -215,9 +219,9 @@ Response Structure
.. versionadded:: 1.3
-:tenantId: The integral, unique identifier of the tenant who owns this :term:`Delivery Service`
-:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [2]_
-:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [2]_
+:tenantId: The integral, unique identifier of the :term:`Tenant` who owns this :term:`Delivery Service`
+:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
+:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
:type: The name of the routing type of this :term:`Delivery Service` e.g. "HTTP"
:typeId: The integral, unique identifier of the routing type of this :term:`Delivery Service`
:xmlId: A unique string that describes this :term:`Delivery Service` - exists for legacy reasons
@@ -318,8 +322,8 @@ Response Structure
]}
-.. [1] Users with the "admin" or "operations" roles will be able to edit *any*:term:`Delivery Service`, whereas other users will only be able to edit :term:`Delivery Service`\ s that their tenant has permissions to edit.
-.. [2] This only applies to HTTP-routed :term:`Delivery Service`\ s
+.. [1] Users with the "admin" or "operations" roles will be able to edit *any*:term:`Delivery Service`, whereas other users will only be able to edit :term:`Delivery Services` that their tenant has permissions to edit.
+.. [#httpOnly] This only applies to HTTP-:ref:`routed <ds-types>` :term:`Delivery Services`
.. [3] See :ref:`ds-multi-site-origin`
-.. [4] This only applies to DNS-routed :term:`Delivery Service`\ s
-.. [5] These fields are required for HTTP-routed and DNS-routed :term:`Delivery Service`\ s, but are optional for (and in fact may have no effect on) STEERING and ANY_MAP :term:`Delivery Service`\ s
+.. [4] This only applies to DNS-:ref:`routed <ds-types>` :term:`Delivery Services`
+.. [5] These fields are required for HTTP-:ref:`routed <ds-types>` and DNS-:ref:`routed <ds-types>` :term:`Delivery Services`, but are optional for (and in fact may have no effect on) STEERING and ANY_MAP :term:`Delivery Services`
diff --git a/docs/source/api/deliveryservices_id_servers.rst b/docs/source/api/deliveryservices_id_servers.rst
index bf421e9..11500c8 100644
--- a/docs/source/api/deliveryservices_id_servers.rst
+++ b/docs/source/api/deliveryservices_id_servers.rst
@@ -18,8 +18,7 @@
***********************************
``deliveryservices/{{ID}}/servers``
***********************************
-.. deprecated:: 1.1
- Use :ref:`to-api-deliveryserviceserver` instead
+.. caution:: It's often much easier to use :ref:`to-api-deliveryservices-xmlid-servers` instead
``GET``
=======
diff --git a/docs/source/api/deliveryservices_xmlid_servers.rst b/docs/source/api/deliveryservices_xmlid_servers.rst
index 48e7844..d15ccfa 100644
--- a/docs/source/api/deliveryservices_xmlid_servers.rst
+++ b/docs/source/api/deliveryservices_xmlid_servers.rst
@@ -18,8 +18,6 @@
***************************************
``deliveryservices/{{xml_id}}/servers``
***************************************
-.. deprecated:: 1.1
- Use :ref:`to-api-deliveryserviceserver` instead
``POST``
========
diff --git a/docs/source/api/servers_id_deliveryservices.rst b/docs/source/api/servers_id_deliveryservices.rst
index 628e8ce..4157cf8 100644
--- a/docs/source/api/servers_id_deliveryservices.rst
+++ b/docs/source/api/servers_id_deliveryservices.rst
@@ -98,8 +98,8 @@ Response Structure
:httpBypassFqdn: The HTTP destination to use for bypass on an HTTP :term:`Delivery Service` - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`
:id: An integral, unique identifier for this :term:`Delivery Service`
:infoUrl: This is a string which is expected to contain at least one URL pointing to more information about the :term:`Delivery Service`. Historically, this has been used to link relevant JIRA tickets
-:initialDispersion: The number of :term:`cache server`\ s between which traffic requesting the same object will be randomly split - meaning that if 4 clients all request the same object (one after another), then if this is above 4 there is a possibility that all 4 are cache misses. For most use-cases, this should be 1\ [1]_
-:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable Edge-tier :term:`cache server`; if ``false`` all addresses will be IPv4, regardless of the client connection\ [1]_
+:initialDispersion: The number of :term:`cache server`\ s between which traffic requesting the same object will be randomly split - meaning that if 4 clients all request the same object (one after another), then if this is above 4 there is a possibility that all 4 are cache misses. For most use-cases, this should be 1\ [#httpOnly]_
+:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable :term:`Edge-tier cache server`; if ``false`` all addresses will be IPv4, regardless of the client connection
:lastUpdated: The date and time at which this :term:`Delivery Service` was last updated, in a :manpage:`ctime(3)`-like format
:logsEnabled: If ``true``, logging is enabled for this :term:`Delivery Service`, otherwise it is disabled
:longDesc: A description of the :term:`Delivery Service`
@@ -112,13 +112,13 @@ Response Structure
:type: The :term:`Type` of match performed using ``pattern`` to determine whether or not to use this :term:`Delivery Service`
HOST_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request\ [1]_
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request, or the name requested for resolution in a DNS request
HEADER_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [1]_
+ Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [#httpOnly]_
PATH_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL
+ Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL\ [#httpOnly]_
STEERING_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Service`\ s
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Services`
:maxDnsAnswers: The maximum number of IPs to put in responses to A/AAAA DNS record requests (0 means all available)\ [3]_
:midHeaderRewrite: Rewrite operations to be performed on TCP headers at the Edge-tier cache level - used by the Header Rewrite :abbr:`ATS (Apache Traffic Server)` plugin
@@ -133,7 +133,7 @@ Response Structure
:profileDescription: The description of the Traffic Router :term:`Profile` with which this :term:`Delivery Service` is associated
:profileId: The integral, unique identifier for the Traffic Router :term:`Profile` with which this :term:`Delivery Service` is associated
:profileName: The name of the Traffic Router :term:`Profile` with which this :term:`Delivery Service` is associated
-:protocol: The protocol which clients will use to communicate with Edge-tier :term:`cache server`\ s\ [1]_ - this is an integer on the interval [0-2] where the values have these meanings:
+:protocol: The protocol which clients will use to communicate with Edge-tier :term:`cache server`\ s\ [#httpOnly]_ - this is an integer on the interval [0-2] where the values have these meanings:
0
HTTP
@@ -189,8 +189,8 @@ Response Structure
.. warning:: This number will not be correct if keys are manually replaced using the API, as the key generation API does not increment it!
:tenantId: The integral, unique identifier of the :term:`Tenant` who owns this :term:`Delivery Service`
-:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [1]_
-:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [1]_
+:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
+:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
:type: The name of the routing type of this :term:`Delivery Service` e.g. "HTTP"
:typeId: The integral, unique identifier of the routing type of this :term:`Delivery Service`
:xmlId: A unique string that describes this :term:`Delivery Service` - exists for legacy reasons, but is used heavily by Traffic Control components
@@ -269,7 +269,7 @@ Response Structure
}
]}
-.. [1] This only applies to HTTP-routed :term:`Delivery Service`\ s
+.. [#httpOnly] This only applies to HTTP-:ref:`routed <ds-types>` :term:`Delivery Services`
.. [2] See :ref:`ds-multi-site-origin`
-.. [3] This only applies to DNS-routed :term:`Delivery Service`\ s
-.. [4] These fields are required for HTTP-routed and DNS-routed :term:`Delivery Service`\ s, but are optional for (and in fact may have no effect on) STEERING and ANY_MAP :term:`Delivery Service`\ s
+.. [3] This only applies to DNS-routed :term:`Delivery Services`
+.. [4] These fields are required for HTTP-routed and DNS-routed :term:`Delivery Services`, but are optional for (and in fact may have no effect on) STEERING and ANY_MAP :term:`Delivery Services`
diff --git a/docs/source/api/users_id_deliveryservices.rst b/docs/source/api/users_id_deliveryservices.rst
index 2bb5882..9934b31 100644
--- a/docs/source/api/users_id_deliveryservices.rst
+++ b/docs/source/api/users_id_deliveryservices.rst
@@ -55,10 +55,19 @@ Response Structure
.. deprecated:: ATCv3.0
This field has been deprecated in Traffic Control 3.x and is subject to removal in Traffic Control 4.x or later
-:ccrDnsTtl: The Time To Live (TTL) of the DNS response for A or AAAA record queries requesting the IP address of the Traffic Router - named "ccrDnsTtl" for legacy reasons
-:cdnId: The integral, unique identifier of the CDN to which the :term:`Delivery Service` belongs
-:cdnName: Name of the CDN to which the :term:`Delivery Service` belongs
-:checkPath: The path portion of the URL to check connections to this :term:`Delivery Service`'s origin server
+:ccrDnsTtl: The Time To Live (TTL) of the DNS response for A or AAAA record queries requesting the IP address of the Traffic Router - named "ccrDnsTtl" for legacy reasons
+:cdnId: The integral, unique identifier of the CDN to which the :term:`Delivery Service` belongs
+:cdnName: Name of the CDN to which the :term:`Delivery Service` belongs
+:checkPath: The path portion of the URL to check connections to this :term:`Delivery Service`'s origin server
+:consistentHashRegex: If defined, this is a regular expression used for the Pattern-Based Consistent Hashing feature.\ [#httpOnly]_
+
+ .. versionadded:: 1.4
+
+:consistentHashQueryParams: A set (actually array due to limitations of JSON) of query parameters which will be considered by Traffic Router when using a client request to consistently find an :term:`Edge-tier cache server` to which to redirect them.\ [#httpOnly]_
+
+ .. versionadded:: 1.4
+
+
:displayName: The display name of the :term:`Delivery Service`
:dnsBypassCname: Domain name to overflow requests for HTTP :term:`Delivery Service`\ s - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`\ [4]_
:dnsBypassIp: The IPv4 IP to use for bypass on a DNS :term:`Delivery Service` - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`\ [4]_
@@ -92,8 +101,8 @@ Response Structure
:httpBypassFqdn: The HTTP destination to use for bypass on an HTTP :term:`Delivery Service` - bypass starts when the traffic on this :term:`Delivery Service` exceeds ``globalMaxMbps``, or when more than ``globalMaxTps`` is being exceeded within the :term:`Delivery Service`
:id: An integral, unique identifier for this :term:`Delivery Service`
:infoUrl: This is a string which is expected to contain at least one URL pointing to more information about the :term:`Delivery Service`. Historically, this has been used to link relevant JIRA tickets
-:initialDispersion: The number of caches between which traffic requesting the same object will be randomly split - meaning that if 4 clients all request the same object (one after another), then if this is above 4 there is a possibility that all 4 are cache misses. For most use-cases, this should be 1\ [2]_
-:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable Edge-tier cache; if ``false`` all addresses will be IPv4, regardless of the client connection\ [2]_
+:initialDispersion: The number of caches between which traffic requesting the same object will be randomly split - meaning that if 4 clients all request the same object (one after another), then if this is above 4 there is a possibility that all 4 are cache misses. For most use-cases, this should be 1\ [#httpOnly]_
+:ipv6RoutingEnabled: If ``true``, clients that connect to Traffic Router using IPv6 will be given the IPv6 address of a suitable :term:`Edge-tier cache server`; if ``false`` all addresses will be IPv4, regardless of the client connection
:lastUpdated: The date and time at which this :term:`Delivery Service` was last updated, in a ``ctime``-like format
:logsEnabled: If ``true``, logging is enabled for this :term:`Delivery Service`, otherwise it is disabled
:longDesc: A description of the :term:`Delivery Service`
@@ -106,13 +115,13 @@ Response Structure
:type: The type of match performed using ``pattern`` to determine whether or not to use this :term:`Delivery Service`
HOST_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request\ [2]_
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``Host:`` HTTP header of an HTTP request, or the name requested for resolution in a DNS request
HEADER_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [2]_
+ Use the :term:`Delivery Service` if ``pattern`` matches an HTTP header (both the name and value) in an HTTP request\ [#httpOnly]_
PATH_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL
+ Use the :term:`Delivery Service` if ``pattern`` matches the request path of this :term:`Delivery Service`'s URL\ [#httpOnly]_
STEERING_REGEXP
- Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Service`\ s
+ Use the :term:`Delivery Service` if ``pattern`` matches the ``xml_id`` of one of this :term:`Delivery Service`'s "Steering" target :term:`Delivery Services`
:maxDnsAnswers: The maximum number of IPs to put in responses to A/AAAA DNS record requests (0 means all available)\ [4]_
:midHeaderRewrite: Rewrite operations to be performed on TCP headers at the Edge-tier cache level - used by the Header Rewrite Apache Trafficserver plugin
@@ -127,7 +136,7 @@ Response Structure
:profileDescription: The description of the Traffic Router Profile with which this :term:`Delivery Service` is associated
:profileId: The integral, unique identifier for the Traffic Router profile with which this :term:`Delivery Service` is associated
:profileName: The name of the Traffic Router Profile with which this :term:`Delivery Service` is associated
-:protocol: The protocol which clients will use to communicate with Edge-tier :term:`cache server`\ s\ [2]_ - this is an integer on the interval [0-2] where the values have these meanings:
+:protocol: The protocol which clients will use to communicate with Edge-tier :term:`cache server`\ s\ [#httpOnly]_ - this is an integer on the interval [0-2] where the values have these meanings:
0
HTTP
@@ -180,8 +189,8 @@ Response Structure
.. warning:: This number will not be correct if keys are manually replaced using the API, as the key generation API does not increment it!
:tenantId: The integral, unique identifier of the tenant who owns this :term:`Delivery Service`
-:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [2]_
-:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [2]_
+:trRequestHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router access logs for requests - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
+:trResponseHeaders: If defined, this takes the form of a string of HTTP headers to be included in Traffic Router responses - it's a template where ``__RETURN__`` translates to a carriage return and line feed (``\r\n``)\ [#httpOnly]_
:type: The name of the routing type of this :term:`Delivery Service` e.g. "HTTP"
:typeId: The integral, unique identifier of the routing type of this :term:`Delivery Service`
:xmlId: A unique string that describes this :term:`Delivery Service` - exists for legacy reasons
@@ -262,7 +271,7 @@ Response Structure
"tenant": "root"
}]}
-.. [1] Users with the roles "admin" and/or "operations" will be able to see *all* :term:`Delivery Service`\ s, whereas any other user will only see the :term:`Delivery Service`\ s their Tenant is allowed to see.
-.. [2] This only applies to HTTP-routed :term:`Delivery Service`\ s
+.. [1] Users with the :term:`Roles` "admin" and/or "operations" will be able to see *all* :term:`Delivery Services`, whereas any other user will only see the :term:`Delivery Services` their :term:`Tenant` is allowed to see.
+.. [#httpOnly] This only applies to HTTP-:ref:`routed <ds-types>` :term:`Delivery Services`
.. [3] See :ref:`ds-multi-site-origin`
-.. [4] This only applies to DNS-routed :term:`Delivery Service`\ s
+.. [4] This only applies to DNS-:ref:`routed <ds-types>` :term:`Delivery Services`
diff --git a/docs/source/glossary.rst b/docs/source/glossary.rst
index 4248964..0ed5e38 100644
--- a/docs/source/glossary.rst
+++ b/docs/source/glossary.rst
@@ -57,9 +57,6 @@ Glossary
An example CDN that shows the hierarchy between four Edge-tier :dfn:`Cache Groups`, two Mid-tier :dfn:`Cache Groups`, and one Origin
- consistent hashing
- See `the Wikipedia article <http://en.wikipedia.org/wiki/Consistent_hashing>`_; Traffic Control uses consistent hashing when using :ref:`http-cr` for the edge tier and when selecting parents in the mid tier.
-
content routing
Directing clients (or client systems) to a particular location or device in a location for optimal delivery of content See also :ref:`http-cr` and :ref:`dns-cr`.
@@ -87,6 +84,7 @@ Glossary
.. seealso:: See :ref:`delivery-services` for a more in-depth explanation of :dfn:`Delivery Services`.
Division
+ Divisions
A group of :term:`Region`\ s.
Edge
@@ -106,6 +104,7 @@ Glossary
.. seealso:: Federations are currently only manageable by directly using the :ref:`to-api`. The endpoints related to federations are :ref:`to-api-federations`, :ref:`to-api-federation_resolvers`, :ref:`to-api-federation_resolvers-id`, :ref:`to-api-federations-id-deliveryservices`, :ref:`to-api-federations-id-deliveryservices-id`, :ref:`to-api-federations-id-federation_resolvers`, :ref:`to-api-federations-id-users`, and :ref:`to-api-federations-id-users-id`.
forward proxy
+ forward proxies
A forward proxy acts on behalf of the client such that the :term:`origin server` is (potentially) unaware of the proxy's existence. All Mid-tier :term:`cache server`\ s in a Traffic Control based CDN are :dfn:`forward proxies`. In a :dfn:`forward proxy` scenario, the client is explicitly configured to use the the proxy's IP address and port as a :dfn:`forward proxy`. The client always connects to the :dfn:`forward proxy` for content. The content provider does not have to change the URL [...]
.. seealso:: `ATS documentation on forward proxy <https://docs.trafficserver.apache.org/en/latest/admin/forward-proxy.en.html>`_.
@@ -221,9 +220,11 @@ Glossary
The :dfn:`parent(s)` of a :term:`cache server` is/are the :term:`cache server`\ (s) belonging to either the "parent" or "secondary parent" :term:`Cache Group`\ (s) of the :term:`Cache Group` to which the :term:`cache server` belongs. For example, in general it is true that an :term:`Edge-tier cache server` has one or more :dfn:`parents` which are :term:`Mid-tier cache servers`.
Physical Location
+ Physical Locations
A pair of geographic coordinates (latitude and longitude) that is used by :term:`Cache Group`\ s to define their location. This information is used by Traffic Router to route client traffic to the geographically nearest :term:`Cache Group`.
Profile
+ Profiles
A :dfn:`Profile` is, most generally, a group of :term:`Parameter`\ s that will be applied to a server. :dfn:`Profiles` are typically re-used by all Edge-Tier :term:`cache server`\ s within a CDN or :term:`Cache Group`. A :dfn:`Profile` will, in addition to configuration :term:`Parameter`\ s, define the CDN to which a server belongs and the "Type" of the profile - which determines some behaviors of Traffic Control components. The allowed "Types" of :dfn:`Profiles` are **not** the same a [...]
UNK_PROFILE
@@ -295,9 +296,11 @@ Glossary
.. danger:: Nearly all of these :dfn:`Profile` types have strict naming requirements, and it may be noted that some of said requirements are prefixes ending with ``_``, while others are either not prefixes or do not end with ``_``. This is exactly true; some requirements **need** that ``_`` and some may or may not have it. It is our suggestion, therefore, that for the time being all prefixes use the ``_`` notation to separate words, so as to avoid causing headaches remembering when tha [...]
Region
+ Regions
A group of :term:`Physical Location`\ s.
reverse proxy
+ reverse proxies
A :dfn:`reverse proxy` acts on behalf of the :term:`origin server` such that the client is (potentially) unaware it is not communicating directly with the :term:`origin`. All Edge-tier :term:`cache server`\ s in a Traffic Control CDN are :dfn:`reverse proxies`. To the end user a Traffic Control-based CDN appears as a :dfn:`reverse proxy` since it retrieves content from the :term:`origin server`, acting on behalf of that :term:`origin server`. The client requests a URL that has a hostna [...]
.. seealso:: `The Apache Traffic Server documentation on reverse proxy <https://docs.trafficserver.apache.org/en/latest/admin/reverse-proxy-http-redirects.en.html#http-reverse-proxy>`_.
@@ -381,12 +384,15 @@ Glossary
<!DOCTYPE html><html><body>This is a fun file</body></html>
Role
+ Roles
Permissions :dfn:`Roles` define the operations a user is allowed to perform, and are currently an ordered list of permission levels.
Snapshot
+ Snapshots
Previously called a "CRConfig" or "CRConfig.json" (and still called such in many places), this is a rather large set of routing information generated from a CDN's configuration and topology.
Status
+ Statuses
A :dfn:`Status` represents the current operating state of a server. The default :dfn:`Statuses` made available on initial startup of Traffic Ops are related to the :ref:`health-proto` and are explained in that section.
Tenant
diff --git a/docs/source/overview/delivery_services.rst b/docs/source/overview/delivery_services.rst
index 9fc8613..653e1e4 100644
--- a/docs/source/overview/delivery_services.rst
+++ b/docs/source/overview/delivery_services.rst
@@ -62,6 +62,38 @@ Check Path
----------
A request path on the :term:`origin server` which is used to by certain :ref:`Traffic Ops Extensions <admin-to-ext-script>` to indicate the "health" of the :term:`origin`.
+.. _ds-consistent-hashing-regex:
+
+Consistent Hashing Regular Expression
+-------------------------------------
+When Traffic Router performs :ref:`consistent-hashing` on a client request to find an :term:`Edge-tier cache server` to which to redirect them, it can optionally first modify the request path by extracting the pieces that match this regular expression.
+
+.. seealso:: :ref:`pattern-based-consistenthash`
+
+.. table:: Aliases
+
+ +----------------------------------+---------------------------------------------------------+----------------------------------------------------------------------------------------------------+
+ | Name | Use(s) | Type(s) |
+ +==================================+=========================================================+====================================================================================================+
+ | consistentHashRegex | In source code and :ref:`to-api` requests and responses | unchanged (regular expression) |
+ +----------------------------------+---------------------------------------------------------+----------------------------------------------------------------------------------------------------+
+ | pattern-based consistent hashing | documentation and the Traffic Portal UI | unchanged (regular expression), but usually used when discussing the concept rather than the field |
+ +----------------------------------+---------------------------------------------------------+----------------------------------------------------------------------------------------------------+
+
+.. _ds-consistent-hashing-qparams:
+
+Consistent Hashing Query Parameters
+-----------------------------------
+When Traffic Router performs :ref:`consistent-hashing` on a client request to find an :term:`Edge-tier cache server` to which to redirect them, it can optionally take into account any number of query parameters. This field defines them, formally as a Set but often represented as an Array/List due to encoding limitations. That is, if the Consistent Hashing Query Parameters on a Delivery Service are ``{test}`` and a client makes a request for ``/?test=something`` they will be directed to a [...]
+
+.. table:: Aliases
+
+ +---------------------------+--------------------------------------------------------------------------+------------------------------------------------------------------------------------------------+
+ | Name | Use(s) | Type(s) |
+ +===========================+==========================================================================+================================================================================================+
+ | consistentHashQueryParams | In source code, Traffic Portal, and :ref:`to-api` requests and responses | unchanged (Array of strings - should ALWAYS be unique, thus treated as a Set in most contexts) |
+ +---------------------------+--------------------------------------------------------------------------+------------------------------------------------------------------------------------------------+
+
.. _ds-deep-caching:
Deep Caching
@@ -292,6 +324,8 @@ Info URL
--------
This should be a URL (though neither the :ref:`to-api` nor the Traffic Ops Database in any way enforce the validity of said URL) to which administrators or others may refer for further information regarding a Delivery Service - e.g. a related JIRA ticket.
+.. _ds-initial-dispersion:
+
Initial Dispersion
------------------
The number of :term:`Edge-tier cache servers` across which a particular asset will be distributed within each :term:`Cache Group`. For most use-cases, this should be 1, meaning that all clients requesting a particular asset will be directed to 1 :term:`cache server` per :term:`Cache Group`. Depending on the popularity and size of assets, consider increasing this number in order to spread the request load across more than 1 :term:`cache server`. The larger this number, the more copies of [...]
diff --git a/infrastructure/cdn-in-a-box/traffic_ops/to-access.sh b/infrastructure/cdn-in-a-box/traffic_ops/to-access.sh
index e98e085..7e3b2ce 100644
--- a/infrastructure/cdn-in-a-box/traffic_ops/to-access.sh
+++ b/infrastructure/cdn-in-a-box/traffic_ops/to-access.sh
@@ -75,7 +75,7 @@ to-auth() {
# if cookiejar is current, nothing to do..
cookie_current $COOKIEJAR && return
- local url=$TO_URL/api/1.3/user/login
+ local url=$TO_URL/api/1.4/user/login
local datatype='Accept: application/json'
cat >"$login" <<-CREDS
{ "u" : "$TO_USER", "p" : "$TO_PASSWORD" }
@@ -89,7 +89,7 @@ CREDS
to-ping() {
# ping endpoint does not require authentication
- curl $CURLAUTH $CURLOPTS -X GET "$TO_URL/api/1.3/ping"
+ curl $CURLAUTH $CURLOPTS -X GET "$TO_URL/api/1.4/ping"
}
to-get() {
diff --git a/infrastructure/cdn-in-a-box/traffic_ops_data/deliveryservices/010-ciab.json b/infrastructure/cdn-in-a-box/traffic_ops_data/deliveryservices/010-ciab.json
index 5b1b5d7..320706d 100644
--- a/infrastructure/cdn-in-a-box/traffic_ops_data/deliveryservices/010-ciab.json
+++ b/infrastructure/cdn-in-a-box/traffic_ops_data/deliveryservices/010-ciab.json
@@ -22,5 +22,11 @@
"regionalGeoBlocking": false,
"routingName": "video",
"anonymousBlockingEnabled": false,
- "signingAlgorithm": null
+ "signingAlgorithm": null,
+ "consistentHashQueryParams": [
+ "zyx",
+ "xxx",
+ "pdq",
+ "abc"
+ ]
}
diff --git a/infrastructure/cdn-in-a-box/traffic_ops_integration_test/run.sh b/infrastructure/cdn-in-a-box/traffic_ops_integration_test/run.sh
index 1ac4874..fe7a443 100755
--- a/infrastructure/cdn-in-a-box/traffic_ops_integration_test/run.sh
+++ b/infrastructure/cdn-in-a-box/traffic_ops_integration_test/run.sh
@@ -18,6 +18,7 @@
# Check that env vars are set
envvars=( DB_SERVER DB_PORT DB_ROOT_PASS DB_USER DB_USER_PASS ADMIN_USER ADMIN_PASS)
+set -ex
for v in $envvars
do
if [[ -z $$v ]]; then echo "$v is unset"; exit 1; fi
diff --git a/lib/go-tc/crconfig.go b/lib/go-tc/crconfig.go
index 7458c00..e0080f8 100644
--- a/lib/go-tc/crconfig.go
+++ b/lib/go-tc/crconfig.go
@@ -101,30 +101,31 @@ type CRConfigTrafficOpsServer struct {
//TODO: drichardson - reconcile this with the DeliveryService struct in deliveryservices.go
type CRConfigDeliveryService struct {
- AnonymousBlockingEnabled *string `json:"anonymousBlockingEnabled,omitempty"`
- ConsistentHashRegex *string `json:"consistentHashRegex,omitempty"`
- CoverageZoneOnly bool `json:"coverageZoneOnly,string"`
- Dispersion *CRConfigDispersion `json:"dispersion,omitempty"`
- Domains []string `json:"domains,omitempty"`
- GeoLocationProvider *string `json:"geolocationProvider,omitempty"`
- MatchSets []*MatchSet `json:"matchsets,omitempty"`
- MissLocation *CRConfigLatitudeLongitudeShort `json:"missLocation,omitempty"`
- Protocol *CRConfigDeliveryServiceProtocol `json:"protocol,omitempty"`
- RegionalGeoBlocking *string `json:"regionalGeoBlocking,omitempty"`
- ResponseHeaders map[string]string `json:"responseHeaders,omitempty"`
- RequestHeaders []string `json:"requestHeaders,omitempty"`
- Soa *SOA `json:"soa,omitempty"`
- SSLEnabled bool `json:"sslEnabled,string"`
- TTL *int `json:"ttl,omitempty"`
- TTLs *CRConfigTTL `json:"ttls,omitempty"`
- MaxDNSIPsForLocation *int `json:"maxDnsIpsForLocation,omitempty"`
- IP6RoutingEnabled *bool `json:"ip6RoutingEnabled,string,omitempty"`
- RoutingName *string `json:"routingName,omitempty"`
- BypassDestination map[string]*CRConfigBypassDestination `json:"bypassDestination,omitempty"`
- DeepCachingType *DeepCachingType `json:"deepCachingType"`
- GeoEnabled []CRConfigGeoEnabled `json:"geoEnabled,omitempty"`
- GeoLimitRedirectURL *string `json:"geoLimitRedirectURL,omitempty"`
- StaticDNSEntries []CRConfigStaticDNSEntry `json:"staticDnsEntries,omitempty"`
+ AnonymousBlockingEnabled *string `json:"anonymousBlockingEnabled,omitempty"`
+ ConsistentHashQueryParams []string `json:"consistentHashQueryParams,omitempty"`
+ ConsistentHashRegex *string `json:"consistentHashRegex,omitempty"`
+ CoverageZoneOnly bool `json:"coverageZoneOnly,string"`
+ Dispersion *CRConfigDispersion `json:"dispersion,omitempty"`
+ Domains []string `json:"domains,omitempty"`
+ GeoLocationProvider *string `json:"geolocationProvider,omitempty"`
+ MatchSets []*MatchSet `json:"matchsets,omitempty"`
+ MissLocation *CRConfigLatitudeLongitudeShort `json:"missLocation,omitempty"`
+ Protocol *CRConfigDeliveryServiceProtocol `json:"protocol,omitempty"`
+ RegionalGeoBlocking *string `json:"regionalGeoBlocking,omitempty"`
+ ResponseHeaders map[string]string `json:"responseHeaders,omitempty"`
+ RequestHeaders []string `json:"requestHeaders,omitempty"`
+ Soa *SOA `json:"soa,omitempty"`
+ SSLEnabled bool `json:"sslEnabled,string"`
+ TTL *int `json:"ttl,omitempty"`
+ TTLs *CRConfigTTL `json:"ttls,omitempty"`
+ MaxDNSIPsForLocation *int `json:"maxDnsIpsForLocation,omitempty"`
+ IP6RoutingEnabled *bool `json:"ip6RoutingEnabled,string,omitempty"`
+ RoutingName *string `json:"routingName,omitempty"`
+ BypassDestination map[string]*CRConfigBypassDestination `json:"bypassDestination,omitempty"`
+ DeepCachingType *DeepCachingType `json:"deepCachingType"`
+ GeoEnabled []CRConfigGeoEnabled `json:"geoEnabled,omitempty"`
+ GeoLimitRedirectURL *string `json:"geoLimitRedirectURL,omitempty"`
+ StaticDNSEntries []CRConfigStaticDNSEntry `json:"staticDnsEntries,omitempty"`
}
type CRConfigGeoEnabled struct {
diff --git a/lib/go-tc/deliveryservices.go b/lib/go-tc/deliveryservices.go
index 128bda3..2dcf0cf 100644
--- a/lib/go-tc/deliveryservices.go
+++ b/lib/go-tc/deliveryservices.go
@@ -74,7 +74,8 @@ type DeleteDeliveryServiceResponse struct {
type DeliveryService struct {
DeliveryServiceV13
- MaxOriginConnections int `json:"maxOriginConnections" db:"max_origin_connections"`
+ MaxOriginConnections int `json:"maxOriginConnections" db:"max_origin_connections"`
+ ConsistentHashQueryParams []string `json:"consistentHashQueryParams"`
}
type DeliveryServiceV13 struct {
@@ -148,8 +149,9 @@ type DeliveryServiceV11 struct {
type DeliveryServiceNullable struct {
DeliveryServiceNullableV13
- ConsistentHashRegex *string `json:"consistentHashRegex"`
- MaxOriginConnections *int `json:"maxOriginConnections" db:"max_origin_connections"`
+ ConsistentHashRegex *string `json:"consistentHashRegex"`
+ ConsistentHashQueryParams []string `json:"consistentHashQueryParams"`
+ MaxOriginConnections *int `json:"maxOriginConnections" db:"max_origin_connections"`
}
type DeliveryServiceNullableV13 struct {
@@ -423,6 +425,32 @@ func (ds *DeliveryServiceNullable) Sanitize() {
*ds.DeepCachingType = DeepCachingTypeFromString(string(*ds.DeepCachingType))
}
+func (ds *DeliveryServiceNullable) validateTypeFields(tx *sql.Tx) error {
+ // Validate the TypeName related fields below
+ err := error(nil)
+
+ typeName, err := ValidateTypeID(tx, ds.TypeID, "deliveryservice")
+ if err != nil {
+ return err
+ }
+
+ errs := validation.Errors{
+ "consistentHashQueryParams": validation.Validate(ds,
+ validation.By(func(dsi interface{}) error {
+ ds := dsi.(*DeliveryServiceNullable)
+ if len(ds.ConsistentHashQueryParams) == 0 || DSType(typeName).IsHTTP() {
+ return nil
+ }
+ return fmt.Errorf("consistentHashQueryParams not allowed for '%s' deliveryservice type", typeName)
+ })),
+ }
+ toErrs := tovalidate.ToErrors(errs)
+ if len(toErrs) > 0 {
+ return errors.New(util.JoinErrsStr(toErrs))
+ }
+ return nil
+}
+
func (ds *DeliveryServiceNullable) Validate(tx *sql.Tx) error {
ds.Sanitize()
neverOrAlways := validation.NewStringRule(tovalidate.IsOneOfStringICase("NEVER", "ALWAYS"),
@@ -433,6 +461,9 @@ func (ds *DeliveryServiceNullable) Validate(tx *sql.Tx) error {
if v12Err := ds.DeliveryServiceNullableV12.Validate(tx); v12Err != nil {
errs = append(errs, v12Err)
}
+ if err := ds.validateTypeFields(tx); err != nil {
+ errs = append(errs, errors.New("type fields: "+err.Error()))
+ }
if len(errs) == 0 {
return nil
}
diff --git a/lib/go-tc/nullable_test.go b/lib/go-tc/nullable_test.go
index bd1f964..9c0fcdf 100644
--- a/lib/go-tc/nullable_test.go
+++ b/lib/go-tc/nullable_test.go
@@ -40,7 +40,6 @@ func TestNullStructs(t *testing.T) {
compareWithNullable(t, DeliveryServiceRequestComment{}, DeliveryServiceRequestCommentNullable{})
compareWithNullable(t, DeliveryServiceRequest{}, DeliveryServiceRequestNullable{})
compareWithNullable(t, DeliveryService{}, DeliveryServiceNullable{})
- compareWithNullable(t, DeliveryServiceV12{}, DeliveryServiceNullableV12{})
compareWithNullable(t, DeliveryServiceV11{}, DeliveryServiceNullableV11{})
compareWithNullable(t, Division{}, DivisionNullable{})
compareWithNullable(t, Domain{}, DomainNullable{})
diff --git a/traffic_ops/app/db/migrations/20190513000000_add-allowed_query_keys.sql b/traffic_ops/app/db/migrations/20190513000000_add-allowed_query_keys.sql
new file mode 100644
index 0000000..8388d3e
--- /dev/null
+++ b/traffic_ops/app/db/migrations/20190513000000_add-allowed_query_keys.sql
@@ -0,0 +1,32 @@
+/*
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+-- +goose Up
+-- SQL in section 'Up' is executed when this migration is applied
+
+-- deliveryservice_consistent_hash_query_param
+CREATE TABLE IF NOT EXISTS deliveryservice_consistent_hash_query_param (
+ name TEXT NOT NULL,
+ deliveryservice_id bigint NOT NULL,
+
+ CONSTRAINT name_empty CHECK (length(name) > 0),
+ CONSTRAINT name_reserved CHECK (name NOT IN ('format','trred')),
+ CONSTRAINT fk_deliveryservice FOREIGN KEY (deliveryservice_id) REFERENCES deliveryservice(id) ON DELETE CASCADE,
+ PRIMARY KEY (name, deliveryservice_id)
+);
+
+-- +goose Down
+-- SQL section 'Down' is executed when this migration is rolled back
+DROP TABLE IF EXISTS deliveryservice_consistent_hash_query_param;
diff --git a/traffic_ops/testing/api/v14/deliveryservices_test.go b/traffic_ops/testing/api/v14/deliveryservices_test.go
index 8af81a7..6f73655 100644
--- a/traffic_ops/testing/api/v14/deliveryservices_test.go
+++ b/traffic_ops/testing/api/v14/deliveryservices_test.go
@@ -61,10 +61,21 @@ func GetTestDeliveryServices(t *testing.T) {
for _, ds := range actualDSes {
actualDSMap[ds.XMLID] = ds
}
+ cnt := 0
for _, ds := range testData.DeliveryServices {
if _, ok := actualDSMap[ds.XMLID]; !ok {
t.Errorf("GET DeliveryService missing: %v\n", ds.XMLID)
}
+ // exactly one ds should have exactly 3 query params. the rest should have none
+ if c := len(ds.ConsistentHashQueryParams); c > 0 {
+ if c != 3 {
+ t.Errorf("deliveryservice %s has %d query params; expected %d or %d", ds.XMLID, c, 3, 0)
+ }
+ cnt++
+ }
+ }
+ if cnt > 1 {
+ t.Errorf("exactly 1 deliveryservice should have more than one query param; found %d", cnt)
}
}
diff --git a/traffic_ops/testing/api/v14/tc-fixtures.json b/traffic_ops/testing/api/v14/tc-fixtures.json
index bd8305a..b76a8f4 100644
--- a/traffic_ops/testing/api/v14/tc-fixtures.json
+++ b/traffic_ops/testing/api/v14/tc-fixtures.json
@@ -235,6 +235,7 @@
"ccrDnsTtl": 3600,
"cdnName": "cdn1",
"checkPath": "",
+ "consistentHashQueryParams": [],
"deepCachingType": "NEVER",
"displayName": "ds1DisplayName",
"dnsBypassCname": null,
@@ -302,6 +303,7 @@
"ccrDnsTtl": 3600,
"cdnName": "cdn1",
"checkPath": "",
+ "consistentHashQueryParams": ["fmt", "limit", "somethingelse"],
"deepCachingType": "NEVER",
"displayName": "d s 1",
"dnsBypassCname": null,
@@ -370,6 +372,7 @@
"ccrDnsTtl": 3600,
"cdnName": "cdn1",
"checkPath": "",
+ "consistentHashQueryParams": null,
"deepCachingType": "NEVER",
"displayName": "d s 1",
"dnsBypassCname": null,
diff --git a/traffic_ops/traffic_ops_golang/api/api.go b/traffic_ops/traffic_ops_golang/api/api.go
index 2251f13..03577e2 100644
--- a/traffic_ops/traffic_ops_golang/api/api.go
+++ b/traffic_ops/traffic_ops_golang/api/api.go
@@ -131,6 +131,7 @@ func handleSimpleErr(w http.ResponseWriter, r *http.Request, statusCode int, use
w.Write([]byte(http.StatusText(http.StatusInternalServerError)))
return
}
+ log.Debugln(userErr.Error())
*r = *r.WithContext(context.WithValue(r.Context(), tc.StatusKey, statusCode))
w.Header().Set(tc.ContentType, tc.ApplicationJson)
w.Write(respBts)
@@ -490,6 +491,16 @@ func parseNotNullConstraint(err *pq.Error) (error, error, int) {
return fmt.Errorf("%s is a required field", toCamelCase(match[1])), nil, http.StatusBadRequest
}
+// parses pq errors for empty string check constraint
+func parseEmptyConstraint(err *pq.Error) (error, error, int) {
+ pattern := regexp.MustCompile(`new row for relation "[^"]*" violates check constraint "(.*)_empty"`)
+ match := pattern.FindStringSubmatch(err.Message)
+ if match == nil {
+ return nil, nil, http.StatusOK
+ }
+ return fmt.Errorf("%s cannot be ", match[1]), nil, http.StatusBadRequest
+}
+
// parses pq errors for violated foreign key constraints
func parseNotPresentFKConstraint(err *pq.Error) (error, error, int) {
pattern := regexp.MustCompile(`Key \(.+\)=\(.+\) is not present in table "(.+)"`)
@@ -552,10 +563,6 @@ func ParseDBError(ierr error) (error, error, int) {
return nil, ierr, http.StatusInternalServerError
}
- if usrErr, sysErr, errCode := parseNotNullConstraint(err); errCode != http.StatusOK {
- return usrErr, sysErr, errCode
- }
-
if usrErr, sysErr, errCode := parseNotPresentFKConstraint(err); errCode != http.StatusOK {
return usrErr, sysErr, errCode
}
@@ -568,6 +575,14 @@ func ParseDBError(ierr error) (error, error, int) {
return usrErr, sysErr, errCode
}
+ if usrErr, sysErr, errCode := parseNotNullConstraint(err); errCode != http.StatusOK {
+ return usrErr, sysErr, errCode
+ }
+
+ if usrErr, sysErr, errCode := parseEmptyConstraint(err); errCode != http.StatusOK {
+ return usrErr, sysErr, errCode
+ }
+
return nil, err, http.StatusInternalServerError
}
diff --git a/traffic_ops/traffic_ops_golang/crconfig/deliveryservice.go b/traffic_ops/traffic_ops_golang/crconfig/deliveryservice.go
index eb33441..629a75e 100644
--- a/traffic_ops/traffic_ops_golang/crconfig/deliveryservice.go
+++ b/traffic_ops/traffic_ops_golang/crconfig/deliveryservice.go
@@ -29,6 +29,7 @@ import (
"github.com/apache/trafficcontrol/lib/go-log"
"github.com/apache/trafficcontrol/lib/go-tc"
+ "github.com/lib/pq"
)
const CDNSOAMinimum = 30 * time.Second
@@ -91,12 +92,41 @@ func makeDSes(cdn string, domain string, tx *sql.Tx) (map[string]tc.CRConfigDeli
}
q := `
-select d.xml_id, d.miss_lat, d.miss_long, d.protocol, d.ccr_dns_ttl as ttl, d.routing_name, d.geo_provider, t.name as type, d.geo_limit, d.geo_limit_countries, d.geolimit_redirect_url, d.initial_dispersion, d.regional_geo_blocking, d.tr_response_headers, d.max_dns_answers, p.name as profile, d.dns_bypass_ip, d.dns_bypass_ip6, d.dns_bypass_ttl, d.dns_bypass_cname, d.http_bypass_fqdn, d.ipv6_routing_enabled, d.deep_caching_type, d.tr_request_headers, d.tr_response_headers, d.anonymous_bloc [...]
-from deliveryservice as d
-inner join type as t on t.id = d.type
-left outer join profile as p on p.id = d.profile
-where d.cdn_id = (select id from cdn where name = $1)
-and d.active = true
+SELECT d.xml_id,
+ d.miss_lat,
+ d.miss_long,
+ d.protocol,
+ d.ccr_dns_ttl AS ttl,
+ d.routing_name,
+ d.geo_provider,
+ t.name AS type,
+ d.geo_limit,
+ d.geo_limit_countries,
+ d.geolimit_redirect_url,
+ d.initial_dispersion,
+ d.regional_geo_blocking,
+ d.tr_response_headers,
+ d.max_dns_answers,
+ p.name AS profile,
+ d.dns_bypass_ip,
+ d.dns_bypass_ip6,
+ d.dns_bypass_ttl,
+ d.dns_bypass_cname,
+ d.http_bypass_fqdn,
+ d.ipv6_routing_enabled,
+ d.deep_caching_type,
+ d.tr_request_headers,
+ d.tr_response_headers,
+ d.anonymous_blocking_enabled,
+ d.consistent_hash_regex,
+ (SELECT ARRAY_AGG(name ORDER BY name)
+ FROM deliveryservice_consistent_hash_query_param
+ WHERE deliveryservice_id = d.id) AS query_keys
+FROM deliveryservice AS d
+INNER JOIN type AS t ON t.id = d.type
+LEFT OUTER JOIN profile AS p ON p.id = d.profile
+WHERE d.cdn_id = (select id FROM cdn WHERE name = $1)
+AND d.active = true
`
q += fmt.Sprintf(" and t.name != '%s'", tc.DSTypeAnyMap)
rows, err := tx.Query(q, cdn)
@@ -107,10 +137,11 @@ and d.active = true
for rows.Next() {
ds := tc.CRConfigDeliveryService{
- Protocol: &tc.CRConfigDeliveryServiceProtocol{},
- ResponseHeaders: map[string]string{},
- Soa: cdnSOA,
- TTLs: &tc.CRConfigTTL{},
+ ConsistentHashQueryParams: []string{},
+ Protocol: &tc.CRConfigDeliveryServiceProtocol{},
+ ResponseHeaders: map[string]string{},
+ Soa: cdnSOA,
+ TTLs: &tc.CRConfigTTL{},
}
missLat := sql.NullFloat64{}
@@ -139,9 +170,40 @@ and d.active = true
trResponseHeaders := sql.NullString{}
anonymousBlocking := false
consistentHashRegex := sql.NullString{}
- if err := rows.Scan(&xmlID, &missLat, &missLon, &protocol, &ds.TTL, &ds.RoutingName, &geoProvider, &ttype, &geoLimit, &geoLimitCountries, &geoLimitRedirectURL, &dispersion, &geoBlocking, &trRespHdrsStr, &maxDNSAnswers, &profile, &dnsBypassIP, &dnsBypassIP6, &dnsBypassTTL, &dnsBypassCName, &httpBypassFQDN, &ip6RoutingEnabled, &deepCachingType, &trRequestHeaders, &trResponseHeaders, &anonymousBlocking, &consistentHashRegex); err != nil {
+ err := rows.Scan(
+ &xmlID,
+ &missLat,
+ &missLon,
+ &protocol,
+ &ds.TTL,
+ &ds.RoutingName,
+ &geoProvider,
+ &ttype,
+ &geoLimit,
+ &geoLimitCountries,
+ &geoLimitRedirectURL,
+ &dispersion,
+ &geoBlocking,
+ &trRespHdrsStr,
+ &maxDNSAnswers,
+ &profile,
+ &dnsBypassIP,
+ &dnsBypassIP6,
+ &dnsBypassTTL,
+ &dnsBypassCName,
+ &httpBypassFQDN,
+ &ip6RoutingEnabled,
+ &deepCachingType,
+ &trRequestHeaders,
+ &trResponseHeaders,
+ &anonymousBlocking,
+ &consistentHashRegex,
+ pq.Array(&ds.ConsistentHashQueryParams),
+ )
+ if err != nil {
return nil, errors.New("scanning deliveryservice: " + err.Error())
}
+
// TODO prevent (lat XOR lon) in the Tx and UI
if missLat.Valid && missLon.Valid {
ds.MissLocation = &tc.CRConfigLatitudeLongitudeShort{Lat: missLat.Float64, Lon: missLon.Float64}
diff --git a/traffic_ops/traffic_ops_golang/crconfig/deliveryservice_test.go b/traffic_ops/traffic_ops_golang/crconfig/deliveryservice_test.go
index b75f0ba..8fd2c3b 100644
--- a/traffic_ops/traffic_ops_golang/crconfig/deliveryservice_test.go
+++ b/traffic_ops/traffic_ops_golang/crconfig/deliveryservice_test.go
@@ -83,8 +83,9 @@ func randDS() tc.CRConfigDeliveryService {
ttlSOA := "86400"
geoProviderStr := GeoProviderMaxmindStr
return tc.CRConfigDeliveryService{
- AnonymousBlockingEnabled: &falseStrPtr,
- CoverageZoneOnly: false,
+ AnonymousBlockingEnabled: &falseStrPtr,
+ CoverageZoneOnly: false,
+ ConsistentHashQueryParams: []string{},
Dispersion: &tc.CRConfigDispersion{
Limit: 42,
Shuffled: true,
@@ -154,12 +155,67 @@ func ExpectedMakeDSes() map[string]tc.CRConfigDeliveryService {
}
func MockMakeDSes(mock sqlmock.Sqlmock, expected map[string]tc.CRConfigDeliveryService, cdn string) {
- // select d.xml_id, d.miss_lat, d.miss_long, d.protocol, d.ccr_dns_ttl as ttl, d.routing_name, d.geo_provider, t.name as type, d.geo_limit, d.geo_limit_countries, d.geolimit_redirect_url, d.initial_dispersion, d.regional_geo_blocking, d.tr_response_headers, d.max_dns_answers, p.name as profile, d.dns_bypass_ip, d.dns_bypass_ip6, d.dns_bypass_ttl, d.dns_bypass_cname, d.http_bypass_fqdn, d.ipv6_routing_enabled, d.deep_caching_type, d.tr_request_headers, d.tr_response_headers, d.anonymous_ [...]
-
- rows := sqlmock.NewRows([]string{"xml_id", "miss_lat", "miss_long", "protocol", "ttl", "routing_name", "geo_provider", "type", "geo_limit", "geo_limit_countries", "geeo_limit_redirect_url", "initial_dispersion", "regional_geo_blocking", "tr_response_headers", "max_dns_answers", "profile", "dns_bypass_ip", "dns_bypass_ip6", "dns_bypass_ttl", "dns_bypass_cname", "http_bypass_fqdn", "ipv6_routing_enabled", "deep_caching_type", "tr_request_headers", "tr_response_headers", "anonymous_blockin [...]
+ rows := sqlmock.NewRows([]string{
+ "xml_id",
+ "miss_lat",
+ "miss_long",
+ "protocol",
+ "ttl",
+ "routing_name",
+ "geo_provider",
+ "type",
+ "geo_limit",
+ "geo_limit_countries",
+ "geeo_limit_redirect_url",
+ "initial_dispersion",
+ "regional_geo_blocking",
+ "tr_response_headers",
+ "max_dns_answers",
+ "profile",
+ "dns_bypass_ip",
+ "dns_bypass_ip6",
+ "dns_bypass_ttl",
+ "dns_bypass_cname",
+ "http_bypass_fqdn",
+ "ipv6_routing_enabled",
+ "deep_caching_type",
+ "tr_request_headers",
+ "tr_response_headers",
+ "anonymous_blocking_enabled",
+ "consistent_hash_regex",
+ "query_keys"})
for dsName, ds := range expected {
- rows = rows.AddRow(dsName, ds.MissLocation.Lat, ds.MissLocation.Lon, 0, *ds.TTL, *ds.RoutingName, 0, "HTTP", 0, "", "", 42, false, "", nil, "", "", "", 0, "", *ds.BypassDestination["HTTP"].FQDN, *ds.IP6RoutingEnabled, nil, "", "", false, "")
+ queryParams := "{" + strings.Join(ds.ConsistentHashQueryParams, ",") + "}"
+ rows = rows.AddRow(
+ dsName,
+ ds.MissLocation.Lat,
+ ds.MissLocation.Lon,
+ 0,
+ *ds.TTL,
+ *ds.RoutingName,
+ 0,
+ "HTTP",
+ 0,
+ "",
+ "",
+ 42,
+ false,
+ "",
+ nil,
+ "",
+ "",
+ "",
+ 0,
+ "",
+ *ds.BypassDestination["HTTP"].FQDN,
+ *ds.IP6RoutingEnabled,
+ nil,
+ "",
+ "",
+ false,
+ "",
+ queryParams)
}
mock.ExpectQuery("select").WithArgs(cdn).WillReturnRows(rows)
}
diff --git a/traffic_ops/traffic_ops_golang/deliveryservice/deliveryservices.go b/traffic_ops/traffic_ops_golang/deliveryservice/deliveryservices.go
index 84814ff..ab5c1ef 100644
--- a/traffic_ops/traffic_ops_golang/deliveryservice/deliveryservices.go
+++ b/traffic_ops/traffic_ops_golang/deliveryservice/deliveryservices.go
@@ -116,7 +116,60 @@ func create(inf *api.APIInfo, ds tc.DeliveryServiceNullable) (tc.DeliveryService
deepCachingType = ds.DeepCachingType.String() // necessary, because DeepCachingType's default needs to insert the string, not "", and Query doesn't call .String().
}
- resultRows, err := tx.Query(insertQuery(), &ds.Active, &ds.AnonymousBlockingEnabled, &ds.CacheURL, &ds.CCRDNSTTL, &ds.CDNID, &ds.CheckPath, &ds.ConsistentHashRegex, &deepCachingType, &ds.DisplayName, &ds.DNSBypassCNAME, &ds.DNSBypassIP, &ds.DNSBypassIP6, &ds.DNSBypassTTL, &ds.DSCP, &ds.EdgeHeaderRewrite, &ds.GeoLimitRedirectURL, &ds.GeoLimit, &ds.GeoLimitCountries, &ds.GeoProvider, &ds.GlobalMaxMBPS, &ds.GlobalMaxTPS, &ds.FQPacingRate, &ds.HTTPBypassFQDN, &ds.InfoURL, &ds.InitialDispers [...]
+ resultRows, err := tx.Query(insertQuery(),
+ &ds.Active,
+ &ds.AnonymousBlockingEnabled,
+ &ds.CacheURL,
+ &ds.CCRDNSTTL,
+ &ds.CDNID,
+ &ds.CheckPath,
+ &ds.ConsistentHashRegex,
+ &deepCachingType,
+ &ds.DisplayName,
+ &ds.DNSBypassCNAME,
+ &ds.DNSBypassIP,
+ &ds.DNSBypassIP6,
+ &ds.DNSBypassTTL,
+ &ds.DSCP,
+ &ds.EdgeHeaderRewrite,
+ &ds.GeoLimitRedirectURL,
+ &ds.GeoLimit,
+ &ds.GeoLimitCountries,
+ &ds.GeoProvider,
+ &ds.GlobalMaxMBPS,
+ &ds.GlobalMaxTPS,
+ &ds.FQPacingRate,
+ &ds.HTTPBypassFQDN,
+ &ds.InfoURL,
+ &ds.InitialDispersion,
+ &ds.IPV6RoutingEnabled,
+ &ds.LogsEnabled,
+ &ds.LongDesc,
+ &ds.LongDesc1,
+ &ds.LongDesc2,
+ &ds.MaxDNSAnswers,
+ &ds.MaxOriginConnections,
+ &ds.MidHeaderRewrite,
+ &ds.MissLat,
+ &ds.MissLong,
+ &ds.MultiSiteOrigin,
+ &ds.OriginShield,
+ &ds.ProfileID,
+ &ds.Protocol,
+ &ds.QStringIgnore,
+ &ds.RangeRequestHandling,
+ &ds.RegexRemap,
+ &ds.RegionalGeoBlocking,
+ &ds.RemapText,
+ &ds.RoutingName,
+ &ds.SigningAlgorithm,
+ &ds.SSLKeyVersion,
+ &ds.TenantID,
+ &ds.TRRequestHeaders,
+ &ds.TRResponseHeaders,
+ &ds.TypeID,
+ &ds.XMLID)
+
if err != nil {
usrErr, sysErr, code := api.ParseDBError(err)
return tc.DeliveryServiceNullable{}, code, usrErr, sysErr
@@ -155,6 +208,13 @@ func create(inf *api.APIInfo, ds tc.DeliveryServiceNullable) (tc.DeliveryService
return tc.DeliveryServiceNullable{}, http.StatusInternalServerError, nil, errors.New("creating default regex: " + err.Error())
}
+ if c, err := createConsistentHashQueryParams(tx, *ds.ID, ds.ConsistentHashQueryParams); err != nil {
+ usrErr, sysErr, code := api.ParseDBError(err)
+ return tc.DeliveryServiceNullable{}, code, usrErr, sysErr
+ } else {
+ api.CreateChangeLogRawTx(api.ApiChange, fmt.Sprintf("Created %d consistent hash query params for delivery service: %s", c, *ds.XMLID), user, tx)
+ }
+
matchlists, err := GetDeliveryServicesMatchLists([]string{*ds.XMLID}, tx)
if err != nil {
return tc.DeliveryServiceNullable{}, http.StatusInternalServerError, nil, errors.New("creating DS: reading matchlists: " + err.Error())
@@ -253,6 +313,22 @@ func createDefaultRegex(tx *sql.Tx, dsID int, xmlID string) error {
return nil
}
+func createConsistentHashQueryParams(tx *sql.Tx, dsID int, consistentHashQueryParams []string) (int, error) {
+ if len(consistentHashQueryParams) == 0 {
+ return 0, nil
+ }
+ c := 0
+ q := `INSERT INTO deliveryservice_consistent_hash_query_param (name, deliveryservice_id) VALUES ($1, $2)`
+ for _, k := range consistentHashQueryParams {
+ if _, err := tx.Exec(q, k, dsID); err != nil {
+ return c, err
+ }
+ c++
+ }
+
+ return c, nil
+}
+
func update(inf *api.APIInfo, ds *tc.DeliveryServiceNullable) (tc.DeliveryServiceNullable, int, error, error) {
tx := inf.Tx.Tx
cfg := inf.Config
@@ -294,7 +370,60 @@ func update(inf *api.APIInfo, ds *tc.DeliveryServiceNullable) (tc.DeliveryServic
deepCachingType = ds.DeepCachingType.String() // necessary, because DeepCachingType's default needs to insert the string, not "", and Query doesn't call .String().
}
- resultRows, err := tx.Query(updateDSQuery(), &ds.Active, &ds.CacheURL, &ds.CCRDNSTTL, &ds.CDNID, &ds.CheckPath, &deepCachingType, &ds.DisplayName, &ds.DNSBypassCNAME, &ds.DNSBypassIP, &ds.DNSBypassIP6, &ds.DNSBypassTTL, &ds.DSCP, &ds.EdgeHeaderRewrite, &ds.GeoLimitRedirectURL, &ds.GeoLimit, &ds.GeoLimitCountries, &ds.GeoProvider, &ds.GlobalMaxMBPS, &ds.GlobalMaxTPS, &ds.FQPacingRate, &ds.HTTPBypassFQDN, &ds.InfoURL, &ds.InitialDispersion, &ds.IPV6RoutingEnabled, &ds.LogsEnabled, &ds.Lon [...]
+ resultRows, err := tx.Query(updateDSQuery(),
+ &ds.Active,
+ &ds.CacheURL,
+ &ds.CCRDNSTTL,
+ &ds.CDNID,
+ &ds.CheckPath,
+ &deepCachingType,
+ &ds.DisplayName,
+ &ds.DNSBypassCNAME,
+ &ds.DNSBypassIP,
+ &ds.DNSBypassIP6,
+ &ds.DNSBypassTTL,
+ &ds.DSCP,
+ &ds.EdgeHeaderRewrite,
+ &ds.GeoLimitRedirectURL,
+ &ds.GeoLimit,
+ &ds.GeoLimitCountries,
+ &ds.GeoProvider,
+ &ds.GlobalMaxMBPS,
+ &ds.GlobalMaxTPS,
+ &ds.FQPacingRate,
+ &ds.HTTPBypassFQDN,
+ &ds.InfoURL,
+ &ds.InitialDispersion,
+ &ds.IPV6RoutingEnabled,
+ &ds.LogsEnabled,
+ &ds.LongDesc,
+ &ds.LongDesc1,
+ &ds.LongDesc2,
+ &ds.MaxDNSAnswers,
+ &ds.MidHeaderRewrite,
+ &ds.MissLat,
+ &ds.MissLong,
+ &ds.MultiSiteOrigin,
+ &ds.OriginShield,
+ &ds.ProfileID,
+ &ds.Protocol,
+ &ds.QStringIgnore,
+ &ds.RangeRequestHandling,
+ &ds.RegexRemap,
+ &ds.RegionalGeoBlocking,
+ &ds.RemapText,
+ &ds.RoutingName,
+ &ds.SigningAlgorithm,
+ &ds.SSLKeyVersion,
+ &ds.TenantID,
+ &ds.TRRequestHeaders,
+ &ds.TRResponseHeaders,
+ &ds.TypeID,
+ &ds.XMLID,
+ &ds.AnonymousBlockingEnabled,
+ &ds.ConsistentHashRegex,
+ &ds.MaxOriginConnections,
+ &ds.ID)
if err != nil {
usrErr, sysErr, code := api.ParseDBError(err)
@@ -374,6 +503,21 @@ func update(inf *api.APIInfo, ds *tc.DeliveryServiceNullable) (tc.DeliveryServic
ds.LastUpdated = &lastUpdated
+ // the update may change or delete the query params -- delete existing and re-add if any provided
+ q := `DELETE FROM deliveryservice_consistent_hash_query_param WHERE deliveryservice_id = $1`
+ if res, err := tx.Exec(q, *ds.ID); err != nil {
+ return tc.DeliveryServiceNullable{}, http.StatusInternalServerError, nil, fmt.Errorf("deleting consistent hash query params for ds %s: %s", *ds.XMLID, err.Error())
+ } else if c, _ := res.RowsAffected(); c > 0 {
+ api.CreateChangeLogRawTx(api.ApiChange, fmt.Sprintf("Deleted %d consistent hash query params for delivery service: %s", c, *ds.XMLID), user, tx)
+ }
+
+ if c, err := createConsistentHashQueryParams(tx, *ds.ID, ds.ConsistentHashQueryParams); err != nil {
+ usrErr, sysErr, code := api.ParseDBError(err)
+ return tc.DeliveryServiceNullable{}, code, usrErr, sysErr
+ } else {
+ api.CreateChangeLogRawTx(api.ApiChange, fmt.Sprintf("Created %d consistent hash query params for delivery service: %s", c, *ds.XMLID), user, tx)
+ }
+
if err := api.CreateChangeLogRawErr(api.ApiChange, "Updated ds: "+*ds.XMLID+" id: "+strconv.Itoa(*ds.ID), user, tx); err != nil {
return tc.DeliveryServiceNullable{}, http.StatusInternalServerError, nil, errors.New("writing change log entry: " + err.Error())
}
@@ -549,18 +693,99 @@ func GetDeliveryServices(query string, queryValues map[string]interface{}, tx *s
dses := []tc.DeliveryServiceNullable{}
dsCDNDomains := map[string]string{}
+
+ // ensure json generated from this slice won't come out as `null` if empty
+ dsQueryParams := []string{}
+
for rows.Next() {
ds := tc.DeliveryServiceNullable{}
cdnDomain := ""
- err := rows.Scan(&ds.Active, &ds.AnonymousBlockingEnabled, &ds.CacheURL, &ds.CCRDNSTTL, &ds.CDNID, &ds.CDNName, &ds.CheckPath, &ds.ConsistentHashRegex, &ds.DeepCachingType, &ds.DisplayName, &ds.DNSBypassCNAME, &ds.DNSBypassIP, &ds.DNSBypassIP6, &ds.DNSBypassTTL, &ds.DSCP, &ds.EdgeHeaderRewrite, &ds.GeoLimitRedirectURL, &ds.GeoLimit, &ds.GeoLimitCountries, &ds.GeoProvider, &ds.GlobalMaxMBPS, &ds.GlobalMaxTPS, &ds.FQPacingRate, &ds.HTTPBypassFQDN, &ds.ID, &ds.InfoURL, &ds.InitialDispersi [...]
+ err := rows.Scan(&ds.Active,
+ &ds.AnonymousBlockingEnabled,
+ &ds.CacheURL,
+ &ds.CCRDNSTTL,
+ &ds.CDNID,
+ &ds.CDNName,
+ &ds.CheckPath,
+ &ds.ConsistentHashRegex,
+ &ds.DeepCachingType,
+ &ds.DisplayName,
+ &ds.DNSBypassCNAME,
+ &ds.DNSBypassIP,
+ &ds.DNSBypassIP6,
+ &ds.DNSBypassTTL,
+ &ds.DSCP,
+ &ds.EdgeHeaderRewrite,
+ &ds.GeoLimitRedirectURL,
+ &ds.GeoLimit,
+ &ds.GeoLimitCountries,
+ &ds.GeoProvider,
+ &ds.GlobalMaxMBPS,
+ &ds.GlobalMaxTPS,
+ &ds.FQPacingRate,
+ &ds.HTTPBypassFQDN,
+ &ds.ID,
+ &ds.InfoURL,
+ &ds.InitialDispersion,
+ &ds.IPV6RoutingEnabled,
+ &ds.LastUpdated,
+ &ds.LogsEnabled,
+ &ds.LongDesc,
+ &ds.LongDesc1,
+ &ds.LongDesc2,
+ &ds.MaxDNSAnswers,
+ &ds.MaxOriginConnections,
+ &ds.MidHeaderRewrite,
+ &ds.MissLat,
+ &ds.MissLong,
+ &ds.MultiSiteOrigin,
+ &ds.OrgServerFQDN,
+ &ds.OriginShield,
+ &ds.ProfileID,
+ &ds.ProfileName,
+ &ds.ProfileDesc,
+ &ds.Protocol,
+ &ds.QStringIgnore,
+ pq.Array(&dsQueryParams),
+ &ds.RangeRequestHandling,
+ &ds.RegexRemap,
+ &ds.RegionalGeoBlocking,
+ &ds.RemapText,
+ &ds.RoutingName,
+ &ds.SigningAlgorithm,
+ &ds.SSLKeyVersion,
+ &ds.TenantID,
+ &ds.Tenant,
+ &ds.TRRequestHeaders,
+ &ds.TRResponseHeaders,
+ &ds.Type,
+ &ds.TypeID,
+ &ds.XMLID,
+ &cdnDomain)
+
if err != nil {
return nil, []error{fmt.Errorf("getting delivery services: %v", err)}, tc.SystemError
}
+
+ ds.ConsistentHashQueryParams = []string{}
+ if len(dsQueryParams) >= 0 {
+ // ensure unique and in consistent order
+ m := make(map[string]struct{}, len(dsQueryParams))
+ for _, k := range dsQueryParams {
+ if _, exists := m[k]; exists {
+ continue
+ }
+ m[k] = struct{}{}
+ ds.ConsistentHashQueryParams = append(ds.ConsistentHashQueryParams, k)
+ }
+ }
+
dsCDNDomains[*ds.XMLID] = cdnDomain
if ds.DeepCachingType != nil {
*ds.DeepCachingType = tc.DeepCachingTypeFromString(string(*ds.DeepCachingType))
}
ds.Signed = ds.SigningAlgorithm != nil && *ds.SigningAlgorithm == tc.SigningAlgorithmURLSig
+
dses = append(dses, ds)
}
@@ -1073,6 +1298,9 @@ profile.name as profile_name,
profile.description as profile_description,
ds.protocol,
ds.qstring_ignore,
+(SELECT ARRAY_AGG(name ORDER BY name)
+ FROM deliveryservice_consistent_hash_query_param
+ WHERE deliveryservice_id = ds.id) AS query_keys,
ds.range_request_handling,
ds.regex_remap,
ds.regional_geo_blocking,
diff --git a/traffic_ops/traffic_ops_golang/deliveryservice/deliveryservicesv13_test.go b/traffic_ops/traffic_ops_golang/deliveryservice/deliveryservices_test.go
similarity index 100%
rename from traffic_ops/traffic_ops_golang/deliveryservice/deliveryservicesv13_test.go
rename to traffic_ops/traffic_ops_golang/deliveryservice/deliveryservices_test.go
diff --git a/traffic_portal/app/src/common/api/DeliveryServiceService.js b/traffic_portal/app/src/common/api/DeliveryServiceService.js
index 334d4bd..93cd347 100644
--- a/traffic_portal/app/src/common/api/DeliveryServiceService.js
+++ b/traffic_portal/app/src/common/api/DeliveryServiceService.js
@@ -30,6 +30,9 @@ var DeliveryServiceService = function(Restangular, $http, $q, locationUtils, htt
this.createDeliveryService = function(ds) {
var request = $q.defer();
+ // strip out any falsy values or duplicates from consistentHashQueryParams
+ ds.consistentHashQueryParams = _.compact(Array.from(new Set(ds.consistentHashQueryParams)));
+
$http.post(ENV.api['root'] + "deliveryservices", ds)
.then(
function(response) {
@@ -46,6 +49,9 @@ var DeliveryServiceService = function(Restangular, $http, $q, locationUtils, htt
this.updateDeliveryService = function(ds) {
var request = $q.defer();
+ // strip out any falsy values or duplicates from consistentHashQueryParams
+ ds.consistentHashQueryParams = _.compact(Array.from(new Set(ds.consistentHashQueryParams)));
+
$http.put(ENV.api['root'] + "deliveryservices/" + ds.id, ds)
.then(
function(response) {
diff --git a/traffic_portal/app/src/common/modules/form/_form.scss b/traffic_portal/app/src/common/modules/form/_form.scss
index 523434d..ff9072c 100644
--- a/traffic_portal/app/src/common/modules/form/_form.scss
+++ b/traffic_portal/app/src/common/modules/form/_form.scss
@@ -16,6 +16,30 @@
*/
+@supports not(-moz-animation: auto) {
+ math {
+ mfenced::before {
+ content: "(";
+ }
+ mfenced::after {
+ content: ")";
+ }
+ mfenced[open]::before {
+ content: attr(open);
+ }
+ mfenced[close]::after {
+ content: attr(close);
+ }
+
+ ms::before {
+ content: open-quote;
+ }
+ ms::after {
+ content: close-quote;
+ }
+ }
+}
+
.btn-link.request-status {
font-size: 20px;
color: #cd1323;
@@ -94,6 +118,13 @@ textarea.autosize:read-only {
font-size: medium;
}
+.add-more-inputs {
+ margin-bottom: 0;
+ .btn {
+ margin-right: 0;
+ }
+}
+
@media (min-width: 768px) {
form label.has-tooltip {
div.helptooltip {
@@ -164,3 +195,4 @@ aside.current-value {
margin-top: 10px;
}
}
+
diff --git a/traffic_portal/app/src/common/modules/form/deliveryService/FormDeliveryServiceController.js b/traffic_portal/app/src/common/modules/form/deliveryService/FormDeliveryServiceController.js
index 643487b..0ada148 100644
--- a/traffic_portal/app/src/common/modules/form/deliveryService/FormDeliveryServiceController.js
+++ b/traffic_portal/app/src/common/modules/form/deliveryService/FormDeliveryServiceController.js
@@ -260,6 +260,21 @@ var FormDeliveryServiceController = function(deliveryService, dsCurrent, origin,
}
};
+ $scope.addQueryParam = function() {
+ $scope.deliveryService.consistentHashQueryParams.push('');
+ };
+
+ $scope.removeQueryParam = function(index) {
+ if ($scope.deliveryService.consistentHashQueryParams.length > 1) {
+ $scope.deliveryService.consistentHashQueryParams.splice(index, 1);
+ } else {
+ // if only one query param is left, don't remove the item from the array. instead, just blank it out
+ // so the dynamic form widget will still be visible. empty strings get stripped out on save anyhow.
+ $scope.deliveryService.consistentHashQueryParams[index] = '';
+ }
+ $scope.deliveryServiceForm.$pristine = false; // this enables the 'update' button in the ds form
+ };
+
$scope.viewTargets = function() {
$location.path($location.path() + '/targets');
};
@@ -310,6 +325,10 @@ var FormDeliveryServiceController = function(deliveryService, dsCurrent, origin,
getCDNs();
getProfiles();
getTenants();
+ if (!deliveryService.consistentHashQueryParams || deliveryService.consistentHashQueryParams.length < 1) {
+ // add an empty one so the dynamic form widget is visible. empty strings get stripped out on save anyhow.
+ $scope.deliveryService.consistentHashQueryParams = [ '' ];
+ }
};
init();
diff --git a/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.HTTP.tpl.html b/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.HTTP.tpl.html
index f17d688..06987ab 100644
--- a/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.HTTP.tpl.html
+++ b/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.HTTP.tpl.html
@@ -1022,6 +1022,37 @@ under the License.
</div>
</div>
+ <div>
+ <ng-form name="qpForm" ng-repeat="qp in deliveryService.consistentHashQueryParams track by $index">
+ <div class="form-group" ng-class="{'has-error': hasError(qpForm.consistentHashQueryParam), 'has-feedback': hasError(qpForm.consistentHashQueryParam)}">
+ <div>
+ <label class="has-tooltip control-label col-md-2 col-sm-2 col-xs-12" for="consistentHashQueryParam">Consistent Hash Query Param #{{$index + 1}}<div class="helptooltip">
+ <div class="helptext">
+ When Traffic Router performs a "consistent hash" on a client request to find an Edge-tier cache server to which to redirect them, it can optionally take into account any number of query parameters. This field defines them, formally as a Set but often represented as an Array/List due to encoding limitations. That is, if the Consistent Hashing Query Parameters on a Delivery Service are <math><mfenced open="{" close="}"><ms>test</ms></mfenced></math> [...]
+ </div>
+ </div>
+ </label>
+ <div class="col-md-10 col-sm-10 col-xs-12">
+ <div>
+ <div class="input-group add-more-inputs">
+ <input id="consistentHashQueryParam" name="consistentHashQueryParam" type="text" class="form-control" ng-model="deliveryService.consistentHashQueryParams[$index]" pattern="\S*">
+ <span class="form-input-group-btn input-group-btn">
+ <button id="removeBtn" name="removeBtn" class="btn btn-default" ng-click="removeQueryParam($index)"><i class="fa fa-minus"></i></button>
+ <button id="addBtn" name="addBtn" class="btn btn-default" ng-show="$index == (deliveryService.consistentHashQueryParams.length - 1)" ng-click="addQueryParam()"><i class="fa fa-plus"></i></button>
+ </span>
+ </div>
+ </div>
+ <small class="input-error" ng-show="hasPropertyError(qpForm.consistentHashQueryParam, 'pattern')">No spaces</small>
+ <aside class="current-value" ng-if="settings.isRequest" ng-show="open() && !_.isEqual(deliveryService.consistentHashQueryParams, dsCurrent.consistentHashQueryParams) && $index == (deliveryService.consistentHashQueryParams.length - 1)">
+ <h3>Current Value</h3>
+ <pre ng-repeat="i in dsCurrent.consistentHashQueryParams track by $index">Consistent Hash Query Param #{{$index + 1}}: {{::dsCurrent.consistentHashQueryParams[$index]}}</pre>
+ </aside>
+ </div>
+ </div>
+ </div>
+ </ng-form>
+ </div>
+
</div>
<div class="modal-footer">
diff --git a/traffic_portal/app/src/traffic_portal_properties.json b/traffic_portal/app/src/traffic_portal_properties.json
index 0163fa2..e0b2dc6 100644
--- a/traffic_portal/app/src/traffic_portal_properties.json
+++ b/traffic_portal/app/src/traffic_portal_properties.json
@@ -152,7 +152,8 @@
"missLong": -87.627778,
"signingAlgorithm": null,
"ccrDnsTtl": 3600,
- "anonymousBlockingEnabled": false
+ "anonymousBlockingEnabled": false,
+ "consistentHashQueryParams": []
},
"STEERING": {
"dscp": 0,
diff --git a/traffic_router/.gitignore b/traffic_router/.gitignore
index 224a8d3..448a380 100644
--- a/traffic_router/.gitignore
+++ b/traffic_router/.gitignore
@@ -8,9 +8,13 @@
/rpm/.settings
/connector/target
/connector/.settings
+/configuration/target
/core/src/test/resources/var/auto-zones
/core/src/test/db/cr-config.json
/core/src/test/db/health.json
/core/src/test/db
/core/src/test/conf/traffic_monitor.properties
/core/src/test/conf/traffic_ops.properties
+/geolocation/target
+/shared/target
+
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryService.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryService.java
index ab52945..9450c42 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryService.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryService.java
@@ -18,9 +18,11 @@ package com.comcast.cdn.traffic_control.traffic_router.core.ds;
import java.io.ByteArrayOutputStream;
import java.io.DataOutputStream;
import java.io.IOException;
+import java.io.UnsupportedEncodingException;
import java.net.InetAddress;
import java.net.MalformedURLException;
import java.net.URL;
+import java.net.URLDecoder;
import java.net.UnknownHostException;
import java.security.GeneralSecurityException;
import java.util.ArrayList;
@@ -30,6 +32,8 @@ import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
+import java.util.SortedSet;
+import java.util.TreeSet;
import java.util.Iterator;
import java.util.concurrent.atomic.AtomicInteger;
@@ -51,7 +55,7 @@ import com.comcast.cdn.traffic_control.traffic_router.core.util.JsonUtils;
import com.comcast.cdn.traffic_control.traffic_router.core.util.JsonUtilsException;
import com.comcast.cdn.traffic_control.traffic_router.core.util.StringProtector;
-@SuppressWarnings({"PMD.TooManyFields","PMD.CyclomaticComplexity", "PMD.AvoidDuplicateLiterals"})
+@SuppressWarnings({"PMD.TooManyFields","PMD.CyclomaticComplexity", "PMD.AvoidDuplicateLiterals", "PMD.ExcessivePublicCount"})
public class DeliveryService {
protected static final Logger LOGGER = Logger.getLogger(DeliveryService.class);
private final String id;
@@ -96,6 +100,7 @@ public class DeliveryService {
private final boolean redirectToHttps;
private final DeepCachingType deepCache;
private String consistentHashRegex;
+ private final Set<String> consistentHashQueryParams;
public enum DeepCachingType {
NEVER,
@@ -125,6 +130,21 @@ public class DeliveryService {
this.soa = dsJo.get("soa");
this.shouldAppendQueryString = JsonUtils.optBoolean(dsJo, "appendQueryString", true);
+ this.consistentHashQueryParams = new HashSet<String>();
+ if (dsJo.has("consistentHashQueryParams")) {
+ final JsonNode cqpNode = dsJo.get("consistentHashQueryParams");
+ if (!cqpNode.isArray()) {
+ LOGGER.error("Delivery Service '" + id + "' has malformed consistentHashQueryParams. Disregarding.");
+ } else {
+ for (final JsonNode n : cqpNode) {
+ final String s = n.asText();
+ if (!s.isEmpty()) {
+ this.consistentHashQueryParams.add(s);
+ }
+ }
+ }
+ }
+
// missLocation: {lat: , long: }
final JsonNode mlJo = dsJo.get("missLocation");
if(mlJo != null) {
@@ -165,6 +185,10 @@ public class DeliveryService {
}
}
+ public Set<String> getConsistentHashQueryParams() {
+ return this.consistentHashQueryParams;
+ }
+
public String getId() {
return id;
}
@@ -658,4 +682,52 @@ public class DeliveryService {
public void setConsistentHashRegex(final String consistentHashRegex) {
this.consistentHashRegex = consistentHashRegex;
}
+
+ /**
+ * Extracts the significant parts of a request's query string based on this
+ * Delivery Service's Consistent Hashing Query Parameters
+ * @param r The request from which to extract query parameters
+ * @return The parts of the request's query string relevant to consistent
+ * hashing. The result is URI-decoded - if decoding fails it will return
+ * a blank string instead.
+ */
+ public String extractSignificantQueryParams(final HTTPRequest r) {
+ if (r.getQueryString() == null || r.getQueryString().isEmpty() || this.getConsistentHashQueryParams().isEmpty()) {
+ return "";
+ }
+
+ final SortedSet<String> qparams = new TreeSet<String>();
+ try {
+ for (final String qparam : r.getQueryString().split("&")) {
+ if (qparam.isEmpty()) {
+ continue;
+ }
+
+ String[] parts = qparam.split("=");
+ for (short i = 0; i < parts.length; ++i) {
+ parts[i] = URLDecoder.decode(parts[i], "UTF-8");
+ }
+
+ if (this.getConsistentHashQueryParams().contains(parts[0])) {
+ qparams.add(String.join("=", parts));
+ }
+ }
+
+ } catch (UnsupportedEncodingException e) {
+ final StringBuffer err = new StringBuffer();
+ err.append("Error decoding query parameters - ");
+ err.append(this.toString());
+ err.append(" - Exception: ");
+ err.append(e.toString());
+ LOGGER.error(err.toString());
+ return "";
+ }
+
+ final StringBuilder s = new StringBuilder();
+ for (final String q : qparams) {
+ s.append(q);
+ }
+
+ return s.toString();
+ }
}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java
index e321ffb..4ba3e99 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java
@@ -77,6 +77,7 @@ import com.comcast.cdn.traffic_control.traffic_router.core.router.StatTracker.Tr
import com.comcast.cdn.traffic_control.traffic_router.core.loc.AnonymousIp;
import com.comcast.cdn.traffic_control.traffic_router.core.loc.AnonymousIpDatabaseService;
+@SuppressWarnings("PMD.ExcessivePublicCount")
public class TrafficRouter {
public static final Logger LOGGER = Logger.getLogger(TrafficRouter.class);
public static final String XTC_STEERING_OPTION = "x-tc-steering-option";
@@ -99,8 +100,8 @@ public class TrafficRouter {
private final Map<String, Geolocation> defaultGeolocationsOverride = new HashMap<String, Geolocation>();
- public TrafficRouter(final CacheRegister cr,
- final GeolocationService geolocationService,
+ public TrafficRouter(final CacheRegister cr,
+ final GeolocationService geolocationService,
final GeolocationService geolocationService6,
final AnonymousIpDatabaseService anonymousIpService,
final StatTracker statTracker,
@@ -137,7 +138,7 @@ public class TrafficRouter {
* Returns a {@link List} of all of the online {@link Cache}s that support the specified
* {@link DeliveryService}. If no online caches are found to support the specified
* DeliveryService an empty list is returned.
- *
+ *
* @param ds
* the DeliveryService to check
* @return collection of supported caches
@@ -313,7 +314,7 @@ public class TrafficRouter {
track.setResult(ResultType.MISS);
track.setResultDetails(ResultDetails.DS_CZ_ONLY);
}
- } else if (track.continueGeo) {
+ } else if (track.continueGeo) {
// continue Geo can be disabled when backup group is used -- ended up an empty cache list if reach here
caches = selectCachesByGeo(request.getClientIP(), ds, cacheLocation, track);
}
@@ -515,6 +516,13 @@ public class TrafficRouter {
return caches;
}
+ /**
+ * Gets multiple routes for STEERING Delivery Services
+ *
+ * @param request The client's HTTP Request
+ * @param track A {@link Track} object used to track routing statistics
+ * @return The list of routes available to service the client's request.
+ */
public HTTPRouteResult multiRoute(final HTTPRequest request, final Track track) throws MalformedURLException, GeolocationException {
final DeliveryService entryDeliveryService = cacheRegister.getDeliveryService(request, true);
@@ -530,12 +538,15 @@ public class TrafficRouter {
final List<SteeringResult> resultsToRemove = new ArrayList<>();
// Pattern based consistent hashing - use consistentHashRegex from steering DS instead of targets
- final String pathToHash = buildPatternBasedHashString(entryDeliveryService, request.getPath());
+ final String steeringHash = buildPatternBasedHashString(entryDeliveryService.getConsistentHashRegex(), request.getPath());
for (final SteeringResult steeringResult : steeringResults) {
final DeliveryService ds = steeringResult.getDeliveryService();
final List<Cache> caches = selectCaches(request, ds, track);
+ // child Delivery Services can use their query parameters
+ final String pathToHash = steeringHash + ds.extractSignificantQueryParams(request);
+
if (caches != null && !caches.isEmpty()) {
final Cache cache = consistentHasher.selectHashable(caches, ds.getDispersion(), pathToHash);
steeringResult.setCache(cache);
@@ -559,19 +570,49 @@ public class TrafficRouter {
return routeResult;
}
- public String buildPatternBasedHashString(final DeliveryService deliveryService, final String requestPath) {
- if (deliveryService.getConsistentHashRegex() != null && !deliveryService.getConsistentHashRegex().isEmpty() && !requestPath.isEmpty()) {
- return buildPatternBasedHashString(deliveryService.getConsistentHashRegex(), requestPath);
+ /**
+ * Creates a string to be used in consistent hashing.
+ *<p>
+ * This uses simply the request path by default, but will consider any and all Query Parameters
+ * that are in deliveryService's {@link DeliveryService.consistentHashQueryParams} set as well.
+ * It will also fall back on the request path if the query parameters are not UTF-8-encoded.
+ *</p>
+ * @param deliveryService The {@link DeliveryService} being requested
+ * @param request An {@link HTTPRequest} representing the client's request.
+ * @return A string appropriate to use for consistent hashing to service the request
+ */
+ @SuppressWarnings({"PMD.CyclomaticComplexity"})
+ public String buildPatternBasedHashString(final DeliveryService deliveryService, final HTTPRequest request) {
+ final String requestPath = request.getPath();
+ final StringBuilder hashString = new StringBuilder("");
+ if (deliveryService.getConsistentHashRegex() != null && !requestPath.isEmpty()) {
+ hashString.append(buildPatternBasedHashString(deliveryService.getConsistentHashRegex(), requestPath));
}
- return requestPath;
+
+ hashString.append(deliveryService.extractSignificantQueryParams(request));
+
+ return hashString.toString();
}
+ /**
+ * Constructs a string to be used in consistent hashing
+ * <p>
+ * If `regex` is `null` or empty - or if an error occurs applying it -, returns `requestPath` unaltered.
+ * </p>
+ * @param regex A regular expression matched against the client's request path to extract information important to consistent hashing
+ * @param requestPath The client's request path - e.g. '/some/path' from 'https://example.com/some/path'
+ * @return The parts of requestPath that matched regex
+ */
public String buildPatternBasedHashString(final String regex, final String requestPath) {
+ if (regex == null || regex.isEmpty()) {
+ return requestPath;
+ }
+
try {
final Pattern pattern = Pattern.compile(regex);
final Matcher matcher = pattern.matcher(requestPath);
- final StringBuilder sb = new StringBuilder();
+ final StringBuilder sb = new StringBuilder("");
if (matcher.find() && matcher.groupCount() > 0) {
for (int i = 1; i <= matcher.groupCount(); i++) {
final String text = matcher.group(i);
@@ -579,11 +620,16 @@ public class TrafficRouter {
}
return sb.toString();
}
- return requestPath;
} catch (final Exception e) {
- return requestPath;
+ final StringBuilder error = new StringBuilder("Failed to construct hash string using regular expression: '");
+ error.append(regex);
+ error.append("' against request path: '");
+ error.append(requestPath);
+ error.append("' Exception: ");
+ error.append(e.toString());
+ LOGGER.error(error.toString());
}
-
+ return requestPath;
}
@SuppressWarnings({ "PMD.CyclomaticComplexity", "PMD.NPathComplexity" })
@@ -629,7 +675,7 @@ public class TrafficRouter {
}
// Pattern based consistent hashing
- final String pathToHash = buildPatternBasedHashString(deliveryService, request.getPath());
+ final String pathToHash = buildPatternBasedHashString(deliveryService, request);
final Cache cache = consistentHasher.selectHashable(caches, deliveryService.getDispersion(), pathToHash);
// Enforce anonymous IP blocking if a DS has anonymous blocking enabled
@@ -662,7 +708,7 @@ public class TrafficRouter {
return null;
}
- final List<SteeringResult> steeringResults = consistentHashMultiDeliveryService(entryDeliveryService, request.getPath());
+ final List<SteeringResult> steeringResults = consistentHashMultiDeliveryService(entryDeliveryService, request);
if (steeringResults == null || steeringResults.isEmpty()) {
track.setResult(ResultType.DS_MISS);
@@ -693,7 +739,7 @@ public class TrafficRouter {
private DeliveryService getDeliveryService(final HTTPRequest request, final Track track) {
final String xtcSteeringOption = request.getHeaderValue(XTC_STEERING_OPTION);
- final DeliveryService deliveryService = consistentHashDeliveryService(cacheRegister.getDeliveryService(request, true), request.getPath(), xtcSteeringOption);
+ final DeliveryService deliveryService = consistentHashDeliveryService(cacheRegister.getDeliveryService(request, true), request, xtcSteeringOption);
if (deliveryService == null) {
track.setResult(ResultType.DS_MISS);
@@ -812,7 +858,7 @@ public class TrafficRouter {
track.continueGeo = false;
return null;
}
- }
+ }
// We had a hit in the CZF but the name does not match a known cache location.
// Check whether the CZF entry has a geolocation and use it if so.
@@ -846,11 +892,35 @@ public class TrafficRouter {
return getCoverageZoneCacheLocation(ip, deliveryService.getId());
}
+ /**
+ * Chooses a {@link Cache} for a Delivery Service based on the Coverage Zone File given a clients IP and request *path*.
+ *
+ * @param ip The client's IP address
+ * @param deliveryServiceId The "xml_id" of a Delivery Service being routed
+ * @param requestPath The client's requested path - e.g. 'http://test.example.com/request/path' -> '/request/path'
+ * @return A cache object chosen to serve the client's request
+ */
public Cache consistentHashForCoverageZone(final String ip, final String deliveryServiceId, final String requestPath) {
return consistentHashForCoverageZone(ip, deliveryServiceId, requestPath, false);
}
+ /**
+ * Chooses a {@link Cache} for a Delivery Service based on the Coverage Zone File or Deep Coverage Zone File given a clients IP and request *path*.
+ *
+ * @param ip The client's IP address
+ * @param deliveryServiceId The "xml_id" of a Delivery Service being routed
+ * @param requestPath The client's requested path - e.g. 'http://test.example.com/request/path' -> '/request/path'
+ * @param useDeep if `true` will attempt to use Deep Coverage Zones - otherwise will only use Coverage Zone File
+ * @return A cache object chosen to serve the client's request
+ */
public Cache consistentHashForCoverageZone(final String ip, final String deliveryServiceId, final String requestPath, final boolean useDeep) {
+ final HTTPRequest r = new HTTPRequest();
+ r.setPath(requestPath);
+ r.setQueryString("");
+ return consistentHashForCoverageZone(ip, deliveryServiceId, r, useDeep);
+ }
+
+ public Cache consistentHashForCoverageZone(final String ip, final String deliveryServiceId, final HTTPRequest request, final boolean useDeep) {
final DeliveryService deliveryService = cacheRegister.getDeliveryService(deliveryServiceId);
if (deliveryService == null) {
LOGGER.error("Failed getting delivery service from cache register for id '" + deliveryServiceId + "'");
@@ -864,11 +934,26 @@ public class TrafficRouter {
return null;
}
- final String pathToHash = buildPatternBasedHashString(deliveryService, requestPath);
+ final String pathToHash = buildPatternBasedHashString(deliveryService, request);
return consistentHasher.selectHashable(caches, deliveryService.getDispersion(), pathToHash);
}
+ /**
+ * Chooses a {@link Cache} for a Delivery Service based on GeoLocation given a clients IP and request *path*.
+ *
+ * @param ip The client's IP address
+ * @param deliveryServiceId The "xml_id" of a Delivery Service being routed
+ * @param requestPath The client's requested path - e.g. 'http://test.example.com/request/path' -> '/request/path'
+ * @return A cache object chosen to serve the client's request
+ */
public Cache consistentHashForGeolocation(final String ip, final String deliveryServiceId, final String requestPath) {
+ final HTTPRequest r = new HTTPRequest();
+ r.setPath(requestPath);
+ r.setQueryString("");
+ return consistentHashForGeolocation(ip, deliveryServiceId, r);
+ }
+
+ public Cache consistentHashForGeolocation(final String ip, final String deliveryServiceId, final HTTPRequest request) {
final DeliveryService deliveryService = cacheRegister.getDeliveryService(deliveryServiceId);
if (deliveryService == null) {
LOGGER.error("Failed getting delivery service from cache register for id '" + deliveryServiceId + "'");
@@ -893,12 +978,15 @@ public class TrafficRouter {
return null;
}
- final String pathToHash = buildPatternBasedHashString(deliveryService, requestPath);
+ final String pathToHash = buildPatternBasedHashString(deliveryService, request);
return consistentHasher.selectHashable(caches, deliveryService.getDispersion(), pathToHash);
}
public String buildPatternBasedHashStringDeliveryService(final String deliveryServiceId, final String requestPath) {
- return buildPatternBasedHashString(cacheRegister.getDeliveryService(deliveryServiceId), requestPath);
+ final HTTPRequest r = new HTTPRequest();
+ r.setPath(requestPath);
+ r.setQueryString("");
+ return buildPatternBasedHashString(cacheRegister.getDeliveryService(deliveryServiceId), r);
}
private boolean isSteeringDeliveryService(final DeliveryService deliveryService) {
@@ -943,7 +1031,7 @@ public class TrafficRouter {
}
}
- public List<SteeringResult> consistentHashMultiDeliveryService(final DeliveryService deliveryService, final String requestPath) {
+ public List<SteeringResult> consistentHashMultiDeliveryService(final DeliveryService deliveryService, final HTTPRequest request) {
if (deliveryService == null) {
return null;
}
@@ -958,7 +1046,7 @@ public class TrafficRouter {
final Steering steering = steeringRegistry.get(deliveryService.getId());
// Pattern based consistent hashing
- final String pathToHash = buildPatternBasedHashString(deliveryService, requestPath);
+ final String pathToHash = buildPatternBasedHashString(deliveryService, request);
final List<SteeringTarget> steeringTargets = consistentHasher.selectHashables(steering.getTargets(), pathToHash);
for (final SteeringTarget steeringTarget : steeringTargets) {
@@ -972,8 +1060,23 @@ public class TrafficRouter {
return steeringResults;
}
+ /**
+ * Chooses a {@link Cache} for a Steering Delivery Service target based on the Coverage Zone File given a clients IP and request *path*.
+ *
+ * @param ip The client's IP address
+ * @param deliveryServiceId The "xml_id" of a Delivery Service being routed
+ * @param requestPath The client's requested path - e.g. 'http://test.example.com/request/path' -> '/request/path'
+ * @return A cache object chosen to serve the client's request
+ */
public Cache consistentHashSteeringForCoverageZone(final String ip, final String deliveryServiceId, final String requestPath) {
- final DeliveryService deliveryService = consistentHashDeliveryService(deliveryServiceId, requestPath);
+ final HTTPRequest r = new HTTPRequest();
+ r.setPath(requestPath);
+ r.setQueryString("");
+ return consistentHashSteeringForCoverageZone(ip, deliveryServiceId, r);
+ }
+
+ public Cache consistentHashSteeringForCoverageZone(final String ip, final String deliveryServiceId, final HTTPRequest request) {
+ final DeliveryService deliveryService = consistentHashDeliveryService(deliveryServiceId, request);
if (deliveryService == null) {
LOGGER.error("Failed getting delivery service from cache register for id '" + deliveryServiceId + "'");
return null;
@@ -986,16 +1089,30 @@ public class TrafficRouter {
return null;
}
- final String pathToHash = buildPatternBasedHashString(deliveryService, requestPath);
+ final String pathToHash = buildPatternBasedHashString(deliveryService, request);
return consistentHasher.selectHashable(caches, deliveryService.getDispersion(), pathToHash);
}
+ /**
+ * Chooses a target Delivery Service of a given Delivery Service to service a given request path
+ *
+ * @param deliveryServiceId The "xml_id" of the Delivery Service being requested
+ * @param requestPath The requested path - e.g. 'http://test.example.com/request/path' -> '/request/path'
+ * @return The chosen target Delivery Service
+ */
public DeliveryService consistentHashDeliveryService(final String deliveryServiceId, final String requestPath) {
- return consistentHashDeliveryService(cacheRegister.getDeliveryService(deliveryServiceId), requestPath, "");
+ final HTTPRequest r = new HTTPRequest();
+ r.setPath(requestPath);
+ r.setQueryString("");
+ return consistentHashDeliveryService(deliveryServiceId, r);
+ }
+
+ public DeliveryService consistentHashDeliveryService(final String deliveryServiceId, final HTTPRequest request) {
+ return consistentHashDeliveryService(cacheRegister.getDeliveryService(deliveryServiceId), request, "");
}
@SuppressWarnings({"PMD.CyclomaticComplexity", "PMD.NPathComplexity"})
- public DeliveryService consistentHashDeliveryService(final DeliveryService deliveryService, final String requestPath, final String xtcSteeringOption) {
+ public DeliveryService consistentHashDeliveryService(final DeliveryService deliveryService, final HTTPRequest request, final String xtcSteeringOption) {
if (deliveryService == null) {
return null;
}
@@ -1010,7 +1127,7 @@ public class TrafficRouter {
return steering.hasTarget(xtcSteeringOption) ? cacheRegister.getDeliveryService(xtcSteeringOption) : null;
}
- final String bypassDeliveryServiceId = steering.getBypassDestination(requestPath);
+ final String bypassDeliveryServiceId = steering.getBypassDestination(request.getPath());
if (bypassDeliveryServiceId != null && !bypassDeliveryServiceId.isEmpty()) {
final DeliveryService bypass = cacheRegister.getDeliveryService(bypassDeliveryServiceId);
if (bypass != null) { // bypass DS target might not be in CRConfig yet. Until then, try existing targets
@@ -1024,7 +1141,7 @@ public class TrafficRouter {
.collect(Collectors.toList());
// Pattern based consistent hashing
- final String pathToHash = buildPatternBasedHashString(deliveryService, requestPath);
+ final String pathToHash = buildPatternBasedHashString(deliveryService, request);
final SteeringTarget steeringTarget = consistentHasher.selectHashable(availableTargets, deliveryService.getDispersion(), pathToHash);
// set target.consistentHashRegex from steering DS, if it is set
@@ -1039,7 +1156,7 @@ public class TrafficRouter {
* Returns a list {@link CacheLocation}s sorted by distance from the client.
* If the client's location could not be determined, then the list is
* unsorted.
- *
+ *
* @param cacheLocations
* the collection of CacheLocations to order
* @return the ordered list of locations
@@ -1065,9 +1182,9 @@ public class TrafficRouter {
return null;
}
- /*
+ /**
* Selects a {@link Cache} from the {@link CacheLocation} provided.
- *
+ *
* @param location
* the caches that will considered
* @param ds
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryServiceTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryServiceTest.java
index e9335df..59da1c9 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryServiceTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryServiceTest.java
@@ -15,6 +15,7 @@
package com.comcast.cdn.traffic_control.traffic_router.core.ds;
+import com.comcast.cdn.traffic_control.traffic_router.core.request.HTTPRequest;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.junit.Test;
@@ -35,6 +36,35 @@ public class DeliveryServiceTest {
}
@Test
+ public void itHandlesLackOfConsistentHashQueryParamsInJSON() throws Exception {
+ final ObjectMapper mapper = new ObjectMapper();
+ final JsonNode json = mapper.readTree("{\"routingName\":\"edge\",\"coverageZoneOnly\":false}");
+ DeliveryService d = new DeliveryService("test", json);
+ assert d.getConsistentHashQueryParams() != null;
+ assert d.getConsistentHashQueryParams().size() == 0;
+ }
+
+ @Test
+ public void itHandlesDuplicatesInConsistentHashQueryParams() throws Exception {
+ final ObjectMapper mapper = new ObjectMapper();
+ final JsonNode json = mapper.readTree("{\"routingName\":\"edge\",\"coverageZoneOnly\":false,\"consistentHashQueryParams\":[\"test\", \"quest\", \"test\"]}");
+ DeliveryService d = new DeliveryService("test", json);
+ assert d.getConsistentHashQueryParams() != null;
+ assert d.getConsistentHashQueryParams().size() == 2;
+ assert d.getConsistentHashQueryParams().contains("test");
+ assert d.getConsistentHashQueryParams().contains("quest");
+ }
+
+ @Test
+ public void itExtractsQueryParams() throws Exception {
+ final JsonNode json = (new ObjectMapper()).readTree("{\"routingName\":\"edge\",\"coverageZoneOnly\":false,\"consistentHashQueryParams\":[\"test\", \"quest\"]}");
+ final HTTPRequest r = new HTTPRequest();
+ r.setPath("/path1234/some_stream_name1234/some_other_info.m3u8");
+ r.setQueryString("test=value&foo=fizz&quest=oth%20ervalue&bar=buzz");
+ assert (new DeliveryService("test", json)).extractSignificantQueryParams(r).equals("quest=oth ervaluetest=value");
+ }
+
+ @Test
public void itConfiguresRequestHeadersFromJSON() throws Exception {
final ObjectMapper mapper = new ObjectMapper();
final String jsonStr = "{\"routingName\":\"edge\",\"requestHeaders\":[\"Cookie\",\"Cache-Control\",\"If-Modified-Since\",\"Content-Type\"],\"coverageZoneOnly\":false}";
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/hashing/ConsistentHasherTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/hashing/ConsistentHasherTest.java
index c0d8d39..fe5c290 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/hashing/ConsistentHasherTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/hashing/ConsistentHasherTest.java
@@ -16,11 +16,13 @@
package com.comcast.cdn.traffic_control.traffic_router.core.hashing;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.Dispersion;
+import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
import com.comcast.cdn.traffic_control.traffic_router.core.hash.ConsistentHasher;
import com.comcast.cdn.traffic_control.traffic_router.core.hash.DefaultHashable;
import com.comcast.cdn.traffic_control.traffic_router.core.hash.Hashable;
import com.comcast.cdn.traffic_control.traffic_router.core.hash.MD5HashFunction;
import com.comcast.cdn.traffic_control.traffic_router.core.hash.NumberSearcher;
+import com.comcast.cdn.traffic_control.traffic_router.core.request.HTTPRequest;
import com.comcast.cdn.traffic_control.traffic_router.core.router.TrafficRouter;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
@@ -39,9 +41,11 @@ import static org.hamcrest.Matchers.allOf;
import static org.hamcrest.Matchers.anyOf;
import static org.hamcrest.Matchers.greaterThan;
import static org.hamcrest.Matchers.lessThan;
+import static org.hamcrest.Matchers.notNullValue;
import static org.hamcrest.core.IsEqual.equalTo;
import static org.junit.Assert.assertThat;
import static org.mockito.Matchers.anyString;
+import static org.mockito.Matchers.any;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import static org.mockito.MockitoAnnotations.initMocks;
@@ -81,6 +85,7 @@ public class ConsistentHasherTest {
trafficRouter = mock(TrafficRouter.class);
when(trafficRouter.buildPatternBasedHashString(anyString(), anyString())).thenCallRealMethod();
+ when(trafficRouter.buildPatternBasedHashString(any(DeliveryService.class), any(HTTPRequest.class))).thenCallRealMethod();
initMocks(this);
}
@@ -205,6 +210,24 @@ public class ConsistentHasherTest {
assertThat(hashableResult1, allOf(equalTo(hashableResult2), equalTo(hashableResult3), equalTo(hashableResult4)));
}
+ @Test
+ public void itHashesQueryParams() throws Exception {
+ final JsonNode j = (new ObjectMapper()).readTree("{\"routingName\":\"edge\",\"coverageZoneOnly\":false,\"consistentHashQueryParams\":[\"test\", \"quest\"]}");
+ final DeliveryService d = new DeliveryService("test", j);
+
+ final HTTPRequest r1 = new HTTPRequest();
+ r1.setPath("/path1234/some_stream_name1234/some_other_info.m3u8");
+ r1.setQueryString("test=value");
+
+ final HTTPRequest r2 = new HTTPRequest();
+ r2.setPath(r1.getPath());
+ r2.setQueryString("quest=other_value");
+
+ final String p1 = trafficRouter.buildPatternBasedHashString(d, r1);
+ final String p2 = trafficRouter.buildPatternBasedHashString(d, r2);
+ assert !p1.equals(p2);
+ }
+
String alphanumericCharacters = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWZYZ";
String exampleValidPathCharacters = alphanumericCharacters + "/=;()-.";
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterTest.java
index d644445..16d6fe2 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterTest.java
@@ -95,7 +95,7 @@ public class TrafficRouterTest {
when(trafficRouter.route(any(HTTPRequest.class), any(Track.class))).thenCallRealMethod();
when(trafficRouter.singleRoute(any(HTTPRequest.class), any(Track.class))).thenCallRealMethod();
when(trafficRouter.selectDeliveryService(any(Request.class), anyBoolean())).thenReturn(deliveryService);
- when(trafficRouter.consistentHashDeliveryService(any(DeliveryService.class), anyString(), anyString())).thenCallRealMethod();
+ when(trafficRouter.consistentHashDeliveryService(any(DeliveryService.class), any(HTTPRequest.class), anyString())).thenCallRealMethod();
}
@Test