You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficcontrol.apache.org by GitBox <gi...@apache.org> on 2018/01/24 17:23:13 UTC
[GitHub] elsloo closed pull request #1520: Add support for Deep Caching/Routing
elsloo closed pull request #1520: Add support for Deep Caching/Routing
URL: https://github.com/apache/incubator-trafficcontrol/pull/1520
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docs/source/admin/traffic_ops/using.rst b/docs/source/admin/traffic_ops/using.rst
index 7a78bd1ea6..e7e8eb215a 100644
--- a/docs/source/admin/traffic_ops/using.rst
+++ b/docs/source/admin/traffic_ops/using.rst
@@ -306,6 +306,11 @@ The fields in the Delivery Service view are:
+--------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Content Routing Type | The type of content routing this delivery service will use. See :ref:`rl-ds-types`. |
+--------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| Deep Caching | (for HTTP routed delivery services only) When to do deep caching for this delivery service (see :ref:`Deep Caching <rl-deep-cache>` for more details): |
+| | |
+| | - NEVER |
+| | - ALWAYS |
++--------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Delivery Service DNS TTL | The Time To Live on the DNS record for the Traffic Router A and AAAA records (``<routing-name>.<deliveryservice>.<cdn-domain>``). |
+--------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Delivery Servics URLs | (Read Only) An example of how the delivery URL may start. This could be multiple rows if multiple HOST_REGEXP entries have been entered. |
@@ -710,7 +715,11 @@ Traffic Router Profile
+-----------------------------------------+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| coveragezone.polling.interval | CRConfig.json | How often to refresh the coverage zone map in ms |
+-----------------------------------------+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-| coveragezone.polling.url | CRConfig.json | The location (URL) to retrieve the coverage zone map file in XML format from. |
+| coveragezone.polling.url | CRConfig.json | The location (URL) to retrieve the coverage zone map file in JSON format from. |
++-----------------------------------------+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| deepcoveragezone.polling.interval | CRConfig.json | How often to refresh the deep coverage zone map in ms |
++-----------------------------------------+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| deepcoveragezone.polling.url | CRConfig.json | The location (URL) to retrieve the deep coverage zone map file in JSON format from. |
+-----------------------------------------+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tld.soa.expire | CRConfig.json | The value for the expire field the Traffic Router DNS Server will respond with on Start of Authority (SOA) records. |
+-----------------------------------------+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@@ -846,23 +855,31 @@ The Coverage Zone File (CZF) should contain a cachegroup name to network prefix
{
"coverageZones": {
"cache-group-01": {
+ "coordinates": {
+ "latitude": 1.1,
+ "longitude": 2.2
+ },
"network6": [
- "1234:5678::\/64",
- "1234:5679::\/64"
+ "1234:5678::/64",
+ "1234:5679::/64"
],
"network": [
- "192.168.8.0\/24",
- "192.168.9.0\/24"
+ "192.168.8.0/24",
+ "192.168.9.0/24"
]
- }
+ },
"cache-group-02": {
+ "coordinates": {
+ "latitude": 3.3,
+ "longitude": 4.4
+ },
"network6": [
- "1234:567a::\/64",
- "1234:567b::\/64"
+ "1234:567a::/64",
+ "1234:567b::/64"
],
"network": [
- "192.168.4.0\/24",
- "192.168.5.0\/24"
+ "192.168.4.0/24",
+ "192.168.5.0/24"
]
}
}
@@ -872,6 +889,62 @@ The CZF is an input to the Traffic Control CDN, and as such does not get generat
The script that generates the CZF file is not part of Traffic Control, since it is different for each situation.
+.. note:: The ``"coordinates"`` section is optional and may be used by Traffic Router for localization in the case of a CZF "hit" where the zone name does not map to a Cache Group name in Traffic Ops (i.e. Traffic Router will route to the closest Cache Group(s) geographically).
+
+.. _rl-deep-czf:
+
+The Deep Coverage Zone File
++++++++++++++++++++++++++++
+The Deep Coverage Zone File (DCZF) format is similar to the CZF format but adds a ``caches`` list under each ``deepCoverageZone``: ::
+
+ {
+ "deepCoverageZones": {
+ "location-01": {
+ "coordinates": {
+ "latitude": 5.5,
+ "longitude": 6.6
+ },
+ "network6": [
+ "1234:5678::/64",
+ "1234:5679::/64"
+ ],
+ "network": [
+ "192.168.8.0/24",
+ "192.168.9.0/24"
+ ],
+ "caches": [
+ "edge-01",
+ "edge-02"
+ ]
+ },
+ "location-02": {
+ "coordinates": {
+ "latitude": 7.7,
+ "longitude": 8.8
+ },
+ "network6": [
+ "1234:567a::/64",
+ "1234:567b::/64"
+ ],
+ "network": [
+ "192.168.4.0/24",
+ "192.168.5.0/24"
+ ],
+ "caches": [
+ "edge-02",
+ "edge-03"
+ ]
+ }
+ }
+ }
+
+Each entry in the ``caches`` list is the hostname of an edge cache registered in Traffic Ops which will be used for "deep" caching in that Deep Coverage Zone. Unlike a regular CZF, coverage zones in the DCZF do not map to a Cache Group in Traffic Ops, so currently the deep coverage zone name only needs to be unique.
+
+If the Traffic Router gets a DCZF "hit" for a requested Delivery Service that has Deep Caching enabled, the client will be routed to an available "deep" cache from that zone's ``caches`` list.
+
+.. note:: The ``"coordinates"`` section is optional.
+
+
.. _rl-working-with-profiles:
Parameters and Profiles
diff --git a/docs/source/admin/traffic_router.rst b/docs/source/admin/traffic_router.rst
index 3a377811e9..474dc96be7 100644
--- a/docs/source/admin/traffic_router.rst
+++ b/docs/source/admin/traffic_router.rst
@@ -102,6 +102,10 @@ Configuration files
| +-------------------------------------------+-----------------------------------------------------------------------------------------------------+---------------------------------------------------+
| | cache.czmap.database.refresh.period | The interval in milliseconds which Traffic Router will poll for a new coverage zone file | 10800000 |
| +-------------------------------------------+-----------------------------------------------------------------------------------------------------+---------------------------------------------------+
+| | cache.dczmap.database | Full path to the local copy of the deep coverage zone file | /opt/traffic_router/db/dczmap.json |
+| +-------------------------------------------+-----------------------------------------------------------------------------------------------------+---------------------------------------------------+
+| | cache.dczmap.database.refresh.period | The interval in milliseconds which Traffic Router will poll for a new deep coverage zone file | 10800000 |
+| +-------------------------------------------+-----------------------------------------------------------------------------------------------------+---------------------------------------------------+
| | cache.health.json | Full path to the local copy of the health state | /opt/traffic_router/db/health.json |
| +-------------------------------------------+-----------------------------------------------------------------------------------------------------+---------------------------------------------------+
| | cache.health.json.refresh.period | The interval in milliseconds which Traffic Router will poll for a new health state file | 1000 |
@@ -183,23 +187,23 @@ Items within brackets below are detailed under the HTTP and DNS sections
Fields Always Present
---------------------
-+------+---------------------------------------------------------------------------------+---------------------------------------------------------------------------+
-|Name |Description |Data |
-+======+=================================================================================+===========================================================================+
-|qtype |Whether the request was for DNS or HTTP |Always DNS or HTTP |
-+------+---------------------------------------------------------------------------------+---------------------------------------------------------------------------+
-|chi |The IP address of the requester |Depends on whether this was a DNS or HTTP request, see below sections |
-+------+---------------------------------------------------------------------------------+---------------------------------------------------------------------------+
-|ttms |The amount of time in milliseconds it took Traffic Router to process the request |A number greater than or equal to zero |
-+------+---------------------------------------------------------------------------------+---------------------------------------------------------------------------+
-|rtype |Routing Result Type |One of ERROR, CZ, GEO, MISS, STATIC_ROUTE, DS_REDIRECT, DS_MISS, INIT, FED |
-+------+---------------------------------------------------------------------------------+---------------------------------------------------------------------------+
-|rloc |GeoLocation of result |Latitude and Longitude in Decimal Degrees |
-+------+---------------------------------------------------------------------------------+---------------------------------------------------------------------------+
-|rdtl |Result Details Associated with unusual conditions |One of DS_NOT_FOUND, DS_NO_BYPASS, DS_BYPASS, DS_CZ_ONLY |
-+------+---------------------------------------------------------------------------------+---------------------------------------------------------------------------+
-|rerr |Message about internal Traffic Router Error |String |
-+------+---------------------------------------------------------------------------------+---------------------------------------------------------------------------+
++------+---------------------------------------------------------------------------------+------------------------------------------------------------------------------------+
+|Name |Description |Data |
++======+=================================================================================+====================================================================================+
+|qtype |Whether the request was for DNS or HTTP |Always DNS or HTTP |
++------+---------------------------------------------------------------------------------+------------------------------------------------------------------------------------+
+|chi |The IP address of the requester |Depends on whether this was a DNS or HTTP request, see below sections |
++------+---------------------------------------------------------------------------------+------------------------------------------------------------------------------------+
+|ttms |The amount of time in milliseconds it took Traffic Router to process the request |A number greater than or equal to zero |
++------+---------------------------------------------------------------------------------+------------------------------------------------------------------------------------+
+|rtype |Routing Result Type |One of ERROR, CZ, DEEP_CZ, GEO, MISS, STATIC_ROUTE, DS_REDIRECT, DS_MISS, INIT, FED |
++------+---------------------------------------------------------------------------------+------------------------------------------------------------------------------------+
+|rloc |GeoLocation of result |Latitude and Longitude in Decimal Degrees |
++------+---------------------------------------------------------------------------------+------------------------------------------------------------------------------------+
+|rdtl |Result Details Associated with unusual conditions |One of DS_NOT_FOUND, DS_NO_BYPASS, DS_BYPASS, DS_CZ_ONLY |
++------+---------------------------------------------------------------------------------+------------------------------------------------------------------------------------+
+|rerr |Message about internal Traffic Router Error |String |
++------+---------------------------------------------------------------------------------+------------------------------------------------------------------------------------+
**rtype meanings**
@@ -210,6 +214,8 @@ Fields Always Present
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|CZ |The result was derived from Coverage Zone data based on the address in the chi field |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+|DEEP_CZ |The result was derived from Deep Coverage Zone data based on the address in the chi field |
++-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|GEO |The result was derived from geolocation service based on the address in the chi field |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|MISS |Traffic Router was unable to resolve a DNS request or find a cache for the requested resource |
@@ -347,6 +353,54 @@ The URL has 3 kinds of formats, which have different meanings:
3. URL with domain that doesn't match with the delivery service. For this URL, the router will return the configured url directly to the client.
+
+.. _rl-deep-cache:
+
+Deep Caching - Deep Coverage Zone Topology
+==========================================
+
+Overview
+--------
+
+Deep Caching is a feature that enables clients to be routed to the closest
+possible "deep" edge caches on a per Delivery Service basis. The term "deep" is
+used in the networking sense, meaning that the edge caches are located deep in
+the network where the number of network hops to a client is as minimal as
+possible. This deep caching topology is desirable because storing content closer
+to the client gives better bandwidth savings, and sometimes the cost of
+bandwidth usage in the network outweighs the cost of adding storage. While it
+may not be feasible to cache an entire copy of the CDN's contents in every deep
+location (for the best possible bandwidth savings), storing just a relatively
+small amount of the CDN's most requested content can lead to very high bandwidth
+savings.
+
+Getting started
+---------------
+
+What you need:
+
+#. Edge caches deployed in "deep" locations and registered in Traffic Ops
+#. A Deep Coverage Zone File (DCZF) mapping these deep cache hostnames to specific network prefixes (see :ref:`rl-deep-czf` for details)
+#. Deep caching parameters in the Traffic Router Profile (see :ref:`rl-ccr-profile` for details):
+
+ * ``deepcoveragezone.polling.interval``
+ * ``deepcoveragezone.polling.url``
+
+#. Deep Caching enabled on one or more HTTP Delivery Services (i.e. ``deepCachingType`` = ALWAYS)
+
+How it works
+------------
+
+Deep Coverage Zone routing is very similar to that of regular Coverage Zone
+routing, except that the DCZF is preferred over the regular CZF for Delivery
+Services with DC (Deep Caching) enabled. If the client requests a DC-enabled
+Delivery Service and their IP address gets a "hit" in the DCZF, Traffic Router
+will attempt to route that client to one of the available deep caches in the
+client's corresponding zone. If there are no deep caches available for a
+client's request, Traffic Router will "fall back" to the regular CZF and
+continue regular CZF routing from there.
+
+
.. _rl-tr-steering:
Steering feature
diff --git a/docs/source/development/traffic_ops_api/v12/deliveryservice.rst b/docs/source/development/traffic_ops_api/v12/deliveryservice.rst
index 2884a4b58a..6dee5784fc 100644
--- a/docs/source/development/traffic_ops_api/v12/deliveryservice.rst
+++ b/docs/source/development/traffic_ops_api/v12/deliveryservice.rst
@@ -66,6 +66,11 @@ Delivery Service
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``checkPath`` | string | The path portion of the URL to check this deliveryservice for health. |
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
+ | ``deepCachingType`` | string | When to do Deep Caching for this Delivery Service: |
+ | | | |
+ | | | - NEVER (default) |
+ | | | - ALWAYS |
+ +--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``displayName`` | string | The display name of the delivery service. |
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``dnsBypassCname`` | string | |
@@ -206,6 +211,7 @@ Delivery Service
"cdnId": "2",
"cdnName": "over-the-top",
"checkPath": "",
+ "deepCachingType": "NEVER",
"displayName": "My Cool Delivery Service",
"dnsBypassCname": "",
"dnsBypassIp": "",
@@ -301,6 +307,11 @@ Delivery Service
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``checkPath`` | string | The path portion of the URL to check this deliveryservice for health. |
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
+ | ``deepCachingType`` | string | When to do Deep Caching for this Delivery Service: |
+ | | | |
+ | | | - NEVER (default) |
+ | | | - ALWAYS |
+ +--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``displayName`` | string | The display name of the delivery service. |
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``dnsBypassCname`` | string | |
@@ -449,6 +460,7 @@ Delivery Service
"cdnId": "2",
"cdnName": "over-the-top",
"checkPath": "",
+ "deepCachingType": "NEVER",
"displayName": "My Cool Delivery Service",
"dnsBypassCname": "",
"dnsBypassIp": "",
@@ -1918,6 +1930,11 @@ URL Sig Keys
+----------------------------------------+--------+----------+---------------------------------------------------------------------------------------------+
| ``>customer`` | string | yes | Name of the customer to associated with the delivery service. |
+----------------------------------------+--------+----------+---------------------------------------------------------------------------------------------+
+ | ``>deepCachingType`` | string | no | When to do Deep Caching for this Delivery Service: |
+ | | | | |
+ | | | | - NEVER (default) |
+ | | | | - ALWAYS |
+ +----------------------------------------+--------+----------+---------------------------------------------------------------------------------------------+
| ``>deliveryProtocol`` | string | yes | Eg. http or http/https |
+----------------------------------------+--------+----------+---------------------------------------------------------------------------------------------+
| ``>routingType`` | string | yes | Eg. DNS or HTTP Redirect |
@@ -1978,6 +1995,7 @@ URL Sig Keys
"details": {
"customer": "XYZ Corporation",
"contentType": "video-on-demand",
+ "deepCachingType": "NEVER",
"deliveryProtocol": "http",
"routingType": "dns",
"routingName": "foo",
@@ -2062,6 +2080,11 @@ URL Sig Keys
+------------------------+----------+---------------------------------------------------------------------------------------------------------+
| checkPath | no | The path portion of the URL to check this deliveryservice for health. |
+------------------------+----------+---------------------------------------------------------------------------------------------------------+
+ | deepCachingType | no | When to do Deep Caching for this Delivery Service: |
+ | | | |
+ | | | - NEVER (default) |
+ | | | - ALWAYS |
+ +------------------------+----------+---------------------------------------------------------------------------------------------------------+
| displayName | yes | Display name |
+------------------------+----------+---------------------------------------------------------------------------------------------------------+
| dnsBypassCname | no | Bypass CNAME |
@@ -2232,6 +2255,11 @@ URL Sig Keys
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``checkPath`` | string | The path portion of the URL to check this deliveryservice for health. |
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
+ | ``deepCachingType`` | string | When to do Deep Caching for this Delivery Service: |
+ | | | |
+ | | | - NEVER (default) |
+ | | | - ALWAYS |
+ +--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``displayName`` | string | The display name of the delivery service. |
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``dnsBypassCname`` | string | |
@@ -2376,6 +2404,7 @@ URL Sig Keys
"cdnId": "2",
"cdnName": "over-the-top",
"checkPath": "",
+ "deepCachingType": "NEVER",
"displayName": "My Cool Delivery Service",
"dnsBypassCname": "",
"dnsBypassIp": "",
@@ -2472,6 +2501,11 @@ URL Sig Keys
+------------------------+----------+---------------------------------------------------------------------------------------------------------+
| checkPath | no | The path portion of the URL to check this deliveryservice for health. |
+------------------------+----------+---------------------------------------------------------------------------------------------------------+
+ | deepCachingType | no | When to do Deep Caching for this Delivery Service: |
+ | | | |
+ | | | - NEVER (default) |
+ | | | - ALWAYS |
+ +------------------------+----------+---------------------------------------------------------------------------------------------------------+
| displayName | yes | Display name |
+------------------------+----------+---------------------------------------------------------------------------------------------------------+
| dnsBypassCname | no | Bypass CNAME |
@@ -2602,7 +2636,7 @@ URL Sig Keys
{
"xmlId": "my_ds_1",
"displayName": "my_ds_displayname_1",
- "tenantId": 1,
+ "tenantId": 1,
"protocol": 1,
"orgServerFqdn": "http://10.75.168.91",
"cdnId": 2,
@@ -2642,6 +2676,11 @@ URL Sig Keys
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``checkPath`` | string | The path portion of the URL to check this deliveryservice for health. |
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
+ | ``deepCachingType`` | string | When to do Deep Caching for this Delivery Service: |
+ | | | |
+ | | | - NEVER (default) |
+ | | | - ALWAYS |
+ +--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``displayName`` | string | The display name of the delivery service. |
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``dnsBypassCname`` | string | |
@@ -2786,6 +2825,7 @@ URL Sig Keys
"cdnId": "2",
"cdnName": "over-the-top",
"checkPath": "",
+ "deepCachingType": "NEVER",
"displayName": "My Cool Delivery Service",
"dnsBypassCname": "",
"dnsBypassIp": "",
@@ -2911,6 +2951,11 @@ URL Sig Keys
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``checkPath`` | string | The path portion of the URL to check this deliveryservice for health. |
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
+ | ``deepCachingType`` | string | When to do Deep Caching for this Delivery Service: |
+ | | | |
+ | | | - NEVER (default) |
+ | | | - ALWAYS |
+ +--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``displayName`` | string | The display name of the delivery service. |
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``dnsBypassCname`` | string | |
@@ -3055,6 +3100,7 @@ URL Sig Keys
"cdnId": "2",
"cdnName": "over-the-top",
"checkPath": "",
+ "deepCachingType": "NEVER",
"displayName": "My Cool Delivery Service",
"dnsBypassCname": "",
"dnsBypassIp": "",
diff --git a/docs/source/development/traffic_ops_api/v12/server.rst b/docs/source/development/traffic_ops_api/v12/server.rst
index 77ae9f80fa..779f19acac 100644
--- a/docs/source/development/traffic_ops_api/v12/server.rst
+++ b/docs/source/development/traffic_ops_api/v12/server.rst
@@ -378,6 +378,11 @@ Server
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``checkPath`` | string | The path portion of the URL to check this deliveryservice for health. |
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
+ | ``deepCachingType`` | string | When to do Deep Caching for this Delivery Service: |
+ | | | |
+ | | | - NEVER (default) |
+ | | | - ALWAYS |
+ +--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``displayName`` | string | The display name of the delivery service. |
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``dnsBypassIp`` | string | The IPv4 IP to use for bypass on a DNS deliveryservice - bypass starts when serving more than the |
@@ -516,6 +521,7 @@ Server
"cdnId": "2",
"cdnName": "over-the-top",
"checkPath": "",
+ "deepCachingType": "NEVER",
"displayName": "My Cool Delivery Service",
"dnsBypassCname": "",
"dnsBypassIp": "",
diff --git a/docs/source/development/traffic_ops_api/v12/user.rst b/docs/source/development/traffic_ops_api/v12/user.rst
index fe07920ce7..27efc22e0f 100644
--- a/docs/source/development/traffic_ops_api/v12/user.rst
+++ b/docs/source/development/traffic_ops_api/v12/user.rst
@@ -454,6 +454,11 @@ Users
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``checkPath`` | string | The path portion of the URL to check this deliveryservice for health. |
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
+ | ``deepCachingType`` | string | When to do Deep Caching for this Delivery Service: |
+ | | | |
+ | | | - NEVER (default) |
+ | | | - ALWAYS |
+ +--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``displayName`` | string | The display name of the delivery service. |
+--------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------+
| ``dnsBypassIp`` | string | The IPv4 IP to use for bypass on a DNS deliveryservice - bypass starts when serving more than the |
@@ -592,6 +597,7 @@ Users
"cdnId": "2",
"cdnName": "over-the-top",
"checkPath": "",
+ "deepCachingType": "NEVER",
"displayName": "My Cool Delivery Service",
"dnsBypassCname": "",
"dnsBypassIp": "",
diff --git a/lib/go-tc/deliveryservices.go b/lib/go-tc/deliveryservices.go
index 66a65b4191..9d621a8985 100644
--- a/lib/go-tc/deliveryservices.go
+++ b/lib/go-tc/deliveryservices.go
@@ -56,6 +56,7 @@ type DeliveryService struct {
Active bool `json:"active"`
DSCP int `json:"dscp"`
RoutingName string `json:"routingName"`
+ DeepCachingType DeepCachingType `json:"deepCachingType"`
Signed bool `json:"signed"`
QStringIgnore int `json:"qstringIgnore"`
GeoLimit int `json:"geoLimit"`
diff --git a/lib/go-tc/enum.go b/lib/go-tc/enum.go
index 46aa9d0882..f8f3a6610a 100644
--- a/lib/go-tc/enum.go
+++ b/lib/go-tc/enum.go
@@ -29,6 +29,8 @@ package tc
*/
import (
+ "errors"
+ "strconv"
"strings"
)
@@ -182,3 +184,49 @@ func CacheStatusFromString(s string) CacheStatus {
return CacheStatusInvalid
}
}
+
+// DeepCachingType represents a Delivery Service's deep caching type. The string values of this type should match the Traffic Ops values.
+type DeepCachingType string
+
+const (
+ DeepCachingTypeAlways = DeepCachingType("ALWAYS")
+ DeepCachingTypeNever = DeepCachingType("NEVER")
+ DeepCachingTypeInvalid = DeepCachingType("")
+)
+
+// String returns a string representation of this deep caching type
+func (t DeepCachingType) String() string {
+ switch t {
+ case DeepCachingTypeAlways:
+ return string(t)
+ case DeepCachingTypeNever:
+ return string(t)
+ default:
+ return "INVALID"
+ }
+}
+
+// DeepCachingTypeFromString returns a DeepCachingType from its string representation, or DeepCachingTypeInvalid if the string is not a valid type.
+func DeepCachingTypeFromString(s string) DeepCachingType {
+ switch strings.ToLower(s) {
+ case "always":
+ return DeepCachingTypeAlways
+ case "never":
+ return DeepCachingTypeNever
+ default:
+ return DeepCachingTypeInvalid
+ }
+}
+
+// UnmarshalJSON unmarshals a JSON representation of a DeepCachingType (i.e. a string) or returns an error if the DeepCachingType is invalid
+func (t *DeepCachingType) UnmarshalJSON(data []byte) error {
+ s, err := strconv.Unquote(string(data))
+ if err != nil {
+ return errors.New(string(data) + " JSON not quoted")
+ }
+ *t = DeepCachingTypeFromString(s)
+ if *t == DeepCachingTypeInvalid {
+ return errors.New(string(data) + " is not a DeepCachingType")
+ }
+ return nil
+}
diff --git a/traffic_ops/app/db/migrations/20180110000000_add_deep_caching_type.sql b/traffic_ops/app/db/migrations/20180110000000_add_deep_caching_type.sql
new file mode 100644
index 0000000000..7c6d94bf06
--- /dev/null
+++ b/traffic_ops/app/db/migrations/20180110000000_add_deep_caching_type.sql
@@ -0,0 +1,27 @@
+/*
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+-- +goose Up
+-- SQL in section 'Up' is executed when this migration is applied
+
+CREATE TYPE deep_caching_type AS ENUM ('NEVER', 'ALWAYS');
+
+ALTER TABLE deliveryservice ADD COLUMN deep_caching_type deep_caching_type DEFAULT 'NEVER';
+
+-- +goose Down
+-- SQL section 'Down' is executed when this migration is rolled back
+
+ALTER TABLE deliveryservice DROP COLUMN IF EXISTS deep_caching_type;
+
+DROP TYPE IF EXISTS deep_caching_type;
diff --git a/traffic_ops/app/lib/API/Deliveryservice.pm b/traffic_ops/app/lib/API/Deliveryservice.pm
index 5db2fd228d..675c1dace0 100644
--- a/traffic_ops/app/lib/API/Deliveryservice.pm
+++ b/traffic_ops/app/lib/API/Deliveryservice.pm
@@ -106,6 +106,7 @@ sub index {
"cdnId" => $row->cdn->id,
"cdnName" => $row->cdn->name,
"checkPath" => $row->check_path,
+ "deepCachingType" => $row->deep_caching_type,
"displayName" => $row->display_name,
"dnsBypassCname" => $row->dns_bypass_cname,
"dnsBypassIp" => $row->dns_bypass_ip,
@@ -226,6 +227,7 @@ sub show {
"cdnId" => $row->cdn->id,
"cdnName" => $row->cdn->name,
"checkPath" => $row->check_path,
+ "deepCachingType" => $row->deep_caching_type,
"displayName" => $row->display_name,
"dnsBypassCname" => $row->dns_bypass_cname,
"dnsBypassIp" => $row->dns_bypass_ip,
@@ -332,6 +334,7 @@ sub update {
ccr_dns_ttl => $params->{ccrDnsTtl},
cdn_id => $params->{cdnId},
check_path => $params->{checkPath},
+ deep_caching_type => $params->{deepCachingType},
display_name => $params->{displayName},
dns_bypass_cname => $params->{dnsBypassCname},
dns_bypass_ip => $params->{dnsBypassIp},
@@ -429,6 +432,7 @@ sub update {
"cdnId" => $rs->cdn->id,
"cdnName" => $rs->cdn->name,
"checkPath" => $rs->check_path,
+ "deepCachingType" => $rs->deep_caching_type,
"displayName" => $rs->display_name,
"dnsBypassCname" => $rs->dns_bypass_cname,
"dnsBypassIp" => $rs->dns_bypass_ip,
@@ -560,6 +564,7 @@ sub safe_update {
"cdnId" => $rs->cdn->id,
"cdnName" => $rs->cdn->name,
"checkPath" => $rs->check_path,
+ "deepCachingType" => $rs->deep_caching_type,
"displayName" => $rs->display_name,
"dnsBypassCname" => $rs->dns_bypass_cname,
"dnsBypassIp" => $rs->dns_bypass_ip,
@@ -666,6 +671,7 @@ sub create {
ccr_dns_ttl => $params->{ccrDnsTtl},
cdn_id => $params->{cdnId},
check_path => $params->{checkPath},
+ deep_caching_type => $params->{deepCachingType},
display_name => $params->{displayName},
dns_bypass_cname => $params->{dnsBypassCname},
dns_bypass_ip => $params->{dnsBypassIp},
@@ -774,6 +780,7 @@ sub create {
"cdnId" => $insert->cdn->id,
"cdnName" => $insert->cdn->name,
"checkPath" => $insert->check_path,
+ "deepCachingType" => $insert->deep_caching_type,
"displayName" => $insert->display_name,
"dnsBypassCname" => $insert->dns_bypass_cname,
"dnsBypassIp" => $insert->dns_bypass_ip,
@@ -966,6 +973,7 @@ sub get_deliveryservices_by_serverId {
"cdnId" => $row->cdn->id,
"cdnName" => $row->cdn->name,
"checkPath" => $row->check_path,
+ "deepCachingType" => $row->deep_caching_type,
"displayName" => $row->display_name,
"dnsBypassCname" => $row->dns_bypass_cname,
"dnsBypassIp" => $row->dns_bypass_ip,
@@ -1064,6 +1072,7 @@ sub get_deliveryservices_by_userId {
"cdnId" => $row->cdn->id,
"cdnName" => $row->cdn->name,
"checkPath" => $row->check_path,
+ "deepCachingType" => $row->deep_caching_type,
"displayName" => $row->display_name,
"dnsBypassCname" => $row->dns_bypass_cname,
"dnsBypassIp" => $row->dns_bypass_ip,
@@ -1335,7 +1344,7 @@ sub is_deliveryservice_request_valid {
my $rules = {
fields => [
- qw/customer contentType deliveryProtocol routingType routingName serviceDesc peakBPSEstimate peakTPSEstimate maxLibrarySizeEstimate originURL hasOriginDynamicRemap originTestFile hasOriginACLWhitelist originHeaders otherOriginSecurity queryStringHandling rangeRequestHandling hasSignedURLs hasNegativeCachingCustomization negativeCachingCustomizationNote serviceAliases rateLimitingGBPS rateLimitingTPS overflowService headerRewriteEdge headerRewriteMid headerRewriteRedirectRouter notes/
+ qw/customer contentType deepCachingType deliveryProtocol routingType routingName serviceDesc peakBPSEstimate peakTPSEstimate maxLibrarySizeEstimate originURL hasOriginDynamicRemap originTestFile hasOriginACLWhitelist originHeaders otherOriginSecurity queryStringHandling rangeRequestHandling hasSignedURLs hasNegativeCachingCustomization negativeCachingCustomizationNote serviceAliases rateLimitingGBPS rateLimitingTPS overflowService headerRewriteEdge headerRewriteMid headerRewriteRedirectRouter notes/
],
# Validation checks to perform
@@ -1370,7 +1379,7 @@ sub is_deliveryservice_valid {
my $rules = {
fields => [
- qw/active cacheurl ccrDnsTtl cdnId checkPath displayName dnsBypassCname dnsBypassIp dnsBypassIp6 dnsBypassTtl dscp edgeHeaderRewrite geoLimitRedirectURL geoLimit geoLimitCountries geoProvider globalMaxMbps globalMaxTps httpBypassFqdn infoUrl initialDispersion ipv6RoutingEnabled logsEnabled longDesc longDesc1 longDesc2 maxDnsAnswers midHeaderRewrite missLat missLong multiSiteOrigin multiSiteOriginAlgorithm orgServerFqdn originShield profileId protocol qstringIgnore rangeRequestHandling regexRemap regionalGeoBlocking remapText routingName signed signingAlgorithm sslKeyVersion tenantId trRequestHeaders trResponseHeaders typeId xmlId/
+ qw/active cacheurl ccrDnsTtl cdnId checkPath deepCachingType displayName dnsBypassCname dnsBypassIp dnsBypassIp6 dnsBypassTtl dscp edgeHeaderRewrite geoLimitRedirectURL geoLimit geoLimitCountries geoProvider globalMaxMbps globalMaxTps httpBypassFqdn infoUrl initialDispersion ipv6RoutingEnabled logsEnabled longDesc longDesc1 longDesc2 maxDnsAnswers midHeaderRewrite missLat missLong multiSiteOrigin multiSiteOriginAlgorithm orgServerFqdn originShield profileId protocol qstringIgnore rangeRequestHandling regexRemap regionalGeoBlocking remapText routingName signed signingAlgorithm sslKeyVersion tenantId trRequestHeaders trResponseHeaders typeId xmlId/
],
# validation checks to perform for ALL delivery services
@@ -1378,6 +1387,7 @@ sub is_deliveryservice_valid {
active => [ is_required("is required") ],
cdnId => [ is_required("is required"), \&is_valid_int_or_undef ],
ccrDnsTtl => [ \&is_valid_int_or_undef ],
+ deepCachingType => [ is_like( qr/^NEVER|ALWAYS$/, "must be NEVER or ALWAYS" ) ],
dnsBypassTtl => [ \&is_valid_int_or_undef ],
dscp => [ is_required("is required"), \&is_valid_int_or_undef ],
displayName => [ is_required("is required"), is_long_at_most( 48, 'too long' ) ],
diff --git a/traffic_ops/app/lib/Fixtures/Deliveryservice.pm b/traffic_ops/app/lib/Fixtures/Deliveryservice.pm
index 0ca854ca77..4069ccc728 100644
--- a/traffic_ops/app/lib/Fixtures/Deliveryservice.pm
+++ b/traffic_ops/app/lib/Fixtures/Deliveryservice.pm
@@ -27,6 +27,7 @@ my %definition_for = (
active => 1,
tenant_id => undef,
dscp => 40,
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
qstring_ignore => 0,
geo_limit => 0,
@@ -66,6 +67,7 @@ my %definition_for = (
active => 1,
tenant_id => undef,
dscp => 40,
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
qstring_ignore => 0,
geo_limit => 0,
@@ -105,6 +107,7 @@ my %definition_for = (
active => 1,
tenant_id => undef,
dscp => 40,
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
qstring_ignore => 0,
geo_limit => 0,
@@ -144,6 +147,7 @@ my %definition_for = (
active => 1,
tenant_id => undef,
dscp => 40,
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
qstring_ignore => 0,
geo_limit => 0,
@@ -183,6 +187,7 @@ my %definition_for = (
active => 1,
tenant_id => undef,
dscp => 40,
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
qstring_ignore => 0,
geo_limit => 0,
@@ -222,6 +227,7 @@ my %definition_for = (
active => 1,
tenant_id => undef,
dscp => 40,
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
qstring_ignore => 0,
geo_limit => 0,
@@ -261,6 +267,7 @@ my %definition_for = (
active => 1,
tenant_id => undef,
dscp => 40,
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
qstring_ignore => 0,
geo_limit => 0,
@@ -298,6 +305,7 @@ my %definition_for = (
active => 1,
tenant_id => undef,
dscp => 40,
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
qstring_ignore => 0,
geo_limit => 0,
@@ -335,6 +343,7 @@ my %definition_for = (
active => 1,
tenant_id => undef,
dscp => 40,
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
qstring_ignore => 0,
geo_limit => 0,
@@ -372,6 +381,7 @@ my %definition_for = (
active => 1,
tenant_id => undef,
dscp => 40,
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
qstring_ignore => 0,
geo_limit => 0,
@@ -409,6 +419,7 @@ my %definition_for = (
active => 1,
tenant_id => undef,
dscp => 40,
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
qstring_ignore => 0,
geo_limit => 0,
@@ -446,6 +457,7 @@ my %definition_for = (
active => 1,
tenant_id => undef,
dscp => 40,
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
qstring_ignore => 0,
geo_limit => 0,
@@ -483,6 +495,7 @@ my %definition_for = (
active => 1,
tenant_id => undef,
dscp => 40,
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
qstring_ignore => 0,
geo_limit => 0,
@@ -520,6 +533,7 @@ my %definition_for = (
active => 1,
tenant_id => 10**9,
dscp => 40,
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
qstring_ignore => 0,
geo_limit => 0,
@@ -559,6 +573,7 @@ my %definition_for = (
active => 1,
tenant_id => undef,
dscp => 40,
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
qstring_ignore => 0,
geo_limit => 0,
diff --git a/traffic_ops/app/lib/Fixtures/Integration/Deliveryservice.pm b/traffic_ops/app/lib/Fixtures/Integration/Deliveryservice.pm
index c88ac07db2..c4e3a3aae7 100644
--- a/traffic_ops/app/lib/Fixtures/Integration/Deliveryservice.pm
+++ b/traffic_ops/app/lib/Fixtures/Integration/Deliveryservice.pm
@@ -47,6 +47,7 @@ my %definition_for = (
type => '16',
dns_bypass_ip6 => undef,
dscp => '40',
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
mid_header_rewrite => 'cond %{REMAP_PSEUDO_HOOK} __RETURN__ set-config proxy.config.http.parent_origin.dead_server_retry_enabled 1',
active => '1',
@@ -121,6 +122,7 @@ my %definition_for = (
max_dns_answers => '0',
miss_lat => '41.881944',
dscp => '40',
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
mid_header_rewrite => undef,
regex_remap => undef,
@@ -143,6 +145,7 @@ my %definition_for = (
qstring_ignore => '0',
ccr_dns_ttl => '3600',
dscp => '40',
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
last_updated => '2015-12-10 15:44:37',
org_server_fqdn => 'http://images.origin.kabletown.net',
@@ -194,6 +197,7 @@ my %definition_for = (
mid_header_rewrite =>
'cond %{REMAP_PSEUDO_HOOK} __RETURN__ set-config proxy.config.http.parent_origin.dead_server_retry_enabled 1__RETURN__ set-config proxy.config.http.parent_origin.simple_retry_enabled 1__RETURN__ set-config proxy.config.http.parent_origin.simple_retry_response_codes "400,404,412"__RETURN__ set-config proxy.config.http.parent_origin.dead_server_retry_response_codes "502,503" __RETURN__ set-config proxy.config.http.connect_attempts_timeout 2 __RETURN__ set-config proxy.config.http.connect_attempts_max_retries 2 __RETURN__ set-config proxy.config.http.connect_attempts_max_retries_dead_server 1__RETURN__ set-config proxy.config.http.transaction_active_timeout_in 5 [L]',
dscp => '40',
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
info_url => 'http://movies.info.kabletown.net',
last_updated => '2015-12-10 15:44:37',
@@ -251,6 +255,7 @@ my %definition_for = (
regex_remap => undef,
dns_bypass_ip6 => undef,
dscp => '40',
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
range_request_handling => '0',
display_name => 'tv-c1',
@@ -305,6 +310,7 @@ my %definition_for = (
check_path => '/crossdomain.xml',
dns_bypass_cname => undef,
dscp => '40',
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
remap_text => undef,
global_max_mbps => '0',
@@ -356,6 +362,7 @@ my %definition_for = (
ccr_dns_ttl => '3600',
dns_bypass_cname => undef,
dscp => '40',
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
global_max_tps => '0',
initial_dispersion => '1',
@@ -394,6 +401,7 @@ my %definition_for = (
signing_algorithm => undef,
dns_bypass_cname => undef,
dscp => '40',
+ deep_caching_type => 'NEVER',
routing_name => 'foo',
geo_limit => '0',
initial_dispersion => '1',
diff --git a/traffic_ops/app/lib/Schema/Result/Deliveryservice.pm b/traffic_ops/app/lib/Schema/Result/Deliveryservice.pm
index f40cd87ea8..b458b402a1 100644
--- a/traffic_ops/app/lib/Schema/Result/Deliveryservice.pm
+++ b/traffic_ops/app/lib/Schema/Result/Deliveryservice.pm
@@ -49,7 +49,6 @@ __PACKAGE__->table("deliveryservice");
=head2 signing_algorithm
data_type: 'deliveryservice_signature_type'
- default_value: null
is_nullable: 1
=head2 qstring_ignore
@@ -297,9 +296,16 @@ __PACKAGE__->table("deliveryservice");
=head2 routing_name
data_type: 'text'
- default_value: 'ds'
+ default_value: 'cdn'
is_nullable: 0
+=head2 deep_caching_type
+
+ data_type: 'enum'
+ default_value: 'NEVER'
+ extra: {custom_type_name => "deep_caching_type",list => ["NEVER","ALWAYS"]}
+ is_nullable: 1
+
=cut
__PACKAGE__->add_columns(
@@ -317,11 +323,7 @@ __PACKAGE__->add_columns(
"dscp",
{ data_type => "bigint", is_nullable => 0 },
"signing_algorithm",
- {
- data_type => "deliveryservice_signature_type",
- default_value => \"null",
- is_nullable => 1,
- },
+ { data_type => "deliveryservice_signature_type", is_nullable => 1 },
"qstring_ignore",
{ data_type => "smallint", is_nullable => 1 },
"geo_limit",
@@ -418,7 +420,17 @@ __PACKAGE__->add_columns(
"tenant_id",
{ data_type => "bigint", is_foreign_key => 1, is_nullable => 1 },
"routing_name",
- { data_type => "text", default_value => "ds", is_nullable => 0 },
+ { data_type => "text", default_value => "cdn", is_nullable => 0 },
+ "deep_caching_type",
+ {
+ data_type => "enum",
+ default_value => "NEVER",
+ extra => {
+ custom_type_name => "deep_caching_type",
+ list => ["NEVER", "ALWAYS"],
+ },
+ is_nullable => 1,
+ },
);
=head1 PRIMARY KEY
@@ -603,7 +615,7 @@ __PACKAGE__->has_many(
{ cascade_copy => 0, cascade_delete => 0 },
);
-=head2 steering_target_deliveryservices_2s
+=head2 steering_target_targets
Type: has_many
@@ -612,9 +624,9 @@ Related object: L<Schema::Result::SteeringTarget>
=cut
__PACKAGE__->has_many(
- "steering_target_deliveryservices_2s",
+ "steering_target_targets",
"Schema::Result::SteeringTarget",
- { "foreign.deliveryservice" => "self.id" },
+ { "foreign.target" => "self.id" },
{ cascade_copy => 0, cascade_delete => 0 },
);
@@ -654,8 +666,8 @@ __PACKAGE__->belongs_to(
);
-# Created by DBIx::Class::Schema::Loader v0.07042 @ 2017-08-09 15:16:31
-# DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:FXNj6wNNq+jajg/rVqOSBw
+# Created by DBIx::Class::Schema::Loader v0.07042 @ 2017-11-09 14:44:59
+# DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:IA1IQ3w/BEYermjf9PoVtA
# You can replace this text with custom code or comments, and it will be preserved on regeneration
#
diff --git a/traffic_ops/app/lib/UI/DeliveryService.pm b/traffic_ops/app/lib/UI/DeliveryService.pm
index c9fac43758..f1c47d9bc1 100644
--- a/traffic_ops/app/lib/UI/DeliveryService.pm
+++ b/traffic_ops/app/lib/UI/DeliveryService.pm
@@ -255,6 +255,7 @@ sub read {
"initial_dispersion" => $row->initial_dispersion,
"regional_geo_blocking" => $row->regional_geo_blocking,
"logs_enabled" => \$row->logs_enabled,
+ "deep_caching_type" => $row->deep_caching_type,
}
);
}
@@ -835,6 +836,7 @@ sub update {
remap_text => $self->paramAsScalar( 'ds.remap_text', undef ),
initial_dispersion => $self->paramAsScalar( 'ds.initial_dispersion', 1 ),
logs_enabled => $self->paramAsScalar('ds.logs_enabled'),
+ deep_caching_type => $self->paramAsScalar('ds.deep_caching_type'),
);
my $typename = $self->typename();
@@ -1073,6 +1075,7 @@ sub create {
initial_dispersion => $self->paramAsScalar( 'ds.initial_dispersion', 1 ),
logs_enabled => $self->paramAsScalar('ds.logs_enabled'),
tenant_id => $tenant_id,
+ deep_caching_type => $self->paramAsScalar('ds.deep_caching_type'),
};
my $insert = $self->db->resultset('Deliveryservice')->create($new_ds);
diff --git a/traffic_ops/app/lib/UI/Topology.pm b/traffic_ops/app/lib/UI/Topology.pm
index e22addd8e8..93c176b7ca 100644
--- a/traffic_ops/app/lib/UI/Topology.pm
+++ b/traffic_ops/app/lib/UI/Topology.pm
@@ -399,6 +399,8 @@ sub gen_crconfig_json {
$data_obj->{'deliveryServices'}->{ $row->xml_id }->{'geoEnabled'} = $geoEnabled;
}
+ $data_obj->{'deliveryServices'}->{ $row->xml_id }->{'deepCachingType'} = $row->deep_caching_type;
+
# Default to 'http only'
$data_obj->{'deliveryServices'}->{ $row->xml_id }->{'sslEnabled'} = 'false';
$data_obj->{'deliveryServices'}->{ $row->xml_id }->{'protocol'}->{'acceptHttps'} = 'false';
@@ -705,6 +707,9 @@ sub stringify_ds {
if ( defined( $ds->{'missLocation'} ) ) {
$string .= "|GeoMiss: " . $ds->{'missLocation'}->{'lat'} . "," . $ds->{'missLocation'}->{'long'};
}
+ if (defined( $ds->{'deepCachingType'} ) ) {
+ $string .= "|deepCachingType: " . $ds->{'deepCachingType'};
+ }
if ( defined( $ds->{'bypassDestination'} ) ) {
$string .= "<br>|BypassDest:";
if ( defined( $ds->{'bypassDestination'}->{'DNS'}->{'ip'} ) ) {
diff --git a/traffic_ops/app/t/api/1.2/deliveryservice.t b/traffic_ops/app/t/api/1.2/deliveryservice.t
index e73a7c52c7..c34eb76308 100644
--- a/traffic_ops/app/t/api/1.2/deliveryservice.t
+++ b/traffic_ops/app/t/api/1.2/deliveryservice.t
@@ -82,6 +82,7 @@ $t->get_ok('/api/1.2/deliveryservices/100/servers/eligible')->status_is(200)->$c
ok $t->get_ok("/api/1.2/deliveryservices")->status_is(200)->or( sub { diag $t->tx->res->content->asset->{content} } )
->json_is( "/response/0/xmlId", "steering-ds1" )
->json_is( "/response/0/routingName", "foo" )
+ ->json_is( "/response/0/deepCachingType", "NEVER" )
->json_is( "/response/0/logsEnabled", 0 )
->json_is( "/response/0/ipv6RoutingEnabled", 1 )
->json_is( "/response/1/xmlId", "steering-ds2" );
@@ -205,6 +206,7 @@ ok $t->get_ok("/api/1.2/deliveryservices?logsEnabled=true")->status_is(200)->or(
ok $t->post_ok('/api/1.2/deliveryservices' => {Accept => 'application/json'} => json => {
"active" => \0,
"cdnId" => 100,
+ "deepCachingType" => "NEVER",
"displayName" => "ds_displayname_1",
"dscp" => 0,
"geoLimit" => 0,
@@ -228,6 +230,7 @@ ok $t->get_ok("/api/1.2/deliveryservices?logsEnabled=true")->status_is(200)->or(
->json_is( "/response/0/tenantId" => $tenant_id)
->json_is( "/response/0/cdnName" => "cdn1")
->json_is( "/response/0/routingName" => "foo")
+ ->json_is( "/response/0/deepCachingType" => "NEVER")
->json_is( "/response/0/displayName" => "ds_displayname_1")
->json_is( "/response/0/xmlId" => "ds_1")
->json_is( "/response/0/multiSiteOrigin" => 0)
@@ -240,6 +243,7 @@ ok $t->get_ok("/api/1.2/deliveryservices?logsEnabled=true")->status_is(200)->or(
ok $t->put_ok('/api/1.2/deliveryservices/' . $ds_id => {Accept => 'application/json'} => json => {
"active" => \1,
"cdnId" => 100,
+ "deepCachingType" => "NEVER",
"displayName" => "ds_displayname_11",
"dscp" => 1,
"geoLimit" => 1,
@@ -265,6 +269,7 @@ ok $t->get_ok("/api/1.2/deliveryservices?logsEnabled=true")->status_is(200)->or(
->json_is( "/response/0/displayName" => "ds_displayname_11")
->json_is( "/response/0/xmlId" => "ds_1")
->json_is( "/response/0/routingName" => "bar")
+ ->json_is( "/response/0/deepCachingType" => "NEVER")
->json_is( "/response/0/multiSiteOrigin" => 0)
->json_is( "/response/0/orgServerFqdn" => "http://10.75.168.91")
->json_is( "/response/0/protocol" => 2)
@@ -277,6 +282,7 @@ ok $t->get_ok("/api/1.2/deliveryservices?logsEnabled=true")->status_is(200)->or(
"displayName" => "ds_displayname_11",
"dscp" => 1,
"routingName" => "baz",
+ "deepCachingType" => "NEVER",
"geoLimit" => 1,
"geoProvider" => 1,
"ipv6RoutingEnabled" => 1,
@@ -306,6 +312,7 @@ ok $t->get_ok("/api/1.2/deliveryservices?logsEnabled=true")->status_is(200)->or(
"displayName" => "ds_displayname_11",
"dscp" => 1,
"routingName" => "foo",
+ "deepCachingType" => "NEVER",
"geoLimit" => 1,
"geoProvider" => 1,
"initialDispersion" => 2,
diff --git a/traffic_ops/app/t/api/1.2/deliveryservice_crud.t b/traffic_ops/app/t/api/1.2/deliveryservice_crud.t
index 7dbe1de7ee..fe666d0619 100644
--- a/traffic_ops/app/t/api/1.2/deliveryservice_crud.t
+++ b/traffic_ops/app/t/api/1.2/deliveryservice_crud.t
@@ -62,6 +62,7 @@ sub run_ut {
ok $t->get_ok("/api/1.2/deliveryservices")->status_is(200)->or( sub { diag $t->tx->res->content->asset->{content} } )
->json_is( "/response/0/xmlId", "steering-ds1" )
->json_is( "/response/0/routingName", "foo" )
+ ->json_is( "/response/0/deepCachingType", "NEVER")
->json_is( "/response/0/logsEnabled", 0 )
->json_is( "/response/0/ipv6RoutingEnabled", 1 )
->json_is( "/response/1/xmlId", "steering-ds2" );
@@ -143,6 +144,7 @@ sub run_ut {
"active" => "false",
"dscp" => 0,
"routingName" => "foo",
+ "deepCachingType" => "NEVER",
"ipv6RoutingEnabled" => "true",
"logsEnabled" => "true",
"initialDispersion" => 1,
@@ -159,6 +161,7 @@ sub run_ut {
->json_is( "/response/0/orgServerFqdn" => "http://10.75.168.91")
->json_is( "/response/0/cdnId" => 100)
->json_is( "/response/0/routingName" => "foo")
+ ->json_is( "/response/0/deepCachingType" => "NEVER")
->json_is( "/response/0/tenantId" => $tenant_id)
->json_is( "/response/0/profileId" => 300)
->json_is( "/response/0/protocol" => "1")
@@ -177,6 +180,7 @@ sub run_ut {
->json_is( "/response/0/orgServerFqdn" => "http://10.75.168.91")
->json_is( "/response/0/cdnId" => 100)
->json_is( "/response/0/routingName" => "foo")
+ ->json_is( "/response/0/deepCachingType" => "NEVER")
->json_is( "/response/0/tenantId" => $tenant_id)
->json_is( "/response/0/tenant" => $tenant_name)
->json_is( "/response/0/profileId" => 300)
@@ -204,6 +208,7 @@ sub run_ut {
"active" => "false",
"dscp" => 0,
"routingName" => "foo",
+ "deepCachingType" => "NEVER",
"ipv6RoutingEnabled" => "true",
"logsEnabled" => "true",
"initialDispersion" => 1,
@@ -246,6 +251,7 @@ sub run_ut {
"active" => "false",
"dscp" => 0,
"routingName" => "foo",
+ "deepCachingType" => "NEVER",
"ipv6RoutingEnabled" => "true",
"logsEnabled" => "true",
"initialDispersion" => 1,
@@ -279,6 +285,7 @@ sub run_ut {
"active" => "false",
"dscp" => 0,
"routingName" => "foo",
+ "deepCachingType" => "NEVER",
"ipv6RoutingEnabled" => "true",
"logsEnabled" => "true",
"initialDispersion" => 1,
@@ -312,6 +319,7 @@ sub run_ut {
"active" => "false",
"dscp" => 0,
"routingName" => "foo",
+ "deepCachingType" => "NEVER",
"ipv6RoutingEnabled" => "true",
"logsEnabled" => "true",
"initialDispersion" => 1,
@@ -345,6 +353,7 @@ sub run_ut {
"active" => "false",
"dscp" => 0,
"routingName" => "foo",
+ "deepCachingType" => "NEVER",
"ipv6RoutingEnabled" => "true",
"logsEnabled" => "true",
"initialDispersion" => 1,
@@ -379,6 +388,7 @@ sub run_ut {
"active" => "false",
"dscp" => 0,
"routingName" => "foo",
+ "deepCachingType" => "NEVER",
"ipv6RoutingEnabled" => "true",
"logsEnabled" => "true",
"initialDispersion" => 1,
@@ -413,6 +423,7 @@ sub run_ut {
"active" => "false",
"dscp" => 0,
"routingName" => "foo",
+ "deepCachingType" => "NEVER",
"ipv6RoutingEnabled" => "true",
"logsEnabled" => "true",
"initialDispersion" => 1,
@@ -463,6 +474,7 @@ sub run_ut {
"active" => "false",
"dscp" => 0,
"routingName" => "foo",
+ "deepCachingType" => "NEVER",
"ipv6RoutingEnabled" => "true",
"logsEnabled" => "true",
"initialDispersion" => 1,
@@ -509,6 +521,7 @@ sub run_ut {
"active" => "false",
"dscp" => 0,
"routingName" => "foo",
+ "deepCachingType" => "NEVER",
"ipv6RoutingEnabled" => "true",
"logsEnabled" => "true",
"initialDispersion" => 1,
diff --git a/traffic_ops/app/t/deliveryservice.t b/traffic_ops/app/t/deliveryservice.t
index e580d2c436..c6b76ce823 100644
--- a/traffic_ops/app/t/deliveryservice.t
+++ b/traffic_ops/app/t/deliveryservice.t
@@ -63,6 +63,7 @@ ok $t->post_ok(
'ds.dns_bypass_ttl' => '30',
'ds.dscp' => '40',
'ds.routing_name' => 'foo',
+ 'ds.deep_caching_type' => 'NEVER',
'ds.geo_limit' => '0',
'ds.geo_limit_countries' => '',
'ds.geo_provider' => '1',
@@ -115,6 +116,7 @@ ok $t->post_ok(
'ds.dns_bypass_ttl' => '30',
'ds.dscp' => '42',
'ds.routing_name' => '',
+ 'ds.deep_caching_type' => 'NEVER',
'ds.geo_limit' => '0',
'ds.geo_limit_countries' => '',
'ds.global_max_mbps' => '',
@@ -166,6 +168,7 @@ ok $t->post_ok(
'ds.dns_bypass_ttl' => '30',
'ds.dscp' => '40',
'ds.routing_name' => 'foo',
+ 'ds.deep_caching_type' => 'NEVER',
'ds.geo_limit' => '1',
'ds.geo_limit_countries' => '',
'ds.global_max_mbps' => '30G',
@@ -246,6 +249,7 @@ ok $t->post_ok(
'ds.dns_bypass_ttl' => '31',
'ds.dscp' => '41',
'ds.routing_name' => 'bar',
+ 'ds.deep_caching_type' => 'NEVER',
'ds.geo_limit' => '2',
'ds.geo_limit_countries' => '',
'ds.geo_provider' => '1',
@@ -294,6 +298,7 @@ ok $t->get_ok('/datadeliveryservice')->status_is(200)
->or( sub { diag $t->tx->res->content->asset->{content}; } )
->json_is( '/1/dscp' => '41' )
->json_is( '/1/routing_name' => 'bar' )
+ ->json_is( '/1/deep_caching_type' => 'NEVER' )
->json_is( '/1/active' => '0' )
->json_is( '/1/profile_description' => 'mid description' )
->json_is( '/1/org_server_fqdn' => 'http://update.knutsel.com' )
diff --git a/traffic_ops/app/templates/delivery_service/_form.html.ep b/traffic_ops/app/templates/delivery_service/_form.html.ep
index 36cff33d05..b2e0156bbc 100644
--- a/traffic_ops/app/templates/delivery_service/_form.html.ep
+++ b/traffic_ops/app/templates/delivery_service/_form.html.ep
@@ -617,4 +617,16 @@
<% } %>
<% } %>
</div>
+ <div class="block form-row" id="deep_caching_type">
+ <% unless (field('ds.deep_caching_type')->valid) { %>
+ <span class="field-with-error"><%= field('ds.deep_caching_type')->error %></span>
+ <% } %>
+ %= label_for 'deep_caching_type' => '* Deep Caching', class => 'label'
+ <% if ($priv_level >= 20) { %>
+ %= field('ds.deep_caching_type')->select([["NEVER" => "NEVER"], ["ALWAYS" => "ALWAYS"]]);
+ <% } else { %>
+ %= field('ds.deep_caching_type')->text(class => 'field', readonly => 'readonly', value => $ds->deep_caching_type);
+
+ <% } %>
+ </div>
</div>
diff --git a/traffic_ops/app/templates/delivery_service/add.html.ep b/traffic_ops/app/templates/delivery_service/add.html.ep
index a04c525c44..c2f0841d6a 100644
--- a/traffic_ops/app/templates/delivery_service/add.html.ep
+++ b/traffic_ops/app/templates/delivery_service/add.html.ep
@@ -115,6 +115,7 @@
$('#range_request_handlingi_row').show(speed);
$('#dns_bypass_ip_row').hide(speed);
$('#dns_bypass_ip6_row').hide(speed);
+ $('#deep_caching_type').show(speed);
$('#regional_geo_blocking_row').show(speed);
$('#dns_bypass_cname_row').hide(speed);
$('#dns_bypass_ttl_row').show(speed);
@@ -157,6 +158,7 @@
$('#range_request_handlingi_row').show(speed);
$('#dns_bypass_ip_row').show(speed);
$('#dns_bypass_ip6_row').show(speed);
+ $('#deep_caching_type').hide(speed);
$('#regional_geo_blocking_row').hide(speed);
$('#dns_bypass_cname_row').show(speed);
$('#dns_bypass_ttl_row').show(speed);
@@ -195,6 +197,7 @@
$('#dns_bypass_ip_row').hide(speed);
$('#dns_bypass_ip6_row').hide(speed);
$('#dns_bypass_cname_row').hide(speed);
+ $('#deep_caching_type').hide(speed);
$('#regional_geo_blocking_row').show(speed);
$('#dns_bypass_ttl_row').hide(speed);
$('#max_dns_answers_row').hide(speed);
@@ -232,6 +235,7 @@
$('#range_request_handlingi_row').hide(speed);
$('#dns_bypass_ip_row').hide(speed);
$('#dns_bypass_ip6_row').hide(speed);
+ $('#deep_caching_type').hide(speed);
$('#regional_geo_blocking_row').show(speed);
$('#dns_bypass_cname_row').hide(speed);
$('#dns_bypass_ttl_row').hide(speed);
diff --git a/traffic_ops/app/templates/delivery_service/edit.html.ep b/traffic_ops/app/templates/delivery_service/edit.html.ep
index 4de0875506..699361ab27 100644
--- a/traffic_ops/app/templates/delivery_service/edit.html.ep
+++ b/traffic_ops/app/templates/delivery_service/edit.html.ep
@@ -209,6 +209,7 @@ function setup_form(speed) {
$('#range_request_handlingi_row').show(speed);
$('#dns_bypass_ip_row').hide(speed);
$('#dns_bypass_ip6_row').hide(speed);
+ $('#deep_caching_type').show(speed);
$('#regional_geo_blocking_row').show(speed);
$('#dns_bypass_cname_row').hide(speed);
$('#dns_bypass_ttl_row').show(speed);
@@ -249,6 +250,7 @@ function setup_form(speed) {
$('#range_request_handlingi_row').show(speed);
$('#dns_bypass_ip_row').show(speed);
$('#dns_bypass_ip6_row').show(speed);
+ $('#deep_caching_type').hide(speed);
$('#regional_geo_blocking_row').hide(speed);
$('#dns_bypass_cname_row').show(speed);
$('#dns_bypass_ttl_row').show(speed);
@@ -286,6 +288,7 @@ function setup_form(speed) {
$('#range_request_handlingi_row').hide(speed);
$('#dns_bypass_ip_row').hide(speed);
$('#dns_bypass_ip6_row').hide(speed);
+ $('#deep_caching_type').hide(speed);
$('#regional_geo_blocking_row').show(speed);
$('#dns_bypass_cname_row').hide(speed);
$('#dns_bypass_ttl_row').hide(speed);
@@ -324,6 +327,7 @@ function setup_form(speed) {
$('#range_request_handlingi_row').hide(speed);
$('#dns_bypass_ip_row').hide(speed);
$('#dns_bypass_ip6_row').hide(speed);
+ $('#deep_caching_type').hide(speed);
$('#regional_geo_blocking_row').show(speed);
$('#dns_bypass_cname_row').hide(speed);
$('#dns_bypass_ttl_row').hide(speed);
diff --git a/traffic_ops/client_tests/fixtures/delivery_service.go b/traffic_ops/client_tests/fixtures/delivery_service.go
index dd869e82c3..f7c7cc978b 100644
--- a/traffic_ops/client_tests/fixtures/delivery_service.go
+++ b/traffic_ops/client_tests/fixtures/delivery_service.go
@@ -26,6 +26,7 @@ func DeliveryServices() *client.GetDeliveryServiceResponse {
XMLID: "ds-test",
Active: true,
DSCP: 40,
+ DeepCachingType: "NEVER",
RoutingName: "edge",
Signed: false,
QStringIgnore: 1,
diff --git a/traffic_ops/client_tests/fixtures/traffic_router_config.go b/traffic_ops/client_tests/fixtures/traffic_router_config.go
index 4dca715a98..82ee5fe5db 100644
--- a/traffic_ops/client_tests/fixtures/traffic_router_config.go
+++ b/traffic_ops/client_tests/fixtures/traffic_router_config.go
@@ -8,9 +8,9 @@ package fixtures
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
- *
+ *
* http://www.apache.org/licenses/LICENSE-2.0
- *
+ *
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -19,7 +19,6 @@ package fixtures
* under the License.
*/
-
import "github.com/apache/incubator-trafficcontrol/traffic_ops/client"
// TrafficRouterConfig returns a default TRConfigResponse to be used for testing.
@@ -114,7 +113,8 @@ func TrafficRouterConfig() *client.TRConfigResponse {
Domains: []string{
"ga.atlanta.kabletown.net",
},
- RoutingName: "edge",
+ RoutingName: "edge",
+ DeepCachingType: "NEVER",
MissLocation: client.MissLocation{
Latitude: 75,
Longitude: 65,
diff --git a/traffic_ops/client_tests/tests/integration/deliveryservice_test.go b/traffic_ops/client_tests/tests/integration/deliveryservice_test.go
index 2aefc4d2a4..b2fb797218 100644
--- a/traffic_ops/client_tests/tests/integration/deliveryservice_test.go
+++ b/traffic_ops/client_tests/tests/integration/deliveryservice_test.go
@@ -62,6 +62,7 @@ func init() {
testDs.CDNID = cdn.ID
testDs.CacheURL = "cacheURL"
testDs.CheckPath = "CheckPath"
+ testDs.DeepCachingType = "NEVER"
testDs.DNSBypassCname = "DNSBypassCNAME"
testDs.DNSBypassIP = "10.10.10.10"
testDs.DNSBypassIP6 = "FF01:0:0:0:0:0:0:FB"
@@ -94,6 +95,7 @@ func init() {
testDs.RangeRequestHandling = 0
testDs.RegexRemap = "regexRemap"
testDs.RemapText = "remapText"
+ testDs.RoutingName = "edge"
testDs.Signed = false
testDs.TRResponseHeaders = "TRResponseHeaders"
testDs.Type = dsType.Name
diff --git a/traffic_portal/app/src/common/modules/form/deliveryService/FormDeliveryServiceController.js b/traffic_portal/app/src/common/modules/form/deliveryService/FormDeliveryServiceController.js
index 71927fc939..e554e76780 100644
--- a/traffic_portal/app/src/common/modules/form/deliveryService/FormDeliveryServiceController.js
+++ b/traffic_portal/app/src/common/modules/form/deliveryService/FormDeliveryServiceController.js
@@ -128,6 +128,11 @@ var FormDeliveryServiceController = function(deliveryService, type, types, $scop
{ value: 56, label: '56 - CS7' }
];
+ $scope.deepCachingTypes = [
+ { value: 'NEVER', label: 'NEVER' },
+ { value: 'ALWAYS', label: 'ALWAYS' }
+ ]
+
$scope.dispersions = [
{ value: 1, label: '1 - OFF' },
{ value: 2, label: '2' },
diff --git a/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.HTTP.tpl.html b/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.HTTP.tpl.html
index b651434c64..a36afe6a94 100644
--- a/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.HTTP.tpl.html
+++ b/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.HTTP.tpl.html
@@ -168,6 +168,16 @@
</div>
</div>
+ <div class="form-group" ng-class="{'has-error': hasError(deliveryServiceForm.deepCachingType), 'has-feedback': hasError(deliveryServiceForm.deepCachingType)}">
+ <label class="control-label col-md-2 col-sm-2 col-xs-12">Deep Caching *</label>
+ <div class="col-md-10 col-sm-10 col-xs-12">
+ <select id="deepCachingType" name="deepCachingType" class="form-control" ng-model="deliveryService.deepCachingType" ng-options="dct.value as dct.label for dct in deepCachingTypes" required>
+ <option value="">Select...</option>
+ </select>
+ <small class="input-error" ng-show="hasPropertyError(deliveryServiceForm.deepCachingType, 'required')">Required</small>
+ </div>
+ </div>
+
<div class="form-group" ng-class="{'has-error': hasError(deliveryServiceForm.dscp), 'has-feedback': hasError(deliveryServiceForm.dscp)}">
<label class="control-label col-md-2 col-sm-2 col-xs-12">DSCP Tag *</label>
<div class="col-md-10 col-sm-10 col-xs-12">
diff --git a/traffic_portal/app/src/traffic_portal_properties.json b/traffic_portal/app/src/traffic_portal_properties.json
index d1bb1f03ba..2935001299 100644
--- a/traffic_portal/app/src/traffic_portal_properties.json
+++ b/traffic_portal/app/src/traffic_portal_properties.json
@@ -83,6 +83,7 @@
},
"HTTP": {
"routingName": "cdn",
+ "deepCachingType": "NEVER",
"dscp": 0,
"ipv6RoutingEnabled": false,
"rangeRequestHandling": 0,
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheLocation.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheLocation.java
index e2bf78d469..67159c1162 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheLocation.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheLocation.java
@@ -23,6 +23,7 @@
import org.apache.commons.lang3.builder.EqualsBuilder;
import org.apache.commons.lang3.builder.HashCodeBuilder;
+import org.apache.log4j.Logger;
import com.comcast.cdn.traffic_control.traffic_router.geolocation.Geolocation;
@@ -31,6 +32,8 @@
*/
public class CacheLocation {
+ public static final Logger LOGGER = Logger.getLogger(CacheLocation.class);
+
private final String id;
private final Geolocation geolocation;
@@ -57,7 +60,29 @@ public CacheLocation(final String id, final Geolocation geolocation) {
* the cache to add
*/
public void addCache(final Cache cache) {
+ synchronized (caches) {
caches.put(cache.getId(), cache);
+ }
+ }
+
+ public void clearCaches() {
+ synchronized (caches) {
+ caches.clear();
+ }
+ }
+
+ public void loadDeepCaches(final Set<String> deepCacheNames, final CacheRegister cacheRegister) {
+ synchronized (caches) {
+ if (caches.isEmpty() && deepCacheNames != null) {
+ for (final String deepCacheName : deepCacheNames) {
+ final Cache deepCache = cacheRegister.getCacheMap().get(deepCacheName);
+ if (deepCache != null) {
+ LOGGER.debug("DDC: Adding " + deepCacheName + " to " + getId());
+ caches.put(deepCache.getId(), deepCache);
+ }
+ }
+ }
+ }
}
@Override
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandler.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandler.java
index a657e23e12..c13b4e6e61 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandler.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandler.java
@@ -36,6 +36,7 @@
import com.comcast.cdn.traffic_control.traffic_router.core.loc.GeolocationDatabaseUpdater;
import com.comcast.cdn.traffic_control.traffic_router.core.loc.NetworkNode;
import com.comcast.cdn.traffic_control.traffic_router.core.loc.NetworkUpdater;
+import com.comcast.cdn.traffic_control.traffic_router.core.loc.DeepNetworkUpdater;
import com.comcast.cdn.traffic_control.traffic_router.core.loc.RegionalGeoUpdater;
import com.comcast.cdn.traffic_control.traffic_router.core.secure.CertificatesPoller;
@@ -76,6 +77,7 @@
private TrafficOpsUtils trafficOpsUtils;
private NetworkUpdater networkUpdater;
+ private DeepNetworkUpdater deepNetworkUpdater;
private FederationsWatcher federationsWatcher;
private RegionalGeoUpdater regionalGeoUpdater;
private SteeringWatcher steeringWatcher;
@@ -102,6 +104,9 @@ public GeolocationDatabaseUpdater getGeolocationDatabaseUpdater() {
public NetworkUpdater getNetworkUpdater () {
return networkUpdater;
}
+ public DeepNetworkUpdater getDeepNetworkUpdater () {
+ return deepNetworkUpdater;
+ }
public RegionalGeoUpdater getRegionalGeoUpdater() {
return regionalGeoUpdater;
@@ -141,6 +146,7 @@ public boolean processConfig(final String jsonStr) throws JsonUtilsException, IO
try {
parseGeolocationConfig(config);
parseCoverageZoneNetworkConfig(config);
+ parseDeepCoverageZoneNetworkConfig(config);
parseRegionalGeoConfig(jo);
final CacheRegister cacheRegister = new CacheRegister();
@@ -214,8 +220,13 @@ public boolean processConfig(final String jsonStr) throws JsonUtilsException, IO
* never have traffic routed to it, as the old List<Cache> does not contain the Cache that was moved to ONLINE.
* NetworkNode is a singleton and is managed asynchronously. As long as we swap out the CacheRegister first,
* then clear cache locations, the lazy loading should work as designed. See issue TC-401 for details.
+ *
+ * Update for DDC (Dynamic Deep Caching): NetworkNode now has a 2nd singleton (deepInstance) that is managed
+ * similarly to the non-deep instance. However, instead of clearing a NetworkNode's CacheLocation, only the
+ * Caches are cleared from the CacheLocation then lazily loaded at request time.
*/
NetworkNode.getInstance().clearCacheLocations();
+ NetworkNode.getDeepInstance().clearCacheLocations(true);
setLastSnapshotTimestamp(sts);
} catch (ParseException e) {
isProcessing.set(false);
@@ -251,6 +262,9 @@ public void setGeolocationDatabaseUpdater(final GeolocationDatabaseUpdater geolo
public void setNetworkUpdater(final NetworkUpdater nu) {
this.networkUpdater = nu;
}
+ public void setDeepNetworkUpdater(final DeepNetworkUpdater dnu) {
+ this.deepNetworkUpdater = dnu;
+ }
public void setRegionalGeoUpdater(final RegionalGeoUpdater regionalGeoUpdater) {
this.regionalGeoUpdater = regionalGeoUpdater;
@@ -541,6 +555,13 @@ private void parseCoverageZoneNetworkConfig(final JsonNode config) throws JsonUt
);
}
+ private void parseDeepCoverageZoneNetworkConfig(final JsonNode config) throws JsonUtilsException {
+ getDeepNetworkUpdater().setDataBaseURL(
+ JsonUtils.optString(config, "deepcoveragezone.polling.url", null),
+ JsonUtils.optLong(config, "deepcoveragezone.polling.interval")
+ );
+ }
+
private void parseRegionalGeoConfig(final JsonNode jo) throws JsonUtilsException {
final JsonNode config = JsonUtils.getJsonNode(jo, "config");
final String url = JsonUtils.optString(config, "regional_geoblock.polling.url", null);
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryService.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryService.java
index 33cd4067e3..4aadd70d2b 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryService.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryService.java
@@ -93,6 +93,12 @@
private final boolean acceptHttp;
private final boolean acceptHttps;
private final boolean redirectToHttps;
+ private final DeepCachingType deepCache;
+
+ public enum DeepCachingType {
+ NEVER,
+ ALWAYS
+ }
public DeliveryService(final String id, final JsonNode dsJo) throws JsonUtilsException {
this.id = id;
@@ -144,7 +150,15 @@ public DeliveryService(final String id, final JsonNode dsJo) throws JsonUtilsExc
acceptHttp = JsonUtils.optBoolean(protocol, "acceptHttp", true);
acceptHttps = JsonUtils.optBoolean(protocol, "acceptHttps");
redirectToHttps = JsonUtils.optBoolean(protocol, "redirectToHttps");
-
+ final String dctString = JsonUtils.optString(dsJo, "deepCachingType", "NEVER").toUpperCase();
+ DeepCachingType dct = DeepCachingType.NEVER;
+ try {
+ dct = DeepCachingType.valueOf(dctString);
+ } catch (IllegalArgumentException e) {
+ LOGGER.error("DeliveryService '" + id + "' has an unrecognized deepCachingType: '" + dctString + "'. Defaulting to 'NEVER' instead");
+ } finally {
+ this.deepCache = dct;
+ }
}
public String getId() {
@@ -389,6 +403,11 @@ public void setDns(final boolean isDns) {
this.isDns = isDns;
}
+ public DeepCachingType getDeepCache() {
+ return deepCache;
+ }
+
+
public boolean appendQueryString() {
return shouldAppendQueryString;
}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/DeepNetworkUpdater.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/DeepNetworkUpdater.java
new file mode 100644
index 0000000000..c978b6349e
--- /dev/null
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/DeepNetworkUpdater.java
@@ -0,0 +1,35 @@
+/*
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.comcast.cdn.traffic_control.traffic_router.core.loc;
+
+import java.io.File;
+import java.io.IOException;
+
+
+public class DeepNetworkUpdater extends NetworkUpdater {
+
+ public DeepNetworkUpdater() {
+ sourceCompressed = false;
+ tmpPrefix = "deepczf";
+ tmpSuffix = ".json";
+ }
+
+ @Override
+ public NetworkNode generateTree(final File dbFile, final boolean verifyOnly) throws IOException {
+ return NetworkNode.generateTree(dbFile, verifyOnly, true);
+ }
+
+}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNode.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNode.java
index 77dc22ba82..9d292aeb22 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNode.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNode.java
@@ -20,11 +20,12 @@
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.Iterator;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
import java.util.Map;
+import java.util.Set;
import java.util.TreeMap;
-import java.util.List;
-import java.util.ArrayList;
-
import com.comcast.cdn.traffic_control.traffic_router.core.util.CidrAddress;
import com.comcast.cdn.traffic_control.traffic_router.core.util.JsonUtils;
@@ -42,12 +43,14 @@
private static final String DEFAULT_SUB_STR = "0.0.0.0/0";
private static NetworkNode instance;
+ private static NetworkNode deepInstance;
private CidrAddress cidrAddress;
private String loc;
private CacheLocation cacheLocation = null;
private Geolocation geolocation = null;
protected Map<NetworkNode,NetworkNode> children;
+ private Set<String> deepCacheNames;
public static NetworkNode getInstance() {
if (instance != null) {
@@ -63,15 +66,38 @@ public static NetworkNode getInstance() {
return instance;
}
- public static NetworkNode generateTree(final File f, final boolean verifyOnly) throws IOException {
+ public static NetworkNode getDeepInstance() {
+ if (deepInstance != null) {
+ return deepInstance;
+ }
+
+ try {
+ deepInstance = new NetworkNode(DEFAULT_SUB_STR);
+ } catch (NetworkNodeException e) {
+ LOGGER.warn(e);
+ }
+
+ return deepInstance;
+ }
+
+ public static NetworkNode generateTree(final File f, final boolean verifyOnly, final boolean useDeep) throws IOException {
final ObjectMapper mapper = new ObjectMapper();
- return generateTree(mapper.readTree(f), verifyOnly);
+ return generateTree(mapper.readTree(f), verifyOnly, useDeep);
+ }
+
+ public static NetworkNode generateTree(final File f, final boolean verifyOnly) throws IOException {
+ return generateTree(f, verifyOnly, false);
}
- @SuppressWarnings("PMD.CyclomaticComplexity")
public static NetworkNode generateTree(final JsonNode json, final boolean verifyOnly) {
+ return generateTree(json, verifyOnly, false);
+ }
+
+ @SuppressWarnings({"PMD.CyclomaticComplexity", "PMD.NPathComplexity"})
+ public static NetworkNode generateTree(final JsonNode json, final boolean verifyOnly, final boolean useDeep) {
try {
- final JsonNode coverageZones = JsonUtils.getJsonNode(json, "coverageZones");
+ final String czKey = useDeep ? "deepCoverageZones" : "coverageZones";
+ final JsonNode coverageZones = JsonUtils.getJsonNode(json, czKey);
final SuperNode root = new SuperNode();
@@ -88,39 +114,17 @@ public static NetworkNode generateTree(final JsonNode json, final boolean verify
geolocation = new Geolocation(latitude, longitude);
}
- try {
- for (final JsonNode network6 : JsonUtils.getJsonNode(locData, "network6")) {
- final String ip = network6.asText();
-
- try {
- root.add6(new NetworkNode(ip, loc, geolocation));
- } catch (NetworkNodeException ex) {
- LOGGER.error(ex, ex);
- return null;
- }
- }
- } catch (JsonUtilsException ex) {
- LOGGER.warn("An exception was caught while accessing the network6 key of " + loc + " in the incoming coverage zone file: " + ex.getMessage());
- }
-
- try {
- for (final JsonNode network : JsonUtils.getJsonNode(locData, "network")) {
- final String ip = network.asText();
-
- try {
- root.add(new NetworkNode(ip, loc, geolocation));
- } catch (NetworkNodeException ex) {
- LOGGER.error(ex, ex);
- return null;
- }
- }
- } catch (JsonUtilsException ex) {
- LOGGER.warn("An exception was caught while accessing the network key of " + loc + " in the incoming coverage zone file: " + ex.getMessage());
+ if (!addNetworkNodesToRoot(root, loc, locData, geolocation, useDeep)) {
+ return null;
}
}
if (!verifyOnly) {
- instance = root;
+ if (useDeep) {
+ deepInstance = root;
+ } else {
+ instance = root;
+ }
}
return root;
@@ -133,6 +137,62 @@ public static NetworkNode generateTree(final JsonNode json, final boolean verify
return null;
}
+ private static boolean addNetworkNodesToRoot(final SuperNode root, final String loc, final JsonNode locData,
+ final Geolocation geolocation, final boolean useDeep) {
+ final CacheLocation deepLoc = new CacheLocation( "deep." + loc, geolocation != null ? geolocation : new Geolocation(0.0, 0.0)); // TODO JvD
+ final Set<String> cacheNames = parseDeepCacheNames(locData);
+
+ for (final String key : new String[]{"network6", "network"}) {
+ try {
+ for (final JsonNode network : JsonUtils.getJsonNode(locData, key)) {
+ final String ip = network.asText();
+
+ try {
+ final NetworkNode nn = new NetworkNode(ip, loc, geolocation);
+ if (useDeep) {
+ // For a deep NetworkNode, we set the CacheLocation here without any Caches.
+ // The deep Caches will be lazily loaded in getCoverageZoneCacheLocation() where we have
+ // access to the latest CacheRegister, similar to how normal NetworkNodes are lazily loaded
+ // with a CacheLocation.
+ nn.setDeepCacheNames(cacheNames);
+ nn.setCacheLocation(deepLoc);
+ }
+ if ("network6".equals(key)) {
+ root.add6(nn);
+ } else {
+ root.add(nn);
+ }
+ } catch (NetworkNodeException ex) {
+ LOGGER.error(ex, ex);
+ return false;
+ }
+ }
+ } catch (JsonUtilsException ex) {
+ LOGGER.warn("An exception was caught while accessing the " + key + " key of " + loc + " in the incoming coverage zone file: " + ex.getMessage());
+ }
+ }
+ return true;
+ }
+
+ private static Set<String> parseDeepCacheNames(final JsonNode locationData) {
+ final Set<String> cacheNames = new HashSet<String>();
+ final JsonNode cacheArray;
+
+ try {
+ cacheArray = JsonUtils.getJsonNode(locationData, "caches");
+ } catch (JsonUtilsException ex) {
+ return cacheNames;
+ }
+
+ for (final JsonNode cache : cacheArray) {
+ final String cacheName = cache.asText();
+ if (!cacheName.isEmpty()) {
+ cacheNames.add(cacheName);
+ }
+ }
+ return cacheNames;
+ }
+
public NetworkNode(final String str) throws NetworkNodeException {
this(str, null);
}
@@ -229,6 +289,14 @@ public void setCacheLocation(final CacheLocation cacheLocation) {
this.cacheLocation = cacheLocation;
}
+ public Set<String> getDeepCacheNames() {
+ return deepCacheNames;
+ }
+
+ public void setDeepCacheNames(final Set<String> deepCacheNames) {
+ this.deepCacheNames = deepCacheNames;
+ }
+
public int size() {
if (children == null) {
return 1;
@@ -244,22 +312,30 @@ public int size() {
}
public void clearCacheLocations() {
+ clearCacheLocations(false);
+ }
+
+ public void clearCacheLocations(final boolean clearCachesOnly) {
synchronized(this) {
- cacheLocation = null;
+ if (clearCachesOnly && cacheLocation != null) {
+ cacheLocation.clearCaches();
+ } else {
+ cacheLocation = null;
+ }
if (this instanceof SuperNode) {
final SuperNode superNode = (SuperNode) this;
if (superNode.children6 != null) {
for (final NetworkNode child : superNode.children6.keySet()) {
- child.clearCacheLocations();
+ child.clearCacheLocations(clearCachesOnly);
}
}
}
if (children != null) {
for (final NetworkNode child : children.keySet()) {
- child.clearCacheLocations();
+ child.clearCacheLocations(clearCachesOnly);
}
}
}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkUpdater.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkUpdater.java
index 92d9e05d89..2ae2938806 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkUpdater.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkUpdater.java
@@ -35,7 +35,7 @@ public boolean loadDatabase() throws IOException {
return false;
}
- return NetworkNode.generateTree(existingDB, false) != null;
+ return generateTree(existingDB, false) != null;
}
@Override
@@ -44,7 +44,11 @@ public boolean verifyDatabase(final File dbFile) throws IOException {
return false;
}
- return NetworkNode.generateTree(dbFile, true) != null;
+ return generateTree(dbFile, true) != null;
+ }
+
+ public NetworkNode generateTree(final File dbFile, final boolean verifyOnly) throws IOException {
+ return NetworkNode.generateTree(dbFile, verifyOnly);
}
}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/StatTracker.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/StatTracker.java
index f628afdca6..b444cd82ac 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/StatTracker.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/StatTracker.java
@@ -44,6 +44,12 @@ public int getGeoCount() {
public void setGeoCount(final int geoCount) {
this.geoCount = geoCount;
}
+ public int getDeepCzCount() {
+ return deepCzCount;
+ }
+ public void setDeepCzCount(final int deepCzCount) {
+ this.deepCzCount = deepCzCount;
+ }
public int getDsrCount() {
return dsrCount;
}
@@ -89,6 +95,7 @@ public void setRegionalAlternateCount(final int regionalAlternateCount) {
public int czCount;
public int geoCount;
+ public int deepCzCount;
public int missCount;
public int dsrCount;
public int errCount;
@@ -104,7 +111,7 @@ public void setRegionalAlternateCount(final int regionalAlternateCount) {
}
public static enum ResultType {
- ERROR, CZ, GEO, MISS, STATIC_ROUTE, DS_REDIRECT, DS_MISS, INIT, FED, RGDENY, RGALT, GEO_REDIRECT
+ ERROR, CZ, GEO, MISS, STATIC_ROUTE, DS_REDIRECT, DS_MISS, INIT, FED, RGDENY, RGALT, GEO_REDIRECT, DEEP_CZ
}
public enum ResultDetails {
@@ -274,6 +281,9 @@ private static void incTally(final Track t, final Tallies tallies) {
case GEO:
tallies.geoCount++;
break;
+ case DEEP_CZ:
+ tallies.deepCzCount++;
+ break;
case MISS:
tallies.missCount++;
break;
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java
index 7db9ab2501..b63e075fb6 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java
@@ -243,9 +243,28 @@ public Geolocation getLocation(final String clientIP, final DeliveryService deli
return null;
}
- protected List<Cache> selectCaches(final Request request, final DeliveryService ds, final Track track) throws GeolocationException {
- final CacheLocation cacheLocation = getCoverageZoneCacheLocation(request.getClientIP(), ds);
- List<Cache> caches = selectCachesByCZ(ds, cacheLocation, track);
+
+ @SuppressWarnings("PMD.CyclomaticComplexity")
+ protected List<Cache> selectCaches(final HTTPRequest request, final DeliveryService ds, final Track track) throws GeolocationException {
+ CacheLocation cacheLocation;
+ ResultType result = ResultType.CZ;
+
+ if (ds.getDeepCache() == DeliveryService.DeepCachingType.ALWAYS) {
+ // Deep caching is enabled. See if there are deep caches available
+ cacheLocation = getDeepCoverageZoneCacheLocation(request.getClientIP(), ds);
+ if (cacheLocation != null && cacheLocation.getCaches().size() != 0) {
+ // Found deep caches for this client, and there are caches that might be available there.
+ result = ResultType.DEEP_CZ;
+ } else {
+ // No deep caches for this client, would have used them if there were any. Fallback to regular CZ
+ cacheLocation = getCoverageZoneCacheLocation(request.getClientIP(), ds);
+ }
+ } else {
+ // Deep caching not enabled for this Delivery Service; use the regular CZ
+ cacheLocation = getCoverageZoneCacheLocation(request.getClientIP(), ds);
+ }
+
+ List<Cache>caches = selectCachesByCZ(ds, cacheLocation, track, result);
if (caches != null) {
return caches;
@@ -434,6 +453,10 @@ public Geolocation getClientLocation(final String clientIp, final DeliveryServic
}
private List<Cache> selectCachesByCZ(final DeliveryService ds, final CacheLocation cacheLocation, final Track track) {
+ return selectCachesByCZ(ds, cacheLocation, track, ResultType.CZ); // ResultType.CZ was the original default before DDC
+ }
+
+ private List<Cache> selectCachesByCZ(final DeliveryService ds, final CacheLocation cacheLocation, final Track track, final ResultType result) {
if (cacheLocation == null || ds == null || !ds.isLocationAvailable(cacheLocation)) {
return null;
}
@@ -441,7 +464,7 @@ public Geolocation getClientLocation(final String clientIp, final DeliveryServic
final List<Cache> caches = selectCaches(cacheLocation, ds);
if (caches != null && track != null) {
- track.setResult(ResultType.CZ);
+ track.setResult(result);
track.setResultLocation(cacheLocation.getGeolocation());
}
@@ -592,6 +615,15 @@ private boolean isTlsMismatch(final HTTPRequest request, final DeliveryService d
return false;
}
+ protected NetworkNode getDeepNetworkNode(final String ip) {
+ try {
+ return NetworkNode.getDeepInstance().getNetwork(ip);
+ } catch (NetworkNodeException e) {
+ LOGGER.warn(e);
+ }
+ return null;
+ }
+
protected NetworkNode getNetworkNode(final String ip) {
try {
return NetworkNode.getInstance().getNetwork(ip);
@@ -602,7 +634,12 @@ protected NetworkNode getNetworkNode(final String ip) {
}
public CacheLocation getCoverageZoneCacheLocation(final String ip, final String deliveryServiceId) {
- final NetworkNode networkNode = getNetworkNode(ip);
+ return getCoverageZoneCacheLocation(ip, deliveryServiceId, false); // default is not deep
+ }
+
+ @SuppressWarnings({"PMD.CyclomaticComplexity", "PMD.NPathComplexity"})
+ public CacheLocation getCoverageZoneCacheLocation(final String ip, final String deliveryServiceId, final boolean useDeep) {
+ final NetworkNode networkNode = useDeep ? getDeepNetworkNode(ip) : getNetworkNode(ip);
if (networkNode == null) {
return null;
@@ -611,10 +648,20 @@ public CacheLocation getCoverageZoneCacheLocation(final String ip, final String
final DeliveryService deliveryService = cacheRegister.getDeliveryService(deliveryServiceId);
CacheLocation cacheLocation = networkNode.getCacheLocation();
+ if (useDeep && cacheLocation != null) {
+ // lazily load deep Caches into the deep CacheLocation
+ cacheLocation.loadDeepCaches(networkNode.getDeepCacheNames(), cacheRegister);
+ }
+
if (cacheLocation != null && !getSupportingCaches(cacheLocation.getCaches(), deliveryService).isEmpty()) {
return cacheLocation;
}
+ if (useDeep) {
+ // there were no available deep caches in the deep CZF
+ return null;
+ }
+
if (networkNode.getLoc() == null) {
return null;
}
@@ -632,6 +679,14 @@ public CacheLocation getCoverageZoneCacheLocation(final String ip, final String
return getClosestCacheLocation(cacheRegister.filterAvailableLocations(deliveryServiceId), networkNode.getGeolocation(), cacheRegister.getDeliveryService(deliveryServiceId));
}
+ public CacheLocation getDeepCoverageZoneCacheLocation(final String ip, final DeliveryService deliveryService) {
+ return getCoverageZoneCacheLocation(ip, deliveryService, true);
+ }
+
+ protected CacheLocation getCoverageZoneCacheLocation(final String ip, final DeliveryService deliveryService, final boolean useDeep) {
+ return getCoverageZoneCacheLocation(ip, deliveryService.getId(), useDeep);
+ }
+
protected CacheLocation getCoverageZoneCacheLocation(final String ip, final DeliveryService deliveryService) {
return getCoverageZoneCacheLocation(ip, deliveryService.getId());
}
diff --git a/traffic_router/core/src/main/webapp/WEB-INF/applicationContext.xml b/traffic_router/core/src/main/webapp/WEB-INF/applicationContext.xml
index 4e3f869b8c..fb5cca3b24 100644
--- a/traffic_router/core/src/main/webapp/WEB-INF/applicationContext.xml
+++ b/traffic_router/core/src/main/webapp/WEB-INF/applicationContext.xml
@@ -109,6 +109,7 @@
<property name="trafficRouterManager" ref="trafficRouterManager" />
<property name="geolocationDatabaseUpdater" ref="geolocationDatabaseUpdater" />
<property name="networkUpdater" ref="networkUpdater" />
+ <property name="deepNetworkUpdater" ref="deepNetworkUpdater" />
<property name="regionalGeoUpdater" ref="regionalGeoUpdater" />
<property name="statTracker" ref="statTracker" />
<property name="configDir" value="/opt/traffic_router/conf" />
@@ -144,6 +145,14 @@
<property name="trafficRouterManager" ref="trafficRouterManager" />
</bean>
+ <bean id="deepNetworkUpdater" class="com.comcast.cdn.traffic_control.traffic_router.core.loc.DeepNetworkUpdater" init-method="init">
+ <property name="executorService" ref="ScheduledExecutorService" />
+ <property name="databasesDirectory" ref="databasesDir" />
+ <property name="databaseName" value="$[cache.dczmap.database:dczmap.json]" />
+ <property name="pollingInterval" value="$[cache.dczmap.database.refresh.period:10800000]" />
+ <property name="trafficRouterManager" ref="trafficRouterManager" />
+ </bean>
+
<bean id="regionalGeoUpdater" class="com.comcast.cdn.traffic_control.traffic_router.core.loc.RegionalGeoUpdater"
init-method="init">
<property name="executorService" ref="ScheduledExecutorService" />
@@ -182,7 +191,7 @@
</bean>
<bean id="languidState" class="com.comcast.cdn.traffic_control.traffic_router.core.util.LanguidState" init-method="init"
- depends-on="geolocationDatabaseUpdater, networkUpdater, regionalGeoUpdater, MonitorWatcher, trafficRouterManager, ConfigHandler">
+ depends-on="geolocationDatabaseUpdater, networkUpdater, deepNetworkUpdater, regionalGeoUpdater, MonitorWatcher, trafficRouterManager, ConfigHandler">
<!-- todo: add bean refs that can be validated inside the method called -->
<property name="trafficRouterManager" ref="trafficRouterManager" />
</bean>
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/external/StatsTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/external/StatsTest.java
index 006419d4b8..b68b43b059 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/external/StatsTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/external/StatsTest.java
@@ -85,12 +85,12 @@ public void itGetsApplicationStats() throws Exception {
Map<String, Object> dnsStats = (Map<String, Object>) statsData.get("dnsMap");
Map<String, Object> cacheDnsStats = (Map<String, Object>) dnsStats.values().iterator().next();
assertThat(cacheDnsStats.keySet(), containsInAnyOrder("czCount", "geoCount", "missCount", "dsrCount", "errCount",
- "staticRouteCount", "fedCount", "regionalDeniedCount", "regionalAlternateCount"));
+ "deepCzCount", "staticRouteCount", "fedCount", "regionalDeniedCount", "regionalAlternateCount"));
Map<String, Object> httpStats = (Map<String, Object>) statsData.get("httpMap");
Map<String, Object> cacheHttpStats = (Map<String, Object>) httpStats.values().iterator().next();
assertThat(cacheHttpStats.keySet(), containsInAnyOrder("czCount", "geoCount", "missCount", "dsrCount", "errCount",
- "staticRouteCount", "fedCount", "regionalDeniedCount", "regionalAlternateCount"));
+ "deepCzCount", "staticRouteCount", "fedCount", "regionalDeniedCount", "regionalAlternateCount"));
Map<String, Object> updateTracker = (Map<String, Object>) statsData.get("updateTracker");
Set<String> keys = updateTracker.keySet();
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/CoverageZoneTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/CoverageZoneTest.java
index 1e7523a554..0387c89c29 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/CoverageZoneTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/CoverageZoneTest.java
@@ -88,6 +88,7 @@ public void before() throws Exception {
trafficRouter = PowerMockito.mock(TrafficRouter.class);
Whitebox.setInternalState(trafficRouter, "cacheRegister", cacheRegister);
when(trafficRouter.getCoverageZoneCacheLocation("12.23.34.45", "delivery-service-1")).thenCallRealMethod();
+ when(trafficRouter.getCoverageZoneCacheLocation("12.23.34.45", "delivery-service-1", false)).thenCallRealMethod();
when(trafficRouter.getCacheRegister()).thenReturn(cacheRegister);
when(trafficRouter.orderCacheLocations(cacheGroups,testLocation)).thenCallRealMethod();
when(trafficRouter.getSupportingCaches(anyListOf(Cache.class), eq(deliveryService))).thenCallRealMethod();
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNodeTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNodeTest.java
index 0a1e7dd7b8..e4301af565 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNodeTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNodeTest.java
@@ -42,36 +42,46 @@
public class NetworkNodeTest {
private static final Logger LOGGER = Logger.getLogger(NetworkNodeTest.class);
private Map<String, List<String>> netMap = new HashMap<String, List<String>>();
+ private Map<String, List<String>> deepNetMap = new HashMap<String, List<String>>();
private NetworkNode root;
+ private NetworkNode deepRoot;
@Before
public void setUp() throws Exception {
- final File file = new File(getClass().getClassLoader().getResource("czmap.json").toURI());
- root = NetworkNode.generateTree(file, false);
+ root = setUp("czmap.json", false);
+ deepRoot = setUp("dczmap.json", true);
+ }
+
+ private NetworkNode setUp(final String filename, final boolean useDeep) throws Exception {
+ final Map<String, List<String>> testNetMap = useDeep ? deepNetMap : netMap;
+ final File file = new File(getClass().getClassLoader().getResource(filename).toURI());
+ final NetworkNode nn = NetworkNode.generateTree(file, false, useDeep);
final ObjectMapper mapper = new ObjectMapper();
final JsonNode jsonNode = mapper.readTree(file);
- final JsonNode coverageZones = jsonNode.get("coverageZones");
+ final String czKey = useDeep ? "deepCoverageZones" : "coverageZones";
+ final JsonNode coverageZones = jsonNode.get(czKey);
final Iterator<String> networkIter = coverageZones.fieldNames();
while (networkIter.hasNext()) {
- final String loc = networkIter.next();
+ final String loc = networkIter.next();
final JsonNode locData = coverageZones.get(loc);
- for (final JsonNode networkType : locData) {
- final String network = networkType.get(0).asText().split("/")[0];
+ for (String networkType : new String[]{"network", "network6"}) {
+ final JsonNode networks = locData.get(networkType);
+ final String network = networks.get(0).asText().split("/")[0];
InetAddress ip = InetAddresses.forString(network);
ip = InetAddresses.increment(ip);
- if (!netMap.containsKey(loc)) {
- netMap.put(loc, new ArrayList<String>());
+ if (!testNetMap.containsKey(loc)) {
+ testNetMap.put(loc, new ArrayList<String>());
}
- final List<String> addressList = netMap.get(loc);
+ final List<String> addressList = testNetMap.get(loc);
addressList.add(InetAddresses.toAddrString(ip));
- netMap.put(loc, addressList);
-
+ testNetMap.put(loc, addressList);
}
}
+ return nn;
}
@Test
@@ -93,15 +103,24 @@ public void testIps() {
@Test
public void testNetworkNodePerformance() {
+ testNetworkNodePerformance(root, netMap);
+ }
+
+ @Test
+ public void testDeepNetworkNodePerformance() {
+ testNetworkNodePerformance(deepRoot, deepNetMap);
+ }
+
+ private void testNetworkNodePerformance(NetworkNode testRoot, Map<String, List<String>> testNetMap) {
final int iterations = 100000;
final long startTime = System.currentTimeMillis();
final long nnTPS = Long.parseLong(System.getProperty("nnTPS", "12000"));
for (int i = 0; i < iterations; i++) {
- for (final String location : netMap.keySet()) {
+ for (final String location : testNetMap.keySet()) {
try {
- for (final String address : netMap.get(location)) {
- final NetworkNode nn = root.getNetwork(address);
+ for (final String address : testNetMap.get(location)) {
+ final NetworkNode nn = testRoot.getNetwork(address);
}
} catch (final NetworkNodeException e) {
e.printStackTrace();
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNodeUnitTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNodeUnitTest.java
index c12bfd60b0..3ebb10ed01 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNodeUnitTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNodeUnitTest.java
@@ -20,8 +20,10 @@
import org.junit.Test;
+import java.util.HashSet;
import java.util.Iterator;
import java.util.Map;
+import java.util.Set;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.equalTo;
@@ -86,6 +88,31 @@ public void itPutsNetworksIntoOrderedHierarchy() throws Exception {
assertThat(iterator.next().getKey(),equalTo(subnet2));
}
+ @Test
+ public void itSupportsDeepCaches() throws Exception {
+ String czmapString = "{" +
+ "\"revision\": \"Mon Dec 21 15:04:01 2015\"," +
+ "\"customerName\": \"Kabletown\"," +
+ "\"deepCoverageZones\": {" +
+ "\"us-co-denver\": {" +
+ "\"network\": [\"192.168.55.0/24\",\"192.168.6.0/24\",\"192.168.0.0/16\"]," +
+ "\"network6\": [\"1234:5678::/64\",\"1234:5679::/64\"]," +
+ "\"caches\": [\"host1\",\"host2\"]" +
+ "}" +
+ "}" +
+ "}";
+
+ final ObjectMapper mapper = new ObjectMapper();
+ final JsonNode json = mapper.readTree(czmapString);
+ NetworkNode networkNode = NetworkNode.generateTree(json, false, true);
+ NetworkNode foundNetworkNode = networkNode.getNetwork("192.168.55.100");
+
+ Set<String> expected = new HashSet<String>();
+ expected.add("host1");
+ expected.add("host2");
+ assertThat(foundNetworkNode.getDeepCacheNames(), equalTo(expected));
+ }
+
@Test
public void itDoesIpV6() throws Exception {
String czmapString = "{" +
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/DeliveryServiceHTTPRoutingMissesTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/DeliveryServiceHTTPRoutingMissesTest.java
index 01d98158c3..1714dc8b09 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/DeliveryServiceHTTPRoutingMissesTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/DeliveryServiceHTTPRoutingMissesTest.java
@@ -47,6 +47,8 @@ public void before() throws Exception {
when(unusedByTest.get("routingName")).thenReturn(mapper.readTree("\"edge\""));
when(unusedByTest.has("coverageZoneOnly")).thenReturn(true);
when(unusedByTest.get("coverageZoneOnly")).thenReturn(mapper.readTree("true"));
+ when(unusedByTest.has("deepCachingType")).thenReturn(true);
+ when(unusedByTest.get("deepCachingType")).thenReturn(mapper.readTree("\"NEVER\""));
deliveryService = new DeliveryService("ignoredbytest", unusedByTest);
httpRequest = mock(HTTPRequest.class);
track = StatTracker.getTrack();
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterTest.java
index 1c6e971f60..2e7a54a801 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterTest.java
@@ -138,7 +138,7 @@ public void itCreatesHttpResults() throws Exception {
List<Cache> caches = new ArrayList<Cache>();
caches.add(cache);
- when(trafficRouter.selectCaches(any(Request.class), any(DeliveryService.class), any(Track.class))).thenReturn(caches);
+ when(trafficRouter.selectCaches(any(HTTPRequest.class), any(DeliveryService.class), any(Track.class))).thenReturn(caches);
when(trafficRouter.selectCachesByGeo(anyString(), any(DeliveryService.class), any(CacheLocation.class), any(Track.class))).thenCallRealMethod();
when(trafficRouter.getClientLocation(anyString(), any(DeliveryService.class), any(CacheLocation.class), any(Track.class))).thenReturn(new Geolocation(40, -100));
when(trafficRouter.getCachesByGeo(any(DeliveryService.class), any(Geolocation.class), any(Track.class))).thenCallRealMethod();
@@ -168,7 +168,7 @@ public void itSetsResultToGeo() throws Exception {
when(deliveryService.isLocationAvailable(cacheLocation)).thenReturn(true);
when(deliveryService.filterAvailableLocations(any(Collection.class))).thenCallRealMethod();
- when(trafficRouter.selectCaches(any(Request.class), any(DeliveryService.class), any(Track.class))).thenCallRealMethod();
+ when(trafficRouter.selectCaches(any(HTTPRequest.class), any(DeliveryService.class), any(Track.class))).thenCallRealMethod();
when(trafficRouter.selectCachesByGeo(anyString(), any(DeliveryService.class), any(CacheLocation.class), any(Track.class))).thenCallRealMethod();
Geolocation clientLocation = new Geolocation(40, -100);
diff --git a/traffic_router/core/src/test/resources/dczmap.json b/traffic_router/core/src/test/resources/dczmap.json
new file mode 100644
index 0000000000..bfc5c5bb7a
--- /dev/null
+++ b/traffic_router/core/src/test/resources/dczmap.json
@@ -0,0 +1,25 @@
+{
+ "deepCoverageZones":
+ {
+ "cache-group-01":
+ {
+ "network6":
+ [
+ "1234:5678::/64",
+ "1234:5679::/64"
+ ],
+ "network":
+ [
+ "192.168.8.0/24",
+ "192.168.9.0/24"
+ ],
+ "caches":
+ [
+ "host1",
+ "host2",
+ "host3",
+ "host4"
+ ]
+ }
+ }
+}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services