You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficcontrol.apache.org by el...@apache.org on 2020/06/08 22:28:06 UTC
[trafficcontrol] branch master updated (281b50e -> 2dd9a0c)
This is an automated email from the ASF dual-hosted git repository.
elsloo pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git.
from 281b50e Made admin user compliant with restrictions (API and TP) (#4764)
new 406a280 Initial commit for localization/routing for HTTP delivery services.
new 5bc20e1 Fixed a bug that led to ADMIN_DOWN hosts being served in ETR related dynamic responses.
new 9c501b1 Ignore PMD violation
new 2dd9a0c Fix up the docs, add changelog entry
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGELOG.md | 1 +
docs/source/admin/traffic_router.rst | 53 ++-
.../traffic_router/images/edge_tr_example.png | Bin 0 -> 448267 bytes
traffic_ops/app/lib/UI/DeliveryService.pm | 2 +-
traffic_ops/app/lib/UI/Topology.pm | 33 +-
.../app/templates/delivery_service/add.html.ep | 4 +-
.../app/templates/delivery_service/edit.html.ep | 6 +-
.../form.deliveryService.HTTP.tpl.html | 16 +
.../form.deliveryService.Steering.tpl.html | 16 +
.../api/controllers/ConsistentHashController.java | 2 +-
.../api/controllers/CoverageZoneController.java | 4 +-
.../controllers/DeepCoverageZoneController.java | 2 +-
.../controllers/DeliveryServicesController.java | 2 +-
.../core/cache/CacheLocationManager.java | 45 ---
.../traffic_router/core/config/ConfigHandler.java | 131 ++++++--
.../traffic_router/core/dns/SignatureManager.java | 2 +-
.../traffic_router/core/dns/ZoneManager.java | 289 ++++++++++------
.../traffic_router/core/ds/DeliveryService.java | 11 +-
.../traffic_router/core/ds/Dispersion.java | 2 +-
.../traffic_router/core/ds/SteeringResult.java | 2 +-
.../traffic_router/core/edge/Cache.java | 112 +++++++
.../core/{cache => edge}/CacheLocation.java | 35 +-
.../core/{cache => edge}/CacheRegister.java | 95 ++++--
.../core/{cache => edge}/InetRecord.java | 44 ++-
.../traffic_router/core/edge/Location.java | 92 +++++
.../core/{cache/Cache.java => edge/Node.java} | 117 +------
.../core/{cache => edge}/Resolver.java | 2 +-
.../core/edge/TrafficRouterLocation.java | 90 +++++
.../traffic_router/core/loc/AnonymousIp.java | 2 +-
.../core/loc/FederationRegistry.java | 2 +-
.../traffic_router/core/loc/NetworkNode.java | 34 +-
.../traffic_router/core/loc/RegionalGeo.java | 2 +-
.../core/monitor/TrafficMonitorWatcher.java | 2 +-
.../traffic_router/core/request/DNSRequest.java | 44 ++-
.../traffic_router/core/router/DNSRouteResult.java | 21 +-
...tionComparator.java => LocationComparator.java} | 8 +-
.../traffic_router/core/router/StatTracker.java | 45 ++-
.../traffic_router/core/router/TrafficRouter.java | 369 ++++++++++++++++++---
.../core/router/TrafficRouterManager.java | 2 +-
.../traffic_router/core/util/DataExporter.java | 18 +-
.../traffic_router/core/CatalinaTrafficRouter.java | 3 +
.../core/config/ConfigHandlerTest.java | 10 +-
.../traffic_router/core/dns/NameServerTest.java | 4 +-
.../traffic_router/core/dns/ZoneManagerTest.java | 14 +-
.../core/dns/ZoneManagerUnitTest.java | 5 +-
.../core/ds/SteeringGeolocationComparatorTest.java | 2 +-
.../core/{cache => edge}/CacheRegisterTest.java | 21 +-
.../traffic_router/core/external/RouterTest.java | 41 ++-
.../traffic_router/core/loc/CoverageZoneTest.java | 12 +-
.../core/loc/FederationRegistryTest.java | 2 +-
.../traffic_router/core/loc/RegionalGeoTest.java | 2 +-
.../core/router/CacheLocationComparatorTest.java | 10 +-
.../core/router/CacheRegisterBuilder.java | 23 +-
.../core/router/DNSRoutingMissesTest.java | 30 +-
.../core/router/GeoSortSteeringResultsTest.java | 2 +-
.../router/TrafficRouterHTTPRoutingMissesTest.java | 2 +-
.../core/router/TrafficRouterTest.java | 36 +-
.../core/src/test/resources/publish/CrConfig.json | 101 +++++-
.../core/src/test/resources/publish/CrConfig2.json | 11 +
.../core/src/test/resources/publish/CrConfig3.json | 13 +-
.../core/src/test/resources/publish/CrConfig4.json | 13 +-
61 files changed, 1544 insertions(+), 572 deletions(-)
create mode 100644 docs/source/admin/traffic_router/images/edge_tr_example.png
delete mode 100644 traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheLocationManager.java
create mode 100644 traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/Cache.java
rename traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/{cache => edge}/CacheLocation.java (90%)
rename traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/{cache => edge}/CacheRegister.java (61%)
rename traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/{cache => edge}/InetRecord.java (66%)
create mode 100644 traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/Location.java
rename traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/{cache/Cache.java => edge/Node.java} (65%)
rename traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/{cache => edge}/Resolver.java (96%)
create mode 100644 traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/TrafficRouterLocation.java
rename traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/{CacheLocationComparator.java => LocationComparator.java} (80%)
rename traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/{cache => edge}/CacheRegisterTest.java (81%)
[trafficcontrol] 03/04: Ignore PMD violation
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elsloo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 9c501b1df7e33f46a3fc47c6f2216b6d5f0cf0bc
Author: Rawlin Peters <ra...@apache.org>
AuthorDate: Sun Feb 23 11:10:08 2020 -0700
Ignore PMD violation
---
.../cdn/traffic_control/traffic_router/core/dns/ZoneManager.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManager.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManager.java
index de2876a..fdbcf53 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManager.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManager.java
@@ -494,7 +494,7 @@ public class ZoneManager extends Resolver {
return records;
}
- @SuppressWarnings({"PMD.CyclomaticComplexity", "PMD.ExcessiveParameterList"})
+ @SuppressWarnings({"PMD.CyclomaticComplexity", "PMD.ExcessiveParameterList", "PMD.NPathComplexity"})
private static void primeZoneCache(final String domain, final Name name, final List<Record> list, final TrafficRouter tr,
final LoadingCache<ZoneKey, Zone> zc, final LoadingCache<ZoneKey, Zone> dzc, final List<Runnable> generationTasks,
final BlockingQueue<Runnable> primingTasks, final DeliveryService ds, final ConcurrentMap<String, ZoneKey> newDomainsToZoneKeys) {
[trafficcontrol] 02/04: Fixed a bug that led to ADMIN_DOWN hosts
being served in ETR related dynamic responses.
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elsloo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 5bc20e189a3796353d56c8bda847191c11ec9e5a
Author: Jeff Elsloo <je...@cable.comcast.com>
AuthorDate: Fri Jan 10 11:00:22 2020 -0700
Fixed a bug that led to ADMIN_DOWN hosts being served in ETR related dynamic responses.
---
.../traffic_router/core/config/ConfigHandler.java | 58 ++++++++++++----------
.../traffic_router/core/edge/CacheRegister.java | 37 ++++++--------
2 files changed, 46 insertions(+), 49 deletions(-)
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandler.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandler.java
index c0eb40a..7cc3ee9 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandler.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandler.java
@@ -883,10 +883,11 @@ public class ConfigHandler {
return locations;
}
+ @SuppressWarnings("PMD.CyclomaticComplexity")
private void parseEdgeTrafficRouterLocations(final JsonNode jo, final CacheRegister cacheRegister) throws JsonUtilsException {
final String locationKey = "location";
final JsonNode trafficRouterJo = JsonUtils.getJsonNode(jo, "contentRouters");
- final Set<TrafficRouterLocation> locations = new HashSet<TrafficRouterLocation>(trafficRouterJo.size());
+ final Map<Geolocation, TrafficRouterLocation> locations = new HashMap<>();
final JsonNode trafficRouterLocJo = jo.get("trafficRouterLocations");
if (trafficRouterLocJo == null) {
@@ -899,46 +900,49 @@ public class ConfigHandler {
for (final Iterator<String> trafficRouterNames = trafficRouterJo.fieldNames(); trafficRouterNames.hasNext();) {
final String trafficRouterName = trafficRouterNames.next();
final JsonNode trafficRouter = trafficRouterJo.get(trafficRouterName);
+
+ // define here to log invalid ip/ip6 input on catch below
+ String ip = null;
+ String ip6 = null;
+
try {
final String trLoc = JsonUtils.getString(trafficRouter, locationKey);
final Location cl = allLocations.get(trLoc);
if (cl != null) {
- LOGGER.info("Edge Traffic Router " + trafficRouterName + " @ " + trLoc + "; " + cl.getGeolocation().toString());
- locations.add(new TrafficRouterLocation(trLoc, cl.getGeolocation()));
- } else {
- LOGGER.error("No trafficRouterLocations configured for " + trLoc + "; unable to use Edge Traffic Router " + trafficRouterName);
- }
-
- } catch (JsonUtilsException e) {
- LOGGER.warn(e,e);
- }
- }
+ TrafficRouterLocation trafficRouterLocation = locations.get(cl.getGeolocation());
- cacheRegister.setEdgeTrafficRouterLocations(locations);
+ if (trafficRouterLocation == null) {
+ trafficRouterLocation = new TrafficRouterLocation(trLoc, cl.getGeolocation());
+ locations.put(cl.getGeolocation(), trafficRouterLocation);
+ }
- for (final Iterator<String> trafficRouterNames = trafficRouterJo.fieldNames(); trafficRouterNames.hasNext();) {
- final String trafficRouterName = trafficRouterNames.next();
- final JsonNode trafficRouter = trafficRouterJo.get(trafficRouterName);
- final String location = JsonUtils.getString(trafficRouter, locationKey);
- final TrafficRouterLocation trafficRouterLocation = cacheRegister.getEdgeTrafficRouterLocation(location);
+ final JsonNode status = trafficRouter.get("status");
- if (trafficRouterLocation != null) {
- final Node edgeTrafficRouter = new Node(trafficRouterName, trafficRouterName, JsonUtils.optInt(jo, "hashCount"));
- final String ip = JsonUtils.getString(trafficRouter, "ip");
- final String ip6 = JsonUtils.optString(trafficRouter, "ip6");
- edgeTrafficRouter.setFqdn(JsonUtils.getString(trafficRouter, "fqdn"));
- edgeTrafficRouter.setPort(JsonUtils.getInt(trafficRouter, "port"));
+ if (status == null || (!"ONLINE".equals(status.asText()) && !"REPORTED".equals(status.asText()))) {
+ LOGGER.warn(String.format("Skipping Edge Traffic Router %s due to %s status", trafficRouterName, status));
+ continue;
+ } else {
+ LOGGER.info(String.format("Edge Traffic Router %s %s @ %s; %s", status, trafficRouterName, trLoc, cl.getGeolocation().toString()));
+ }
- try {
+ final Node edgeTrafficRouter = new Node(trafficRouterName, trafficRouterName, JsonUtils.optInt(jo, "hashCount"));
+ ip = JsonUtils.getString(trafficRouter, "ip");
+ ip6 = JsonUtils.optString(trafficRouter, "ip6");
+ edgeTrafficRouter.setFqdn(JsonUtils.getString(trafficRouter, "fqdn"));
+ edgeTrafficRouter.setPort(JsonUtils.getInt(trafficRouter, "port"));
edgeTrafficRouter.setIpAddress(ip, ip6, 0);
trafficRouterLocation.addTrafficRouter(trafficRouterName, edgeTrafficRouter);
- } catch (UnknownHostException ex) {
- LOGGER.warn(ex + " : " + ip);
+ } else {
+ LOGGER.error("No Location found for " + trLoc + "; unable to use Edge Traffic Router " + trafficRouterName);
}
+ } catch (JsonUtilsException e) {
+ LOGGER.warn(e, e);
+ } catch (UnknownHostException ex) {
+ LOGGER.warn(String.format("%s; input was ip=%s, ip6=%s", ex, ip, ip6), ex);
}
}
- cacheRegister.setEdgeTrafficRouterCount();
+ cacheRegister.setEdgeTrafficRouterLocations(locations.values());
}
}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/CacheRegister.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/CacheRegister.java
index 8da63df..c802a84 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/CacheRegister.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/CacheRegister.java
@@ -15,13 +15,7 @@
package com.comcast.cdn.traffic_control.traffic_router.core.edge;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-import java.util.TreeSet;
+import java.util.*;
import com.fasterxml.jackson.databind.JsonNode;
@@ -74,20 +68,8 @@ public class CacheRegister {
return new ArrayList<TrafficRouterLocation>(edgeTrafficRouterLocations.values());
}
- public void setEdgeTrafficRouterCount() {
- int count = 0;
-
- for (final TrafficRouterLocation location : getEdgeTrafficRouterLocations()) {
- final List<Node> trafficRouters = location.getTrafficRouters();
-
- if (trafficRouters == null) {
- continue;
- }
-
- count += trafficRouters.size();
- }
-
- edgeTrafficRouterCount = count;
+ private void setEdgeTrafficRouterCount(final int count) {
+ this.edgeTrafficRouterCount = count;
}
public int getEdgeTrafficRouterCount() {
@@ -117,11 +99,22 @@ public class CacheRegister {
}
}
- public void setEdgeTrafficRouterLocations(final Set<TrafficRouterLocation> locations) {
+ public void setEdgeTrafficRouterLocations(final Collection<TrafficRouterLocation> locations) {
+ int count = 0;
+
edgeTrafficRouterLocations.clear();
+
for (final TrafficRouterLocation newLoc : locations) {
edgeTrafficRouterLocations.put(newLoc.getId(), newLoc);
+
+ final List<Node> trafficRouters = newLoc.getTrafficRouters();
+
+ if (trafficRouters != null) {
+ count += trafficRouters.size();
+ }
}
+
+ setEdgeTrafficRouterCount(count);
}
public boolean hasEdgeTrafficRouters() {
[trafficcontrol] 01/04: Initial commit for localization/routing for
HTTP delivery services.
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elsloo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 406a2801a62a11b254521067003402d830ca82cd
Author: Jeff Elsloo <je...@cable.comcast.com>
AuthorDate: Mon Nov 28 15:06:12 2016 -0700
Initial commit for localization/routing for HTTP delivery services.
Rebase/checkpoint commit for localization/routing for HTTP delivery services.
Fix testDynamicZoneCache failure when location not found in czmap
Minor code cleanup for Edge Traffic Routing
Changed logic around initial limits when creating list of edge TRs in a miss scenario.
Fixed a small bug that prevented localized Traffic Routers from being identified for NS records and other RR types that do not associate with a DS.
Minor bug fix and cleanup around logging.
Add field for Max DNS IPs to HTTP & STEERING DSes in Traffic Portal
Also, show the field when editing a STEERING DS in the old UI.
Fixed NPE caused by a null track object due to not wanting to overwrite logging data set by localization decisions on a DNS delivery service.
Fixed a case where result location was not being set.
Adds configurable hash count by profile to the 'contentRouters' section of CRConfig.
CDN-567 Move edge traffic router locations out of edgeLocations
Separate localized DNS stats from the 'dnsMap' in the stats API endpoint
The 'dnsMap' key is typically filled with Delivery Service FQDNs, but
with DNS localization for Traffic Router NS or A/AAAA records, the
'dnsMap' has begun to fill up with non-Delivery Service keys, e.g. when
doing this:
`dig @trafficrouter.cdn-domain.com deliveryservice.cdn-domain.com ns`
... the 'deliveryservice.cdn-domain.com' key is put in the 'dnsMap' when
that's not an example Delivery URL.
This commit separates entries like the example above into a
'localizedDnsMap' key to keep it more separate from Delivery Service
FQDNs. To keep it from filling up memory, localized DNS queries that
return NXDOMAIN are tallied into the 'nxdomainTallies' key, and ones
that return NODATA are not tallied at all.
Added some defensive coding around parsing trafficRouterLocations and slightly refactored to work around a PMD warning.
Updated logging message to contain the correct key name for trafficRouterLocations.
Added edge traffic routing docs.
Moved TR admin docs to their own directory structure.
Zone cache fixes, performance enhancements and new tuning parameters.
Updated HTTP DS page to match updates made to DNS DS page.
Updated Steering DS page to match updates made to HTTP and DNS DS pages.
Removed import that belongs in the Cache class.
Updated ConfigHandlerTest import to reflect correct package name for CacheLocation.LocalizationMethod.
Corrected package imports for new tests in RegionalGeoTest
manual merge of docs/source/admin/traffic_ops/using.rst
Fix ZoneManagerTest to work with cachegroups that aren't in the CZF
---
docs/source/admin/index.rst | 2 +-
.../traffic_router/images/edge_tr_example.png | Bin 0 -> 448267 bytes
.../index.rst} | 53 ++-
traffic_ops/app/lib/UI/DeliveryService.pm | 2 +-
traffic_ops/app/lib/UI/Topology.pm | 33 +-
.../app/templates/delivery_service/add.html.ep | 4 +-
.../app/templates/delivery_service/edit.html.ep | 6 +-
.../form.deliveryService.HTTP.tpl.html | 16 +
.../form.deliveryService.Steering.tpl.html | 16 +
.../api/controllers/ConsistentHashController.java | 2 +-
.../api/controllers/CoverageZoneController.java | 4 +-
.../controllers/DeepCoverageZoneController.java | 2 +-
.../controllers/DeliveryServicesController.java | 2 +-
.../core/cache/CacheLocationManager.java | 45 ---
.../traffic_router/core/config/ConfigHandler.java | 127 +++++--
.../traffic_router/core/dns/SignatureManager.java | 2 +-
.../traffic_router/core/dns/ZoneManager.java | 287 ++++++++++------
.../traffic_router/core/ds/DeliveryService.java | 11 +-
.../traffic_router/core/ds/Dispersion.java | 2 +-
.../traffic_router/core/ds/SteeringResult.java | 2 +-
.../traffic_router/core/edge/Cache.java | 112 +++++++
.../core/{cache => edge}/CacheLocation.java | 35 +-
.../core/{cache => edge}/CacheRegister.java | 90 +++--
.../core/{cache => edge}/InetRecord.java | 44 ++-
.../traffic_router/core/edge/Location.java | 92 +++++
.../core/{cache/Cache.java => edge/Node.java} | 117 +------
.../core/{cache => edge}/Resolver.java | 2 +-
.../core/edge/TrafficRouterLocation.java | 90 +++++
.../traffic_router/core/loc/AnonymousIp.java | 2 +-
.../core/loc/FederationRegistry.java | 2 +-
.../traffic_router/core/loc/NetworkNode.java | 34 +-
.../traffic_router/core/loc/RegionalGeo.java | 2 +-
.../core/monitor/TrafficMonitorWatcher.java | 2 +-
.../traffic_router/core/request/DNSRequest.java | 44 ++-
.../traffic_router/core/router/DNSRouteResult.java | 21 +-
...tionComparator.java => LocationComparator.java} | 8 +-
.../traffic_router/core/router/StatTracker.java | 45 ++-
.../traffic_router/core/router/TrafficRouter.java | 369 ++++++++++++++++++---
.../core/router/TrafficRouterManager.java | 2 +-
.../traffic_router/core/util/DataExporter.java | 18 +-
.../traffic_router/core/CatalinaTrafficRouter.java | 3 +
.../core/config/ConfigHandlerTest.java | 10 +-
.../traffic_router/core/dns/NameServerTest.java | 4 +-
.../traffic_router/core/dns/ZoneManagerTest.java | 14 +-
.../core/dns/ZoneManagerUnitTest.java | 5 +-
.../core/ds/SteeringGeolocationComparatorTest.java | 2 +-
.../core/{cache => edge}/CacheRegisterTest.java | 21 +-
.../traffic_router/core/external/RouterTest.java | 41 ++-
.../traffic_router/core/loc/CoverageZoneTest.java | 12 +-
.../core/loc/FederationRegistryTest.java | 2 +-
.../traffic_router/core/loc/RegionalGeoTest.java | 2 +-
.../core/router/CacheLocationComparatorTest.java | 10 +-
.../core/router/CacheRegisterBuilder.java | 23 +-
.../core/router/DNSRoutingMissesTest.java | 30 +-
.../core/router/GeoSortSteeringResultsTest.java | 2 +-
.../router/TrafficRouterHTTPRoutingMissesTest.java | 2 +-
.../core/router/TrafficRouterTest.java | 36 +-
.../core/src/test/resources/publish/CrConfig.json | 101 +++++-
.../core/src/test/resources/publish/CrConfig2.json | 11 +
.../core/src/test/resources/publish/CrConfig3.json | 13 +-
.../core/src/test/resources/publish/CrConfig4.json | 13 +-
61 files changed, 1540 insertions(+), 566 deletions(-)
diff --git a/docs/source/admin/index.rst b/docs/source/admin/index.rst
index 8c5a889..af8b55a 100644
--- a/docs/source/admin/index.rst
+++ b/docs/source/admin/index.rst
@@ -39,7 +39,7 @@ Once everything is installed, you will need to configure the servers to talk to
traffic_ops.rst
traffic_portal/*
traffic_monitor.rst
- traffic_router.rst
+ traffic_router/index.rst
traffic_router/migrationto2-3.rst
traffic_stats.rst
traffic_server.rst
diff --git a/docs/source/admin/traffic_router/images/edge_tr_example.png b/docs/source/admin/traffic_router/images/edge_tr_example.png
new file mode 100644
index 0000000..1ef935d
Binary files /dev/null and b/docs/source/admin/traffic_router/images/edge_tr_example.png differ
diff --git a/docs/source/admin/traffic_router.rst b/docs/source/admin/traffic_router/index.rst
similarity index 93%
rename from docs/source/admin/traffic_router.rst
rename to docs/source/admin/traffic_router/index.rst
index 80803b5..fd965f5 100644
--- a/docs/source/admin/traffic_router.rst
+++ b/docs/source/admin/traffic_router/index.rst
@@ -378,9 +378,60 @@ Rolling Zone Signing Keys
-------------------------
Traffic Router currently follows the :abbr:`ZSK (Zone Signing Key)` pre-publishing operational best practice described in :rfc:`6781#section-4.1.1.1`. Once :abbr:`DNSSEC (Domain Name System Security Extensions)` is enabled for a CDN in Traffic Portal, key rolls are triggered by Traffic Ops via the automated key generation process, and Traffic Router selects the active :abbr:`ZSK (Zone Signing Keys)`\ s based on the expiration information returned from the 'keystore' API of Traffic Ops.
+.. _tr-edge_traffic_routing:
+
+Edge Traffic Routing
+====================
+
+Overview
+--------
+Edge Traffic Routing is a feature that enables localization for more DNS record types than just the routing name for DNS delivery services. This feature has two main components: localization of HTTP delivery service routing names and localization for CDN-managed NS records. This allows Traffic Router to scale horizontally more easily, as there is a practical limit to how many records can be in an RRset for ``NS``, ``A``, or ``AAAA`` record types. The practical limit is typically an answe [...]
+
+Localizing ``NS`` records reduces latency for the resolver, which, due to caching in DNS is of little utility for ``NS`` records, but it will force resolvers to use the closest Traffic Routers for all queries. This is important for lookups of CDN routing name records that have very short TTLs, meaning DNS traffic for routing name records is frequent. Like localization for routing names of DNS delivery services, localizing the routing name for HTTP delivery services provides the client (e [...]
+
+Edge DNS Routing
+----------------
+Edge DNS routing refers to the localization of ``NS`` records in Traffic Router. This can be turned on and off via the ``edge.dns.routing`` parameter; the number of records returned is controlled via ``edge.dns.limit`` and there is a hardcoded default limit of 4 when this feature is enabled. See :ref:`ccr-profile` documentation for parameter details.
+
+Edge HTTP Routing
+-----------------
+Edge HTTP routing refers to the localization of ``A`` and ``AAAA`` records that represent routing names for HTTP delivery services. This can be turned on and off via the ``edge.http.routing`` parameter; the number of records returned is controlled via ``edge.http.limit`` and there is a hardcoded default limit of 4 when this feature is enabled. The default or global limit can be overridden by modifying the ``maxDnsAnswers`` setting on the delivery service. See :ref:`ccr-profile` documenta [...]
+
+Edge Traffic Router Selection
+-----------------------------
+Traffic Router performs localization on client requests in order to determine which Traffic Routers should service a given request. After localization, Traffic Router will perform a consistent hash on the incoming name and will use the value to refine Traffic Router selection. There are two main cases for Traffic Router selection: a localization hit, and a localization miss.
+
+Localization Hit: Consistent Hash (CH)
+**************************************
+When a client is localized, Traffic Router selects the nearest Traffic Router Location (cachegroup) based on proximity. Proximity is determined by using the latitude and longitude of the client, regardless of whether it is a coverage zone or geolocation hit, and the latitude and longitude specified on Traffic Router Locations. Once the location is identified, a consistent hash is performed on the incoming name and the list of Traffic Routers is ordered based upon the consistent hash. Onc [...]
+
+Localization Miss: Consistent Hash + Round Robin (CH + RR)
+**********************************************************
+When a client cannot be localized, Traffic Router still needs to produce a list of Traffic Routers to service the request. Because the number of Traffic Routers in the CDN could far exceed the practical limits of what constitutes a "normal" sized answer, a selection algorithm is applied to select Traffic Routers. Much like the localization hit scenario, the incoming request name is consistent hashed and results size is limited by the same parameters. Because no client location is known, [...]
+
+For example, with four Traffic Router Locations each containing 10 Traffic Routers and a limit of 6, the algorithm would:
+
+- Consistent hash the incoming name
+- Order the Traffic Routers at each location by the consistent hash
+- Select the Traffic Router at at the first position of each location
+- Select the Traffic Router at the second position of each location, stopping after selecting the 6th Traffic Router
+
+Because the algorithm employs consistent hashing, the answers should be consistent as long as the topology remains the same. This approach can be thought of as the consistent hash round robin (CH + RR) selection process.
+
+Example Request Flow
+********************
+The following is an example of the request flow when a client requests the routing name for an example delivery service, ``tr.service.cdn.example.com``. The request flow assumes that the resolver is cold and has yet to build a local cache of lookups, meaning it has to walk the domain hierarchy asking for ``NS`` records until it reaches ``service.cdn.example.com``. This example starts after the resolver has determined which name servers are authoritative for ``cdn.example.com``. Note that [...]
+
+.. figure:: images/edge_tr_example.png
+ :scale: 30%
+ :align: center
+ :alt: Example Request Flow for Edge Traffic Routing
+
+ Example Request Flow for Edge Traffic Routing. Note this picks up when the resolver hits the CDN managed domain.
+
.. _tr-logs:
-Troubleshooting and Log Files
+Troubleshooting and log files
=============================
Traffic Router log files can be found under :file:`/opt/traffic_router/var/log` and :file:`/opt/tomcat/logs`. Initialization and shutdown logs are in :file:`/opt/tomcat/logs/catalina{date}.out`. Application related logging is in :file:`/opt/traffic_router/var/log/traffic_router.log`, while access logs are written to :file:`/opt/traffic_router/var/log/access.log`.
diff --git a/traffic_ops/app/lib/UI/DeliveryService.pm b/traffic_ops/app/lib/UI/DeliveryService.pm
index f515f99..f28654c 100644
--- a/traffic_ops/app/lib/UI/DeliveryService.pm
+++ b/traffic_ops/app/lib/UI/DeliveryService.pm
@@ -898,6 +898,7 @@ sub update {
logs_enabled => $self->paramAsScalar('ds.logs_enabled'),
deep_caching_type => $self->paramAsScalar('ds.deep_caching_type'),
anonymous_blocking_enabled => $self->paramAsScalar('ds.anonymous_blocking_enabled'),
+ max_dns_answers => $self->paramAsScalar('ds.max_dns_answers'),
);
my $typename = $self->typename();
@@ -905,7 +906,6 @@ sub update {
$hash{dns_bypass_ip} = $self->paramAsScalar('ds.dns_bypass_ip');
$hash{dns_bypass_ip6} = $self->paramAsScalar('ds.dns_bypass_ip6');
$hash{dns_bypass_cname} = $self->paramAsScalar('ds.dns_bypass_cname');
- $hash{max_dns_answers} = $self->paramAsScalar('ds.max_dns_answers');
$hash{dns_bypass_ttl} =
$self->paramAsScalar('ds.dns_bypass_ttl') eq ""
? undef
diff --git a/traffic_ops/app/lib/UI/Topology.pm b/traffic_ops/app/lib/UI/Topology.pm
index 17cbb74..7efd8a6 100644
--- a/traffic_ops/app/lib/UI/Topology.pm
+++ b/traffic_ops/app/lib/UI/Topology.pm
@@ -234,6 +234,15 @@ sub gen_crconfig_json {
);
my $r = $rs_param->single;
my $port = ( defined($r) && defined( $r->value ) ) ? $r->value : 80;
+ my $pid = $row->profile->id;
+ my $weight =
+ defined( $param_cache{$pid}->{'weight'} )
+ ? $param_cache{$pid}->{'weight'}
+ : 0.999;
+ my $weight_multiplier =
+ defined( $param_cache{$pid}->{'weightMultiplier'} )
+ ? $param_cache{$pid}->{'weightMultiplier'}
+ : 1000;
$data_obj->{'contentRouters'}->{ $row->host_name }->{'fqdn'} = $row->host_name . "." . $row->domain_name;
$data_obj->{'contentRouters'}->{ $row->host_name }->{'status'} = $row->status->name;
@@ -244,6 +253,15 @@ sub gen_crconfig_json {
$data_obj->{'contentRouters'}->{ $row->host_name }->{'ip'} = $row->ip_address;
$data_obj->{'contentRouters'}->{ $row->host_name }->{'ip6'} = ( $row->ip6_address || "" );
$data_obj->{'contentRouters'}->{ $row->host_name }->{'profile'} = $row->profile->name;
+ $data_obj->{'contentRouters'}->{ $row->host_name }->{'hashCount'} = int( $weight * $weight_multiplier );
+
+ # Add Traffic Router cache groups to edgeLocations if valid lat/long is specified (0, 0 is unlikely to be valid)
+ # This is necessary to enable localization for Traffic Router related DNS records
+ if ( defined( $row->cachegroup->latitude ) && defined( $row->cachegroup->longitude ) &&
+ ($row->cachegroup->latitude + 0) != 0 && ($row->cachegroup->longitude + 0) != 0 ) {
+ $data_obj->{'trafficRouterLocations'}->{ $row->cachegroup->name }->{'latitude'} = $row->cachegroup->latitude + 0;
+ $data_obj->{'trafficRouterLocations'}->{ $row->cachegroup->name }->{'longitude'} = $row->cachegroup->longitude + 0;
+ }
}
elsif ( $row->type->name =~ m/^EDGE/ || $row->type->name =~ m/^MID/ ) {
@@ -454,6 +472,12 @@ sub gen_crconfig_json {
$data_obj->{'deliveryServices'}->{ $row->xml_id }->{'geolocationProvider'} = 'maxmindGeolocationService';
}
+ if ( defined( $row->max_dns_answers )
+ && $row->max_dns_answers ne "" )
+ {
+ $data_obj->{'deliveryServices'}->{ $row->xml_id }->{'maxDnsIpsForLocation'} = $row->max_dns_answers;
+ }
+
if ( $protocol =~ m/DNS/ ) {
#$data_obj->{'deliveryServices'}->{$row->xml_id}->{'matchsets'}->[0]->{'protocol'} = 'DNS';
@@ -475,11 +499,6 @@ sub gen_crconfig_json {
{
$data_obj->{'deliveryServices'}->{ $row->xml_id }->{'bypassDestination'}->{'DNS'}->{'ttl'} = $row->dns_bypass_ttl;
}
- if ( defined( $row->max_dns_answers )
- && $row->max_dns_answers ne "" )
- {
- $data_obj->{'deliveryServices'}->{ $row->xml_id }->{'maxDnsIpsForLocation'} = $row->max_dns_answers;
- }
}
elsif ( $protocol =~ m/HTTP/ ) {
@@ -715,6 +734,10 @@ sub crconfig_strings {
my $return = &stringify_ccr( $config_json->{'contentRouters'}->{$ccr} );
push( @ccr_strings, $return );
}
+ foreach my $trafficRouterLocation ( sort keys %{ $config_json->{'trafficRouterLocations'} } ) {
+ my $return = &stringify_cachegroup( $config_json->{'trafficRouterLocations'}->{$trafficRouterLocation} );
+ push( @ccr_strings, $return );
+ }
return ( \@ds_strings, \@loc_strings, \@cs_strings, \@csds_strings, \@rascal_strings, \@ccr_strings, \@config_strings );
diff --git a/traffic_ops/app/templates/delivery_service/add.html.ep b/traffic_ops/app/templates/delivery_service/add.html.ep
index bb8521b..e47f881 100644
--- a/traffic_ops/app/templates/delivery_service/add.html.ep
+++ b/traffic_ops/app/templates/delivery_service/add.html.ep
@@ -119,7 +119,7 @@
$('#regional_geo_blocking_row').show(speed);
$('#dns_bypass_cname_row').hide(speed);
$('#dns_bypass_ttl_row').show(speed);
- $('#max_dns_answers_row').hide(speed);
+ $('#max_dns_answers_row').show(speed);
$('#org_server_fqdn_row').show(speed);
$('#global_max_mbps_row').show(speed);
$('#global_max_tps_row').show(speed);
@@ -245,7 +245,7 @@
$('#regional_geo_blocking_row').show(speed);
$('#dns_bypass_cname_row').hide(speed);
$('#dns_bypass_ttl_row').hide(speed);
- $('#max_dns_answers_row').hide(speed);
+ $('#max_dns_answers_row').show(speed);
$('#ccr_dns_ttl_row').show(speed);
$('#org_server_fqdn_row').hide(speed);
$('#global_max_mbps_row').hide(speed);
diff --git a/traffic_ops/app/templates/delivery_service/edit.html.ep b/traffic_ops/app/templates/delivery_service/edit.html.ep
index 452a56f..0f3dd14 100644
--- a/traffic_ops/app/templates/delivery_service/edit.html.ep
+++ b/traffic_ops/app/templates/delivery_service/edit.html.ep
@@ -110,7 +110,7 @@
$('#dns_bypass_ip6_row').hide(speed);
$('#dns_bypass_cname_row').hide(speed);
$('#dns_bypass_ttl_row').hide(speed);
- $('#max_dns_answers_row').hide(speed);
+ $('#max_dns_answers_row').show(speed);
$('#http_bypass_fqdn_row').show(speed);
$('#initial_dispersion_row').show(speed);
$('#regional_geo_blocking_row').show(speed);
@@ -213,7 +213,7 @@ function setup_form(speed) {
$('#regional_geo_blocking_row').show(speed);
$('#dns_bypass_cname_row').hide(speed);
$('#dns_bypass_ttl_row').show(speed);
- $('#max_dns_answers_row').hide(speed);
+ $('#max_dns_answers_row').show(speed);
$('#org_server_fqdn_row').show(speed);
$('#global_max_mbps_row').show(speed);
$('#global_max_tps_row').show(speed);
@@ -337,7 +337,7 @@ function setup_form(speed) {
$('#regional_geo_blocking_row').show(speed);
$('#dns_bypass_cname_row').hide(speed);
$('#dns_bypass_ttl_row').hide(speed);
- $('#max_dns_answers_row').hide(speed);
+ $('#max_dns_answers_row').show(speed);
$('#ccr_dns_ttl_row').show(speed);
$('#org_server_fqdn_row').hide(speed);
$('#global_max_mbps_row').hide(speed);
diff --git a/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.HTTP.tpl.html b/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.HTTP.tpl.html
index 457c25a..fae9a39 100644
--- a/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.HTTP.tpl.html
+++ b/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.HTTP.tpl.html
@@ -903,6 +903,22 @@ under the License.
</aside>
</div>
</div>
+ <div class="form-group" ng-class="{'has-error': hasError(routingConfig.maxDnsAnswers), 'has-feedback': hasError(routingConfig.maxDnsAnswers)}">
+ <label class="has-tooltip control-label col-md-2 col-sm-2 col-xs-12" for="maxDnsAnswers">Max DNS Answers<div class="helptooltip">
+ <div class="helptext">
+ Max number of IP addresses in DNS answer (0 means all)
+ </div>
+ </div>
+ </label>
+ <div class="col-md-10 col-sm-10 col-xs-12">
+ <input id="maxDnsAnswers" name="maxDnsAnswers" type="number" class="form-control" placeholder="Max number of IP addresses in DNS answer (0 means all)" ng-model="deliveryService.maxDnsAnswers" step="1" min="0">
+ <small class="input-error" ng-show="hasPropertyError(routingConfig.maxDnsAnswers, 'min')">Too Small (must be greater than or equal to 0)</small>
+ <aside class="current-value" ng-if="settings.isRequest" ng-show="open() && deliveryService.maxDnsAnswers != dsCurrent.maxDnsAnswers">
+ <h3>Current Value</h3>
+ <pre>{{::dsCurrent.maxDnsAnswers}}</pre>
+ </aside>
+ </div>
+ </div>
<div class="form-group" ng-class="{'has-error': hasError(routingConfig.ccrDnsTtl), 'has-feedback': hasError(routingConfig.ccrDnsTtl)}">
<label class="has-tooltip control-label col-md-2 col-sm-2 col-xs-12" for="ccrDnsTtl">Delivery Service DNS TTL<div class="helptooltip">
<div class="helptext">The <abbr title="Time To Live">TTL</abbr> on the DNS record for the Traffic Router A and AAAA records. Setting too high or too low will result in poor caching performance.</div>
diff --git a/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.Steering.tpl.html b/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.Steering.tpl.html
index f3df675..d1baf2c 100644
--- a/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.Steering.tpl.html
+++ b/traffic_portal/app/src/common/modules/form/deliveryService/form.deliveryService.Steering.tpl.html
@@ -425,6 +425,22 @@ under the License.
</aside>
</div>
</div>
+ <div class="form-group" ng-class="{'has-error': hasError(routingConfig.maxDnsAnswers), 'has-feedback': hasError(routingConfig.maxDnsAnswers)}">
+ <label class="has-tooltip control-label col-md-2 col-sm-2 col-xs-12" for="maxDnsAnswers">Max DNS Answers<div class="helptooltip">
+ <div class="helptext">
+ Max number of IP addresses in DNS answer (0 means all)
+ </div>
+ </div>
+ </label>
+ <div class="col-md-10 col-sm-10 col-xs-12">
+ <input id="maxDnsAnswers" name="maxDnsAnswers" type="number" class="form-control" placeholder="Max number of IP addresses in DNS answer (0 means all)" ng-model="deliveryService.maxDnsAnswers" step="1" min="0">
+ <small class="input-error" ng-show="hasPropertyError(routingConfig.maxDnsAnswers, 'min')">Too Small (must be greater than or equal to 0)</small>
+ <aside class="current-value" ng-if="settings.isRequest" ng-show="open() && deliveryService.maxDnsAnswers != dsCurrent.maxDnsAnswers">
+ <h3>Current Value</h3>
+ <pre>{{::dsCurrent.maxDnsAnswers}}</pre>
+ </aside>
+ </div>
+ </div>
<div class="form-group" ng-class="{'has-error': hasError(routingConfig.ccrDnsTtl), 'has-feedback': hasError(routingConfig.ccrDnsTtl)}">
<label class="has-tooltip control-label col-md-2 col-sm-2 col-xs-12" for="ccrDnsTtl">Delivery Service DNS TTL<div class="helptooltip">
<div class="helptext">The <abbr title="Time To Live">TTL</abbr> on the DNS record for the Traffic Router A and AAAA records. Setting too high or too low will result in poor caching performance.</div>
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/ConsistentHashController.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/ConsistentHashController.java
index aa018b7..d9a9a4c 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/ConsistentHashController.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/ConsistentHashController.java
@@ -15,7 +15,7 @@
package com.comcast.cdn.traffic_control.traffic_router.api.controllers;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
import com.comcast.cdn.traffic_control.traffic_router.core.router.TrafficRouterManager;
import org.springframework.beans.factory.annotation.Autowired;
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/CoverageZoneController.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/CoverageZoneController.java
index b47743f..918158f 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/CoverageZoneController.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/CoverageZoneController.java
@@ -15,8 +15,8 @@
package com.comcast.cdn.traffic_control.traffic_router.api.controllers;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation;
import com.comcast.cdn.traffic_control.traffic_router.core.router.TrafficRouterManager;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/DeepCoverageZoneController.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/DeepCoverageZoneController.java
index 56d5ab0..c45f119 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/DeepCoverageZoneController.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/DeepCoverageZoneController.java
@@ -15,7 +15,7 @@
package com.comcast.cdn.traffic_control.traffic_router.api.controllers;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation;
import com.comcast.cdn.traffic_control.traffic_router.core.router.TrafficRouterManager;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/DeliveryServicesController.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/DeliveryServicesController.java
index 4e5213b..45108dc 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/DeliveryServicesController.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/api/controllers/DeliveryServicesController.java
@@ -50,7 +50,7 @@ public class DeliveryServicesController {
}
final TrafficRouter trafficRouter = trafficRouterManager.getTrafficRouter();
- final DeliveryService deliveryService = trafficRouter.getCacheRegister().getDeliveryService(new HTTPRequest(request, decodedUrl), true);
+ final DeliveryService deliveryService = trafficRouter.getCacheRegister().getDeliveryService(new HTTPRequest(request, decodedUrl));
if (deliveryService == null) {
return ResponseEntity.status(HttpStatus.NOT_FOUND).body(null);
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheLocationManager.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheLocationManager.java
deleted file mode 100644
index e3de4f1..0000000
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheLocationManager.java
+++ /dev/null
@@ -1,45 +0,0 @@
-/*
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package com.comcast.cdn.traffic_control.traffic_router.core.cache;
-
-import java.util.Collection;
-import java.util.Map;
-
-/**
- * Provides access to the configured {@link CacheLocation}s and the {@link Cache}s that are a part
- * of them.
- */
-public interface CacheLocationManager {
- /**
- * Gets the {@link CacheLocation} specified by the provided ID.
- *
- * @param id
- * the ID for the desired <code>CacheLocation</code>
- * @return the <code>CacheLocation</code> or null if no location exists for the specified ID.
- */
- CacheLocation getCacheLocation(final String id);
-
- /**
- * Returns the configured {@link CacheLocation}s.
- *
- * @return the configured <code>CacheLocations</code> or an empty {@link Collection} if no
- * locations are configured
- */
- Collection<CacheLocation> getCacheLocations();
-
- void setCacheMap(Map<String,Cache> map);
- Map<String,Cache> getCacheMap();
-}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandler.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandler.java
index a4b6fcd..c0eb40a 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandler.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandler.java
@@ -49,10 +49,13 @@ import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.log4j.Logger;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache.DeliveryServiceReference;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache.DeliveryServiceReference;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Location;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Node;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.TrafficRouterLocation;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryServiceMatcher;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryServiceMatcher.Type;
@@ -221,6 +224,8 @@ public class ConfigHandler {
parseDeliveryServiceMatchSets(deliveryServicesJson, deliveryServiceMap, cacheRegister);
parseLocationConfig(JsonUtils.getJsonNode(jo, "edgeLocations"), cacheRegister);
+
+ parseEdgeTrafficRouterLocations(jo, cacheRegister);
parseCacheConfig(JsonUtils.getJsonNode(jo, "contentServers"), cacheRegister);
parseMonitorConfig(JsonUtils.getJsonNode(jo, "monitors"));
@@ -247,8 +252,8 @@ public class ConfigHandler {
* similarly to the non-deep instance. However, instead of clearing a NetworkNode's CacheLocation, only the
* Caches are cleared from the CacheLocation then lazily loaded at request time.
*/
- NetworkNode.getInstance().clearCacheLocations();
- NetworkNode.getDeepInstance().clearCacheLocations(true);
+ NetworkNode.getInstance().clearLocations();
+ NetworkNode.getDeepInstance().clearLocations(true);
setLastSnapshotTimestamp(sts);
} catch (ParseException e) {
isProcessing.set(false);
@@ -455,8 +460,9 @@ public class ConfigHandler {
}
private void parseDeliveryServiceMatchSets(final JsonNode allDeliveryServices, final Map<String, DeliveryService> deliveryServiceMap, final CacheRegister cacheRegister) throws JsonUtilsException {
- final TreeSet<DeliveryServiceMatcher> dnsServiceMatchers = new TreeSet<>();
- final TreeSet<DeliveryServiceMatcher> httpServiceMatchers = new TreeSet<>();
+ final TreeSet<DeliveryServiceMatcher> deliveryServiceMatchers = new TreeSet<>();
+ final JsonNode config = cacheRegister.getConfig();
+ final boolean regexSuperhackEnabled = JsonUtils.optBoolean(config, "confighandler.regex.superhack.enabled", true);
final Iterator<String> deliveryServiceIds = allDeliveryServices.fieldNames();
while (deliveryServiceIds.hasNext()) {
@@ -465,29 +471,31 @@ public class ConfigHandler {
final JsonNode matchsets = JsonUtils.getJsonNode(deliveryServiceJson, "matchsets");
final DeliveryService deliveryService = deliveryServiceMap.get(deliveryServiceId);
- for (final JsonNode matchset : matchsets) {
- final String protocol = JsonUtils.getString(matchset, "protocol");
-
+ for (int i = 0; i < matchsets.size(); i++) {
+ final JsonNode matchset = matchsets.get(i);
final DeliveryServiceMatcher deliveryServiceMatcher = new DeliveryServiceMatcher(deliveryService);
+ deliveryServiceMatchers.add(deliveryServiceMatcher);
- if ("HTTP".equals(protocol)) {
- httpServiceMatchers.add(deliveryServiceMatcher);
- } else if ("DNS".equals(protocol)) {
- dnsServiceMatchers.add(deliveryServiceMatcher);
- }
-
- for (final JsonNode matcherJo : JsonUtils.getJsonNode(matchset, "matchlist")) {
+ final JsonNode list = JsonUtils.getJsonNode(matchset, "matchlist");
+ for (int j = 0; j < list.size(); j++) {
+ final JsonNode matcherJo = list.get(j);
final Type type = Type.valueOf(JsonUtils.getString(matcherJo, "match-type"));
final String target = JsonUtils.optString(matcherJo, "target");
- deliveryServiceMatcher.addMatch(type, JsonUtils.getString(matcherJo, "regex"), target);
+
+ String regex = JsonUtils.getString(matcherJo, "regex");
+
+ if (regexSuperhackEnabled && i == 0 && j == 0 && type.equals(Type.HOST)) {
+ regex = regex.replaceFirst("^\\.\\*\\\\\\.", "(.*\\\\.|^)");
+ }
+
+ deliveryServiceMatcher.addMatch(type, regex, target);
}
}
}
cacheRegister.setDeliveryServiceMap(deliveryServiceMap);
- cacheRegister.setDnsDeliveryServiceMatchers(dnsServiceMatchers);
- cacheRegister.setHttpDeliveryServiceMatchers(httpServiceMatchers);
+ cacheRegister.setDeliveryServiceMatchers(deliveryServiceMatchers);
initGeoFailedRedirect(deliveryServiceMap, cacheRegister);
}
@@ -516,7 +524,7 @@ public class ConfigHandler {
ds.setGeoRedirectFile(url.getFile());
//try select the ds by the redirect fake HTTPRequest
- final DeliveryService rds = cacheRegister.getDeliveryService(req, true);
+ final DeliveryService rds = cacheRegister.getDeliveryService(req);
if (rds == null || rds.getId() != ds.getId()) {
//the redirect url not belongs to this ds
ds.setGeoRedirectUrlType("NOT_DS_URL");
@@ -858,4 +866,79 @@ public class ConfigHandler {
public boolean isProcessingConfig() {
return isProcessing.get();
}
+
+ private Map<String, Location> getEdgeTrafficRouterLocationMap(final JsonNode jo) {
+ final Map<String, Location> locations = new HashMap<>(jo.size());
+ final Iterator<String> locs = jo.fieldNames();
+ while (locs.hasNext()) {
+ final String loc = locs.next();
+ try {
+ final JsonNode locJo = JsonUtils.getJsonNode(jo, loc);
+ locations.put(loc, new Location(loc, new Geolocation(JsonUtils.getDouble(locJo, "latitude"), JsonUtils.getDouble(locJo, "longitude"))));
+ } catch (JsonUtilsException e) {
+ LOGGER.warn(e, e);
+ }
+ }
+
+ return locations;
+ }
+
+ private void parseEdgeTrafficRouterLocations(final JsonNode jo, final CacheRegister cacheRegister) throws JsonUtilsException {
+ final String locationKey = "location";
+ final JsonNode trafficRouterJo = JsonUtils.getJsonNode(jo, "contentRouters");
+ final Set<TrafficRouterLocation> locations = new HashSet<TrafficRouterLocation>(trafficRouterJo.size());
+ final JsonNode trafficRouterLocJo = jo.get("trafficRouterLocations");
+
+ if (trafficRouterLocJo == null) {
+ LOGGER.warn("No trafficRouterLocations key found in configuration; unable to configure localized traffic routers");
+ return;
+ }
+
+ final Map<String, Location> allLocations = getEdgeTrafficRouterLocationMap(trafficRouterLocJo);
+
+ for (final Iterator<String> trafficRouterNames = trafficRouterJo.fieldNames(); trafficRouterNames.hasNext();) {
+ final String trafficRouterName = trafficRouterNames.next();
+ final JsonNode trafficRouter = trafficRouterJo.get(trafficRouterName);
+ try {
+ final String trLoc = JsonUtils.getString(trafficRouter, locationKey);
+ final Location cl = allLocations.get(trLoc);
+
+ if (cl != null) {
+ LOGGER.info("Edge Traffic Router " + trafficRouterName + " @ " + trLoc + "; " + cl.getGeolocation().toString());
+ locations.add(new TrafficRouterLocation(trLoc, cl.getGeolocation()));
+ } else {
+ LOGGER.error("No trafficRouterLocations configured for " + trLoc + "; unable to use Edge Traffic Router " + trafficRouterName);
+ }
+
+ } catch (JsonUtilsException e) {
+ LOGGER.warn(e,e);
+ }
+ }
+
+ cacheRegister.setEdgeTrafficRouterLocations(locations);
+
+ for (final Iterator<String> trafficRouterNames = trafficRouterJo.fieldNames(); trafficRouterNames.hasNext();) {
+ final String trafficRouterName = trafficRouterNames.next();
+ final JsonNode trafficRouter = trafficRouterJo.get(trafficRouterName);
+ final String location = JsonUtils.getString(trafficRouter, locationKey);
+ final TrafficRouterLocation trafficRouterLocation = cacheRegister.getEdgeTrafficRouterLocation(location);
+
+ if (trafficRouterLocation != null) {
+ final Node edgeTrafficRouter = new Node(trafficRouterName, trafficRouterName, JsonUtils.optInt(jo, "hashCount"));
+ final String ip = JsonUtils.getString(trafficRouter, "ip");
+ final String ip6 = JsonUtils.optString(trafficRouter, "ip6");
+ edgeTrafficRouter.setFqdn(JsonUtils.getString(trafficRouter, "fqdn"));
+ edgeTrafficRouter.setPort(JsonUtils.getInt(trafficRouter, "port"));
+
+ try {
+ edgeTrafficRouter.setIpAddress(ip, ip6, 0);
+ trafficRouterLocation.addTrafficRouter(trafficRouterName, edgeTrafficRouter);
+ } catch (UnknownHostException ex) {
+ LOGGER.warn(ex + " : " + ip);
+ }
+ }
+ }
+
+ cacheRegister.setEdgeTrafficRouterCount();
+ }
}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/SignatureManager.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/SignatureManager.java
index ce3d708..dc5f75a 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/SignatureManager.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/SignatureManager.java
@@ -40,7 +40,7 @@ import org.xbill.DNS.DSRecord;
import org.xbill.DNS.Name;
import org.xbill.DNS.Record;
import org.xbill.DNS.TextParseException;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
import com.comcast.cdn.traffic_control.traffic_router.core.dns.ZoneManager.ZoneCacheType;
import com.comcast.cdn.traffic_control.traffic_router.core.util.TrafficOpsUtils;
import com.comcast.cdn.traffic_control.traffic_router.core.util.ProtectedFetcher;
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManager.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManager.java
index 9c32bb5..de2876a 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManager.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManager.java
@@ -46,7 +46,6 @@ import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
-import com.comcast.cdn.traffic_control.traffic_router.core.router.TrafficRouterManager;
import com.comcast.cdn.traffic_control.traffic_router.core.util.JsonUtils;
import com.comcast.cdn.traffic_control.traffic_router.core.util.JsonUtilsException;
import com.fasterxml.jackson.databind.JsonNode;
@@ -69,19 +68,22 @@ import org.xbill.DNS.TXTRecord;
import org.xbill.DNS.Type;
import org.xbill.DNS.Zone;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache.DeliveryServiceReference;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.InetRecord;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Resolver;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache.DeliveryServiceReference;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.InetRecord;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Resolver;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.TrafficRouterLocation;
import com.comcast.cdn.traffic_control.traffic_router.core.request.DNSRequest;
import com.comcast.cdn.traffic_control.traffic_router.core.router.DNSRouteResult;
import com.comcast.cdn.traffic_control.traffic_router.core.router.StatTracker;
import com.comcast.cdn.traffic_control.traffic_router.core.router.StatTracker.Track;
import com.comcast.cdn.traffic_control.traffic_router.core.router.TrafficRouter;
+import com.comcast.cdn.traffic_control.traffic_router.core.router.TrafficRouterManager;
import com.comcast.cdn.traffic_control.traffic_router.core.util.TrafficOpsUtils;
+import com.comcast.cdn.traffic_control.traffic_router.geolocation.GeolocationException;
import com.google.common.cache.CacheBuilder;
import com.google.common.cache.CacheBuilderSpec;
import com.google.common.cache.CacheLoader;
@@ -110,7 +112,6 @@ public class ZoneManager extends Resolver {
private static SignatureManager signatureManager;
private static Name topLevelDomain;
- private static Set<String> dnsRoutingNames;
private static final String AAAA = "AAAA";
protected static enum ZoneCacheType {
@@ -118,7 +119,6 @@ public class ZoneManager extends Resolver {
}
public ZoneManager(final TrafficRouter tr, final StatTracker statTracker, final TrafficOpsUtils trafficOpsUtils, final TrafficRouterManager trafficRouterManager) throws IOException {
- initDnsRoutingNames(tr.getCacheRegister());
initTopLevelDomain(tr.getCacheRegister());
initSignatureManager(tr.getCacheRegister(), trafficOpsUtils, trafficRouterManager);
initZoneCache(tr);
@@ -136,16 +136,6 @@ public class ZoneManager extends Resolver {
initZoneCache(trafficRouter);
}
- private static void initDnsRoutingNames(final CacheRegister cacheRegister) {
- final Set<String> dnsRoutingNames = new HashSet<>();
- for (final DeliveryService ds : cacheRegister.getDeliveryServices().values()) {
- if (ds.isDns()) {
- dnsRoutingNames.add(ds.getRoutingName());
- }
- }
- setDnsRoutingNames(dnsRoutingNames);
- }
-
@SuppressWarnings("PMD.UseStringBufferForStringAppends")
private static void initTopLevelDomain(final CacheRegister data) throws TextParseException {
String tld = JsonUtils.optString(data.getConfig(), "domain_name");
@@ -479,20 +469,20 @@ public class ZoneManager extends Resolver {
final Name name = newName(domain);
final List<Record> list = zoneMap.get(domain);
final Name admin = newName(ZoneUtils.getAdminString(soa, "admin", "traffic_ops", domain));
- list.add(new SOARecord(name, DClass.IN,
+ list.add(new SOARecord(name, DClass.IN,
ZoneUtils.getLong(ttl, "SOA", 86400), getGlueName(ds, trafficRouters.get(hostname), name, hostname), admin,
- ZoneUtils.getLong(soa, "serial", ZoneUtils.getSerial(data.getStats())),
- ZoneUtils.getLong(soa, "refresh", 28800),
- ZoneUtils.getLong(soa, "retry", 7200),
- ZoneUtils.getLong(soa, "expire", 604800),
+ ZoneUtils.getLong(soa, "serial", ZoneUtils.getSerial(data.getStats())),
+ ZoneUtils.getLong(soa, "refresh", 28800),
+ ZoneUtils.getLong(soa, "retry", 7200),
+ ZoneUtils.getLong(soa, "expire", 604800),
ZoneUtils.getLong(soa, "minimum", 60)));
- addTrafficRouters(list, trafficRouters, name, ttl, domain, ds);
+ addTrafficRouters(list, trafficRouters, name, ttl, domain, ds, tr);
addStaticDnsEntries(list, ds, domain);
final List<Record> records = new ArrayList<Record>();
+ final long maxTTL = ZoneUtils.getMaximumTTL(list);
try {
- final long maxTTL = ZoneUtils.getMaximumTTL(list);
records.addAll(signatureManager.generateDSRecords(name, maxTTL));
list.addAll(signatureManager.generateDNSKEYRecords(name, maxTTL));
} catch (NoSuchAlgorithmException ex) {
@@ -533,25 +523,60 @@ public class ZoneManager extends Resolver {
final CacheRegister data = tr.getCacheRegister();
final JsonNode config = data.getConfig();
final boolean primeDynCache = JsonUtils.optBoolean(config, "dynamic.cache.primer.enabled", true);
-
- // prime the dynamic zone cache
- if (primeDynCache && ds != null && ds.isDns()) {
- primingTasks.add(() -> {
- try {
- primeDNSDeliveryServices(domain, tr, dzc, zone, ds, data);
- } catch (TextParseException ex) {
- LOGGER.fatal("Unable to prime dynamic zone " + domain, ex);
- }
- });
+ if (!primeDynCache || ds == null || (!ds.isDns() && !tr.isEdgeHTTPRouting())){
+ return;
}
+ primingTasks.add(() -> {
+ try {
+ // prime the dynamic zone cache
+ if (ds.isDns()) {
+ primeDNSDeliveryServices(domain, name, tr, dzc, zone, ds, data);
+ } else if (!ds.isDns() && tr.isEdgeHTTPRouting()) {
+ primeHTTPDeliveryServices(domain, tr, dzc, zone, ds, data);
+ }
+ } catch (TextParseException ex) {
+ LOGGER.fatal("Unable to prime dynamic zone " + domain, ex);
+ }
+ });
} catch (ExecutionException ex) {
LOGGER.fatal("Unable to load zone into cache: " + ex.getMessage(), ex);
}
});
}
+ private static void primeHTTPDeliveryServices(final String domain, final TrafficRouter tr, final LoadingCache<ZoneKey, Zone> dzc,
+ final Zone zone, final DeliveryService ds, final CacheRegister data) throws TextParseException {
+ final Name edgeName = newName(ds.getRoutingName(), domain);
+
+ LOGGER.info("Priming " + edgeName);
+
+ final DNSRequest request = new DNSRequest(zone, edgeName, Type.A);
+ request.setDnssec(signatureManager.isDnssecEnabled());
+ request.setHostname(edgeName.toString(true)); // Name.toString(true) - omit the trailing dot
+
+ // prime the miss case first
+ try {
+ final DNSRouteResult result = new DNSRouteResult();
+ result.setAddresses(tr.selectTrafficRoutersMiss(request.getZoneName(), ds));
+ fillDynamicZone(dzc, zone, request, result);
+ } catch (GeolocationException ex) {
+ LOGGER.warn(ex, ex);
+ }
+
+ // prime answers for each of our edge locations
+ for (final TrafficRouterLocation trLocation : data.getEdgeTrafficRouterLocations()) {
+ try {
+ final DNSRouteResult result = new DNSRouteResult();
+ result.setAddresses(tr.selectTrafficRoutersLocalized(trLocation.getGeolocation(), request.getZoneName(), ds));
+ fillDynamicZone(dzc, zone, request, result);
+ } catch (GeolocationException ex) {
+ LOGGER.warn(ex, ex);
+ }
+ }
+ }
+
@SuppressWarnings("PMD.CyclomaticComplexity")
- private static void primeDNSDeliveryServices(final String domain, final TrafficRouter tr, final LoadingCache<ZoneKey, Zone> dzc,
+ private static void primeDNSDeliveryServices(final String domain, final Name name, final TrafficRouter tr, final LoadingCache<ZoneKey, Zone> dzc,
final Zone zone, final DeliveryService ds, final CacheRegister data) throws TextParseException {
final Name edgeName = newName(ds.getRoutingName(), domain);
final JsonNode config = data.getConfig();
@@ -559,7 +584,8 @@ public class ZoneManager extends Resolver {
LOGGER.info("Priming " + edgeName);
- final DNSRequest request = new DNSRequest();
+ final DNSRequest request = new DNSRequest(zone, name, Type.A);
+ request.setDnssec(signatureManager.isDnssecEnabled());
request.setHostname(edgeName.toString(true)); // Name.toString(true) - omit the trailing dot
for (final CacheLocation cacheLocation : data.getCacheLocations()) {
@@ -572,23 +598,39 @@ public class ZoneManager extends Resolver {
// calculate number of permutations if maxDnsIpsForLocation > 0 and we're not using consistent DNS routing
int p = 1;
- if (ds.getMaxDnsIps() > 0 && !tr.isConsistentDNSRouting() && caches.size() > ds.getMaxDnsIps()) {
+ if (ds.isDns() && ds.getMaxDnsIps() > 0 && !tr.isConsistentDNSRouting() && caches.size() > ds.getMaxDnsIps()) {
for (int c = caches.size(); c > (caches.size() - ds.getMaxDnsIps()); c--) {
p *= c;
}
}
- final Set<List<InetRecord>> pset = new HashSet<>();
+ final Set<List<InetRecord>> pset = new HashSet<List<InetRecord>>();
for (int i = 0; i < primerLimit; i++) {
final List<InetRecord> records = tr.inetRecordsFromCaches(ds, caches, request);
+ final DNSRouteResult result = new DNSRouteResult();
+ result.setAddresses(records);
if (!pset.contains(records)) {
- fillDynamicZone(dzc, zone, edgeName, records, signatureManager.isDnssecEnabled());
+ if (!tr.isEdgeDNSRouting()) {
+ fillDynamicZone(dzc, zone, request, result);
+ } else {
+ try {
+ final DNSRouteResult hitResult = new DNSRouteResult();
+ final List<InetRecord> hitRecords = tr.selectTrafficRoutersLocalized(cacheLocation.getGeolocation(), request.getZoneName(), ds);
+ hitRecords.addAll(records);
+ hitResult.setAddresses(hitRecords);
+ fillDynamicZone(dzc, zone, request, hitResult);
+ } catch (GeolocationException ex) {
+ LOGGER.warn(ex, ex);
+ }
+ }
+
pset.add(records);
- LOGGER.debug("Primed " + ds.getId() + " @ " + cacheLocation.getId() + "; permutation " + pset.size() + "/" + p);
}
+ LOGGER.debug("Primed " + ds.getId() + " @ " + cacheLocation.getId() + "; permutation " + pset.size() + "/" + p);
+
if (pset.size() == p) {
break;
}
@@ -671,10 +713,10 @@ public class ZoneManager extends Resolver {
}
}
- @SuppressWarnings("PMD.CyclomaticComplexity")
+ @SuppressWarnings({"PMD.CyclomaticComplexity", "PMD.NPathComplexity"})
private static void addTrafficRouters(final List<Record> list, final JsonNode trafficRouters, final Name name,
- final JsonNode ttl, final String domain, final DeliveryService ds)
- throws TextParseException, UnknownHostException {
+ final JsonNode ttl, final String domain, final DeliveryService ds, final TrafficRouter tr) throws TextParseException, UnknownHostException {
+
final boolean ip6RoutingEnabled = (ds == null || (ds != null && ds.isIp6RoutingEnabled())) ? true : false;
final Iterator<String> keyIter = trafficRouters.fieldNames();
@@ -682,19 +724,20 @@ public class ZoneManager extends Resolver {
final String key = keyIter.next();
final JsonNode trJo = trafficRouters.get(key);
- if(trJo.has("status") && "OFFLINE".equals(trJo.get("status").asText())) {
- // if "status": "OFFLINE"
+ if (!trJo.has("status") || "OFFLINE".equals(trJo.get("status").asText()) || "ADMIN_DOWN".equals(trJo.get("status").asText())) {
continue;
}
final Name trName = newName(key, domain);
- String ip6 = JsonUtils.optString(trJo, IP6);
+ // NSRecords will be replaced later if tr.isEdgeDNSRouting() is true; we need these to allow stub zones to be signed, etc
list.add(new NSRecord(name, DClass.IN, ZoneUtils.getLong(ttl, "NS", 60), getGlueName(ds, trJo, name, key)));
list.add(new ARecord(trName,
DClass.IN, ZoneUtils.getLong(ttl, "A", 60),
InetAddress.getByName(JsonUtils.optString(trJo, IP))));
+ String ip6 = trJo.get("ip6").asText();
+
if (ip6 != null && !ip6.isEmpty() && ip6RoutingEnabled) {
ip6 = ip6.replaceAll("/.*", "");
list.add(new AAAARecord(trName,
@@ -703,7 +746,8 @@ public class ZoneManager extends Resolver {
Inet6Address.getByName(ip6)));
}
- if (ds != null && !ds.isDns()) {
+ // only add static routing name entries for HTTP DSs if necessary
+ if (ds != null && !ds.isDns() && !tr.isEdgeHTTPRouting()) {
addHttpRoutingRecords(list, ds.getRoutingName(), domain, trJo, ttl, ip6RoutingEnabled);
}
}
@@ -840,22 +884,23 @@ public class ZoneManager extends Resolver {
* @return the Zone to use to resolve the specified Name
*/
public Zone getZone(final Name name, final int qtype) {
- Zone result = null;
final Map<ZoneKey, Zone> zoneMap = zoneCache.asMap();
final List<ZoneKey> sorted = new ArrayList<ZoneKey>(zoneMap.keySet());
+ Zone result = null;
+ Name target = name;
+
Collections.sort(sorted);
- // put the superDomains at the beginning of the list so we look there first for DS records
if (qtype == Type.DS) {
- Collections.reverse(sorted);
+ target = new Name(name, 1); // DS records are in the parent zone, change target accordingly
}
for (final ZoneKey key : sorted) {
final Zone zone = zoneMap.get(key);
final Name origin = zone.getOrigin();
- if (name.subdomain(origin)) {
+ if (target.subdomain(origin)) {
result = zone;
break;
}
@@ -870,30 +915,22 @@ public class ZoneManager extends Resolver {
*
* @param staticZone
* The Zone that would normally serve this request
- * @param name
- * the Name that is being requested
- * @param clientAddress
- * the IP address of the requestor
- * @param dnssecRequest
- * whether the client requested DNSSEC
+ * @param builder
+ * DNSAccessRecord.Builder access logging
+ * @param request
+ * DNSRequest representing the query
* @return the new Zone to serve the request or null if the static Zone should be used
*/
- private Zone createDynamicZone(final Zone staticZone, final Name name, final int qtype, final InetAddress clientAddress, final boolean dnssecRequest, final DNSAccessRecord.Builder builder) {
- if (clientAddress==null) {
- return staticZone;
- }
-
- final DNSRequest request = new DNSRequest();
- request.setClientIP(clientAddress.getHostAddress());
- request.setHostname(name.relativize(Name.root).toString());
- request.setQtype(qtype);
+ private Zone createDynamicZone(final Zone staticZone, final DNSAccessRecord.Builder builder, final DNSRequest request) {
final Track track = StatTracker.getTrack();
try {
final DNSRouteResult result = trafficRouter.route(request, track);
if (result != null) {
- return fillDynamicZone(dynamicZoneCache, staticZone, name, result.getAddresses(), dnssecRequest);
+ final Zone dynamicZone = fillDynamicZone(dynamicZoneCache, staticZone, request, result);
+ track.setResultCode(dynamicZone, request.getName(), request.getQueryType());
+ return dynamicZone;
} else {
return null;
}
@@ -909,27 +946,64 @@ public class ZoneManager extends Resolver {
return null;
}
- private static Zone fillDynamicZone(final LoadingCache<ZoneKey, Zone> dzc, final Zone staticZone, final Name name, final List<InetRecord> addresses, final boolean dnssecRequest) {
- if (addresses == null) {
+ @SuppressWarnings({"PMD.CyclomaticComplexity", "PMD.NPathComplexity"})
+ private static Zone fillDynamicZone(final LoadingCache<ZoneKey, Zone> dzc, final Zone staticZone, final DNSRequest request, final DNSRouteResult result) {
+ if (result == null || result.getAddresses() == null) {
return null;
}
try {
- final List<Record> records = createZoneRecords(staticZone);
- int recordsAdded = 0;
+ boolean nsSeen = false;
+ final List<Record> records = new ArrayList<>();
+
+ for (final InetRecord address : result.getAddresses()) {
+ final DeliveryService ds = result.getDeliveryService();
+ Name name = request.getName();
+
+ if (address.getType() == Type.NS) {
+ name = staticZone.getOrigin();
+ } else if (ds != null && (address.getType() == Type.A || address.getType() == Type.AAAA)) {
+ final String routingName = ds.getRoutingName();
+ name = new Name(routingName, staticZone.getOrigin()); // routingname.ds.cdn.tld
+ }
- for (final InetRecord address : addresses) {
final Record record = createRecord(name, address);
if (record != null) {
records.add(record);
- recordsAdded++;
}
+
+ if (record instanceof NSRecord) {
+ nsSeen = true;
+ }
+ }
+
+ // populate the dynamic zone with any static entries that aren't NS records or routing names
+ final Iterator<RRset> it = staticZone.iterator();
+
+ while (it.hasNext()) {
+ final RRset rrset = it.next();
+ final Iterator<Record> rit = rrset.rrs();
+
+ while (rit.hasNext()) {
+ final Record r = rit.next();
+
+ if (r instanceof NSRecord) { // NSRecords are handled below
+ continue;
+ }
+
+ records.add(r);
+ }
+
}
- if (recordsAdded > 0) {
+ if (!records.isEmpty()) {
+ if (!nsSeen) {
+ records.addAll(createZoneNSRecords(staticZone));
+ }
+
try {
- final ZoneKey zoneKey = signatureManager.generateDynamicZoneKey(staticZone.getOrigin(), records, dnssecRequest);
+ final ZoneKey zoneKey = signatureManager.generateDynamicZoneKey(staticZone.getOrigin(), records, request.isDnssec());
final Zone zone = dzc.get(zoneKey);
return zone;
} catch (ExecutionException e) {
@@ -950,6 +1024,16 @@ public class ZoneManager extends Resolver {
if (address.isAlias()) {
record = new CNAMERecord(name, DClass.IN, address.getTTL(), newName(address.getAlias()));
+ } else if (address.getType() == Type.NS) {
+ final Name tld = getTopLevelDomain();
+ String target = address.getTarget();
+
+ // fix up target to be TR host name plus top level domain
+ if (name.subdomain(tld) && !name.equals(tld)) {
+ target = String.format("%s.%s", target.split("\\.", 2)[0], tld.toString());
+ }
+
+ record = new NSRecord(name, DClass.IN, address.getTTL(), newName(target));
} else if (address.isInet4()) { // address instanceof Inet4Address
record = new ARecord(name, DClass.IN, address.getTTL(), address.getAddress());
} else if (address.isInet6()) {
@@ -959,11 +1043,8 @@ public class ZoneManager extends Resolver {
return record;
}
- private static List<Record> createZoneRecords(final Zone staticZone) throws IOException {
+ private static List<Record> createZoneNSRecords(final Zone staticZone) throws IOException {
final List<Record> records = new ArrayList<Record>();
- records.add(staticZone.getSOA());
-
- @SuppressWarnings("unchecked")
final Iterator<Record> ns = staticZone.getNS().rrs();
while (ns.hasNext()) {
@@ -981,7 +1062,6 @@ public class ZoneManager extends Resolver {
final RRset[] answers = sr.answers();
for (final RRset answer : answers) {
- @SuppressWarnings("unchecked")
final Iterator<Record> it = answer.rrs();
while (it.hasNext()) {
@@ -1025,10 +1105,15 @@ public class ZoneManager extends Resolver {
Zone zone = getZone(name);
final InetAddress addr = InetAddress.getByName(address);
final int qtype = (addr instanceof Inet6Address) ? Type.AAAA : Type.A;
- final Zone dynamicZone = createDynamicZone(zone, name, qtype, addr, true, builder);
+ final DNSRequest request = new DNSRequest(zone, name, qtype);
+ request.setClientIP(addr.getHostAddress());
+ request.setHostname(name.relativize(Name.root).toString());
+ request.setDnssec(true);
+
+ final Zone dynamicZone = createDynamicZone(zone, builder, request);
- if (dynamicZone != null) {
- zone = dynamicZone;
+ if (dynamicZone != null) {
+ zone = dynamicZone;
}
if (zone == null) {
@@ -1051,31 +1136,27 @@ public class ZoneManager extends Resolver {
return null;
}
- final SetResponse sr = zone.findRecords(qname, qtype);
-
- if (sr.isSuccessful()) {
- return zone;
- } else if (isDnsRoutingName(qname.toString().toLowerCase().split("\\.")[0])) {
- final Zone dynamicZone = createDynamicZone(zone, qname, qtype, clientAddress, isDnssecRequest, builder);
+ // all queries must be dynamic when edge DNS routing is enabled, as NS RRsets are used for the authority section and must be localized
+ if (!trafficRouter.isEdgeDNSRouting()) {
+ final SetResponse sr = zone.findRecords(qname, qtype);
- if (dynamicZone != null) {
- return dynamicZone;
+ if (sr.isSuccessful()) {
+ return zone;
}
}
- return zone;
- }
+ final DNSRequest request = new DNSRequest(zone, qname, qtype);
+ request.setClientIP(clientAddress.getHostAddress());
+ request.setHostname(qname.relativize(Name.root).toString());
+ request.setDnssec(isDnssecRequest);
- private static boolean isDnsRoutingName(final String routingName) {
- return getDnsRoutingNames().contains(routingName);
- }
+ final Zone dynamicZone = createDynamicZone(zone, builder, request);
- private static Set<String> getDnsRoutingNames() {
- return dnsRoutingNames;
- }
+ if (dynamicZone != null) {
+ return dynamicZone;
+ }
- private static void setDnsRoutingNames(final Set<String> dnsRoutingNames) {
- ZoneManager.dnsRoutingNames = dnsRoutingNames;
+ return zone;
}
public static File getZoneDirectory() {
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryService.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryService.java
index 9db83ab..4a5a8cb 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryService.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/DeliveryService.java
@@ -41,10 +41,11 @@ import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.databind.JsonNode;
import org.apache.log4j.Logger;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.InetRecord;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache.DeliveryServiceReference;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.InetRecord;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Location;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache.DeliveryServiceReference;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation;
import com.comcast.cdn.traffic_control.traffic_router.geolocation.Geolocation;
import com.comcast.cdn.traffic_control.traffic_router.core.request.DNSRequest;
import com.comcast.cdn.traffic_control.traffic_router.core.request.HTTPRequest;
@@ -551,7 +552,7 @@ public class DeliveryService {
return isAvailable;
}
- public boolean isLocationAvailable(final CacheLocation cl) {
+ public boolean isLocationAvailable(final Location cl) {
if(cl==null) {
return false;
}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/Dispersion.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/Dispersion.java
index ee41ff3..108fdf8 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/Dispersion.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/Dispersion.java
@@ -24,7 +24,7 @@ import java.util.SortedMap;
import com.comcast.cdn.traffic_control.traffic_router.core.util.JsonUtils;
import com.fasterxml.jackson.databind.JsonNode;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
public class Dispersion {
public final static int DEFAULT_LIMIT = 1;
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/SteeringResult.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/SteeringResult.java
index b86ccbd..882060e 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/SteeringResult.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/SteeringResult.java
@@ -15,7 +15,7 @@
package com.comcast.cdn.traffic_control.traffic_router.core.ds;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
public class SteeringResult {
private SteeringTarget steeringTarget;
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/Cache.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/Cache.java
new file mode 100644
index 0000000..bfadbea
--- /dev/null
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/Cache.java
@@ -0,0 +1,112 @@
+/*
+ * Copyright 2015 Comcast Cable Communications Management, LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.comcast.cdn.traffic_control.traffic_router.core.edge;
+
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Map;
+
+import com.comcast.cdn.traffic_control.traffic_router.geolocation.Geolocation;
+
+import org.apache.commons.lang3.builder.EqualsBuilder;
+import org.apache.commons.lang3.builder.HashCodeBuilder;
+
+import com.comcast.cdn.traffic_control.traffic_router.core.config.ParseException;
+
+public class Cache extends Node {
+ private final Map<String, DeliveryServiceReference> deliveryServices = new HashMap<String, DeliveryServiceReference>();
+ private final Geolocation geolocation;
+
+ public Cache(final String id, final String hashId, final int hashCount, final Geolocation geolocation) {
+ super(id, hashId, hashCount);
+ this.geolocation = geolocation;
+ }
+
+ public Cache(final String id, final String hashId, final int hashCount) {
+ this(id, hashId, hashCount, null);
+ }
+
+ @Override
+ public boolean equals(final Object obj) {
+ if (this == obj) {
+ return true;
+ } else if (obj instanceof Cache) {
+ final Cache rhs = (Cache) obj;
+ return new EqualsBuilder()
+ .append(getId(), rhs.getId())
+ .isEquals();
+ } else {
+ return false;
+ }
+ }
+
+ public Collection<DeliveryServiceReference> getDeliveryServices() {
+ return deliveryServices.values();
+ }
+
+ public Geolocation getGeolocation() {
+ return geolocation;
+ }
+
+ @Override
+ public int hashCode() {
+ return new HashCodeBuilder(1, 31)
+ .append(getId())
+ .toHashCode();
+ }
+
+ public void setDeliveryServices(final Collection<DeliveryServiceReference> deliveryServices) {
+ for (final DeliveryServiceReference deliveryServiceReference : deliveryServices) {
+ this.deliveryServices.put(deliveryServiceReference.getDeliveryServiceId(), deliveryServiceReference);
+ }
+ }
+
+ public boolean hasDeliveryService(final String deliveryServiceId) {
+ return deliveryServices.containsKey(deliveryServiceId);
+ }
+
+ @Override
+ public String toString() {
+ return "Cache [id=" + id + "] ";
+ }
+
+ /**
+ * Contains a reference to a DeliveryService ID and the FQDN that should be used if this Cache
+ * is used when supporting the DeliveryService.
+ */
+ public static class DeliveryServiceReference {
+ private final String deliveryServiceId;
+ private final String fqdn;
+
+ public DeliveryServiceReference(final String deliveryServiceId, final String fqdn) throws ParseException {
+ if (fqdn.split("\\.", 2).length != 2) {
+ throw new ParseException("Invalid FQDN (" + fqdn + ") on delivery service " + deliveryServiceId + "; please verify the HOST regex(es) in Traffic Ops");
+ }
+
+ this.deliveryServiceId = deliveryServiceId;
+ this.fqdn = fqdn;
+ }
+
+ public String getDeliveryServiceId() {
+ return deliveryServiceId;
+ }
+
+ public String getFqdn() {
+ return fqdn;
+ }
+ }
+}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheLocation.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/CacheLocation.java
similarity index 90%
rename from traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheLocation.java
rename to traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/CacheLocation.java
index 1daa027..b924b0f 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheLocation.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/CacheLocation.java
@@ -13,7 +13,7 @@
* limitations under the License.
*/
-package com.comcast.cdn.traffic_control.traffic_router.core.cache;
+package com.comcast.cdn.traffic_control.traffic_router.core.edge;
import java.util.ArrayList;
import java.util.Arrays;
@@ -32,13 +32,10 @@ import com.comcast.cdn.traffic_control.traffic_router.geolocation.Geolocation;
/**
* A physical location that has caches.
*/
-public class CacheLocation {
+public class CacheLocation extends Location {
public static final Logger LOGGER = Logger.getLogger(CacheLocation.class);
- private final String id;
- private final Geolocation geolocation;
-
private final Map<String, Cache> caches;
private List<String> backupCacheGroups = null;
private boolean useClosestGeoOnBackupFailure = true;
@@ -87,8 +84,7 @@ public class CacheLocation {
final boolean useClosestGeoOnBackupFailure,
final Set<LocalizationMethod> enabledLocalizationMethods
) {
- this.id = id;
- this.geolocation = geolocation;
+ super(id, geolocation);
this.backupCacheGroups = backupCacheGroups;
this.useClosestGeoOnBackupFailure = useClosestGeoOnBackupFailure;
this.enabledLocalizationMethods = enabledLocalizationMethods;
@@ -169,15 +165,6 @@ public class CacheLocation {
}
/**
- * Gets geolocation.
- *
- * @return the geolocation
- */
- public Geolocation getGeolocation() {
- return geolocation;
- }
-
- /**
* Gets backupCacheGroups.
*
* @return the backupCacheGroups
@@ -196,16 +183,6 @@ public class CacheLocation {
}
/**
- * Gets id.
- *
- * @return the id
- */
- public String getId() {
- return id;
- }
-
-
- /**
* Determines if the specified {@link Cache} exists at this location.
*
* @param id
@@ -222,10 +199,4 @@ public class CacheLocation {
.append(getId())
.toHashCode();
}
-
- public Map<String,String> getProperties() {
- final Map<String,String> map = geolocation.getProperties();
- map.put("id", id);
- return map;
- }
}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheRegister.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/CacheRegister.java
similarity index 61%
rename from traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheRegister.java
rename to traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/CacheRegister.java
index fbf4e20..8da63df 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheRegister.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/CacheRegister.java
@@ -13,8 +13,9 @@
* limitations under the License.
*/
-package com.comcast.cdn.traffic_control.traffic_router.core.cache;
+package com.comcast.cdn.traffic_control.traffic_router.core.edge;
+import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
@@ -29,26 +30,26 @@ import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryServiceMat
import com.comcast.cdn.traffic_control.traffic_router.core.request.Request;
@SuppressWarnings("PMD.LooseCoupling")
-public class CacheRegister implements CacheLocationManager {
+public class CacheRegister {
private final Map<String, CacheLocation> configuredLocations;
+ private final Map<String, TrafficRouterLocation> edgeTrafficRouterLocations;
private JsonNode trafficRouters;
private Map<String,Cache> allCaches;
- private TreeSet<DeliveryServiceMatcher> dnsServiceMatchers;
- private TreeSet<DeliveryServiceMatcher> httpServiceMatchers;
+ private TreeSet<DeliveryServiceMatcher> deliveryServiceMatchers;
private Map<String, DeliveryService> dsMap;
private JsonNode config;
private JsonNode stats;
+ private int edgeTrafficRouterCount;
public CacheRegister() {
configuredLocations = new HashMap<String, CacheLocation>();
+ edgeTrafficRouterLocations = new HashMap<String, TrafficRouterLocation>();
}
- @Override
public CacheLocation getCacheLocation(final String id) {
return configuredLocations.get(id);
}
- @Override
public Set<CacheLocation> getCacheLocations() {
final Set<CacheLocation> result = new HashSet<CacheLocation>(configuredLocations.size());
result.addAll(configuredLocations.values());
@@ -56,15 +57,53 @@ public class CacheRegister implements CacheLocationManager {
}
public CacheLocation getCacheLocationById(final String id) {
- for (final CacheLocation cacheLocation : configuredLocations.values()) {
- if (id.equals(cacheLocation.getId())) {
- return cacheLocation;
+ for (final CacheLocation location : configuredLocations.values()) {
+ if (id.equals(location.getId())) {
+ return location;
}
}
return null;
}
+ public TrafficRouterLocation getEdgeTrafficRouterLocation(final String id) {
+ return edgeTrafficRouterLocations.get(id);
+ }
+
+ public List<TrafficRouterLocation> getEdgeTrafficRouterLocations() {
+ return new ArrayList<TrafficRouterLocation>(edgeTrafficRouterLocations.values());
+ }
+
+ public void setEdgeTrafficRouterCount() {
+ int count = 0;
+
+ for (final TrafficRouterLocation location : getEdgeTrafficRouterLocations()) {
+ final List<Node> trafficRouters = location.getTrafficRouters();
+
+ if (trafficRouters == null) {
+ continue;
+ }
+
+ count += trafficRouters.size();
+ }
+
+ edgeTrafficRouterCount = count;
+ }
+
+ public int getEdgeTrafficRouterCount() {
+ return edgeTrafficRouterCount;
+ }
+
+ public List<Node> getAllEdgeTrafficRouters() {
+ final List<Node> edgeTrafficRouters = new ArrayList<>();
+
+ for (final TrafficRouterLocation location : getEdgeTrafficRouterLocations()) {
+ edgeTrafficRouters.addAll(location.getTrafficRouters());
+ }
+
+ return edgeTrafficRouters;
+ }
+
/**
* Sets the configured locations.
*
@@ -78,6 +117,17 @@ public class CacheRegister implements CacheLocationManager {
}
}
+ public void setEdgeTrafficRouterLocations(final Set<TrafficRouterLocation> locations) {
+ edgeTrafficRouterLocations.clear();
+ for (final TrafficRouterLocation newLoc : locations) {
+ edgeTrafficRouterLocations.put(newLoc.getId(), newLoc);
+ }
+ }
+
+ public boolean hasEdgeTrafficRouters() {
+ return !edgeTrafficRouterLocations.isEmpty();
+ }
+
public void setCacheMap(final Map<String,Cache> map) {
allCaches = map;
}
@@ -85,13 +135,9 @@ public class CacheRegister implements CacheLocationManager {
public Map<String,Cache> getCacheMap() {
return allCaches;
}
-
- public void setDnsDeliveryServiceMatchers(final TreeSet<DeliveryServiceMatcher> dnsServices) {
- this.dnsServiceMatchers = dnsServices;
- }
- public void setHttpDeliveryServiceMatchers(final TreeSet<DeliveryServiceMatcher> httpServices) {
- this.httpServiceMatchers = httpServices;
+ public void setDeliveryServiceMatchers(final TreeSet<DeliveryServiceMatcher> matchers) {
+ this.deliveryServiceMatchers = matchers;
}
/**
@@ -101,18 +147,12 @@ public class CacheRegister implements CacheLocationManager {
* the request to match
* @return the DeliveryService that matches the request
*/
- public DeliveryService getDeliveryService(final Request request, final boolean isHttp) {
- TreeSet<DeliveryServiceMatcher> matchers = dnsServiceMatchers;
-
- if (isHttp) {
- matchers = httpServiceMatchers;
- }
-
- if (matchers == null) {
+ public DeliveryService getDeliveryService(final Request request) {
+ if (deliveryServiceMatchers == null) {
return null;
}
- for (final DeliveryServiceMatcher m : matchers) {
+ for (final DeliveryServiceMatcher m : deliveryServiceMatchers) {
if (m.matches(request)) {
return m.getDeliveryService();
}
@@ -125,7 +165,7 @@ public class CacheRegister implements CacheLocationManager {
return dsMap.get(deliveryServiceId);
}
- public List<CacheLocation> filterAvailableLocations(final String deliveryServiceId) {
+ public List<CacheLocation> filterAvailableCacheLocations(final String deliveryServiceId) {
final DeliveryService deliveryService = dsMap.get(deliveryServiceId);
if (deliveryService == null) {
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/InetRecord.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/InetRecord.java
similarity index 66%
rename from traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/InetRecord.java
rename to traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/InetRecord.java
index f30dd85..2076094 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/InetRecord.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/InetRecord.java
@@ -13,7 +13,9 @@
* limitations under the License.
*/
-package com.comcast.cdn.traffic_control.traffic_router.core.cache;
+package com.comcast.cdn.traffic_control.traffic_router.core.edge;
+
+import org.xbill.DNS.Type;
import java.net.Inet4Address;
import java.net.Inet6Address;
@@ -24,18 +26,28 @@ public class InetRecord {
final private InetAddress ad;
final private long ttl;
- final private String alias;
-
+ final private int type;
+ final private String target;
+
public InetRecord(final InetAddress ad, final long ttl) {
this.ad = ad;
this.ttl = ttl;
- this.alias = null;
+ this.target = null;
+ this.type = (ad instanceof Inet4Address) ? Type.A : Type.AAAA;
}
public InetRecord(final String alias, final long ttl) {
this.ad = null;
this.ttl = ttl;
- this.alias = alias;
+ this.target = alias;
+ this.type = Type.CNAME;
+ }
+
+ public InetRecord(final String target, final long ttl, final int type) {
+ this.ad = null;
+ this.target = target;
+ this.ttl = ttl;
+ this.type = type;
}
public boolean isInet4() {
@@ -58,16 +70,25 @@ public class InetRecord {
return "InetRecord{" +
"ad=" + ad +
", ttl=" + ttl +
- ", alias='" + alias + '\'' +
+ ", target='" + target + '\'' +
+ ", type=" + Type.string(type) +
'}';
}
public boolean isAlias() {
- return (alias != null);
+ return (target != null && type == Type.CNAME);
}
public String getAlias() {
- return alias;
+ return target;
+ }
+
+ public String getTarget() {
+ return target;
+ }
+
+ public int getType() {
+ return type;
}
@Override
@@ -78,9 +99,9 @@ public class InetRecord {
final InetRecord that = (InetRecord) o;
- if (ttl != that.ttl) return false;
+ if (ttl != that.ttl || type != that.type) return false;
if (ad != null ? !ad.equals(that.ad) : that.ad != null) return false;
- return !(alias != null ? !alias.equals(that.alias) : that.alias != null);
+ return !(target != null ? !target.equals(that.target) : that.target != null);
}
@@ -88,7 +109,8 @@ public class InetRecord {
public int hashCode() {
int result = ad != null ? ad.hashCode() : 0;
result = 31 * result + (int) (ttl ^ (ttl >>> 32));
- result = 31 * result + (alias != null ? alias.hashCode() : 0);
+ result = 31 * result + (int) (type ^ (type >>> 32));
+ result = 31 * result + (target != null ? target.hashCode() : 0);
return result;
}
}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/Location.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/Location.java
new file mode 100644
index 0000000..76d9fd0
--- /dev/null
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/Location.java
@@ -0,0 +1,92 @@
+/*
+ * Copyright 2015 Comcast Cable Communications Management, LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.comcast.cdn.traffic_control.traffic_router.core.edge;
+
+import java.util.Map;
+
+import org.apache.commons.lang3.builder.EqualsBuilder;
+import org.apache.commons.lang3.builder.HashCodeBuilder;
+
+import com.comcast.cdn.traffic_control.traffic_router.geolocation.Geolocation;
+
+/**
+ * A physical location that has caches.
+ */
+public class Location {
+
+ private final String id;
+ private final Geolocation geolocation;
+
+ /**
+ * Creates a Location with the specified ID at the specified location.
+ *
+ * @param id
+ * the id of the location
+ * @param geolocation
+ * the coordinates of this location
+ */
+ public Location(final String id, final Geolocation geolocation) {
+ this.id = id;
+ this.geolocation = geolocation;
+ }
+
+ @Override
+ public boolean equals(final Object obj) {
+ if (this == obj) {
+ return true;
+ } else if (obj instanceof Location) {
+ final Location rhs = (Location) obj;
+ return new EqualsBuilder()
+ .append(getId(), rhs.getId())
+ .isEquals();
+ } else {
+ return false;
+ }
+ }
+
+ /**
+ * Gets geolocation.
+ *
+ * @return the geolocation
+ */
+ public Geolocation getGeolocation() {
+ return geolocation;
+ }
+
+ /**
+ * Gets id.
+ *
+ * @return the id
+ */
+ public String getId() {
+ return id;
+ }
+
+
+ @Override
+ public int hashCode() {
+ return new HashCodeBuilder(1, 31)
+ .append(getId())
+ .toHashCode();
+ }
+
+ public Map<String,String> getProperties() {
+ final Map<String,String> map = geolocation.getProperties();
+ map.put("id", id);
+ return map;
+ }
+}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/Cache.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/Node.java
similarity index 65%
rename from traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/Cache.java
rename to traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/Node.java
index 26ae125..21c89fc 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/Cache.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/Node.java
@@ -13,67 +13,53 @@
* limitations under the License.
*/
-package com.comcast.cdn.traffic_control.traffic_router.core.cache;
+package com.comcast.cdn.traffic_control.traffic_router.core.edge;
import java.net.Inet6Address;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.ArrayList;
-import java.util.Collection;
-import java.util.HashMap;
import java.util.List;
-import java.util.Map;
-import com.comcast.cdn.traffic_control.traffic_router.core.hash.DefaultHashable;
-import com.comcast.cdn.traffic_control.traffic_router.core.hash.Hashable;
import com.comcast.cdn.traffic_control.traffic_router.core.util.JsonUtils;
-import com.comcast.cdn.traffic_control.traffic_router.geolocation.Geolocation;
import com.fasterxml.jackson.databind.JsonNode;
+import com.comcast.cdn.traffic_control.traffic_router.core.hash.DefaultHashable;
import org.apache.commons.lang3.builder.EqualsBuilder;
import org.apache.commons.lang3.builder.HashCodeBuilder;
import org.apache.log4j.Logger;
-import com.comcast.cdn.traffic_control.traffic_router.core.config.ParseException;
-
-public class Cache implements Comparable<Cache>, Hashable<Cache> {
- private static final Logger LOGGER = Logger.getLogger(Cache.class);
+public class Node extends DefaultHashable {
+ private static final Logger LOGGER = Logger.getLogger(Node.class);
private static final int REPLICAS = 1000;
public enum IpVersions {
IPV4ONLY, IPV6ONLY, BOTH
}
private IpVersions ipAvailableVersions;
- private final String id;
+ protected final String id;
private String fqdn;
private List<InetRecord> ipAddresses;
private List<InetRecord> unavailableIpAddresses;
private int port;
- private final Map<String, DeliveryServiceReference> deliveryServices = new HashMap<String, DeliveryServiceReference>();
- private final Geolocation geolocation;
- private final Hashable hashable = new DefaultHashable();
private int httpsPort = 443;
- public Cache(final String id, final String hashId, final int hashCount, final Geolocation geolocation) {
+ public Node(final String id) {
this.id = id;
- hashable.generateHashes(hashId, hashCount > 0 ? hashCount : REPLICAS);
- this.geolocation = geolocation;
+ generateHashes(id, REPLICAS);
}
- public Cache(final String id, final String hashId, final int hashCount) {
- this(id, hashId, hashCount, null);
- }
-
- @Override
- public int compareTo(final Cache o) {
- return getId().compareTo(o.getId());
+ // alternate constructor
+ public Node(final String id, final String hashId, final int hashCount) {
+ this.id = id;
+ generateHashes(hashId, hashCount > 0 ? hashCount : REPLICAS);
}
@Override
public boolean equals(final Object obj) {
if (this == obj) {
return true;
- } else if (obj instanceof Cache) {
- final Cache rhs = (Cache) obj;
+ } else if (obj instanceof Node) {
+ final Node rhs = (Node) obj;
return new EqualsBuilder()
.append(getId(), rhs.getId())
.isEquals();
@@ -82,14 +68,6 @@ public class Cache implements Comparable<Cache>, Hashable<Cache> {
}
}
- public Collection<DeliveryServiceReference> getDeliveryServices() {
- return deliveryServices.values();
- }
-
- public Geolocation getGeolocation() {
- return geolocation;
- }
-
public String getFqdn() {
return fqdn;
}
@@ -141,16 +119,6 @@ public class Cache implements Comparable<Cache>, Hashable<Cache> {
.toHashCode();
}
- public void setDeliveryServices(final Collection<DeliveryServiceReference> deliveryServices) {
- for (final DeliveryServiceReference deliveryServiceReference : deliveryServices) {
- this.deliveryServices.put(deliveryServiceReference.getDeliveryServiceId(), deliveryServiceReference);
- }
- }
-
- public boolean hasDeliveryService(final String deliveryServiceId) {
- return deliveryServices.containsKey(deliveryServiceId);
- }
-
public void setFqdn(final String fqdn) {
this.fqdn = fqdn;
}
@@ -165,33 +133,7 @@ public class Cache implements Comparable<Cache>, Hashable<Cache> {
@Override
public String toString() {
- return "Cache [id=" + id + "] ";
- }
-
- /**
- * Contains a reference to a DeliveryService ID and the FQDN that should be used if this Cache
- * is used when supporting the DeliveryService.
- */
- public static class DeliveryServiceReference {
- private final String deliveryServiceId;
- private final String fqdn;
-
- public DeliveryServiceReference(final String deliveryServiceId, final String fqdn) throws ParseException {
- if (fqdn.split("\\.", 2).length != 2) {
- throw new ParseException("Invalid FQDN (" + fqdn + ") on delivery service " + deliveryServiceId + "; please verify the HOST regex(es) in Traffic Ops");
- }
-
- this.deliveryServiceId = deliveryServiceId;
- this.fqdn = fqdn;
- }
-
- public String getDeliveryServiceId() {
- return deliveryServiceId;
- }
-
- public String getFqdn() {
- return fqdn;
- }
+ return "Node [id=" + id + "] ";
}
boolean isAvailable = false;
@@ -283,22 +225,6 @@ public class Cache implements Comparable<Cache>, Hashable<Cache> {
}
- @Override
- public Hashable<Cache> generateHashes(final String hashId, final int hashCount) {
- hashable.generateHashes(hashId, hashCount);
- return this;
- }
-
- @Override
- public double getClosestHash(final double hash) {
- return hashable.getClosestHash(hash);
- }
-
- @Override
- public List<Double> getHashValues() {
- return hashable.getHashValues();
- }
-
public int getHttpsPort() {
return httpsPort;
}
@@ -307,21 +233,6 @@ public class Cache implements Comparable<Cache>, Hashable<Cache> {
this.httpsPort = httpsPort;
}
- @Override
- public boolean hasHashes() {
- return hashable.hasHashes();
- }
-
- @Override
- public int getOrder() {
- return hashable.getOrder();
- }
-
- @Override
- public void setOrder(final int order) {
- hashable.setOrder(order);
- }
-
public IpVersions getIpAvailableVersions() {
return ipAvailableVersions;
}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/Resolver.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/Resolver.java
similarity index 96%
rename from traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/Resolver.java
rename to traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/Resolver.java
index 56c25a1..0547df2 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/Resolver.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/Resolver.java
@@ -13,7 +13,7 @@
* limitations under the License.
*/
-package com.comcast.cdn.traffic_control.traffic_router.core.cache;
+package com.comcast.cdn.traffic_control.traffic_router.core.edge;
import java.net.Inet4Address;
import java.net.InetAddress;
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/TrafficRouterLocation.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/TrafficRouterLocation.java
new file mode 100644
index 0000000..8535a2f
--- /dev/null
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/TrafficRouterLocation.java
@@ -0,0 +1,90 @@
+/*
+ * Copyright 2015 Comcast Cable Communications Management, LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.comcast.cdn.traffic_control.traffic_router.core.edge;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.commons.lang3.builder.EqualsBuilder;
+import org.apache.commons.lang3.builder.HashCodeBuilder;
+
+import com.comcast.cdn.traffic_control.traffic_router.geolocation.Geolocation;
+
+/**
+ * A physical location that has caches.
+ */
+public class TrafficRouterLocation extends Location {
+ private final Map<String, Node> trafficRouters;
+
+ /**
+ * Creates a TrafficRouteRLocation with the specified ID at the specified location.
+ *
+ * @param id
+ * the id of the location
+ * @param geolocation
+ * the coordinates of this location
+ */
+ public TrafficRouterLocation(final String id, final Geolocation geolocation) {
+ super(id, geolocation);
+ trafficRouters = new HashMap<String, Node>();
+ }
+
+ /**
+ * Adds the specified cache to this location.
+ *
+ * @param name
+ * the name of the Traffic Router to add
+ * @param trafficRouter
+ * the Node representing a Traffic Router
+ */
+ public void addTrafficRouter(final String name, final Node trafficRouter) {
+ trafficRouters.put(name, trafficRouter);
+ }
+
+ @Override
+ public boolean equals(final Object obj) {
+ if (this == obj) {
+ return true;
+ } else if (obj instanceof TrafficRouterLocation) {
+ final TrafficRouterLocation rhs = (TrafficRouterLocation) obj;
+ return new EqualsBuilder()
+ .append(getId(), rhs.getId())
+ .isEquals();
+ } else {
+ return false;
+ }
+ }
+
+ @Override
+ public int hashCode() {
+ return new HashCodeBuilder(1, 31)
+ .append(getId())
+ .toHashCode();
+ }
+
+ /**
+ * Retrieves the {@link Set} of Traffic Routers at this location.
+ *
+ * @return the caches
+ */
+ public List<Node> getTrafficRouters() {
+ return new ArrayList<Node>(trafficRouters.values());
+ }
+}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/AnonymousIp.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/AnonymousIp.java
index b61af09..83abfe9 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/AnonymousIp.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/AnonymousIp.java
@@ -25,7 +25,7 @@ import com.google.common.net.InetAddresses;
import com.maxmind.geoip2.model.AnonymousIpResponse;
import org.apache.log4j.Logger;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
import com.comcast.cdn.traffic_control.traffic_router.core.request.HTTPRequest;
import com.comcast.cdn.traffic_control.traffic_router.core.request.Request;
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/FederationRegistry.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/FederationRegistry.java
index 40ffe4f..56c0d56 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/FederationRegistry.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/FederationRegistry.java
@@ -15,7 +15,7 @@
package com.comcast.cdn.traffic_control.traffic_router.core.loc;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.InetRecord;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.InetRecord;
import com.comcast.cdn.traffic_control.traffic_router.core.util.CidrAddress;
import com.comcast.cdn.traffic_control.traffic_router.core.util.ComparableTreeSet;
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNode.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNode.java
index 9d292ae..138964e 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNode.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/NetworkNode.java
@@ -27,6 +27,8 @@ import java.util.Map;
import java.util.Set;
import java.util.TreeMap;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Location;
import com.comcast.cdn.traffic_control.traffic_router.core.util.CidrAddress;
import com.comcast.cdn.traffic_control.traffic_router.core.util.JsonUtils;
import com.comcast.cdn.traffic_control.traffic_router.core.util.JsonUtilsException;
@@ -36,8 +38,6 @@ import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.log4j.Logger;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation;
-
public class NetworkNode implements Comparable<NetworkNode> {
private static final Logger LOGGER = Logger.getLogger(NetworkNode.class);
private static final String DEFAULT_SUB_STR = "0.0.0.0/0";
@@ -47,7 +47,7 @@ public class NetworkNode implements Comparable<NetworkNode> {
private CidrAddress cidrAddress;
private String loc;
- private CacheLocation cacheLocation = null;
+ private Location location = null;
private Geolocation geolocation = null;
protected Map<NetworkNode,NetworkNode> children;
private Set<String> deepCacheNames;
@@ -155,7 +155,7 @@ public class NetworkNode implements Comparable<NetworkNode> {
// access to the latest CacheRegister, similar to how normal NetworkNodes are lazily loaded
// with a CacheLocation.
nn.setDeepCacheNames(cacheNames);
- nn.setCacheLocation(deepLoc);
+ nn.setLocation(deepLoc);
}
if ("network6".equals(key)) {
root.add6(nn);
@@ -281,12 +281,12 @@ public class NetworkNode implements Comparable<NetworkNode> {
return geolocation;
}
- public CacheLocation getCacheLocation() {
- return cacheLocation;
+ public Location getLocation() {
+ return location;
}
- public void setCacheLocation(final CacheLocation cacheLocation) {
- this.cacheLocation = cacheLocation;
+ public void setLocation(final Location location) {
+ this.location = location;
}
public Set<String> getDeepCacheNames() {
@@ -311,16 +311,16 @@ public class NetworkNode implements Comparable<NetworkNode> {
return size;
}
- public void clearCacheLocations() {
- clearCacheLocations(false);
+ public void clearLocations() {
+ clearLocations(false);
}
- public void clearCacheLocations(final boolean clearCachesOnly) {
+ public void clearLocations(final boolean clearCachesOnly) {
synchronized(this) {
- if (clearCachesOnly && cacheLocation != null) {
- cacheLocation.clearCaches();
+ if (clearCachesOnly && location != null && location instanceof CacheLocation) {
+ ((CacheLocation) location).clearCaches();
} else {
- cacheLocation = null;
+ location = null;
}
if (this instanceof SuperNode) {
@@ -328,14 +328,14 @@ public class NetworkNode implements Comparable<NetworkNode> {
if (superNode.children6 != null) {
for (final NetworkNode child : superNode.children6.keySet()) {
- child.clearCacheLocations(clearCachesOnly);
+ child.clearLocations(clearCachesOnly);
}
}
}
if (children != null) {
for (final NetworkNode child : children.keySet()) {
- child.clearCacheLocations(clearCachesOnly);
+ child.clearLocations(clearCachesOnly);
}
}
}
@@ -393,4 +393,4 @@ public class NetworkNode implements Comparable<NetworkNode> {
return "[" + str + "/" + cidrAddress.getNetmaskLength() + "] - location:" + this.getLoc();
}
-}
+}
\ No newline at end of file
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/RegionalGeo.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/RegionalGeo.java
index 7805d91..d302322 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/RegionalGeo.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/RegionalGeo.java
@@ -27,8 +27,8 @@ import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.log4j.Logger;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
import com.comcast.cdn.traffic_control.traffic_router.core.loc.RegionalGeoResult.RegionalGeoResultType;
import com.comcast.cdn.traffic_control.traffic_router.core.request.HTTPRequest;
import com.comcast.cdn.traffic_control.traffic_router.core.request.Request;
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/monitor/TrafficMonitorWatcher.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/monitor/TrafficMonitorWatcher.java
index da9f84e..f571376 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/monitor/TrafficMonitorWatcher.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/monitor/TrafficMonitorWatcher.java
@@ -297,7 +297,7 @@ public class TrafficMonitorWatcher implements ApplicationListener<ApplicationCon
String hostList = System.getenv("TRAFFIC_MONITOR_HOSTS");
if (hostList != null && !hostList.isEmpty()) {
- LOGGER.warn("hostlist initialized to '" + hostList + "' from env var 'TRAFFIC_MONITOR_HOSTS");
+ LOGGER.warn("hostlist initialized to '" + hostList + "' from env var 'TRAFFIC_MONITOR_HOSTS'");
}
if (hostList == null || hostList.isEmpty()) {
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/request/DNSRequest.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/request/DNSRequest.java
index bd218a0..e30b809 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/request/DNSRequest.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/request/DNSRequest.java
@@ -15,19 +15,53 @@
package com.comcast.cdn.traffic_control.traffic_router.core.request;
+import org.xbill.DNS.Name;
+import org.xbill.DNS.Zone;
+
public class DNSRequest extends Request {
- private int qtype;
+ private final Name name;
+ private final String zoneName;
+ private final int queryType;
+ private boolean dnssec = false;
+
+ public DNSRequest(final String zoneName, final Name name, final int queryType) {
+ super();
- public int getQtype() {
- return qtype;
+ this.queryType = queryType;
+ this.name = name;
+ this.zoneName = zoneName;
}
- public void setQtype(final int qtype) {
- this.qtype = qtype;
+ public DNSRequest(final Zone zone, final Name name, final int queryType) {
+ super();
+
+ this.queryType = queryType;
+ this.name = name;
+ this.zoneName = zone.getOrigin().toString().toLowerCase();
+ }
+
+ public int getQueryType() {
+ return queryType;
}
@Override
public String getType() {
return "dns";
}
+
+ public boolean isDnssec() {
+ return dnssec;
+ }
+
+ public void setDnssec(final boolean dnssec) {
+ this.dnssec = dnssec;
+ }
+
+ public Name getName() {
+ return name;
+ }
+
+ public String getZoneName() {
+ return zoneName;
+ }
}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/DNSRouteResult.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/DNSRouteResult.java
index ba81318..cb7db8f 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/DNSRouteResult.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/DNSRouteResult.java
@@ -15,12 +15,15 @@
package com.comcast.cdn.traffic_control.traffic_router.core.router;
+import java.util.ArrayList;
import java.util.List;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.InetRecord;
+import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.InetRecord;
public class DNSRouteResult implements RouteResult {
private List<InetRecord> addresses;
+ private DeliveryService deliveryService;
public Object getResult() {
return getAddresses();
@@ -33,4 +36,20 @@ public class DNSRouteResult implements RouteResult {
public void setAddresses(final List<InetRecord> addresses) {
this.addresses = addresses;
}
+
+ public void addAddresses(final List<InetRecord> addresses) {
+ if (this.addresses == null) {
+ this.addresses = new ArrayList<>();
+ }
+
+ this.addresses.addAll(addresses);
+ }
+
+ public DeliveryService getDeliveryService() {
+ return deliveryService;
+ }
+
+ public void setDeliveryService(final DeliveryService deliveryService) {
+ this.deliveryService = deliveryService;
+ }
}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/CacheLocationComparator.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/LocationComparator.java
similarity index 80%
rename from traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/CacheLocationComparator.java
rename to traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/LocationComparator.java
index bc64c4a..0606c05 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/CacheLocationComparator.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/LocationComparator.java
@@ -17,20 +17,20 @@ package com.comcast.cdn.traffic_control.traffic_router.core.router;
import java.util.Comparator;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Location;
import com.comcast.cdn.traffic_control.traffic_router.geolocation.Geolocation;
/**
*
*/
-public class CacheLocationComparator implements Comparator<CacheLocation> {
+public class LocationComparator implements Comparator<Location> {
private final Geolocation sourceLocation;
/**
* @param sourceLocation
*/
- public CacheLocationComparator(final Geolocation sourceLocation) {
+ public LocationComparator(final Geolocation sourceLocation) {
this.sourceLocation = sourceLocation;
}
@@ -40,7 +40,7 @@ public class CacheLocationComparator implements Comparator<CacheLocation> {
* @see java.util.Comparator#compare(java.lang.Object, java.lang.Object)
*/
@Override
- public int compare(final CacheLocation loc1, final CacheLocation loc2) {
+ public int compare(final Location loc1, final Location loc2) {
final Double d1 = sourceLocation.getDistanceFrom(loc1.getGeolocation());
final Double d2 = sourceLocation.getDistanceFrom(loc2.getGeolocation());
return d1.compareTo(d2);
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/StatTracker.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/StatTracker.java
index b4cb3f7..1b9d74c 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/StatTracker.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/StatTracker.java
@@ -19,11 +19,15 @@ import java.util.HashMap;
import java.util.List;
import java.util.Map;
+import org.xbill.DNS.Name;
+import org.xbill.DNS.SetResponse;
+import org.xbill.DNS.Zone;
+
import com.comcast.cdn.traffic_control.traffic_router.geolocation.Geolocation;
import com.comcast.cdn.traffic_control.traffic_router.core.loc.RegionalGeoResult;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
import com.comcast.cdn.traffic_control.traffic_router.core.router.StatTracker.Track.ResultType;
import com.comcast.cdn.traffic_control.traffic_router.core.router.StatTracker.Track.RouteType;
@@ -116,12 +120,18 @@ public class StatTracker {
public enum ResultDetails {
NO_DETAILS, DS_NOT_FOUND, DS_TLS_MISMATCH, DS_NO_BYPASS, DS_BYPASS, DS_CZ_ONLY, DS_CLIENT_GEO_UNSUPPORTED, GEO_NO_CACHE_FOUND,
- REGIONAL_GEO_NO_RULE, REGIONAL_GEO_ALTERNATE_WITHOUT_CACHE, REGIONAL_GEO_ALTERNATE_WITH_CACHE, DS_CZ_BACKUP_CG
+ REGIONAL_GEO_NO_RULE, REGIONAL_GEO_ALTERNATE_WITHOUT_CACHE, REGIONAL_GEO_ALTERNATE_WITH_CACHE, DS_CZ_BACKUP_CG,
+ DS_INVALID_ROUTING_NAME, LOCALIZED_DNS
+ }
+
+ public enum ResultCode {
+ NO_RESULT_CODE, NXDOMAIN, NODATA
}
long time;
RouteType routeType;
String fqdn;
+ ResultCode resultCode = ResultCode.NO_RESULT_CODE;
ResultType result = ResultType.ERROR;
ResultDetails resultDetails;
Geolocation resultLocation;
@@ -146,6 +156,20 @@ public class StatTracker {
this.routeType = routeType;
this.fqdn = fqdn;
}
+ public void setResultCode(final Zone zone, final Name qname, final int qtype) {
+ if (zone == null) {
+ return;
+ }
+ final SetResponse sr = zone.findRecords(qname, qtype);
+ if (sr.isNXDOMAIN()) {
+ resultCode = ResultCode.NXDOMAIN;
+ } else if (sr.isNXRRSET()) {
+ resultCode = ResultCode.NODATA;
+ }
+ }
+ public ResultCode getResultCode() {
+ return resultCode;
+ }
public void setResult(final ResultType result) {
this.result = result;
}
@@ -212,6 +236,7 @@ public class StatTracker {
final private Map<String, Tallies> dnsMap = new HashMap<String, Tallies>();
final private Map<String, Tallies> httpMap = new HashMap<String, Tallies>();
+
public Map<String, Tallies> getDnsMap() {
return dnsMap;
}
@@ -264,21 +289,23 @@ public class StatTracker {
synchronized(this) {
Map<String,Tallies> map;
- if(t.routeType == RouteType.DNS) {
+ final String fqdn = t.fqdn == null ? "null" : t.fqdn;
+ if (t.routeType == RouteType.DNS) {
totalDnsCount++;
totalDnsTime += t.time;
map = dnsMap;
+
+ if (t.resultDetails == Track.ResultDetails.LOCALIZED_DNS) {
+ return;
+ }
+
} else {
totalHttpCount++;
totalHttpTime += t.time;
map = httpMap;
}
- Tallies tallies = map.get(t.fqdn);
- if(tallies == null) {
- tallies = new Tallies();
- map.put((t.fqdn==null)?"null":t.fqdn, tallies);
- }
- incTally(t, tallies);
+ map.putIfAbsent(fqdn, new Tallies());
+ incTally(t, map.get(fqdn));
}
}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java
index c8b7807..78bbdd2 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java
@@ -48,13 +48,17 @@ import org.apache.log4j.Logger;
import org.springframework.beans.BeansException;
import org.springframework.context.ApplicationContext;
import org.xbill.DNS.Name;
+import org.xbill.DNS.Type;
import org.xbill.DNS.Zone;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation.LocalizationMethod;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.InetRecord;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation.LocalizationMethod;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Node;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.InetRecord;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Location;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.TrafficRouterLocation;
import com.comcast.cdn.traffic_control.traffic_router.core.dns.ZoneManager;
import com.comcast.cdn.traffic_control.traffic_router.core.dns.DNSAccessRecord;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
@@ -78,13 +82,15 @@ import com.comcast.cdn.traffic_control.traffic_router.core.router.StatTracker.Tr
import com.comcast.cdn.traffic_control.traffic_router.core.loc.AnonymousIp;
import com.comcast.cdn.traffic_control.traffic_router.core.loc.AnonymousIpDatabaseService;
-@SuppressWarnings("PMD.ExcessivePublicCount")
+@SuppressWarnings({"PMD.TooManyFields", "PMD.ExcessivePublicCount"})
public class TrafficRouter {
public static final Logger LOGGER = Logger.getLogger(TrafficRouter.class);
public static final String XTC_STEERING_OPTION = "x-tc-steering-option";
public static final String CLIENT_STEERING_DIVERSITY = "client.steering.forced.diversity";
public static final String DNSSEC_ENABLED = "dnssec.enabled";
public static final String DNSSEC_ZONE_DIFFING = "dnssec.zone.diffing.enabled";
+ private static final long DEFAULT_EDGE_NS_TTL = 3600;
+ private static final int DEFAULT_EDGE_TR_LIMIT = 4;
private final CacheRegister cacheRegister;
private final ZoneManager zoneManager;
@@ -95,10 +101,15 @@ public class TrafficRouter {
private final boolean consistentDNSRouting;
private final boolean clientSteeringDiversityEnabled;
private final boolean dnssecZoneDiffingEnabled;
+ private final boolean edgeDNSRouting;
+ private final boolean edgeHTTPRouting;
+ private final long edgeNSttl; // 1 hour default
+ private final int edgeDNSRoutingLimit;
+ private final int edgeHTTPRoutingLimit;
private final Random random = new Random(System.nanoTime());
private Set<String> requestHeaders = new HashSet<String>();
- private static final Geolocation GEO_ZERO_ZERO = new Geolocation(0,0);
+ private static final Geolocation GEO_ZERO_ZERO = new Geolocation(0, 0);
private ApplicationContext applicationContext;
private final ConsistentHasher consistentHasher = new ConsistentHasher();
@@ -119,10 +130,11 @@ public class TrafficRouter {
this.geolocationService6 = geolocationService6;
this.anonymousIpService = anonymousIpService;
this.federationRegistry = federationRegistry;
- this.consistentDNSRouting = JsonUtils.optBoolean(cr.getConfig(), "consistent.dns.routing");
this.clientSteeringDiversityEnabled = JsonUtils.optBoolean(cr.getConfig(), CLIENT_STEERING_DIVERSITY);
this.dnssecZoneDiffingEnabled = JsonUtils.optBoolean(cr.getConfig(), DNSSEC_ENABLED) && JsonUtils.optBoolean(cr.getConfig(), DNSSEC_ZONE_DIFFING);
- this.zoneManager = new ZoneManager(this, statTracker, trafficOpsUtils, trafficRouterManager);
+ this.consistentDNSRouting = JsonUtils.optBoolean(cr.getConfig(), "consistent.dns.routing"); // previous/default behavior
+ this.edgeDNSRouting = JsonUtils.optBoolean(cr.getConfig(), "edge.dns.routing") && cr.hasEdgeTrafficRouters();
+ this.edgeHTTPRouting = JsonUtils.optBoolean(cr.getConfig(), "edge.http.routing") && cr.hasEdgeTrafficRouters();
if (cr.getConfig() != null) {
// maxmindDefaultOverride: {countryCode: , lat: , long: }
@@ -136,6 +148,18 @@ public class TrafficRouter {
}
}
}
+
+ final JsonNode ttls = cacheRegister.getConfig().get("ttls");
+
+ if (ttls != null && ttls.has("NS")) {
+ this.edgeNSttl = JsonUtils.optLong(ttls, "NS");
+ } else {
+ this.edgeNSttl = DEFAULT_EDGE_NS_TTL;
+ }
+
+ this.edgeDNSRoutingLimit = JsonUtils.optInt(cr.getConfig(), "edge.dns.limit", DEFAULT_EDGE_TR_LIMIT);
+ this.edgeHTTPRoutingLimit = JsonUtils.optInt(cr.getConfig(), "edge.http.limit", DEFAULT_EDGE_TR_LIMIT); // NOTE: this can be overridden per-DS via maxDnsAnswers
+ this.zoneManager = new ZoneManager(this, statTracker, trafficOpsUtils, trafficRouterManager);
}
public ZoneManager getZoneManager() {
@@ -170,13 +194,14 @@ public class TrafficRouter {
public CacheRegister getCacheRegister() {
return cacheRegister;
}
- protected DeliveryService selectDeliveryService(final Request request, final boolean isHttp) {
- if(cacheRegister==null) {
+
+ protected DeliveryService selectDeliveryService(final Request request) {
+ if (cacheRegister == null) {
LOGGER.warn("no caches yet");
return null;
}
- final DeliveryService deliveryService = cacheRegister.getDeliveryService(request, isHttp);
+ final DeliveryService deliveryService = cacheRegister.getDeliveryService(request);
if (LOGGER.isDebugEnabled()) {
LOGGER.debug("Selected DeliveryService: " + deliveryService);
@@ -204,7 +229,9 @@ public class TrafficRouter {
return false;
}
final Map<String, Cache> cacheMap = cacheRegister.getCacheMap();
- if(cacheMap == null) { return false; }
+ if (cacheMap == null) {
+ return false;
+ }
for (final String cacheName : cacheMap.keySet()) {
final String monitorCacheName = cacheName.replaceFirst("@.*", "");
final JsonNode state = cacheStates.get(monitorCacheName);
@@ -267,7 +294,8 @@ public class TrafficRouter {
final int locationLimit = ds.getLocationLimit();
final List<CacheLocation> geoEnabledCacheLocations = filterEnabledLocations(getCacheRegister().getCacheLocations(), LocalizationMethod.GEO);
final List<CacheLocation> cacheLocations1 = ds.filterAvailableLocations(geoEnabledCacheLocations);
- final List<CacheLocation> cacheLocations = orderCacheLocations(cacheLocations1, clientLocation);
+ @SuppressWarnings("unchecked")
+ final List<CacheLocation> cacheLocations = (List<CacheLocation>) orderLocations(cacheLocations1, clientLocation);
for (final CacheLocation location : cacheLocations) {
final List<Cache> caches = selectCaches(location, ds);
@@ -279,7 +307,7 @@ public class TrafficRouter {
return caches;
}
locationsTested++;
- if(locationLimit != 0 && locationsTested >= locationLimit) {
+ if (locationLimit != 0 && locationsTested >= locationLimit) {
return null;
}
}
@@ -348,7 +376,7 @@ public class TrafficRouter {
//will use the NGB redirect
LOGGER.debug(String
.format("client is blocked by geolimit, use the NGB redirect url: %s",
- deliveryService.getGeoRedirectUrl()));
+ deliveryService.getGeoRedirectUrl()));
return enforceGeoRedirect(track, deliveryService, clientIp, track.getClientGeolocation());
} else {
track.setResultDetails(ResultDetails.DS_CLIENT_GEO_UNSUPPORTED);
@@ -370,29 +398,192 @@ public class TrafficRouter {
return caches;
}
- @SuppressWarnings({"PMD.CyclomaticComplexity", "PMD.NPathComplexity"})
public DNSRouteResult route(final DNSRequest request, final Track track) throws GeolocationException {
+ final DeliveryService ds = selectDeliveryService(request);
+
track.setRouteType(RouteType.DNS, request.getHostname());
- final DeliveryService ds = selectDeliveryService(request, false);
+ // TODO: getHostname or getName -- !ds.getRoutingName().equalsIgnoreCase(request.getHostname().split("\\.")[0]))
+ if (ds != null && ds.isDns() && request.getName().toString().toLowerCase().matches(ds.getRoutingName().toLowerCase() + "\\..*")) {
+ return getEdgeCaches(request, ds, track);
+ } else {
+ return getEdgeTrafficRouters(request, ds, track);
+ }
+ }
- if (ds == null) {
- track.setResult(ResultType.STATIC_ROUTE);
- track.setResultDetails(ResultDetails.DS_NOT_FOUND);
- return null;
+ private DNSRouteResult getEdgeTrafficRouters(final DNSRequest request, final DeliveryService ds, final Track track) throws GeolocationException {
+ final DNSRouteResult result = new DNSRouteResult();
+
+ result.setDeliveryService(ds);
+ result.setAddresses(selectTrafficRouters(request, ds, track));
+
+ return result;
+ }
+
+ private List<InetRecord> selectTrafficRouters(final DNSRequest request, final DeliveryService ds) throws GeolocationException {
+ return selectTrafficRouters(request, ds, null);
+ }
+
+ private List<InetRecord> selectTrafficRouters(final DNSRequest request, final DeliveryService ds, final Track track) throws GeolocationException {
+ final List<InetRecord> result = new ArrayList<>();
+ ResultType resultType = null;
+
+ if (track != null) {
+ track.setResultDetails(ResultDetails.LOCALIZED_DNS);
+ }
+
+ Geolocation clientGeolocation = null;
+
+ final NetworkNode networkNode = getNetworkNode(request.getClientIP());
+
+ if (networkNode != null && networkNode.getGeolocation() != null) {
+ clientGeolocation = networkNode.getGeolocation();
+ resultType = ResultType.CZ;
+ } else {
+ clientGeolocation = getClientGeolocation(request.getClientIP(), track, ds);
+ resultType = ResultType.GEO;
+ }
+
+ if (clientGeolocation == null) {
+ result.addAll(selectTrafficRoutersMiss(request.getZoneName(), ds));
+ resultType = ResultType.MISS;
+ } else {
+ result.addAll(selectTrafficRoutersLocalized(clientGeolocation, request.getZoneName(), ds, track, request.getQueryType()));
+
+ if (track != null) {
+ track.setClientGeolocation(clientGeolocation);
+ }
+ }
+
+ if (track != null) {
+ track.setResult(resultType);
+ }
+
+ return result;
+ }
+
+ @SuppressWarnings({"PMD.CyclomaticComplexity", "PMD.NPathComplexity"})
+ public List<InetRecord> selectTrafficRoutersMiss(final String zoneName, final DeliveryService ds) throws GeolocationException {
+ final List<InetRecord> trafficRouterRecords = new ArrayList<>();
+
+ if (!isEdgeDNSRouting() && !isEdgeHTTPRouting()) {
+ return trafficRouterRecords;
+ }
+
+ final List<TrafficRouterLocation> trafficRouterLocations = getCacheRegister().getEdgeTrafficRouterLocations();
+ final List<Node> edgeTrafficRouters = new ArrayList<>();
+ final Map<String, List<Node>> orderedNodes = new HashMap<>();
+
+ int limit = (getEdgeDNSRoutingLimit() > getEdgeHTTPRoutingLimit(ds)) ? getEdgeDNSRoutingLimit() : getEdgeHTTPRoutingLimit(ds);
+ int index = 0;
+ boolean exhausted = false;
+
+ // if limits don't exist, or do exist and are higher than the number of edge TRs, use the number of edge TRs as the limit
+ if (limit == 0 || limit > getCacheRegister().getEdgeTrafficRouterCount()) {
+ limit = getCacheRegister().getEdgeTrafficRouterCount();
+ }
+
+ // grab one TR per location until the limit is reached
+ while (edgeTrafficRouters.size() < limit && !exhausted) {
+ final int initialCount = edgeTrafficRouters.size();
+
+ for (final TrafficRouterLocation location : trafficRouterLocations) {
+ if (edgeTrafficRouters.size() >= limit) {
+ break;
+ }
+
+ if (!orderedNodes.containsKey(location.getId())) {
+ orderedNodes.put(location.getId(), consistentHasher.selectHashables(location.getTrafficRouters(), zoneName));
+ }
+
+ final List<Node> trafficRouters = orderedNodes.get(location.getId());
+
+ if (trafficRouters == null || trafficRouters.isEmpty() || index >= trafficRouters.size()) {
+ continue;
+ }
+
+ edgeTrafficRouters.add(trafficRouters.get(index));
+ }
+
+ /*
+ * we iterated through every location and attempted to add edge TR at index, but none were added....
+ * normally, these values would never match unless we ran out of options...
+ * if so, we've exhausted our options so we need to break out of the while loop
+ */
+ if (initialCount == edgeTrafficRouters.size()) {
+ exhausted = true;
+ }
+
+ index++;
+ }
+
+ if (!edgeTrafficRouters.isEmpty()) {
+ if (isEdgeDNSRouting()) {
+ trafficRouterRecords.addAll(nsRecordsFromNodes(ds, edgeTrafficRouters));
+ }
+
+ if (ds != null && !ds.isDns() && isEdgeHTTPRouting()) { // only generate edge routing records for HTTP DSs when necessary
+ trafficRouterRecords.addAll(inetRecordsFromNodes(ds, edgeTrafficRouters));
+ }
}
- if (!ds.getRoutingName().equalsIgnoreCase(request.getHostname().split("\\.")[0])) {
- // request matched the Delivery Service but is using the wrong routing name
+ return trafficRouterRecords;
+ }
+
+ public List<InetRecord> selectTrafficRoutersLocalized(final Geolocation clientGeolocation, final String name, final DeliveryService ds) throws GeolocationException {
+ return selectTrafficRoutersLocalized(clientGeolocation, name, ds, null, 0);
+ }
+
+ @SuppressWarnings({"PMD.CyclomaticComplexity", "PMD.NPathComplexity"})
+ public List<InetRecord> selectTrafficRoutersLocalized(final Geolocation clientGeolocation, final String zoneName, final DeliveryService ds, final Track track, final int queryType) throws GeolocationException {
+ final List<InetRecord> trafficRouterRecords = new ArrayList<>();
+
+ if (!isEdgeDNSRouting() && !isEdgeHTTPRouting()) {
+ return trafficRouterRecords;
+ }
+
+ final List<TrafficRouterLocation> trafficRouterLocations = (List<TrafficRouterLocation>) orderLocations(getCacheRegister().getEdgeTrafficRouterLocations(), clientGeolocation);
+
+ for (final TrafficRouterLocation location : trafficRouterLocations) {
+ final List<Node> trafficRouters = consistentHasher.selectHashables(location.getTrafficRouters(), zoneName);
+
+ if (trafficRouters == null || trafficRouters.isEmpty()) {
+ continue;
+ }
+
+ if (isEdgeDNSRouting()) {
+ trafficRouterRecords.addAll(nsRecordsFromNodes(ds, trafficRouters));
+ }
+
+ if (ds != null && !ds.isDns() && isEdgeHTTPRouting()) { // only generate edge routing records for HTTP DSs when necessary
+ trafficRouterRecords.addAll(inetRecordsFromNodes(ds, trafficRouters));
+ }
+
+ if (track != null) {
+ track.setResultLocation(location.getGeolocation());
+ }
+
+ break;
+ }
+
+
+ return trafficRouterRecords;
+ }
+
+ @SuppressWarnings({"PMD.CyclomaticComplexity", "PMD.NPathComplexity"})
+ private DNSRouteResult getEdgeCaches(final DNSRequest request, final DeliveryService ds, final Track track) throws GeolocationException {
+ final DNSRouteResult result = new DNSRouteResult();
+ result.setDeliveryService(ds);
+
+ if (ds == null) {
track.setResult(ResultType.STATIC_ROUTE);
track.setResultDetails(ResultDetails.DS_NOT_FOUND);
return null;
}
- final DNSRouteResult result = new DNSRouteResult();
-
if (!ds.isAvailable()) {
result.setAddresses(ds.getFailureDnsResponse(request, track));
+ result.addAddresses(selectTrafficRouters(request, ds));
return result;
}
@@ -403,6 +594,7 @@ public class TrafficRouter {
track.setResult(ResultType.CZ);
track.setClientGeolocation(cacheLocation.getGeolocation());
result.setAddresses(inetRecordsFromCaches(ds, caches, request));
+ result.addAddresses(selectTrafficRouters(request, ds));
return result;
}
@@ -410,6 +602,7 @@ public class TrafficRouter {
track.setResult(ResultType.MISS);
track.setResultDetails(ResultDetails.DS_CZ_ONLY);
result.setAddresses(ds.getFailureDnsResponse(request, track));
+ result.addAddresses(selectTrafficRouters(request, ds));
return result;
}
@@ -437,11 +630,56 @@ public class TrafficRouter {
result.setAddresses(ds.getFailureDnsResponse(request, track));
}
+ result.addAddresses(selectTrafficRouters(request, ds));
+
return result;
}
+ private List<InetRecord> nsRecordsFromNodes(final DeliveryService ds, final List<Node> nodes) {
+ final List<InetRecord> nsRecords = new ArrayList<>();
+ final int limit = (getEdgeDNSRoutingLimit() > nodes.size()) ? nodes.size() : getEdgeDNSRoutingLimit();
+
+ long ttl = getEdgeNSttl();
+
+ if (ds != null && ds.getTtls().has("NS")) {
+ ttl = JsonUtils.optLong(ds.getTtls(), "NS"); // no exception
+ }
+
+ for (int i = 0; i < limit; i++) {
+ final Node node = nodes.get(i);
+ nsRecords.add(new InetRecord(node.getFqdn(), ttl, Type.NS));
+ }
+
+ return nsRecords;
+ }
+
+ public List<InetRecord> inetRecordsFromNodes(final DeliveryService ds, final List<Node> nodes) {
+ final List<InetRecord> addresses = new ArrayList<>();
+ final int limit = (getEdgeHTTPRoutingLimit(ds) > nodes.size()) ? nodes.size() : getEdgeHTTPRoutingLimit(ds);
+
+ if (ds == null) {
+ return addresses;
+ }
+
+ final JsonNode ttls = ds.getTtls();
+
+ for (int i = 0; i < limit; i++) {
+ final Node node = nodes.get(i);
+
+ if (node.getIp4() != null) {
+ addresses.add(new InetRecord(node.getIp4(), JsonUtils.optLong(ttls, "A")));
+ }
+
+ if (node.getIp6() != null && ds.isIp6RoutingEnabled()) {
+ addresses.add(new InetRecord(node.getIp6(), JsonUtils.optLong(ttls,"AAAA")));
+ }
+ }
+
+ return addresses;
+ }
+
public List<InetRecord> inetRecordsFromCaches(final DeliveryService ds, final List<Cache> caches, final Request request) {
- final List<InetRecord> addresses = new ArrayList<InetRecord>();
+ final List<InetRecord> addresses = new ArrayList<>();
final int maxDnsIps = ds.getMaxDnsIps();
List<Cache> selectedCaches;
@@ -478,18 +716,27 @@ public class TrafficRouter {
}
public Geolocation getClientGeolocation(final String clientIp, final Track track, final DeliveryService deliveryService) throws GeolocationException {
- if (track.isClientGeolocationQueried()) {
+ if (track != null && track.isClientGeolocationQueried()) {
return track.getClientGeolocation();
}
- final Geolocation clientGeolocation = getLocation(clientIp, deliveryService);
- track.setClientGeolocation(clientGeolocation);
- track.setClientGeolocationQueried(true);
+ final Geolocation clientGeolocation;
+
+ if (deliveryService != null) {
+ clientGeolocation = getLocation(clientIp, deliveryService);
+ } else {
+ clientGeolocation = getLocation(clientIp);
+ }
+
+ if (track != null) {
+ track.setClientGeolocation(clientGeolocation);
+ track.setClientGeolocationQueried(true);
+ }
return clientGeolocation;
}
- public Geolocation getClientLocation(final String clientIp, final DeliveryService ds, final CacheLocation cacheLocation, final Track track) throws GeolocationException {
+ public Geolocation getClientLocation(final String clientIp, final DeliveryService ds, final Location cacheLocation, final Track track) throws GeolocationException {
if (cacheLocation != null) {
return cacheLocation.getGeolocation();
}
@@ -537,7 +784,7 @@ public class TrafficRouter {
*/
@SuppressWarnings({"PMD.CyclomaticComplexity", "PMD.NPathComplexity"})
public HTTPRouteResult multiRoute(final HTTPRequest request, final Track track) throws MalformedURLException, GeolocationException {
- final DeliveryService entryDeliveryService = cacheRegister.getDeliveryService(request, true);
+ final DeliveryService entryDeliveryService = cacheRegister.getDeliveryService(request);
final List<SteeringResult> steeringResults = getSteeringResults(request, track, entryDeliveryService);
@@ -794,7 +1041,7 @@ public class TrafficRouter {
private DeliveryService getDeliveryService(final HTTPRequest request, final Track track) {
final String xtcSteeringOption = request.getHeaderValue(XTC_STEERING_OPTION);
- final DeliveryService deliveryService = consistentHashDeliveryService(cacheRegister.getDeliveryService(request, true), request, xtcSteeringOption);
+ final DeliveryService deliveryService = consistentHashDeliveryService(cacheRegister.getDeliveryService(request), request, xtcSteeringOption);
if (deliveryService == null) {
track.setResult(ResultType.DS_MISS);
@@ -855,7 +1102,7 @@ public class TrafficRouter {
}
final DeliveryService deliveryService = cacheRegister.getDeliveryService(deliveryServiceId);
- CacheLocation cacheLocation = networkNode.getCacheLocation();
+ CacheLocation cacheLocation = (CacheLocation) networkNode.getLocation();
if (useDeep && cacheLocation != null) {
// lazily load deep Caches into the deep CacheLocation
@@ -888,7 +1135,7 @@ public class TrafficRouter {
if (cacheLocation != null && !getSupportingCaches(cacheLocation.getCaches(), deliveryService).isEmpty()) {
// lazy loading in case a CacheLocation has not yet been associated with this NetworkNode
- networkNode.setCacheLocation(cacheLocation);
+ networkNode.setLocation(cacheLocation);
return cacheLocation;
}
@@ -917,9 +1164,10 @@ public class TrafficRouter {
// We had a hit in the CZF but the name does not match a known cache location.
// Check whether the CZF entry has a geolocation and use it if so.
- List<CacheLocation> availableLocations = cacheRegister.filterAvailableLocations(deliveryServiceId);
+ List<CacheLocation> availableLocations = cacheRegister.filterAvailableCacheLocations(deliveryServiceId);
availableLocations = filterEnabledLocations(availableLocations, localizationMethod);
final CacheLocation closestCacheLocation = getClosestCacheLocation(availableLocations, networkNode.getGeolocation(), cacheRegister.getDeliveryService(deliveryServiceId));
+
if (closestCacheLocation != null) {
LOGGER.debug("Got closest CZ cache group " + closestCacheLocation.getId() + " for " + ip + ", ds " + deliveryServiceId);
if (track != null) {
@@ -939,11 +1187,11 @@ public class TrafficRouter {
return getCoverageZoneCacheLocation(ip, deliveryService, true, null);
}
- protected CacheLocation getCoverageZoneCacheLocation(final String ip, final DeliveryService deliveryService, final boolean useDeep, final Track track) {
+ public CacheLocation getCoverageZoneCacheLocation(final String ip, final DeliveryService deliveryService, final boolean useDeep, final Track track) {
return getCoverageZoneCacheLocation(ip, deliveryService.getId(), useDeep, track);
}
- protected CacheLocation getCoverageZoneCacheLocation(final String ip, final DeliveryService deliveryService) {
+ public CacheLocation getCoverageZoneCacheLocation(final String ip, final DeliveryService deliveryService) {
return getCoverageZoneCacheLocation(ip, deliveryService.getId());
}
@@ -1017,8 +1265,8 @@ public class TrafficRouter {
List<Cache> caches = null;
if (deliveryService.isCoverageZoneOnly() && deliveryService.getGeoRedirectUrl() != null) {
- //use the NGB redirect
- caches = enforceGeoRedirect(StatTracker.getTrack(), deliveryService, ip, null);
+ //use the NGB redirect
+ caches = enforceGeoRedirect(StatTracker.getTrack(), deliveryService, ip, null);
} else {
final CacheLocation cacheLocation = getCoverageZoneCacheLocation(ip, deliveryServiceId);
@@ -1049,7 +1297,7 @@ public class TrafficRouter {
}
private boolean isMultiRouteRequest(final HTTPRequest request) {
- final DeliveryService deliveryService = cacheRegister.getDeliveryService(request, true);
+ final DeliveryService deliveryService = cacheRegister.getDeliveryService(request);
if (deliveryService == null || !isSteeringDeliveryService(deliveryService)) {
return false;
@@ -1208,17 +1456,16 @@ public class TrafficRouter {
}
/**
- * Returns a list {@link CacheLocation}s sorted by distance from the client.
+ * Returns a list {@link Location}s sorted by distance from the client.
* If the client's location could not be determined, then the list is
* unsorted.
*
- * @param cacheLocations
- * the collection of CacheLocations to order
+ * @param locations the collection of Locations to order
* @return the ordered list of locations
*/
- public List<CacheLocation> orderCacheLocations(final List<CacheLocation> cacheLocations, final Geolocation clientLocation) {
- Collections.sort(cacheLocations, new CacheLocationComparator(clientLocation));
- return cacheLocations;
+ public List<? extends Location> orderLocations(final List<? extends Location> locations, final Geolocation clientLocation) {
+ Collections.sort(locations, new LocationComparator(clientLocation));
+ return locations;
}
private CacheLocation getClosestCacheLocation(final List<CacheLocation> cacheLocations, final Geolocation clientLocation, final DeliveryService deliveryService) {
@@ -1226,7 +1473,7 @@ public class TrafficRouter {
return null;
}
- final List<CacheLocation> orderedLocations = orderCacheLocations(cacheLocations, clientLocation);
+ final List<CacheLocation> orderedLocations = (List<CacheLocation>) orderLocations(cacheLocations, clientLocation);
for (final CacheLocation cacheLocation : orderedLocations) {
if (!getSupportingCaches(cacheLocation.getCaches(), deliveryService).isEmpty()) {
@@ -1364,4 +1611,28 @@ public class TrafficRouter {
public void setSteeringRegistry(final SteeringRegistry steeringRegistry) {
this.steeringRegistry = steeringRegistry;
}
+
+ public boolean isEdgeDNSRouting() {
+ return edgeDNSRouting;
+ }
+
+ public boolean isEdgeHTTPRouting() {
+ return edgeHTTPRouting;
+ }
+
+ private long getEdgeNSttl() {
+ return edgeNSttl;
+ }
+
+ private int getEdgeDNSRoutingLimit() {
+ return edgeDNSRoutingLimit;
+ }
+
+ private int getEdgeHTTPRoutingLimit(final DeliveryService ds) {
+ if (ds != null && ds.getMaxDnsIps() != 0 && ds.getMaxDnsIps() != edgeHTTPRoutingLimit) {
+ return ds.getMaxDnsIps();
+ }
+
+ return edgeHTTPRoutingLimit;
+ }
}
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterManager.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterManager.java
index 163f996..71e7c30 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterManager.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterManager.java
@@ -25,7 +25,7 @@ import com.comcast.cdn.traffic_control.traffic_router.core.loc.FederationRegistr
import com.fasterxml.jackson.databind.JsonNode;
import org.apache.log4j.Logger;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
import com.comcast.cdn.traffic_control.traffic_router.core.dns.NameServer;
import com.comcast.cdn.traffic_control.traffic_router.geolocation.GeolocationService;
import com.comcast.cdn.traffic_control.traffic_router.core.util.TrafficOpsUtils;
diff --git a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/util/DataExporter.java b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/util/DataExporter.java
index fffc8fb..ee19629 100644
--- a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/util/DataExporter.java
+++ b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/util/DataExporter.java
@@ -27,13 +27,15 @@ import java.util.Properties;
import com.fasterxml.jackson.databind.JsonNode;
import com.google.common.cache.CacheStats;
+
import org.apache.log4j.Logger;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.InetRecord;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Resolver;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.InetRecord;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Location;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Resolver;
import com.comcast.cdn.traffic_control.traffic_router.geolocation.Geolocation;
import com.comcast.cdn.traffic_control.traffic_router.geolocation.GeolocationException;
import com.comcast.cdn.traffic_control.traffic_router.core.loc.NetworkNode;
@@ -90,7 +92,7 @@ public class DataExporter {
final Map<String, Object> map = new HashMap<String, Object>();
map.put("requestIp", ip);
- final CacheLocation cl = getLocationFromCzm(ip);
+ final Location cl = getLocationFromCzm(ip);
if (cl != null) {
map.put("locationByCoverageZone", cl.getProperties());
@@ -127,7 +129,7 @@ public class DataExporter {
return map;
}
- private CacheLocation getLocationFromCzm(final String ip) {
+ private Location getLocationFromCzm(final String ip) {
NetworkNode nn = null;
try {
@@ -139,7 +141,7 @@ public class DataExporter {
if (nn == null) { return null; }
final String locId = nn.getLoc();
- final CacheLocation cl = nn.getCacheLocation();
+ final Location cl = nn.getLocation();
if (cl != null) {
return cl;
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/CatalinaTrafficRouter.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/CatalinaTrafficRouter.java
index 8e9eac5..9dcca02 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/CatalinaTrafficRouter.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/CatalinaTrafficRouter.java
@@ -74,6 +74,9 @@ public class CatalinaTrafficRouter {
}
+ System.out.println("[" + System.currentTimeMillis() + "] >>>>>>>>>>>>>>>> Traffic Router listening on DNS port " + System.getProperty("dns.udp.port") + " udp");
+ System.out.println("[" + System.currentTimeMillis() + "] >>>>>>>>>>>>>>>> Traffic Router listening on DNS port " + System.getProperty("dns.tcp.port") + " tcp");
+
StandardHost standardHost = (StandardHost) trafficRouterService.getContainer().findChild("localhost");
standardHost.setAppBase(appBase);
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandlerTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandlerTest.java
index dcb47ad..9e9e000 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandlerTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/config/ConfigHandlerTest.java
@@ -23,7 +23,7 @@ import java.util.Set;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation.LocalizationMethod;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation.LocalizationMethod;
import com.comcast.cdn.traffic_control.traffic_router.core.request.HTTPRequest;
import org.junit.Before;
import org.junit.Test;
@@ -42,7 +42,7 @@ import org.mockito.stubbing.Answer;
import org.powermock.api.mockito.PowerMockito;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
import org.powermock.reflect.Whitebox;
@@ -160,7 +160,7 @@ public class ConfigHandlerTest {
CacheRegister register = PowerMockito.mock(CacheRegister.class);
- when(register.getDeliveryService(any(HTTPRequest.class), anyBoolean())).thenReturn(null);
+ when(register.getDeliveryService(any(HTTPRequest.class))).thenReturn(null);
Whitebox.invokeMethod(handler, "initGeoFailedRedirect", dsMap, register);
assertThat(urlType[0], equalTo("NOT_DS_URL"));
@@ -202,7 +202,7 @@ public class ConfigHandlerTest {
when(ds.getId()).thenReturn(anotherId);
CacheRegister register = PowerMockito.mock(CacheRegister.class);
- when(register.getDeliveryService(any(HTTPRequest.class), anyBoolean())).thenReturn(anotherDs);
+ when(register.getDeliveryService(any(HTTPRequest.class))).thenReturn(anotherDs);
Whitebox.invokeMethod(handler, "initGeoFailedRedirect", dsMap, register);
assertThat(urlType[0], equalTo("NOT_DS_URL"));
@@ -241,7 +241,7 @@ public class ConfigHandlerTest {
CacheRegister register = PowerMockito.mock(CacheRegister.class);
- when(register.getDeliveryService(any(HTTPRequest.class), anyBoolean())).thenReturn(ds);
+ when(register.getDeliveryService(any(HTTPRequest.class))).thenReturn(ds);
Whitebox.invokeMethod(handler, "initGeoFailedRedirect", dsMap, register);
assertThat(urlType[0], equalTo("DS_URL"));
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/NameServerTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/NameServerTest.java
index 02a3bb9..5b48aab 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/NameServerTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/NameServerTest.java
@@ -18,7 +18,9 @@ package com.comcast.cdn.traffic_control.traffic_router.core.dns;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
import com.comcast.cdn.traffic_control.traffic_router.core.router.TrafficRouter;
import com.comcast.cdn.traffic_control.traffic_router.core.router.TrafficRouterManager;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.dns.DNSAccessRecord;
+import com.comcast.cdn.traffic_control.traffic_router.core.dns.NameServer;
import com.comcast.cdn.traffic_control.traffic_router.core.util.JsonUtils;
import com.fasterxml.jackson.databind.node.ArrayNode;
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManagerTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManagerTest.java
index 4438b1c..fbaa618 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManagerTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManagerTest.java
@@ -48,9 +48,9 @@ import org.xbill.DNS.Type;
import org.xbill.DNS.Zone;
import com.comcast.cdn.traffic_control.traffic_router.core.TestBase;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
import com.comcast.cdn.traffic_control.traffic_router.core.router.TrafficRouter;
import com.comcast.cdn.traffic_control.traffic_router.core.router.TrafficRouterManager;
@@ -73,7 +73,7 @@ public class ZoneManagerTest {
public void setUp() throws Exception {
trafficRouterManager = (TrafficRouterManager) context.getBean("trafficRouterManager");
trafficRouterManager.getTrafficRouter().setApplicationContext(context);
- final File file = new File("src/test/resources/czmap.json");
+ final File file = new File("src/test/db/czmap.json");
final ObjectMapper mapper = new ObjectMapper();
final JsonNode jsonNode = mapper.readTree(file);
final JsonNode coverageZones = jsonNode.get("coverageZones");
@@ -107,14 +107,14 @@ public class ZoneManagerTest {
final Name edgeName = new Name(ds.getRoutingName() + "." + domain + ".");
- for (CacheLocation location : cacheRegister.getCacheLocations()) {
+ for (InetAddress source : netMap.values()) {
+ final CacheLocation location = trafficRouter.getCoverageZoneCacheLocation(source.getHostAddress(), ds);
final List<Cache> caches = trafficRouter.selectCachesByCZ(ds, location);
if (caches == null) {
continue;
}
- final InetAddress source = netMap.get(location.getId());
final DNSAccessRecord.Builder builder = new DNSAccessRecord.Builder(1, source);
final Set<Zone> zones = new HashSet<Zone>();
final int maxDnsIps = ds.getMaxDnsIps();
@@ -162,4 +162,4 @@ public class ZoneManagerTest {
return p;
}
-}
\ No newline at end of file
+}
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManagerUnitTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManagerUnitTest.java
index 4ae1ae1..c5c2135 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManagerUnitTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManagerUnitTest.java
@@ -15,7 +15,7 @@
package com.comcast.cdn.traffic_control.traffic_router.core.dns;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
import com.comcast.cdn.traffic_control.traffic_router.core.router.StatTracker;
import com.comcast.cdn.traffic_control.traffic_router.core.router.StatTracker.Track.ResultType;
import com.comcast.cdn.traffic_control.traffic_router.core.router.TrafficRouterManager;
@@ -63,8 +63,6 @@ public class ZoneManagerUnitTest {
when(trafficRouter.getCacheRegister()).thenReturn(cacheRegister);
PowerMockito.spy(ZoneManager.class);
- PowerMockito.doNothing().when(ZoneManager.class, "initDnsRoutingNames", cacheRegister);
- PowerMockito.doReturn(true).when(ZoneManager.class, "isDnsRoutingName", anyString());
PowerMockito.doNothing().when(ZoneManager.class, "initTopLevelDomain", cacheRegister);
PowerMockito.doNothing().when(ZoneManager.class, "initZoneCache", trafficRouter);
@@ -84,6 +82,7 @@ public class ZoneManagerUnitTest {
Zone zone = mock(Zone.class);
when(zone.findRecords(any(Name.class), anyInt())).thenReturn(setResponse);
+ when(zone.getOrigin()).thenReturn(new Name(qname, 1));
DNSAccessRecord.Builder builder = new DNSAccessRecord.Builder(1L, client);
builder = spy(builder);
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/SteeringGeolocationComparatorTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/SteeringGeolocationComparatorTest.java
index ed72081..f769f80 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/SteeringGeolocationComparatorTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/ds/SteeringGeolocationComparatorTest.java
@@ -15,7 +15,7 @@
package com.comcast.cdn.traffic_control.traffic_router.core.ds;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
import com.comcast.cdn.traffic_control.traffic_router.geolocation.Geolocation;
import org.junit.Before;
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheRegisterTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/CacheRegisterTest.java
similarity index 81%
rename from traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheRegisterTest.java
rename to traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/CacheRegisterTest.java
index 72100f8..303ef01 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/cache/CacheRegisterTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/edge/CacheRegisterTest.java
@@ -13,7 +13,7 @@
* limitations under the License.
*/
-package com.comcast.cdn.traffic_control.traffic_router.core.cache;
+package com.comcast.cdn.traffic_control.traffic_router.core.edge;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryServiceMatcher;
@@ -59,15 +59,13 @@ public class CacheRegisterTest {
deliveryServiceMatchers.add(deliveryServiceMatcher2);
deliveryServiceMatchers.add(deliveryServiceMatcher3);
- cacheRegister.setHttpDeliveryServiceMatchers(deliveryServiceMatchers);
-
DeliveryServiceMatcher dnsMatcher1 = new DeliveryServiceMatcher(deliveryService1);
dnsMatcher1.addMatch(HOST, ".*\\.service01-kabletown\\..*", "");
TreeSet<DeliveryServiceMatcher> dnsMatchers = new TreeSet<DeliveryServiceMatcher>();
- dnsMatchers.add(dnsMatcher1);
+ deliveryServiceMatchers.add(dnsMatcher1);
- cacheRegister.setDnsDeliveryServiceMatchers(dnsMatchers);
+ cacheRegister.setDeliveryServiceMatchers(deliveryServiceMatchers);
}
@Test
@@ -76,25 +74,20 @@ public class CacheRegisterTest {
httpRequest.setHostname("foo.service01-kabletown.com");
httpRequest.setPath("foo/abcde/bar");
- assertThat(cacheRegister.getDeliveryService(httpRequest, true).getId(), equalTo("delivery service 2"));
+ assertThat(cacheRegister.getDeliveryService(httpRequest).getId(), equalTo("delivery service 2"));
Request request = new Request();
request.setHostname("foo.service01-kabletown.com");
- assertThat(cacheRegister.getDeliveryService(request, false).getId(), equalTo("delivery service 1"));
+ assertThat(cacheRegister.getDeliveryService(request).getId(), equalTo("delivery service 1"));
}
@Test
public void itReturnsNullForDeliveryServiceWhenItHasNoMatchers() {
- cacheRegister.setHttpDeliveryServiceMatchers(null);
+ cacheRegister.setDeliveryServiceMatchers(null);
HTTPRequest httpRequest = new HTTPRequest();
httpRequest.setHostname("foo.service01-kabletown.com");
httpRequest.setPath("foo/abcde/bar");
- assertThat(cacheRegister.getDeliveryService(httpRequest, true), nullValue());
-
- cacheRegister.setDnsDeliveryServiceMatchers(null);
- Request request = new Request();
- request.setHostname("foo.service01-kabletown.com");
- assertThat(cacheRegister.getDeliveryService(request, false), nullValue());
+ assertThat(cacheRegister.getDeliveryService(httpRequest), nullValue());
}
}
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/external/RouterTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/external/RouterTest.java
index 7a2ed1c..9b35db1 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/external/RouterTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/external/RouterTest.java
@@ -38,6 +38,7 @@ import org.junit.FixMethodOrder;
import org.junit.Test;
import org.junit.experimental.categories.Category;
import org.junit.runners.MethodSorters;
+import org.xbill.DNS.*;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.SNIHostName;
@@ -49,8 +50,10 @@ import javax.net.ssl.SSLSocket;
import javax.net.ssl.TrustManagerFactory;
import java.io.IOException;
import java.io.InputStream;
+import java.net.UnknownHostException;
import java.security.KeyStore;
import java.util.ArrayList;
+import java.util.Arrays;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
@@ -97,6 +100,8 @@ public class RouterTest {
private String testHttpPort = System.getProperty("testHttpServerPort", "8889");
private KeyStore trustStore;
+ private int routerDnsPort = Integer.valueOf(System.getProperty("dns.udp.port", "1053"));
+
@Before
public void before() throws Exception {
ObjectMapper objectMapper = new ObjectMapper(new JsonFactory());
@@ -234,6 +239,37 @@ public class RouterTest {
}
@Test
+ public void itAUsesEdgeTrafficRoutersForHttpRouting() throws TextParseException, UnknownHostException {
+ Set<String> edgeIpAddresses = new HashSet<>();
+ // this will actually be the "miss" scenario since the resolver is localhost, which will be a CZF miss
+ // in the miss case, we serve one TR from each location, and these are what we'd serve with our test CrConfig.json
+ edgeIpAddresses.add("12.34.21.2");
+ edgeIpAddresses.add("12.34.21.3");
+ edgeIpAddresses.add("12.34.21.7");
+ edgeIpAddresses.add("12.34.21.8");
+ edgeIpAddresses.add("2001:dead:beef:124:1:0:0:2");
+ edgeIpAddresses.add("2001:dead:beef:124:1:0:0:3");
+ edgeIpAddresses.add("2001:dead:beef:124:1:0:0:7");
+ edgeIpAddresses.add("2001:dead:beef:124:1:0:0:8");
+
+ SimpleResolver resolver = new SimpleResolver("localhost");
+ resolver.setPort(routerDnsPort);
+
+ for (int type : Arrays.asList(Type.A, Type.AAAA)) {
+ Lookup lookup = new Lookup(new Name("tr.http-only-test.thecdn.example.com."), type);
+ lookup.setResolver(resolver);
+ lookup.run();
+
+ assertThat(lookup.getResult(), equalTo(Lookup.SUCCESSFUL));
+ assertThat(lookup.getAnswers().length, equalTo(4));
+
+ for (Record record : lookup.getAnswers()) {
+ assertThat(record.rdataToString(), isIn(edgeIpAddresses));
+ }
+ }
+ }
+
+ @Test
public void itRedirectsValidHttpRequests() throws IOException, InterruptedException {
HttpGet httpGet = new HttpGet("http://localhost:" + routerHttpPort + "/stuff?fakeClientIpAddress=12.34.56.78");
httpGet.addHeader("Host", "tr." + deliveryServiceId + ".bar");
@@ -468,7 +504,6 @@ public class RouterTest {
// verify that if we get a new cr-config that turns off https for the problematic delivery service
// that it's able to get through while TR is still concurrently trying to get certs
-
String testHttpPort = System.getProperty("testHttpServerPort", "8889");
httpPost = new HttpPost("http://localhost:"+ testHttpPort + "/crconfig-3");
httpClient.execute(httpPost).close();
@@ -551,6 +586,8 @@ public class RouterTest {
httpGet = new HttpGet("http://localhost:" + routerHttpPort + "/stuff?fakeClientIpAddress=12.34.56.78");
httpGet.addHeader("Host", "tr." + httpOnlyId + ".bar");
+ System.out.println(httpGet.toString());
+ System.out.println(Arrays.toString(httpGet.getAllHeaders()));
try (CloseableHttpResponse response = httpClient.execute(httpGet)) {
assertThat(response.getStatusLine().getStatusCode(), equalTo(302));
@@ -577,10 +614,8 @@ public class RouterTest {
ObjectMapper objectMapper = new ObjectMapper(new JsonFactory());
assertThat(entity.getContent(), not(nullValue()));
- System.out.println(entity.getContent());
JsonNode json = objectMapper.readTree(entity.getContent());
- System.out.println(json);
assertThat(json.has("location"), equalTo(true));
assertThat(json.get("location").asText(), isIn(validLocations));
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/CoverageZoneTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/CoverageZoneTest.java
index 90b5979..9be34b9 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/CoverageZoneTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/CoverageZoneTest.java
@@ -15,10 +15,10 @@
package com.comcast.cdn.traffic_control.traffic_router.core.loc;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation.LocalizationMethod;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation.LocalizationMethod;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
import com.comcast.cdn.traffic_control.traffic_router.core.router.TrafficRouter;
import com.comcast.cdn.traffic_control.traffic_router.geolocation.Geolocation;
@@ -95,7 +95,7 @@ public class CoverageZoneTest {
when(cacheRegister.getCacheLocationById("east-cache-group")).thenReturn(eastCacheGroup);
- when(cacheRegister.filterAvailableLocations("delivery-service-1")).thenReturn(cacheGroups);
+ when(cacheRegister.filterAvailableCacheLocations("delivery-service-1")).thenReturn(cacheGroups);
when(cacheRegister.getDeliveryService("delivery-service-1")).thenReturn(deliveryService);
trafficRouter = PowerMockito.mock(TrafficRouter.class);
@@ -103,7 +103,7 @@ public class CoverageZoneTest {
when(trafficRouter.getCoverageZoneCacheLocation("12.23.34.45", "delivery-service-1")).thenCallRealMethod();
when(trafficRouter.getCoverageZoneCacheLocation("12.23.34.45", "delivery-service-1", false, null)).thenCallRealMethod();
when(trafficRouter.getCacheRegister()).thenReturn(cacheRegister);
- when(trafficRouter.orderCacheLocations(anyListOf(CacheLocation.class),any(Geolocation.class))).thenCallRealMethod();
+ when(trafficRouter.orderLocations(anyListOf(CacheLocation.class),any(Geolocation.class))).thenCallRealMethod();
when(trafficRouter.getSupportingCaches(anyListOf(Cache.class), eq(deliveryService))).thenCallRealMethod();
when(trafficRouter.filterEnabledLocations(anyListOf(CacheLocation.class), any(CacheLocation.LocalizationMethod.class))).thenCallRealMethod();
PowerMockito.when(trafficRouter, "getNetworkNode", "12.23.34.45").thenReturn(eastNetworkNode);
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/FederationRegistryTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/FederationRegistryTest.java
index 63e67b7..11f80eb 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/FederationRegistryTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/FederationRegistryTest.java
@@ -15,7 +15,7 @@
package com.comcast.cdn.traffic_control.traffic_router.core.loc;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.InetRecord;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.InetRecord;
import com.comcast.cdn.traffic_control.traffic_router.core.util.CidrAddress;
import com.comcast.cdn.traffic_control.traffic_router.core.util.ComparableTreeSet;
import org.junit.Before;
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/RegionalGeoTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/RegionalGeoTest.java
index f959257..40a8de5 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/RegionalGeoTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/loc/RegionalGeoTest.java
@@ -27,7 +27,7 @@ import java.util.regex.Pattern;
import java.util.Set;
import java.util.HashSet;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
import com.comcast.cdn.traffic_control.traffic_router.core.request.HTTPRequest;
import com.comcast.cdn.traffic_control.traffic_router.core.request.Request;
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/CacheLocationComparatorTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/CacheLocationComparatorTest.java
index 9dd3389..f3f0c43 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/CacheLocationComparatorTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/CacheLocationComparatorTest.java
@@ -19,14 +19,14 @@ import static org.junit.Assert.assertEquals;
import org.junit.Test;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation;
import com.comcast.cdn.traffic_control.traffic_router.geolocation.Geolocation;
public class CacheLocationComparatorTest {
@Test
public void testCompareBothLocEqual() {
- final CacheLocationComparator comparator = new CacheLocationComparator(new Geolocation(1f, 1f));
+ final LocationComparator comparator = new LocationComparator(new Geolocation(1f, 1f));
final CacheLocation loc1 = new CacheLocation("loc1", new Geolocation(0f, 0f));
final CacheLocation loc2 = new CacheLocation("loc2", new Geolocation(0f, 0f));
@@ -36,7 +36,7 @@ public class CacheLocationComparatorTest {
@Test
public void testCompareBothLocNull() {
- final CacheLocationComparator comparator = new CacheLocationComparator(new Geolocation(1f, 1f));
+ final LocationComparator comparator = new LocationComparator(new Geolocation(1f, 1f));
final CacheLocation loc1 = new CacheLocation("loc1", null);
final CacheLocation loc2 = new CacheLocation("loc2", null);
@@ -46,7 +46,7 @@ public class CacheLocationComparatorTest {
@Test
public void testCompareLocsDifferent() {
- final CacheLocationComparator comparator = new CacheLocationComparator(new Geolocation(1f, 1f));
+ final LocationComparator comparator = new LocationComparator(new Geolocation(1f, 1f));
final CacheLocation loc1 = new CacheLocation("loc1", new Geolocation(1f, 1f));
final CacheLocation loc2 = new CacheLocation("loc2", new Geolocation(0f, 0f));
@@ -56,7 +56,7 @@ public class CacheLocationComparatorTest {
@Test
public void testCompareOneLocNull() {
- final CacheLocationComparator comparator = new CacheLocationComparator(new Geolocation(1f, 1f));
+ final LocationComparator comparator = new LocationComparator(new Geolocation(1f, 1f));
final CacheLocation loc1 = new CacheLocation("loc1", new Geolocation(0f, 0f));
final CacheLocation loc2 = new CacheLocation("loc2", null);
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/CacheRegisterBuilder.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/CacheRegisterBuilder.java
index de6ab36..962bbfe 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/CacheRegisterBuilder.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/CacheRegisterBuilder.java
@@ -16,9 +16,9 @@
package com.comcast.cdn.traffic_control.traffic_router.core.router;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
import com.comcast.cdn.traffic_control.traffic_router.core.config.ParseException;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryServiceMatcher;
@@ -119,8 +119,7 @@ public class CacheRegisterBuilder {
}
public static void parseDeliveryServiceConfig(final JsonNode deliveryServices, final CacheRegister cacheRegister) throws JsonUtilsException {
- final TreeSet<DeliveryServiceMatcher> dnsServiceMatchers = new TreeSet<DeliveryServiceMatcher>();
- final TreeSet<DeliveryServiceMatcher> httpServiceMatchers = new TreeSet<DeliveryServiceMatcher>();
+ final TreeSet<DeliveryServiceMatcher> deliveryServiceMatchers = new TreeSet<DeliveryServiceMatcher>();
final Map<String,DeliveryService> dsMap = new HashMap<String,DeliveryService>();
final Iterator<String> keyIter = deliveryServices.fieldNames();
@@ -136,17 +135,8 @@ public class CacheRegisterBuilder {
for (final JsonNode matchset : matchsets) {
final String protocol = JsonUtils.getString(matchset, "protocol");
- if ("DNS".equals(protocol)) {
- isDns = true;
- }
-
final DeliveryServiceMatcher m = new DeliveryServiceMatcher(ds);
-
- if ("HTTP".equals(protocol)) {
- httpServiceMatchers.add(m);
- } else if("DNS".equals(protocol)) {
- dnsServiceMatchers.add(m);
- }
+ deliveryServiceMatchers.add(m);
for (JsonNode matchlist : matchset.get("matchlist")) {
final DeliveryServiceMatcher.Type type = DeliveryServiceMatcher.Type.valueOf(JsonUtils.getString(matchlist, "match-type"));
@@ -158,8 +148,7 @@ public class CacheRegisterBuilder {
}
cacheRegister.setDeliveryServiceMap(dsMap);
- cacheRegister.setDnsDeliveryServiceMatchers(dnsServiceMatchers);
- cacheRegister.setHttpDeliveryServiceMatchers(httpServiceMatchers);
+ cacheRegister.setDeliveryServiceMatchers(deliveryServiceMatchers);
}
}
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/DNSRoutingMissesTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/DNSRoutingMissesTest.java
index bb80022..34ba482 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/DNSRoutingMissesTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/DNSRoutingMissesTest.java
@@ -15,8 +15,8 @@
package com.comcast.cdn.traffic_control.traffic_router.core.router;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
import com.comcast.cdn.traffic_control.traffic_router.core.loc.FederationRegistry;
import com.comcast.cdn.traffic_control.traffic_router.core.request.DNSRequest;
@@ -51,11 +51,11 @@ public class DNSRoutingMissesTest {
@Before
public void before() throws Exception {
- request = new DNSRequest();
+ Name name = Name.fromString("edge.foo-img.kabletown.com");
+ request = new DNSRequest("foo-img.kabletown.com", name, Type.A);
request.setClientIP("192.168.34.56");
- request.setHostname(Name.fromString("edge.foo-img.kabletown.com").relativize(Name.root).toString());
- request.setQtype(Type.A);
+ request.setHostname(name.relativize(Name.root).toString());
FederationRegistry federationRegistry = mock(FederationRegistry.class);
when(federationRegistry.findInetRecords(anyString(), any(CidrAddress.class))).thenReturn(null);
@@ -73,8 +73,8 @@ public class DNSRoutingMissesTest {
public void itSetsDetailsWhenNoDeliveryService() throws Exception {
trafficRouter.route(request, track);
- verify(track).setResult(ResultType.STATIC_ROUTE);
- verify(track).setResultDetails(ResultDetails.DS_NOT_FOUND);
+ verify(track).setResult(ResultType.MISS);
+ verify(track).setResultDetails(ResultDetails.LOCALIZED_DNS);
}
// When the delivery service is unavailable ...
@@ -84,8 +84,9 @@ public class DNSRoutingMissesTest {
when(deliveryService.isAvailable()).thenReturn(false);
when(deliveryService.getFailureDnsResponse(request, track)).thenCallRealMethod();
when(deliveryService.getRoutingName()).thenReturn("edge");
+ when(deliveryService.isDns()).thenReturn(true);
- doReturn(deliveryService).when(trafficRouter).selectDeliveryService(request, false);
+ doReturn(deliveryService).when(trafficRouter).selectDeliveryService(request);
trafficRouter.route(request, track);
@@ -99,8 +100,9 @@ public class DNSRoutingMissesTest {
when(deliveryService.isAvailable()).thenReturn(false);
when(deliveryService.getFailureDnsResponse(request, track)).thenCallRealMethod();
when(deliveryService.getRoutingName()).thenReturn("edge");
+ when(deliveryService.isDns()).thenReturn(true);
- doReturn(deliveryService).when(trafficRouter).selectDeliveryService(request, false);
+ doReturn(deliveryService).when(trafficRouter).selectDeliveryService(request);
JsonNode bypassDestination = mock(JsonNode.class);
when(bypassDestination.get("DNS")).thenReturn(null);
@@ -121,10 +123,10 @@ public class DNSRoutingMissesTest {
DeliveryService deliveryService = mock(DeliveryService.class);
doReturn(true).when(deliveryService).isAvailable();
when(deliveryService.getRoutingName()).thenReturn("edge");
-
+ when(deliveryService.isDns()).thenReturn(true);
when(deliveryService.isCoverageZoneOnly()).thenReturn(true);
- doReturn(deliveryService).when(trafficRouter).selectDeliveryService(any(Request.class), anyBoolean());
+ doReturn(deliveryService).when(trafficRouter).selectDeliveryService(any(Request.class));
trafficRouter.route(request, track);
verify(track).setResult(ResultType.MISS);
@@ -139,10 +141,11 @@ public class DNSRoutingMissesTest {
DeliveryService deliveryService = mock(DeliveryService.class);
doReturn(true).when(deliveryService).isAvailable();
when(deliveryService.getRoutingName()).thenReturn("edge");
+ when(deliveryService.isDns()).thenReturn(true);
when(deliveryService.isCoverageZoneOnly()).thenReturn(false);
- doReturn(deliveryService).when(trafficRouter).selectDeliveryService(request, false);
+ doReturn(deliveryService).when(trafficRouter).selectDeliveryService(request);
trafficRouter.route(request, track);
@@ -162,8 +165,9 @@ public class DNSRoutingMissesTest {
when(deliveryService.isLocationAvailable(cacheLocation)).thenReturn(false);
when(deliveryService.isCoverageZoneOnly()).thenReturn(false);
when(deliveryService.getRoutingName()).thenReturn("edge");
+ when(deliveryService.isDns()).thenReturn(true);
- doReturn(deliveryService).when(trafficRouter).selectDeliveryService(request, false);
+ doReturn(deliveryService).when(trafficRouter).selectDeliveryService(request);
doReturn(cacheLocation).when(trafficRouter).getCoverageZoneCacheLocation("192.168.34.56", deliveryService);
doReturn(cacheRegister).when(trafficRouter).getCacheRegister();
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/GeoSortSteeringResultsTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/GeoSortSteeringResultsTest.java
index 81d3ed0..6d9d117 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/GeoSortSteeringResultsTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/GeoSortSteeringResultsTest.java
@@ -15,7 +15,7 @@
package com.comcast.cdn.traffic_control.traffic_router.core.router;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.SteeringResult;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.SteeringTarget;
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterHTTPRoutingMissesTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterHTTPRoutingMissesTest.java
index 97146af..4ba2a09 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterHTTPRoutingMissesTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterHTTPRoutingMissesTest.java
@@ -15,7 +15,7 @@
package com.comcast.cdn.traffic_control.traffic_router.core.router;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
import com.comcast.cdn.traffic_control.traffic_router.core.request.HTTPRequest;
import org.junit.Before;
import org.junit.Test;
diff --git a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterTest.java b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterTest.java
index ad8abd2..aaa5505 100644
--- a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterTest.java
+++ b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouterTest.java
@@ -15,12 +15,11 @@
package com.comcast.cdn.traffic_control.traffic_router.core.router;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.Cache;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheLocation.LocalizationMethod;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.CacheRegister;
-import com.comcast.cdn.traffic_control.traffic_router.core.cache.InetRecord;
-import com.comcast.cdn.traffic_control.traffic_router.core.config.CertificateChecker;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.Cache;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheLocation.LocalizationMethod;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
+import com.comcast.cdn.traffic_control.traffic_router.core.edge.InetRecord;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.DeliveryService;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.Dispersion;
import com.comcast.cdn.traffic_control.traffic_router.core.ds.SteeringRegistry;
@@ -34,6 +33,7 @@ import com.comcast.cdn.traffic_control.traffic_router.core.router.StatTracker.Tr
import com.comcast.cdn.traffic_control.traffic_router.core.util.CidrAddress;
import org.junit.Before;
import org.junit.Test;
+import org.xbill.DNS.Name;
import org.xbill.DNS.Type;
import java.util.ArrayList;
@@ -84,7 +84,7 @@ public class TrafficRouterTest {
trafficRouter = mock(TrafficRouter.class);
CacheRegister cacheRegister = mock(CacheRegister.class);
- when(cacheRegister.getDeliveryService(any(HTTPRequest.class), eq(true))).thenReturn(deliveryService);
+ when(cacheRegister.getDeliveryService(any(HTTPRequest.class))).thenReturn(deliveryService);
setInternalState(trafficRouter, "cacheRegister", cacheRegister);
setInternalState(trafficRouter, "federationRegistry", federationRegistry);
@@ -94,19 +94,21 @@ public class TrafficRouterTest {
when(trafficRouter.route(any(DNSRequest.class), any(Track.class))).thenCallRealMethod();
when(trafficRouter.route(any(HTTPRequest.class), any(Track.class))).thenCallRealMethod();
when(trafficRouter.singleRoute(any(HTTPRequest.class), any(Track.class))).thenCallRealMethod();
- when(trafficRouter.selectDeliveryService(any(Request.class), anyBoolean())).thenReturn(deliveryService);
+ when(trafficRouter.selectDeliveryService(any(Request.class))).thenReturn(deliveryService);
when(trafficRouter.consistentHashDeliveryService(any(DeliveryService.class), any(HTTPRequest.class), anyString())).thenCallRealMethod();
}
@Test
public void itCreatesDnsResultsFromFederationMappingHit() throws Exception {
- DNSRequest request = new DNSRequest();
+ final Name name = Name.fromString("edge.example.com");
+ DNSRequest request = new DNSRequest("example.com", name, Type.A);
request.setClientIP("192.168.10.11");
- request.setHostname("edge.example.com");
+ request.setHostname(name.relativize(Name.root).toString());
Track track = spy(StatTracker.getTrack());
when(deliveryService.getRoutingName()).thenReturn("edge");
+ when(deliveryService.isDns()).thenReturn(true);
DNSRouteResult result = trafficRouter.route(request, track);
@@ -144,7 +146,7 @@ public class TrafficRouterTest {
when(trafficRouter.getClientLocation(anyString(), any(DeliveryService.class), any(CacheLocation.class), any(Track.class))).thenReturn(new Geolocation(40, -100));
when(trafficRouter.getCachesByGeo(any(DeliveryService.class), any(Geolocation.class), any(Track.class))).thenCallRealMethod();
when(trafficRouter.getCacheRegister()).thenReturn(cacheRegister);
- when(trafficRouter.orderCacheLocations(any(List.class), any(Geolocation.class))).thenCallRealMethod();
+ when(trafficRouter.orderLocations(any(List.class), any(Geolocation.class))).thenCallRealMethod();
HTTPRouteResult httpRouteResult = trafficRouter.route(httpRequest, track);
@@ -178,7 +180,7 @@ public class TrafficRouterTest {
when(trafficRouter.getCachesByGeo(any(DeliveryService.class), any(Geolocation.class), any(Track.class))).thenCallRealMethod();
when(trafficRouter.filterEnabledLocations(any(List.class), any(LocalizationMethod.class))).thenCallRealMethod();
- when(trafficRouter.orderCacheLocations(any(List.class), any(Geolocation.class))).thenCallRealMethod();
+ when(trafficRouter.orderLocations(any(List.class), any(Geolocation.class))).thenCallRealMethod();
when(trafficRouter.getSupportingCaches(any(List.class), any(DeliveryService.class))).thenCallRealMethod();
HTTPRequest httpRequest = new HTTPRequest();
@@ -194,13 +196,13 @@ public class TrafficRouterTest {
assertThat(track.getResultLocation(), equalTo(new Geolocation(50, 50)));
when(federationRegistry.findInetRecords(anyString(), any(CidrAddress.class))).thenReturn(null);
- when(deliveryService.getRoutingName()).thenReturn("ccr");
+ when(deliveryService.getRoutingName()).thenReturn("edge");
+ when(deliveryService.isDns()).thenReturn(true);
- DNSRequest dnsRequest = new DNSRequest();
- dnsRequest.setClientIP("192.168.1.2");
+ final Name name = Name.fromString("edge.example.com");
+ DNSRequest dnsRequest = new DNSRequest("example.com", name, Type.A);
dnsRequest.setClientIP("10.10.10.10");
- dnsRequest.setHostname("ccr.example.com");
- dnsRequest.setQtype(Type.A);
+ dnsRequest.setHostname(name.relativize(Name.root).toString());
track = StatTracker.getTrack();
trafficRouter.route(dnsRequest, track);
diff --git a/traffic_router/core/src/test/resources/publish/CrConfig.json b/traffic_router/core/src/test/resources/publish/CrConfig.json
index 645e98d..c9e8cab 100644
--- a/traffic_router/core/src/test/resources/publish/CrConfig.json
+++ b/traffic_router/core/src/test/resources/publish/CrConfig.json
@@ -564,6 +564,90 @@
"fqdn": "testing-tr-01.cdn.example.com",
"ip": "12.34.20.1",
"profile": "ROUTER_TEST"
+ },
+ "edge-testing-tr-08": {
+ "port": "80",
+ "status": "ONLINE",
+ "location": "location-9",
+ "ip6": "2001:dead:beef:124:1::8",
+ "api.port": "3333",
+ "fqdn": "edge-testing-tr-08.cdn.example.com",
+ "ip": "12.34.21.8",
+ "profile": "ROUTER_TEST"
+ },
+
+ "edge-testing-tr-06": {
+ "port": "80",
+ "status": "ONLINE",
+ "location": "location-9",
+ "ip6": "2001:dead:beef:124:1::6",
+ "api.port": "3333",
+ "fqdn": "edge-testing-tr-06.cdn.example.com",
+ "ip": "12.34.21.6",
+ "profile": "ROUTER_TEST"
+ },
+
+ "edge-testing-tr-07": {
+ "port": "80",
+ "status": "ONLINE",
+ "location": "location-9",
+ "ip6": "2001:dead:beef:124:1::7",
+ "api.port": "3333",
+ "fqdn": "edge-testing-tr-07.cdn.example.com",
+ "ip": "12.34.21.7",
+ "profile": "ROUTER_TEST"
+ },
+
+ "edge-testing-tr-01": {
+ "port": "80",
+ "status": "ONLINE",
+ "location": "location-0",
+ "ip6": "2001:dead:beef:124:1::1",
+ "api.port": "3333",
+ "fqdn": "edge-testing-tr-01.cdn.example.com",
+ "ip": "12.34.21.1",
+ "profile": "ROUTER_TEST"
+ },
+
+ "edge-testing-tr-03": {
+ "port": "80",
+ "status": "ONLINE",
+ "location": "location-0",
+ "ip6": "2001:dead:beef:124:1::3",
+ "api.port": "3333",
+ "fqdn": "edge-testing-tr-03.cdn.example.com",
+ "ip": "12.34.21.3",
+ "profile": "ROUTER_TEST"
+ },
+ "edge-testing-tr-04": {
+ "port": "80",
+ "status": "ONLINE",
+ "location": "location-0",
+ "ip6": "2001:dead:beef:124:1::4",
+ "api.port": "3333",
+ "fqdn": "edge-testing-tr-04.cdn.example.com",
+ "ip": "12.34.21.4",
+ "profile": "ROUTER_TEST"
+ },
+ "edge-testing-tr-02": {
+ "port": "80",
+ "status": "ONLINE",
+ "location": "location-0",
+ "ip6": "2001:dead:beef:124:1::2",
+ "api.port": "3333",
+ "fqdn": "edge-testing-tr-02.cdn.example.com",
+ "ip": "12.34.21.2",
+ "profile": "ROUTER_TEST"
+ },
+ "edge-testing-tr-05": {
+ "port": "80",
+ "status": "ONLINE",
+ "location": "location-9",
+ "ip6": "2001:dead:beef:124:1::5",
+ "api.port": "3333",
+ "fqdn": "edge-testing-tr-05.cdn.example.com",
+ "ip": "12.34.21.5",
+ "profile": "ROUTER_TEST"
}
},
"monitors": {
@@ -1700,7 +1784,7 @@
"NS": "3600"
},
"ttl": "3600",
- "ip6RoutingEnabled": "false",
+ "ip6RoutingEnabled": "true",
"dispersion": {
"limit": 1,
"shuffled": "true"
@@ -1822,6 +1906,16 @@
"latitude": 37.0
}
},
+ "trafficRouterLocations": {
+ "location-0": {
+ "longitude": -84.0,
+ "latitude": 33.0
+ },
+ "location-9": {
+ "longitude": -87.0,
+ "latitude": 41.0
+ }
+ },
"config": {
"certificate.api.url": "http://${toHostname}/api/2.0/cdns/name/${cdnName}/sslkeys",
"client.steering.forced.diversity": "true",
@@ -1862,6 +1956,9 @@
"deepcoveragezone.polling.url": "http://localhost:8889/dczmap.json",
"api.auth.url": "http://${toHostname}/api/2.0/user/login",
"certificates.polling.interval": "10000",
- "dnssec.enabled": "false"
+ "dnssec.enabled": "false",
+ "edge.dns.routing": "true",
+ "edge.http.routing": "true"
}
}
+
diff --git a/traffic_router/core/src/test/resources/publish/CrConfig2.json b/traffic_router/core/src/test/resources/publish/CrConfig2.json
index 32a9d35..12f949b 100644
--- a/traffic_router/core/src/test/resources/publish/CrConfig2.json
+++ b/traffic_router/core/src/test/resources/publish/CrConfig2.json
@@ -1200,6 +1200,16 @@
"latitude": 36.0
}
},
+ "trafficRouterLocations": {
+ "location-0": {
+ "longitude": -84.0,
+ "latitude": 33.0
+ },
+ "location-9": {
+ "longitude": -87.0,
+ "latitude": 41.0
+ }
+ },
"config": {
"federationmapping.polling.url": "http://${toHostname}/api/2.0/federations/all",
"certificate.api.url": "http://${toHostname}/api/2.0/cdns/name/${cdnName}/sslkeys",
@@ -1241,3 +1251,4 @@
"dnssec.enabled": "false"
}
}
+
diff --git a/traffic_router/core/src/test/resources/publish/CrConfig3.json b/traffic_router/core/src/test/resources/publish/CrConfig3.json
index 02f85f0..52ed5cf4 100644
--- a/traffic_router/core/src/test/resources/publish/CrConfig3.json
+++ b/traffic_router/core/src/test/resources/publish/CrConfig3.json
@@ -1195,7 +1195,7 @@
"matchsets": [{
"protocol": "DNS",
"matchlist": [{
- "regex": ".*\\.http-only-test\\..*",
+ "regex": ".*\\.http-dns-test\\..*",
"match-type": "HOST"
}]
}],
@@ -1251,6 +1251,16 @@
"latitude": 36.0
}
},
+ "trafficRouterLocations": {
+ "location-0": {
+ "longitude": -84.0,
+ "latitude": 33.0
+ },
+ "location-9": {
+ "longitude": -87.0,
+ "latitude": 41.0
+ }
+ },
"config": {
"federationmapping.polling.url": "http://${toHostname}/api/2.0/federations/all",
"certificate.api.url": "http://${toHostname}/api/2.0/cdns/name/${cdnName}/sslkeys",
@@ -1292,3 +1302,4 @@
"dnssec.enabled": "false"
}
}
+
diff --git a/traffic_router/core/src/test/resources/publish/CrConfig4.json b/traffic_router/core/src/test/resources/publish/CrConfig4.json
index 6503234..1f27b41 100644
--- a/traffic_router/core/src/test/resources/publish/CrConfig4.json
+++ b/traffic_router/core/src/test/resources/publish/CrConfig4.json
@@ -1189,7 +1189,7 @@
"matchsets": [{
"protocol": "DNS",
"matchlist": [{
- "regex": ".*\\.http-only-test\\..*",
+ "regex": ".*\\.http-dns-test\\..*",
"match-type": "HOST"
}]
}],
@@ -1245,6 +1245,16 @@
"latitude": 36.0
}
},
+ "trafficRouterLocations": {
+ "location-0": {
+ "longitude": -84.0,
+ "latitude": 33.0
+ },
+ "location-9": {
+ "longitude": -87.0,
+ "latitude": 41.0
+ }
+ },
"config": {
"federationmapping.polling.url": "http://${toHostname}/api/2.0/federations/all",
"certificate.api.url": "http://${toHostname}/api/2.0/cdns/name/${cdnName}/sslkeys",
@@ -1286,3 +1296,4 @@
"dnssec.enabled": "false"
}
}
+
[trafficcontrol] 04/04: Fix up the docs, add changelog entry
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
elsloo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 2dd9a0cdf1fa3f7262f8518b779e50c4e3c36804
Author: Rawlin Peters <ra...@apache.org>
AuthorDate: Mon Jun 8 14:18:30 2020 -0600
Fix up the docs, add changelog entry
---
CHANGELOG.md | 1 +
docs/source/admin/index.rst | 2 +-
.../admin/{traffic_router/index.rst => traffic_router.rst} | 12 ++++++------
3 files changed, 8 insertions(+), 7 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 572f3d0..7e35fec 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -17,6 +17,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
- Traffic Portal: Added the ability to assign topologies to delivery services.
- Traffic Portal: Added the ability to view all delivery services and cache groups associated with a topology.
- Updated /servers/details to use multiple interfaces in API v3
+- Added [Edge Traffic Routing](https://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_router.html#edge-traffic-routing) feature which allows Traffic Router to localize more DNS record types than just the routing name for DNS delivery services
- Astats csv support - astats will now respond to `Accept: text/csv` and return a csv formatted stats list
### Fixed
diff --git a/docs/source/admin/index.rst b/docs/source/admin/index.rst
index af8b55a..8c5a889 100644
--- a/docs/source/admin/index.rst
+++ b/docs/source/admin/index.rst
@@ -39,7 +39,7 @@ Once everything is installed, you will need to configure the servers to talk to
traffic_ops.rst
traffic_portal/*
traffic_monitor.rst
- traffic_router/index.rst
+ traffic_router.rst
traffic_router/migrationto2-3.rst
traffic_stats.rst
traffic_server.rst
diff --git a/docs/source/admin/traffic_router/index.rst b/docs/source/admin/traffic_router.rst
similarity index 99%
rename from docs/source/admin/traffic_router/index.rst
rename to docs/source/admin/traffic_router.rst
index fd965f5..beee5c5 100644
--- a/docs/source/admin/traffic_router/index.rst
+++ b/docs/source/admin/traffic_router.rst
@@ -391,22 +391,22 @@ Localizing ``NS`` records reduces latency for the resolver, which, due to cachin
Edge DNS Routing
----------------
-Edge DNS routing refers to the localization of ``NS`` records in Traffic Router. This can be turned on and off via the ``edge.dns.routing`` parameter; the number of records returned is controlled via ``edge.dns.limit`` and there is a hardcoded default limit of 4 when this feature is enabled. See :ref:`ccr-profile` documentation for parameter details.
+Edge DNS routing refers to the localization of ``NS`` records in Traffic Router. This can be turned on and off via the ``edge.dns.routing`` parameter; the number of records returned is controlled via ``edge.dns.limit`` and there is a hardcoded default limit of 4 when this feature is enabled. See :ref:`tr-profile` documentation for parameter details.
Edge HTTP Routing
-----------------
-Edge HTTP routing refers to the localization of ``A`` and ``AAAA`` records that represent routing names for HTTP delivery services. This can be turned on and off via the ``edge.http.routing`` parameter; the number of records returned is controlled via ``edge.http.limit`` and there is a hardcoded default limit of 4 when this feature is enabled. The default or global limit can be overridden by modifying the ``maxDnsAnswers`` setting on the delivery service. See :ref:`ccr-profile` documenta [...]
+Edge HTTP routing refers to the localization of ``A`` and ``AAAA`` records that represent routing names for HTTP delivery services. This can be turned on and off via the ``edge.http.routing`` parameter; the number of records returned is controlled via ``edge.http.limit`` and there is a hardcoded default limit of 4 when this feature is enabled. The default or global limit can be overridden by modifying the ``maxDnsAnswers`` setting on the delivery service. See :ref:`tr-profile` documentat [...]
Edge Traffic Router Selection
-----------------------------
Traffic Router performs localization on client requests in order to determine which Traffic Routers should service a given request. After localization, Traffic Router will perform a consistent hash on the incoming name and will use the value to refine Traffic Router selection. There are two main cases for Traffic Router selection: a localization hit, and a localization miss.
Localization Hit: Consistent Hash (CH)
-**************************************
+""""""""""""""""""""""""""""""""""""""
When a client is localized, Traffic Router selects the nearest Traffic Router Location (cachegroup) based on proximity. Proximity is determined by using the latitude and longitude of the client, regardless of whether it is a coverage zone or geolocation hit, and the latitude and longitude specified on Traffic Router Locations. Once the location is identified, a consistent hash is performed on the incoming name and the list of Traffic Routers is ordered based upon the consistent hash. Onc [...]
Localization Miss: Consistent Hash + Round Robin (CH + RR)
-**********************************************************
+""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
When a client cannot be localized, Traffic Router still needs to produce a list of Traffic Routers to service the request. Because the number of Traffic Routers in the CDN could far exceed the practical limits of what constitutes a "normal" sized answer, a selection algorithm is applied to select Traffic Routers. Much like the localization hit scenario, the incoming request name is consistent hashed and results size is limited by the same parameters. Because no client location is known, [...]
For example, with four Traffic Router Locations each containing 10 Traffic Routers and a limit of 6, the algorithm would:
@@ -419,10 +419,10 @@ For example, with four Traffic Router Locations each containing 10 Traffic Route
Because the algorithm employs consistent hashing, the answers should be consistent as long as the topology remains the same. This approach can be thought of as the consistent hash round robin (CH + RR) selection process.
Example Request Flow
-********************
+""""""""""""""""""""
The following is an example of the request flow when a client requests the routing name for an example delivery service, ``tr.service.cdn.example.com``. The request flow assumes that the resolver is cold and has yet to build a local cache of lookups, meaning it has to walk the domain hierarchy asking for ``NS`` records until it reaches ``service.cdn.example.com``. This example starts after the resolver has determined which name servers are authoritative for ``cdn.example.com``. Note that [...]
-.. figure:: images/edge_tr_example.png
+.. figure:: traffic_router/images/edge_tr_example.png
:scale: 30%
:align: center
:alt: Example Request Flow for Edge Traffic Routing