You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficcontrol.apache.org by ne...@apache.org on 2020/03/17 16:12:11 UTC
[trafficcontrol] branch master updated: Change ORT/atstccfg to
generate all files at once, massive performance improvement (#4493)
This is an automated email from the ASF dual-hosted git repository.
neuman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
The following commit(s) were added to refs/heads/master by this push:
new 2748999 Change ORT/atstccfg to generate all files at once, massive performance improvement (#4493)
2748999 is described below
commit 27489995f56eb966335cb81ac0e0b374c45158e0
Author: Robert O Butts <ro...@users.noreply.github.com>
AuthorDate: Tue Mar 17 10:12:01 2020 -0600
Change ORT/atstccfg to generate all files at once, massive performance improvement (#4493)
* WIP change atstccfg to generate all files at once
Changes atstccfg to build the meta config, and then build all configs
from it, and output them to a specified directory, in a single run.
This is much faster than independent runs called by ORT, and also
a step toward replacing ORT with atstccfg.
This changeset does not modify ORT as necessary to handle the single,
run, and is thus WIP and not complete or functional yet.
* WIP Change atstccfg to geneate multipart file
* WIP add atstccfg args for TO data that ORT needs
* Remove ORT unused api_in_use conditionals
* Remove ORT unused funcs
* Add ORT atstccfg reval-only flag.
Adds a flag to the atstccfg config generator, to only get files
named regex_revalidate.config.
This is needed by ORT, which has a mode to only get reval files.
* Change ORT to use atstccfg single-file generation
* Remove ORT atstccfg unused func, rename func
Removes unused funcs getting data from TO (which were previously used
with the one-at-a-time generation, but not needed by the new
single-file gen).
Also renamed GetCached to GetRetry, to reflect what it actually does
now.
* Fix lib/go-tc missing symbol from 4.0.x.
Adds back a symbol which was in 4.0.x. This allows users to vendor
libraries that use lib/go-tc, namely traffic_ops/client.
This is critically important. Without maintaining backward
compatibility in lib/go-tc (which Go Modules will require anyway),
it's impossible to vendor the client, and thus (since we keep the
client in master matching the latest TO Server) impossible to write a
new app using the client to request a production Traffic Ops which
isn't the version of the very latest master branch.
Unless that app also vendored lib/go-tc. But then the Go symbols
of the vendored and unvendored tc don't match and won't compile,
unless the app itself also vendors lib/go-tc. Except any external
libraries it uses which also use lib/go-tc (for example,
lib/go-atscfg) _can't_ be made to use the vendored version. Making
it actually impossible to decompose your app into libraries.
Moreover, writing functions to translate between vendored and
unvendored symbols is also impossible, because Go won't let you
import or declare variables of a vendored type, making it actually
impossible to declare a function with a vendored symbol as the input
or output.
It is theoretically possible to make the code compile, but it is
extremely, extremely painful. I am speaking from experience.
* Vendor ORT traffic_ops/client
ORT is required to work with at least one version back, so users
aren't required to update in a particular order.
This is necesary, because the master client requests endpoints
which don't exist in the previous TC version.
* Fix intermittent test failure
The unit test was verifying a port 42 wasn't in a config file.
But the timestamp is in the header comment, so whenever the time
contained '42' the test failed.
This changes the port to be a bigger number, so it's less likely.
* Fix broken test, remove unused files
Removes a unit test for functions that no longer exist.
Also moves the last non-caching func from caching.go and removes the
file. Since atstccfg no longer does caching.
* Add ORT atstccfg preprocessing
* Remove unnecessary type declaration
* Fix ORT erroneous return preprocessing
* Remove comment
* Remove ORT atstccfg unused return val
* Fix ORT atstccfg debug logging to info
atstccfg doesn't use the debug logger and it's always nil. Fixed
uses to Info.
* Remove ORT atstccfg unused OutputDir config
* Remove ORT atstccfg PrintGeneratedFiles flag
Flag no longer makes sense, as atstccfg generates all files now.
Some files are dynamically generated, so callers should only have
used the flag to determine if a file was generated by atstccfg or
Traffic Ops, not whether the file exists.
* Fix ORT atstccfg Cfg struct to be ordered
* Fix ORT atstccfg wrong usage string
* Fix ORT atstccfg misnamed log msg
* Add ORT atstccfg comment
* Remove ORT unused imports, variables
* Fix ORT spacing
* Add ORT readme
* Add API Test for TO client GetUpdateStatus
* Add ORT atstccfg unit tests
* Fix ORT atstccfg missing licenses
* Add ORT atstccfg sslkeys file
* Fix ORT to ensure files have exactly 1 \n$
Fixes ORT to trim all trailing whitespace from files, and add exactly
one trailing newline as required by POSIX.
This specifically fixes it to trim carriage returns left in by
the mixed/multipart message.
---
lib/go-atscfg/astatsdotconfig.go | 2 +
lib/go-atscfg/atscfg.go | 2 +
lib/go-atscfg/atsdotrules.go | 2 +
lib/go-atscfg/bgfetchdotconfig.go | 2 +
lib/go-atscfg/cachedotconfig.go | 2 +
lib/go-atscfg/cacheurldotconfig.go | 2 +
lib/go-atscfg/chkconfig.go | 2 +-
lib/go-atscfg/dropqstringdotconfig.go | 1 +
lib/go-atscfg/facts.go | 2 +
lib/go-atscfg/headerrewritedotconfig.go | 1 +
lib/go-atscfg/hostingdotconfig.go | 2 +-
lib/go-atscfg/ipallowdotconfig.go | 1 +
lib/go-atscfg/loggingdotconfig.go | 1 +
lib/go-atscfg/loggingdotyaml.go | 1 +
lib/go-atscfg/logsdotxml.go | 1 +
lib/go-atscfg/meta.go | 34 +-
lib/go-atscfg/packages.go | 2 +
lib/go-atscfg/parentdotconfig.go | 2 +
lib/go-atscfg/plugindotconfig.go | 1 +
lib/go-atscfg/recordsdotconfig.go | 1 +
lib/go-atscfg/regexremapdotconfig.go | 2 +
lib/go-atscfg/regexrevalidatedotconfig.go | 2 +
lib/go-atscfg/remapdotconfig.go | 1 +
lib/go-atscfg/servercachedotconfig_test.go | 6 +-
lib/go-atscfg/serverunknown.go | 2 +
lib/go-atscfg/setdscpdotconfig.go | 2 +
lib/go-atscfg/sslmulticertdotconfig.go | 65 +-
lib/go-atscfg/storagedotconfig.go | 2 +
lib/go-atscfg/sysctldotconf.go | 1 +
lib/go-atscfg/unknownconfig.go | 2 +
lib/go-atscfg/urisigningconfig.go | 2 +
lib/go-atscfg/volumedotconfig.go | 2 +
lib/go-tc/stats_summary.go | 9 +-
traffic_ops/client/server.go | 20 +
traffic_ops/ort/README.md | 99 +++
traffic_ops/ort/atstccfg/atstccfg.go | 69 +-
traffic_ops/ort/atstccfg/cfgfile/all.go | 134 +++
.../ort/atstccfg/cfgfile/astatsdotconfig.go | 30 +-
traffic_ops/ort/atstccfg/cfgfile/atsdotrules.go | 31 +-
.../ort/atstccfg/cfgfile/bgfetchdotconfig.go | 18 +-
traffic_ops/ort/atstccfg/cfgfile/cachedotconfig.go | 49 +-
.../ort/atstccfg/cfgfile/cacheurldotconfig.go | 46 +-
traffic_ops/ort/atstccfg/cfgfile/cfgfile.go | 485 +++++++++++
traffic_ops/ort/atstccfg/cfgfile/cfgfile_test.go | 111 +++
traffic_ops/ort/atstccfg/cfgfile/chkconfig.go | 50 +-
.../ort/atstccfg/cfgfile/dropqstringdotconfig.go | 31 +-
traffic_ops/ort/atstccfg/cfgfile/facts.go | 17 +-
.../ort/atstccfg/cfgfile/headerrewritedotconfig.go | 47 +-
.../atstccfg/cfgfile/headerrewritemiddotconfig.go | 59 +-
.../ort/atstccfg/cfgfile/hostingdotconfig.go | 99 +--
.../ort/atstccfg/cfgfile/ipallowdotconfig.go | 78 +-
.../ort/atstccfg/cfgfile/loggingdotconfig.go | 41 +-
traffic_ops/ort/atstccfg/cfgfile/loggingdotyaml.go | 41 +-
.../ort/atstccfg/cfgfile/logsxmldotconfig.go | 41 +-
traffic_ops/ort/atstccfg/cfgfile/meta.go | 148 ++--
traffic_ops/ort/atstccfg/cfgfile/packages.go | 57 +-
.../ort/atstccfg/cfgfile/parentdotconfig.go | 185 ++---
.../ort/atstccfg/cfgfile/plugindotconfig.go | 41 +-
.../ort/atstccfg/cfgfile/recordsdotconfig.go | 41 +-
.../ort/atstccfg/cfgfile/regexremapdotconfig.go | 36 +-
.../atstccfg/cfgfile/regexrevalidatedotconfig.go | 55 +-
traffic_ops/ort/atstccfg/cfgfile/remapdotconfig.go | 164 +---
traffic_ops/ort/atstccfg/cfgfile/routing.go | 190 +++++
.../ort/atstccfg/cfgfile/servercachedotconfig.go | 52 +-
.../ort/atstccfg/cfgfile/serverunknownconfig.go | 63 +-
.../ort/atstccfg/cfgfile/setdscpdotconfig.go | 19 +-
traffic_ops/ort/atstccfg/cfgfile/sslkeys.go | 76 ++
.../ort/atstccfg/cfgfile/sslmulticertdotconfig.go | 40 +-
.../ort/atstccfg/cfgfile/storagedotconfig.go | 41 +-
traffic_ops/ort/atstccfg/cfgfile/sysctldotconf.go | 30 +-
traffic_ops/ort/atstccfg/cfgfile/unknownconfig.go | 51 +-
.../ort/atstccfg/cfgfile/urisigningconfig.go | 15 +-
traffic_ops/ort/atstccfg/cfgfile/urlsigconfig.go | 37 +-
.../ort/atstccfg/cfgfile/volumedotconfig.go | 41 +-
traffic_ops/ort/atstccfg/config/config.go | 114 +--
traffic_ops/ort/atstccfg/getdata/getdata.go | 215 +++++
traffic_ops/ort/atstccfg/plugin/plugin.go | 8 +-
traffic_ops/ort/atstccfg/routing.go | 235 ------
traffic_ops/ort/atstccfg/toreq/caching.go | 280 -------
traffic_ops/ort/atstccfg/toreq/caching_test.go | 163 ----
traffic_ops/ort/atstccfg/toreq/toreq.go | 280 +++----
traffic_ops/ort/atstccfg/toreq/trafficops.go | 116 +--
traffic_ops/ort/traffic_ops_ort.pl | 916 +++++----------------
.../github.com/apache/trafficcontrol/LICENSE | 454 ++++++++++
.../github.com/apache/trafficcontrol/VERSION | 1 +
.../github.com/apache/trafficcontrol/changeset.txt | 2 +
.../trafficcontrol/traffic_ops/client/README.md | 53 ++
.../trafficcontrol/traffic_ops/client/about.go | 43 +
.../trafficcontrol/traffic_ops/client/asn.go | 132 +++
.../trafficcontrol/traffic_ops/client/atsconfig.go | 88 ++
.../traffic_ops/client/cachegroup.go | 306 +++++++
.../traffic_ops/client/cachegroup_parameters.go | 109 +++
.../trafficcontrol/traffic_ops/client/cdn.go | 176 ++++
.../traffic_ops/client/cdn_domains.go | 29 +
.../traffic_ops/client/cdnfederations.go | 73 ++
.../traffic_ops/client/coordinate.go | 155 ++++
.../trafficcontrol/traffic_ops/client/crconfig.go | 53 ++
.../traffic_ops/client/deliveryservice.go | 456 ++++++++++
.../client/deliveryservice_endpoints.go | 70 ++
.../client/deliveryservice_request_comments.go | 115 +++
.../traffic_ops/client/deliveryservice_requests.go | 243 ++++++
.../deliveryservices_required_capabilities.go | 83 ++
.../traffic_ops/client/deliveryserviceserver.go | 119 +++
.../trafficcontrol/traffic_ops/client/division.go | 146 ++++
.../trafficcontrol/traffic_ops/client/dsuser.go | 60 ++
.../trafficcontrol/traffic_ops/client/endpoints.go | 18 +
.../traffic_ops/client/federation.go | 211 +++++
.../traffic_ops/client/federation_resolver.go | 116 +++
.../trafficcontrol/traffic_ops/client/hardware.go | 52 ++
.../trafficcontrol/traffic_ops/client/iso.go | 40 +-
.../trafficcontrol/traffic_ops/client/job.go | 196 +++++
.../trafficcontrol/traffic_ops/client/log.go | 61 ++
.../trafficcontrol/traffic_ops/client/origin.go | 194 +++++
.../trafficcontrol/traffic_ops/client/parameter.go | 209 +++++
.../traffic_ops/client/phys_location.go | 143 ++++
.../trafficcontrol/traffic_ops/client/ping.go | 39 +
.../trafficcontrol/traffic_ops/client/profile.go | 264 ++++++
.../traffic_ops/client/profile_parameter.go | 96 +++
.../trafficcontrol/traffic_ops/client/region.go | 161 ++++
.../trafficcontrol/traffic_ops/client/role.go | 160 ++++
.../trafficcontrol/traffic_ops/client}/server.go | 65 +-
.../client/server_server_capabilities.go | 83 ++
.../traffic_ops/client/server_update_status.go | 81 ++
.../traffic_ops/client/servercapability.go | 104 +++
.../traffic_ops/client/servercheck.go | 56 ++
.../traffic_ops/client/serversstatus.go | 44 +
.../trafficcontrol/traffic_ops/client/session.go | 503 +++++++++++
.../traffic_ops/client/staticdnsentry.go | 174 ++++
.../traffic_ops/client/stats_summary.go | 125 +++
.../trafficcontrol/traffic_ops/client/status.go | 150 ++++
.../trafficcontrol/traffic_ops/client/steering.go | 38 +
.../traffic_ops/client/steeringtarget.go | 96 +++
.../trafficcontrol/traffic_ops/client/tenant.go | 117 +++
.../traffic_ops/client/tenant_endpoints.go | 25 +
.../traffic_ops/client/toextension.go | 69 ++
.../traffic_ops/client/traffic_monitor.go | 70 ++
.../traffic_ops/client/traffic_stats.go | 29 +
.../trafficcontrol/traffic_ops/client/type.go | 159 ++++
.../trafficcontrol/traffic_ops/client/update.go | 72 ++
.../trafficcontrol/traffic_ops/client/user.go | 185 +++++
.../trafficcontrol/traffic_ops/client/util.go | 58 ++
traffic_ops/testing/api/v1/updatestatus_test.go | 40 +
traffic_ops/v1-client/server.go | 20 +
143 files changed, 9424 insertions(+), 3332 deletions(-)
diff --git a/lib/go-atscfg/astatsdotconfig.go b/lib/go-atscfg/astatsdotconfig.go
index 910f048..0fbc0bf 100644
--- a/lib/go-atscfg/astatsdotconfig.go
+++ b/lib/go-atscfg/astatsdotconfig.go
@@ -22,6 +22,8 @@ package atscfg
const AstatsSeparator = "="
const AstatsFileName = "astats.config"
+const ContentTypeAstatsDotConfig = ContentTypeTextASCII
+
func MakeAStatsDotConfig(
profileName string,
paramData map[string]string, // GetProfileParamData(tx, profile.ID, AstatsFileName)
diff --git a/lib/go-atscfg/atscfg.go b/lib/go-atscfg/atscfg.go
index c6edf3e..2e38e4e 100644
--- a/lib/go-atscfg/atscfg.go
+++ b/lib/go-atscfg/atscfg.go
@@ -34,6 +34,8 @@ const DefaultATSVersion = "5" // TODO Emulates Perl; change to 6? ATC no longer
const HeaderCommentDateFormat = "Mon Jan 2 15:04:05 MST 2006"
+const ContentTypeTextASCII = `text/plain; charset=us-ascii`
+
type ServerCapability string
type ServerInfo struct {
diff --git a/lib/go-atscfg/atsdotrules.go b/lib/go-atscfg/atsdotrules.go
index e2b46ab..5ea8f43 100644
--- a/lib/go-atscfg/atsdotrules.go
+++ b/lib/go-atscfg/atsdotrules.go
@@ -23,6 +23,8 @@ import (
"strings"
)
+const ContentTypeATSDotRules = ContentTypeTextASCII
+
func MakeATSDotRules(
profileName string,
paramData map[string]string, // GetProfileParamData(tx, profile.ID, StorageFileName)
diff --git a/lib/go-atscfg/bgfetchdotconfig.go b/lib/go-atscfg/bgfetchdotconfig.go
index 4d2c2aa..3480613 100644
--- a/lib/go-atscfg/bgfetchdotconfig.go
+++ b/lib/go-atscfg/bgfetchdotconfig.go
@@ -23,6 +23,8 @@ import (
"github.com/apache/trafficcontrol/lib/go-tc"
)
+const ContentTypeBGFetchDotConfig = ContentTypeTextASCII
+
func MakeBGFetchDotConfig(
cdnName tc.CDNName,
toToolName string, // tm.toolname global parameter (TODO: cache itself?)
diff --git a/lib/go-atscfg/cachedotconfig.go b/lib/go-atscfg/cachedotconfig.go
index ce7919c..ba95e0e 100644
--- a/lib/go-atscfg/cachedotconfig.go
+++ b/lib/go-atscfg/cachedotconfig.go
@@ -26,6 +26,8 @@ import (
"github.com/apache/trafficcontrol/lib/go-tc"
)
+const ContentTypeCacheDotConfig = ContentTypeTextASCII
+
type ProfileDS struct {
Type tc.DSType
OriginFQDN *string
diff --git a/lib/go-atscfg/cacheurldotconfig.go b/lib/go-atscfg/cacheurldotconfig.go
index 8886ae0..f6a077c 100644
--- a/lib/go-atscfg/cacheurldotconfig.go
+++ b/lib/go-atscfg/cacheurldotconfig.go
@@ -26,6 +26,8 @@ import (
"github.com/apache/trafficcontrol/lib/go-tc"
)
+const ContentTypeCacheURLDotConfig = ContentTypeTextASCII
+
type CacheURLDS struct {
OrgServerFQDN string
QStringIgnore int
diff --git a/lib/go-atscfg/chkconfig.go b/lib/go-atscfg/chkconfig.go
index 56f9df5..581ca38 100644
--- a/lib/go-atscfg/chkconfig.go
+++ b/lib/go-atscfg/chkconfig.go
@@ -26,8 +26,8 @@ import (
)
const ChkconfigFileName = `chkconfig`
-
const ChkconfigParamConfigFile = `chkconfig`
+const ContentTypeChkconfig = ContentTypeTextASCII
type ChkConfigEntry struct {
Name string `json:"name"`
diff --git a/lib/go-atscfg/dropqstringdotconfig.go b/lib/go-atscfg/dropqstringdotconfig.go
index b3efa7c..3c7c59b 100644
--- a/lib/go-atscfg/dropqstringdotconfig.go
+++ b/lib/go-atscfg/dropqstringdotconfig.go
@@ -21,6 +21,7 @@ package atscfg
const DropQStringDotConfigFileName = "drop_qstring.config"
const DropQStringDotConfigParamName = "content"
+const ContentTypeDropQStringDotConfig = ContentTypeTextASCII
func MakeDropQStringDotConfig(
profileName string,
diff --git a/lib/go-atscfg/facts.go b/lib/go-atscfg/facts.go
index b8413c0..70102ce 100644
--- a/lib/go-atscfg/facts.go
+++ b/lib/go-atscfg/facts.go
@@ -19,6 +19,8 @@ package atscfg
* under the License.
*/
+const ContentType12MFacts = ContentTypeTextASCII
+
func Make12MFacts(
profileName string,
toToolName string, // tm.toolname global parameter (TODO: cache itself?)
diff --git a/lib/go-atscfg/headerrewritedotconfig.go b/lib/go-atscfg/headerrewritedotconfig.go
index fa22a32..083ee91 100644
--- a/lib/go-atscfg/headerrewritedotconfig.go
+++ b/lib/go-atscfg/headerrewritedotconfig.go
@@ -30,6 +30,7 @@ import (
)
const HeaderRewritePrefix = "hdr_rw_"
+const ContentTypeHeaderRewriteDotConfig = ContentTypeTextASCII
const MaxOriginConnectionsNoMax = 0 // 0 indicates no limit on origin connections
diff --git a/lib/go-atscfg/hostingdotconfig.go b/lib/go-atscfg/hostingdotconfig.go
index 55542e5..f3f53bc 100644
--- a/lib/go-atscfg/hostingdotconfig.go
+++ b/lib/go-atscfg/hostingdotconfig.go
@@ -27,8 +27,8 @@ import (
)
const HostingConfigFileName = `hosting.config`
-
const HostingConfigParamConfigFile = `storage.config`
+const ContentTypeHostingDotConfig = ContentTypeTextASCII
const ParamDrivePrefix = "Drive_Prefix"
const ParamRAMDrivePrefix = "RAM_Drive_Prefix"
diff --git a/lib/go-atscfg/ipallowdotconfig.go b/lib/go-atscfg/ipallowdotconfig.go
index 4dd0168..74b27c1 100644
--- a/lib/go-atscfg/ipallowdotconfig.go
+++ b/lib/go-atscfg/ipallowdotconfig.go
@@ -30,6 +30,7 @@ import (
)
const IPAllowConfigFileName = `ip_allow.config`
+const ContentTypeIPAllowDotConfig = ContentTypeTextASCII
type IPAllowData struct {
Src string
diff --git a/lib/go-atscfg/loggingdotconfig.go b/lib/go-atscfg/loggingdotconfig.go
index f0f5dd7..fdb24fc 100644
--- a/lib/go-atscfg/loggingdotconfig.go
+++ b/lib/go-atscfg/loggingdotconfig.go
@@ -29,6 +29,7 @@ import (
const MaxLogObjects = 10
const LoggingFileName = "logging.config"
+const ContentTypeLoggingDotConfig = ContentTypeTextASCII
// MakeStorageDotConfig creates storage.config for a given ATS Profile.
// The paramData is the map of parameter names to values, for all parameters assigned to the given profile, with the config_file "storage.config".
diff --git a/lib/go-atscfg/loggingdotyaml.go b/lib/go-atscfg/loggingdotyaml.go
index 6f9ffae..02e3968 100644
--- a/lib/go-atscfg/loggingdotyaml.go
+++ b/lib/go-atscfg/loggingdotyaml.go
@@ -27,6 +27,7 @@ import (
)
const LoggingYAMLFileName = "logging.yaml"
+const ContentTypeLoggingDotYAML = "application/yaml; charset=us-ascii" // Note YAML has no IANA standard mime type. This is one of several common usages, and is likely to be the standardized value. If you're reading this, please check IANA to see if YAML has been added, and change this to the IANA definition if so. Also note we include 'charset=us-ascii' because YAML is commonly UTF-8, but ATS is likely to be unable to handle UTF.
func MakeLoggingDotYAML(
profileName string,
diff --git a/lib/go-atscfg/logsdotxml.go b/lib/go-atscfg/logsdotxml.go
index 681fa69..991b411 100644
--- a/lib/go-atscfg/logsdotxml.go
+++ b/lib/go-atscfg/logsdotxml.go
@@ -25,6 +25,7 @@ import (
)
const LogsXMLFileName = "logs_xml.config"
+const ContentTypeLogsDotXML = `text/xml`
func MakeLogsXMLDotConfig(
profileName string,
diff --git a/lib/go-atscfg/meta.go b/lib/go-atscfg/meta.go
index e1929f2..9c67445 100644
--- a/lib/go-atscfg/meta.go
+++ b/lib/go-atscfg/meta.go
@@ -50,6 +50,29 @@ func MakeMetaConfig(
scopeParams map[string]string, // map[configFileName]scopeParam
dsNames map[tc.DeliveryServiceName]struct{},
) string {
+ return MetaObjToMetaConfig(MakeMetaObj(serverHostName, server, tmURL, tmReverseProxyURL, locationParams, uriSignedDSes, scopeParams, dsNames))
+}
+
+func MetaObjToMetaConfig(atsData tc.ATSConfigMetaData) string {
+ bts, err := json.Marshal(atsData)
+ if err != nil {
+ // should never happen
+ log.Errorln("marshalling meta config: " + err.Error())
+ bts = []byte("error encoding to json, see log for details")
+ }
+ return string(bts)
+}
+
+func MakeMetaObj(
+ serverHostName tc.CacheName,
+ server *ServerInfo,
+ tmURL string, // global tm.url Parameter
+ tmReverseProxyURL string, // global tm.rev_proxy.url Parameter
+ locationParams map[string]ConfigProfileParams, // map[configFile]params; 'location' and 'URL' Parameters on serverHostName's Profile
+ uriSignedDSes []tc.DeliveryServiceName,
+ scopeParams map[string]string, // map[configFileName]scopeParam
+ dsNames map[tc.DeliveryServiceName]struct{},
+) tc.ATSConfigMetaData {
if tmURL == "" {
log.Errorln("ats.GetConfigMetadata: global tm.url parameter missing or empty! Setting empty in meta config!")
}
@@ -133,14 +156,7 @@ locationParamsFor:
atsData.ConfigFiles = append(atsData.ConfigFiles, atsCfg)
}
-
- bts, err := json.Marshal(atsData)
- if err != nil {
- // should never happen
- log.Errorln("marshalling chkconfig NameVersions: " + err.Error())
- bts = []byte("error encoding to json, see log for details")
- }
- return string(bts)
+ return atsData
}
func getServerScope(cfgFile string, serverType string, scopeParams map[string]string) tc.ATSConfigMetaDataConfigFileScope {
@@ -182,7 +198,7 @@ func getScope(cfgFile string, scopeParams map[string]string) tc.ATSConfigMetaDat
return tc.ATSConfigMetaDataConfigFileScopeProfiles
case cfgFile == "bg_fetch.config",
cfgFile == "regex_revalidate.config",
- cfgFile == "ssl_multicert.config",
+ cfgFile == SSLMultiCertConfigFileName,
strings.HasPrefix(cfgFile, "cacheurl") && strings.HasSuffix(cfgFile, ".config"),
strings.HasPrefix(cfgFile, "hdr_rw_") && strings.HasSuffix(cfgFile, ".config"),
strings.HasPrefix(cfgFile, "regex_remap_") && strings.HasSuffix(cfgFile, ".config"),
diff --git a/lib/go-atscfg/packages.go b/lib/go-atscfg/packages.go
index a7b73ac..b675072 100644
--- a/lib/go-atscfg/packages.go
+++ b/lib/go-atscfg/packages.go
@@ -28,6 +28,8 @@ import (
const PackagesFileName = `packages`
const PackagesParamConfigFile = `package`
+const ContentTypePackages = ContentTypeTextASCII
+
type Package struct {
Name string `json:"name"`
Version string `json:"version"`
diff --git a/lib/go-atscfg/parentdotconfig.go b/lib/go-atscfg/parentdotconfig.go
index a5a62fa..a80a7e2 100644
--- a/lib/go-atscfg/parentdotconfig.go
+++ b/lib/go-atscfg/parentdotconfig.go
@@ -31,6 +31,8 @@ import (
"github.com/apache/trafficcontrol/lib/go-util"
)
+const ContentTypeParentDotConfig = ContentTypeTextASCII
+
const ParentConfigParamQStringHandling = "psel.qstring_handling"
const ParentConfigParamMSOAlgorithm = "mso.algorithm"
const ParentConfigParamMSOParentRetry = "mso.parent_retry"
diff --git a/lib/go-atscfg/plugindotconfig.go b/lib/go-atscfg/plugindotconfig.go
index 9a1ba52..5a9eb5a 100644
--- a/lib/go-atscfg/plugindotconfig.go
+++ b/lib/go-atscfg/plugindotconfig.go
@@ -21,6 +21,7 @@ package atscfg
const PluginSeparator = " "
const PluginFileName = "plugin.config"
+const ContentTypePluginDotConfig = ContentTypeTextASCII
func MakePluginDotConfig(
profileName string,
diff --git a/lib/go-atscfg/recordsdotconfig.go b/lib/go-atscfg/recordsdotconfig.go
index b00e992..18aa3f8 100644
--- a/lib/go-atscfg/recordsdotconfig.go
+++ b/lib/go-atscfg/recordsdotconfig.go
@@ -25,6 +25,7 @@ import (
const RecordsSeparator = " "
const RecordsFileName = "records.config"
+const ContentTypeRecordsDotConfig = ContentTypeTextASCII
func MakeRecordsDotConfig(
profileName string,
diff --git a/lib/go-atscfg/regexremapdotconfig.go b/lib/go-atscfg/regexremapdotconfig.go
index 1ab4f51..f726793 100644
--- a/lib/go-atscfg/regexremapdotconfig.go
+++ b/lib/go-atscfg/regexremapdotconfig.go
@@ -26,6 +26,8 @@ import (
"github.com/apache/trafficcontrol/lib/go-tc"
)
+const ContentTypeRegexRemapDotConfig = ContentTypeTextASCII
+
type CDNDS struct {
OrgServerFQDN string
QStringIgnore int
diff --git a/lib/go-atscfg/regexrevalidatedotconfig.go b/lib/go-atscfg/regexrevalidatedotconfig.go
index 0624bf6..ad614f9 100644
--- a/lib/go-atscfg/regexrevalidatedotconfig.go
+++ b/lib/go-atscfg/regexrevalidatedotconfig.go
@@ -40,6 +40,8 @@ const DefaultMaxRevalDurationDays = 90
const JobKeywordPurge = "PURGE"
const RegexRevalidateMinTTL = time.Hour
+const ContentTypeRegexRevalidateDotConfig = ContentTypeTextASCII
+
type Job struct {
AssetURL string
PurgeEnd time.Time
diff --git a/lib/go-atscfg/remapdotconfig.go b/lib/go-atscfg/remapdotconfig.go
index 8429b1c..55b056b 100644
--- a/lib/go-atscfg/remapdotconfig.go
+++ b/lib/go-atscfg/remapdotconfig.go
@@ -31,6 +31,7 @@ import (
const CacheURLParameterConfigFile = "cacheurl.config"
const CacheKeyParameterConfigFile = "cachekey.config"
+const ContentTypeRemapDotConfig = ContentTypeTextASCII
type RemapConfigDSData struct {
ID int
diff --git a/lib/go-atscfg/servercachedotconfig_test.go b/lib/go-atscfg/servercachedotconfig_test.go
index 9f812b5..eb5fcb3 100644
--- a/lib/go-atscfg/servercachedotconfig_test.go
+++ b/lib/go-atscfg/servercachedotconfig_test.go
@@ -37,11 +37,11 @@ func TestMakeServerCacheDotConfig(t *testing.T) {
Type: tc.DSTypeHTTP,
},
"ds1": ServerCacheConfigDS{
- OrgServerFQDN: "https://ds1.example.test:42/path",
+ OrgServerFQDN: "https://ds1.example.test:4321/path",
Type: tc.DSTypeDNS,
},
"ds2": ServerCacheConfigDS{
- OrgServerFQDN: "https://ds2.example.test:42",
+ OrgServerFQDN: "https://ds2.example.test:4321",
Type: tc.DSTypeHTTP,
},
"ds3": ServerCacheConfigDS{
@@ -92,7 +92,7 @@ func TestMakeServerCacheDotConfig(t *testing.T) {
"ds6": {},
"ds7": {},
"ds8": {},
- "42": {},
+ "4321": {},
"1234": {},
}
diff --git a/lib/go-atscfg/serverunknown.go b/lib/go-atscfg/serverunknown.go
index 58ba06e..68c3732 100644
--- a/lib/go-atscfg/serverunknown.go
+++ b/lib/go-atscfg/serverunknown.go
@@ -26,6 +26,8 @@ import (
"github.com/apache/trafficcontrol/lib/go-tc"
)
+const ContentTypeServerUnknownConfig = ContentTypeTextASCII
+
func MakeServerUnknown(
serverName tc.CacheName,
serverDomain string,
diff --git a/lib/go-atscfg/setdscpdotconfig.go b/lib/go-atscfg/setdscpdotconfig.go
index c311de1..3f2ee3c 100644
--- a/lib/go-atscfg/setdscpdotconfig.go
+++ b/lib/go-atscfg/setdscpdotconfig.go
@@ -25,6 +25,8 @@ import (
"github.com/apache/trafficcontrol/lib/go-tc"
)
+const ContentTypeSetDSCPDotConfig = ContentTypeTextASCII
+
func MakeSetDSCPDotConfig(
cdnName tc.CDNName,
toToolName string, // tm.toolname global parameter (TODO: cache itself?)
diff --git a/lib/go-atscfg/sslmulticertdotconfig.go b/lib/go-atscfg/sslmulticertdotconfig.go
index 4040885..64cd48b 100644
--- a/lib/go-atscfg/sslmulticertdotconfig.go
+++ b/lib/go-atscfg/sslmulticertdotconfig.go
@@ -26,7 +26,11 @@ import (
"github.com/apache/trafficcontrol/lib/go-tc"
)
+const ContentTypeSSLMultiCertDotConfig = ContentTypeTextASCII
+const SSLMultiCertConfigFileName = `ssl_multicert.config`
+
type SSLMultiCertDS struct {
+ XMLID string
Type tc.DSType
Protocol int
ExampleURLs []string
@@ -56,7 +60,46 @@ func MakeSSLMultiCertDotConfig(
) string {
text := GenericHeaderComment(string(cdnName), toToolName, toURL)
+ dses = GetSSLMultiCertDotConfigDeliveryServices(dses)
+
for dsName, ds := range dses {
+ cerName, keyName := GetSSLMultiCertDotConfigCertAndKeyName(dsName, ds)
+ text += `ssl_cert_name=` + cerName + "\t" + ` ssl_key_name=` + keyName + "\n"
+ }
+ return text
+}
+
+// GetSSLMultiCertDotConfigCertAndKeyName returns the cert file name and key file name for the given delivery service.
+func GetSSLMultiCertDotConfigCertAndKeyName(dsName tc.DeliveryServiceName, ds SSLMultiCertDS) (string, string) {
+ hostName := ds.ExampleURLs[0] // first one is the one we want
+
+ scheme := "https://"
+ if !strings.HasPrefix(hostName, scheme) {
+ scheme = "http://"
+ }
+ newHost := hostName
+ if len(hostName) < len(scheme) {
+ log.Errorln("MakeSSLMultiCertDotConfig got ds '" + string(dsName) + "' example url '" + hostName + "' with no scheme! ssl_multicert.config will likely be malformed!")
+ } else {
+ newHost = hostName[len(scheme):]
+ }
+ keyName := newHost + ".key"
+
+ newHost = strings.Replace(newHost, ".", "_", -1)
+
+ cerName := newHost + "_cert.cer"
+ return cerName, keyName
+}
+
+// GetSSLMultiCertDotConfigDeliveryServices takes a list of delivery services, and returns the delivery services which will be inserted into the config by MakeSSLMultiCertDotConfig.
+// This is public, so users can see which Delivery Services are used, without parsing the config file.
+// For example, this is useful to determine which certificates are needed.
+func GetSSLMultiCertDotConfigDeliveryServices(dses map[tc.DeliveryServiceName]SSLMultiCertDS) map[tc.DeliveryServiceName]SSLMultiCertDS {
+ usedDSes := map[tc.DeliveryServiceName]SSLMultiCertDS{}
+ for dsName, ds := range dses {
+ if ds.Type == tc.DSTypeAnyMap {
+ continue
+ }
if ds.Type.IsSteering() {
continue // Steering delivery service SSLs should not be on the edges.
}
@@ -66,25 +109,7 @@ func MakeSSLMultiCertDotConfig(
if len(ds.ExampleURLs) == 0 {
continue // TODO warn? error? Perl doesn't
}
-
- hostName := ds.ExampleURLs[0] // first one is the one we want
-
- scheme := "https://"
- if !strings.HasPrefix(hostName, scheme) {
- scheme = "http://"
- }
- newHost := hostName
- if len(hostName) < len(scheme) {
- log.Errorln("MakeSSLMultiCertDotConfig got ds '" + string(dsName) + "' example url '" + hostName + "' with no scheme! ssl_multicert.config will likely be malformed!")
- } else {
- newHost = hostName[len(scheme):]
- }
- keyName := newHost + ".key"
-
- newHost = strings.Replace(newHost, ".", "_", -1)
-
- cerName := newHost + "_cert.cer"
- text += `ssl_cert_name=` + cerName + "\t" + ` ssl_key_name=` + keyName + "\n"
+ usedDSes[dsName] = ds
}
- return text
+ return usedDSes
}
diff --git a/lib/go-atscfg/storagedotconfig.go b/lib/go-atscfg/storagedotconfig.go
index ab6f1bd..a78cdff 100644
--- a/lib/go-atscfg/storagedotconfig.go
+++ b/lib/go-atscfg/storagedotconfig.go
@@ -26,6 +26,8 @@ import (
"github.com/apache/trafficcontrol/lib/go-log"
)
+const ContentTypeStorageDotConfig = ContentTypeTextASCII
+
// MakeStorageDotConfig creates storage.config for a given ATS Profile.
// The paramData is the map of parameter names to values, for all parameters assigned to the given profile, with the config_file "storage.config".
func MakeStorageDotConfig(
diff --git a/lib/go-atscfg/sysctldotconf.go b/lib/go-atscfg/sysctldotconf.go
index fa83280..ba3c144 100644
--- a/lib/go-atscfg/sysctldotconf.go
+++ b/lib/go-atscfg/sysctldotconf.go
@@ -21,6 +21,7 @@ package atscfg
const SysctlSeparator = " = "
const SysctlFileName = "sysctl.conf"
+const ContentTypeSysctlDotConf = ContentTypeTextASCII
func MakeSysCtlDotConf(
profileName string,
diff --git a/lib/go-atscfg/unknownconfig.go b/lib/go-atscfg/unknownconfig.go
index b7a85de..255d3b1 100644
--- a/lib/go-atscfg/unknownconfig.go
+++ b/lib/go-atscfg/unknownconfig.go
@@ -23,6 +23,8 @@ import (
"strings"
)
+const ContentTypeUnknownConfig = ContentTypeTextASCII
+
func MakeUnknownConfig(
profileName string,
paramData map[string]string, // GetProfileParamData(tx, profile.ID, AstatsFileName)
diff --git a/lib/go-atscfg/urisigningconfig.go b/lib/go-atscfg/urisigningconfig.go
index a312def..a9db57e 100644
--- a/lib/go-atscfg/urisigningconfig.go
+++ b/lib/go-atscfg/urisigningconfig.go
@@ -19,6 +19,8 @@ package atscfg
* under the License.
*/
+const ContentTypeURISigningDotConfig = `application/json; charset=us-ascii`
+
func MakeURISigningConfig(
uriSigningKeysBts []byte,
) string {
diff --git a/lib/go-atscfg/volumedotconfig.go b/lib/go-atscfg/volumedotconfig.go
index 22f7261..af92fb4 100644
--- a/lib/go-atscfg/volumedotconfig.go
+++ b/lib/go-atscfg/volumedotconfig.go
@@ -23,6 +23,8 @@ import (
"strconv"
)
+const ContentTypeVolumeDotConfig = ContentTypeTextASCII
+
// MakeVolumeDotConfig creates volume.config for a given ATS Profile.
// The paramData is the map of parameter names to values, for all parameters assigned to the given profile, with the config_file "storage.config".
func MakeVolumeDotConfig(
diff --git a/lib/go-tc/stats_summary.go b/lib/go-tc/stats_summary.go
index 0c1e291..cb26769 100644
--- a/lib/go-tc/stats_summary.go
+++ b/lib/go-tc/stats_summary.go
@@ -152,7 +152,14 @@ func (ss *StatsSummaryLastUpdated) UnmarshalJSON(data []byte) error {
return nil
}
-// StatsSummaryLastUpdatedResponse ...
+type LastUpdated struct {
+ Version string `json:"version"`
+ Response struct {
+ SummaryTime string `json:"summaryTime"`
+ } `json:"response"`
+}
+
type StatsSummaryLastUpdatedResponse struct {
+ Version string `json:"version"`
Response StatsSummaryLastUpdated `json:"response"`
}
diff --git a/traffic_ops/client/server.go b/traffic_ops/client/server.go
index 61ebcb8..6127d87 100644
--- a/traffic_ops/client/server.go
+++ b/traffic_ops/client/server.go
@@ -305,3 +305,23 @@ func (to *Session) GetServerIDDeliveryServices(server int) ([]tc.DeliveryService
err = json.NewDecoder(resp.Body).Decode(&data)
return data.Response, reqInf, err
}
+
+// GetServerUpdateStatus GETs the Server Update Status by the Server hostname.
+func (to *Session) GetServerUpdateStatus(hostName string) (tc.ServerUpdateStatus, ReqInf, error) {
+ path := API_SERVERS + `/` + hostName + `/update_status`
+ resp, remoteAddr, err := to.request(http.MethodGet, path, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return tc.ServerUpdateStatus{}, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ data := []tc.ServerUpdateStatus{}
+ if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return tc.ServerUpdateStatus{}, reqInf, err
+ }
+ if len(data) == 0 {
+ return tc.ServerUpdateStatus{}, reqInf, errors.New("Traffic Ops returned no update statuses for that server")
+ }
+ return data[0], reqInf, nil
+}
diff --git a/traffic_ops/ort/README.md b/traffic_ops/ort/README.md
new file mode 100644
index 0000000..62e6192
--- /dev/null
+++ b/traffic_ops/ort/README.md
@@ -0,0 +1,99 @@
+# ORT
+
+ORT, traffic_ops_ort.pl, is used to apply configuration from Traffic Control, stored in Traffic Ops, to the cache.
+
+Typical usage is to install ORT on the cache machine, and then run it periodically via a CRON job.
+
+## Options
+
+ORT has the following command-line options:
+
+
+option | default | description
+----------------------- | ------- | ---
+dispersion | 300 | wait a random number of seconds between 0 and <time> before starting.
+retries | 5 | retry connection to Traffic Ops URL <number> times.
+wait_for_parents | 1 | do not update if parent_pending = 1 in the update json.
+login_dispersion | 0 | wait a random number of seconds between 0 and <time> before login.
+rev_proxy_disable | 0 | bypass the reverse proxy even if one has been configured.
+skip_os_check | 0 | bypass the check for a supported CentOS version.
+override_hostname_short | "" | override the short hostname of the OS for config generation.
+
+# Modes
+
+ORT can be run in a number of modes.
+
+The syncds mode is the normal mode of operation, which should typically be run periodically via cron or a similar tool.
+
+The badass mode is typically an emergency-fix mode, which will override and replace all files with the configuration generated from the current Traffic Ops data, regardless whether ORT (presumably incorrectly) thinks the files need updating or not. It is recommended to run this mode when something goes wrong, and the configuration on the cache is incorrect, and the data in Traffic Ops and config generation is believed to be correct. It is not recommended to run this in normal operation; [...]
+
+The revalidate mode will apply Revalidations from Traffic Ops (regex_revalidate.config) but no other configuration. This mode was intended to quickly apply revalidations when ORT took a long time to run. It is less relevant with ORT's current speed, but may still be useful on slow networks or very large deployments.
+
+mode | description
+------------| ---
+interactive | asks questions during config process
+report | prints config differences and exits
+badass | attempts to fix all config differences that it can
+syncds | syncs delivery services with what is configured in Traffic Ops
+revalidate | checks for updated revalidations in Traffic Ops and applies them
+
+# Behavior
+
+When ORT is run, it will:
+
+1. Delete all of its temporary directories over a week old. Currently, the base temp directory is hard-coded to /tmp/ort.
+1. Determine if Updates have been Queued on the server (by checking the Server's Update Pending or Revalidate Pending flag in Traffic Ops).
+ 1. If Updates were not queued and the script is running in syncds mode (the normal mode), exit.
+1. Get the config files from Traffic Ops, via atstccfg.
+1. Process CentOS Yum packages.
+ 1. These are specified via Parameters on the Server's Profile, with the Config File 'package', where the Parameter Name is the package name, and the Parameter Value is the package version.
+ 1. Uninstall any packages which are installed but whose version does not match.
+ 1. Install all packages in the Server Profile.
+1. Process chkconfig directives.
+ 1. These are specified via Parameters on the Server's Profile, with the Config File 'chkconfig', where the Parameter Name is the package name, and the Parameter Value is the chkconfig directive line.
+ 1. All chkconfig directives in the Server's Profile are applied to the CentOS chkconfig.
+ 1. **NOTE** the default profiles distributed by Traffic Control have an ATS chkconfig with a runlevel before networking is enabled, which is likely incorrect.
+ 1. **NOTE** this is not used by CentOS 7+ and ATS 7+. SystemD does not use chkconfig, and ATS 7+ uses a SystemD script not an init script.
+1. Process each config file
+ 1. If ORT is in revalidate mode, this will only be regex_revalidate.config
+ 1. Perform any special processing. See [Special Processing](#special-processing).
+ 1. If a file exists at the path of the file, load it from disk and compare the two.
+ 1. If there are no changes, don't apply the new file.
+ 1. If there are changes, backup the existing file in the temp directory, and write the new file.
+1. If configuration was changed which requires an ATS reload to apply, perform a service reload of ATS.
+1. If configuration was changed which requires an ATS restart to apply, and ORT is in badass mode, perform a service restart of ATS.
+1. If a sysctl.conf config file was changed, and ORT is in badass mode, run `sysctl -p`.
+1. If a ntpd.conf config file was changed, and ORT is in badass mode, perform a service restart of ntpd.
+1. Update Traffic Ops to unset the Update Pending or Revalidate Pending flag of this Server.
+
+# Special Processing
+
+Certain config files perform extra processing.
+
+## Global replacements
+
+All config files have certain text directives replaced. This is done by the atstccfg config generator before the file is returned to ORT.
+
+* `__SERVER_TCP_PORT__` is replaced with the Server's Port from Traffic Ops; unless the server's port is 80, 0, or null, in which case any occurrences preceded by a colon are removed.
+* `__CACHE_IPV4__` is replaced with the Server's IP address from Traffic Ops.
+* `__HOSTNAME__` is replaced with the Server's (short) HostName from Traffic Ops.
+* `__FULL_HOSTNAME__` is replaced with the Server's HostName, a dot, and the Server's DomainName from Traffic Ops (i.e. the Server's Fully Qualified Domain Name).
+* `__RETURN__` is replaced with a newline character, and any whitespace before or after it is removed.
+
+## remap.config
+
+ORT processes `##OVERRIDE##` directives in the remap.config file.
+
+The ##OVERRIDE## template string allows the Delivery Service Raw Remap Text field to override to fully override the Delivery Service’s line in the remap.config ATS configuration file, generated by Traffic Ops. The end result is the original, generated line commented out, prepended with ##OVERRIDDEN## and the ##OVERRIDE## rule is activated in its place. This behavior is used to incrementally deploy plugins used in this configuration file. Normally, this entails cloning the Delivery Servic [...]
+
+## 50-ats.rules
+
+This is presumed to be a udev file for devices which are block devices to be used as disk storage by ATS.
+
+ORT verifies all devices in the file are owned by the owner listed in the file, and logs errors otherwise.
+
+ORT verifies all devices in the file do not have filesystems. If any device has a filesystem, ORT assumes it was a mistake to assign as an ATS storage device, and logs a fatal error.
+
+# Trivia
+
+ORT stands for "Operational Readiness Test." The acronym is a legacy artifact and does not reflect the current purpose, which is to apply configuration from Traffic Ops.
diff --git a/traffic_ops/ort/atstccfg/atstccfg.go b/traffic_ops/ort/atstccfg/atstccfg.go
index fd0eba7..1c64711 100644
--- a/traffic_ops/ort/atstccfg/atstccfg.go
+++ b/traffic_ops/ort/atstccfg/atstccfg.go
@@ -49,13 +49,13 @@ package main
import (
"fmt"
- "net/http"
"os"
- "strconv"
"strings"
"github.com/apache/trafficcontrol/lib/go-log"
+ "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/cfgfile"
"github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
+ "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/getdata"
"github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/plugin"
"github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
@@ -72,67 +72,46 @@ func main() {
os.Exit(0)
}
- if cfg.PrintGeneratedFiles {
- fmt.Println(strings.Join(GetGeneratedFilesList(), "\n"))
- os.Exit(config.ExitCodeSuccess)
- }
-
plugins := plugin.Get(cfg)
plugins.OnStartup(plugin.StartupData{Cfg: cfg})
- log.Infoln("URL: '" + cfg.TOURL.String() + "' User: '" + cfg.TOUser + "' Pass len: '" + strconv.Itoa(len(cfg.TOPass)) + "'")
- log.Infoln("TempDir: '" + cfg.TempDir + "'")
-
- toFQDN := cfg.TOURL.Scheme + "://" + cfg.TOURL.Host
- log.Infoln("TO FQDN: '" + toFQDN + "'")
- log.Infoln("TO URL: '" + cfg.TOURL.String() + "'")
-
- toClient, err := toreq.GetClient(toFQDN, cfg.TOUser, cfg.TOPass, cfg.TempDir, cfg.CacheFileMaxAge, cfg.TOTimeout, cfg.TOInsecure)
+ tccfg, err := toreq.GetTCCfg(cfg)
if err != nil {
- log.Errorln("Logging in to Traffic Ops: " + err.Error())
+ log.Errorln(err)
os.Exit(config.ExitCodeErrGeneric)
}
- tccfg := config.TCCfg{Cfg: cfg, TOClient: &toClient}
-
onReqData := plugin.OnRequestData{Cfg: tccfg}
if handled := plugins.OnRequest(onReqData); handled {
return
}
- cfgFile, code, err := GetConfigFile(tccfg)
- log.Infof("GetConfigFile returned %v %v\n", code, err)
- if err != nil {
- log.Errorln("Getting config file '" + cfg.TOURL.String() + "': " + err.Error())
- if code == 0 {
- code = config.ExitCodeErrGeneric
+ if tccfg.GetData != "" {
+ if err := getdata.WriteData(tccfg); err != nil {
+ log.Errorln("writing data: " + err.Error())
+ os.Exit(config.ExitCodeErrGeneric)
}
- log.Infof("GetConfigFile exiting with code %v\n", code)
- os.Exit(code)
+ os.Exit(config.ExitCodeSuccess)
}
- fmt.Println(cfgFile)
- os.Exit(config.ExitCodeSuccess)
-}
-func GetGeneratedFilesList() []string {
- names := []string{}
- for scope, fileFuncs := range ConfigFileFuncs() {
- for cfgFile, _ := range fileFuncs {
- names = append(names, scope+"/"+cfgFile)
+ if tccfg.SetRevalStatus != "" || tccfg.SetQueueStatus != "" {
+ if err := getdata.SetQueueRevalStatuses(tccfg); err != nil {
+ log.Errorln("writing queue and reval statuses: " + err.Error())
+ os.Exit(config.ExitCodeErrGeneric)
}
+ os.Exit(config.ExitCodeSuccess)
}
- names = append(names, "profiles/url_sig_*.config") // url_sig configs are generated, but not in the funcs because they're not a literal match
- names = append(names, "profiles/uri_signing_*.config") // uri_signing configs are generated, but not in the funcs because they're not a literal match
- names = append(names, "profiles/*") // unknown profiles configs are generated, a.k.a. "take-and-bake"
-
- return names
-}
+ configs, err := cfgfile.GetAllConfigs(tccfg)
+ if err != nil {
+ log.Errorln("Getting config for'" + cfg.CacheHostName + "': " + err.Error())
+ os.Exit(config.ExitCodeErrGeneric)
+ }
-func HTTPCodeToExitCode(httpCode int) int {
- switch httpCode {
- case http.StatusNotFound:
- return config.ExitCodeNotFound
+ if err := cfgfile.WriteConfigs(configs, os.Stdout); err != nil {
+ log.Errorln("Writing configs for '" + cfg.CacheHostName + "': " + err.Error())
+ os.Exit(config.ExitCodeErrGeneric)
}
- return config.ExitCodeErrGeneric
+
+ os.Exit(config.ExitCodeSuccess)
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/all.go b/traffic_ops/ort/atstccfg/cfgfile/all.go
new file mode 100644
index 0000000..84b43c3
--- /dev/null
+++ b/traffic_ops/ort/atstccfg/cfgfile/all.go
@@ -0,0 +1,134 @@
+package cfgfile
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import (
+ "errors"
+ "io"
+ "math/rand"
+ "mime/multipart"
+ "path/filepath"
+ "regexp"
+ "strconv"
+ "strings"
+
+ "github.com/apache/trafficcontrol/lib/go-atscfg"
+ "github.com/apache/trafficcontrol/lib/go-rfc"
+ "github.com/apache/trafficcontrol/lib/go-tc"
+ "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
+)
+
+// GetAllConfigs gets all config files for cfg.CacheHostName.
+func GetAllConfigs(cfg config.TCCfg) ([]ATSConfigFile, error) {
+ toData, err := GetTOData(cfg)
+ if err != nil {
+ return nil, errors.New("getting data from traffic ops: " + err.Error())
+ }
+
+ meta, err := GetMeta(toData)
+ if err != nil {
+ return nil, errors.New("creating meta: " + err.Error())
+ }
+
+ hasSSLMultiCertConfig := false
+ configs := []ATSConfigFile{}
+ for _, fi := range meta.ConfigFiles {
+ if cfg.RevalOnly && fi.FileNameOnDisk != atscfg.RegexRevalidateFileName {
+ continue
+ }
+ txt, contentType, err := GetConfigFile(toData, fi)
+ if err != nil {
+ return nil, errors.New("getting config file '" + fi.APIURI + "': " + err.Error())
+ }
+ if fi.FileNameOnDisk == atscfg.SSLMultiCertConfigFileName {
+ hasSSLMultiCertConfig = true
+ }
+ txt = PreprocessConfigFile(toData.Server, txt)
+ configs = append(configs, ATSConfigFile{ATSConfigMetaDataConfigFile: fi, Text: txt, ContentType: contentType})
+ }
+
+ if hasSSLMultiCertConfig {
+ sslConfigs, err := GetSSLCertsAndKeyFiles(toData)
+ if err != nil {
+ return nil, errors.New("getting ssl key and cert config files: " + err.Error())
+ }
+ configs = append(configs, sslConfigs...)
+ }
+
+ return configs, nil
+}
+
+const HdrConfigFilePath = "Path"
+
+// WriteConfigs writes the given configs as a RFC2046§5.1 MIME multipart/mixed message.
+func WriteConfigs(configs []ATSConfigFile, output io.Writer) error {
+ w := multipart.NewWriter(output)
+
+ // Create a unique boundary. Because we're using a text encoding, we need to make sure the boundary text doesn't occur in any body.
+ boundary := w.Boundary()
+ randSet := `abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ`
+ for _, cfg := range configs {
+ for strings.Contains(cfg.Text, boundary) {
+ boundary += string(randSet[rand.Intn(len(randSet))])
+ }
+ }
+ if err := w.SetBoundary(boundary); err != nil {
+ return errors.New("setting multipart writer boundary '" + boundary + "': " + err.Error())
+ }
+
+ io.WriteString(output, `MIME-Version: 1.0`+"\r\n"+`Content-Type: multipart/mixed; boundary="`+boundary+`"`+"\r\n\r\n")
+
+ for _, cfg := range configs {
+ hdr := map[string][]string{
+ rfc.ContentType: {cfg.ContentType},
+ HdrConfigFilePath: {filepath.Join(cfg.Location, cfg.FileNameOnDisk)},
+ }
+ partW, err := w.CreatePart(hdr)
+ if err != nil {
+ return errors.New("creating multipart part for config file '" + cfg.FileNameOnDisk + "': " + err.Error())
+ }
+ if _, err := io.WriteString(partW, cfg.Text); err != nil {
+ return errors.New("writing to multipart part for config file '" + cfg.FileNameOnDisk + "': " + err.Error())
+ }
+ }
+
+ if err := w.Close(); err != nil {
+ return errors.New("closing multipart writer and writing final boundary: " + err.Error())
+ }
+ return nil
+}
+
+var returnRegex = regexp.MustCompile(`\s*__RETURN__\s*`)
+
+// PreprocessConfigFile does global preprocessing on the given config file cfgFile.
+// This is mostly string replacements of __X__ directives. See the code for the full list of replacements.
+// These things were formerly done by ORT, but need to be processed by atstccfg now, because ORT no longer has the metadata necessary.
+func PreprocessConfigFile(server tc.Server, cfgFile string) string {
+ if server.TCPPort != 80 && server.TCPPort != 0 {
+ cfgFile = strings.Replace(cfgFile, `__SERVER_TCP_PORT__`, strconv.Itoa(server.TCPPort), -1)
+ } else {
+ cfgFile = strings.Replace(cfgFile, `:__SERVER_TCP_PORT__`, ``, -1)
+ }
+ cfgFile = strings.Replace(cfgFile, `__CACHE_IPV4__`, server.IPAddress, -1)
+ cfgFile = strings.Replace(cfgFile, `__HOSTNAME__`, server.HostName, -1)
+ cfgFile = strings.Replace(cfgFile, `__FULL_HOSTNAME__`, server.HostName+`.`+server.DomainName, -1)
+ cfgFile = returnRegex.ReplaceAllString(cfgFile, "\n")
+ return cfgFile
+}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/astatsdotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/astatsdotconfig.go
index 3f6a521..4a5757d 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/astatsdotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/astatsdotconfig.go
@@ -20,37 +20,13 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileProfileAstatsDotConfig(cfg config.TCCfg, profileNameOrID string) (string, error) {
- profileName, err := toreq.GetProfileNameFromProfileNameOrID(cfg, profileNameOrID)
- if err != nil {
- return "", errors.New("getting profile name from '" + profileNameOrID + "': " + err.Error())
- }
-
- profileParameters, err := toreq.GetProfileParameters(cfg, profileName)
- if err != nil {
- return "", errors.New("getting profile '" + profileName + "' parameters: " + err.Error())
- }
- if len(profileParameters) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params (and there should always be storage.config params), to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
+func GetConfigFileProfileAstatsDotConfig(toData *TOData) (string, string, error) {
paramData := map[string]string{}
// TODO add configFile query param to profile/parameters endpoint, to only get needed data
- for _, param := range profileParameters {
+ for _, param := range toData.ServerParams {
if param.ConfigFile != atscfg.AstatsFileName {
continue
}
@@ -60,5 +36,5 @@ func GetConfigFileProfileAstatsDotConfig(cfg config.TCCfg, profileNameOrID strin
paramData[param.Name] = param.Value
}
- return atscfg.MakeAStatsDotConfig(profileName, paramData, toToolName, toURL), nil
+ return atscfg.MakeAStatsDotConfig(toData.Server.Profile, paramData, toData.TOToolName, toData.TOURL), atscfg.ContentTypeAstatsDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/atsdotrules.go b/traffic_ops/ort/atstccfg/cfgfile/atsdotrules.go
index 775b173..f70b83f 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/atsdotrules.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/atsdotrules.go
@@ -20,39 +20,15 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
const ATSDotRulesFileName = StorageFileName
-func GetConfigFileProfileATSDotRules(cfg config.TCCfg, profileNameOrID string) (string, error) {
- profileName, err := toreq.GetProfileNameFromProfileNameOrID(cfg, profileNameOrID)
- if err != nil {
- return "", errors.New("getting profile name from '" + profileNameOrID + "': " + err.Error())
- }
-
- profileParameters, err := toreq.GetProfileParameters(cfg, profileName)
- if err != nil {
- return "", errors.New("getting profile '" + profileName + "' parameters: " + err.Error())
- }
- if len(profileParameters) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params (and there should always be storage.config params), to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
+func GetConfigFileProfileATSDotRules(toData *TOData) (string, string, error) {
paramData := map[string]string{}
// TODO add configFile query param to profile/parameters endpoint, to only get needed data
- for _, param := range profileParameters {
+ for _, param := range toData.ServerParams {
if param.ConfigFile != ATSDotRulesFileName {
continue
}
@@ -61,6 +37,5 @@ func GetConfigFileProfileATSDotRules(cfg config.TCCfg, profileNameOrID string) (
}
paramData[param.Name] = param.Value
}
-
- return atscfg.MakeATSDotRules(profileName, paramData, toToolName, toURL), nil
+ return atscfg.MakeATSDotRules(toData.Server.Profile, paramData, toData.TOToolName, toData.TOURL), atscfg.ContentTypeATSDotRules, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/bgfetchdotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/bgfetchdotconfig.go
index afce617..432c246 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/bgfetchdotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/bgfetchdotconfig.go
@@ -20,22 +20,10 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
+ "github.com/apache/trafficcontrol/lib/go-tc"
)
-func GetConfigFileCDNBGFetchDotConfig(cfg config.TCCfg, cdnNameOrID string) (string, error) {
- cdnName, err := toreq.GetCDNNameFromCDNNameOrID(cfg, cdnNameOrID)
- if err != nil {
- return "", errors.New("getting CDN name from '" + cdnNameOrID + "': " + err.Error())
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
- return atscfg.MakeBGFetchDotConfig(cdnName, toToolName, toURL), nil
+func GetConfigFileCDNBGFetchDotConfig(toData *TOData) (string, string, error) {
+ return atscfg.MakeBGFetchDotConfig(tc.CDNName(toData.Server.CDNName), toData.TOToolName, toData.TOURL), atscfg.ContentTypeBGFetchDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/cachedotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/cachedotconfig.go
index 96b57bd..a256846 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/cachedotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/cachedotconfig.go
@@ -20,56 +20,20 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
"github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileProfileCacheDotConfig(cfg config.TCCfg, profileNameOrID string) (string, error) {
- profileName, err := toreq.GetProfileNameFromProfileNameOrID(cfg, profileNameOrID)
- if err != nil {
- return "", errors.New("getting profile name from '" + profileNameOrID + "': " + err.Error())
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- servers, err := toreq.GetServers(cfg)
- if err != nil {
- return "", errors.New("getting servers: " + err.Error())
- }
-
- profileServerIDs := []int{}
+func GetConfigFileProfileCacheDotConfig(toData *TOData) (string, string, error) {
profileServerIDsMap := map[int]struct{}{}
- profileServers := []tc.Server{}
- for _, sv := range servers {
- if sv.Profile != profileName {
+ for _, sv := range toData.Servers {
+ if sv.Profile != toData.Server.Profile {
continue
}
- profileServers = append(profileServers, sv)
- profileServerIDs = append(profileServerIDs, sv.ID)
profileServerIDsMap[sv.ID] = struct{}{}
}
- dsServers, err := toreq.GetDeliveryServiceServers(cfg, nil, profileServerIDs)
- if err != nil {
- return "", errors.New("getting parent.config cachegroup parent server delivery service servers: " + err.Error())
- }
-
- profile, err := toreq.GetProfileByName(cfg, profileName)
- if err != nil {
- return "", errors.New("getting profile '" + profileNameOrID + "': " + err.Error())
- }
-
- dses, err := toreq.GetCDNDeliveryServices(cfg, profile.CDNID)
- if err != nil {
- return "", errors.New("getting delivery services: " + err.Error())
- }
+ dsServers := FilterDSS(toData.DeliveryServiceServers, nil, profileServerIDsMap)
dsIDs := map[int]struct{}{}
for _, dss := range dsServers {
@@ -83,7 +47,7 @@ func GetConfigFileProfileCacheDotConfig(cfg config.TCCfg, profileNameOrID string
}
profileDSes := []atscfg.ProfileDS{}
- for _, ds := range dses {
+ for _, ds := range toData.DeliveryServices {
if ds.ID == nil || ds.Type == nil || ds.OrgServerFQDN == nil {
continue // TODO warn? err?
}
@@ -99,5 +63,6 @@ func GetConfigFileProfileCacheDotConfig(cfg config.TCCfg, profileNameOrID string
origin := *ds.OrgServerFQDN
profileDSes = append(profileDSes, atscfg.ProfileDS{Type: *ds.Type, OriginFQDN: &origin})
}
- return atscfg.MakeCacheDotConfig(profileName, profileDSes, toToolName, toURL), nil
+
+ return atscfg.MakeCacheDotConfig(toData.Server.Profile, profileDSes, toData.TOToolName, toData.TOURL), atscfg.ContentTypeCacheDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/cacheurldotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/cacheurldotconfig.go
index db96037..2459c18 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/cacheurldotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/cacheurldotconfig.go
@@ -20,46 +20,19 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
"github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileCDNCacheURL(cfg config.TCCfg, cdnNameOrID string, fileName string) (string, error) {
- cdnName, err := toreq.GetCDNNameFromCDNNameOrID(cfg, cdnNameOrID)
- if err != nil {
- return "", errors.New("getting CDN name from '" + cdnNameOrID + "': " + err.Error())
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- cdn, err := toreq.GetCDN(cfg, cdnName)
- if err != nil {
- return "", errors.New("getting cdn '" + string(cdnName) + "': " + err.Error())
- }
-
- dses, err := toreq.GetCDNDeliveryServices(cfg, cdn.ID)
- if err != nil {
- return "", errors.New("getting delivery services: " + err.Error())
- }
-
- dsIDs := []int{}
- for _, ds := range dses {
+func GetConfigFileCDNCacheURL(toData *TOData, fileName string) (string, string, error) {
+ dsIDs := map[int]struct{}{}
+ for _, ds := range toData.DeliveryServices {
if ds.ID != nil {
- dsIDs = append(dsIDs, *ds.ID)
+ dsIDs[*ds.ID] = struct{}{}
}
}
- dss, err := toreq.GetDeliveryServiceServers(cfg, dsIDs, nil)
- if err != nil {
- return "", errors.New("getting delivery service servers: " + err.Error())
- }
+ dss := FilterDSS(toData.DeliveryServiceServers, dsIDs, nil)
dssMap := map[int][]int{} // map[dsID]serverID
for _, dss := range dss {
@@ -70,7 +43,7 @@ func GetConfigFileCDNCacheURL(cfg config.TCCfg, cdnNameOrID string, fileName str
}
dsesWithServers := []tc.DeliveryServiceNullable{}
- for _, ds := range dses {
+ for _, ds := range toData.DeliveryServices {
if ds.ID == nil {
continue // TODO warn
}
@@ -86,10 +59,9 @@ func GetConfigFileCDNCacheURL(cfg config.TCCfg, cdnNameOrID string, fileName str
cfgDSes := atscfg.DeliveryServicesToCacheURLDSes(dsesWithServers)
- txt := atscfg.MakeCacheURLDotConfig(cdnName, toToolName, toURL, fileName, cfgDSes)
- return txt, nil
+ return atscfg.MakeCacheURLDotConfig(tc.CDNName(toData.Server.CDNName), toData.TOToolName, toData.TOURL, fileName, cfgDSes), atscfg.ContentTypeCacheURLDotConfig, nil
}
-func GetConfigFileCDNCacheURLPlain(cfg config.TCCfg, cdnNameOrID string) (string, error) {
- return GetConfigFileCDNCacheURL(cfg, cdnNameOrID, "cacheurl.config")
+func GetConfigFileCDNCacheURLPlain(toData *TOData) (string, string, error) {
+ return GetConfigFileCDNCacheURL(toData, "cacheurl.config")
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/cfgfile.go b/traffic_ops/ort/atstccfg/cfgfile/cfgfile.go
new file mode 100644
index 0000000..57fad0f
--- /dev/null
+++ b/traffic_ops/ort/atstccfg/cfgfile/cfgfile.go
@@ -0,0 +1,485 @@
+package cfgfile
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import (
+ "encoding/json"
+ "errors"
+ "strconv"
+ "strings"
+ "time"
+
+ "github.com/apache/trafficcontrol/lib/go-atscfg"
+ "github.com/apache/trafficcontrol/lib/go-log"
+ "github.com/apache/trafficcontrol/lib/go-tc"
+ "github.com/apache/trafficcontrol/lib/go-util"
+ "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
+ "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
+)
+
+// TOData is the Traffic Ops data needed to generate configs.
+// See each field for details on the data required.
+// - If a field says 'must', the creation of TOData is guaranteed to do so, and users of the struct may rely on that.
+// - If it says 'may', the creation may or may not do so, and therefore users of the struct must filter if they
+// require the potential fields to be omitted to generate correctly.
+type TOData struct {
+ // Servers must be all the servers from Traffic Ops. May include servers not on the current cdn.
+ Servers []tc.Server
+
+ // CacheGroups must be all cachegroups in Traffic Ops with Servers on the current server's cdn. May also include CacheGroups without servers on the current cdn.
+ CacheGroups []tc.CacheGroupNullable
+
+ // GlobalParams must be all Parameters in Traffic Ops on the tc.GlobalProfileName Profile. Must not include other parameters.
+ GlobalParams []tc.Parameter
+
+ // ScopeParams must be all Parameters in Traffic Ops with the name "scope". Must not include other Parameters.
+ ScopeParams []tc.Parameter
+
+ // ServerParams must be all Parameters on the Profile of the current server. Must not include other Parameters.
+ ServerParams []tc.Parameter
+
+ // CacheKeyParams must be all Parameters with the ConfigFile atscfg.CacheKeyParameterConfigFile.
+ CacheKeyParams []tc.Parameter
+
+ // ParentConfigParams must be all Parameters with the ConfigFile "parent.config.
+ ParentConfigParams []tc.Parameter
+
+ // DeliveryServices must include all Delivery Services on the current server's cdn, including those not assigned to the server. Must not include delivery services on other cdns.
+ DeliveryServices []tc.DeliveryServiceNullable
+
+ // DeliveryServiceServers must include all delivery service servers in Traffic Ops for all delivery services on the current cdn, including those not assigned to the current server.
+ DeliveryServiceServers []tc.DeliveryServiceServer
+
+ // Server must be the server we're fetching configs from
+ Server tc.Server
+
+ // TOToolName must be the Parameter named 'tm.toolname' on the tc.GlobalConfigFileName Profile.
+ TOToolName string
+
+ // TOToolName must be the Parameter named 'tm.url' on the tc.GlobalConfigFileName Profile.
+ TOURL string
+
+ // Jobs must be all Jobs on the server's CDN. May include jobs on other CDNs.
+ Jobs []tc.Job
+
+ // CDN must be the CDN of the server.
+ CDN tc.CDN
+
+ // DeliveryServiceRegexes must be all regexes on all delivery services on this server's cdn.
+ DeliveryServiceRegexes []tc.DeliveryServiceRegexes
+
+ // Profile must be the Profile of the server being requested.
+ Profile tc.Profile
+
+ // URISigningKeys must be a map of every delivery service which is URI Signed, to its keys.
+ URISigningKeys map[tc.DeliveryServiceName][]byte
+
+ // URLSigKeys must be a map of every delivery service which uses URL Sig, to its keys.
+ URLSigKeys map[tc.DeliveryServiceName]tc.URLSigKeys
+
+ // ServerCapabilities must be a map of all server IDs on this server's CDN, to a set of their capabilities. May also include servers from other cdns.
+ ServerCapabilities map[int]map[atscfg.ServerCapability]struct{}
+
+ // DSRequiredCapabilities must be a map of all delivery service IDs on this server's CDN, to a set of their required capabilities. Delivery Services with no required capabilities may not have an entry in the map.
+ DSRequiredCapabilities map[int]map[atscfg.ServerCapability]struct{}
+
+ // SSLKeys must be all the ssl keys for the server's cdn.
+ SSLKeys []tc.CDNSSLKeys
+}
+
+// TODO: validate all "profile scope" files are the server's profile.
+// If they ever weren't, we'll send bad data, because we're only getting the server's profile data.
+// Getting all data for all profiles in TOData isn't reasonable.
+// TODO info log profile name, cdn name (ORT was logging, and doesn't get that data anymore, so we need to log here)
+func GetTOData(cfg config.TCCfg) (*TOData, error) {
+ start := time.Now()
+ defer func() { log.Infof("GetTOData took %v\n", time.Since(start)) }()
+
+ toData := &TOData{}
+
+ serversF := func() error {
+ defer func(start time.Time) { log.Infof("serversF took %v\n", time.Since(start)) }(time.Now())
+ // TODO TOAPI add /servers?cdn=1 query param
+ servers, err := toreq.GetServers(cfg)
+ if err != nil {
+ return errors.New("getting servers: " + err.Error())
+ }
+
+ toData.Servers = servers
+
+ server := tc.Server{ID: atscfg.InvalidID}
+ for _, toServer := range servers {
+ if toServer.HostName == cfg.CacheHostName {
+ server = toServer
+ break
+ }
+ }
+ if server.ID == atscfg.InvalidID {
+ return errors.New("server '" + cfg.CacheHostName + " not found in servers")
+ }
+
+ toData.Server = server
+
+ sslF := func() error {
+ defer func(start time.Time) { log.Infof("sslF took %v\n", time.Since(start)) }(time.Now())
+ keys, err := toreq.GetCDNSSLKeys(cfg, tc.CDNName(server.CDNName))
+ if err != nil {
+ return errors.New("getting cdn '" + server.CDNName + "': " + err.Error())
+ }
+ toData.SSLKeys = keys
+ return nil
+ }
+ dsF := func() error {
+ defer func(start time.Time) { log.Infof("dsF took %v\n", time.Since(start)) }(time.Now())
+ dses, err := toreq.GetCDNDeliveryServices(cfg, server.CDNID)
+ if err != nil {
+ return errors.New("getting delivery services: " + err.Error())
+ }
+ toData.DeliveryServices = dses
+
+ uriSignKeysF := func() error {
+ defer func(start time.Time) { log.Infof("uriF took %v\n", time.Since(start)) }(time.Now())
+ uriSigningKeys := map[tc.DeliveryServiceName][]byte{}
+ for _, ds := range dses {
+ if ds.XMLID == nil {
+ continue // TODO warn?
+ }
+ // TODO read meta config gen, and only include servers which are included in the meta (assigned to edge or all for mids? read the meta gen to find out)
+ if ds.SigningAlgorithm == nil || *ds.SigningAlgorithm != tc.SigningAlgorithmURISigning {
+ continue
+ }
+ keys, err := toreq.GetURISigningKeys(cfg, *ds.XMLID)
+ if err != nil {
+ if strings.Contains(strings.ToLower(err.Error()), "not found") {
+ log.Errorln("Delivery service '" + *ds.XMLID + "' is uri_signing, but keys were not found! Skipping!")
+ continue
+ } else {
+ return errors.New("getting uri signing keys for ds '" + *ds.XMLID + "': " + err.Error())
+ }
+ }
+ uriSigningKeys[tc.DeliveryServiceName(*ds.XMLID)] = keys
+ }
+ toData.URISigningKeys = uriSigningKeys
+ return nil
+ }
+
+ urlSigKeysF := func() error {
+ defer func(start time.Time) { log.Infof("urlF took %v\n", time.Since(start)) }(time.Now())
+ urlSigKeys := map[tc.DeliveryServiceName]tc.URLSigKeys{}
+ for _, ds := range dses {
+ if ds.XMLID == nil {
+ continue // TODO warn?
+ }
+ // TODO read meta config gen, and only include servers which are included in the meta (assigned to edge or all for mids? read the meta gen to find out)
+ if ds.SigningAlgorithm == nil || *ds.SigningAlgorithm != tc.SigningAlgorithmURLSig {
+ continue
+ }
+ keys, err := toreq.GetURLSigKeys(cfg, *ds.XMLID)
+ if err != nil {
+ if strings.Contains(strings.ToLower(err.Error()), "not found") {
+ log.Errorln("Delivery service '" + *ds.XMLID + "' is url_sig, but keys were not found! Skipping!: " + err.Error())
+ continue
+ } else {
+ return errors.New("getting url sig keys for ds '" + *ds.XMLID + "': " + err.Error())
+ }
+ }
+ urlSigKeys[tc.DeliveryServiceName(*ds.XMLID)] = keys
+ }
+ toData.URLSigKeys = urlSigKeys
+ return nil
+ }
+
+ fs := []func() error{}
+ if !cfg.RevalOnly {
+ fs = append([]func() error{uriSignKeysF, urlSigKeysF}, fs...) // skip keys for reval-only, which doesn't need them
+ }
+ return util.JoinErrs(runParallel(fs))
+ }
+ serverParamsF := func() error {
+ defer func(start time.Time) { log.Infof("serverParamsF took %v\n", time.Since(start)) }(time.Now())
+ params, err := toreq.GetServerProfileParameters(cfg, server.Profile)
+ if err != nil {
+ return errors.New("getting server profile '" + server.Profile + "' parameters: " + err.Error())
+ } else if len(params) == 0 {
+ return errors.New("getting server profile '" + server.Profile + "' parameters: no parameters (profile not found?)")
+ }
+ toData.ServerParams = params
+ return nil
+ }
+ cdnF := func() error {
+ defer func(start time.Time) { log.Infof("cdnF took %v\n", time.Since(start)) }(time.Now())
+ cdn, err := toreq.GetCDN(cfg, tc.CDNName(server.CDNName))
+ if err != nil {
+ return errors.New("getting cdn '" + server.CDNName + "': " + err.Error())
+ }
+ toData.CDN = cdn
+ return nil
+ }
+ profileF := func() error {
+ defer func(start time.Time) { log.Infof("profileF took %v\n", time.Since(start)) }(time.Now())
+ profile, err := toreq.GetProfileByName(cfg, server.Profile)
+ if err != nil {
+ return errors.New("getting profile '" + server.Profile + "': " + err.Error())
+ }
+ toData.Profile = profile
+ return nil
+ }
+ fs := []func() error{dsF, serverParamsF, cdnF, profileF}
+ if !cfg.RevalOnly {
+ fs = append([]func() error{sslF}, fs...) // skip ssl keys for reval only, which doesn't need them
+ }
+ return util.JoinErrs(runParallel(fs))
+ }
+
+ cgF := func() error {
+ defer func(start time.Time) { log.Infof("cfF took %v\n", time.Since(start)) }(time.Now())
+ cacheGroups, err := toreq.GetCacheGroups(cfg)
+ if err != nil {
+ return errors.New("getting cachegroups: " + err.Error())
+ }
+ toData.CacheGroups = cacheGroups
+ return nil
+ }
+ globalParamsF := func() error {
+ defer func(start time.Time) { log.Infof("globalParamsF took %v\n", time.Since(start)) }(time.Now())
+ globalParams, err := toreq.GetGlobalParameters(cfg)
+ if err != nil {
+ return errors.New("getting global parameters: " + err.Error())
+ }
+ toData.GlobalParams = globalParams
+ toData.TOToolName, toData.TOURL = toreq.GetTOToolNameAndURL(globalParams)
+ return nil
+ }
+ scopeParamsF := func() error {
+ defer func(start time.Time) { log.Infof("scopeParamsF took %v\n", time.Since(start)) }(time.Now())
+ scopeParams, err := toreq.GetParametersByName(cfg, "scope")
+ if err != nil {
+ return errors.New("getting scope parameters: " + err.Error())
+ }
+ toData.ScopeParams = scopeParams
+ return nil
+ }
+ dssF := func() error {
+ defer func(start time.Time) { log.Infof("dssF took %v\n", time.Since(start)) }(time.Now())
+ dss, err := toreq.GetDeliveryServiceServers(cfg, nil, nil)
+ if err != nil {
+ return errors.New("getting delivery service servers: " + err.Error())
+ }
+ toData.DeliveryServiceServers = dss
+ return nil
+ }
+ jobsF := func() error {
+ defer func(start time.Time) { log.Infof("jobsF took %v\n", time.Since(start)) }(time.Now())
+ jobs, err := toreq.GetJobs(cfg) // TODO add cdn query param to jobs endpoint
+ if err != nil {
+ return errors.New("getting jobs: " + err.Error())
+ }
+ toData.Jobs = jobs
+ return nil
+ }
+ capsF := func() error {
+ defer func(start time.Time) { log.Infof("capsF took %v\n", time.Since(start)) }(time.Now())
+ caps, err := toreq.GetServerCapabilitiesByID(cfg, nil) // TODO change to not take a param; it doesn't use it to request TO anyway.
+ if err != nil {
+ log.Errorln("Server Capabilities error, skipping!")
+ // return errors.New("getting server caps from Traffic Ops: " + err.Error())
+ } else {
+ toData.ServerCapabilities = caps
+ }
+ return nil
+ }
+ dsCapsF := func() error {
+ defer func(start time.Time) { log.Infof("dscapsF took %v\n", time.Since(start)) }(time.Now())
+ caps, err := toreq.GetDeliveryServiceRequiredCapabilitiesByID(cfg, nil)
+ if err != nil {
+ log.Errorln("DS Required Capabilities error, skipping!")
+ // return errors.New("getting DS required capabilities: " + err.Error())
+ } else {
+ toData.DSRequiredCapabilities = caps
+ }
+ return nil
+ }
+ dsrF := func() error {
+ defer func(start time.Time) { log.Infof("dsrF took %v\n", time.Since(start)) }(time.Now())
+ dsr, err := toreq.GetDeliveryServiceRegexes(cfg)
+ if err != nil {
+ return errors.New("getting delivery service regexes: " + err.Error())
+ }
+ toData.DeliveryServiceRegexes = dsr
+ return nil
+ }
+ cacheKeyParamsF := func() error {
+ defer func(start time.Time) { log.Infof("cacheKeyParamsF took %v\n", time.Since(start)) }(time.Now())
+ params, err := toreq.GetConfigFileParameters(cfg, atscfg.CacheKeyParameterConfigFile)
+ if err != nil {
+ return errors.New("getting cache key parameters: " + err.Error())
+ }
+ toData.CacheKeyParams = params
+ return nil
+ }
+ parentConfigParamsF := func() error {
+ defer func(start time.Time) { log.Infof("parentConfigParamsF took %v\n", time.Since(start)) }(time.Now())
+ parentConfigParams, err := toreq.GetConfigFileParameters(cfg, "parent.config") // TODO make const in lib/go-atscfg
+ if err != nil {
+ return errors.New("getting parent.config parameters: " + err.Error())
+ }
+ toData.ParentConfigParams = parentConfigParams
+ return nil
+ }
+
+ fs := []func() error{dssF, serversF, cgF, globalParamsF, scopeParamsF, jobsF}
+ if !cfg.RevalOnly {
+ // skip data not needed for reval, if we're reval-only
+ fs = append([]func() error{dsrF, cacheKeyParamsF, parentConfigParamsF, capsF, dsCapsF}, fs...)
+ }
+ errs := runParallel(fs)
+ return toData, util.JoinErrs(errs)
+}
+
+// runParallel runs all funcs in fs in parallel goroutines, and returns after all funcs have returned.
+// Returns a slice of the errors returned by each func. The order of the errors will not be the same as the order of fs.
+// All funcs in fs must be safe to run in parallel.
+func runParallel(fs []func() error) []error {
+ errs := []error{}
+ doneChan := make(chan error, len(fs))
+ for _, fPtr := range fs {
+ f := fPtr // because functions are pointers, f will change in the loop. Need create a new variable here to close around.
+ go func() { doneChan <- f() }()
+ }
+ for i := 0; i < len(fs); i++ {
+ errs = append(errs, <-doneChan)
+ }
+ return errs
+}
+
+func FilterDSS(dsses []tc.DeliveryServiceServer, dsIDs map[int]struct{}, serverIDs map[int]struct{}) []tc.DeliveryServiceServer {
+ // TODO filter only DSes on this server's CDN? Does anything ever needs DSS cross-CDN? Surely not.
+ // Then, we can remove a bunch of config files that filter only DSes on the current cdn.
+ filtered := []tc.DeliveryServiceServer{}
+ for _, dss := range dsses {
+ if dss.Server == nil || dss.DeliveryService == nil {
+ continue // TODO warn?
+ }
+ if len(dsIDs) > 0 {
+ if _, ok := dsIDs[*dss.DeliveryService]; !ok {
+ continue
+ }
+ }
+ if len(serverIDs) > 0 {
+ if _, ok := serverIDs[*dss.Server]; !ok {
+ continue
+ }
+ }
+ filtered = append(filtered, dss)
+ }
+ return filtered
+}
+
+// TCParamsToParamsWithProfiles unmarshals the Profiles that the tc struct doesn't.
+func TCParamsToParamsWithProfiles(tcParams []tc.Parameter) ([]ParameterWithProfiles, error) {
+ params := make([]ParameterWithProfiles, 0, len(tcParams))
+ for _, tcParam := range tcParams {
+ param := ParameterWithProfiles{Parameter: tcParam}
+
+ profiles := []string{}
+ if err := json.Unmarshal(tcParam.Profiles, &profiles); err != nil {
+ return nil, errors.New("unmarshalling JSON from parameter '" + strconv.Itoa(param.ID) + "': " + err.Error())
+ }
+ param.ProfileNames = profiles
+ param.Profiles = nil
+ params = append(params, param)
+ }
+ return params, nil
+}
+
+type ParameterWithProfiles struct {
+ tc.Parameter
+ ProfileNames []string
+}
+
+type ParameterWithProfilesMap struct {
+ tc.Parameter
+ ProfileNames map[string]struct{}
+}
+
+func ParameterWithProfilesToMap(tcParams []ParameterWithProfiles) []ParameterWithProfilesMap {
+ params := []ParameterWithProfilesMap{}
+ for _, tcParam := range tcParams {
+ param := ParameterWithProfilesMap{Parameter: tcParam.Parameter, ProfileNames: map[string]struct{}{}}
+ for _, profile := range tcParam.ProfileNames {
+ param.ProfileNames[profile] = struct{}{}
+ }
+ params = append(params, param)
+ }
+ return params
+}
+
+// FilterParams filters params and returns only the parameters which match configFile, name, and value.
+// If configFile, name, or value is the empty string, it is not filtered.
+// Returns a slice of parameters.
+func FilterParams(params []tc.Parameter, configFile string, name string, value string, omitName string) []tc.Parameter {
+ filtered := []tc.Parameter{}
+ for _, param := range params {
+ if configFile != "" && param.ConfigFile != configFile {
+ continue
+ }
+ if name != "" && param.Name != name {
+ continue
+ }
+ if value != "" && param.Value != value {
+ continue
+ }
+ if omitName != "" && param.Name == omitName {
+ continue
+ }
+ filtered = append(filtered, param)
+ }
+ return filtered
+}
+
+// ParamsToMap converts a []tc.Parameter to a map[paramName]paramValue.
+// If multiple params have the same value, the first one in params will be used an an error will be logged.
+// See ParamArrToMultiMap.
+func ParamsToMap(params []tc.Parameter) map[string]string {
+ mp := map[string]string{}
+ for _, param := range params {
+ if val, ok := mp[param.Name]; ok {
+ log.Errorln("config generation got multiple parameters for name '" + param.Name + "' - using '" + val + "'")
+ continue
+ }
+ mp[param.Name] = param.Value
+ }
+ return mp
+}
+
+// ParamArrToMultiMap converts a []tc.Parameter to a map[paramName][]paramValue.
+func ParamsToMultiMap(params []tc.Parameter) map[string][]string {
+ mp := map[string][]string{}
+ for _, param := range params {
+ mp[param.Name] = append(mp[param.Name], param.Value)
+ }
+ return mp
+}
+
+type ATSConfigFile struct {
+ tc.ATSConfigMetaDataConfigFile
+ Text string
+ ContentType string
+}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/cfgfile_test.go b/traffic_ops/ort/atstccfg/cfgfile/cfgfile_test.go
new file mode 100644
index 0000000..1e4be28
--- /dev/null
+++ b/traffic_ops/ort/atstccfg/cfgfile/cfgfile_test.go
@@ -0,0 +1,111 @@
+package cfgfile
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import (
+ "bytes"
+ "strings"
+ "testing"
+
+ "github.com/apache/trafficcontrol/lib/go-tc"
+)
+
+func TestWriteConfigs(t *testing.T) {
+ buf := &bytes.Buffer{}
+ configs := []ATSConfigFile{
+ {
+ ATSConfigMetaDataConfigFile: tc.ATSConfigMetaDataConfigFile{
+ FileNameOnDisk: "config0.txt",
+ Location: "/my/config0/location",
+ },
+ Text: "config0",
+ ContentType: "text/plain",
+ },
+ {
+ ATSConfigMetaDataConfigFile: tc.ATSConfigMetaDataConfigFile{
+ FileNameOnDisk: "config1.txt",
+ Location: "/my/config1/location",
+ },
+ Text: "config2,foo",
+ ContentType: "text/csv",
+ },
+ }
+
+ if err := WriteConfigs(configs, buf); err != nil {
+ t.Fatalf("WriteConfigs error expected nil, actual: %v", err)
+ }
+
+ actual := buf.String()
+
+ expected0 := "Content-Type: text/plain\r\nPath: /my/config0/location/config0.txt\r\n\r\nconfig0\r\n"
+
+ if !strings.Contains(actual, expected0) {
+ t.Errorf("WriteConfigs expecte '%v' actual '%v'", expected0, actual)
+ }
+
+ expected1 := "Content-Type: text/csv\r\nPath: /my/config1/location/config1.txt\r\n\r\nconfig2,foo\r\n"
+ if !strings.Contains(actual, expected1) {
+ t.Errorf("WriteConfigs expected config1 '%v' actual '%v'", expected1, actual)
+ }
+
+ expectedPrefix := "MIME-Version: 1.0\r\nContent-Type: multipart/mixed; boundary="
+ if !strings.HasPrefix(actual, expectedPrefix) {
+ t.Errorf("WriteConfigs expected prefix '%v' actual '%v'", expectedPrefix, actual)
+ }
+}
+
+func TestPreprocessConfigFile(t *testing.T) {
+ // the TCP port replacement is fundamentally different for 80 vs non-80, so test both
+ {
+ server := tc.Server{
+ TCPPort: 8080,
+ IPAddress: "127.0.2.1",
+ HostName: "my-edge",
+ DomainName: "example.net",
+ }
+ cfgFile := "abc__SERVER_TCP_PORT__def__CACHE_IPV4__ghi __RETURN__ \t __HOSTNAME__ jkl __FULL_HOSTNAME__ \n__SOMETHING__ __ELSE__\nmno\r\n"
+
+ actual := PreprocessConfigFile(server, cfgFile)
+
+ expected := "abc8080def127.0.2.1ghi\nmy-edge jkl my-edge.example.net \n__SOMETHING__ __ELSE__\nmno\r\n"
+
+ if expected != actual {
+ t.Errorf("PreprocessConfigFile expected '%v' actual '%v'", expected, actual)
+ }
+ }
+
+ {
+ server := tc.Server{
+ TCPPort: 80,
+ IPAddress: "127.0.2.1",
+ HostName: "my-edge",
+ DomainName: "example.net",
+ }
+ cfgFile := "abc__SERVER_TCP_PORT__def__CACHE_IPV4__ghi __RETURN__ \t __HOSTNAME__ jkl __FULL_HOSTNAME__ \n__SOMETHING__ __ELSE__\nmno:__SERVER_TCP_PORT__\r\n"
+
+ actual := PreprocessConfigFile(server, cfgFile)
+
+ expected := "abc__SERVER_TCP_PORT__def127.0.2.1ghi\nmy-edge jkl my-edge.example.net \n__SOMETHING__ __ELSE__\nmno\r\n"
+
+ if expected != actual {
+ t.Errorf("PreprocessConfigFile expected '%v' actual '%v'", expected, actual)
+ }
+ }
+}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/chkconfig.go b/traffic_ops/ort/atstccfg/cfgfile/chkconfig.go
index 0034c03..ff850bf 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/chkconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/chkconfig.go
@@ -20,61 +20,17 @@ package cfgfile
*/
import (
- "errors"
- "strconv"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileServerChkconfig(cfg config.TCCfg, serverNameOrID string) (string, error) {
- // TODO TOAPI add /servers?cdn=1 query param
- servers, err := toreq.GetServers(cfg)
- if err != nil {
- return "", errors.New("getting servers: " + err.Error())
- }
-
- server := tc.Server{ID: atscfg.InvalidID}
- if serverID, err := strconv.Atoi(serverNameOrID); err == nil {
- for _, toServer := range servers {
- if toServer.ID == serverID {
- server = toServer
- break
- }
- }
- } else {
- serverName := serverNameOrID
- for _, toServer := range servers {
- if toServer.HostName == serverName {
- server = toServer
- break
- }
- }
- }
- if server.ID == atscfg.InvalidID {
- return "", errors.New("server '" + serverNameOrID + " not found in servers")
- }
-
- profileParams, err := toreq.GetProfileParameters(cfg, server.Profile)
- if err != nil {
- return "", errors.New("getting profile '" + server.Profile + "' parameters: " + err.Error())
- }
- if len(profileParams) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params, to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
+func GetConfigFileServerChkconfig(toData *TOData) (string, string, error) {
fileParams := map[string][]string{}
- for _, param := range profileParams {
+ for _, param := range toData.ServerParams {
if param.ConfigFile != atscfg.ChkconfigParamConfigFile {
continue
}
fileParams[param.Name] = append(fileParams[param.Name], param.Value)
}
- txt := atscfg.MakeChkconfig(fileParams)
- return txt, nil
+ return atscfg.MakeChkconfig(fileParams), atscfg.ContentTypeChkconfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/dropqstringdotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/dropqstringdotconfig.go
index 1ea61cb..28e805b 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/dropqstringdotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/dropqstringdotconfig.go
@@ -20,37 +20,12 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileProfileDropQStringDotConfig(cfg config.TCCfg, profileNameOrID string) (string, error) {
- profileName, err := toreq.GetProfileNameFromProfileNameOrID(cfg, profileNameOrID)
- if err != nil {
- return "", errors.New("getting profile name from '" + profileNameOrID + "': " + err.Error())
- }
-
- profileParameters, err := toreq.GetProfileParameters(cfg, profileName)
- if err != nil {
- return "", errors.New("getting profile '" + profileName + "' parameters: " + err.Error())
- }
- if len(profileParameters) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params (and there should always be volume.config params), to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
+func GetConfigFileProfileDropQStringDotConfig(toData *TOData) (string, string, error) {
dropQStringVal := (*string)(nil)
- // TODO add configFile query param to profile/parameters endpoint, to only get needed data
- for _, param := range profileParameters {
+ for _, param := range toData.ServerParams {
if param.ConfigFile != atscfg.DropQStringDotConfigFileName {
continue
}
@@ -61,5 +36,5 @@ func GetConfigFileProfileDropQStringDotConfig(cfg config.TCCfg, profileNameOrID
break
}
- return atscfg.MakeDropQStringDotConfig(profileName, toToolName, toURL, dropQStringVal), nil
+ return atscfg.MakeDropQStringDotConfig(toData.Server.Profile, toData.TOToolName, toData.TOURL, dropQStringVal), atscfg.ContentTypeDropQStringDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/facts.go b/traffic_ops/ort/atstccfg/cfgfile/facts.go
index a51dd5c..21ca9f1 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/facts.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/facts.go
@@ -20,22 +20,9 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileProfile12MFacts(cfg config.TCCfg, profileNameOrID string) (string, error) {
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- profileName, err := toreq.GetProfileNameFromProfileNameOrID(cfg, profileNameOrID)
- if err != nil {
- return "", errors.New("getting profile name from '" + profileNameOrID + "': " + err.Error())
- }
- return atscfg.Make12MFacts(profileName, toToolName, toURL), nil
+func GetConfigFileProfile12MFacts(toData *TOData) (string, string, error) {
+ return atscfg.Make12MFacts(toData.Server.Profile, toData.TOToolName, toData.TOURL), atscfg.ContentType12MFacts, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/headerrewritedotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/headerrewritedotconfig.go
index 0759b92..3a8966a 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/headerrewritedotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/headerrewritedotconfig.go
@@ -25,36 +25,13 @@ import (
"github.com/apache/trafficcontrol/lib/go-atscfg"
"github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileCDNHeaderRewrite(cfg config.TCCfg, cdnNameOrID string, fileName string) (string, error) {
-
- cdnName, err := toreq.GetCDNNameFromCDNNameOrID(cfg, cdnNameOrID)
- if err != nil {
- return "", errors.New("getting CDN name from '" + cdnNameOrID + "': " + err.Error())
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- cdn, err := toreq.GetCDN(cfg, cdnName)
- if err != nil {
- return "", errors.New("getting cdn '" + string(cdnName) + "': " + err.Error())
- }
-
+func GetConfigFileCDNHeaderRewrite(toData *TOData, fileName string) (string, string, error) {
dsName := strings.TrimSuffix(strings.TrimPrefix(fileName, atscfg.HeaderRewritePrefix), atscfg.ConfigSuffix) // TODO verify prefix and suffix? Perl doesn't
- // TODO determine if it's faster overall for ORT to get all DSes (which we already have for other configs), or to get the single DS we need
- dses, err := toreq.GetCDNDeliveryServices(cfg, cdn.ID)
- if err != nil {
- return "", errors.New("getting delivery services: " + err.Error())
- }
tcDS := tc.DeliveryServiceNullable{}
- for _, ds := range dses {
+ for _, ds := range toData.DeliveryServices {
if ds.XMLID == nil || *ds.XMLID != dsName {
continue
}
@@ -62,22 +39,19 @@ func GetConfigFileCDNHeaderRewrite(cfg config.TCCfg, cdnNameOrID string, fileNam
break
}
if tcDS.ID == nil {
- return "", errors.New("ds '" + dsName + "' not found")
+ return "", "", errors.New("ds '" + dsName + "' not found")
}
if tcDS.CDNName == nil {
- return "", errors.New("ds '" + dsName + "' missing cdn")
+ return "", "", errors.New("ds '" + dsName + "' missing cdn")
}
cfgDS, err := atscfg.HeaderRewriteDSFromDS(&tcDS)
if err != nil {
- return "", errors.New("converting ds to config ds: " + err.Error())
+ return "", "", errors.New("converting ds to config ds: " + err.Error())
}
- dsServers, err := toreq.GetDeliveryServiceServers(cfg, []int{cfgDS.ID}, nil)
- if err != nil {
- return "", errors.New("getting delivery service servers: " + err.Error())
- }
+ dsServers := FilterDSS(toData.DeliveryServiceServers, map[int]struct{}{cfgDS.ID: {}}, nil)
dsServerIDs := map[int]struct{}{}
for _, dss := range dsServers {
@@ -90,13 +64,8 @@ func GetConfigFileCDNHeaderRewrite(cfg config.TCCfg, cdnNameOrID string, fileNam
dsServerIDs[*dss.Server] = struct{}{}
}
- servers, err := toreq.GetServers(cfg)
- if err != nil {
- return "", errors.New("getting servers: " + err.Error())
- }
-
assignedEdges := []atscfg.HeaderRewriteServer{}
- for _, server := range servers {
+ for _, server := range toData.Servers {
if server.CDNName != *tcDS.CDNName {
continue
}
@@ -110,5 +79,5 @@ func GetConfigFileCDNHeaderRewrite(cfg config.TCCfg, cdnNameOrID string, fileNam
assignedEdges = append(assignedEdges, cfgServer)
}
- return atscfg.MakeHeaderRewriteDotConfig(cdnName, toToolName, toURL, cfgDS, assignedEdges), nil
+ return atscfg.MakeHeaderRewriteDotConfig(tc.CDNName(toData.Server.CDNName), toData.TOToolName, toData.TOURL, cfgDS, assignedEdges), atscfg.ContentTypeHeaderRewriteDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/headerrewritemiddotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/headerrewritemiddotconfig.go
index def1e22..9d042be 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/headerrewritemiddotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/headerrewritemiddotconfig.go
@@ -25,35 +25,13 @@ import (
"github.com/apache/trafficcontrol/lib/go-atscfg"
"github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileCDNHeaderRewriteMid(cfg config.TCCfg, cdnNameOrID string, fileName string) (string, error) {
- cdnName, err := toreq.GetCDNNameFromCDNNameOrID(cfg, cdnNameOrID)
- if err != nil {
- return "", errors.New("getting CDN name from '" + cdnNameOrID + "': " + err.Error())
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- cdn, err := toreq.GetCDN(cfg, cdnName)
- if err != nil {
- return "", errors.New("getting cdn '" + string(cdnName) + "': " + err.Error())
- }
-
+func GetConfigFileCDNHeaderRewriteMid(toData *TOData, fileName string) (string, string, error) {
dsName := strings.TrimSuffix(strings.TrimPrefix(fileName, atscfg.HeaderRewriteMidPrefix), atscfg.ConfigSuffix) // TODO verify prefix and suffix? Perl doesn't
- // TODO determine if it's faster overall for ORT to get all DSes (which we already have for other configs), or to get the single DS we need
- dses, err := toreq.GetCDNDeliveryServices(cfg, cdn.ID)
- if err != nil {
- return "", errors.New("getting delivery services: " + err.Error())
- }
tcDS := tc.DeliveryServiceNullable{}
- for _, ds := range dses {
+ for _, ds := range toData.DeliveryServices {
if ds.XMLID == nil || *ds.XMLID != dsName {
continue
}
@@ -61,22 +39,19 @@ func GetConfigFileCDNHeaderRewriteMid(cfg config.TCCfg, cdnNameOrID string, file
break
}
if tcDS.ID == nil {
- return "", errors.New("ds '" + dsName + "' not found")
+ return "", "", errors.New("ds '" + dsName + "' not found")
}
if tcDS.CDNName == nil {
- return "", errors.New("ds '" + dsName + "' missing cdn")
+ return "", "", errors.New("ds '" + dsName + "' missing cdn")
}
cfgDS, err := atscfg.HeaderRewriteDSFromDS(&tcDS)
if err != nil {
- return "", errors.New("converting ds to config ds: " + err.Error())
+ return "", "", errors.New("converting ds to config ds: " + err.Error())
}
- dsServers, err := toreq.GetDeliveryServiceServers(cfg, []int{cfgDS.ID}, nil)
- if err != nil {
- return "", errors.New("getting delivery service servers: " + err.Error())
- }
+ dsServers := FilterDSS(toData.DeliveryServiceServers, map[int]struct{}{cfgDS.ID: {}}, nil)
dsServerIDs := map[int]struct{}{}
for _, dss := range dsServers {
@@ -89,19 +64,9 @@ func GetConfigFileCDNHeaderRewriteMid(cfg config.TCCfg, cdnNameOrID string, file
dsServerIDs[*dss.Server] = struct{}{}
}
- servers, err := toreq.GetServers(cfg)
- if err != nil {
- return "", errors.New("getting servers: " + err.Error())
- }
-
- cgs, err := toreq.GetCacheGroups(cfg)
- if err != nil {
- return "", errors.New("getting cachegroups: " + err.Error())
- }
-
serverCGs := map[tc.CacheGroupName]struct{}{}
- for _, sv := range servers {
- if tc.CDNName(sv.CDNName) != cdnName {
+ for _, sv := range toData.Servers {
+ if sv.CDNName != toData.Server.CDNName {
continue
}
if tc.CacheStatus(sv.Status) != tc.CacheStatusReported && tc.CacheStatus(sv.Status) != tc.CacheStatusOnline {
@@ -111,7 +76,7 @@ func GetConfigFileCDNHeaderRewriteMid(cfg config.TCCfg, cdnNameOrID string, file
}
parentCGs := map[string]struct{}{} // names of cachegroups which are parent cachegroups of the cachegroup of any edge assigned to the given DS
- for _, cg := range cgs {
+ for _, cg := range toData.CacheGroups {
if cg.Name == nil {
continue // TODO warn?
}
@@ -125,7 +90,7 @@ func GetConfigFileCDNHeaderRewriteMid(cfg config.TCCfg, cdnNameOrID string, file
}
parentServers := []tc.Server{}
- for _, sv := range servers {
+ for _, sv := range toData.Servers {
if _, ok := parentCGs[sv.Cachegroup]; !ok {
continue
}
@@ -133,7 +98,7 @@ func GetConfigFileCDNHeaderRewriteMid(cfg config.TCCfg, cdnNameOrID string, file
}
assignedMids := []atscfg.HeaderRewriteServer{}
- for _, server := range servers {
+ for _, server := range toData.Servers {
if server.CDNName != *tcDS.CDNName {
continue
}
@@ -147,5 +112,5 @@ func GetConfigFileCDNHeaderRewriteMid(cfg config.TCCfg, cdnNameOrID string, file
assignedMids = append(assignedMids, cfgServer)
}
- return atscfg.MakeHeaderRewriteMidDotConfig(cdnName, toToolName, toURL, cfgDS, assignedMids), nil
+ return atscfg.MakeHeaderRewriteMidDotConfig(tc.CDNName(toData.Server.CDNName), toData.TOToolName, toData.TOURL, cfgDS, assignedMids), atscfg.ContentTypeHeaderRewriteDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/hostingdotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/hostingdotconfig.go
index 52a1a71..66a6414 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/hostingdotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/hostingdotconfig.go
@@ -21,110 +21,44 @@ package cfgfile
import (
"errors"
- "strconv"
"strings"
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/lib/go-log"
"github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
const ServerHostingDotConfigMidIncludeInactive = false
const ServerHostingDotConfigEdgeIncludeInactive = true
-func GetConfigFileServerHostingDotConfig(cfg config.TCCfg, serverNameOrID string) (string, error) {
- // TODO TOAPI add /servers?cdn=1 query param
- servers, err := toreq.GetServers(cfg)
- if err != nil {
- return "", errors.New("getting servers: " + err.Error())
- }
-
- server := tc.Server{ID: atscfg.InvalidID}
- if serverID, err := strconv.Atoi(serverNameOrID); err == nil {
- for _, toServer := range servers {
- if toServer.ID == serverID {
- server = toServer
- break
- }
- }
- } else {
- serverName := serverNameOrID
- for _, toServer := range servers {
- if toServer.HostName == serverName {
- server = toServer
- break
- }
- }
- }
- if server.ID == atscfg.InvalidID {
- return "", errors.New("server '" + serverNameOrID + " not found in servers")
- }
-
- serverName := tc.CacheName(server.HostName)
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- profileParams, err := toreq.GetProfileParameters(cfg, server.Profile)
- if err != nil {
- return "", errors.New("getting profile '" + server.Profile + "' parameters: " + err.Error())
- }
- if len(profileParams) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params, to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- fileParams := map[string]string{}
- for _, param := range profileParams {
- if param.ConfigFile != atscfg.HostingConfigParamConfigFile {
- continue
- }
- if val, ok := fileParams[param.Name]; ok {
- log.Errorln("hosting config parameter name '" + param.Name + "' got multiple values - using '" + val + "'")
- continue
- }
- fileParams[param.Name] = param.Value
- }
-
- dses, err := toreq.GetCDNDeliveryServices(cfg, server.CDNID)
- if err != nil {
- return "", errors.New("getting delivery services: " + err.Error())
- }
+func GetConfigFileServerHostingDotConfig(toData *TOData) (string, string, error) {
+ fileParams := ParamsToMap(FilterParams(toData.ServerParams, atscfg.HostingConfigParamConfigFile, "", "", ""))
cdnServers := map[tc.CacheName]tc.Server{}
- for _, sv := range servers {
- if sv.CDNID != server.CDNID {
+ for _, sv := range toData.Servers {
+ if sv.CDNID != toData.Server.CDNID {
continue
}
cdnServers[tc.CacheName(sv.HostName)] = sv
}
- serverIDs := []int{}
+ serverIDs := map[int]struct{}{}
for _, sv := range cdnServers {
- serverIDs = append(serverIDs, sv.ID)
+ serverIDs[sv.ID] = struct{}{}
}
- dsIDs := []int{}
- for _, ds := range dses {
+ dsIDs := map[int]struct{}{}
+ for _, ds := range toData.DeliveryServices {
if ds.ID != nil {
- dsIDs = append(dsIDs, *ds.ID)
+ dsIDs[*ds.ID] = struct{}{}
}
}
- dsServers, err := toreq.GetDeliveryServiceServers(cfg, dsIDs, serverIDs)
- if err != nil {
- return "", errors.New("getting delivery service servers: " + err.Error())
- }
+ dsServers := FilterDSS(toData.DeliveryServiceServers, dsIDs, serverIDs)
dsServerMap := map[int]map[int]struct{}{} // set[dsID][serverID]
for _, dss := range dsServers {
if dss.Server == nil || dss.DeliveryService == nil {
- return "", errors.New("deliveryserviceservers returned dss with nil values")
+ return "", "", errors.New("deliveryserviceservers returned dss with nil values")
}
if _, ok := dsServerMap[*dss.DeliveryService]; !ok {
dsServerMap[*dss.DeliveryService] = map[int]struct{}{}
@@ -134,9 +68,9 @@ func GetConfigFileServerHostingDotConfig(cfg config.TCCfg, serverNameOrID string
hostingDSes := map[tc.DeliveryServiceName]tc.DeliveryServiceNullable{}
- isMid := strings.HasPrefix(server.Type, tc.MidTypePrefix)
+ isMid := strings.HasPrefix(toData.Server.Type, tc.MidTypePrefix)
- for _, ds := range dses {
+ for _, ds := range toData.DeliveryServices {
if ds.Active == nil || ds.Type == nil || ds.XMLID == nil || ds.CDNID == nil || ds.ID == nil || ds.OrgServerFQDN == nil {
// some DSes have nil origins. I think MSO? TODO: verify
continue
@@ -146,7 +80,7 @@ func GetConfigFileServerHostingDotConfig(cfg config.TCCfg, serverNameOrID string
continue
}
- if *ds.CDNID != server.CDNID {
+ if *ds.CDNID != toData.Server.CDNID {
continue
}
@@ -169,7 +103,7 @@ func GetConfigFileServerHostingDotConfig(cfg config.TCCfg, serverNameOrID string
continue
}
- if _, ok := dsServerMap[*ds.ID][server.ID]; !ok {
+ if _, ok := dsServerMap[*ds.ID][toData.Server.ID]; !ok {
continue
}
}
@@ -186,6 +120,5 @@ func GetConfigFileServerHostingDotConfig(cfg config.TCCfg, serverNameOrID string
origins = append(origins, origin)
}
- txt := atscfg.MakeHostingDotConfig(serverName, toToolName, toURL, fileParams, origins)
- return txt, nil
+ return atscfg.MakeHostingDotConfig(tc.CacheName(toData.Server.HostName), toData.TOToolName, toData.TOURL, fileParams, origins), atscfg.ContentTypeHostingDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/ipallowdotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/ipallowdotconfig.go
index 119d06b..364510e 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/ipallowdotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/ipallowdotconfig.go
@@ -21,85 +21,26 @@ package cfgfile
import (
"errors"
- "strconv"
"strings"
"github.com/apache/trafficcontrol/lib/go-atscfg"
"github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileServerIPAllowDotConfig(cfg config.TCCfg, serverNameOrID string) (string, error) {
- // TODO TOAPI add /servers?cdn=1 query param
- servers, err := toreq.GetServers(cfg)
- if err != nil {
- return "", errors.New("getting servers: " + err.Error())
- }
-
- server := tc.Server{ID: atscfg.InvalidID}
- if serverID, err := strconv.Atoi(serverNameOrID); err == nil {
- for _, toServer := range servers {
- if toServer.ID == serverID {
- server = toServer
- break
- }
- }
- } else {
- serverName := serverNameOrID
- for _, toServer := range servers {
- if toServer.HostName == serverName {
- server = toServer
- break
- }
- }
- }
- if server.ID == atscfg.InvalidID {
- return "", errors.New("server '" + serverNameOrID + " not found in servers")
- }
-
- serverName := tc.CacheName(server.HostName)
- serverType := tc.CacheType(server.Type)
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- profileParams, err := toreq.GetProfileParameters(cfg, server.Profile)
- if err != nil {
- return "", errors.New("getting profile '" + server.Profile + "' parameters: " + err.Error())
- }
- if len(profileParams) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params, to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- fileParams := map[string][]string{}
- for _, param := range profileParams {
- if param.ConfigFile != atscfg.IPAllowConfigFileName {
- continue
- }
- fileParams[param.Name] = append(fileParams[param.Name], param.Value)
- }
-
- cacheGroups, err := toreq.GetCacheGroups(cfg)
- if err != nil {
- return "", errors.New("getting cachegroups: " + err.Error())
- }
+func GetConfigFileServerIPAllowDotConfig(toData *TOData) (string, string, error) {
+ fileParams := ParamsToMultiMap(FilterParams(toData.ServerParams, atscfg.IPAllowConfigFileName, "", "", ""))
cgMap := map[string]tc.CacheGroupNullable{}
- for _, cg := range cacheGroups {
+ for _, cg := range toData.CacheGroups {
if cg.Name == nil {
- return "", errors.New("got cachegroup with nil name!'")
+ return "", "", errors.New("got cachegroup with nil name!'")
}
cgMap[*cg.Name] = cg
}
- serverCG, ok := cgMap[server.Cachegroup]
+ serverCG, ok := cgMap[toData.Server.Cachegroup]
if !ok {
- return "", errors.New("server cachegroup not in cachegroups!")
+ return "", "", errors.New("server cachegroup not in cachegroups!")
}
childCGs := map[string]tc.CacheGroupNullable{}
@@ -110,13 +51,12 @@ func GetConfigFileServerIPAllowDotConfig(cfg config.TCCfg, serverNameOrID string
}
childServers := map[tc.CacheName]atscfg.IPAllowServer{}
- for _, sv := range servers {
+ for _, sv := range toData.Servers {
_, ok := childCGs[sv.Cachegroup]
- if ok || (strings.HasPrefix(string(serverType), tc.MidTypePrefix) && string(sv.Type) == tc.MonitorTypeName) {
+ if ok || (strings.HasPrefix(toData.Server.Type, tc.MidTypePrefix) && string(sv.Type) == tc.MonitorTypeName) {
childServers[tc.CacheName(sv.HostName)] = atscfg.IPAllowServer{IPAddress: sv.IPAddress, IP6Address: sv.IP6Address}
}
}
- txt := atscfg.MakeIPAllowDotConfig(serverName, serverType, toToolName, toURL, fileParams, childServers)
- return txt, nil
+ return atscfg.MakeIPAllowDotConfig(tc.CacheName(toData.Server.HostName), tc.CacheType(toData.Server.Type), toData.TOToolName, toData.TOURL, fileParams, childServers), atscfg.ContentTypeIPAllowDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/loggingdotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/loggingdotconfig.go
index 420d036..9783413 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/loggingdotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/loggingdotconfig.go
@@ -20,45 +20,10 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileProfileLoggingDotConfig(cfg config.TCCfg, profileNameOrID string) (string, error) {
- profileName, err := toreq.GetProfileNameFromProfileNameOrID(cfg, profileNameOrID)
- if err != nil {
- return "", errors.New("getting profile name from '" + profileNameOrID + "': " + err.Error())
- }
-
- profileParameters, err := toreq.GetProfileParameters(cfg, profileName)
- if err != nil {
- return "", errors.New("getting profile '" + profileName + "' parameters: " + err.Error())
- }
- if len(profileParameters) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params (and there should always be volume.config params), to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- paramData := map[string]string{}
- // TODO add configFile query param to profile/parameters endpoint, to only get needed data
- for _, param := range profileParameters {
- if param.ConfigFile != atscfg.LoggingFileName {
- continue
- }
- if param.Name == "location" {
- continue
- }
- paramData[param.Name] = param.Value
- }
-
- return atscfg.MakeLoggingDotConfig(profileName, paramData, toToolName, toURL), nil
+func GetConfigFileProfileLoggingDotConfig(toData *TOData) (string, string, error) {
+ params := ParamsToMap(FilterParams(toData.ServerParams, atscfg.LoggingFileName, "", "", "location"))
+ return atscfg.MakeLoggingDotConfig(toData.Server.Profile, params, toData.TOToolName, toData.TOURL), atscfg.ContentTypeLoggingDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/loggingdotyaml.go b/traffic_ops/ort/atstccfg/cfgfile/loggingdotyaml.go
index 55e121b..5e2a5ae 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/loggingdotyaml.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/loggingdotyaml.go
@@ -20,45 +20,10 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileProfileLoggingDotYAML(cfg config.TCCfg, profileNameOrID string) (string, error) {
- profileName, err := toreq.GetProfileNameFromProfileNameOrID(cfg, profileNameOrID)
- if err != nil {
- return "", errors.New("getting profile name from '" + profileNameOrID + "': " + err.Error())
- }
-
- profileParameters, err := toreq.GetProfileParameters(cfg, profileName)
- if err != nil {
- return "", errors.New("getting profile '" + profileName + "' parameters: " + err.Error())
- }
- if len(profileParameters) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params (and there should always be volume.config params), to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- paramData := map[string]string{}
- // TODO add configFile query param to profile/parameters endpoint, to only get needed data
- for _, param := range profileParameters {
- if param.ConfigFile != atscfg.LoggingYAMLFileName {
- continue
- }
- if param.Name == "location" {
- continue
- }
- paramData[param.Name] = param.Value
- }
-
- return atscfg.MakeLoggingDotYAML(profileName, paramData, toToolName, toURL), nil
+func GetConfigFileProfileLoggingDotYAML(toData *TOData) (string, string, error) {
+ params := ParamsToMap(FilterParams(toData.ServerParams, atscfg.LoggingYAMLFileName, "", "", "location"))
+ return atscfg.MakeLoggingDotYAML(toData.Server.Profile, params, toData.TOToolName, toData.TOURL), atscfg.ContentTypeLoggingDotYAML, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/logsxmldotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/logsxmldotconfig.go
index cd144f9..493fbfe 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/logsxmldotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/logsxmldotconfig.go
@@ -20,45 +20,10 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileProfileLogsXMLDotConfig(cfg config.TCCfg, profileNameOrID string) (string, error) {
- profileName, err := toreq.GetProfileNameFromProfileNameOrID(cfg, profileNameOrID)
- if err != nil {
- return "", errors.New("getting profile name from '" + profileNameOrID + "': " + err.Error())
- }
-
- profileParameters, err := toreq.GetProfileParameters(cfg, profileName)
- if err != nil {
- return "", errors.New("getting profile '" + profileName + "' parameters: " + err.Error())
- }
- if len(profileParameters) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params (and there should always be volume.config params), to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- paramData := map[string]string{}
- // TODO add configFile query param to profile/parameters endpoint, to only get needed data
- for _, param := range profileParameters {
- if param.ConfigFile != atscfg.LogsXMLFileName {
- continue
- }
- if param.Name == "location" {
- continue
- }
- paramData[param.Name] = param.Value
- }
-
- return atscfg.MakeLogsXMLDotConfig(profileName, paramData, toToolName, toURL), nil
+func GetConfigFileProfileLogsXMLDotConfig(toData *TOData) (string, string, error) {
+ params := ParamsToMap(FilterParams(toData.ServerParams, atscfg.LogsXMLFileName, "", "", "location"))
+ return atscfg.MakeLogsXMLDotConfig(toData.Server.Profile, params, toData.TOToolName, toData.TOURL), atscfg.ContentTypeLogsDotXML, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/meta.go b/traffic_ops/ort/atstccfg/cfgfile/meta.go
index 313c08e..3f64896 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/meta.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/meta.go
@@ -21,59 +21,23 @@ package cfgfile
import (
"errors"
- "strconv"
"github.com/apache/trafficcontrol/lib/go-atscfg"
"github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileMeta(cfg config.TCCfg, serverNameOrID string) (string, error) {
- servers, err := toreq.GetServers(cfg)
- if err != nil {
- return "", errors.New("getting servers: " + err.Error())
- }
-
- server := tc.Server{ID: atscfg.InvalidID}
- if serverID, err := strconv.Atoi(serverNameOrID); err == nil {
- for _, toServer := range servers {
- if toServer.ID == serverID {
- server = toServer
- break
- }
- }
- } else {
- serverName := serverNameOrID
- for _, toServer := range servers {
- if toServer.HostName == serverName {
- server = toServer
- break
- }
- }
- }
- if server.ID == atscfg.InvalidID {
- return "", errors.New("server '" + serverNameOrID + " not found in servers")
- }
-
- serverHostName := server.HostName
-
- cacheGroups, err := toreq.GetCacheGroups(cfg)
- if err != nil {
- return "", errors.New("getting cachegroups: " + err.Error())
- }
-
+func GetMeta(toData *TOData) (*tc.ATSConfigMetaData, error) {
cgMap := map[string]tc.CacheGroupNullable{}
- for _, cg := range cacheGroups {
+ for _, cg := range toData.CacheGroups {
if cg.Name == nil {
- return "", errors.New("got cachegroup with nil name!'")
+ return nil, errors.New("got cachegroup with nil name!'")
}
cgMap[*cg.Name] = cg
}
- serverCG, ok := cgMap[server.Cachegroup]
+ serverCG, ok := cgMap[toData.Server.Cachegroup]
if !ok {
- return "", errors.New("server '" + serverNameOrID + "' cachegroup '" + server.Cachegroup + "' not found in CacheGroups")
+ return nil, errors.New("server '" + toData.Server.HostName + "' cachegroup '" + toData.Server.Cachegroup + "' not found in CacheGroups")
}
parentCGID := -1
@@ -81,15 +45,15 @@ func GetConfigFileMeta(cfg config.TCCfg, serverNameOrID string) (string, error)
if serverCG.ParentName != nil && *serverCG.ParentName != "" {
parentCG, ok := cgMap[*serverCG.ParentName]
if !ok {
- return "", errors.New("server '" + serverNameOrID + "' cachegroup '" + server.Cachegroup + "' parent '" + *serverCG.ParentName + "' not found in CacheGroups")
+ return nil, errors.New("server '" + toData.Server.HostName + "' cachegroup '" + toData.Server.Cachegroup + "' parent '" + *serverCG.ParentName + "' not found in CacheGroups")
}
if parentCG.ID == nil {
- return "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil ID!'")
+ return nil, errors.New("got cachegroup '" + *parentCG.Name + "' with nil ID!'")
}
parentCGID = *parentCG.ID
if parentCG.Type == nil {
- return "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil Type!'")
+ return nil, errors.New("got cachegroup '" + *parentCG.Name + "' with nil Type!'")
}
parentCGType = *parentCG.Type
}
@@ -99,46 +63,41 @@ func GetConfigFileMeta(cfg config.TCCfg, serverNameOrID string) (string, error)
if serverCG.SecondaryParentName != nil && *serverCG.SecondaryParentName != "" {
parentCG, ok := cgMap[*serverCG.SecondaryParentName]
if !ok {
- return "", errors.New("server '" + serverNameOrID + "' cachegroup '" + server.Cachegroup + "' secondary parent '" + *serverCG.SecondaryParentName + "' not found in CacheGroups")
+ return nil, errors.New("server '" + toData.Server.HostName + "' cachegroup '" + toData.Server.Cachegroup + "' secondary parent '" + *serverCG.SecondaryParentName + "' not found in CacheGroups")
}
if parentCG.ID == nil {
- return "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil ID!'")
+ return nil, errors.New("got cachegroup '" + *parentCG.Name + "' with nil ID!'")
}
secondaryParentCGID = *parentCG.ID
if parentCG.Type == nil {
- return "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil Type!'")
+ return nil, errors.New("got cachegroup '" + *parentCG.Name + "' with nil Type!'")
}
secondaryParentCGType = *parentCG.Type
}
serverInfo := atscfg.ServerInfo{
- CacheGroupID: server.CachegroupID,
- CDN: tc.CDNName(server.CDNName),
- CDNID: server.CDNID,
- DomainName: server.DomainName,
- HostName: server.HostName,
- ID: server.ID,
- IP: server.IPAddress,
+ CacheGroupID: toData.Server.CachegroupID,
+ CDN: tc.CDNName(toData.Server.CDNName),
+ CDNID: toData.Server.CDNID,
+ DomainName: toData.Server.DomainName,
+ HostName: toData.Server.HostName,
+ ID: toData.Server.ID,
+ IP: toData.Server.IPAddress,
ParentCacheGroupID: parentCGID,
ParentCacheGroupType: parentCGType,
- ProfileID: atscfg.ProfileID(server.ProfileID),
- ProfileName: server.Profile,
- Port: server.TCPPort,
+ ProfileID: atscfg.ProfileID(toData.Server.ProfileID),
+ ProfileName: toData.Server.Profile,
+ Port: toData.Server.TCPPort,
SecondaryParentCacheGroupID: secondaryParentCGID,
SecondaryParentCacheGroupType: secondaryParentCGType,
- Type: server.Type,
- }
-
- globalParams, err := toreq.GetGlobalParameters(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
+ Type: toData.Server.Type,
}
toReverseProxyURL := ""
toURL := ""
- for _, param := range globalParams {
+ for _, param := range toData.GlobalParams {
if param.Name == "tm.rev_proxy.url" {
toReverseProxyURL = param.Value
} else if param.Name == "tm.url" {
@@ -149,23 +108,13 @@ func GetConfigFileMeta(cfg config.TCCfg, serverNameOrID string) (string, error)
}
}
- scopeParamsRaw, err := toreq.GetParametersByName(cfg, "scope")
- if err != nil {
- return "", errors.New("getting scope parameters: " + err.Error())
- }
-
scopeParams := map[string]string{}
- for _, param := range scopeParamsRaw {
+ for _, param := range toData.ScopeParams {
scopeParams[param.ConfigFile] = param.Value
}
- serverProfileParameters, err := toreq.GetServerProfileParameters(cfg, server.Profile)
- if err != nil {
- return "", errors.New("getting server profile '" + server.Profile + "' parameters: " + err.Error())
- }
-
locationParams := map[string]atscfg.ConfigProfileParams{}
- for _, param := range serverProfileParameters {
+ for _, param := range toData.ServerParams {
if param.Name == "location" {
p := locationParams[param.ConfigFile]
p.FileNameOnDisk = param.ConfigFile
@@ -178,37 +127,35 @@ func GetConfigFileMeta(cfg config.TCCfg, serverNameOrID string) (string, error)
}
}
- deliveryServices, err := toreq.GetCDNDeliveryServices(cfg, server.CDNID)
- if err != nil {
- return "", errors.New("getting delivery services: " + err.Error())
- }
-
dsNames := map[tc.DeliveryServiceName]struct{}{}
- if tc.CacheTypeFromString(server.Type) != tc.CacheTypeMid {
- dsIDs := []int{}
- for _, ds := range deliveryServices {
+ if tc.CacheTypeFromString(toData.Server.Type) != tc.CacheTypeMid {
+ dsIDs := map[int]struct{}{}
+ for _, ds := range toData.DeliveryServices {
if ds.ID == nil {
// TODO log error?
continue
}
- dsIDs = append(dsIDs, *ds.ID)
+ dsIDs[*ds.ID] = struct{}{}
}
- serverIDs := []int{server.ID}
-
- dsServers, err := toreq.GetDeliveryServiceServers(cfg, dsIDs, serverIDs)
- if err != nil {
- return "", errors.New("getting meta config delivery service servers: " + err.Error())
- }
+ // TODO verify?
+ // serverIDs := []int{toData.Server.ID}
- dssMap := map[int]struct{}{} // set of map[dsID]. We know we only asked for our own server, so we don't care about the servers returned.
- for _, dss := range dsServers {
- if dss.DeliveryService == nil {
- continue // TODO log?
+ dssMap := map[int]struct{}{}
+ for _, dss := range toData.DeliveryServiceServers {
+ if dss.Server == nil || dss.DeliveryService == nil {
+ continue // TODO warn?
+ }
+ if *dss.Server != toData.Server.ID {
+ continue
+ }
+ if _, ok := dsIDs[*dss.DeliveryService]; !ok {
+ continue
}
dssMap[*dss.DeliveryService] = struct{}{}
}
- for _, ds := range deliveryServices {
+
+ for _, ds := range toData.DeliveryServices {
if ds.ID == nil {
continue
}
@@ -221,14 +168,14 @@ func GetConfigFileMeta(cfg config.TCCfg, serverNameOrID string) (string, error)
dsNames[tc.DeliveryServiceName(*ds.XMLID)] = struct{}{}
}
} else {
- for _, ds := range deliveryServices {
+ for _, ds := range toData.DeliveryServices {
if ds.ID == nil {
continue
}
if ds.XMLID == nil {
continue // TODO log?
}
- if ds.CDNID == nil || *ds.CDNID != server.CDNID {
+ if ds.CDNID == nil || *ds.CDNID != toData.Server.CDNID {
continue
}
dsNames[tc.DeliveryServiceName(*ds.XMLID)] = struct{}{}
@@ -236,7 +183,7 @@ func GetConfigFileMeta(cfg config.TCCfg, serverNameOrID string) (string, error)
}
uriSignedDSes := []tc.DeliveryServiceName{}
- for _, ds := range deliveryServices {
+ for _, ds := range toData.DeliveryServices {
if ds.ID == nil {
continue
}
@@ -252,5 +199,6 @@ func GetConfigFileMeta(cfg config.TCCfg, serverNameOrID string) (string, error)
uriSignedDSes = append(uriSignedDSes, tc.DeliveryServiceName(*ds.XMLID))
}
- return atscfg.MakeMetaConfig(tc.CacheName(serverHostName), &serverInfo, toURL, toReverseProxyURL, locationParams, uriSignedDSes, scopeParams, dsNames), nil
+ metaObj := atscfg.MakeMetaObj(tc.CacheName(toData.Server.HostName), &serverInfo, toURL, toReverseProxyURL, locationParams, uriSignedDSes, scopeParams, dsNames)
+ return &metaObj, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/packages.go b/traffic_ops/ort/atstccfg/cfgfile/packages.go
index e2d0bdb..a41be67 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/packages.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/packages.go
@@ -20,61 +20,10 @@ package cfgfile
*/
import (
- "errors"
- "strconv"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileServerPackages(cfg config.TCCfg, serverNameOrID string) (string, error) {
- // TODO TOAPI add /servers?cdn=1 query param
- servers, err := toreq.GetServers(cfg)
- if err != nil {
- return "", errors.New("getting servers: " + err.Error())
- }
-
- server := tc.Server{ID: atscfg.InvalidID}
- if serverID, err := strconv.Atoi(serverNameOrID); err == nil {
- for _, toServer := range servers {
- if toServer.ID == serverID {
- server = toServer
- break
- }
- }
- } else {
- serverName := serverNameOrID
- for _, toServer := range servers {
- if toServer.HostName == serverName {
- server = toServer
- break
- }
- }
- }
- if server.ID == atscfg.InvalidID {
- return "", errors.New("server '" + serverNameOrID + " not found in servers")
- }
-
- profileParams, err := toreq.GetProfileParameters(cfg, server.Profile)
- if err != nil {
- return "", errors.New("getting profile '" + server.Profile + "' parameters: " + err.Error())
- }
- if len(profileParams) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params, to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- fileParams := map[string][]string{}
- for _, param := range profileParams {
- if param.ConfigFile != atscfg.PackagesParamConfigFile {
- continue
- }
- fileParams[param.Name] = append(fileParams[param.Name], param.Value)
- }
-
- txt := atscfg.MakePackages(fileParams)
- return txt, nil
+func GetConfigFileServerPackages(toData *TOData) (string, string, error) {
+ params := ParamsToMultiMap(FilterParams(toData.ServerParams, atscfg.PackagesParamConfigFile, "", "", ""))
+ return atscfg.MakePackages(params), atscfg.ContentTypePackages, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/parentdotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/parentdotconfig.go
index d0e59b4..d996c6c 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/parentdotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/parentdotconfig.go
@@ -30,54 +30,20 @@ import (
"github.com/apache/trafficcontrol/lib/go-atscfg"
"github.com/apache/trafficcontrol/lib/go-log"
"github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileServerParentDotConfig(cfg config.TCCfg, serverNameOrID string) (string, error) {
- // TODO TOAPI add /servers?cdn=1 query param
- servers, err := toreq.GetServers(cfg)
- if err != nil {
- return "", errors.New("getting servers: " + err.Error())
- }
-
- server := tc.Server{ID: atscfg.InvalidID}
- if serverID, err := strconv.Atoi(serverNameOrID); err == nil {
- for _, toServer := range servers {
- if toServer.ID == serverID {
- server = toServer
- break
- }
- }
- } else {
- serverName := serverNameOrID
- for _, toServer := range servers {
- if toServer.HostName == serverName {
- server = toServer
- break
- }
- }
- }
- if server.ID == atscfg.InvalidID {
- return "", errors.New("server '" + serverNameOrID + " not found in servers")
- }
-
- cacheGroups, err := toreq.GetCacheGroups(cfg)
- if err != nil {
- return "", errors.New("getting cachegroups: " + err.Error())
- }
-
+func GetConfigFileServerParentDotConfig(toData *TOData) (string, string, error) {
cgMap := map[string]tc.CacheGroupNullable{}
- for _, cg := range cacheGroups {
+ for _, cg := range toData.CacheGroups {
if cg.Name == nil {
- return "", errors.New("got cachegroup with nil name!'")
+ return "", "", errors.New("got cachegroup with nil name!'")
}
cgMap[*cg.Name] = cg
}
- serverCG, ok := cgMap[server.Cachegroup]
+ serverCG, ok := cgMap[toData.Server.Cachegroup]
if !ok {
- return "", errors.New("server '" + serverNameOrID + "' cachegroup '" + server.Cachegroup + "' not found in CacheGroups")
+ return "", "", errors.New("server '" + toData.Server.HostName + "' cachegroup '" + toData.Server.Cachegroup + "' not found in CacheGroups")
}
parentCGID := -1
@@ -85,15 +51,15 @@ func GetConfigFileServerParentDotConfig(cfg config.TCCfg, serverNameOrID string)
if serverCG.ParentName != nil && *serverCG.ParentName != "" {
parentCG, ok := cgMap[*serverCG.ParentName]
if !ok {
- return "", errors.New("server '" + serverNameOrID + "' cachegroup '" + server.Cachegroup + "' parent '" + *serverCG.ParentName + "' not found in CacheGroups")
+ return "", "", errors.New("server '" + toData.Server.HostName + "' cachegroup '" + toData.Server.Cachegroup + "' parent '" + *serverCG.ParentName + "' not found in CacheGroups")
}
if parentCG.ID == nil {
- return "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil ID!'")
+ return "", "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil ID!'")
}
parentCGID = *parentCG.ID
if parentCG.Type == nil {
- return "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil Type!'")
+ return "", "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil Type!'")
}
parentCGType = *parentCG.Type
}
@@ -103,47 +69,47 @@ func GetConfigFileServerParentDotConfig(cfg config.TCCfg, serverNameOrID string)
if serverCG.SecondaryParentName != nil && *serverCG.SecondaryParentName != "" {
parentCG, ok := cgMap[*serverCG.SecondaryParentName]
if !ok {
- return "", errors.New("server '" + serverNameOrID + "' cachegroup '" + server.Cachegroup + "' secondary parent '" + *serverCG.SecondaryParentName + "' not found in CacheGroups")
+ return "", "", errors.New("server '" + toData.Server.HostName + "' cachegroup '" + toData.Server.Cachegroup + "' secondary parent '" + *serverCG.SecondaryParentName + "' not found in CacheGroups")
}
if parentCG.ID == nil {
- return "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil ID!'")
+ return "", "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil ID!'")
}
secondaryParentCGID = *parentCG.ID
if parentCG.Type == nil {
- return "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil Type!'")
+ return "", "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil Type!'")
}
secondaryParentCGType = *parentCG.Type
}
serverInfo := atscfg.ServerInfo{
- CacheGroupID: server.CachegroupID,
- CDN: tc.CDNName(server.CDNName),
- CDNID: server.CDNID,
- DomainName: server.DomainName,
- HostName: server.HostName,
- ID: server.ID,
- IP: server.IPAddress,
+ CacheGroupID: toData.Server.CachegroupID,
+ CDN: tc.CDNName(toData.Server.CDNName),
+ CDNID: toData.Server.CDNID,
+ DomainName: toData.Server.DomainName,
+ HostName: toData.Server.HostName,
+ ID: toData.Server.ID,
+ IP: toData.Server.IPAddress,
ParentCacheGroupID: parentCGID,
ParentCacheGroupType: parentCGType,
- ProfileID: atscfg.ProfileID(server.ProfileID),
- ProfileName: server.Profile,
- Port: server.TCPPort,
+ ProfileID: atscfg.ProfileID(toData.Server.ProfileID),
+ ProfileName: toData.Server.Profile,
+ Port: toData.Server.TCPPort,
SecondaryParentCacheGroupID: secondaryParentCGID,
SecondaryParentCacheGroupType: secondaryParentCGType,
- Type: server.Type,
+ Type: toData.Server.Type,
}
parentCacheGroups := map[string]struct{}{}
if serverInfo.IsTopLevelCache() {
log.Infoln("This cache Is Top Level!")
- for _, cg := range cacheGroups {
+ for _, cg := range toData.CacheGroups {
if cg.Type == nil {
- return "", errors.New("cachegroup type is nil!")
+ return "", "", errors.New("cachegroup type is nil!")
}
if cg.Name == nil {
- return "", errors.New("cachegroup type is nil!")
+ return "", "", errors.New("cachegroup type is nil!")
}
if *cg.Type != tc.CacheGroupOriginTypeName {
@@ -152,18 +118,18 @@ func GetConfigFileServerParentDotConfig(cfg config.TCCfg, serverNameOrID string)
parentCacheGroups[*cg.Name] = struct{}{}
}
} else {
- if server.Cachegroup == "" {
- return "", errors.New("server cachegroup is nil!")
+ if toData.Server.Cachegroup == "" {
+ return "", "", errors.New("server cachegroup is nil!")
}
- for _, cg := range cacheGroups {
+ for _, cg := range toData.CacheGroups {
if cg.Type == nil {
- return "", errors.New("cachegroup type is nil!")
+ return "", "", errors.New("cachegroup type is nil!")
}
if cg.Name == nil {
- return "", errors.New("cachegroup type is nil!")
+ return "", "", errors.New("cachegroup type is nil!")
}
- if *cg.Name == server.Cachegroup {
+ if *cg.Name == toData.Server.Cachegroup {
if cg.ParentName != nil && *cg.ParentName != "" {
parentCacheGroups[*cg.ParentName] = struct{}{}
}
@@ -176,8 +142,8 @@ func GetConfigFileServerParentDotConfig(cfg config.TCCfg, serverNameOrID string)
}
cgServers := map[int]tc.Server{} // map[serverID]server
- for _, sv := range servers {
- if sv.CDNName != server.CDNName {
+ for _, sv := range toData.Servers {
+ if sv.CDNName != toData.Server.CDNName {
continue
}
if _, ok := parentCacheGroups[sv.Cachegroup]; !ok {
@@ -194,26 +160,18 @@ func GetConfigFileServerParentDotConfig(cfg config.TCCfg, serverNameOrID string)
cgServers[sv.ID] = sv
}
- cgServerIDs := []int{}
+ cgServerIDs := map[int]struct{}{}
for serverID, _ := range cgServers {
- cgServerIDs = append(cgServerIDs, serverID)
+ cgServerIDs[serverID] = struct{}{}
}
- cgServerIDs = append(cgServerIDs, server.ID)
+ cgServerIDs[toData.Server.ID] = struct{}{}
- serverCapabilities, err := toreq.GetServerCapabilitiesByID(cfg, cgServerIDs)
- if err != nil {
- return "", errors.New("getting server capabilities: " + err.Error())
- }
-
- cgDSServers, err := toreq.GetDeliveryServiceServers(cfg, nil, cgServerIDs)
- if err != nil {
- return "", errors.New("getting parent.config cachegroup parent server delivery service servers: " + err.Error())
- }
+ cgDSServers := FilterDSS(toData.DeliveryServiceServers, nil, cgServerIDs)
parentServerDSes := map[int]map[int]struct{}{} // map[serverID][dsID] // cgServerDSes
for _, dss := range cgDSServers {
if dss.Server == nil || dss.DeliveryService == nil {
- return "", errors.New("getting parent.config cachegroup parent server delivery service servers: got dss with nil members!" + err.Error())
+ return "", "", errors.New("getting parent.config cachegroup parent server delivery service servers: got dss with nil members!")
}
if parentServerDSes[*dss.Server] == nil {
parentServerDSes[*dss.Server] = map[int]struct{}{}
@@ -221,13 +179,8 @@ func GetConfigFileServerParentDotConfig(cfg config.TCCfg, serverNameOrID string)
parentServerDSes[*dss.Server][*dss.DeliveryService] = struct{}{}
}
- serverProfileParameters, err := toreq.GetServerProfileParameters(cfg, server.Profile)
- if err != nil {
- return "", errors.New("getting server profile '" + server.Profile + "' parameters: " + err.Error())
- }
-
atsVersionParam := ""
- for _, param := range serverProfileParameters {
+ for _, param := range toData.ServerParams {
if param.ConfigFile != "package" || param.Name != "trafficserver" {
continue
}
@@ -240,40 +193,12 @@ func GetConfigFileServerParentDotConfig(cfg config.TCCfg, serverNameOrID string)
atsMajorVer, err := atscfg.GetATSMajorVersionFromATSVersion(atsVersionParam)
if err != nil {
- return "", errors.New("getting ATS major version from version parameter (profile '" + server.Profile + "' configFile 'package' name 'trafficserver'): " + err.Error())
+ return "", "", errors.New("getting ATS major version from version parameter (profile '" + toData.Server.Profile + "' configFile 'package' name 'trafficserver'): " + err.Error())
}
- globalParams, err := toreq.GetGlobalParameters(cfg)
+ parentConfigParamsWithProfiles, err := TCParamsToParamsWithProfiles(toData.ParentConfigParams)
if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- toToolName := ""
- toURL := ""
- for _, param := range globalParams {
- if param.Name == "tm.toolname" {
- toToolName = param.Value
- } else if param.Name == "tm.url" {
- toURL = param.Value
- }
- if toToolName != "" && toURL != "" {
- break
- }
- }
-
- deliveryServices, err := toreq.GetCDNDeliveryServices(cfg, server.CDNID)
- if err != nil {
- return "", errors.New("getting delivery services: " + err.Error())
- }
-
- parentConfigParams, err := toreq.GetConfigFileParameters(cfg, "parent.config")
- if err != nil {
- return "", errors.New("getting parent.config parameters: " + err.Error())
- }
-
- parentConfigParamsWithProfiles, err := toreq.TCParamsToParamsWithProfiles(parentConfigParams)
- if err != nil {
- return "", errors.New("unmarshalling parent.config parameters profiles: " + err.Error())
+ return "", "", errors.New("unmarshalling parent.config parameters profiles: " + err.Error())
}
// this is an optimization, to avoid looping over all params, for every DS. Instead, we loop over all params only once, and put them in a profile map.
@@ -298,12 +223,7 @@ func GetConfigFileServerParentDotConfig(cfg config.TCCfg, serverNameOrID string)
}
}
- cdn, err := toreq.GetCDN(cfg, serverInfo.CDN)
- if err != nil {
- return "", errors.New("getting cdn '" + string(serverInfo.CDN) + "': " + err.Error())
- }
-
- serverCDNDomain := cdn.DomainName
+ serverCDNDomain := toData.CDN.DomainName
parentConfigServerCacheProfileParams := map[string]atscfg.ProfileCache{} // map[profileName]ProfileCache
for _, cgServer := range cgServers {
@@ -351,7 +271,7 @@ func GetConfigFileServerParentDotConfig(cfg config.TCCfg, serverNameOrID string)
}
dsIDMap := map[int]tc.DeliveryServiceNullable{}
- for _, ds := range deliveryServices {
+ for _, ds := range toData.DeliveryServices {
if ds.ID == nil {
log.Errorln("delivery services got nil ID!")
os.Exit(1)
@@ -385,19 +305,14 @@ func GetConfigFileServerParentDotConfig(cfg config.TCCfg, serverNameOrID string)
allDSes = append(allDSes, int(ds))
}
- dsRequiredCapabilities, err := toreq.GetDeliveryServiceRequiredCapabilitiesByID(cfg, allDSes)
- if err != nil {
- return "", errors.New("getting DS required capabilities: " + err.Error())
- }
-
parentConfigDSes := []atscfg.ParentConfigDSTopLevel{}
- for _, tcDS := range deliveryServices {
+ for _, tcDS := range toData.DeliveryServices {
if tcDS.ID == nil {
continue // TODO warn?
}
if !serverInfo.IsTopLevelCache() {
- if _, ok := parentServerDSes[server.ID][*tcDS.ID]; !ok {
+ if _, ok := parentServerDSes[toData.Server.ID][*tcDS.ID]; !ok {
continue // skip DSes not assigned to this server.
}
}
@@ -471,7 +386,7 @@ func GetConfigFileServerParentDotConfig(cfg config.TCCfg, serverNameOrID string)
}
}
- ds.RequiredCapabilities = dsRequiredCapabilities[*tcDS.ID]
+ ds.RequiredCapabilities = toData.DSRequiredCapabilities[*tcDS.ID]
parentConfigDSes = append(parentConfigDSes, ds)
}
@@ -506,7 +421,7 @@ func GetConfigFileServerParentDotConfig(cfg config.TCCfg, serverNameOrID string)
CDN: cgServer.CDNID,
TypeName: cgServer.Type,
Domain: cgServer.DomainName,
- Capabilities: serverCapabilities[cgServer.ID],
+ Capabilities: toData.ServerCapabilities[cgServer.ID],
}
if cgServer.Type == tc.OriginTypeName {
@@ -516,7 +431,7 @@ func GetConfigFileServerParentDotConfig(cfg config.TCCfg, serverNameOrID string)
// log.Warnln("ds %v has no origins! Skipping!\n", dsID) // TODO determine if this is normal
continue
}
- if atscfg.HasRequiredCapabilities(serverCapabilities[int(cgServer.ID)], dsRequiredCapabilities[dsID]) {
+ if atscfg.HasRequiredCapabilities(toData.ServerCapabilities[int(cgServer.ID)], toData.DSRequiredCapabilities[dsID]) {
orgHost := atscfg.OriginHost(orgURI.Host)
originServers[orgHost] = append(originServers[orgHost], realCGServer)
} else {
@@ -539,7 +454,7 @@ func GetConfigFileServerParentDotConfig(cfg config.TCCfg, serverNameOrID string)
parentInfos := atscfg.MakeParentInfo(&serverInfo, serverCDNDomain, profileCaches, originServers)
- return atscfg.MakeParentDotConfig(&serverInfo, atsMajorVer, toToolName, toURL, parentConfigDSes, serverParams, parentInfos), nil
+ return atscfg.MakeParentDotConfig(&serverInfo, atsMajorVer, toData.TOToolName, toData.TOURL, parentConfigDSes, serverParams, parentInfos), atscfg.ContentTypeParentDotConfig, nil
}
// GetDSOrigins takes a map[deliveryServiceID]DeliveryService, and returns a map[DeliveryServiceID]OriginURI.
diff --git a/traffic_ops/ort/atstccfg/cfgfile/plugindotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/plugindotconfig.go
index 48f45fd..e3209ed 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/plugindotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/plugindotconfig.go
@@ -20,45 +20,10 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileProfilePluginDotConfig(cfg config.TCCfg, profileNameOrID string) (string, error) {
- profileName, err := toreq.GetProfileNameFromProfileNameOrID(cfg, profileNameOrID)
- if err != nil {
- return "", errors.New("getting profile name from '" + profileNameOrID + "': " + err.Error())
- }
-
- profileParameters, err := toreq.GetProfileParameters(cfg, profileName)
- if err != nil {
- return "", errors.New("getting profile '" + profileName + "' parameters: " + err.Error())
- }
- if len(profileParameters) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params (and there should always be storage.config params), to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- paramData := map[string]string{}
- // TODO add configFile query param to profile/parameters endpoint, to only get needed data
- for _, param := range profileParameters {
- if param.ConfigFile != atscfg.PluginFileName {
- continue
- }
- if param.Name == "location" {
- continue
- }
- paramData[param.Name] = param.Value
- }
-
- return atscfg.MakePluginDotConfig(profileName, paramData, toToolName, toURL), nil
+func GetConfigFileProfilePluginDotConfig(toData *TOData) (string, string, error) {
+ params := ParamsToMap(FilterParams(toData.ServerParams, atscfg.PluginFileName, "", "", "location"))
+ return atscfg.MakePluginDotConfig(toData.Server.Profile, params, toData.TOToolName, toData.TOURL), atscfg.ContentTypePluginDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/recordsdotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/recordsdotconfig.go
index d322460..eae25cc 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/recordsdotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/recordsdotconfig.go
@@ -20,45 +20,10 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileProfileRecordsDotConfig(cfg config.TCCfg, profileNameOrID string) (string, error) {
- profileName, err := toreq.GetProfileNameFromProfileNameOrID(cfg, profileNameOrID)
- if err != nil {
- return "", errors.New("getting profile name from '" + profileNameOrID + "': " + err.Error())
- }
-
- profileParameters, err := toreq.GetProfileParameters(cfg, profileName)
- if err != nil {
- return "", errors.New("getting profile '" + profileName + "' parameters: " + err.Error())
- }
- if len(profileParameters) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params (and there should always be storage.config params), to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- paramData := map[string]string{}
- // TODO add configFile query param to profile/parameters endpoint, to only get needed data
- for _, param := range profileParameters {
- if param.ConfigFile != atscfg.RecordsFileName {
- continue
- }
- if param.Name == "location" {
- continue
- }
- paramData[param.Name] = param.Value
- }
-
- return atscfg.MakeRecordsDotConfig(profileName, paramData, toToolName, toURL), nil
+func GetConfigFileProfileRecordsDotConfig(toData *TOData) (string, string, error) {
+ params := ParamsToMap(FilterParams(toData.ServerParams, atscfg.RecordsFileName, "", "", "location"))
+ return atscfg.MakeRecordsDotConfig(toData.Server.Profile, params, toData.TOToolName, toData.TOURL), atscfg.ContentTypeRecordsDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/regexremapdotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/regexremapdotconfig.go
index c846a02..2876a2c 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/regexremapdotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/regexremapdotconfig.go
@@ -20,48 +20,27 @@ package cfgfile
*/
import (
- "errors"
"strings"
"github.com/apache/trafficcontrol/lib/go-atscfg"
"github.com/apache/trafficcontrol/lib/go-tc"
"github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileCDNRegexRemap(cfg config.TCCfg, cdnNameOrID string, fileName string) (string, error) {
+func GetConfigFileCDNRegexRemap(toData *TOData, fileName string) (string, string, error) {
configSuffix := `.config`
if !strings.HasPrefix(fileName, atscfg.RegexRemapPrefix) || !strings.HasSuffix(fileName, configSuffix) {
- return `{"alerts":[{"level":"error","text":"Error - regex remap file '` + fileName + `' not of the form 'regex_remap_*.config! Please file a bug with Traffic Control, this should never happen."}]}`, config.ErrBadRequest
+ return `{"alerts":[{"level":"error","text":"Error - regex remap file '` + fileName + `' not of the form 'regex_remap_*.config! Please file a bug with Traffic Control, this should never happen."}]}`, "", config.ErrBadRequest
}
+
dsName := strings.TrimSuffix(strings.TrimPrefix(fileName, atscfg.RegexRemapPrefix), configSuffix)
if dsName == "" {
- return `{"alerts":[{"level":"error","text":"Error - regex remap file '` + fileName + `' has no delivery service name!"}]}`, config.ErrBadRequest
- }
-
- cdnName, err := toreq.GetCDNNameFromCDNNameOrID(cfg, cdnNameOrID)
- if err != nil {
- return "", errors.New("getting CDN name from '" + cdnNameOrID + "': " + err.Error())
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- cdn, err := toreq.GetCDN(cfg, cdnName)
- if err != nil {
- return "", errors.New("getting cdn '" + string(cdnName) + "': " + err.Error())
- }
-
- dses, err := toreq.GetCDNDeliveryServices(cfg, cdn.ID)
- if err != nil {
- return "", errors.New("getting delivery services: " + err.Error())
+ return `{"alerts":[{"level":"error","text":"Error - regex remap file '` + fileName + `' has no delivery service name!"}]}`, "", config.ErrBadRequest
}
// only send the requested DS to atscfg. The atscfg.Make will work correctly even if we send it other DSes, but this will prevent atscfg.DeliveryServicesToCDNDSes from logging errors about AnyMap and Steering DSes without origins.
ds := tc.DeliveryServiceNullable{}
- for _, dsesDS := range dses {
+ for _, dsesDS := range toData.DeliveryServices {
if dsesDS.XMLID == nil {
continue // TODO log?
}
@@ -71,11 +50,10 @@ func GetConfigFileCDNRegexRemap(cfg config.TCCfg, cdnNameOrID string, fileName s
ds = dsesDS
}
if ds.ID == nil {
- return `{"alerts":[{"level":"error","text":"Error - delivery service '` + dsName + `' not found! Do you have a regex_remap_*.config location Parameter for a delivery service that doesn't exist?"}]}`, config.ErrNotFound
+ return `{"alerts":[{"level":"error","text":"Error - delivery service '` + dsName + `' not found! Do you have a regex_remap_*.config location Parameter for a delivery service that doesn't exist?"}]}`, "", config.ErrNotFound
}
cfgDSes := atscfg.DeliveryServicesToCDNDSes([]tc.DeliveryServiceNullable{ds})
- txt := atscfg.MakeRegexRemapDotConfig(cdnName, toToolName, toURL, fileName, cfgDSes)
- return txt, nil
+ return atscfg.MakeRegexRemapDotConfig(tc.CDNName(toData.Server.CDNName), toData.TOToolName, toData.TOURL, fileName, cfgDSes), atscfg.ContentTypeRegexRemapDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/regexrevalidatedotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/regexrevalidatedotconfig.go
index 9aed4d3..72e7830 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/regexrevalidatedotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/regexrevalidatedotconfig.go
@@ -20,62 +20,22 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
"github.com/apache/trafficcontrol/lib/go-log"
"github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/lib/go-util"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileCDNRegexRevalidateDotConfig(cfg config.TCCfg, cdnNameOrID string) (string, error) {
- cdnName, err := toreq.GetCDNNameFromCDNNameOrID(cfg, cdnNameOrID)
- if err != nil {
- return "", errors.New("getting CDN name from '" + cdnNameOrID + "': " + err.Error())
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- fileParamsWithoutProfiles, err := toreq.GetConfigFileParameters(cfg, atscfg.RegexRevalidateFileName)
- if err != nil {
- return "", errors.New("getting regexreval parameters: " + err.Error())
- }
-
- fileParams, err := toreq.TCParamsToParamsWithProfiles(fileParamsWithoutProfiles)
- if err != nil {
- return "", errors.New("unmarshalling regexreval parameters profiles: " + err.Error())
- }
-
+func GetConfigFileCDNRegexRevalidateDotConfig(toData *TOData) (string, string, error) {
params := map[string][]string{}
- for _, param := range fileParams {
- if !util.StrInArray(param.ProfileNames, tc.GlobalProfileName) {
- continue // TODO add profile query params to TO endpoint
+ for _, param := range toData.GlobalParams {
+ if param.ConfigFile != atscfg.RegexRevalidateFileName {
+ continue
}
params[param.Name] = append(params[param.Name], param.Value)
}
- allJobs, err := toreq.GetJobs(cfg) // TODO add cdn query param to jobs endpoint
- if err != nil {
- return "", errors.New("unmarshalling regexreval parameters profiles: " + err.Error())
- }
-
- cdn, err := toreq.GetCDN(cfg, cdnName)
- if err != nil {
- return "", errors.New("getting cdn '" + string(cdnName) + "': " + err.Error())
- }
-
- dses, err := toreq.GetCDNDeliveryServices(cfg, cdn.ID)
- if err != nil {
- return "", errors.New("getting delivery services: " + err.Error())
- }
-
dsNames := map[string]struct{}{}
- for _, ds := range dses {
+ for _, ds := range toData.DeliveryServices {
if ds.XMLID == nil {
log.Errorln("Regex Revalidate got Delivery Service from Traffic Ops with a nil xmlId! Skipping!")
continue
@@ -84,13 +44,12 @@ func GetConfigFileCDNRegexRevalidateDotConfig(cfg config.TCCfg, cdnNameOrID stri
}
jobs := []tc.Job{}
- for _, job := range allJobs {
+ for _, job := range toData.Jobs {
if _, ok := dsNames[job.DeliveryService]; !ok {
continue
}
jobs = append(jobs, job)
}
- txt := atscfg.MakeRegexRevalidateDotConfig(cdnName, params, toToolName, toURL, jobs)
- return txt, nil // TODO implement
+ return atscfg.MakeRegexRevalidateDotConfig(tc.CDNName(toData.Server.CDNName), params, toData.TOToolName, toData.TOURL, jobs), atscfg.ContentTypeRegexRevalidateDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/remapdotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/remapdotconfig.go
index 5839dce..a74b385 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/remapdotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/remapdotconfig.go
@@ -29,59 +29,13 @@ import (
"github.com/apache/trafficcontrol/lib/go-log"
"github.com/apache/trafficcontrol/lib/go-tc"
"github.com/apache/trafficcontrol/lib/go-util"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileServerRemapDotConfig(cfg config.TCCfg, serverNameOrID string) (string, error) {
+func GetConfigFileServerRemapDotConfig(toData *TOData) (string, string, error) {
// TODO TOAPI add /servers?cdn=1 query param
- servers, err := toreq.GetServers(cfg)
- if err != nil {
- return "", errors.New("getting servers: " + err.Error())
- }
-
- server := tc.Server{ID: atscfg.InvalidID}
- if serverID, err := strconv.Atoi(serverNameOrID); err == nil {
- for _, toServer := range servers {
- if toServer.ID == serverID {
- server = toServer
- break
- }
- }
- } else {
- serverName := serverNameOrID
- for _, toServer := range servers {
- if toServer.HostName == serverName {
- server = toServer
- break
- }
- }
- }
- if server.ID == atscfg.InvalidID {
- return "", errors.New("server '" + serverNameOrID + " not found in servers")
- }
-
- serverName := server.HostName
-
- cdn, err := toreq.GetCDN(cfg, tc.CDNName(server.CDNName))
- if err != nil {
- return "", errors.New("getting cdn '" + string(server.CDNName) + "': " + err.Error())
- }
-
- serverCDNDomain := cdn.DomainName
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- serverProfileParameters, err := toreq.GetServerProfileParameters(cfg, server.Profile)
- if err != nil {
- return "", errors.New("getting server profile '" + server.Profile + "' parameters: " + err.Error())
- }
atsVersionParam := ""
- for _, param := range serverProfileParameters {
+ for _, param := range toData.ServerParams {
if param.ConfigFile != "package" || param.Name != "trafficserver" {
continue
}
@@ -94,35 +48,27 @@ func GetConfigFileServerRemapDotConfig(cfg config.TCCfg, serverNameOrID string)
atsMajorVer, err := atscfg.GetATSMajorVersionFromATSVersion(atsVersionParam)
if err != nil {
- return "", errors.New("getting ATS major version from version parameter (profile '" + server.Profile + "' configFile 'package' name 'trafficserver'): " + err.Error())
+ return "", "", errors.New("getting ATS major version from version parameter (profile '" + toData.Server.Profile + "' configFile 'package' name 'trafficserver'): " + err.Error())
}
- deliveryServices, err := toreq.GetCDNDeliveryServices(cfg, server.CDNID)
- if err != nil {
- return "", errors.New("getting delivery services: " + err.Error())
- }
-
- dsIDs := []int{}
- for _, ds := range deliveryServices {
+ dsIDs := map[int]struct{}{}
+ for _, ds := range toData.DeliveryServices {
if ds.ID == nil {
// TODO log error?
continue
}
- dsIDs = append(dsIDs, *ds.ID)
+ dsIDs[*ds.ID] = struct{}{}
}
- isMid := strings.HasPrefix(server.Type, string(tc.CacheTypeMid))
+ isMid := strings.HasPrefix(toData.Server.Type, string(tc.CacheTypeMid))
- serverIDs := ([]int)(nil)
+ serverIDs := map[int]struct{}{}
if !isMid {
- // mids use all servers, so pass nil=all. Edges only use this current server
- serverIDs = append(serverIDs, server.ID)
+ // mids use all servers, so pass empty=all. Edges only use this current server
+ serverIDs[toData.Server.ID] = struct{}{}
}
- dsServers, err := toreq.GetDeliveryServiceServers(cfg, dsIDs, serverIDs)
- if err != nil {
- return "", errors.New("getting parent.config cachegroup parent server delivery service servers: " + err.Error())
- }
+ dsServers := FilterDSS(toData.DeliveryServiceServers, dsIDs, serverIDs)
dssMap := map[int]map[int]struct{}{} // set of map[dsID][serverID]
for _, dss := range dsServers {
@@ -142,7 +88,7 @@ func GetConfigFileServerRemapDotConfig(cfg config.TCCfg, serverNameOrID string)
}
filteredDSes := []tc.DeliveryServiceNullable{}
- for _, ds := range deliveryServices {
+ for _, ds := range toData.DeliveryServices {
if ds.ID == nil {
continue // TODO log?
}
@@ -158,13 +104,8 @@ func GetConfigFileServerRemapDotConfig(cfg config.TCCfg, serverNameOrID string)
filteredDSes = append(filteredDSes, ds)
}
- dsRegexes, err := toreq.GetDeliveryServiceRegexes(cfg)
- if err != nil {
- return "", errors.New("getting delivery service regexes: " + err.Error())
- }
-
dsRegexMap := map[tc.DeliveryServiceName][]tc.DeliveryServiceRegex{}
- for _, dsRegex := range dsRegexes {
+ for _, dsRegex := range toData.DeliveryServiceRegexes {
sort.Sort(DeliveryServiceRegexesSortByTypeThenSetNum(dsRegex.Regexes))
dsRegexMap[tc.DeliveryServiceName(dsRegex.DSName)] = dsRegex.Regexes
}
@@ -195,7 +136,7 @@ func GetConfigFileServerRemapDotConfig(cfg config.TCCfg, serverNameOrID string)
RoutingName: ds.RoutingName,
Pattern: util.StrPtr(dsRegex.Pattern),
RegexType: util.StrPtr(dsRegex.Type),
- Domain: util.StrPtr(serverCDNDomain), // note this is intentionally the CDN domain, not the DS or Server Domain. Must be the remap domain.
+ Domain: util.StrPtr(toData.CDN.DomainName), // note this is intentionally the CDN domain, not the DS or Server Domain. Must be the remap domain.
OriginShield: ds.OriginShield,
ProfileID: ds.ProfileID,
Protocol: ds.Protocol,
@@ -206,13 +147,8 @@ func GetConfigFileServerRemapDotConfig(cfg config.TCCfg, serverNameOrID string)
}
}
- serverProfileParams, err := toreq.GetProfileParameters(cfg, server.Profile)
- if err != nil {
- return "", errors.New("getting profile parameters from server (profile '" + server.Profile + ": " + err.Error())
- }
-
serverPackageParamData := map[string]string{}
- for _, param := range serverProfileParams {
+ for _, param := range toData.ServerParams {
if param.ConfigFile != "package" { // TODO put in const
continue
}
@@ -228,34 +164,29 @@ func GetConfigFileServerRemapDotConfig(cfg config.TCCfg, serverNameOrID string)
}
paramValue := param.Value
if paramValue == "STRING __HOSTNAME__" {
- paramValue = server.HostName + "." + server.DomainName // TODO strings.Replace to replace all anywhere, instead of just an exact match?
+ paramValue = toData.Server.HostName + "." + toData.Server.DomainName // TODO strings.Replace to replace all anywhere, instead of just an exact match?
}
serverPackageParamData[paramName] = paramValue
}
cacheURLParams := map[string]string{}
- for _, param := range serverProfileParams {
+ for _, param := range toData.ServerParams {
if param.ConfigFile != atscfg.CacheURLParameterConfigFile {
continue
}
if existingVal, ok := cacheURLParams[param.Name]; ok {
- log.Warnln("generating remap.config: server profile '" + server.Profile + "' cacheurl.config has multiple parameters for '" + param.Name + "' - using '" + existingVal + "' and ignoring the rest!")
+ log.Warnln("generating remap.config: server profile '" + toData.Server.Profile + "' cacheurl.config has multiple parameters for '" + param.Name + "' - using '" + existingVal + "' and ignoring the rest!")
continue
}
cacheURLParams[param.Name] = param.Value
}
- cacheKeyParams, err := toreq.GetConfigFileParameters(cfg, atscfg.CacheKeyParameterConfigFile)
- if err != nil {
- return "", errors.New("getting cache key parameters: " + err.Error())
- }
-
- cacheKeyParamsWithProfiles, err := toreq.TCParamsToParamsWithProfiles(cacheKeyParams)
+ cacheKeyParamsWithProfiles, err := TCParamsToParamsWithProfiles(toData.CacheKeyParams)
if err != nil {
- return "", errors.New("decoding cache key parameter profiles: " + err.Error())
+ return "", "", errors.New("decoding cache key parameter profiles: " + err.Error())
}
- cacheKeyParamsWithProfilesMap := toreq.ParameterWithProfilesToMap(cacheKeyParamsWithProfiles)
+ cacheKeyParamsWithProfilesMap := ParameterWithProfilesToMap(cacheKeyParamsWithProfiles)
dsProfileNamesToIDs := map[string]int{}
for _, ds := range filteredDSes {
@@ -285,22 +216,17 @@ func GetConfigFileServerRemapDotConfig(cfg config.TCCfg, serverNameOrID string)
// TODO put parentcg logic in func, to remove duplication with parent.config
- cacheGroups, err := toreq.GetCacheGroups(cfg)
- if err != nil {
- return "", errors.New("getting cachegroups: " + err.Error())
- }
-
cgMap := map[string]tc.CacheGroupNullable{}
- for _, cg := range cacheGroups {
+ for _, cg := range toData.CacheGroups {
if cg.Name == nil {
- return "", errors.New("got cachegroup with nil name!'")
+ return "", "", errors.New("got cachegroup with nil name!'")
}
cgMap[*cg.Name] = cg
}
- serverCG, ok := cgMap[server.Cachegroup]
+ serverCG, ok := cgMap[toData.Server.Cachegroup]
if !ok {
- return "", errors.New("server '" + serverNameOrID + "' cachegroup '" + server.Cachegroup + "' not found in CacheGroups")
+ return "", "", errors.New("server '" + toData.Server.HostName + "' cachegroup '" + toData.Server.Cachegroup + "' not found in CacheGroups")
}
parentCGID := -1
@@ -308,15 +234,15 @@ func GetConfigFileServerRemapDotConfig(cfg config.TCCfg, serverNameOrID string)
if serverCG.ParentName != nil && *serverCG.ParentName != "" {
parentCG, ok := cgMap[*serverCG.ParentName]
if !ok {
- return "", errors.New("server '" + serverNameOrID + "' cachegroup '" + server.Cachegroup + "' parent '" + *serverCG.ParentName + "' not found in CacheGroups")
+ return "", "", errors.New("server '" + toData.Server.HostName + "' cachegroup '" + toData.Server.Cachegroup + "' parent '" + *serverCG.ParentName + "' not found in CacheGroups")
}
if parentCG.ID == nil {
- return "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil ID!'")
+ return "", "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil ID!'")
}
parentCGID = *parentCG.ID
if parentCG.Type == nil {
- return "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil Type!'")
+ return "", "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil Type!'")
}
parentCGType = *parentCG.Type
}
@@ -326,41 +252,39 @@ func GetConfigFileServerRemapDotConfig(cfg config.TCCfg, serverNameOrID string)
if serverCG.SecondaryParentName != nil && *serverCG.SecondaryParentName != "" {
parentCG, ok := cgMap[*serverCG.SecondaryParentName]
if !ok {
- return "", errors.New("server '" + serverNameOrID + "' cachegroup '" + server.Cachegroup + "' secondary parent '" + *serverCG.SecondaryParentName + "' not found in CacheGroups")
+ return "", "", errors.New("server '" + toData.Server.HostName + "' cachegroup '" + toData.Server.Cachegroup + "' secondary parent '" + *serverCG.SecondaryParentName + "' not found in CacheGroups")
}
if parentCG.ID == nil {
- return "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil ID!'")
+ return "", "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil ID!'")
}
secondaryParentCGID = *parentCG.ID
if parentCG.Type == nil {
- return "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil Type!'")
+ return "", "", errors.New("got cachegroup '" + *parentCG.Name + "' with nil Type!'")
}
secondaryParentCGType = *parentCG.Type
}
serverInfo := &atscfg.ServerInfo{
- CacheGroupID: server.CachegroupID,
- CDN: tc.CDNName(server.CDNName),
- CDNID: server.CDNID,
- DomainName: serverCDNDomain, // note this is intentionally the CDN domain, not the server domain. It's what's remapped to.
- HostName: server.HostName,
- ID: server.ID,
- IP: server.IPAddress,
+ CacheGroupID: toData.Server.CachegroupID,
+ CDN: tc.CDNName(toData.Server.CDNName),
+ CDNID: toData.Server.CDNID,
+ DomainName: toData.CDN.DomainName, // note this is intentionally the CDN domain, not the server domain. It's what's remapped to.
+ HostName: toData.Server.HostName,
+ ID: toData.Server.ID,
+ IP: toData.Server.IPAddress,
ParentCacheGroupID: parentCGID,
ParentCacheGroupType: parentCGType,
- ProfileID: atscfg.ProfileID(server.ProfileID),
- ProfileName: server.Profile,
- Port: server.TCPPort,
- HTTPSPort: server.HTTPSPort,
+ ProfileID: atscfg.ProfileID(toData.Server.ProfileID),
+ ProfileName: toData.Server.Profile,
+ Port: toData.Server.TCPPort,
+ HTTPSPort: toData.Server.HTTPSPort,
SecondaryParentCacheGroupID: secondaryParentCGID,
SecondaryParentCacheGroupType: secondaryParentCGType,
- Type: server.Type,
+ Type: toData.Server.Type,
}
-
- txt := atscfg.MakeRemapDotConfig(tc.CacheName(serverName), toToolName, toURL, atsMajorVer, cacheURLParams, dsProfilesCacheKeyConfigParams, serverPackageParamData, serverInfo, remapConfigDSData)
- return txt, nil
+ return atscfg.MakeRemapDotConfig(tc.CacheName(toData.Server.HostName), toData.TOToolName, toData.TOURL, atsMajorVer, cacheURLParams, dsProfilesCacheKeyConfigParams, serverPackageParamData, serverInfo, remapConfigDSData), atscfg.ContentTypeRemapDotConfig, nil
}
type DeliveryServiceRegexesSortByTypeThenSetNum []tc.DeliveryServiceRegex
diff --git a/traffic_ops/ort/atstccfg/cfgfile/routing.go b/traffic_ops/ort/atstccfg/cfgfile/routing.go
new file mode 100644
index 0000000..c3459f9
--- /dev/null
+++ b/traffic_ops/ort/atstccfg/cfgfile/routing.go
@@ -0,0 +1,190 @@
+package cfgfile
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import (
+ "errors"
+ "strings"
+ "time"
+
+ "github.com/apache/trafficcontrol/lib/go-log"
+ "github.com/apache/trafficcontrol/lib/go-tc"
+ "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
+)
+
+var scopeConfigFileFuncs = map[string]func(toData *TOData, fileName string) (string, string, error){
+ "cdns": GetConfigFileCDN,
+ "servers": GetConfigFileServer,
+ "profiles": GetConfigFileProfile,
+}
+
+// GetConfigFile returns the text of the generated config file, the MIME Content Type of the config file, and any error.
+func GetConfigFile(toData *TOData, fileInfo tc.ATSConfigMetaDataConfigFile) (string, string, error) {
+ path := fileInfo.APIURI
+ // TODO remove the URL path parsing. It's a legacy from when config files were endpoints in the meta config.
+ // We should replace it with actually calling the right file and name directly.
+ start := time.Now()
+ defer func() {
+ log.Infof("GetConfigFile %v took %v\n", path, time.Since(start).Round(time.Millisecond))
+ }()
+
+ pathParts := strings.Split(path, "/")
+ if len(pathParts) < 8 {
+ return "", "", errors.New("unknown config file '" + path + "'")
+ }
+ scope := pathParts[3]
+ resource := pathParts[4]
+ fileName := pathParts[7]
+
+ log.Infoln("GetConfigFile scope '" + scope + "' resource '" + resource + "' fileName '" + fileName + "'")
+
+ if scopeConfigFileFunc, ok := scopeConfigFileFuncs[scope]; ok {
+ return scopeConfigFileFunc(toData, fileName)
+ }
+
+ return "", "", errors.New("unknown config file '" + fileInfo.APIURI + "'")
+}
+
+type ConfigFilePrefixSuffixFunc struct {
+ Prefix string
+ Suffix string
+ Func func(toData *TOData, fileName string) (string, string, error)
+}
+
+func GetConfigFileCDN(toData *TOData, fileName string) (string, string, error) {
+ log.Infoln("GetConfigFileCDN cdn '" + toData.Server.CDNName + "' fileName '" + fileName + "'")
+
+ txt := ""
+ contentType := ""
+ err := error(nil)
+ if getCfgFunc, ok := CDNConfigFileFuncs()[fileName]; ok {
+ txt, contentType, err = getCfgFunc(toData)
+ } else {
+ for _, prefixSuffixFunc := range ConfigFileCDNPrefixSuffixFuncs {
+ if strings.HasPrefix(fileName, prefixSuffixFunc.Prefix) && strings.HasSuffix(fileName, prefixSuffixFunc.Suffix) && len(fileName) > len(prefixSuffixFunc.Prefix)+len(prefixSuffixFunc.Suffix) {
+ txt, contentType, err = prefixSuffixFunc.Func(toData, fileName)
+ break
+ }
+ }
+ }
+
+ if err == nil && txt == "" {
+ err = config.ErrNotFound
+ }
+
+ if err != nil {
+ return "", "", err
+ }
+ return txt, contentType, nil
+}
+
+func GetConfigFileProfile(toData *TOData, fileName string) (string, string, error) {
+ log.Infoln("GetConfigFileProfile profile '" + toData.Server.Profile + "' fileName '" + fileName + "'")
+
+ txt := ""
+ contentType := ""
+ err := error(nil)
+ if getCfgFunc, ok := ProfileConfigFileFuncs()[fileName]; ok {
+ txt, contentType, err = getCfgFunc(toData)
+ } else if strings.HasPrefix(fileName, "url_sig_") && strings.HasSuffix(fileName, ".config") && len(fileName) > len("url_sig_")+len(".config") {
+ txt, contentType, err = GetConfigFileProfileURLSigConfig(toData, fileName)
+ } else if strings.HasPrefix(fileName, "uri_signing_") && strings.HasSuffix(fileName, ".config") && len(fileName) > len("uri_signing")+len(".config") {
+ txt, contentType, err = GetConfigFileProfileURISigningConfig(toData, fileName)
+ } else {
+ txt, contentType, err = GetConfigFileProfileUnknownConfig(toData, fileName)
+ }
+
+ if err != nil {
+ return "", "", err
+ }
+ return txt, contentType, nil
+}
+
+// ConfigFileFuncs returns a map[scope][configFile]configFileFunc.
+func ConfigFileFuncs() map[string]map[string]func(toData *TOData) (string, string, error) {
+ return map[string]map[string]func(toData *TOData) (string, string, error){
+ "cdns": CDNConfigFileFuncs(),
+ "servers": ServerConfigFileFuncs(),
+ "profiles": ProfileConfigFileFuncs(),
+ }
+}
+
+func CDNConfigFileFuncs() map[string]func(toData *TOData) (string, string, error) {
+ return map[string]func(toData *TOData) (string, string, error){
+ "regex_revalidate.config": GetConfigFileCDNRegexRevalidateDotConfig,
+ "bg_fetch.config": GetConfigFileCDNBGFetchDotConfig,
+ "ssl_multicert.config": GetConfigFileCDNSSLMultiCertDotConfig,
+ "cacheurl.config": GetConfigFileCDNCacheURLPlain,
+ }
+}
+
+var ConfigFileCDNPrefixSuffixFuncs = []ConfigFilePrefixSuffixFunc{
+ {"hdr_rw_mid_", ".config", GetConfigFileCDNHeaderRewriteMid},
+ {"hdr_rw_", ".config", GetConfigFileCDNHeaderRewrite},
+ {"cacheurl", ".config", GetConfigFileCDNCacheURL},
+ {"regex_remap_", ".config", GetConfigFileCDNRegexRemap},
+ {"set_dscp_", ".config", GetConfigFileCDNSetDSCP},
+}
+
+func ProfileConfigFileFuncs() map[string]func(toData *TOData) (string, string, error) {
+ return map[string]func(toData *TOData) (string, string, error){
+ "12M_facts": GetConfigFileProfile12MFacts,
+ "50-ats.rules": GetConfigFileProfileATSDotRules,
+ "astats.config": GetConfigFileProfileAstatsDotConfig,
+ "cache.config": GetConfigFileProfileCacheDotConfig,
+ "drop_qstring.config": GetConfigFileProfileDropQStringDotConfig,
+ "logging.config": GetConfigFileProfileLoggingDotConfig,
+ "logging.yaml": GetConfigFileProfileLoggingDotYAML,
+ "logs_xml.config": GetConfigFileProfileLogsXMLDotConfig,
+ "plugin.config": GetConfigFileProfilePluginDotConfig,
+ "records.config": GetConfigFileProfileRecordsDotConfig,
+ "storage.config": GetConfigFileProfileStorageDotConfig,
+ "sysctl.conf": GetConfigFileProfileSysCtlDotConf,
+ "volume.config": GetConfigFileProfileVolumeDotConfig,
+ }
+}
+
+func ServerConfigFileFuncs() map[string]func(toData *TOData) (string, string, error) {
+ return map[string]func(toData *TOData) (string, string, error){
+ "parent.config": GetConfigFileServerParentDotConfig,
+ "remap.config": GetConfigFileServerRemapDotConfig,
+ "cache.config": GetConfigFileServerCacheDotConfig,
+ "ip_allow.config": GetConfigFileServerIPAllowDotConfig,
+ "hosting.config": GetConfigFileServerHostingDotConfig,
+ "packages": GetConfigFileServerPackages,
+ "chkconfig": GetConfigFileServerChkconfig,
+ }
+}
+
+func GetConfigFileServer(toData *TOData, fileName string) (string, string, error) {
+ log.Infoln("GetConfigFileServer server '" + toData.Server.HostName + "' fileName '" + fileName + "'")
+ txt := ""
+ contentType := ""
+ err := error(nil)
+ if getCfgFunc, ok := ServerConfigFileFuncs()[fileName]; ok {
+ txt, contentType, err = getCfgFunc(toData)
+ } else {
+ txt, contentType, err = GetConfigFileServerUnknownConfig(toData, fileName)
+ }
+ if err != nil {
+ return "", "", err
+ }
+ return txt, contentType, nil
+}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/servercachedotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/servercachedotconfig.go
index e57588d..a33eb2a 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/servercachedotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/servercachedotconfig.go
@@ -21,57 +21,25 @@ package cfgfile
import (
"errors"
- "strconv"
"strings"
"github.com/apache/trafficcontrol/lib/go-atscfg"
"github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
const ServerCacheDotConfigIncludeInactiveDSes = false // TODO move to lib/go-atscfg
-func GetConfigFileServerCacheDotConfig(cfg config.TCCfg, serverNameOrID string) (string, error) {
+func GetConfigFileServerCacheDotConfig(toData *TOData) (string, string, error) {
// TODO TOAPI add /servers?cdn=1 query param
- servers, err := toreq.GetServers(cfg)
- if err != nil {
- return "", errors.New("getting servers: " + err.Error())
- }
-
- server := tc.Server{ID: atscfg.InvalidID}
- if serverID, err := strconv.Atoi(serverNameOrID); err == nil {
- for _, toServer := range servers {
- if toServer.ID == serverID {
- server = toServer
- break
- }
- }
- } else {
- serverName := serverNameOrID
- for _, toServer := range servers {
- if toServer.HostName == serverName {
- server = toServer
- break
- }
- }
- }
- if server.ID == atscfg.InvalidID {
- return "", errors.New("server '" + serverNameOrID + " not found in servers")
- }
- if !strings.HasPrefix(string(server.Type), tc.MidTypePrefix) {
+ // TODO remove this, we generated the scope, we know it's right? Or should we have an extra safety check?
+ if !strings.HasPrefix(string(toData.Server.Type), tc.MidTypePrefix) {
// emulates Perl
- return "", errors.New("Error - incorrect file scope for route used. Please use the profiles route.")
- }
-
- dses, err := toreq.GetCDNDeliveryServices(cfg, server.CDNID)
- if err != nil {
- return "", errors.New("getting delivery services: " + err.Error())
+ return "", "", errors.New("Error - incorrect file scope for route used. Please use the profiles route.")
}
dsData := map[tc.DeliveryServiceName]atscfg.ServerCacheConfigDS{}
- for _, ds := range dses {
+ for _, ds := range toData.DeliveryServices {
if ds.XMLID == nil || ds.Active == nil || ds.OrgServerFQDN == nil || ds.Type == nil {
// TODO orgserverfqdn is nil for some DSes - MSO? Verify.
continue
@@ -83,13 +51,7 @@ func GetConfigFileServerCacheDotConfig(cfg config.TCCfg, serverNameOrID string)
dsData[tc.DeliveryServiceName(*ds.XMLID)] = atscfg.ServerCacheConfigDS{OrgServerFQDN: *ds.OrgServerFQDN, Type: *ds.Type}
}
- serverName := tc.CacheName(server.HostName)
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
+ serverName := tc.CacheName(toData.Server.HostName)
- txt := atscfg.MakeServerCacheDotConfig(serverName, toToolName, toURL, dsData)
- return txt, nil
+ return atscfg.MakeServerCacheDotConfig(serverName, toData.TOToolName, toData.TOURL, dsData), atscfg.ContentTypeCacheDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/serverunknownconfig.go b/traffic_ops/ort/atstccfg/cfgfile/serverunknownconfig.go
index 0f17a9e..de343a6 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/serverunknownconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/serverunknownconfig.go
@@ -20,68 +20,11 @@ package cfgfile
*/
import (
- "errors"
- "strconv"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
"github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileServerUnknownConfig(cfg config.TCCfg, serverNameOrID string, fileName string) (string, error) {
- servers, err := toreq.GetServers(cfg)
- if err != nil {
- return "", errors.New("getting servers: " + err.Error())
- }
-
- server := tc.Server{ID: atscfg.InvalidID}
- if serverID, err := strconv.Atoi(serverNameOrID); err == nil {
- for _, toServer := range servers {
- if toServer.ID == serverID {
- server = toServer
- break
- }
- }
- } else {
- serverName := serverNameOrID
- for _, toServer := range servers {
- if toServer.HostName == serverName {
- server = toServer
- break
- }
- }
- }
- if server.ID == atscfg.InvalidID {
- return "", errors.New("server '" + serverNameOrID + " not found in servers")
- }
-
- serverName := tc.CacheName(server.HostName)
- serverDomain := server.DomainName
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- profileParams, err := toreq.GetProfileParameters(cfg, server.Profile)
- if err != nil {
- return "", errors.New("getting profile '" + server.Profile + "' parameters: " + err.Error())
- }
- if len(profileParams) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params, to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- fileParams := map[string][]string{}
-
- for _, param := range profileParams {
- if param.ConfigFile != fileName {
- continue
- }
- fileParams[param.Name] = append(fileParams[param.Name], param.Value)
- }
-
- return atscfg.MakeServerUnknown(serverName, serverDomain, toToolName, toURL, fileParams), nil
+func GetConfigFileServerUnknownConfig(toData *TOData, fileName string) (string, string, error) {
+ params := ParamsToMultiMap(FilterParams(toData.ServerParams, fileName, "", "", ""))
+ return atscfg.MakeServerUnknown(tc.CacheName(toData.Server.HostName), toData.Server.DomainName, toData.TOToolName, toData.TOURL, params), atscfg.ContentTypeServerUnknownConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/setdscpdotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/setdscpdotconfig.go
index 7089035..e20bbde 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/setdscpdotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/setdscpdotconfig.go
@@ -20,30 +20,17 @@ package cfgfile
*/
import (
- "errors"
"strings"
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
+ "github.com/apache/trafficcontrol/lib/go-tc"
)
-func GetConfigFileCDNSetDSCP(cfg config.TCCfg, cdnNameOrID string, fileName string) (string, error) {
- cdnName, err := toreq.GetCDNNameFromCDNNameOrID(cfg, cdnNameOrID)
- if err != nil {
- return "", errors.New("getting CDN name from '" + cdnNameOrID + "': " + err.Error())
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
+func GetConfigFileCDNSetDSCP(toData *TOData, fileName string) (string, string, error) {
// TODO verify prefix, suffix, and that it's a number? Perl doesn't.
dscpNumStr := fileName
dscpNumStr = strings.TrimPrefix(dscpNumStr, "set_dscp_")
dscpNumStr = strings.TrimSuffix(dscpNumStr, ".config")
- txt := atscfg.MakeSetDSCPDotConfig(cdnName, toToolName, toURL, dscpNumStr)
- return txt, nil
+ return atscfg.MakeSetDSCPDotConfig(tc.CDNName(toData.Server.CDNName), toData.TOToolName, toData.TOURL, dscpNumStr), atscfg.ContentTypeSetDSCPDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/sslkeys.go b/traffic_ops/ort/atstccfg/cfgfile/sslkeys.go
new file mode 100644
index 0000000..3ba773c
--- /dev/null
+++ b/traffic_ops/ort/atstccfg/cfgfile/sslkeys.go
@@ -0,0 +1,76 @@
+package cfgfile
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import (
+ "encoding/base64"
+
+ "github.com/apache/trafficcontrol/lib/go-atscfg"
+ "github.com/apache/trafficcontrol/lib/go-log"
+ "github.com/apache/trafficcontrol/lib/go-tc"
+)
+
+func GetSSLCertsAndKeyFiles(toData *TOData) ([]ATSConfigFile, error) {
+ dses := atscfg.DeliveryServicesToSSLMultiCertDSes(toData.DeliveryServices)
+ dses = atscfg.GetSSLMultiCertDotConfigDeliveryServices(dses)
+
+ configs := []ATSConfigFile{}
+ for _, keys := range toData.SSLKeys {
+ dsName := tc.DeliveryServiceName(keys.DeliveryService)
+ ds, ok := dses[dsName]
+ if !ok {
+ continue
+ }
+
+ cert, err := base64.StdEncoding.DecodeString(keys.Certificate.Crt)
+ if err != nil {
+ log.Errorln("Delivery Service '" + string(dsName) + "' skipping HTTPS certificate! Failed to decode cert: " + err.Error())
+ continue
+ }
+ if len(cert) > 0 && cert[len(cert)-1] != '\n' {
+ cert = append(cert, '\n') // it's going to be a file, needs a trailing newline to be POSIX-compliant.
+ }
+
+ key, err := base64.StdEncoding.DecodeString(keys.Certificate.Key)
+ if err != nil {
+ log.Errorln("Delivery Service '" + string(dsName) + "' skipping HTTPS certificate! Failed to decode key: " + err.Error())
+ continue
+ }
+ if len(key) > 0 && key[len(key)-1] != '\n' {
+ key = append(key, '\n') // it's going to be a file, needs a trailing newline to be POSIX-compliant.
+ }
+
+ certName, keyName := atscfg.GetSSLMultiCertDotConfigCertAndKeyName(dsName, ds)
+
+ keyFile := ATSConfigFile{}
+ keyFile.FileNameOnDisk = keyName
+ keyFile.Location = "/opt/trafficserver/etc/trafficserver/ssl/" // TODO read config, don't hard code
+ keyFile.Text = string(key)
+ configs = append(configs, keyFile)
+
+ certFile := ATSConfigFile{}
+ certFile.FileNameOnDisk = certName
+ certFile.Location = "/opt/trafficserver/etc/trafficserver/ssl/" // TODO read config, don't hard code
+ certFile.Text = string(cert)
+ configs = append(configs, certFile)
+ }
+
+ return configs, nil
+}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/sslmulticertdotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/sslmulticertdotconfig.go
index e27205c..3eee6db 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/sslmulticertdotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/sslmulticertdotconfig.go
@@ -20,46 +20,12 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
"github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileCDNSSLMultiCertDotConfig(cfg config.TCCfg, cdnNameOrID string) (string, error) {
- cdnName, err := toreq.GetCDNNameFromCDNNameOrID(cfg, cdnNameOrID)
- if err != nil {
- return "", errors.New("getting CDN name from '" + cdnNameOrID + "': " + err.Error())
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- cdn, err := toreq.GetCDN(cfg, cdnName)
- if err != nil {
- return "", errors.New("getting cdn '" + string(cdnName) + "': " + err.Error())
- }
-
- dses, err := toreq.GetCDNDeliveryServices(cfg, cdn.ID)
- if err != nil {
- return "", errors.New("getting delivery services: " + err.Error())
- }
-
- filteredDSes := []tc.DeliveryServiceNullable{}
- for _, ds := range dses {
- // ANY_MAP and STEERING DSes don't have origins, and thus can't be put into the ssl config.
- if ds.Type != nil && (*ds.Type == tc.DSTypeAnyMap || *ds.Type == tc.DSTypeSteering) {
- continue
- }
- filteredDSes = append(filteredDSes, ds)
- }
-
- cfgDSes := atscfg.DeliveryServicesToSSLMultiCertDSes(filteredDSes)
+func GetConfigFileCDNSSLMultiCertDotConfig(toData *TOData) (string, string, error) {
+ cfgDSes := atscfg.DeliveryServicesToSSLMultiCertDSes(toData.DeliveryServices)
- txt := atscfg.MakeSSLMultiCertDotConfig(cdnName, toToolName, toURL, cfgDSes)
- return txt, nil
+ return atscfg.MakeSSLMultiCertDotConfig(tc.CDNName(toData.Server.CDNName), toData.TOToolName, toData.TOURL, cfgDSes), atscfg.ContentTypeSSLMultiCertDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/storagedotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/storagedotconfig.go
index c0aebea..f15e488 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/storagedotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/storagedotconfig.go
@@ -20,47 +20,12 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
const StorageFileName = "storage.config"
-func GetConfigFileProfileStorageDotConfig(cfg config.TCCfg, profileNameOrID string) (string, error) {
- profileName, err := toreq.GetProfileNameFromProfileNameOrID(cfg, profileNameOrID)
- if err != nil {
- return "", errors.New("getting profile name from '" + profileNameOrID + "': " + err.Error())
- }
-
- profileParameters, err := toreq.GetProfileParameters(cfg, profileName)
- if err != nil {
- return "", errors.New("getting profile '" + profileName + "' parameters: " + err.Error())
- }
- if len(profileParameters) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params (and there should always be volume.config params), to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- paramData := map[string]string{}
- // TODO add configFile query param to profile/parameters endpoint, to only get needed data
- for _, param := range profileParameters {
- if param.ConfigFile != StorageFileName {
- continue
- }
- if param.Name == "location" {
- continue
- }
- paramData[param.Name] = param.Value
- }
-
- return atscfg.MakeStorageDotConfig(profileName, paramData, toToolName, toURL), nil
+func GetConfigFileProfileStorageDotConfig(toData *TOData) (string, string, error) {
+ params := ParamsToMap(FilterParams(toData.ServerParams, StorageFileName, "", "", "location"))
+ return atscfg.MakeStorageDotConfig(toData.Server.Profile, params, toData.TOToolName, toData.TOURL), atscfg.ContentTypeStorageDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/sysctldotconf.go b/traffic_ops/ort/atstccfg/cfgfile/sysctldotconf.go
index 545030d..fea54ff 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/sysctldotconf.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/sysctldotconf.go
@@ -20,37 +20,13 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileProfileSysCtlDotConf(cfg config.TCCfg, profileNameOrID string) (string, error) {
- profileName, err := toreq.GetProfileNameFromProfileNameOrID(cfg, profileNameOrID)
- if err != nil {
- return "", errors.New("getting profile name from '" + profileNameOrID + "': " + err.Error())
- }
-
- profileParameters, err := toreq.GetProfileParameters(cfg, profileName)
- if err != nil {
- return "", errors.New("getting profile '" + profileName + "' parameters: " + err.Error())
- }
- if len(profileParameters) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params (and there should always be storage.config params), to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
+func GetConfigFileProfileSysCtlDotConf(toData *TOData) (string, string, error) {
paramData := map[string]string{}
// TODO add configFile query param to profile/parameters endpoint, to only get needed data
- for _, param := range profileParameters {
+ for _, param := range toData.ServerParams {
if param.ConfigFile != atscfg.SysctlFileName {
continue
}
@@ -60,5 +36,5 @@ func GetConfigFileProfileSysCtlDotConf(cfg config.TCCfg, profileNameOrID string)
paramData[param.Name] = param.Value
}
- return atscfg.MakeSysCtlDotConf(profileName, paramData, toToolName, toURL), nil
+ return atscfg.MakeSysCtlDotConf(toData.Server.Profile, paramData, toData.TOToolName, toData.TOURL), atscfg.ContentTypeSysctlDotConf, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/unknownconfig.go b/traffic_ops/ort/atstccfg/cfgfile/unknownconfig.go
index 74c5729..aae7fc2 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/unknownconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/unknownconfig.go
@@ -20,41 +20,13 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
"github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
-func GetConfigFileProfileUnknownConfig(cfg config.TCCfg, profileNameOrID string, fileName string) (string, error) {
- profileName, err := toreq.GetProfileNameFromProfileNameOrID(cfg, profileNameOrID)
- if err != nil {
- return "", errors.New("getting profile name from '" + profileNameOrID + "': " + err.Error())
- }
-
- profileParameters, err := toreq.GetProfileParameters(cfg, profileName)
- if err != nil {
- return "", errors.New("getting profile '" + profileName + "' parameters: " + err.Error())
- }
- if len(profileParameters) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params (and there should always be storage.config params), to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- scopeParams, err := toreq.GetParametersByName(cfg, "scope")
- if err != nil {
- return "", errors.New("getting scope parameters: " + err.Error())
- }
-
+func GetConfigFileProfileUnknownConfig(toData *TOData, fileName string) (string, string, error) {
inScope := false
- for _, scopeParam := range scopeParams {
+ for _, scopeParam := range toData.ScopeParams {
if scopeParam.ConfigFile != fileName {
continue
}
@@ -64,22 +36,9 @@ func GetConfigFileProfileUnknownConfig(cfg config.TCCfg, profileNameOrID string,
inScope = true
break
}
-
if !inScope {
- return `{"alerts":[{"level":"error","text":"Error - incorrect file scope for route used. Please use the servers route."}]}`, config.ErrBadRequest
+ return `{"alerts":[{"level":"error","text":"Error - incorrect file scope for route used. Please use the servers route."}]}`, "", config.ErrBadRequest
}
-
- paramData := map[string]string{}
- // TODO add configFile query param to profile/parameters endpoint, to only get needed data
- for _, param := range profileParameters {
- if param.ConfigFile != fileName {
- continue
- }
- if param.Name == "location" {
- continue
- }
- paramData[param.Name] = param.Value
- }
-
- return atscfg.MakeUnknownConfig(profileName, paramData, toToolName, toURL), nil
+ params := ParamsToMap(FilterParams(toData.ServerParams, fileName, "", "", "location"))
+ return atscfg.MakeUnknownConfig(toData.Server.Profile, params, toData.TOToolName, toData.TOURL), atscfg.ContentTypeUnknownConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/cfgfile/urisigningconfig.go b/traffic_ops/ort/atstccfg/cfgfile/urisigningconfig.go
index 409653a..bd7d3f8 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/urisigningconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/urisigningconfig.go
@@ -24,23 +24,22 @@ import (
"strings"
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
+ "github.com/apache/trafficcontrol/lib/go-tc"
)
-func GetConfigFileProfileURISigningConfig(cfg config.TCCfg, profileNameOrID string, fileName string) (string, error) {
+func GetConfigFileProfileURISigningConfig(toData *TOData, fileName string) (string, string, error) {
dsName := GetDSFromURISigningConfigFileName(fileName)
if dsName == "" {
// extra safety, this should never happen, the routing shouldn't get here
- return "", errors.New("getting ds name: malformed config file '" + fileName + "'")
+ return "", "", errors.New("getting ds name: malformed config file '" + fileName + "'")
}
- uriSigningKeys, err := toreq.GetURISigningKeys(cfg, dsName)
- if err != nil {
- return "", errors.New("getting uri signing keys for ds '" + dsName + "': " + err.Error())
+ uriSigningKeys, ok := toData.URISigningKeys[tc.DeliveryServiceName(dsName)]
+ if !ok {
+ return "", "", errors.New("no keys fetched for ds '" + dsName + "!")
}
- return atscfg.MakeURISigningConfig(uriSigningKeys), nil
+ return atscfg.MakeURISigningConfig(uriSigningKeys), atscfg.ContentTypeURISigningDotConfig, nil
}
// GetDSFromURISigningConfigFileName returns the DS of a URI Signing config file name.
diff --git a/traffic_ops/ort/atstccfg/cfgfile/urlsigconfig.go b/traffic_ops/ort/atstccfg/cfgfile/urlsigconfig.go
index 46e82df..986598b 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/urlsigconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/urlsigconfig.go
@@ -24,34 +24,13 @@ import (
"strings"
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
+ "github.com/apache/trafficcontrol/lib/go-tc"
)
-func GetConfigFileProfileURLSigConfig(cfg config.TCCfg, profileNameOrID string, fileName string) (string, error) {
- profileName, err := toreq.GetProfileNameFromProfileNameOrID(cfg, profileNameOrID)
- if err != nil {
- return "", errors.New("getting profile name from '" + profileNameOrID + "': " + err.Error())
- }
-
- profileParameters, err := toreq.GetProfileParameters(cfg, profileName)
- if err != nil {
- return "", errors.New("getting profile '" + profileName + "' parameters: " + err.Error())
- }
- if len(profileParameters) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params (and there should always be storage.config params), to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
+func GetConfigFileProfileURLSigConfig(toData *TOData, fileName string) (string, string, error) {
paramData := map[string]string{}
// TODO add configFile query param to profile/parameters endpoint, to only get needed data
- for _, param := range profileParameters {
+ for _, param := range toData.ServerParams {
if param.ConfigFile != fileName {
continue
}
@@ -64,15 +43,15 @@ func GetConfigFileProfileURLSigConfig(cfg config.TCCfg, profileNameOrID string,
dsName := GetDSFromURLSigConfigFileName(fileName)
if dsName == "" {
// extra safety, this should never happen, the routing shouldn't get here
- return "", errors.New("getting ds name: malformed config file '" + fileName + "'")
+ return "", "", errors.New("getting ds name: malformed config file '" + fileName + "'")
}
- urlSigKeys, err := toreq.GetURLSigKeys(cfg, dsName)
- if err != nil {
- return "", errors.New("getting url sig keys for ds '" + dsName + "': " + err.Error())
+ urlSigKeys, ok := toData.URLSigKeys[tc.DeliveryServiceName(dsName)]
+ if !ok {
+ return "", "", errors.New("no keys fetched for ds '" + dsName + "!")
}
- return atscfg.MakeURLSigConfig(profileName, urlSigKeys, paramData, toToolName, toURL), nil
+ return atscfg.MakeURLSigConfig(toData.Server.Profile, urlSigKeys, paramData, toData.TOToolName, toData.TOURL), atscfg.ContentTypeParentDotConfig, nil
}
// GetDSFromURLSigConfigFileName returns the DS of a URLSig config file name.
diff --git a/traffic_ops/ort/atstccfg/cfgfile/volumedotconfig.go b/traffic_ops/ort/atstccfg/cfgfile/volumedotconfig.go
index 83b5c54..86a442a 100644
--- a/traffic_ops/ort/atstccfg/cfgfile/volumedotconfig.go
+++ b/traffic_ops/ort/atstccfg/cfgfile/volumedotconfig.go
@@ -20,47 +20,12 @@ package cfgfile
*/
import (
- "errors"
-
"github.com/apache/trafficcontrol/lib/go-atscfg"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
)
const VolumeFileName = StorageFileName
-func GetConfigFileProfileVolumeDotConfig(cfg config.TCCfg, profileNameOrID string) (string, error) {
- profileName, err := toreq.GetProfileNameFromProfileNameOrID(cfg, profileNameOrID)
- if err != nil {
- return "", errors.New("getting profile name from '" + profileNameOrID + "': " + err.Error())
- }
-
- profileParameters, err := toreq.GetProfileParameters(cfg, profileName)
- if err != nil {
- return "", errors.New("getting profile '" + profileName + "' parameters: " + err.Error())
- }
- if len(profileParameters) == 0 {
- // The TO endpoint behind toclient.GetParametersByProfileName returns an empty object with a 200, if the Profile doesn't exist.
- // So we act as though we got a 404 if there are no params (and there should always be volume.config params), to make ORT behave correctly.
- return "", config.ErrNotFound
- }
-
- toToolName, toURL, err := toreq.GetTOToolNameAndURLFromTO(cfg)
- if err != nil {
- return "", errors.New("getting global parameters: " + err.Error())
- }
-
- paramData := map[string]string{}
- // TODO add configFile query param to profile/parameters endpoint, to only get needed data
- for _, param := range profileParameters {
- if param.ConfigFile != VolumeFileName {
- continue
- }
- if param.Name == "location" {
- continue
- }
- paramData[param.Name] = param.Value
- }
-
- return atscfg.MakeVolumeDotConfig(profileName, paramData, toToolName, toURL), nil
+func GetConfigFileProfileVolumeDotConfig(toData *TOData) (string, string, error) {
+ params := ParamsToMap(FilterParams(toData.ServerParams, VolumeFileName, "", "", ""))
+ return atscfg.MakeVolumeDotConfig(toData.Server.Profile, params, toData.TOToolName, toData.TOURL), atscfg.ContentTypeVolumeDotConfig, nil
}
diff --git a/traffic_ops/ort/atstccfg/config/config.go b/traffic_ops/ort/atstccfg/config/config.go
index 6244fb0..07ba346 100644
--- a/traffic_ops/ort/atstccfg/config/config.go
+++ b/traffic_ops/ort/atstccfg/config/config.go
@@ -25,7 +25,6 @@ import (
"math"
"net/url"
"os"
- "path/filepath"
"strings"
"time"
@@ -36,16 +35,9 @@ import (
)
const AppName = "atstccfg"
-const Version = "0.1"
+const Version = "0.2"
const UserAgent = AppName + "/" + Version
-const APIVersion = "1.2"
-const TempSubdir = AppName + "_cache"
-const TempCookieFileName = "cookies"
-const TOCookieName = "mojolicious"
-
-const GlobalProfileName = "GLOBAL"
-
const ExitCodeSuccess = 0
const ExitCodeErrGeneric = 1
const ExitCodeNotFound = 104
@@ -55,19 +47,21 @@ var ErrNotFound = errors.New("not found")
var ErrBadRequest = errors.New("bad request")
type Cfg struct {
- CacheFileMaxAge time.Duration
- LogLocationErr string
- LogLocationInfo string
- LogLocationWarn string
- NumRetries int
- TempDir string
- TOInsecure bool
- TOPass string
- TOTimeout time.Duration
- TOURL *url.URL
- TOUser string
- ListPlugins bool
- PrintGeneratedFiles bool
+ CacheHostName string
+ GetData string
+ ListPlugins bool
+ LogLocationErr string
+ LogLocationInfo string
+ LogLocationWarn string
+ NumRetries int
+ RevalOnly bool
+ SetQueueStatus string
+ SetRevalStatus string
+ TOInsecure bool
+ TOPass string
+ TOTimeout time.Duration
+ TOURL *url.URL
+ TOUser string
}
type TCCfg struct {
@@ -82,23 +76,24 @@ func (cfg Cfg) DebugLog() log.LogLocation { return log.LogLocation(log.LogLoca
func (cfg Cfg) EventLog() log.LogLocation { return log.LogLocation(log.LogLocationNull) } // atstccfg doesn't use the event logger.
// GetCfg gets the application configuration, from arguments and environment variables.
-// Note if PrintGeneratedFiles is configured, the config will be returned with PrintGeneratedFiles true and all other values set to their defaults. This is because other values may have requirements and return errors, where if PrintGeneratedFiles is set by the user, no other setting should be considered.
func GetCfg() (Cfg, error) {
toURLPtr := flag.StringP("traffic-ops-url", "u", "", "Traffic Ops URL. Must be the full URL, including the scheme. Required. May also be set with the environment variable TO_URL.")
toUserPtr := flag.StringP("traffic-ops-user", "U", "", "Traffic Ops username. Required. May also be set with the environment variable TO_USER.")
toPassPtr := flag.StringP("traffic-ops-password", "P", "", "Traffic Ops password. Required. May also be set with the environment variable TO_PASS.")
- noCachePtr := flag.BoolP("no-cache", "n", false, "Whether not to use existing cache files. Optional. Cache files will still be created, existing ones just won't be used.")
numRetriesPtr := flag.IntP("num-retries", "r", 5, "The number of times to retry getting a file if it fails.")
logLocationErrPtr := flag.StringP("log-location-error", "e", "stderr", "Where to log errors. May be a file path, stdout, stderr, or null.")
logLocationWarnPtr := flag.StringP("log-location-warning", "w", "stderr", "Where to log warnings. May be a file path, stdout, stderr, or null.")
logLocationInfoPtr := flag.StringP("log-location-info", "i", "stderr", "Where to log information messages. May be a file path, stdout, stderr, or null.")
- printGeneratedFilesPtr := flag.BoolP("print-generated-files", "g", false, "Whether to print a list of files which are generated (and not proxied to Traffic Ops).")
toInsecurePtr := flag.BoolP("traffic-ops-insecure", "s", false, "Whether to ignore HTTPS certificate errors from Traffic Ops. It is HIGHLY RECOMMENDED to never use this in a production environment, but only for debugging.")
- toTimeoutMSPtr := flag.IntP("traffic-ops-timeout-milliseconds", "t", 10000, "Timeout in seconds for Traffic Ops requests.")
- cacheFileMaxAgeSecondsPtr := flag.IntP("cache-file-max-age-seconds", "a", 900, "Maximum age to use cached files.")
+ toTimeoutMSPtr := flag.IntP("traffic-ops-timeout-milliseconds", "t", 60000, "Timeout in seconds for Traffic Ops requests.")
versionPtr := flag.BoolP("version", "v", false, "Print version information and exit.")
listPluginsPtr := flag.BoolP("list-plugins", "l", false, "Print the list of plugins.")
helpPtr := flag.BoolP("help", "h", false, "Print usage information and exit")
+ cacheHostNamePtr := flag.StringP("cache-host-name", "n", "", "Host name of the cache to generate config for. Must be the server host name in Traffic Ops, not a URL, and not the FQDN")
+ getDataPtr := flag.StringP("get-data", "d", "", "non-config-file Traffic Ops Data to get. Valid values are update-status, packages, chkconfig, system-info, and statuses")
+ setQueueStatusPtr := flag.StringP("set-queue-status", "q", "", "POSTs to Traffic Ops setting the queue status of the server. Must be 'true' or 'false'. Requires --set-reval-status also be set")
+ setRevalStatusPtr := flag.StringP("set-reval-status", "a", "", "POSTs to Traffic Ops setting the revaliate status of the server. Must be 'true' or 'false'. Requires --set-queue-status also be set")
+ revalOnlyPtr := flag.BoolP("revalidate-only", "y", false, "Whether to exclude files not named 'regex_revalidate.config'")
flag.Parse()
@@ -108,8 +103,6 @@ func GetCfg() (Cfg, error) {
} else if *helpPtr {
flag.PrintDefaults()
os.Exit(0)
- } else if *printGeneratedFilesPtr {
- return Cfg{PrintGeneratedFiles: true}, nil
} else if *listPluginsPtr {
return Cfg{ListPlugins: true}, nil
}
@@ -117,15 +110,18 @@ func GetCfg() (Cfg, error) {
toURL := *toURLPtr
toUser := *toUserPtr
toPass := *toPassPtr
- noCache := *noCachePtr
numRetries := *numRetriesPtr
logLocationErr := *logLocationErrPtr
logLocationWarn := *logLocationWarnPtr
logLocationInfo := *logLocationInfoPtr
toInsecure := *toInsecurePtr
toTimeout := time.Millisecond * time.Duration(*toTimeoutMSPtr)
- cacheFileMaxAge := time.Second * time.Duration(*cacheFileMaxAgeSecondsPtr)
listPlugins := *listPluginsPtr
+ cacheHostName := *cacheHostNamePtr
+ getData := *getDataPtr
+ setQueueStatus := *setQueueStatusPtr
+ setRevalStatus := *setRevalStatusPtr
+ revalOnly := *revalOnlyPtr
urlSourceStr := "argument" // for error messages
if toURL == "" {
@@ -139,14 +135,18 @@ func GetCfg() (Cfg, error) {
toPass = os.Getenv("TO_PASS")
}
+ usageStr := "Usage: ./" + AppName + " --traffic-ops-url=myurl --traffic-ops-user=myuser --traffic-ops-password=mypass --cache-host-name=my-cache"
if strings.TrimSpace(toURL) == "" {
- return Cfg{}, errors.New("Missing required argument --traffic-ops-url or TO_URL environment variable. Usage: ./" + AppName + " --traffic-ops-url myurl --traffic-ops-user myuser --traffic-ops-password mypass")
+ return Cfg{}, errors.New("Missing required argument --traffic-ops-url or TO_URL environment variable. " + usageStr)
}
if strings.TrimSpace(toUser) == "" {
- return Cfg{}, errors.New("Missing required argument --traffic-ops-user or TO_USER environment variable. Usage: ./" + AppName + " --traffic-ops-url myurl --traffic-ops-user myuser --traffic-ops-password mypass")
+ return Cfg{}, errors.New("Missing required argument --traffic-ops-user or TO_USER environment variable. " + usageStr)
}
if strings.TrimSpace(toPass) == "" {
- return Cfg{}, errors.New("Missing required argument --traffic-ops-password or TO_PASS environment variable. Usage: ./" + AppName + " --traffic-ops-url myurl --traffic-ops-user myuser --traffic-ops-password mypass")
+ return Cfg{}, errors.New("Missing required argument --traffic-ops-password or TO_PASS environment variable. " + usageStr)
+ }
+ if strings.TrimSpace(cacheHostName) == "" {
+ return Cfg{}, errors.New("Missing required argument --cache-host-name. " + usageStr)
}
toURLParsed, err := url.Parse(toURL)
@@ -156,41 +156,26 @@ func GetCfg() (Cfg, error) {
return Cfg{}, errors.New("invalid Traffic Ops URL from " + urlSourceStr + " '" + toURL + "': " + err.Error())
}
- tmpDir := os.TempDir()
- tmpDir = filepath.Join(tmpDir, TempSubdir)
-
cfg := Cfg{
- CacheFileMaxAge: cacheFileMaxAge,
LogLocationErr: logLocationErr,
LogLocationWarn: logLocationWarn,
LogLocationInfo: logLocationInfo,
NumRetries: numRetries,
- TempDir: tmpDir,
TOInsecure: toInsecure,
TOPass: toPass,
TOTimeout: toTimeout,
TOURL: toURLParsed,
TOUser: toUser,
ListPlugins: listPlugins,
+ CacheHostName: cacheHostName,
+ GetData: getData,
+ SetRevalStatus: setRevalStatus,
+ SetQueueStatus: setQueueStatus,
+ RevalOnly: revalOnly,
}
-
if err := log.InitCfg(cfg); err != nil {
return Cfg{}, errors.New("Initializing loggers: " + err.Error() + "\n")
}
-
- if noCache {
- if err := os.RemoveAll(tmpDir); err != nil {
- log.Errorln("deleting cache directory '" + tmpDir + "': " + err.Error())
- }
- }
-
- if err := os.MkdirAll(tmpDir, 0700); err != nil {
- return Cfg{}, errors.New("creating temp directory '" + tmpDir + "': " + err.Error())
- }
- if err := ValidateDirWriteable(tmpDir); err != nil {
- return Cfg{}, errors.New("validating temp directory is writeable '" + tmpDir + "': " + err.Error())
- }
-
return cfg, nil
}
@@ -207,27 +192,6 @@ func ValidateURL(u *url.URL) error {
return nil
}
-func ValidateDirWriteable(dir string) error {
- testFileName := "testwrite.txt"
- testFilePath := filepath.Join(dir, testFileName)
- if err := os.RemoveAll(testFilePath); err != nil {
- // TODO don't log? This can be normal
- log.Infoln("error removing temp test file '" + testFilePath + "' (ok if it didn't exist): " + err.Error())
- }
-
- fl, err := os.Create(testFilePath)
- if err != nil {
- return errors.New("creating temp test file '" + testFilePath + "': " + err.Error())
- }
- defer fl.Close()
-
- if _, err := fl.WriteString("test"); err != nil {
- return errors.New("writing to temp test file '" + testFilePath + "': " + err.Error())
- }
-
- return nil
-}
-
func RetryBackoffSeconds(currentRetry int) int {
// TODO make configurable?
return int(math.Pow(2.0, float64(currentRetry)))
diff --git a/traffic_ops/ort/atstccfg/getdata/getdata.go b/traffic_ops/ort/atstccfg/getdata/getdata.go
new file mode 100644
index 0000000..cae65c3
--- /dev/null
+++ b/traffic_ops/ort/atstccfg/getdata/getdata.go
@@ -0,0 +1,215 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+// package getdata gets and posts non-config data from Traffic Ops which is related to config generation and needed by ORT.
+// For example, the --get-data, --set-queue-status, and --set-reval-status arguments.
+package getdata
+
+import (
+ "encoding/json"
+ "errors"
+ "fmt"
+ "io"
+ "io/ioutil"
+ "net/http"
+ "os"
+ "strings"
+
+ "github.com/apache/trafficcontrol/lib/go-atscfg"
+ "github.com/apache/trafficcontrol/lib/go-log"
+ "github.com/apache/trafficcontrol/lib/go-tc"
+ "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
+ "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
+)
+
+func GetDataFuncs() map[string]func(config.TCCfg, io.Writer) error {
+ return map[string]func(config.TCCfg, io.Writer) error{
+ `update-status`: WriteServerUpdateStatus,
+ `packages`: WritePackages,
+ `chkconfig`: WriteChkconfig,
+ `system-info`: WriteSystemInfo,
+ `statuses`: WriteStatuses,
+ }
+}
+
+func WriteData(cfg config.TCCfg) error {
+ log.Infoln("Getting data '" + cfg.GetData + "'")
+ dataF, ok := GetDataFuncs()[cfg.GetData]
+ if !ok {
+ return errors.New("unknown data request '" + cfg.GetData + "'")
+ }
+ return dataF(cfg, os.Stdout)
+}
+
+func SetQueueRevalStatuses(cfg config.TCCfg) error {
+ log.Infoln("setting queue reval statuses '" + cfg.SetQueueStatus + "', '" + cfg.SetRevalStatus + "'")
+ if cfg.SetQueueStatus == "" || cfg.SetRevalStatus == "" {
+ return errors.New("must set both reval and queue status")
+ }
+ queueStatus := false
+ revalStatus := false
+ if strings.ToLower(string(cfg.SetQueueStatus[0])) != "f" {
+ queueStatus = true
+ }
+ if strings.ToLower(string(cfg.SetRevalStatus[0])) != "f" {
+ revalStatus = true
+ }
+ return SetUpdateStatus(cfg, tc.CacheName(cfg.CacheHostName), queueStatus, revalStatus)
+}
+
+const SystemInfoParamConfigFile = `global`
+
+// WriteSystemInfo writes the "system info" to output.
+//
+// This is the same info at /api/1.x/system/info, which is actually just all Parameters with the config_file 'global'.
+// Note this is different than the more common "global parameters", which usually refers to all Parameters on the Profile named 'GLOBAL'.
+//
+// This is identical to the /api/1.x/system/info endpoint, except it does not include a '{response: {parameters:' wrapper.
+//
+func WriteSystemInfo(cfg config.TCCfg, output io.Writer) error {
+ paramArr, err := toreq.GetConfigFileParameters(cfg, SystemInfoParamConfigFile)
+ if err != nil {
+ return errors.New("getting system info parameters: " + err.Error())
+ }
+ params := map[string]string{}
+ for _, param := range paramArr {
+ params[param.Name] = param.Value
+ }
+ if err := json.NewEncoder(output).Encode(params); err != nil {
+ return errors.New("encoding system info parameters: " + err.Error())
+ }
+ return nil
+}
+
+// WriteStatuses writes the Traffic Ops statuses to output.
+// Note this is identical to /api/1.x/statuses except it omits the '{response:' wrapper.
+func WriteStatuses(cfg config.TCCfg, output io.Writer) error {
+ statuses, err := toreq.GetStatuses(cfg)
+ if err != nil {
+ return errors.New("getting statuses: " + err.Error())
+ }
+ if err := json.NewEncoder(output).Encode(statuses); err != nil {
+ return errors.New("encoding statuses: " + err.Error())
+ }
+ return nil
+}
+
+// WriteUpdateStatus writes the Traffic Ops server update status to output.
+// Note this is identical to /api/1.x/servers/name/update_status except it omits the '[]' wrapper.
+func WriteServerUpdateStatus(cfg config.TCCfg, output io.Writer) error {
+ status, err := toreq.GetServerUpdateStatus(cfg)
+ if err != nil {
+ return errors.New("getting server update status: " + err.Error())
+ }
+ if err := json.NewEncoder(output).Encode(status); err != nil {
+ return errors.New("encoding server update status: " + err.Error())
+ }
+ return nil
+}
+
+// WriteORTServerPackages writes the packages for serverName to output.
+// Note this is identical to /ort/serverName/packages.
+func WritePackages(cfg config.TCCfg, output io.Writer) error {
+ packages, err := GetPackages(cfg)
+ if err != nil {
+ return errors.New("getting ORT server packages: " + err.Error())
+ }
+ if err := json.NewEncoder(output).Encode(packages); err != nil {
+ return errors.New("writing packages: " + err.Error())
+ }
+ return nil
+}
+
+func GetPackages(cfg config.TCCfg) ([]atscfg.Package, error) {
+ server, err := toreq.GetServerByHostName(cfg, string(cfg.CacheHostName))
+ if err != nil {
+ return nil, errors.New("getting server: " + err.Error())
+ }
+ params, err := toreq.GetServerProfileParameters(cfg, server.Profile)
+ if err != nil {
+ return nil, errors.New("getting server profile '" + server.Profile + "' parameters: " + err.Error())
+ }
+ packages := []atscfg.Package{}
+ for _, param := range params {
+ if param.ConfigFile != atscfg.PackagesParamConfigFile {
+ continue
+ }
+ packages = append(packages, atscfg.Package{Name: param.Name, Version: param.Value})
+ }
+ return packages, nil
+}
+
+// WriteChkconfig writes the chkconfig for cfg.CacheHostName to output.
+// Note this is identical to /ort/serverName/chkconfig.
+func WriteChkconfig(cfg config.TCCfg, output io.Writer) error {
+ chkconfig, err := GetChkconfig(cfg)
+ if err != nil {
+ return errors.New("getting chkconfig: " + err.Error())
+ }
+ if err := json.NewEncoder(output).Encode(chkconfig); err != nil {
+ return errors.New("writing chkconfig: " + err.Error())
+ }
+ return nil
+}
+
+func GetChkconfig(cfg config.TCCfg) ([]atscfg.ChkConfigEntry, error) {
+ server, err := toreq.GetServerByHostName(cfg, string(cfg.CacheHostName))
+ if err != nil {
+ return nil, errors.New("getting server: " + err.Error())
+ }
+ params, err := toreq.GetServerProfileParameters(cfg, server.Profile)
+ if err != nil {
+ return nil, errors.New("getting server profile '" + server.Profile + "' parameters: " + err.Error())
+ }
+ chkconfig := []atscfg.ChkConfigEntry{}
+ for _, param := range params {
+ if param.ConfigFile != atscfg.ChkconfigParamConfigFile {
+ continue
+ }
+ chkconfig = append(chkconfig, atscfg.ChkConfigEntry{Name: param.Name, Val: param.Value})
+ }
+ return chkconfig, nil
+}
+
+// SetUpdateStatus sets the queue and reval status of serverName in Traffic Ops.
+func SetUpdateStatus(cfg config.TCCfg, serverName tc.CacheName, queue bool, revalPending bool) error {
+ // TODO change this to an API path, when one exists
+ path := `/update/` + string(serverName) + `?updated=` + jsonBoolStr(queue) + `&reval_updated=` + jsonBoolStr(revalPending)
+ // RawRequest should generally never be used, but the alternatve here is to manually get the cookie and do an http.Get. We need to hit a non-API endpoint, no API endpoint exists for what we need.
+ resp, _, err := (*cfg.TOClient).RawRequest(http.MethodPost, path, nil)
+ if err != nil {
+ return errors.New("setting update statuses: " + err.Error())
+ }
+ defer resp.Body.Close()
+ if resp.StatusCode != 200 {
+ bodyBts, err := ioutil.ReadAll(resp.Body)
+ if err == nil {
+ return fmt.Errorf("Traffic Ops returned %v %v", resp.StatusCode, string(bodyBts))
+ }
+ return fmt.Errorf("Traffic Ops returned %v (error reading body)", resp.StatusCode)
+ }
+ return nil
+}
+
+func jsonBoolStr(b bool) string {
+ if b {
+ return `true`
+ }
+ return `false`
+}
diff --git a/traffic_ops/ort/atstccfg/plugin/plugin.go b/traffic_ops/ort/atstccfg/plugin/plugin.go
index aa90684..d2d96a9 100644
--- a/traffic_ops/ort/atstccfg/plugin/plugin.go
+++ b/traffic_ops/ort/atstccfg/plugin/plugin.go
@@ -63,7 +63,7 @@ func AddPlugin(priority uint64, funcs Funcs) {
}
pluginName := strings.TrimSuffix(path.Base(filename), ".go")
- log.Debugln("AddPlugin adding " + pluginName)
+ log.Infoln("AddPlugin adding " + pluginName)
initPlugins = append(initPlugins, pluginObj{funcs: funcs, priority: priority, name: pluginName})
}
@@ -122,13 +122,13 @@ func (ps plugins) OnStartup(d StartupData) {
// OnRequest returns a boolean whether to immediately stop processing the request. If a plugin returns true, this is immediately returned with no further plugins processed.
func (ps plugins) OnRequest(d OnRequestData) bool {
- log.Debugf("DEBUG plugins.OnRequest calling %+v plugins\n", len(ps.slice))
+ log.Infof("plugins.OnRequest calling %+v plugins\n", len(ps.slice))
for _, p := range ps.slice {
if p.funcs.onRequest == nil {
- log.Debugln("plugins.OnRequest plugging " + p.name + " - no onRequest func")
+ log.Infoln("plugins.OnRequest plugging " + p.name + " - no onRequest func")
continue
}
- log.Debugln("plugins.OnRequest plugging " + p.name)
+ log.Infoln("plugins.OnRequest plugging " + p.name)
if stop := p.funcs.onRequest(d); stop {
return true
}
diff --git a/traffic_ops/ort/atstccfg/routing.go b/traffic_ops/ort/atstccfg/routing.go
deleted file mode 100644
index ab2239d..0000000
--- a/traffic_ops/ort/atstccfg/routing.go
+++ /dev/null
@@ -1,235 +0,0 @@
-package main
-
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-import (
- "errors"
- "net/http"
- "strings"
- "time"
-
- "github.com/apache/trafficcontrol/lib/go-log"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/cfgfile"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/toreq"
-)
-
-var scopeConfigFileFuncs = map[string]func(cfg config.TCCfg, resource string, fileName string) (string, int, error){
- "cdns": GetConfigFileCDN,
- "servers": GetConfigFileServer,
- "profiles": GetConfigFileProfile,
-}
-
-func GetConfigFile(cfg config.TCCfg) (string, int, error) {
- start := time.Now()
- defer func() {
- log.Infof("GetConfigFile %v took %v\n", cfg.TOURL.Path, time.Since(start).Round(time.Millisecond))
- }()
-
- pathParts := strings.Split(cfg.TOURL.Path, "/")
-
- if len(pathParts) == 7 && pathParts[1] == `api` && pathParts[3] == `servers` && pathParts[5] == `configfiles` && pathParts[6] == `ats` {
- // "/api/1.x/servers/name/configfiles/ats" is the "meta" config route, which lists all the other configs for this server.
- server := pathParts[4]
- log.Infoln("GetConfigFile is meta config request for server '" + server + "'; generating")
- txt, err := cfgfile.GetConfigFileMeta(cfg, server)
- if err != nil {
- if err == config.ErrNotFound {
- return "", config.ExitCodeNotFound, err
- } else if err == config.ErrBadRequest {
- return "", config.ExitCodeBadRequest, err
- } else {
- return "", config.ExitCodeErrGeneric, err
- }
- }
- return txt, config.ExitCodeSuccess, nil
- }
-
- if len(pathParts) < 8 {
- log.Infoln("GetConfigFile pathParts < 7, calling TO")
- return GetConfigFileFromTrafficOps(cfg)
- }
- scope := pathParts[3]
- resource := pathParts[4]
- fileName := pathParts[7]
-
- log.Infoln("GetConfigFile scope '" + scope + "' resource '" + resource + "' fileName '" + fileName + "'")
-
- if scopeConfigFileFunc, ok := scopeConfigFileFuncs[scope]; ok {
- return scopeConfigFileFunc(cfg, resource, fileName)
- }
-
- log.Infoln("GetConfigFile unknown scope, calling TO")
- return GetConfigFileFromTrafficOps(cfg)
-}
-
-type ConfigFilePrefixSuffixFunc struct {
- Prefix string
- Suffix string
- Func func(cfg config.TCCfg, resource string, fileName string) (string, error)
-}
-
-func GetConfigFileCDN(cfg config.TCCfg, cdnNameOrID string, fileName string) (string, int, error) {
- log.Infoln("GetConfigFileCDN cdn '" + cdnNameOrID + "' fileName '" + fileName + "'")
-
- txt := ""
- err := error(nil)
- if getCfgFunc, ok := CDNConfigFileFuncs()[fileName]; ok {
- txt, err = getCfgFunc(cfg, cdnNameOrID)
- } else {
- for _, prefixSuffixFunc := range ConfigFileCDNPrefixSuffixFuncs {
- if strings.HasPrefix(fileName, prefixSuffixFunc.Prefix) && strings.HasSuffix(fileName, prefixSuffixFunc.Suffix) && len(fileName) > len(prefixSuffixFunc.Prefix)+len(prefixSuffixFunc.Suffix) {
- txt, err = prefixSuffixFunc.Func(cfg, cdnNameOrID, fileName)
- break
- }
- }
- }
-
- if err == nil && txt == "" {
- err = config.ErrNotFound
- }
-
- if err != nil {
- code := config.ExitCodeErrGeneric
- if err == config.ErrNotFound {
- code = config.ExitCodeNotFound
- } else if err == config.ErrBadRequest {
- code = config.ExitCodeBadRequest
- }
- return "", code, err
- }
- return txt, config.ExitCodeSuccess, nil
-}
-
-func GetConfigFileProfile(cfg config.TCCfg, profileNameOrID string, fileName string) (string, int, error) {
- log.Infoln("GetConfigFileProfile profile '" + profileNameOrID + "' fileName '" + fileName + "'")
-
- txt := ""
- err := error(nil)
- if getCfgFunc, ok := ProfileConfigFileFuncs()[fileName]; ok {
- txt, err = getCfgFunc(cfg, profileNameOrID)
- } else if strings.HasPrefix(fileName, "url_sig_") && strings.HasSuffix(fileName, ".config") && len(fileName) > len("url_sig_")+len(".config") {
- txt, err = cfgfile.GetConfigFileProfileURLSigConfig(cfg, profileNameOrID, fileName)
- } else if strings.HasPrefix(fileName, "uri_signing_") && strings.HasSuffix(fileName, ".config") && len(fileName) > len("uri_signing")+len(".config") {
- txt, err = cfgfile.GetConfigFileProfileURISigningConfig(cfg, profileNameOrID, fileName)
- } else {
- txt, err = cfgfile.GetConfigFileProfileUnknownConfig(cfg, profileNameOrID, fileName)
- }
-
- if err != nil {
- code := config.ExitCodeErrGeneric
- if err == config.ErrNotFound {
- code = config.ExitCodeNotFound
- } else if err == config.ErrBadRequest {
- code = config.ExitCodeBadRequest
- }
- return "", code, err
- }
- return txt, config.ExitCodeSuccess, nil
-}
-
-// ConfigFileFuncs returns a map[scope][configFile]configFileFunc.
-func ConfigFileFuncs() map[string]map[string]func(cfg config.TCCfg, serverNameOrID string) (string, error) {
- return map[string]map[string]func(cfg config.TCCfg, serverNameOrID string) (string, error){
- "cdns": CDNConfigFileFuncs(),
- "servers": ServerConfigFileFuncs(),
- "profiles": ProfileConfigFileFuncs(),
- }
-}
-
-func CDNConfigFileFuncs() map[string]func(cfg config.TCCfg, cdnNameOrID string) (string, error) {
- return map[string]func(cfg config.TCCfg, cdnNameOrID string) (string, error){
- "regex_revalidate.config": cfgfile.GetConfigFileCDNRegexRevalidateDotConfig,
- "bg_fetch.config": cfgfile.GetConfigFileCDNBGFetchDotConfig,
- "ssl_multicert.config": cfgfile.GetConfigFileCDNSSLMultiCertDotConfig,
- "cacheurl.config": cfgfile.GetConfigFileCDNCacheURLPlain,
- }
-}
-
-var ConfigFileCDNPrefixSuffixFuncs = []ConfigFilePrefixSuffixFunc{
- {"hdr_rw_mid_", ".config", cfgfile.GetConfigFileCDNHeaderRewriteMid},
- {"hdr_rw_", ".config", cfgfile.GetConfigFileCDNHeaderRewrite},
- {"cacheurl", ".config", cfgfile.GetConfigFileCDNCacheURL},
- {"regex_remap_", ".config", cfgfile.GetConfigFileCDNRegexRemap},
- {"set_dscp_", ".config", cfgfile.GetConfigFileCDNSetDSCP},
-}
-
-func ProfileConfigFileFuncs() map[string]func(cfg config.TCCfg, serverNameOrID string) (string, error) {
- return map[string]func(cfg config.TCCfg, serverNameOrID string) (string, error){
- "12M_facts": cfgfile.GetConfigFileProfile12MFacts,
- "50-ats.rules": cfgfile.GetConfigFileProfileATSDotRules,
- "astats.config": cfgfile.GetConfigFileProfileAstatsDotConfig,
- "cache.config": cfgfile.GetConfigFileProfileCacheDotConfig,
- "drop_qstring.config": cfgfile.GetConfigFileProfileDropQStringDotConfig,
- "logging.config": cfgfile.GetConfigFileProfileLoggingDotConfig,
- "logging.yaml": cfgfile.GetConfigFileProfileLoggingDotYAML,
- "logs_xml.config": cfgfile.GetConfigFileProfileLogsXMLDotConfig,
- "plugin.config": cfgfile.GetConfigFileProfilePluginDotConfig,
- "records.config": cfgfile.GetConfigFileProfileRecordsDotConfig,
- "storage.config": cfgfile.GetConfigFileProfileStorageDotConfig,
- "sysctl.conf": cfgfile.GetConfigFileProfileSysCtlDotConf,
- "volume.config": cfgfile.GetConfigFileProfileVolumeDotConfig,
- }
-}
-
-func ServerConfigFileFuncs() map[string]func(cfg config.TCCfg, serverNameOrID string) (string, error) {
- return map[string]func(cfg config.TCCfg, serverNameOrID string) (string, error){
- "parent.config": cfgfile.GetConfigFileServerParentDotConfig,
- "remap.config": cfgfile.GetConfigFileServerRemapDotConfig,
- "cache.config": cfgfile.GetConfigFileServerCacheDotConfig,
- "ip_allow.config": cfgfile.GetConfigFileServerIPAllowDotConfig,
- "hosting.config": cfgfile.GetConfigFileServerHostingDotConfig,
- "packages": cfgfile.GetConfigFileServerPackages,
- "chkconfig": cfgfile.GetConfigFileServerChkconfig,
- }
-}
-
-func GetConfigFileServer(cfg config.TCCfg, serverNameOrID string, fileName string) (string, int, error) {
- log.Infoln("GetConfigFileServer server '" + serverNameOrID + "' fileName '" + fileName + "'")
- txt := ""
- err := error(nil)
- if getCfgFunc, ok := ServerConfigFileFuncs()[fileName]; ok {
- txt, err = getCfgFunc(cfg, serverNameOrID)
- } else {
- txt, err = cfgfile.GetConfigFileServerUnknownConfig(cfg, serverNameOrID, fileName)
- }
- if err != nil {
- return "", config.ExitCodeErrGeneric, err
- }
- return txt, config.ExitCodeSuccess, nil
-}
-
-func GetConfigFileFromTrafficOps(cfg config.TCCfg) (string, int, error) {
- path := cfg.TOURL.Path
- if cfg.TOURL.RawQuery != "" {
- path += "?" + cfg.TOURL.RawQuery
- }
- log.Infoln("GetConfigFile path '" + path + "' not generated locally, requesting from Traffic Ops")
- log.Infoln("GetConfigFile url '" + cfg.TOURL.String() + "'")
-
- body, code, err := toreq.TrafficOpsRequest(cfg, http.MethodGet, cfg.TOURL.String(), nil)
- if err != nil {
- return "", code, errors.New("Requesting path '" + path + "': " + err.Error())
- }
-
- toreq.WriteCookiesToFile(toreq.CookiesToString((*cfg.TOClient).Client.Jar.Cookies(cfg.TOURL)), cfg.TempDir)
-
- return string(body), HTTPCodeToExitCode(code), nil
-}
diff --git a/traffic_ops/ort/atstccfg/toreq/caching.go b/traffic_ops/ort/atstccfg/toreq/caching.go
deleted file mode 100644
index 198fbe2..0000000
--- a/traffic_ops/ort/atstccfg/toreq/caching.go
+++ /dev/null
@@ -1,280 +0,0 @@
-package toreq
-
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-import (
- "bufio"
- "encoding/json"
- "errors"
- "fmt"
- "io"
- "io/ioutil"
- "net/http"
- "os"
- "path/filepath"
- "strconv"
- "strings"
- "time"
-
- "github.com/apache/trafficcontrol/lib/go-log"
- "github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
-)
-
-// DefaultCacheFormat is the encoder, decoder, and file extension to use for caching.
-// This may be changed at compile-time. See CacheFormatJSON.
-// Note this is not used for all cache files. Notably, Delivery Service Servers use a custom CSV format, which is faster.
-var DefaultCacheFormat = CacheFormatJSON
-
-// GetCached attempts to get the given object from tempDir/cacheFileName.
-// If the cache file doesn't exist, is too old, or is malformed, it uses getter to get the object, and stores it in cacheFileName.
-// The object is placed in obj (which must be a pointer to the type of object to decode), and the error from getter is returned.
-// The cache format is defined by CacheEncoder and CacheDecoder, which may be changed at compile-time.
-func GetCached(cfg config.TCCfg, cacheFileName string, obj interface{}, getter func(obj interface{}) error) error {
- return GetCachedWithFormat(cfg, cacheFileName, obj, getter, DefaultCacheFormat)
-}
-
-// GetCachedDSS is like GetCached, but optimized for Delivery Service Servers, which is massive.
-func GetCachedDSS(cfg config.TCCfg, cacheFileName string, obj *[]tc.DeliveryServiceServer, getter func(obj interface{}) error) error {
- return GetCachedWithFormat(cfg, cacheFileName, obj, getter, CacheFormatDSS)
-}
-
-func GetCachedWithFormat(cfg config.TCCfg, cacheFileName string, obj interface{}, getter func(obj interface{}) error, cacheFormat CacheFormat) error {
- cacheFileName += cacheFormat.Extension
- start := time.Now()
- err := ReadCache(cacheFormat.Decoder, cfg.TempDir, cacheFileName, cfg.CacheFileMaxAge, obj)
- if err == nil {
- log.Infof("ReadCache %v (hit) took %v\n", cacheFileName, time.Since(start).Round(time.Millisecond))
- return nil
- }
-
- log.Infoln("ReadCache failed to get object from '" + cfg.TempDir + "/" + cacheFileName + "', calling getter: " + err.Error())
-
- currentRetry := 0
- for {
- err := getter(obj)
- if err == nil {
- break
- }
- if strings.Contains(strings.ToLower(err.Error()), "not found") {
- // if the server returned a 404, retrying won't help
- return errors.New("getting uncached: " + err.Error())
- }
- if currentRetry == cfg.NumRetries {
- return errors.New("getting uncached: " + err.Error())
- }
-
- sleepSeconds := config.RetryBackoffSeconds(currentRetry)
- log.Warnf("getting '%v', sleeping for %v seconds: %v\n", cacheFileName, sleepSeconds, err)
- currentRetry++
- time.Sleep(time.Second * time.Duration(sleepSeconds)) // TODO make backoff configurable?
- }
-
- WriteCache(cacheFormat.Encoder, cfg.TempDir, cacheFileName, obj)
- log.Infof("GetCachedJSON %v (miss) took %v\n", cacheFileName, time.Since(start).Round(time.Millisecond))
- return nil
-}
-
-// WriteCache attempts to write obj to tempDir/cacheFileName.
-// If there is an error, it is written to stderr but not returned.
-func WriteCache(encode EncodeFunc, tempDir string, cacheFileName string, obj interface{}) {
- if encode == nil {
- log.Errorln("object '" + cacheFileName + "': nil encode func! Should never happen! Can't write file!")
- return
- }
-
- objPath := filepath.Join(tempDir, cacheFileName)
- // Use os.OpenFile, not os.Create, in order to set perms to 0600 - cookies allow login, therefore the file MUST only allow access by the current user, for security reasons
- objFile, err := os.OpenFile(objPath, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0600)
- if err != nil {
- log.Errorln("creating object cache file '" + objPath + "': " + err.Error())
- return
- }
- defer objFile.Close()
-
- if err := encode(objFile, obj); err != nil {
- log.Errorln("writing '" + cacheFileName + "': " + err.Error())
- return
- }
-}
-
-// ReadCache gets obj from tempDir/cacheFileName, if it exists and isn't older than CacheFileMaxAge.
-// The obj must be a non-nil pointer to the object to decode into.
-func ReadCache(decode DecodeFunc, tempDir string, cacheFileName string, cacheFileMaxAge time.Duration, obj interface{}) error {
- if decode == nil {
- return errors.New("nil decode func")
- }
- objPath := filepath.Join(tempDir, cacheFileName)
- objFile, err := os.Open(objPath)
- if err != nil {
- return errors.New("opening object file '" + objPath + "':" + err.Error())
- }
- defer objFile.Close()
-
- objFileInfo, err := objFile.Stat()
- if err != nil {
- return errors.New("getting object file info '" + objPath + "':" + err.Error())
- }
- if objFileAge := time.Now().Sub(objFileInfo.ModTime()); objFileAge > cacheFileMaxAge {
- return fmt.Errorf("object file too old, max age %dms less than file age %dms", int(cacheFileMaxAge/time.Millisecond), int(objFileAge/time.Millisecond))
- }
- if err := decode(objFile, obj); err != nil {
- log.Errorln("GetJSONObjFromFile loaded '" + cacheFileName + "' but failed to parse JSON: " + err.Error())
- return errors.New("unmarshalling object from file '" + objPath + "':" + err.Error())
- }
- return nil
-}
-
-type CacheFormat struct {
- Encoder EncodeFunc
- Decoder DecodeFunc
- Extension string
-}
-
-var CacheFormatJSON = CacheFormat{JSONEncode, JSONDecode, JSONExtension}
-
-// CacheFormatDSS is a special format encoder/decoder optimized for Delivery Service Servers.
-// The encoder and decoder both return errors if obj is not a *[]tc.DeliveryServiceServer.
-var CacheFormatDSS = CacheFormat{DSSEncode, DSSDecode, DSSExtension}
-
-type EncodeFunc func(w io.Writer, obj interface{}) error
-type DecodeFunc func(r io.Reader, obj interface{}) error
-
-const JSONExtension = ".json"
-
-func JSONDecode(r io.Reader, obj interface{}) error { return json.NewDecoder(r).Decode(obj) }
-func JSONEncode(w io.Writer, obj interface{}) error { return json.NewEncoder(w).Encode(obj) }
-
-func StringToCookies(cookiesStr string) []*http.Cookie {
- hdr := http.Header{}
- hdr.Add("Cookie", cookiesStr)
- req := http.Request{Header: hdr}
- return req.Cookies()
-}
-
-func CookiesToString(cookies []*http.Cookie) string {
- strs := []string{}
- for _, cookie := range cookies {
- strs = append(strs, cookie.String())
- }
- return strings.Join(strs, "; ")
-}
-
-// WriteCookiesFile writes the given cookies to the temp file. On error, returns nothing, but writes to stderr.
-func WriteCookiesToFile(cookiesStr string, tempDir string) {
- cookiePath := filepath.Join(tempDir, config.TempCookieFileName)
- // Use os.OpenFile, not os.Create, in order to set perms to 0600 - cookies allow login, therefore the file MUST only allow access by the current user, for security reasons
- if cookieFile, err := os.OpenFile(cookiePath, os.O_RDWR|os.O_CREATE, 0600); err != nil {
- log.Errorln("creating cookie file '" + cookiePath + "': " + err.Error())
- } else {
- defer cookieFile.Close()
- if _, err := cookieFile.WriteString(cookiesStr + "\n"); err != nil {
- log.Errorln("writing cookie file '" + cookiePath + "': " + err.Error())
- }
- }
-}
-
-func GetCookiesFromFile(tempDir string, cacheFileMaxAge time.Duration) (string, error) {
- cookiePath := filepath.Join(tempDir, config.TempCookieFileName)
-
- cookieFile, err := os.Open(cookiePath)
- if err != nil {
- return "", errors.New("opening cookie file '" + cookiePath + "':" + err.Error())
- }
- defer cookieFile.Close()
-
- cookieFileInfo, err := cookieFile.Stat()
- if err != nil {
- return "", errors.New("getting cookie file info '" + cookiePath + "':" + err.Error())
- }
-
- cookieFileAge := time.Now().Sub(cookieFileInfo.ModTime())
- if cookieFileAge > cacheFileMaxAge {
- return "", fmt.Errorf("cookie file too old, max age %dms less than file age %dms", int(cacheFileMaxAge/time.Millisecond), int(cookieFileAge/time.Millisecond))
- }
-
- bts, err := ioutil.ReadAll(cookieFile)
- if err != nil {
- return "", errors.New("reading cookies from file '" + cookiePath + "':" + err.Error())
- }
- return string(bts), nil
-}
-
-// DSSExtension is the file extension for the format read and written by DSSEncode and DSSDecode.
-const DSSExtension = ".csv"
-
-// DSSEncode is an EncodeFunc optimized for Delivery Service Servers.
-// If iObj is not a *[]tc.DeliveryServiceServer it immediately returns an error.
-func DSSEncode(w io.Writer, iObj interface{}) error {
- // Please don't change this to use encoding/csv unless you benchmark and prove it's at least as fast. DSS is massive, and performance is important.
- obj, ok := iObj.(*[]tc.DeliveryServiceServer)
- if !ok {
- return fmt.Errorf("object is '%T' must be a *[]tc.DeliveryServiceServer\n", iObj)
- }
- for _, dss := range *obj {
- if dss.DeliveryService == nil || dss.Server == nil {
- continue // TODO warn?
- }
- if _, err := io.WriteString(w, strconv.Itoa(*dss.DeliveryService)+`,`+strconv.Itoa(*dss.Server)+"\n"); err != nil {
- return fmt.Errorf("writing object cache file: " + err.Error())
- }
- }
- return nil
-}
-
-// DSSDecode is a DecodeFunc optimized for Delivery Service Servers.
-// If iObj is not a *[]tc.DeliveryServiceServer it immediately returns an error.
-func DSSDecode(r io.Reader, iObj interface{}) error {
- // Please don't change this to use encoding/csv unless you benchmark and prove it's at least as fast. DSS is massive, and performance is important.
- obj, ok := iObj.(*[]tc.DeliveryServiceServer)
- if !ok {
- return fmt.Errorf("object is '%T' must be a *[]tc.DeliveryServiceServer\n", iObj)
- }
- objFileReader := bufio.NewReader(r)
- for {
- dsStr, err := objFileReader.ReadString(',')
- if err != nil {
- if err == io.EOF {
- return nil
- }
- return errors.New("malformed object file:" + err.Error())
- }
- dsStr = dsStr[:len(dsStr)-1] // ReadString(',') includes the comma; remove it
-
- ds, err := strconv.Atoi(dsStr)
- if err != nil {
- return errors.New("malformed object file: first field should be ds id, but is not an integer: '" + dsStr + "'")
- }
- svStr, err := objFileReader.ReadString('\n')
- if err != nil {
- if err == io.EOF {
- return errors.New("malformed object file: final line had ds but not server, or file is missing a trailing newline")
- }
- return errors.New("malformed object file:" + err.Error())
- }
- svStr = svStr[:len(svStr)-1] // ReadString('\n') includes the newline; remove it
- sv, err := strconv.Atoi(svStr)
- if err != nil {
- return errors.New("malformed object file: second field should be server id, but is not an integer: '" + svStr + "'")
- }
- *obj = append(*obj, tc.DeliveryServiceServer{DeliveryService: &ds, Server: &sv})
- }
- return nil
-}
diff --git a/traffic_ops/ort/atstccfg/toreq/caching_test.go b/traffic_ops/ort/atstccfg/toreq/caching_test.go
deleted file mode 100644
index e9542b2..0000000
--- a/traffic_ops/ort/atstccfg/toreq/caching_test.go
+++ /dev/null
@@ -1,163 +0,0 @@
-package toreq
-
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-import (
- "encoding/json"
- "os"
- "path/filepath"
- "reflect"
- "testing"
- "time"
-
- "github.com/apache/trafficcontrol/lib/go-tc"
- "github.com/apache/trafficcontrol/lib/go-util"
-)
-
-func TestWriteCacheJSON(t *testing.T) {
- type Obj struct {
- S string `json:"s"`
- }
-
- fileName := "TestWriteCacheJSON.json"
- tmpDir := os.TempDir()
- filePath := filepath.Join(tmpDir, fileName)
- defer os.Remove(filePath)
-
- {
- largeObj := Obj{
- S: `
- Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
-
- Curabitur pretium tincidunt lacus. Nulla gravida orci a odio. Nullam varius, turpis et commodo pharetra, est eros bibendum elit, nec luctus magna felis sollicitudin mauris. Integer in mauris eu nibh euismod gravida. Duis ac tellus et risus vulputate vehicula. Donec lobortis risus a elit. Etiam tempor. Ut ullamcorper, ligula eu tempor congue, eros est euismod turpis, id tincidunt sapien risus a quam. Maecenas fermentum consequat mi. Donec fermentum. Pellentesque malesuada nulla a mi. [...]
-`,
- }
-
- WriteCache(CacheFormatJSON.Encoder, tmpDir, fileName, largeObj)
- loadedLargeObj := Obj{}
- if err := ReadCache(CacheFormatJSON.Decoder, tmpDir, fileName, time.Hour, &loadedLargeObj); err != nil {
- t.Fatalf("ReadCache large error expected nil, actual: " + err.Error())
- } else if largeObj.S != loadedLargeObj.S {
- t.Fatalf("ReadCache expected %+v actual %+v\n", largeObj.S, loadedLargeObj.S)
- }
- }
-
- {
- // Write a smaller object to the same file, to make sure it properly truncates, and doesn't leave old text lying around (resulting in malformed json)
- smallObj := Obj{S: `foo`}
- WriteCache(CacheFormatJSON.Encoder, tmpDir, fileName, smallObj)
- loadedSmallObj := Obj{}
- if err := ReadCache(CacheFormatJSON.Decoder, tmpDir, fileName, time.Hour, &loadedSmallObj); err != nil {
- t.Fatalf("GetJSONObjFromFile small error expected nil, actual: " + err.Error())
- } else if smallObj.S != loadedSmallObj.S {
- t.Fatalf("GetJSONObjFromFile expected %+v actual %+v\n", smallObj.S, loadedSmallObj.S)
- }
- }
-}
-
-func TestDefaultCacheFormatIsomorphic(t *testing.T) {
- // Test whether DefaultCacheFormat is isomorphic.
- // That is, whether serializing and deserializing produces the original object.
- // This might seem silly, but encoding/gob is not isomorphic and fails this test.
- // We requires an isomorphic cache format, or config files will be wrong.
-
- // Delivery Service is one of TC's most complex objects, so use an array of it to test.
- // It's important to test null pointers, as well as pointers to default values (a common failure).
-
- dsMatchPtr := []tc.DeliveryServiceMatch{
- tc.DeliveryServiceMatch{
- Type: tc.DSMatchTypeHostRegex,
- SetNumber: 0,
- Pattern: "foo",
- },
- tc.DeliveryServiceMatch{
- Type: tc.DSMatchTypeInvalid,
- SetNumber: 42,
- Pattern: "",
- },
- }
- dsTypeInvalidPtr := tc.DSTypeInvalid
-
- ds1 := tc.DeliveryServiceNullable{}
- ds1.Active = util.BoolPtr(false)
- ds1.AnonymousBlockingEnabled = nil
- ds1.CacheURL = util.StrPtr("")
- ds1.CCRDNSTTL = util.IntPtr(0)
- ds1.CDNID = nil
- ds1.CDNName = nil
- ds1.CheckPath = util.StrPtr("foo")
- ds1.DisplayName = util.StrPtr("")
- ds1.DSCP = nil
- ds1.LogsEnabled = util.BoolPtr(true)
- // ds1.MatchList = &dsMatchPtr
- ds1.MissLat = nil
- ds1.MissLong = util.FloatPtr(0)
- ds1.Signed = false
- ds1.Type = &dsTypeInvalidPtr
- // ds1.ExampleURLs = []string{"foo", ""}
-
- ds2 := tc.DeliveryServiceNullable{}
- ds2.Active = nil
- ds2.AnonymousBlockingEnabled = util.BoolPtr(false)
- ds2.CacheURL = util.StrPtr("")
- ds2.CCRDNSTTL = util.IntPtr(0)
- ds2.CDNID = nil
- ds2.CDNName = nil
- ds2.CheckPath = util.StrPtr("foo")
- ds2.DisplayName = util.StrPtr("")
- ds2.DSCP = nil
- ds2.LogsEnabled = util.BoolPtr(true)
- ds2.MatchList = &dsMatchPtr
- ds2.MissLat = util.FloatPtr(0)
- ds2.MissLong = util.FloatPtr(42)
- ds2.Signed = true
- ds2.Type = nil
- ds2.ExampleURLs = nil
-
- dses := []tc.DeliveryServiceNullable{ds1, ds2}
-
- fileName := "TestDefaultCacheFormatIsomorphic"
-
- tmpDir := os.TempDir()
- filePath := filepath.Join(tmpDir, fileName)
- defer os.Remove(filePath)
-
- WriteCache(DefaultCacheFormat.Encoder, tmpDir, fileName, dses)
-
- readDSes := []tc.DeliveryServiceNullable{}
- if err := ReadCache(DefaultCacheFormat.Decoder, tmpDir, fileName, time.Hour, &readDSes); err != nil {
- t.Fatalf("ReadCache error expected nil, actual: " + err.Error())
- }
- if len(readDSes) != 2 {
- t.Fatalf("ReadCache error expected 2 dses, actual: %v", len(readDSes))
- }
-
- if !reflect.DeepEqual(dses[0], readDSes[0]) {
- dsj, _ := json.MarshalIndent(dses[0], "", " ")
- dsrj, _ := json.MarshalIndent(readDSes[0], "", " ")
- t.Errorf("ReadCache expected %+v actual %+v\n", string(dsj), string(dsrj))
- }
-
- if !reflect.DeepEqual(dses[1], readDSes[1]) {
- dsj, _ := json.MarshalIndent(dses[1], "", " ")
- dsrj, _ := json.MarshalIndent(readDSes[1], "", " ")
- t.Errorf("ReadCache expected %+v actual %+v\n", string(dsj), string(dsrj))
- }
-}
diff --git a/traffic_ops/ort/atstccfg/toreq/toreq.go b/traffic_ops/ort/atstccfg/toreq/toreq.go
index 8b1d5ed..7262dd8 100644
--- a/traffic_ops/ort/atstccfg/toreq/toreq.go
+++ b/traffic_ops/ort/atstccfg/toreq/toreq.go
@@ -22,8 +22,9 @@ package toreq
import (
"encoding/base64"
"errors"
- "fmt"
"strconv"
+ "strings"
+ "time"
"github.com/apache/trafficcontrol/lib/go-atscfg"
"github.com/apache/trafficcontrol/lib/go-log"
@@ -32,78 +33,42 @@ import (
"github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
)
-func GetProfile(cfg config.TCCfg, profileID int) (tc.Profile, error) {
- profile := tc.Profile{}
- err := GetCached(cfg, "profile_"+strconv.Itoa(profileID), &profile, func(obj interface{}) error {
- toProfiles, reqInf, err := (*cfg.TOClient).GetProfileByID(profileID)
- if err != nil {
- return errors.New("getting profile '" + strconv.Itoa(profileID) + "' from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
- }
- if len(toProfiles) != 1 {
- return errors.New("getting profile '" + strconv.Itoa(profileID) + "'from Traffic Ops '" + MaybeIPStr(reqInf) + "': expected 1 Profile, got " + strconv.Itoa(len(toProfiles)))
- }
-
- profile := obj.(*tc.Profile)
- *profile = toProfiles[0]
- return nil
- })
- if err != nil {
- return tc.Profile{}, errors.New("getting profile '" + strconv.Itoa(profileID) + "': " + err.Error())
- }
- return profile, nil
-}
-
func GetProfileByName(cfg config.TCCfg, profileName string) (tc.Profile, error) {
profile := tc.Profile{}
- err := GetCached(cfg, "profile_"+profileName, &profile, func(obj interface{}) error {
+
+ err := GetRetry(cfg, "profile_"+profileName, &profile, func(obj interface{}) error {
toProfiles, reqInf, err := (*cfg.TOClient).GetProfileByName(profileName)
if err != nil {
- return errors.New("getting profile '" + profileName + "' from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting profile '" + profileName + "' from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
if len(toProfiles) != 1 {
- return errors.New("getting profile '" + profileName + "'from Traffic Ops '" + MaybeIPStr(reqInf) + "': expected 1 Profile, got " + strconv.Itoa(len(toProfiles)))
+ return errors.New("getting profile '" + profileName + "'from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': expected 1 Profile, got " + strconv.Itoa(len(toProfiles)))
}
profile := obj.(*tc.Profile)
*profile = toProfiles[0]
return nil
})
+
if err != nil {
return tc.Profile{}, errors.New("getting profile '" + profileName + "': " + err.Error())
}
return profile, nil
}
-func GetProfileParameters(cfg config.TCCfg, profileName string) ([]tc.Parameter, error) {
- profileParameters := []tc.Parameter{}
- err := GetCached(cfg, "profile_"+profileName+"_parameters", &profileParameters, func(obj interface{}) error {
- toParams, reqInf, err := (*cfg.TOClient).GetParametersByProfileName(profileName)
- if err != nil {
- return errors.New("getting profile '" + profileName + "' parameters from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
- }
- params := obj.(*[]tc.Parameter)
- *params = toParams
- return nil
- })
- if err != nil {
- return nil, errors.New("getting profile '" + profileName + "' parameters: " + err.Error())
- }
- return profileParameters, nil
-}
-
func GetGlobalParameters(cfg config.TCCfg) ([]tc.Parameter, error) {
globalParams := []tc.Parameter{}
- err := GetCached(cfg, "profile_global_parameters", &globalParams, func(obj interface{}) error {
- toParams, reqInf, err := (*cfg.TOClient).GetParametersByProfileName(config.GlobalProfileName)
+ err := GetRetry(cfg, "profile_global_parameters", &globalParams, func(obj interface{}) error {
+ toParams, reqInf, err := (*cfg.TOClient).GetParametersByProfileName(tc.GlobalProfileName)
if err != nil {
- return errors.New("getting global profile '" + config.GlobalProfileName + "' parameters from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting global profile '" + tc.GlobalProfileName + "' parameters from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
params := obj.(*[]tc.Parameter)
*params = toParams
return nil
})
if err != nil {
- return nil, errors.New("getting global profile '" + config.GlobalProfileName + "' parameters: " + err.Error())
+ return nil, errors.New("getting global profile '" + tc.GlobalProfileName + "' parameters: " + err.Error())
}
return globalParams, nil
}
@@ -131,21 +96,12 @@ func GetTOToolNameAndURL(globalParams []tc.Parameter) (string, string) {
return toToolName, toURL
}
-func GetTOToolNameAndURLFromTO(cfg config.TCCfg) (string, string, error) {
- globalParams, err := GetGlobalParameters(cfg)
- if err != nil {
- return "", "", err
- }
- toToolName, toURL := GetTOToolNameAndURL(globalParams)
- return toToolName, toURL, nil
-}
-
func GetServers(cfg config.TCCfg) ([]tc.Server, error) {
servers := []tc.Server{}
- err := GetCached(cfg, "servers", &servers, func(obj interface{}) error {
+ err := GetRetry(cfg, "servers", &servers, func(obj interface{}) error {
toServers, reqInf, err := (*cfg.TOClient).GetServers()
if err != nil {
- return errors.New("getting servers from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting servers from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
servers := obj.(*[]tc.Server)
*servers = toServers
@@ -159,12 +115,12 @@ func GetServers(cfg config.TCCfg) ([]tc.Server, error) {
func GetServerByHostName(cfg config.TCCfg, serverHostName string) (tc.Server, error) {
server := tc.Server{}
- err := GetCached(cfg, "server-name-"+serverHostName, &server, func(obj interface{}) error {
+ err := GetRetry(cfg, "server-name-"+serverHostName, &server, func(obj interface{}) error {
toServers, reqInf, err := (*cfg.TOClient).GetServerByHostName(serverHostName)
if err != nil {
- return errors.New("getting server name '" + serverHostName + "' from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting server name '" + serverHostName + "' from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
} else if len(toServers) < 1 {
- return errors.New("getting server name '" + serverHostName + "' from Traffic Ops '" + MaybeIPStr(reqInf) + "': no servers returned")
+ return errors.New("getting server name '" + serverHostName + "' from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': no servers returned")
}
server := obj.(*tc.Server)
*server = toServers[0]
@@ -176,31 +132,12 @@ func GetServerByHostName(cfg config.TCCfg, serverHostName string) (tc.Server, er
return server, nil
}
-func GetServerByID(cfg config.TCCfg, serverID int) (tc.Server, error) {
- server := tc.Server{}
- err := GetCached(cfg, "server-id-"+strconv.Itoa(serverID), &server, func(obj interface{}) error {
- toServers, reqInf, err := (*cfg.TOClient).GetServerByID(serverID)
- if err != nil {
- return fmt.Errorf("getting server id %v from Traffic Ops '%v': %v", serverID, MaybeIPStr(reqInf), err)
- } else if len(toServers) < 1 {
- return fmt.Errorf("getting server id %v from Traffic Ops '%v': %v", serverID, MaybeIPStr(reqInf), "no servers returned")
- }
- server := obj.(*tc.Server)
- *server = toServers[0]
- return nil
- })
- if err != nil {
- return tc.Server{}, fmt.Errorf("getting server id %v: %v", serverID, err)
- }
- return server, nil
-}
-
func GetCacheGroups(cfg config.TCCfg) ([]tc.CacheGroupNullable, error) {
cacheGroups := []tc.CacheGroupNullable{}
- err := GetCached(cfg, "cachegroups", &cacheGroups, func(obj interface{}) error {
+ err := GetRetry(cfg, "cachegroups", &cacheGroups, func(obj interface{}) error {
toCacheGroups, reqInf, err := (*cfg.TOClient).GetCacheGroupsNullable()
if err != nil {
- return errors.New("getting cachegroups from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting cachegroups from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
cacheGroups := obj.(*[]tc.CacheGroupNullable)
*cacheGroups = toCacheGroups
@@ -236,11 +173,11 @@ func GetDeliveryServiceServers(cfg config.TCCfg, dsIDs []int, serverIDs []int) (
}
dsServers := []tc.DeliveryServiceServer{}
- err := GetCachedDSS(cfg, "deliveryservice_servers_s"+serverIDsStr+"_d_"+dsIDsStr, &dsServers, func(obj interface{}) error {
+ err := GetRetry(cfg, "deliveryservice_servers_s"+serverIDsStr+"_d_"+dsIDsStr, &dsServers, func(obj interface{}) error {
const noLimit = 999999 // TODO add "no limit" param to DSS endpoint
toDSS, reqInf, err := (*cfg.TOClient).GetDeliveryServiceServersWithLimits(noLimit, dsIDsToFetch, sIDsToFetch)
if err != nil {
- return errors.New("getting delivery service servers from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting delivery service servers from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
dss := obj.(*[]tc.DeliveryServiceServer)
*dss = toDSS.Response
@@ -284,10 +221,10 @@ func GetDeliveryServiceServers(cfg config.TCCfg, dsIDs []int, serverIDs []int) (
func GetServerProfileParameters(cfg config.TCCfg, profileName string) ([]tc.Parameter, error) {
serverProfileParameters := []tc.Parameter{}
- err := GetCached(cfg, "profile_"+profileName+"_parameters", &serverProfileParameters, func(obj interface{}) error {
+ err := GetRetry(cfg, "profile_"+profileName+"_parameters", &serverProfileParameters, func(obj interface{}) error {
toParams, reqInf, err := (*cfg.TOClient).GetParametersByProfileName(profileName)
if err != nil {
- return errors.New("getting server profile '" + profileName + "' parameters from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting server profile '" + profileName + "' parameters from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
params := obj.(*[]tc.Parameter)
*params = toParams
@@ -301,10 +238,10 @@ func GetServerProfileParameters(cfg config.TCCfg, profileName string) ([]tc.Para
func GetCDNDeliveryServices(cfg config.TCCfg, cdnID int) ([]tc.DeliveryServiceNullable, error) {
deliveryServices := []tc.DeliveryServiceNullable{}
- err := GetCached(cfg, "cdn_"+strconv.Itoa(cdnID)+"_deliveryservices", &deliveryServices, func(obj interface{}) error {
+ err := GetRetry(cfg, "cdn_"+strconv.Itoa(cdnID)+"_deliveryservices", &deliveryServices, func(obj interface{}) error {
toDSes, reqInf, err := (*cfg.TOClient).GetDeliveryServicesByCDNID(cdnID)
if err != nil {
- return errors.New("getting delivery services from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting delivery services from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
dses := obj.(*[]tc.DeliveryServiceNullable)
*dses = toDSes
@@ -318,10 +255,10 @@ func GetCDNDeliveryServices(cfg config.TCCfg, cdnID int) ([]tc.DeliveryServiceNu
func GetConfigFileParameters(cfg config.TCCfg, configFile string) ([]tc.Parameter, error) {
params := []tc.Parameter{}
- err := GetCached(cfg, "config_file_"+configFile+"_parameters", ¶ms, func(obj interface{}) error {
+ err := GetRetry(cfg, "config_file_"+configFile+"_parameters", ¶ms, func(obj interface{}) error {
toParams, reqInf, err := (*cfg.TOClient).GetParameterByConfigFile(configFile)
if err != nil {
- return errors.New("getting delivery services from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting delivery services from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
params := obj.(*[]tc.Parameter)
*params = toParams
@@ -335,33 +272,13 @@ func GetConfigFileParameters(cfg config.TCCfg, configFile string) ([]tc.Paramete
func GetCDN(cfg config.TCCfg, cdnName tc.CDNName) (tc.CDN, error) {
cdn := tc.CDN{}
- err := GetCached(cfg, "cdn_"+string(cdnName), &cdn, func(obj interface{}) error {
+ err := GetRetry(cfg, "cdn_"+string(cdnName), &cdn, func(obj interface{}) error {
toCDNs, reqInf, err := (*cfg.TOClient).GetCDNByName(string(cdnName))
if err != nil {
- return errors.New("getting cdn from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting cdn from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
if len(toCDNs) != 1 {
- return errors.New("getting cdn from Traffic Ops '" + MaybeIPStr(reqInf) + "': expected 1 CDN, got " + strconv.Itoa(len(toCDNs)))
- }
- cdn := obj.(*tc.CDN)
- *cdn = toCDNs[0]
- return nil
- })
- if err != nil {
- return tc.CDN{}, errors.New("getting cdn: " + err.Error())
- }
- return cdn, nil
-}
-
-func GetCDNByID(cfg config.TCCfg, cdnID int) (tc.CDN, error) {
- cdn := tc.CDN{}
- err := GetCached(cfg, "cdn_id_"+strconv.Itoa(cdnID), &cdn, func(obj interface{}) error {
- toCDNs, reqInf, err := (*cfg.TOClient).GetCDNByID(cdnID)
- if err != nil {
- return errors.New("getting cdn from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
- }
- if len(toCDNs) != 1 {
- return errors.New("getting cdn from Traffic Ops '" + MaybeIPStr(reqInf) + "': expected 1 CDN, got " + strconv.Itoa(len(toCDNs)))
+ return errors.New("getting cdn from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': expected 1 CDN, got " + strconv.Itoa(len(toCDNs)))
}
cdn := obj.(*tc.CDN)
*cdn = toCDNs[0]
@@ -375,10 +292,10 @@ func GetCDNByID(cfg config.TCCfg, cdnID int) (tc.CDN, error) {
func GetURLSigKeys(cfg config.TCCfg, dsName string) (tc.URLSigKeys, error) {
keys := tc.URLSigKeys{}
- err := GetCached(cfg, "urlsigkeys_"+string(dsName), &keys, func(obj interface{}) error {
+ err := GetRetry(cfg, "urlsigkeys_"+string(dsName), &keys, func(obj interface{}) error {
toKeys, reqInf, err := (*cfg.TOClient).GetDeliveryServiceURLSigKeys(dsName)
if err != nil {
- return errors.New("getting url sig keys from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting url sig keys from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
keys := obj.(*tc.URLSigKeys)
*keys = toKeys
@@ -392,10 +309,10 @@ func GetURLSigKeys(cfg config.TCCfg, dsName string) (tc.URLSigKeys, error) {
func GetURISigningKeys(cfg config.TCCfg, dsName string) ([]byte, error) {
keys := []byte{}
- err := GetCached(cfg, "urisigningkeys_"+string(dsName), &keys, func(obj interface{}) error {
+ err := GetRetry(cfg, "urisigningkeys_"+string(dsName), &keys, func(obj interface{}) error {
toKeys, reqInf, err := (*cfg.TOClient).GetDeliveryServiceURISigningKeys(dsName)
if err != nil {
- return errors.New("getting url sig keys from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting url sig keys from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
keys := obj.(*[]byte)
@@ -410,10 +327,10 @@ func GetURISigningKeys(cfg config.TCCfg, dsName string) ([]byte, error) {
func GetParametersByName(cfg config.TCCfg, paramName string) ([]tc.Parameter, error) {
params := []tc.Parameter{}
- err := GetCached(cfg, "parameters_name_"+paramName, ¶ms, func(obj interface{}) error {
+ err := GetRetry(cfg, "parameters_name_"+paramName, ¶ms, func(obj interface{}) error {
toParams, reqInf, err := (*cfg.TOClient).GetParameterByName(paramName)
if err != nil {
- return errors.New("getting parameters name '" + paramName + "' from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting parameters name '" + paramName + "' from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
params := obj.(*[]tc.Parameter)
*params = toParams
@@ -425,38 +342,12 @@ func GetParametersByName(cfg config.TCCfg, paramName string) ([]tc.Parameter, er
return params, nil
}
-func GetCacheGroupParameters(cfg config.TCCfg, cacheGroupID int) ([]tc.Parameter, error) {
- params := []tc.Parameter{}
- err := GetCached(cfg, "cachegroup_parameters_id_"+strconv.Itoa(cacheGroupID), ¶ms, func(obj interface{}) error {
- toParams, reqInf, err := (*cfg.TOClient).GetCacheGroupParameters(cacheGroupID)
- if err != nil {
- return errors.New("getting cachegroup parameters id '" + strconv.Itoa(cacheGroupID) + "' from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
- }
- params := obj.(*[]tc.Parameter)
- for _, cgParam := range toParams {
- *params = append(*params, tc.Parameter{
- ConfigFile: cgParam.ConfigFile,
- ID: cgParam.ID,
- LastUpdated: cgParam.LastUpdated,
- Name: cgParam.Name,
- Secure: cgParam.Secure,
- Value: cgParam.Value,
- })
- }
- return nil
- })
- if err != nil {
- return nil, errors.New("getting params cachegroup id '" + strconv.Itoa(cacheGroupID) + "': " + err.Error())
- }
- return params, nil
-}
-
func GetDeliveryServiceRegexes(cfg config.TCCfg) ([]tc.DeliveryServiceRegexes, error) {
regexes := []tc.DeliveryServiceRegexes{}
- err := GetCached(cfg, "ds_regexes", ®exes, func(obj interface{}) error {
+ err := GetRetry(cfg, "ds_regexes", ®exes, func(obj interface{}) error {
toRegexes, reqInf, err := (*cfg.TOClient).GetDeliveryServiceRegexes()
if err != nil {
- return errors.New("getting ds regexes from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting ds regexes from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
regexes := obj.(*[]tc.DeliveryServiceRegexes)
*regexes = toRegexes
@@ -470,10 +361,10 @@ func GetDeliveryServiceRegexes(cfg config.TCCfg) ([]tc.DeliveryServiceRegexes, e
func GetJobs(cfg config.TCCfg) ([]tc.Job, error) {
jobs := []tc.Job{}
- err := GetCached(cfg, "jobs", &jobs, func(obj interface{}) error {
+ err := GetRetry(cfg, "jobs", &jobs, func(obj interface{}) error {
toJobs, reqInf, err := (*cfg.TOClient).GetJobs(nil, nil)
if err != nil {
- return errors.New("getting jobs from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting jobs from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
jobs := obj.(*[]tc.Job)
*jobs = toJobs
@@ -493,11 +384,11 @@ func GetServerCapabilitiesByID(cfg config.TCCfg, serverIDs []int) (map[int]map[a
}
serverCaps := map[int]map[atscfg.ServerCapability]struct{}{}
- err := GetCached(cfg, "server_capabilities_s_"+serverIDsStr, &serverCaps, func(obj interface{}) error {
+ err := GetRetry(cfg, "server_capabilities_s_"+serverIDsStr, &serverCaps, func(obj interface{}) error {
// TODO add list of IDs to API+Client
toServerCaps, reqInf, err := (*cfg.TOClient).GetServerServerCapabilities(nil, nil, nil)
if err != nil {
- return errors.New("getting server caps from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting server caps from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
serverCaps := obj.(*map[int]map[atscfg.ServerCapability]struct{})
@@ -529,11 +420,11 @@ func GetDeliveryServiceRequiredCapabilitiesByID(cfg config.TCCfg, dsIDs []int) (
}
dsCaps := map[int]map[atscfg.ServerCapability]struct{}{}
- err := GetCached(cfg, "ds_capabilities_d_"+dsIDsStr, &dsCaps, func(obj interface{}) error {
+ err := GetRetry(cfg, "ds_capabilities_d_"+dsIDsStr, &dsCaps, func(obj interface{}) error {
// TODO add list of IDs to API+Client
toDSCaps, reqInf, err := (*cfg.TOClient).GetDeliveryServicesRequiredCapabilities(nil, nil, nil)
if err != nil {
- return errors.New("getting ds caps from Traffic Ops '" + MaybeIPStr(reqInf) + "': " + err.Error())
+ return errors.New("getting ds caps from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
dsCaps := obj.(*map[int]map[atscfg.ServerCapability]struct{})
@@ -557,32 +448,81 @@ func GetDeliveryServiceRequiredCapabilitiesByID(cfg config.TCCfg, dsIDs []int) (
return dsCaps, nil
}
-func GetProfileNameFromProfileNameOrID(cfg config.TCCfg, profileNameOrID string) (string, error) {
- profileName := profileNameOrID
- if profileID, err := strconv.Atoi(profileNameOrID); err == nil {
- profile, err := GetProfile(cfg, profileID)
+func GetCDNSSLKeys(cfg config.TCCfg, cdnName tc.CDNName) ([]tc.CDNSSLKeys, error) {
+ keys := []tc.CDNSSLKeys{}
+ err := GetRetry(cfg, "cdn_sslkeys_"+string(cdnName), &keys, func(obj interface{}) error {
+ toKeys, reqInf, err := (*cfg.TOClient).GetCDNSSLKeys(string(cdnName))
if err != nil {
- return "", errors.New("getting profile '" + profileNameOrID + "': " + err.Error())
+ return errors.New("getting cdn ssl keys from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
- if profile.Name == "" {
- return "", errors.New("getting profile '" + profileNameOrID + "': got profile with empty name")
+ keys := obj.(*[]tc.CDNSSLKeys)
+ *keys = toKeys
+ return nil
+ })
+ if err != nil {
+ return []tc.CDNSSLKeys{}, errors.New("getting cdn ssl keys: " + err.Error())
+ }
+ return keys, nil
+}
+
+func GetStatuses(cfg config.TCCfg) ([]tc.Status, error) {
+ statuses := []tc.Status{}
+ err := GetRetry(cfg, "statuses", &statuses, func(obj interface{}) error {
+ toStatus, reqInf, err := (*cfg.TOClient).GetStatuses()
+ if err != nil {
+ return errors.New("getting server update status from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
- profileName = profile.Name
+ status := obj.(*[]tc.Status)
+ *status = toStatus
+ return nil
+ })
+ if err != nil {
+ return nil, errors.New("getting server update status: " + err.Error())
}
- return profileName, nil
+ return statuses, nil
}
-func GetCDNNameFromCDNNameOrID(cfg config.TCCfg, cdnNameOrID string) (tc.CDNName, error) {
- cdnName := cdnNameOrID
- if cdnID, err := strconv.Atoi(cdnNameOrID); err == nil {
- cdn, err := GetCDNByID(cfg, cdnID)
+func GetServerUpdateStatus(cfg config.TCCfg) (tc.ServerUpdateStatus, error) {
+ status := tc.ServerUpdateStatus{}
+ err := GetRetry(cfg, "server_update_status_"+string(cfg.CacheHostName), &status, func(obj interface{}) error {
+ toStatus, reqInf, err := (*cfg.TOClient).GetServerUpdateStatus(string(cfg.CacheHostName))
if err != nil {
- return "", errors.New("getting cdn '" + cdnNameOrID + "': " + err.Error())
+ return errors.New("getting server update status from Traffic Ops '" + MaybeIPStr(reqInf.RemoteAddr) + "': " + err.Error())
}
- if cdn.Name == "" {
- return "", errors.New("getting cdn '" + cdnNameOrID + "': got cdn with empty name")
+ status := obj.(*tc.ServerUpdateStatus)
+ *status = toStatus
+ return nil
+ })
+ if err != nil {
+ return tc.ServerUpdateStatus{}, errors.New("getting server update status: " + err.Error())
+ }
+ return status, nil
+}
+
+// GetRetry attempts to get the given object from tempDir/cacheFileName, retrying with exponential backoff up to cfg.NumRetries.
+// The cacheFileName is not used in actual fetching or logic, but only for logging. It can be any printable string, but should be unique and reflect the object being fetched.
+func GetRetry(cfg config.TCCfg, cacheFileName string, obj interface{}, getter func(obj interface{}) error) error {
+ start := time.Now()
+ currentRetry := 0
+ for {
+ err := getter(obj)
+ if err == nil {
+ break
+ }
+ if strings.Contains(strings.ToLower(err.Error()), "not found") {
+ // if the server returned a 404, retrying won't help
+ return errors.New("getting uncached: " + err.Error())
+ }
+ if currentRetry == cfg.NumRetries {
+ return errors.New("getting uncached: " + err.Error())
}
- cdnName = cdn.Name
+
+ sleepSeconds := config.RetryBackoffSeconds(currentRetry)
+ log.Warnf("getting '%v', sleeping for %v seconds: %v\n", cacheFileName, sleepSeconds, err)
+ currentRetry++
+ time.Sleep(time.Second * time.Duration(sleepSeconds)) // TODO make backoff configurable?
}
- return tc.CDNName(cdnName), nil
+
+ log.Infof("GetRetry %v retries %v took %v\n", cacheFileName, currentRetry, time.Since(start).Round(time.Millisecond))
+ return nil
}
diff --git a/traffic_ops/ort/atstccfg/toreq/trafficops.go b/traffic_ops/ort/atstccfg/toreq/trafficops.go
index 1578437..983a901 100644
--- a/traffic_ops/ort/atstccfg/toreq/trafficops.go
+++ b/traffic_ops/ort/atstccfg/toreq/trafficops.go
@@ -23,87 +23,34 @@ import (
"bytes"
"crypto/sha512"
"encoding/base64"
- "encoding/json"
"errors"
"io"
"io/ioutil"
"net"
"net/http"
- "net/http/cookiejar"
"net/http/httptrace"
- "net/url"
"strconv"
"time"
- "golang.org/x/net/publicsuffix"
-
"github.com/apache/trafficcontrol/lib/go-log"
- "github.com/apache/trafficcontrol/lib/go-tc"
toclient "github.com/apache/trafficcontrol/traffic_ops/client"
"github.com/apache/trafficcontrol/traffic_ops/ort/atstccfg/config"
)
-// GetClient returns a TO Client, using a cached cookie if it exists, or logging in otherwise
-func GetClient(toURL string, toUser string, toPass string, tempDir string, cacheFileMaxAge time.Duration, toTimeout time.Duration, toInsecure bool) (*toclient.Session, error) {
- cookies, err := GetCookiesFromFile(tempDir, cacheFileMaxAge)
- if err != nil {
- log.Infoln("failed to get cookies from cache file (trying real TO): " + err.Error())
- cookies = ""
- }
-
- if cookies == "" {
- err := error(nil)
- cookies, err = GetCookiesFromTO(toURL, toUser, toPass, tempDir, toTimeout, toInsecure)
- if err != nil {
- return nil, errors.New("getting cookies from Traffic Ops: " + err.Error())
- }
- log.Infoln("using cookies from TO")
- } else {
- log.Infoln("using cookies from cache file")
- }
-
- useCache := false
- toClient := toclient.NewNoAuthSession(toURL, toInsecure, config.UserAgent, useCache, toTimeout)
- toClient.UserName = toUser
- toClient.Password = toPass
-
- jar, err := cookiejar.New(&cookiejar.Options{PublicSuffixList: publicsuffix.List})
- if err != nil {
- return nil, errors.New("making cookie jar: " + err.Error())
- }
- toClient.Client.Jar = jar
-
- toURLParsed, err := url.Parse(toURL)
- if err != nil {
- return nil, errors.New("parsing Traffic Ops URL '" + toURL + "': " + err.Error())
- }
+// GetTCCfg takes the config.Cfg and logs into Traffic Ops, returning the TCCfg which contains the logged-in client.
+func GetTCCfg(cfg config.Cfg) (config.TCCfg, error) {
+ log.Infoln("URL: '" + cfg.TOURL.String() + "' User: '" + cfg.TOUser + "' Pass len: '" + strconv.Itoa(len(cfg.TOPass)) + "'")
- toClient.Client.Jar.SetCookies(toURLParsed, StringToCookies(cookies))
- return toClient, nil
-}
-
-// GetCookies gets the cookies from logging in to Traffic Ops.
-// If this succeeds, it also writes the cookies to TempSubdir/TempCookieFileName.
-func GetCookiesFromTO(toURL string, toUser string, toPass string, tempDir string, toTimeout time.Duration, toInsecure bool) (string, error) {
- toURLParsed, err := url.Parse(toURL)
- if err != nil {
- return "", errors.New("parsing Traffic Ops URL '" + toURL + "': " + err.Error())
- }
+ toFQDN := cfg.TOURL.Scheme + "://" + cfg.TOURL.Host
+ log.Infoln("TO FQDN: '" + toFQDN + "'")
+ log.Infoln("TO URL: '" + cfg.TOURL.String() + "'")
- toUseCache := false
- toClient, toIP, err := toclient.LoginWithAgent(toURL, toUser, toPass, toInsecure, config.UserAgent, toUseCache, toTimeout)
+ toClient, toIP, err := toclient.LoginWithAgent(toFQDN, cfg.TOUser, cfg.TOPass, cfg.TOInsecure, config.UserAgent, false, cfg.TOTimeout)
if err != nil {
- toIPStr := ""
- if toIP != nil {
- toIPStr = toIP.String()
- }
- return "", errors.New("logging in to Traffic Ops IP '" + toIPStr + "': " + err.Error())
+ return config.TCCfg{}, errors.New("Logging in to Traffic Ops '" + MaybeIPStr(toIP) + "': " + err.Error())
}
- cookiesStr := CookiesToString(toClient.Client.Jar.Cookies(toURLParsed))
- WriteCookiesToFile(cookiesStr, tempDir)
-
- return cookiesStr, nil
+ return config.TCCfg{Cfg: cfg, TOClient: &toClient}, nil
}
// TrafficOpsRequest makes a request to Traffic Ops for the given method, url, and body.
@@ -255,48 +202,9 @@ func rawTrafficOpsRequest(toClient *toclient.Session, method string, url string,
}
// MaybeIPStr returns the Traffic Ops IP string if it isn't nil, or the empty string if it is.
-func MaybeIPStr(reqInf toclient.ReqInf) string {
- if reqInf.RemoteAddr != nil {
- return reqInf.RemoteAddr.String()
+func MaybeIPStr(addr net.Addr) string {
+ if addr != nil {
+ return addr.String()
}
return ""
}
-
-// TCParamsToParamsWithProfiles unmarshals the Profiles that the tc struct doesn't.
-func TCParamsToParamsWithProfiles(tcParams []tc.Parameter) ([]ParameterWithProfiles, error) {
- params := make([]ParameterWithProfiles, 0, len(tcParams))
- for _, tcParam := range tcParams {
- param := ParameterWithProfiles{Parameter: tcParam}
-
- profiles := []string{}
- if err := json.Unmarshal(tcParam.Profiles, &profiles); err != nil {
- return nil, errors.New("unmarshalling JSON from parameter '" + strconv.Itoa(param.ID) + "': " + err.Error())
- }
- param.ProfileNames = profiles
- param.Profiles = nil
- params = append(params, param)
- }
- return params, nil
-}
-
-type ParameterWithProfiles struct {
- tc.Parameter
- ProfileNames []string
-}
-
-type ParameterWithProfilesMap struct {
- tc.Parameter
- ProfileNames map[string]struct{}
-}
-
-func ParameterWithProfilesToMap(tcParams []ParameterWithProfiles) []ParameterWithProfilesMap {
- params := []ParameterWithProfilesMap{}
- for _, tcParam := range tcParams {
- param := ParameterWithProfilesMap{Parameter: tcParam.Parameter, ProfileNames: map[string]struct{}{}}
- for _, profile := range tcParam.ProfileNames {
- param.ProfileNames[profile] = struct{}{}
- }
- params = append(params, param)
- }
- return params
-}
diff --git a/traffic_ops/ort/traffic_ops_ort.pl b/traffic_ops/ort/traffic_ops_ort.pl
index aae7a3b..bed05ad 100755
--- a/traffic_ops/ort/traffic_ops_ort.pl
+++ b/traffic_ops/ort/traffic_ops_ort.pl
@@ -18,13 +18,9 @@ use warnings;
use feature qw(switch);
use JSON;
use File::Basename;
-use File::Path;
use Fcntl qw(:flock);
use MIME::Base64;
-use LWP::UserAgent;
-use Crypt::SSLeay;
use Getopt::Long;
-use Digest::SHA qw(sha512_base64);
$| = 1;
my $date = `/bin/date`;
@@ -43,10 +39,6 @@ my $reval_in_use = 0;
my $rev_proxy_disable = 0;
my $skip_os_check = 0;
my $override_hostname_short = '';
-my $override_hostname_full = '';
-my $override_domainname = '';
-my $use_cache = 1;
-my $cache_max_age_seconds = 900;
GetOptions( "dispersion=i" => \$dispersion, # dispersion (in seconds)
"retries=i" => \$retries,
@@ -55,10 +47,6 @@ GetOptions( "dispersion=i" => \$dispersion, # dispersion (in seconds)
"rev_proxy_disable=i" => \$rev_proxy_disable,
"skip_os_check=i" => \$skip_os_check,
"override_hostname_short=s" => \$override_hostname_short,
- "override_hostname_full=s" => \$override_hostname_full,
- "override_domainname=s" => \$override_domainname,
- "use_cache=i" => \$use_cache,
- "cache_max_age_seconds=i" => \$cache_max_age_seconds,
);
if ( $#ARGV < 1 ) {
@@ -79,20 +67,17 @@ given ( $ARGV[1] ) {
default { &usage(); }
}
-my $traffic_ops_host = undef;
-my $to_url = undef;
-my $to_rev_proxy_url = undef;
-my $TM_LOGIN = undef;
+my $TO_URL = undef;
+my $TO_USER = undef;
+my $TO_PASS = undef;
if ( defined( $ARGV[2] ) ) {
if ( $ARGV[2] !~ /^https*:\/\/.*$/ ) {
&usage();
}
else {
- $traffic_ops_host = $ARGV[2];
- $traffic_ops_host =~ s/\/*$//g;
- # Stash to_url for later use...
- $to_url = $traffic_ops_host;
+ $TO_URL = $ARGV[2];
+ $TO_URL =~ s/\/*$//g;
}
}
else {
@@ -104,7 +89,7 @@ if ( defined( $ARGV[3] ) ) {
&usage();
}
else {
- $TM_LOGIN = $ARGV[3];
+ ( $TO_USER, $TO_PASS ) = split( /:/, $ARGV[3] );
}
}
else {
@@ -156,36 +141,17 @@ my $CFG_FILE_CHANGED = 2;
my $CFG_FILE_PREREQ_FAILED = 3;
my $CFG_FILE_ALREADY_PROCESSED = 4;
-#### LWP globals
-my $api_in_use = 1;
-my $rev_proxy_in_use = 0;
-my $atstccfg_cache_cleared = 0;
-my $lwp_conn = &setup_lwp();
my $unixtime = time();
my $hostname_short = `/bin/hostname -s`;
if ($override_hostname_short ne '') {
$hostname_short = $override_hostname_short;
}
chomp($hostname_short);
-my $hostname_full = `/bin/hostname`;
-if ($override_hostname_full ne '') {
- $hostname_full = $override_hostname_full;
-}
-chomp($hostname_full);
-my $server_ipv4;
-my $server_tcp_port;
-
-my $domainname = &set_domainname();
-if ($override_domainname ne '') {
- $domainname = $override_domainname;
-}
my $atstccfg_cmd = '/opt/ort/atstccfg';
-$lwp_conn->agent("$hostname_short-$unixtime");
-
my $TMP_BASE = "/tmp/ort";
-my $cookie = &get_cookie( $traffic_ops_host, $TM_LOGIN );
+my $atstccfg_log_path = $TMP_BASE . '/atstccfg.log';
# add any special yum options for your environment here; this variable is used with all yum commands
my $YUM_OPTS = "";
@@ -199,23 +165,16 @@ my $return = &check_output($out);
my @config_files = ();
#### Process reboot tracker
-my $reboot_needed = 0;
-my $traffic_ctl_needed = 0;
+my $traffic_ctl_needed = 0;
my $sysctl_p_needed = 0;
my $ntpd_restart_needed = 0;
my $trafficserver_restart_needed = 0;
-#### Process runnning tracker
-my $ats_running = 0;
-my $teakd_running = 0;
-
#### Process installed tracker
my $installed_new_ssl_keys = 0;
my %install_tracker;
-my $config_dirs = undef;
my $cfg_file_tracker = undef;
-my $ssl_tracker = undef;
####-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-####
#### Start main flow
@@ -228,12 +187,7 @@ if ( $script_mode == $BADASS || $script_mode == $INTERACTIVE || $script_mode ==
&clean_tmp_dirs();
}
-my $header_comment = &get_header_comment($traffic_ops_host);
-
-if ( !defined $traffic_ops_host ) {
- print "FATAL Could not resolve Traffic Ops host!\n";
- exit 1;
-}
+my $header_comment = &get_header_comment();
#### If this is a syncds run, check to see if we can bail.
my $syncds_update = 0;
@@ -248,26 +202,19 @@ else {
( $syncds_update ) = &check_syncds_state();
}
-
-( my $my_profile_name, $cfg_file_tracker, my $my_cdn_name ) = &get_cfg_file_list( $hostname_short, $traffic_ops_host, $script_mode );
+$cfg_file_tracker = &get_cfg_file_list( $hostname_short, $script_mode );
if ( $script_mode == $REVALIDATE ) {
( $log_level >> $INFO ) && print "\nINFO: ======== Revalidating, no package processing needed ========\n";
}
else {
( $log_level >> $INFO ) && print "\nINFO: ======== Start processing packages ========\n";
- &process_packages( $hostname_short, $traffic_ops_host );
+ &process_packages( $hostname_short );
# get the ats user's UID after package installation in case this is the initial badass
( $log_level >> $INFO ) && print "\nINFO: ======== Start second package processing run ========\n";
- &process_chkconfig( $hostname_short, $traffic_ops_host );
+ &process_chkconfig( $hostname_short );
}
-
-
-#### First time
-&process_config_files();
-
-#### Second time, in case there were new files added to the registry
&process_config_files();
foreach my $file ( keys ( %{$cfg_file_tracker} ) ) {
@@ -314,7 +261,7 @@ sub revalidate_while_sleeping {
$syncds_update = &check_revalidate_state(1);
if ( $syncds_update > 0 ) {
$script_mode = $REVALIDATE;
- ( my $my_profile_name, $cfg_file_tracker, my $my_cdn_name ) = &get_cfg_file_list( $hostname_short, $traffic_ops_host, $script_mode );
+ $cfg_file_tracker = &get_cfg_file_list( $hostname_short, $script_mode );
&process_config_files();
@@ -359,55 +306,34 @@ sub usage {
print "\t rev_proxy_disable=<0|1> => bypass the reverse proxy even if one has been configured Default = 0.\n";
print "\t skip_os_check=<0|1> => bypass the check for a supported CentOS version. Default = 0.\n";
print "\t override_hostname_short=<text> => override the short hostname of the OS for config generation. Default = ''.\n";
- print "\t override_hostname_full=<text> => override the full hostname of the OS for config generation. Default = ''.\n";
- print "\t override_domainname=<text> => override the domainname of the OS for config generation. Default = ''.\n";
- print "\t use_cache=<0|1> => whether to use cached Traffic Ops data for config generation. Default = 1, use cache.\n";
- print "\t cache_max_age_seconds=<time> => the max time in seconds to use cache files. Default = 900 (15 minutes).\n";
print "====-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-====\n";
exit 1;
}
sub process_cfg_file {
my $cfg_file = shift;
- my $result = ( defined( $cfg_file_tracker->{$cfg_file}->{'contents'} ) ) ? $cfg_file_tracker->{$cfg_file}->{'contents'} : undef;
- my $return_code = 0;
- my $uri;
+ if ( $cfg_file eq "" ) {
+ ( $log_level >> $ERROR ) && print "ERROR Config file name is empty!\n";
+ $cfg_file_tracker->{$cfg_file}->{'audit_failed'}++;
+ return $CFG_FILE_NOT_PROCESSED;
+ }
+
+ my $result = $cfg_file_tracker->{$cfg_file}->{'body'};
+ my $return_code = 0;
return $CFG_FILE_ALREADY_PROCESSED
if ( defined( $cfg_file_tracker->{$cfg_file}->{'audit_complete'} ) && $cfg_file_tracker->{$cfg_file}->{'audit_complete'} > 0 );
- return $CFG_FILE_NOT_PROCESSED if ( !&validate_filename($cfg_file) );
-
( $log_level >> $INFO ) && print "\nINFO: ======== Start processing config file: $cfg_file ========\n";
- my $config_dir = $cfg_file_tracker->{$cfg_file}->{'location'};
- if (!$config_dir) {
- ( $log_level >> $ERROR ) && print "ERROR No location information for $cfg_file.\n";
+ my $cfg_dir = dirname($cfg_file_tracker->{$cfg_file}->{'headers'}->{'path'});
+ if ( length($cfg_dir) == 0 ) {
+ ( $log_level >> $ERROR ) && print "ERROR No location information for $cfg_file.\n";
return $CFG_FILE_NOT_PROCESSED;
}
- $uri = &set_uri($cfg_file);
-
- &smart_mkdir($config_dir);
-
- $result = &lwp_get($uri) if ( !defined($result) && defined($uri) );
-
- return $CFG_FILE_NOT_PROCESSED if ( !&validate_result( \$uri, \$result ) );
-
- # Process __SERVER_TCP_PORT__, __HOSTNAME__, __FULL_HOSTNAME__ and __CACHE_IPV4__ values from traffic ops API.
- if ( $api_in_use == 1 ) {
- if ( $server_tcp_port != 80 ) {
- $result =~ s/__SERVER_TCP_PORT__/$server_tcp_port/g;
- }
- else {
- $result =~ s/:__SERVER_TCP_PORT__//g;
- }
- $result =~ s/__CACHE_IPV4__/$server_ipv4/g;
- $result =~ s/__HOSTNAME__/$hostname_short/g;
- $result =~ s/__FULL_HOSTNAME__/$hostname_full/g;
- $result =~ s/\s*__RETURN__\s*/\n/g;
- }
+ &smart_mkdir($cfg_dir);
# Process ##OVERRIDE## remap rules (from anymap rawtext)
if ( $cfg_file eq "remap.config" ) {
@@ -416,9 +342,7 @@ sub process_cfg_file {
my @db_file_lines = @{ &scrape_unencode_text($result) };
@db_file_lines = @{ &scrape_canned_comments(\@db_file_lines) };
-
- my $file = $config_dir . "/" . $cfg_file;
-
+ my $file = $cfg_file_tracker->{$cfg_file}->{'headers'}->{'path'};
return $CFG_FILE_PREREQ_FAILED if ( !&prereqs_ok( $cfg_file, \@db_file_lines ) );
my @disk_file_lines;
@@ -464,9 +388,6 @@ sub process_cfg_file {
if ( $cfg_file eq "50-ats.rules" ) {
&adv_processing_udev( \@db_file_lines );
}
- elsif ( $cfg_file eq "ssl_multicert.config" ) {
- &adv_processing_ssl( \@db_file_lines );
- }
( $log_level >> $INFO )
&& print "INFO: ======== End processing config file: $cfg_file for service: " . $cfg_file_tracker->{$cfg_file}->{'service'} . " ========\n";
@@ -766,10 +687,10 @@ sub update_trops {
}
if ($update_result) {
#need to know if reval_pending is supported
- my ($upd_json, $uri) = get_update_status();
+ my $upd_json = get_update_status();
- my $upd_pending = ( defined( $upd_json->[0]->{'upd_pending'} ) ) ? $upd_json->[0]->{'upd_pending'} : undef;
- my $reval_pending = ( defined( $upd_json->[0]->{'reval_pending'} ) ) ? $upd_json->[0]->{'reval_pending'} : undef;
+ my $upd_pending = ( defined( $upd_json->{'upd_pending'} ) ) ? $upd_json->{'upd_pending'} : undef;
+ my $reval_pending = ( defined( $upd_json->{'reval_pending'} ) ) ? $upd_json->{'reval_pending'} : undef;
if ( $script_mode == $INTERACTIVE ) {
( $log_level >> $ERROR ) && print "ERROR Traffic Ops needs updated. Should I do that now? [Y/n] (n): ";
@@ -802,62 +723,51 @@ sub send_update_to_trops {
my $status = shift;
my $reval_status = shift;
- # TODO: this is a non-standard API endpoint that will be removed with the Perl; need to update it.
- my $uri = "/update/$hostname_short";
- ( $log_level >> $DEBUG ) && print "DEBUG Setting update flag in Traffic Ops to $status.\n";
-
- my %headers = ( 'Cookie' => $cookie );
- my $url = $traffic_ops_host . $uri;
- my $response = $lwp_conn->post( $url, [ 'updated' => $status, 'reval_updated' => $reval_status ], %headers );
-
- &check_lwp_response_code($response, $ERROR);
+ my $upd_str='false';
+ my $reval_str='false';
+ if ( $status != 0 ) {
+ $upd_str='true';
+ }
+ if ( $reval_status != 0 ) {
+ $reval_str='true';
+ }
- ( $log_level >> $DEBUG ) && print "DEBUG Response from Traffic Ops is: " . $response->content() . ".\n";
-}
+ my $response = `$atstccfg_cmd --traffic-ops-user='$TO_USER' --traffic-ops-password='$TO_PASS' --traffic-ops-url='$TO_URL' --cache-host-name='$hostname_short' --log-location-error=stderr --log-location-warning=stderr --log-location-info=null --set-queue-status=$upd_str --set-reval-status=$reval_str 2>$atstccfg_log_path`;
+ my $atstccfg_exit_code = $?;
+ if ($atstccfg_exit_code != 0) {
+ ( $log_level >> $ERROR ) && printf("ERROR sending update status with atstccfg (via Traffic Ops). See $atstccfg_log_path.\n");
+ }
-sub get_print_current_client_connections {
- my $cmd = $TRAFFIC_CTL . " metric get proxy.process.http.current_client_connections";
- my $current_connections = `$cmd 2>/dev/null`;
- chomp($current_connections);
- ( $log_level >> $DEBUG ) && print "DEBUG There are currently $current_connections connections.\n";
+ ( $log_level >> $DEBUG ) && print "DEBUG Response from Traffic Ops is: " . $response . ".\n";
}
sub get_update_status {
- my $uri = "/api/2.0/servers/$hostname_short/update_status";
- my $upd_ref = &lwp_get($uri);
- if ($upd_ref eq '404') {
- ( $log_level >> $ERROR ) && printf("ERROR ORT version incompatible with current version of Traffic Ops. Please upgrade to Traffic Ops 2.2.\n");
- exit 1;
- }
-
- if ( $upd_ref =~ m/^\d{3}$/ ) {
- ( $log_level >> $ERROR ) && print "ERROR Update URL: $uri returned $upd_ref. Exiting, not sure what else to do.\n";
+ my $upd_ref = `$atstccfg_cmd --traffic-ops-user='$TO_USER' --traffic-ops-password='$TO_PASS' --traffic-ops-url='$TO_URL' --cache-host-name='$hostname_short' --log-location-error=stderr --log-location-warning=stderr --log-location-info=null --get-data=update-status 2>$atstccfg_log_path`;
+ my $atstccfg_exit_code = $?;
+ if ($atstccfg_exit_code != 0) {
+ ( $log_level >> $ERROR ) && printf("ERROR getting update status from atstccfg (via Traffic Ops). See $atstccfg_log_path.\n");
exit 1;
}
my $upd_json = decode_json($upd_ref);
##Some versions of Traffic Ops had the 1.3 API but did not have the use_reval_pending field. If this field is not present, exit.
- if ( !defined( $upd_json->[0]->{'use_reval_pending'} ) ) {
- my $info_uri = "/api/2.0/system/info";
- my $info_ref = &lwp_get($info_uri);
- if ($info_ref eq '404') {
+ if ( !defined( $upd_json->{'use_reval_pending'} ) ) {
+ my $info_ref = `$atstccfg_cmd --traffic-ops-user='$TO_USER' --traffic-ops-password='$TO_PASS' --traffic-ops-url='$TO_URL' --cache-host-name='$hostname_short' --log-location-error=stderr --log-location-warning=stderr --log-location-info=null --get-data=system-info 2>$atstccfg_log_path`;
+ my $atstccfg_exit_code = $?;
+ if ($atstccfg_exit_code != 0) {
( $log_level >> $ERROR ) && printf("ERROR Unable to get status of use_reval_pending parameter. Stopping.\n");
exit 1;
}
- if ( $info_ref =~ m/^\d{3}$/ ) {
- ( $log_level >> $ERROR ) && print "ERROR Update URL: $info_uri returned $info_ref. Exiting, not sure what else to do.\n";
- exit 1;
- }
my $info_json = decode_json($info_ref);
- if (defined( $info_json->{'response'}->{'parameters'}->{'use_reval_pending'} ) ) {
- $reval_in_use = $info_json->{'response'}->{'parameters'}->{'use_reval_pending'};
+ if (defined( $info_json->{'use_reval_pending'} ) ) {
+ $reval_in_use = $info_json->{'use_reval_pending'};
}
}
else {
- $reval_in_use = $upd_json->[0]->{'use_reval_pending'};
+ $reval_in_use = $upd_json->{'use_reval_pending'};
}
- return ($upd_json, $uri);
+ return $upd_json;
}
sub check_revalidate_state {
@@ -869,20 +779,20 @@ sub check_revalidate_state {
if ( $script_mode == $REVALIDATE || $sleep_override == 1 ) {
## The herd is about to get /update/<hostname>
- my ($upd_json, $uri) = get_update_status();
+ my $upd_json = get_update_status();
if ( $reval_in_use == 0 ) {
( $log_level >> $ERROR ) && print "ERROR Update URL: Instant invalidate is not enabled. Separated revalidation requires upgrading to Traffic Ops version 2.2 and enabling this feature.\n";
return($UPDATE_TROPS_NOTNEEDED);
}
- my $reval_pending = $upd_json->[0]->{'reval_pending'};
+ my $reval_pending = $upd_json->{'reval_pending'};
if ( $reval_pending == 1 ) {
( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that a revalidation is waiting to be applied.\n";
$syncds_update = $UPDATE_TROPS_NEEDED;
- my $parent_reval_pending = $upd_json->[0]->{'parent_reval_pending'};
+ my $parent_reval_pending = $upd_json->{'parent_reval_pending'};
if ( !defined($parent_reval_pending) ) {
- ( $log_level >> $ERROR ) && print "ERROR Update URL: $uri did not have an parent_reval_pending key. Separated revalidation requires upgrading to Traffic Ops version 2.2. Unable to continue!\n";
+ ( $log_level >> $ERROR ) && print "ERROR Update URL: did not have an parent_reval_pending key. Separated revalidation requires upgrading to Traffic Ops version 2.2. Unable to continue!\n";
return($UPDATE_TROPS_NOTNEEDED);
}
if ( $parent_reval_pending == 1 ) {
@@ -898,13 +808,14 @@ sub check_revalidate_state {
( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that no revalidations are waiting to be applied.\n";
}
- my $stj = &lwp_get("/api/2.0/statuses");
- if ( $stj =~ m/^\d{3}$/ ) {
- ( $log_level >> $ERROR ) && print "Statuses URL: $uri returned $stj! Skipping creation of status file.\n";
+ my $stj = `$atstccfg_cmd --traffic-ops-user='$TO_USER' --traffic-ops-password='$TO_PASS' --traffic-ops-url='$TO_URL' --cache-host-name='$hostname_short' --log-location-error=stderr --log-location-warning=stderr --log-location-info=null --get-data=statuses 2>$atstccfg_log_path`;
+ my $atstccfg_exit_code = $?;
+ if ( $atstccfg_exit_code != 0 ) {
+ ( $log_level >> $ERROR ) && print "Statuses URL: returned $stj! Skipping creation of status file.\n";
}
my $statuses = decode_json($stj);
- my $my_status = ( defined( $upd_json->[0]->{'status'} ) ) ? $upd_json->[0]->{'status'} : undef;
+ my $my_status = ( defined( $upd_json->{'status'} ) ) ? $upd_json->{'status'} : undef;
if ( defined($my_status) ) {
( $log_level >> $DEBUG ) && print "DEBUG Found $my_status status from Traffic Ops.\n";
@@ -949,16 +860,16 @@ sub check_syncds_state {
## The herd is about to get /update/<hostname>
## need to check if revalidation is being used first.
- my ($upd_json, $uri) = get_update_status();
+ my $upd_json = get_update_status();
- my $upd_pending = ( defined( $upd_json->[0]->{'upd_pending'} ) ) ? $upd_json->[0]->{'upd_pending'} : undef;
+ my $upd_pending = ( defined( $upd_json->{'upd_pending'} ) ) ? $upd_json->{'upd_pending'} : undef;
if ( !defined($upd_pending) ) {
- ( $log_level >> $ERROR ) && print "ERROR Update URL: $uri did not have an upd_pending key.\n";
+ ( $log_level >> $ERROR ) && print "ERROR Update URL: did not have an upd_pending key.\n";
if ( $script_mode != $SYNCDS ) {
return $syncds_update;
}
else {
- ( $log_level >> $ERROR ) && print "ERROR Invalid JSON for $uri. Exiting, not sure what else to do.\n";
+ ( $log_level >> $ERROR ) && print "ERROR Invalid JSON for update_status. Exiting, not sure what else to do.\n";
exit 1;
}
}
@@ -968,15 +879,15 @@ sub check_syncds_state {
( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that an update is waiting to be applied.\n";
$syncds_update = $UPDATE_TROPS_NEEDED;
- my $parent_pending = ( defined( $upd_json->[0]->{'parent_pending'} ) ) ? $upd_json->[0]->{'parent_pending'} : undef;
- my $parent_reval_pending = ( defined( $upd_json->[0]->{'parent_reval_pending'} ) ) ? $upd_json->[0]->{'parent_reval_pending'} : undef;
+ my $parent_pending = ( defined( $upd_json->{'parent_pending'} ) ) ? $upd_json->{'parent_pending'} : undef;
+ my $parent_reval_pending = ( defined( $upd_json->{'parent_reval_pending'} ) ) ? $upd_json->{'parent_reval_pending'} : undef;
if ( !defined($parent_pending) ) {
- ( $log_level >> $ERROR ) && print "ERROR Update URL: $uri did not have an parent_pending key.\n";
+ ( $log_level >> $ERROR ) && print "ERROR Update URL: did not have an parent_pending key.\n";
if ( $script_mode != $SYNCDS ) {
return $syncds_update;
}
else {
- ( $log_level >> $ERROR ) && print "ERROR Invalid JSON for $uri. Exiting, not sure what else to do.\n";
+ ( $log_level >> $ERROR ) && print "ERROR Invalid JSON for update_status. Exiting, not sure what else to do.\n";
exit 1;
}
}
@@ -987,11 +898,11 @@ sub check_syncds_state {
( $log_level >> $WARN ) && print "WARN In syncds mode, sleeping for " . $dispersion . "s to see if the update my parents need is cleared.\n";
( $dispersion > 0 ) && &sleep_timer($dispersion);
}
- ($upd_json, $uri) = get_update_status();
+ $upd_json = get_update_status();
- $parent_pending = ( defined( $upd_json->[0]->{'parent_pending'} ) ) ? $upd_json->[0]->{'parent_pending'} : undef;
+ $parent_pending = ( defined( $upd_json->{'parent_pending'} ) ) ? $upd_json->{'parent_pending'} : undef;
if ( !defined($parent_pending) ) {
- ( $log_level >> $ERROR ) && print "ERROR Invalid JSON for $uri. Exiting, not sure what else to do.\n";
+ ( $log_level >> $ERROR ) && print "ERROR Invalid JSON for update_status. Exiting, not sure what else to do.\n";
}
if ( $parent_pending == 1 || $parent_reval_pending == 1 ) {
( $log_level >> $ERROR ) && print "ERROR My parents still need an update, bailing.\n";
@@ -1015,13 +926,15 @@ sub check_syncds_state {
else {
( $log_level >> $ERROR ) && print "ERROR Traffic Ops is signaling that no update is waiting to be applied.\n";
}
- my $stj = &lwp_get("/api/2.0/statuses");
- if ( $stj =~ m/^\d{3}$/ ) {
- ( $log_level >> $ERROR ) && print "Statuses URL: $uri returned $stj! Skipping creation of status file.\n";
+
+ my $stj = `$atstccfg_cmd --traffic-ops-user='$TO_USER' --traffic-ops-password='$TO_PASS' --traffic-ops-url='$TO_URL' --cache-host-name='$hostname_short' --log-location-error=stderr --log-location-warning=stderr --log-location-info=null --get-data=statuses 2>$atstccfg_log_path`;
+ my $atstccfg_exit_code = $?;
+ if ( $atstccfg_exit_code != 0 ) {
+ ( $log_level >> $ERROR ) && print "Statuses URL: returned $stj! Skipping creation of status file.\n";
}
my $statuses = decode_json($stj);
- my $my_status = ( defined( $upd_json->[0]->{'status'} ) ) ? $upd_json->[0]->{'status'} : undef;
+ my $my_status = ( defined( $upd_json->{'status'} ) ) ? $upd_json->{'status'} : undef;
if ( defined($my_status) ) {
( $log_level >> $DEBUG ) && print "DEBUG Found $my_status status from Traffic Ops.\n";
@@ -1038,7 +951,7 @@ sub check_syncds_state {
( $log_level >> $ERROR ) && print "ERROR status file $status_file does not exist.\n";
}
- for my $status ( @{$statuses->{'response'}} ) {
+ for my $status ( @{$statuses} ) {
next if ( $status->{name} eq $my_status );
my $other_status = $status_dir . "/" . $status->{name};
@@ -1128,11 +1041,12 @@ sub sleep_timer {
( $log_level >> $WARN ) && print "\n";
}
-sub process_config_files {
-
+sub process_config_files{
( $log_level >> $INFO ) && print "\nINFO: ======== Start processing config files ========\n";
+
foreach my $file ( keys %{$cfg_file_tracker} ) {
( $log_level >> $DEBUG ) && print "DEBUG Starting processing of config file: $file\n";
+ my $cfg_dir = dirname($cfg_file_tracker->{$file}->{'headers'}->{'path'});
my $return = undef;
if (
$script_mode == $SYNCDS
@@ -1168,18 +1082,18 @@ sub process_config_files {
}
elsif ($script_mode == $SYNCDS
&& $file =~ m/\_facts/
- && ( defined( $cfg_file_tracker->{$file}->{'location'} ) && $cfg_file_tracker->{$file}->{'location'} =~ m/\/opt\/ort/ ) )
+ && $cfg_dir =~ m/\/opt\/ort/ )
{
( $log_level >> $DEBUG ) && print "DEBUG In syncds mode, I'm about to process config file: $file\n";
$cfg_file_tracker->{$file}->{'service'} = "puppet";
$return = &process_cfg_file($file);
}
- elsif ( $script_mode == $SYNCDS && defined( $cfg_file_tracker->{$file}->{'location'} ) && $cfg_file_tracker->{$file}->{'location'} =~ m/cron/ ) {
+ elsif ( $script_mode == $SYNCDS && $cfg_dir =~ m/cron/ ) {
( $log_level >> $DEBUG ) && print "DEBUG In syncds mode, I'm about to process config file: $file\n";
$cfg_file_tracker->{$file}->{'service'} = "system";
$return = &process_cfg_file($file);
}
- elsif ( $script_mode == $SYNCDS && defined( $cfg_file_tracker->{$file}->{'url'} ) && defined ( $cfg_file_tracker->{$file}->{'location'} ) ) {
+ elsif ( $script_mode == $SYNCDS ) {
( $log_level >> $DEBUG ) && print "DEBUG In syncds mode, I'm about to process config file: $file\n";
$cfg_file_tracker->{$file}->{'service'} = "trafficserver";
$return = &process_cfg_file($file);
@@ -1187,8 +1101,7 @@ sub process_config_files {
elsif ( $script_mode != $SYNCDS ) {
if (
package_installed("trafficserver")
- && ( defined( $cfg_file_tracker->{$file}->{'location'} )
- && ( $cfg_file_tracker->{$file}->{'location'} =~ m/trafficserver/ || $cfg_file_tracker->{$file}->{'location'} =~ m/udev/ ) )
+ && ( $cfg_dir =~ m/trafficserver/ || $cfg_dir =~ m/udev/ )
)
{
$cfg_file_tracker->{$file}->{'service'} = "trafficserver";
@@ -1216,6 +1129,7 @@ sub process_config_files {
$syncds_update = $UPDATE_TROPS_FAILED;
}
}
+
foreach my $file ( keys %{$cfg_file_tracker} ) {
if ( $cfg_file_tracker->{$file}->{'change_needed'}
&& !$cfg_file_tracker->{$file}->{'change_applied'}
@@ -1239,20 +1153,16 @@ sub process_config_files {
}
}
}
+
( $log_level >> $INFO ) && print "\nINFO: ======== End processing config files ========\n\n";
}
sub touch_file {
my $return = 0;
my $file = shift;
- if ( defined( $cfg_file_tracker->{$file}->{'location'} ) ) {
- $file = $cfg_file_tracker->{$file}->{'location'} . "/" . $file;
- ( $log_level >> $DEBUG ) && print "DEBUG About to touch $file.\n";
- }
- else {
- ( $log_level >> $ERROR ) && print "ERROR $file has not location defined. Not touching $file.\n";
- return $return;
- }
+ $file = $cfg_file_tracker->{$file}->{'headers'}->{'path'};
+ ( $log_level >> $DEBUG ) && print "DEBUG About to touch $file.\n";
+
if ( $script_mode == $INTERACTIVE ) {
( $log_level >> $ERROR ) && print "ERROR $file needs touched. Should I do that now? [Y/n] (n): ";
my $select = 'n';
@@ -1322,7 +1232,9 @@ sub check_plugins {
( my $plugin_name ) = split( /\s+/, $linep );
$plugin_name =~ s/\s+//g;
( $log_level >> $DEBUG ) && print "DEBUG Found plugin $plugin_name in $cfg_file.\n";
+
my $return_code = &check_this_plugin($plugin_name);
+
if ( $return_code == $PLUGIN_YES ) {
( $log_level >> $DEBUG ) && print "DEBUG Package for plugin: $plugin_name is installed.\n";
}
@@ -1361,7 +1273,9 @@ sub check_plugins {
}
$plugin_name =~ s/\s//g;
( $log_level >> $DEBUG ) && print "DEBUG Found plugin $plugin_name in $cfg_file.\n";
+
$return_code = &check_this_plugin($plugin_name);
+
if ( $return_code == $PLUGIN_YES ) {
( $log_level >> $DEBUG ) && print "DEBUG Package for plugin: $plugin_name is installed.\n";
}
@@ -1373,6 +1287,7 @@ sub check_plugins {
}
}
( $log_level >> $TRACE ) && print "TRACE Returning $return_code for checking plugins for $cfg_file.\n";
+
return $return_code;
}
@@ -1450,8 +1365,15 @@ sub check_ntp {
}
}
+
+my %checked_plugins = ();
sub check_this_plugin {
my $plugin = shift;
+
+ if ( exists( $checked_plugins{$plugin} ) ) {
+ return ($checked_plugins{$plugin});
+ }
+
my $full_plugin = $TS_HOME . "/libexec/trafficserver/" . $plugin;
( $log_level >> $DEBUG ) && print "DEBUG Checking package dependency for plugin: $plugin.\n";
@@ -1462,125 +1384,15 @@ sub check_this_plugin {
$trafficserver_restart_needed++;
}
+ $checked_plugins{$plugin} = $PLUGIN_YES;
return ($PLUGIN_YES);
}
else {
+ $checked_plugins{$plugin} = $PLUGIN_NO;
return ($PLUGIN_NO);
}
}
-sub atstccfg_code_to_http_code {
- # this is necessary, because Linux codes can only be 0-256, so we map e.g. 104 -> 404 to fake the Traffic Ops response code.
- my $code = shift;
-
- my $generic_http_err = 500;
- my %atstccfg_to_http_codes = (
- 0, 200,
- 1, 500,
- 104, 404,
- );
- my $http_code = $atstccfg_to_http_codes{$code};
- if (!defined($http_code)) {
- $http_code = $generic_http_err;
- }
- return $http_code;
-}
-
-sub lwp_get {
- my $uri = shift;
- my $retry_counter = $retries;
-
- ( $log_level >> $DEBUG ) && print "DEBUG Total connections in LWP cache: " . $lwp_conn->conn_cache->get_connections("https") . "\n";
- my %headers = ( 'Cookie' => $cookie );
-
- my $response;
- my $response_content;
-
- # TODO add retry_counter arg to atstccfg
- while(1) { # no retry counter, atstccfg handles retries
- ( $log_level >> $INFO ) && print "INFO Traffic Ops host: " . $traffic_ops_host . "\n";
- ( $log_level >> $DEBUG ) && print "DEBUG lwp_get called with $uri\n";
- my $request = $traffic_ops_host . $uri;
- if ( $uri =~ m/^http/ ) {
- $request = $uri;
- ( $log_level >> $DEBUG ) && print "DEBUG Complete URL found. Downloading from external source $request.\n";
- }
- if ( ($uri =~ m/sslkeys/ || $uri =~ m/url\_sig/ || $uri =~ m/uri\_signing/) && $rev_proxy_in_use == 1 ) {
- $request = $to_url . $uri;
- ( $log_level >> $INFO ) && print "INFO Secure data request - bypassing reverse proxy and using $to_url.\n";
- }
-
- my $atstccfg_log_path = "$TMP_BASE/atstccfg.log";
-
- my ( $TO_USER, $TO_PASS ) = split( /:/, $TM_LOGIN );
-
- # atstccfg_cache_cleared is a global variable we use to clear the atstccfg cache on the first atstccfg call.
- # Telling atstccfg to use-cache=false will cause it to delete the cache directory
- # Which is what we want: when ORT starts to run, delete the cache. We only want to use the atstccfg cache within the same ORT run, not across different runs.
-
- my $no_cache_arg = '';
- if ( $use_cache == 0 || $atstccfg_cache_cleared == 0 ) {
- $atstccfg_cache_cleared = 1;
- $no_cache_arg = '--no-cache';
- }
-
- my $cache_age_arg='';
- if (length $cache_max_age_seconds > 0) {
- $cache_age_arg = "--cache-file-max-age-seconds=$cache_max_age_seconds";
- }
-
- $response_content = `$atstccfg_cmd $no_cache_arg $cache_age_arg --traffic-ops-user='$TO_USER' --traffic-ops-password='$TO_PASS' --traffic-ops-url='$request' --log-location-error=stderr --log-location-warning=stderr --log-location-info=null 2>$atstccfg_log_path`;
-
- my $atstccfg_exit_code = $?;
- $atstccfg_exit_code = atstccfg_code_to_http_code($atstccfg_exit_code);
-
- if ($atstccfg_exit_code != 200) {
- if ( $uri =~ m/configfiles\/ats/ && $atstccfg_exit_code == 404) {
- return $atstccfg_exit_code;
- }
- if ($uri =~ m/update_status/ && $atstccfg_exit_code == 404) {
- return $$atstccfg_exit_code;
- }
- if ( $atstccfg_exit_code != 200 && $rev_proxy_in_use == 1 ) {
- ( $log_level >> $ERROR ) && print "ERROR There appears to be an issue with the Traffic Ops Reverse Proxy. Reverting to primary Traffic Ops host.\n";
- $traffic_ops_host = $to_url;
- $rev_proxy_in_use = 0;
- next;
- }
-
- ( $log_level >> $FATAL ) && print "FATAL atstccfg returned $atstccfg_exit_code - see $atstccfg_log_path\n";
- exit 1;
- }
-
- # https://github.com/Comcast/traffic_control/issues/1168
- if ( ( $uri =~ m/url\_sig\_(.*)\.config$/ || $uri =~ m/uri\_signing\_(.*)\.config$/ ) && $response_content =~ m/No RIAK servers are set to ONLINE/ ) {
- ( $log_level >> $FATAL ) && print "FATAL result for $uri is: ..." . $response_content . "...\n";
- exit 1;
- }
-
- ( $log_level >> $DEBUG ) && print "DEBUG result for $uri is: ..." . $response_content . "...\n";
-
- &eval_json($request, $response_content) if ( $uri =~ m/\.json$/ );
- last;
- }
-
- return $response_content;
-}
-
-sub eval_json {
- my $uri = shift;
- my $lwp_response_content = shift;
- eval {
- decode_json($lwp_response_content);
- 1;
- } or do {
- my $error = $@;
- ( $log_level >> $FATAL ) && print "FATAL " . $uri . " did not return valid JSON: " . $lwp_response_content . " | Error: $error\n";
- exit 1;
- }
-
-}
-
sub replace_cfg_file {
my $cfg_file = shift;
my $return_code = 0;
@@ -1600,15 +1412,15 @@ sub replace_cfg_file {
( $log_level >> $ERROR )
&& print "ERROR Copying "
. $cfg_file_tracker->{$cfg_file}->{'backup_from_trops'} . " to "
- . $cfg_file_tracker->{$cfg_file}->{'location'}
- . "/$cfg_file\n";
- system("/bin/cp $cfg_file_tracker->{$cfg_file}->{'backup_from_trops'} $cfg_file_tracker->{$cfg_file}->{'location'}/$cfg_file");
+ . $cfg_file_tracker->{$cfg_file}->{'headers'}->{'path'}
+ . "\n";
+ system("/bin/cp $cfg_file_tracker->{$cfg_file}->{'backup_from_trops'} $cfg_file_tracker->{$cfg_file}->{'headers'}->{'path'}");
if ( $cfg_file =~ /cron/ ) {
- chown 0, 0, "$cfg_file_tracker->{$cfg_file}->{'location'}/$cfg_file";
+ chown 0, 0, "$cfg_file_tracker->{$cfg_file}->{'headers'}->{'path'}";
}
else {
my $ats_uid = getpwnam("ats");
- chown $ats_uid, $ats_uid, "$cfg_file_tracker->{$cfg_file}->{'location'}/$cfg_file";
+ chown $ats_uid, $ats_uid, "$cfg_file_tracker->{$cfg_file}->{'headers'}->{'path'}";
}
$cfg_file_tracker->{$cfg_file}->{'change_applied'}++;
( $log_level >> $TRACE ) && print "TRACE Setting change applied for $cfg_file.\n";
@@ -1630,6 +1442,7 @@ sub replace_cfg_file {
sub process_reload_restarts {
my $cfg_file = shift;
+ my $cfg_dir = dirname($cfg_file_tracker->{$cfg_file}->{'headers'}->{'path'});
( $log_level >> $DEBUG ) && print "DEBUG Applying config for: $cfg_file.\n";
if ( $cfg_file =~ m/url\_sig\_(.*)\.config/ ) {
@@ -1648,12 +1461,12 @@ sub process_reload_restarts {
( $log_level >> $DEBUG ) && print "DEBUG $cfg_file changed, trafficserver restart needed.\n";
$trafficserver_restart_needed++;
}
- elsif ( $cfg_file_tracker->{$cfg_file}->{'location'} =~ m/ssl/ && ( $cfg_file =~ m/\.cer$/ || $cfg_file =~ m/\.key$/ ) ) {
+ elsif ( $cfg_dir =~ m/ssl/ && ( $cfg_file =~ m/\.cer$/ || $cfg_file =~ m/\.key$/ ) ) {
( $log_level >> $DEBUG ) && print "DEBUG SSL key/cert $cfg_file changed, touch ssl_multicert.config, and traffic_ctl config reload needed.\n";
$installed_new_ssl_keys++;
$traffic_ctl_needed++;
}
- elsif ( $cfg_file_tracker->{$cfg_file}->{'location'} =~ m/trafficserver/ ) {
+ elsif ( $cfg_dir =~ m/trafficserver/ ) {
( $log_level >> $DEBUG ) && print "DEBUG $cfg_file changed, traffic_ctl config reload needed.\n";
$traffic_ctl_needed++;
}
@@ -1692,94 +1505,6 @@ sub check_output {
}
}
-sub get_cookie {
- my $to_host = shift;
- my $to_login = shift;
- my ( $u, $p ) = split( /:/, $to_login );
- my %headers;
-
- if ( $login_dispersion > 0 ) {
- ( $log_level >> $WARN ) && print "WARN Login dispersion is enabled.\n";
- &sleep_rand($login_dispersion);
- }
-
- my $url = $to_host . "/api/2.0/user/login";
- my $json = qq/{ "u": "$u", "p": "$p"}/;
- my $response = $lwp_conn->post($url, Content => $json);
-
- &check_lwp_response_code($response, $FATAL);
-
- my $cookie;
- if ( $response->header('Set-Cookie') ) {
- ($cookie) = split(/\;/, $response->header('Set-Cookie'));
- }
-
- if ( $cookie =~ m/mojolicious/ ) {
- ( $log_level >> $DEBUG ) && print "DEBUG Cookie is $cookie.\n";
- return $cookie;
- }
- else {
- ( $log_level >> $FATAL ) && print "FATAL mojolicious cookie not found from Traffic Ops!\n";
- exit 1;
- }
-}
-
-sub check_lwp_response_code {
- my $lwp_response = shift;
- my $panic_level = shift;
- my $log_level_str = &log_level_to_string($panic_level);
- my $url = $lwp_response->request->uri;
-
- if ( !defined($lwp_response->code()) ) {
- ( $log_level >> $panic_level ) && print $log_level_str . " $url failed!\n";
- exit 1 if ($log_level_str eq 'FATAL');
- return 1;
- }
- elsif ( $lwp_response->code() >= 400 ) {
- ( $log_level >> $panic_level ) && print $log_level_str . " $url returned HTTP " . $lwp_response->code() . "! " . $lwp_response->message() . " \n";
- exit 1 if ($log_level_str eq 'FATAL');
- return 1;
- }
- else {
- ( $log_level >> $DEBUG ) && print "DEBUG $url returned HTTP " . $lwp_response->code() . ".\n";
- return 0;
- }
-}
-
-sub check_lwp_response_message_integrity {
- my $lwp_response = shift;
- my $panic_level = shift;
- my $log_level_str = &log_level_to_string($panic_level);
- my $url = $lwp_response->request->uri;
-
- my $mic_header = 'Whole-Content-SHA512';
-
- if ( defined($lwp_response->header($mic_header)) ) {
- if ( $lwp_response->header($mic_header) ne sha512_base64($lwp_response->content()) . '==') {
- ( $log_level >> $panic_level ) && print $log_level_str . " $url returned a $mic_header of " . $lwp_response->header($mic_header) . ", however actual body SHA512 is " . sha512_base64($lwp_response->content()) . '==' . "!\n";
- exit 1 if ($log_level_str eq 'FATAL');
- return 1;
- } else {
- ( $log_level >> $DEBUG ) && print "DEBUG $url returned a $mic_header of " . $lwp_response->header($mic_header) . ", and actual body SHA512 is " . sha512_base64($lwp_response->content()) . '==' . "\n";
- return 0;
- }
- }
- elsif ( defined($lwp_response->header('Content-Length')) ) {
- if ( $lwp_response->header('Content-Length') != length($lwp_response->content()) ) {
- ( $log_level >> $panic_level ) && print $log_level_str . " $url returned a Content-Length of " . $lwp_response->header('Content-Length') . ", however actual content length is " . length($lwp_response->content()) . "!\n";
- exit 1 if ($log_level_str eq 'FATAL');
- return 1;
- } else {
- ( $log_level >> $DEBUG ) && print "DEBUG $url returned a Content-Length of " . $lwp_response->header('Content-Length') . ", and actual content length is " . length($lwp_response->content()). "\n";
- return 0;
- }
- }
- else {
- ( $log_level >> $panic_level ) && print $log_level_str . " $url did not return a $mic_header or Content-Length header! Cannot Message Integrity Check!\n";
- return 1;
- }
-}
-
sub check_script_mode {
#### No default script mode
my $script_mode = undef;
@@ -1833,151 +1558,99 @@ sub check_log_level {
}
}
-sub set_domainname {
- my $hostname;
- if ($RELEASE eq "EL7") {
- $hostname = `cat /etc/hostname`;
- chomp($hostname);
- } else {
- $hostname = `cat /etc/sysconfig/network | grep HOSTNAME`;
- chomp($hostname);
- $hostname =~ s/HOSTNAME\=//g;
- }
- chomp($hostname);
- $hostname =~ s/HOSTNAME\=//g;
- my $domainname;
- ( my @parts ) = split( /\./, $hostname );
- for ( my $i = 1; $i < scalar(@parts); $i++ ) {
- $domainname .= $parts[$i] . ".";
- }
- $domainname =~ s/\.$//g;
- return $domainname;
-}
-
+# get_cfg_file_list gets the config files from atstccfg via Traffic Ops.
+# See parse_multipart_config_files for the return type.
sub get_cfg_file_list {
my $host_name = shift;
- my $tm_host = shift;
my $script_mode = shift;
- my $cfg_files;
- my $profile_name;
- my $cdn_name;
- my $uri = "/api/2.0/servers/$host_name/configfiles/ats"; #This doesn't actually exist in 2.0, but atstccfg doesn't care
- my $result = &lwp_get($uri);
+ my $atstccfg_reval_arg = '';
+ if ( $script_mode == $REVALIDATE ) {
+ $atstccfg_reval_arg = '--revalidate-only';
+ }
- if ($result eq '404') {
- $api_in_use = 0;
- ( $log_level >> $ERROR ) && printf("ERROR Traffic Ops version does not support config files API. Please upgrade to Traffic Ops 2.2.\n");
+ my $result = `$atstccfg_cmd --traffic-ops-user='$TO_USER' --traffic-ops-password='$TO_PASS' --traffic-ops-url='$TO_URL' --cache-host-name='$host_name' $atstccfg_reval_arg --log-location-error=stderr --log-location-warning=stderr --log-location-info=null 2>$atstccfg_log_path`;
+ my $atstccfg_exit_code = $?;
+ if ($atstccfg_exit_code != 0) {
+ ( $log_level >> $ERROR ) && printf("ERROR getting config files from atstccfg via Traffic Ops. See $atstccfg_log_path for details\n");
exit 1;
}
- my $ort_ref = decode_json($result);
+ return &parse_multipart_config_files($result);
+}
- if ($api_in_use == 1) {
- $to_rev_proxy_url = $ort_ref->{'info'}->{'toRevProxyUrl'};
- if ( $to_rev_proxy_url && $rev_proxy_disable == 0 ) {
- $to_rev_proxy_url =~ s/\/*$//g;
- # Note: If traffic_ops_url is changing, would be suggested to get a new cookie.
- # Secrets might not be the same on all Traffic Ops instance.
- $traffic_ops_host = $to_rev_proxy_url;
- $rev_proxy_in_use = 1;
- ( $log_level >> $INFO ) && printf("INFO Found Traffic Ops Reverse Proxy URL from Traffic Ops: $to_rev_proxy_url\n");
- } else {
- if ( $rev_proxy_disable == 1 ) {
- ( $log_level >> $INFO ) && printf("INFO Reverse proxy disabled - connecting directly to traffic ops for all files.\n");
- }
- $traffic_ops_host = $to_url;
- }
- $profile_name = $ort_ref->{'info'}->{'profileName'};
- ( $log_level >> $INFO ) && printf("INFO Found profile from Traffic Ops: $profile_name\n");
- $cdn_name = $ort_ref->{'info'}->{'cdnName'};
- ( $log_level >> $INFO ) && printf("INFO Found CDN_name from Traffic Ops: $cdn_name\n");
- $server_tcp_port = $ort_ref->{'info'}->{'serverTcpPort'};
- ( $log_level >> $INFO ) && printf("INFO Found cache server tcp port from Traffic Ops: $server_tcp_port\n");
- $server_ipv4 = $ort_ref->{'info'}->{'serverIpv4'};
- ( $log_level >> $INFO ) && printf("INFO Found cache server ipv4 from Traffic Ops: $server_ipv4\n");
+# parse_multipart_config_files parses the multipart/mixed message returned by atstccfg, and returns a map of file names to content.
+# Returns a hash of the file name to a sub-hash. Each sub-hash contains the values 'headers' and 'body'.
+# The 'headers' is a hash of lowercased header names to header values.
+# The 'body' is the string body
+sub parse_multipart_config_files {
+ my $multipart_txt = shift;
+
+ # Note this doesn't enforce RFC compliance for the boundary. We assume any char is valid, where in reality only certain characters are allowed in the unquoted value. See RFC2616s3.6
+ my $boundary = '';
+ if ($multipart_txt =~ m/boundary="/) {
+ ($boundary) = $multipart_txt =~ m/boundary="((?:[^"\\]|\\.)*)"/g; # this regex gets the quoted boundary="foo" value
+ } else {
+ ($boundary) = $multipart_txt =~ m/boundary=([^ \r\n\t]+)/g; # this regex gets the unquoted boundary=foo value
}
- else {
- $profile_name = $ort_ref->{'profile'}->{'name'};
- ( $log_level >> $INFO ) && printf("INFO Found profile from Traffic Ops: $profile_name\n");
- $cdn_name = $ort_ref->{'other'}->{'CDN_name'};
- ( $log_level >> $INFO ) && printf("INFO Found CDN_name from Traffic Ops: $cdn_name\n");
+
+ if ( length($boundary) == 0 ) {
+ ( $log_level >> $FATAL ) && print "FATAL Error getting package list from Traffic Ops! Could not find boundary for multipart message\n";
+ exit 1;
}
- if ( $script_mode == $REVALIDATE ) {
- foreach my $cfg_file ( @{$ort_ref->{'configFiles'}} ) {
- if ( $cfg_file->{'fnameOnDisk'} eq "regex_revalidate.config" ) {
- my $fname_on_disk = &get_filename_on_disk( $cfg_file->{'fnameOnDisk'} );
- ( $log_level >> $INFO )
- && printf( "INFO Found config file (on disk: %-41s): %-41s with location: %-50s\n", $fname_on_disk, $cfg_file->{'fnameOnDisk'}, $cfg_file->{'location'} );
- $cfg_files->{$fname_on_disk}->{'location'} = $cfg_file->{'location'};
- if ($api_in_use == 1) {
- $cfg_files->{$fname_on_disk}->{'apiUri'} = $cfg_file->{'apiUri'};
- }
- $cfg_files->{$fname_on_disk}->{'fname-in-TO'} = $cfg_file->{'fnameOnDisk'};
- }
+
+ my @files = split("--" . $boundary . "\r\n", $multipart_txt);
+
+ my %all_files;
+ for my $i (1 .. $#files) { # start at 1, because 0 is the MIME-Version and Content-Type: multipart/mixed
+ my $file = $files[$i];
+ my @headers_body = split("\r\n\r\n", $file);
+
+ if ( @headers_body < 2 ) {
+ print "FATAL Error getting package list from Traffic Ops! malformed headers on file $i\n";
+ exit 1;
}
- }
- else {
- if ( $reval_in_use == 1 ) {
- ( $log_level >> $WARN ) && printf("WARN Instant Invalidate is enabled. Skipping regex_revalidate.config.\n");
- if ( $api_in_use == 1 ) {
- my @new = grep { $_->{'fnameOnDisk'} ne 'regex_revalidate.config' } @{$ort_ref->{'configFiles'}};
- $ort_ref->{'configFiles'} = \@new;
- }
- else {
- delete $ort_ref->{'config_files'}->{'regex_revalidate.config'};
- }
- }
- if ( $api_in_use == 1 ) {
- foreach my $cfg_file (@{$ort_ref->{'configFiles'}} ) {
- my $fname_on_disk = &get_filename_on_disk( $cfg_file->{'fnameOnDisk'} );
- ( $log_level >> $INFO )
- && printf( "INFO Found config file (on disk: %-41s): %-41s with location: %-50s\n", $fname_on_disk, $cfg_file->{'fnameOnDisk'}, $cfg_file->{'location'} );
- $cfg_files->{$fname_on_disk}->{'location'} = $cfg_file->{'location'};
- if ( defined($cfg_file->{'apiUri'} ) ) {
- $cfg_files->{$fname_on_disk}->{'apiUri'} = $cfg_file->{'apiUri'};
- ( $log_level >> $DEBUG ) && print "DEBUG apiUri found: $cfg_files->{$fname_on_disk}->{'apiUri'}.\n";
- }
- elsif ( defined($cfg_file->{'url'} ) ) {
- $cfg_files->{$fname_on_disk}->{'url'} = $cfg_file->{'url'};
- ( $log_level >> $DEBUG ) && print "DEBUG URL found: $cfg_files->{$fname_on_disk}->{'url'}.\n";
- }
- $cfg_files->{$fname_on_disk}->{'fname-in-TO'} = $cfg_file->{'fnameOnDisk'};
- }
+ my @headers_arr = split("\r\n", $headers_body[0]);
+ my %headers;
+ for my $i (0 .. $#headers_arr) {
+ my $header = $headers_arr[$i];
+ my @header_name_val = split(": ", $header, 2);
+ my $header_name = lc($header_name_val[0]);
+ my $header_val = $header_name_val[1];
+ $headers{$header_name} = $header_val;
}
- else {
- foreach my $cfg_file ( keys %{ $ort_ref->{'config_files'} } ) {
- my $fname_on_disk = &get_filename_on_disk( $cfg_file );
- ( $log_level >> $INFO )
- && printf( "INFO Found config file (on disk: %-41s): %-41s with location: %-50s\n", $fname_on_disk, $cfg_file, $ort_ref->{'config_files'}->{$cfg_file}->{'location'} );
- $cfg_files->{$fname_on_disk}->{'location'} = $ort_ref->{'config_files'}->{$cfg_file}->{'location'};
- if ($api_in_use == 1) {
- $cfg_files->{$fname_on_disk}->{'apiUri'} = $cfg_file->{'apiUri'};
- }
- $cfg_files->{$fname_on_disk}->{'fname-in-TO'} = $cfg_file;
- }
+
+ my $file_name = basename($headers{'path'});
+ if ( length($file_name) == 0 ) {
+ print "FATAL Error getting package list from Traffic Ops! Headers on file $i missing Path!\n";
+ exit 1;
}
+
+ my %file_obj;
+ $file_obj{'headers'} = \%headers;
+ my $body = $headers_body[1];
+ $body =~ s/\s+$//; # trim trailing whitespace
+ $body = $body . "\n"; # add a single trailing newline, POSIX files require trailing newlines
+ $file_obj{'body'} = $body;
+ $all_files{$file_name} = \%file_obj;
}
- return ( $profile_name, $cfg_files, $cdn_name );
-}
-sub get_filename_on_disk {
- my $config_file = shift;
- $config_file =~ s/^to\_ext\_(.*)\.config$/$1\.config/ if ($config_file =~ m/^to\_ext\_/);
- return $config_file;
+ return \%all_files;
}
sub get_header_comment {
- my $to_host = shift;
my $toolname;
- my $uri = "/api/2.0/system/info";
- my $result = &lwp_get($uri);
-
+ my $result = `$atstccfg_cmd --traffic-ops-user='$TO_USER' --traffic-ops-password='$TO_PASS' --traffic-ops-url='$TO_URL' --cache-host-name='$hostname_short' --log-location-error=stderr --log-location-warning=stderr --log-location-info=null --get-data=system-info 2>$atstccfg_log_path`;
+ my $atstccfg_exit_code = $?;
+ if ($atstccfg_exit_code != 0) {
+ ( $log_level >> $ERROR ) && printf("ERROR Unable to get system info. Stopping.\n");
+ exit 1;
+ }
my $result_ref = decode_json($result);
- if ( defined( $result_ref->{'response'}->{'parameters'}->{'tm.toolname'} ) ) {
- $toolname = $result_ref->{'response'}->{'parameters'}->{'tm.toolname'};
+ if ( defined( $result_ref->{'tm.toolname'} ) ) {
+ $toolname = $result_ref->{'tm.toolname'};
( $log_level >> $INFO ) && printf("INFO Found tm.toolname: $toolname\n");
}
else {
@@ -1985,7 +1658,6 @@ sub get_header_comment {
$toolname = "";
}
return $toolname;
-
}
sub __package_action {
@@ -2067,6 +1739,7 @@ sub package_was_installed {
}
}
+my %packages_installed = ();
sub package_installed {
my $package_name = shift;
my $package_version = shift;
@@ -2076,6 +1749,13 @@ sub package_installed {
$package_name = $package_name . "-" . $package_version;
}
+ if ( exists( $packages_installed{$package_name} ) ) {
+ print("package_installed $package_name (cached)\n");
+ return ($packages_installed{$package_name});
+ }
+
+ print("package_installed $package_name (uncached)\n");
+
my $out = `/bin/rpm -q $package_name 2>&1`;
# rpm returns 0 if installed, 1 if not installed
@@ -2086,6 +1766,7 @@ sub package_installed {
@package_list = split( /\n/, $out );
}
+ $packages_installed{$package_name} = @package_list;
return (@package_list);
}
@@ -2136,15 +1817,16 @@ sub remove_packages {
sub process_packages {
my $host_name = shift;
- my $tm_host = shift;
my $proceed = 0;
- # TODO: this is a non-standard API endpoint that will be removed with the Perl; need to update it.
- my $uri = "/ort/$host_name/packages";
- my $result = &lwp_get($uri);
+ my $result = `$atstccfg_cmd --traffic-ops-user='$TO_USER' --traffic-ops-password='$TO_PASS' --traffic-ops-url='$TO_URL' --cache-host-name='$hostname_short' --log-location-error=stderr --log-location-warning=stderr --log-location-info=null --get-data=packages 2>$atstccfg_log_path`;
+ my $atstccfg_exit_code = $?;
+ if ($atstccfg_exit_code != 0) {
+ ( $log_level >> $FATAL ) && print "FATAL Error getting package list from Traffic Ops!\n";
+ exit 1;
+ }
- if ( defined($result) && $result ne "" && $result !~ m/^(\d){3}$/ ) {
my %package_map;
my @package_list = @{ decode_json($result) };
@@ -2297,11 +1979,6 @@ sub process_packages {
( $log_level >> $TRACE ) && print "TRACE All required packages are installed.\n";
}
}
- }
- else {
- ( $log_level >> $FATAL ) && print "FATAL Error getting package list from Traffic Ops!\n";
- exit 1;
- }
}
sub set_chkconfig {
@@ -2386,13 +2063,15 @@ sub chkconfig_matches {
sub process_chkconfig {
my $host_name = shift;
- my $tm_host = shift;
my $proceed = 0;
- # TODO: this is a non-standard API endpoint that will be removed with the Perl; need to update it.
- my $uri = "/ort/$host_name/chkconfig";
- my $result = &lwp_get($uri);
+ my $result = `$atstccfg_cmd --traffic-ops-user='$TO_USER' --traffic-ops-password='$TO_PASS' --traffic-ops-url='$TO_URL' --cache-host-name='$hostname_short' --log-location-error=stderr --log-location-warning=stderr --log-location-info=null --get-data=chkconfig 2>$atstccfg_log_path`;
+ my $atstccfg_exit_code = $?;
+ if ($atstccfg_exit_code != 0) {
+ ( $log_level >> $FATAL ) && print "FATAL Error getting package list from Traffic Ops!\n";
+ exit 1;
+ }
if ( defined($result) && $result ne "" && $result !~ m/^\d{3}$/ ) {
my @chkconfig_list = @{ decode_json($result) };
@@ -2501,6 +2180,7 @@ sub get_answer {
}
sub start_restart_services {
+ my $ats_running = 0;
#### Start ATS
if ( package_installed("trafficserver") ) {
( $log_level >> $DEBUG ) && print "DEBUG trafficserver is installed.\n";
@@ -2581,15 +2261,13 @@ sub start_restart_services {
#### Start teakd
if ( package_installed("teakd") ) {
( $log_level >> $DEBUG ) && print "DEBUG teakd is installed.\n";
- $teakd_running = &start_service("teakd");
+ &start_service("teakd");
# Do something here in the future.
}
-
}
sub run_sysctl_p {
-
if ( $script_mode == $INTERACTIVE ) {
my $select = 'n';
( $log_level >> $ERROR ) && print "ERROR sysctl configuration has changed. 'sysctl -p' needs to be run. Should I do that now? (Y/[n]):";
@@ -2621,49 +2299,6 @@ sub run_sysctl_p {
}
}
-sub validate_result {
-
- my $url = ${ $_[0] };
- my $result = ${ $_[1] };
-
- if ( $result =~ m/^\d{3}$/ ) {
- ( $log_level >> $ERROR ) && print "ERROR Result from getting $url is HTTP $result!\n";
- return 0;
- }
-
- my $size = length($result);
- if ( $size == 0 ) {
- ( $log_level >> $ERROR ) && print "ERROR URL: $url returned empty!\n";
- return 0;
- }
- elsif ( $size < 125 ) {
- ( $log_level >> $WARN ) && print "WARN URL: $url returned only the header.\n";
- return 1;
- }
- else {
- ( $log_level >> $DEBUG ) && print "DEBUG URL: $url returned $size bytes.\n";
- return 1;
- }
-}
-
-sub set_uri {
- my $filename = shift;
-
- my $filepath = $cfg_file_tracker->{$filename}->{'location'};
- my $URI;
- if ( $api_in_use == 1 && defined($cfg_file_tracker->{$filename}->{'apiUri'}) ) {
- $URI = $cfg_file_tracker->{$filename}->{'apiUri'};
- }
- elsif ( $api_in_use == 1 && defined($cfg_file_tracker->{$filename}->{'url'}) ) {
- $URI = $cfg_file_tracker->{$filename}->{'url'};
- ( $log_level >> $DEBUG ) && print "DEBUG Setting external download URL.\n";
- }
-
- return if (!defined($cfg_file_tracker->{$filename}->{'fname-in-TO'}));
-
- return $URI;
-}
-
sub scrape_unencode_text {
my $text = shift;
@@ -2704,11 +2339,8 @@ sub scrape_canned_comments {
}
sub can_read_write_file {
-
my $filename = shift;
-
- my $filepath = $cfg_file_tracker->{$filename}->{'location'};
- my $file = $filepath . "/" . $filename;
+ my $file = $cfg_file_tracker->{$filename}->{'headers'}->{'path'};
my $username = $ENV{LOGNAME} || $ENV{USER} || getpwuid($<);
( $log_level >> $TRACE ) && print "TRACE User to validate $file against: $username\n";
@@ -2729,25 +2361,7 @@ sub can_read_write_file {
return 1;
}
-sub file_exists {
-
- my $filename = shift;
- my $filepath = $cfg_file_tracker->{$filename}->{'location'};
- my $file = $filepath . "/" . $filename;
-
- if ( !-e $file ) {
- ( $log_level >> $ERROR ) && print "ERROR $filename does not exist!\n";
- $cfg_file_tracker->{$filename}->{'audit_failed'}++;
- return 0;
- }
-
- ( $log_level >> $TRACE ) && print "TRACE $filename exists on disk.\n";
- return 1;
-
-}
-
sub open_file_get_contents {
-
my $file = shift;
my @disk_file_lines;
@@ -2773,7 +2387,6 @@ sub open_file_get_contents {
}
sub prereqs_ok {
-
my $filename = shift;
my $file_lines_ref = shift;
@@ -2787,11 +2400,9 @@ sub prereqs_ok {
}
}
return 1;
-
}
sub diff_file_lines {
-
my $cfg_file = shift;
my @db_file_lines = @{ $_[0] };
my @disk_file_lines = @{ $_[1] };
@@ -2802,8 +2413,7 @@ sub diff_file_lines {
my @db_lines_missing;
my @disk_lines_missing;
- my $filepath = $cfg_file_tracker->{$cfg_file}->{'location'};
- my $file = $filepath . "/" . $cfg_file;
+ my $file = $cfg_file_tracker->{$cfg_file}->{'headers'}->{'path'};
foreach my $line (@db_file_lines) {
( $log_level >> $TRACE ) && print "TRACE Line from TrOps: $line!\n";
@@ -2882,26 +2492,12 @@ sub diff_file_lines {
return ( \@db_lines_missing, \@disk_lines_missing );
}
-sub validate_filename {
-
- my $filename = shift;
-
- if ( $filename eq "" ) {
- ( $log_level >> $ERROR ) && print "ERROR Config file name is empty!\n";
- $cfg_file_tracker->{$filename}->{'audit_failed'}++;
- return 0;
- }
- return 1;
-}
-
sub backup_file {
-
my $filename = shift;
my $result_ref = shift;
my $result = ${$result_ref};
- my $filepath = $cfg_file_tracker->{$filename}->{'location'};
- my $file = $filepath . "/" . $filename;
+ my $file = $cfg_file_tracker->{$filename}->{'headers'}->{'path'};
if ( $script_mode != $REPORT ) {
my $ats_uid = getpwnam("ats");
@@ -2935,7 +2531,6 @@ sub backup_file {
}
sub adv_preprocessing_remap {
-
my $buffer = ${ $_[0] };
( my @file_lines ) = split( /\n/, $buffer );
@@ -3004,7 +2599,6 @@ sub adv_preprocessing_remap {
}
sub adv_processing_udev {
-
my @db_file_lines = @{ $_[0] };
( $log_level >> $DEBUG ) && print "DEBUG Entering advanced processing for 50-ats.rules.\n";
@@ -3072,91 +2666,6 @@ sub adv_processing_udev {
return 0;
}
-sub adv_processing_ssl {
-
- my @db_file_lines = @{ $_[0] };
- if (@db_file_lines > 1) { #header line is always present, so look for 2 lines or more
- ( $log_level >> $DEBUG ) && print "DEBUG Entering advanced processing for ssl_multicert.config.\n";
- my $uri = "/api/2.0/cdns/name/$my_cdn_name/sslkeys";
- my $result = &lwp_get($uri);
- if ( $result =~ m/^\d{3}$/ ) {
- if ( $script_mode == $REPORT ) {
- ( $log_level >> $ERROR ) && print "ERROR SSL URL: $uri returned $result.\n";
- return 1;
- } else {
- ( $log_level >> $FATAL ) && print "FATAL SSL URL: $uri returned $result. Exiting.\n";
- exit 1;
- }
- }
- my $result_json = decode_json($result);
- my $certs = $result_json->{'response'};
-
- foreach my $line (@db_file_lines) {
- ( $log_level >> $DEBUG ) && print "DEBUG line in ssl_multicert.config from Traffic Ops: $line \n";
- if ( $line =~ m/^\s*ssl_cert_name\=(.*)\s+ssl_key_name\=(.*)\s*$/ ) {
- push( @{ $ssl_tracker->{'db_config'} }, { cert_name => $1, key_name => $2 } );
- }
- }
-
- foreach my $keypair ( @{ $ssl_tracker->{'db_config'} } ) {
- ( $log_level >> $DEBUG ) && print "DEBUG Processing SSL key: " . $keypair->{'key_name'} . "\n";
- my $remap = $keypair->{'key_name'};
- $remap =~ s/\.key$//;
- $remap =~ /^(.*?)(\..*)/;
- # HTTP delivery services use wildcard certs
- my $wildcard = "*$2";
- my $found = 0;
- foreach my $record (@$certs){
- if ($record->{'hostname'} eq $remap || $record->{'hostname'} eq $wildcard) {
- $found = 1;
- my $ssl_key = decode_base64($record->{'certificate'}->{'key'});
- my $ssl_cert = decode_base64($record->{'certificate'}->{'crt'});
- ( $log_level >> $DEBUG ) && print "DEBUG private key for $remap is:\n$ssl_key\n";
- ( $log_level >> $DEBUG ) && print "DEBUG certificate for $remap is:\n$ssl_cert\n";
-
- $cfg_file_tracker->{ $keypair->{'key_name'} }->{'location'} = "/opt/trafficserver/etc/trafficserver/ssl/";
- $cfg_file_tracker->{ $keypair->{'key_name'} }->{'service'} = "trafficserver";
- $cfg_file_tracker->{ $keypair->{'key_name'} }->{'component'} = "SSL";
- $cfg_file_tracker->{ $keypair->{'key_name'} }->{'contents'} = $ssl_key;
- $cfg_file_tracker->{ $keypair->{'key_name'} }->{'fname-in-TO'} = $keypair->{'key_name'};
-
- $cfg_file_tracker->{ $keypair->{'cert_name'} }->{'location'} = "/opt/trafficserver/etc/trafficserver/ssl/";
- $cfg_file_tracker->{ $keypair->{'cert_name'} }->{'service'} = "trafficserver";
- $cfg_file_tracker->{ $keypair->{'cert_name'} }->{'component'} = "SSL";
- $cfg_file_tracker->{ $keypair->{'cert_name'} }->{'contents'} = $ssl_cert;
- $cfg_file_tracker->{ $keypair->{'cert_name'} }->{'fname-in-TO'} = $keypair->{'cert_name'};
- }
- }
- #if no cert is found, log error and exit
- if (!$found) {
- ( $log_level >> $FATAL ) && print "FATAL SSL certificate for $remap not found!\n";
- exit 1;
- }
- }
- }
- return 0;
-}
-
-sub setup_lwp {
- my $browser = LWP::UserAgent->new( keep_alive => 100, ssl_opts => { verify_hostname => 0, SSL_verify_mode => 0x00 } );
-
- my $lwp_cc = $browser->conn_cache(LWP::ConnCache->new());
- $browser->timeout(30);
-
- return $browser;
-}
-
-sub log_level_to_string {
- return "ALL" if ( $_[0] == 7 );
- return "TRACE" if ( $_[0] == 6 );
- return "DEBUG" if ( $_[0] == 5 );
- return "INFO" if ( $_[0] == 4 );
- return "WARN" if ( $_[0] == 3 );
- return "ERROR" if ( $_[0] == 2 );
- return "FATAL" if ( $_[0] == 1 );
- return "NONE" if ( $_[0] == 0 );
-}
-
{
my $fh;
@@ -3170,4 +2679,3 @@ sub log_level_to_string {
}
}
}
-
diff --git a/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/LICENSE b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/LICENSE
new file mode 100644
index 0000000..8ab4ab4
--- /dev/null
+++ b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/LICENSE
@@ -0,0 +1,454 @@
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+
+APACHE TRAFFICCONTROL SUBCOMPONENTS:
+
+Apache TrafficControl includes a number of subcomponents with
+separate copyright notices and license terms. Your use of the source
+code for the these subcomponents is subject to the terms and
+conditions of the following licenses.
+
+For readability, subcomponent licenses have been broken out into their own
+files, largely located in the licenses directory. Each subcomponent enumerated
+below lists the files it comprises and a link to the full text of the license.
+
+For the fontawesome component:
+@traffic_portal/app/src/assets/fonts/[Ff]ont[Aa]wesome*
+
+These binary fontawesome fonts are provided under the SIL Open Font License 1.1:
+./licenses/SIL-1.1
+
+The fontawesome CSS files are provided under an MIT license:
+./licenses/MIT-fontawesome
+
+For the bootstrap component:
+@misc/traffic-control-cdn/*/bootstrap*
+./licenses/MIT-bootstrap
+
+/*! normalize.css v3.0.1 | MIT License | git.io/normalize */
+./licenses/MIT-normalize
+
+For the sorttable component:
+@traffic_monitor/static/sorttable.js
+./licenses/MIT-sorttable
+
+For the jQuery component:
+./licenses/MIT-jquery
+
+For the JSON formatter component:
+@traffic_portal/app/src/assets/css/jsonformatter.min_0.6.0.css
+@traffic_portal/app/src/assets/js/jsonformatter.min_0.6.0.js
+./licenses/AL2-jsonformatter
+
+For the DataTables component:
+@traffic_portal/app/src/assets/css/jquery.dataTables.min_1.10.9.css
+@traffic_portal/app/src/assets/js/jquery.dataTables.min_1.10.19_patched.js
+@traffic_portal/app/src/assets/images/sort_*
+./licenses/MIT-datatables
+
+For the DataTables ColReorder component:
+@traffic_portal/app/src/assets/css/colReorder.dataTables.min_1.5.1.css
+@traffic_portal/app/src/assets/js/colReorder.dataTables.min_1.5.1.js
+./licenses/MIT-ColReorder
+
+For the DataTables Buttons component:
+@traffic_portal/app/src/assets/js/dataTables.buttons.min_1.5.6.js
+./licenses/MIT-datatables-buttons
+
+For the HTML5 Buttons component:
+@traffic_portal/app/src/assets/js/buttons.html5.min_1.5.6.js
+./licenses/MIT-buttons-html5
+
+For the moment.js component:
+@traffic_portal/app/src/assets/js/moment-min_2.22.1.js
+./licenses/MIT-momentjs
+
+For the jsdiff component:
+@traffic_portal/app/src/assets/js/jsdiff-min_3.5.0.js
+./licenses/BSD-jsdiff
+
+For the fast-json-patch component:
+@traffic_portal/app/src/assets/js/fast-json-patch_v2.1.0.js
+./licenses/MIT-fast-json-patch
+
+For the angular-moment-picker component:
+@traffic_portal/app/src/assets/css/angular-moment-picker_0.10.2.css
+@traffic_portal/app/src/assets/js/moment-picker/angular-moment-picker.min_0.10.2.js
+./licenses/MIT-angular_moment_picker
+
+For the Chart.js component:
+@traffic_portal/app/src/assets/js/chartjs/Chart.min_2.7.2.js
+./licenses/MIT-chartjs
+
+For the angular-chart.js component:
+@traffic_portal/app/src/assets/js/chartjs/angular-chart.min_1.1.1.js
+./licenses/BSD-angular-chartjs
+
+For the downloadjs component:
+@traffic_portal/app/src/assets/js/downloadjs-min_v4.21.js
+./licenses/MIT-downloadjs
+
+For the angular-loading-bar component:
+@traffic_portal/app/src/styles/loading.scss
+./licenses/MIT-angular_loading_bar
+
+For the gentelella admin theme:
+@traffic_portal/app/src/styles/theme.scss
+./licenses/MIT-gentelella
+
+For the underscore component:
+@traffic_portal/app/src/assets/js/underscore-min_1.8.3.js
+./licenses/MIT-underscore
+
+For the stoppableListener component:
+@traffic_monitor/vendor/github.com/hydrogen18/stoppableListener/*
+./traffic_monitor/vendor/github.com/hydrogen18/stoppableListener/LICENSE
+
+For the fsnotify component:
+@vendor/gopkg.in/fsnotify.v1/*
+./vendor/gopkg.in/fsnotify.v1/LICENSE
+
+For the yaml component:
+@vendor/gopkg.in/yaml.v2/*
+./vendor/gopkg.in/yaml.v2/LICENSE
+./vendor/gopkg.in/yaml.v2/LICENSE.libyaml
+
+For the jwt-go component:
+@experimental/traffic_ops_auth/vendor/github.com/dgrijalva/jwt-go/*
+./experimental/traffic_ops_auth/vendor/github.com/dgrijalva/jwt-go/LICENSE
+
+For the pq component:
+@experimental/traffic_ops_auth/vendor/github.com/lib/pq/*
+@vendor/github.com/lib/pq/*
+./vendor/github.com/lib/pq/LICENSE.md
+
+For the influxdb component:
+@vendor/github.com/influxdata/influxdb/*
+./vendor/github.com/influxdata/influxdb/LICENSE
+./vendor/github.com/influxdata/influxdb/LICENSE_OF_DEPENDENCIES.md
+
+For the errors component:
+@traffic_stats/vendor/github.com/pkg/errors/*
+./traffic_stats/vendor/github.com/pkg/errors/LICENSE
+
+For the MaxMind DB GeoLite2 Database:
+@traffic_router/core/src/test/resources/geo/GeoLite2-City.mmdb.gz
+./licenses/CC_A_SA_4-maxmind
+
+For the gofmt github hook:
+@misc/git/pre-commit-hooks/01-gofmt
+./licenses/BSD-go
+
+For the IPv6 Perl Module component:
+@traffic_ops/app/bin/checks/NetPacket/IPv6.pm
+./licenses/ISC-netpacket_ipv6
+
+Several subsections of main.css are under the MIT license, as noted in the file:
+@traffic_portal/app/src/styles/main.scss
+@traffic_portal/app/src/styles/main.scss
+
+ For the bootstrap-vertical-tabs component:
+ ./licenses/MIT-bootstrap_vertical_tabs
+
+ For the cropper component:
+ ./licenses/MIT-cropper
+
+ For the bootstrap-progressbar component:
+ ./licenses/MIT-bootstrap_progressbar
+
+For the seelog component:
+@traffic_stats/vendor/github.com/cihub/seelog/*
+./licenses/BSD-seelog
+
+The invalid_passwords.txt file is from the Projects/OWASP SecLists Project provided under a MIT license:
+@traffic_ops/app/conf/invalid_passwords.txt
+./licenses/MIT-SecLists
+
+For the go-sqlmock component:
+@traffic_ops/vendor/gopkg.in/DATA-DOG/go-sqlmock.v1/*
+./traffic_ops/vendor/gopkg.in/DATA-DOG/go-sqlmock.v1/LICENSE
+
+For the go-jwx component:
+@traffic_ops/vendor/github.com/lestrrat/go-jwx/*
+./traffic_ops/vendor/github.com/lestrrat/go-jwx/LICENSE
+
+For the protobuf component:
+@traffic_ops/vendor/github.com/golang/protobuf/*
+./traffic_ops/vendor/github.com/golang/protobuf/LICENSE
+
+For the sqlx component:
+@vendor/github.com/jmoiron/sqlx/*
+./vendor/github.com/jmoiron/sqlx/LICENSE
+
+For the backoff component:
+@traffic_ops/vendor/github.com/basho/backoff/*
+./traffic_ops/vendor/github.com/basho/backoff/README.md
+
+The riak-go-client component is used under the Apache license:
+@traffic_ops/vendor/github.com/basho/riak-go-client/*
+./traffic_ops/vendor/github.com/basho/riak-go-client/LICENSE
+
+The envconfig component is used under the MIT license:
+@traffic_ops/vendor/github.com/kelseyhightower/envconfig/*
+./traffic_ops/vendor/github.com/kelseyhightower/envconfig/LICENSE
+
+The govalidator component is used under the MIT license:
+@vendor/github.com/asaskevich/govalidator/*
+./vendor/github.com/asaskevich/govalidator/LICENSE
+
+The ozzo-validation component is used under the MIT license:
+@vendor/github.com/go-ozzo/ozzo-validation/*
+./vendor/github.com/go-ozzo/ozzo-validation/LICENSE
+
+The bytefmt component is used under the Apache 2.0 license:
+@grove/vendor/code.cloudfoundry.org/bytefmt/*
+./grove/vendor/code.cloudfoundry.org/bytefmt/LICENSE
+
+The bytefmt component is used under the MIT license:
+@grove/vendor/github.com/coreos/bbolt/*
+./grove/vendor/github.com/coreos/bbolt/LICENSE
+
+For the siphash component:
+@grove/vendor/github.com/dchest/siphash/*
+./licenses/CC0
+
+For the miekg/dns component:
+@traffic_ops/traffic_ops_golang/vendor/github.com/miekg/dns/*
+./licenses/BSD-miekg-dns
+
+The asn1-ber.v1 component is used under the MIT license:
+@traffic_ops/vendor/gopkg.in/asn1-ber.v1/*
+./traffic_ops/vendor/gopkg.in/asn1-ber.v1/LICENSE
+
+The ldap.v2 component is used under the MIT license:
+@traffic_ops/vendor/gopkg.in/ldap.v2/*
+./traffic_ops/vendor/gopkg.in/ldap.v2/LICENSE
+
+The json-iterator/go component is used under the MIT license:
+@vendor/github.com/json-iterator/go/*
+./vendor/github.com/json-iterator/go/LICENSE
+
+The modern-go/concurrent component is used under the Apache 2.0 license:
+@vendor/github.com/modern-go/concurrent/*
+./vendor/github.com/modern-go/concurrent/LICENSE
+
+The modern-go/reflect2 component is used under the Apache 2.0 license:
+@vendor/github.com/modern-go/reflect2/*
+./vendor/github.com/modern-go/reflect2/LICENSE
+
+For the lestrrat-go/jwx (commit e35178a) component:
+@traffic_ops/traffic_ops_golang/vendor/github.com/lestrrat-go/jwx/*
+./traffic_ops/traffic_ops_golang/vendor/github.com/lestrrat-go/jwx/LICENSE
+
+For the dgrijalva/jwt-go (commit 5e25c22) component:
+@traffic_ops/traffic_ops_golang/vendor/github.com/dgrijalva/jwt-go/*
+./traffic_ops/traffic_ops_golang/vendor/github.com/dgrijalva/jwt-go/LICENSE
+
+For the ogier/pflag (commit 45c278a) component:
+@vendor/github.com/ogier/pflag/*
+./licenses/BSD-pflag
+
+For the errors (commit 27936f6) component:
+@vendor/github.com/pkg/errors/*
+./vendor/github.com/pkg/errors/LICENSE
diff --git a/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/VERSION b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/VERSION
new file mode 100644
index 0000000..fcdb2e1
--- /dev/null
+++ b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/VERSION
@@ -0,0 +1 @@
+4.0.0
diff --git a/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/changeset.txt b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/changeset.txt
new file mode 100644
index 0000000..2e13fdf
--- /dev/null
+++ b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/changeset.txt
@@ -0,0 +1,2 @@
+4.0.x
+2294a43da787cc4505d892261b10563a5c218841
diff --git a/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/README.md b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/README.md
new file mode 100644
index 0000000..f445a22
--- /dev/null
+++ b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/README.md
@@ -0,0 +1,53 @@
+# TO Client Library Golang
+
+## Getting Started
+1. Obtain the latest version of the library
+
+`go get github.com/apache/trafficcontrol/traffic_ops/client`
+
+2. Get a basic TO session started and fetch a list of CDNs
+```go
+package main
+
+import (
+ "fmt"
+ "os"
+ "time"
+
+ "github.com/apache/trafficcontrol/lib/go-tc"
+ toclient "github.com/apache/trafficcontrol/traffic_ops/client"
+)
+
+const TOURL = "http://localhost"
+const TOUser = "user"
+const TOPassword = "password"
+const AllowInsecureConnections = true
+const UserAgent = "MySampleApp"
+const UseClientCache = false
+const TrafficOpsRequestTimeout = time.Second * time.Duration(10)
+
+func main() {
+ session, remoteaddr, err := toclient.LoginWithAgent(
+ TOURL,
+ TOUser,
+ TOPassword,
+ AllowInsecureConnections,
+ UserAgent,
+ UseClientCache,
+ TrafficOpsRequestTimeout)
+ if err != nil {
+ fmt.Printf("An error occurred while logging in:\n\t%v\n", err)
+ os.Exit(1)
+ }
+ fmt.Println("Connected to: " + remoteaddr.String())
+ var cdns []v13.CDN
+ cdns, _, err = session.GetCDNs()
+ if err != nil {
+ fmt.Printf("An error occurred while getting cdns:\n\t%v\n", err)
+ os.Exit(1)
+ }
+ for _, cdn := range cdns {
+ fmt.Println(cdn.Name)
+ }
+}
+```
diff --git a/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/about.go b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/about.go
new file mode 100644
index 0000000..8aa345e
--- /dev/null
+++ b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/about.go
@@ -0,0 +1,43 @@
+/*
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package client
+
+import (
+ "encoding/json"
+ "net/http"
+)
+
+const (
+ API_v13_ABOUT = "/api/1.3/about"
+)
+
+// GetAbout gets data about the TO instance
+func (to *Session) GetAbout() (map[string]string, ReqInf, error) {
+ route := API_v13_ABOUT
+ resp, remoteAddr, err := to.request(http.MethodGet, route, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data map[string]string
+ if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return nil, reqInf, err
+ }
+
+ return data, reqInf, nil
+}
diff --git a/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/asn.go b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/asn.go
new file mode 100644
index 0000000..2a38c65
--- /dev/null
+++ b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/asn.go
@@ -0,0 +1,132 @@
+/*
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package client
+
+import (
+ "encoding/json"
+ "fmt"
+ "net"
+ "net/http"
+
+ "github.com/apache/trafficcontrol/lib/go-tc"
+)
+
+const (
+ API_v2_ASNs = "/api/1.3/asns"
+)
+
+// Create a ASN
+func (to *Session) CreateASN(entity tc.ASN) (tc.Alerts, ReqInf, error) {
+
+ var remoteAddr net.Addr
+ reqBody, err := json.Marshal(entity)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ resp, remoteAddr, err := to.request(http.MethodPost, API_v2_ASNs, reqBody)
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ defer resp.Body.Close()
+ var alerts tc.Alerts
+ err = json.NewDecoder(resp.Body).Decode(&alerts)
+ return alerts, reqInf, nil
+}
+
+// Update a ASN by ID
+func (to *Session) UpdateASNByID(id int, entity tc.ASN) (tc.Alerts, ReqInf, error) {
+
+ var remoteAddr net.Addr
+ reqBody, err := json.Marshal(entity)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ route := fmt.Sprintf("%s/%d", API_v2_ASNs, id)
+ resp, remoteAddr, err := to.request(http.MethodPut, route, reqBody)
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ defer resp.Body.Close()
+ var alerts tc.Alerts
+ err = json.NewDecoder(resp.Body).Decode(&alerts)
+ return alerts, reqInf, nil
+}
+
+// Returns a list of ASNs
+func (to *Session) GetASNs() ([]tc.ASN, ReqInf, error) {
+ resp, remoteAddr, err := to.request(http.MethodGet, API_v2_ASNs, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data tc.ASNsResponse
+ err = json.NewDecoder(resp.Body).Decode(&data)
+ return data.Response, reqInf, nil
+}
+
+// GET a ASN by the id
+func (to *Session) GetASNByID(id int) ([]tc.ASN, ReqInf, error) {
+ route := fmt.Sprintf("%s/%d", API_v2_ASNs, id)
+ resp, remoteAddr, err := to.request(http.MethodGet, route, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data tc.ASNsResponse
+ if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return nil, reqInf, err
+ }
+
+ return data.Response, reqInf, nil
+}
+
+// GET an ASN by the asn number
+func (to *Session) GetASNByASN(asn int) ([]tc.ASN, ReqInf, error) {
+ url := fmt.Sprintf("%s?asn=%d", API_v2_ASNs, asn)
+ resp, remoteAddr, err := to.request(http.MethodGet, url, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data tc.ASNsResponse
+ if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return nil, reqInf, err
+ }
+
+ return data.Response, reqInf, nil
+}
+
+// DELETE an ASN by asn number
+func (to *Session) DeleteASNByASN(asn int) (tc.Alerts, ReqInf, error) {
+ route := fmt.Sprintf("%s/asn/%d", API_v2_ASNs, asn)
+ resp, remoteAddr, err := to.request(http.MethodDelete, route, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ defer resp.Body.Close()
+ var alerts tc.Alerts
+ err = json.NewDecoder(resp.Body).Decode(&alerts)
+ return alerts, reqInf, nil
+}
diff --git a/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/atsconfig.go b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/atsconfig.go
new file mode 100644
index 0000000..6432c23
--- /dev/null
+++ b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/atsconfig.go
@@ -0,0 +1,88 @@
+/*
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package client
+
+import (
+ "encoding/json"
+ "io/ioutil"
+ "net/http"
+ "strconv"
+
+ "github.com/apache/trafficcontrol/lib/go-tc"
+)
+
+func (to *Session) GetATSServerConfigList(serverID int) (tc.ATSConfigMetaData, ReqInf, error) {
+ resp, remoteAddr, err := to.request(http.MethodGet, apiBase+"/servers/"+strconv.Itoa(serverID)+"/configfiles/ats", nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return tc.ATSConfigMetaData{}, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ data := tc.ATSConfigMetaData{}
+ if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return tc.ATSConfigMetaData{}, reqInf, err
+ }
+ return data, reqInf, nil
+}
+
+func (to *Session) GetATSServerConfigListByName(serverHostName string) (tc.ATSConfigMetaData, ReqInf, error) {
+ resp, remoteAddr, err := to.request(http.MethodGet, apiBase+"/servers/"+serverHostName+"/configfiles/ats", nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return tc.ATSConfigMetaData{}, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ data := tc.ATSConfigMetaData{}
+ if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return tc.ATSConfigMetaData{}, reqInf, err
+ }
+ return data, reqInf, nil
+}
+
+func (to *Session) GetATSServerConfig(serverID int, fileName string) (string, ReqInf, error) {
+ return to.getConfigFile(apiBase + "/servers/" + strconv.Itoa(serverID) + "/configfiles/ats/" + fileName)
+}
+
+func (to *Session) GetATSServerConfigByName(serverHostName string, fileName string) (string, ReqInf, error) {
+ return to.getConfigFile(apiBase + "/servers/" + serverHostName + "/configfiles/ats/" + fileName)
+}
+
+func (to *Session) GetATSProfileConfig(profileID int, fileName string) (string, ReqInf, error) {
+ return to.getConfigFile(apiBase + "/profiles/" + strconv.Itoa(profileID) + "/configfiles/ats/" + fileName)
+}
+
+func (to *Session) GetATSProfileConfigByName(profileName string, fileName string) (string, ReqInf, error) {
+ return to.getConfigFile(apiBase + "/profiles/" + profileName + "/configfiles/ats/" + fileName)
+}
+
+func (to *Session) GetATSCDNConfig(cdnID int, fileName string) (string, ReqInf, error) {
+ return to.getConfigFile(apiBase + "/cdns/" + strconv.Itoa(cdnID) + "/configfiles/ats/" + fileName)
+}
+
+func (to *Session) GetATSCDNConfigByName(cdnName string, fileName string) (string, ReqInf, error) {
+ return to.getConfigFile(apiBase + "/cdns/" + cdnName + "/configfiles/ats/" + fileName)
+}
+
+func (to *Session) getConfigFile(uri string) (string, ReqInf, error) {
+ resp, remoteAddr, err := to.request(http.MethodGet, uri, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return "", reqInf, err
+ }
+ defer resp.Body.Close()
+
+ bts, err := ioutil.ReadAll(resp.Body)
+ return string(bts), reqInf, err
+}
diff --git a/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/cachegroup.go b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/cachegroup.go
new file mode 100644
index 0000000..ad43ced
--- /dev/null
+++ b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/cachegroup.go
@@ -0,0 +1,306 @@
+/*
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package client
+
+import (
+ "encoding/json"
+ "errors"
+ "fmt"
+ "net"
+ "net/http"
+ "net/url"
+
+ "github.com/apache/trafficcontrol/lib/go-tc"
+)
+
+const (
+ API_v13_CacheGroups = "/api/1.3/cachegroups"
+)
+
+// Create a CacheGroup
+func (to *Session) CreateCacheGroupNullable(cachegroup tc.CacheGroupNullable) (*tc.CacheGroupDetailResponse, ReqInf, error) {
+ if cachegroup.TypeID == nil && cachegroup.Type != nil {
+ ty, _, err := to.GetTypeByName(*cachegroup.Type)
+ if err != nil {
+ return nil, ReqInf{}, err
+ }
+ if len(ty) == 0 {
+ return nil, ReqInf{}, errors.New("no type named " + *cachegroup.Type)
+ }
+ cachegroup.TypeID = &ty[0].ID
+ }
+
+ if cachegroup.ParentCachegroupID == nil && cachegroup.ParentName != nil {
+ p, _, err := to.GetCacheGroupByName(*cachegroup.ParentName)
+ if err != nil {
+ return nil, ReqInf{}, err
+ }
+ if len(p) == 0 {
+ return nil, ReqInf{}, errors.New("no cachegroup named " + *cachegroup.ParentName)
+ }
+ cachegroup.ParentCachegroupID = &p[0].ID
+ }
+
+ if cachegroup.SecondaryParentCachegroupID == nil && cachegroup.SecondaryParentName != nil {
+ p, _, err := to.GetCacheGroupByName(*cachegroup.SecondaryParentName)
+ if err != nil {
+ return nil, ReqInf{}, err
+ }
+ if len(p) == 0 {
+ return nil, ReqInf{}, errors.New("no cachegroup named " + *cachegroup.ParentName)
+ }
+ cachegroup.SecondaryParentCachegroupID = &p[0].ID
+ }
+
+ reqBody, err := json.Marshal(cachegroup)
+ if err != nil {
+ return nil, ReqInf{}, err
+ }
+ resp, remoteAddr, err := to.request(http.MethodPost, API_v13_CacheGroups, reqBody)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+ var cachegroupResp tc.CacheGroupDetailResponse
+ if err = json.NewDecoder(resp.Body).Decode(&cachegroupResp); err != nil {
+ return nil, reqInf, err
+ }
+ return &cachegroupResp, reqInf, nil
+}
+
+// Create a CacheGroup
+// Deprecated: Use CreateCacheGroupNullable
+func (to *Session) CreateCacheGroup(cachegroup tc.CacheGroup) (tc.Alerts, ReqInf, error) {
+ if cachegroup.TypeID == 0 && cachegroup.Type != "" {
+ ty, _, err := to.GetTypeByName(cachegroup.Type)
+ if err != nil {
+ return tc.Alerts{}, ReqInf{}, err
+ }
+ if len(ty) == 0 {
+ return tc.Alerts{}, ReqInf{}, errors.New("no type named " + cachegroup.Type)
+ }
+ cachegroup.TypeID = ty[0].ID
+ }
+
+ if cachegroup.ParentCachegroupID == 0 && cachegroup.ParentName != "" {
+ p, _, err := to.GetCacheGroupByName(cachegroup.ParentName)
+ if err != nil {
+ return tc.Alerts{}, ReqInf{}, err
+ }
+ if len(p) == 0 {
+ return tc.Alerts{}, ReqInf{}, errors.New("no cachegroup named " + cachegroup.ParentName)
+ }
+ cachegroup.ParentCachegroupID = p[0].ID
+ }
+
+ if cachegroup.SecondaryParentCachegroupID == 0 && cachegroup.SecondaryParentName != "" {
+ p, _, err := to.GetCacheGroupByName(cachegroup.SecondaryParentName)
+ if err != nil {
+ return tc.Alerts{}, ReqInf{}, err
+ }
+ if len(p) == 0 {
+ return tc.Alerts{}, ReqInf{}, errors.New("no cachegroup named " + cachegroup.ParentName)
+ }
+ cachegroup.SecondaryParentCachegroupID = p[0].ID
+ }
+
+ reqBody, err := json.Marshal(cachegroup)
+ if err != nil {
+ return tc.Alerts{}, ReqInf{}, err
+ }
+ resp, remoteAddr, err := to.request(http.MethodPost, API_v13_CacheGroups, reqBody)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ defer resp.Body.Close()
+ var alerts tc.Alerts
+ err = json.NewDecoder(resp.Body).Decode(&alerts)
+ return alerts, reqInf, nil
+}
+
+// Update a CacheGroup by ID
+func (to *Session) UpdateCacheGroupNullableByID(id int, cachegroup tc.CacheGroupNullable) (*tc.CacheGroupDetailResponse, ReqInf, error) {
+
+ var remoteAddr net.Addr
+ reqBody, err := json.Marshal(cachegroup)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ route := fmt.Sprintf("%s/%d", API_v13_CacheGroups, id)
+ resp, remoteAddr, err := to.request(http.MethodPut, route, reqBody)
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+ var cachegroupResp tc.CacheGroupDetailResponse
+ if err = json.NewDecoder(resp.Body).Decode(&cachegroupResp); err != nil {
+ return nil, reqInf, err
+ }
+ return &cachegroupResp, reqInf, nil
+}
+
+// Update a CacheGroup by ID
+// Deprecated: use UpdateCachegroupNullableByID
+func (to *Session) UpdateCacheGroupByID(id int, cachegroup tc.CacheGroup) (tc.Alerts, ReqInf, error) {
+
+ var remoteAddr net.Addr
+ reqBody, err := json.Marshal(cachegroup)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ route := fmt.Sprintf("%s/%d", API_v13_CacheGroups, id)
+ resp, remoteAddr, err := to.request(http.MethodPut, route, reqBody)
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ defer resp.Body.Close()
+ var alerts tc.Alerts
+ err = json.NewDecoder(resp.Body).Decode(&alerts)
+ return alerts, reqInf, nil
+}
+
+// Returns a list of CacheGroups
+func (to *Session) GetCacheGroupsNullable() ([]tc.CacheGroupNullable, ReqInf, error) {
+ resp, remoteAddr, err := to.request(http.MethodGet, API_v13_CacheGroups, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data tc.CacheGroupsNullableResponse
+ if err = json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return nil, reqInf, err
+ }
+ return data.Response, reqInf, nil
+}
+
+// Returns a list of CacheGroups
+// Deprecated: use GetCacheGroupsNullable
+func (to *Session) GetCacheGroups() ([]tc.CacheGroup, ReqInf, error) {
+ resp, remoteAddr, err := to.request(http.MethodGet, API_v13_CacheGroups, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data tc.CacheGroupsResponse
+ err = json.NewDecoder(resp.Body).Decode(&data)
+ return data.Response, reqInf, nil
+}
+
+// CacheGroups gets the CacheGroups in an array of CacheGroup structs
+// (note CacheGroup used to be called location)
+// Deprecated: use GetCacheGroups.
+func (to *Session) CacheGroups() ([]tc.CacheGroup, error) {
+ cgs, _, err := to.GetCacheGroups()
+ return cgs, err
+}
+
+// GET a CacheGroup by the CacheGroup id
+func (to *Session) GetCacheGroupNullableByID(id int) ([]tc.CacheGroupNullable, ReqInf, error) {
+ route := fmt.Sprintf("%s/%d", API_v13_CacheGroups, id)
+ resp, remoteAddr, err := to.request(http.MethodGet, route, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data tc.CacheGroupsNullableResponse
+ if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return nil, reqInf, err
+ }
+
+ return data.Response, reqInf, nil
+}
+
+// GET a CacheGroup by the CacheGroup id
+// Deprecated: use GetCacheGroupNullableByID
+func (to *Session) GetCacheGroupByID(id int) ([]tc.CacheGroup, ReqInf, error) {
+ route := fmt.Sprintf("%s/%d", API_v13_CacheGroups, id)
+ resp, remoteAddr, err := to.request(http.MethodGet, route, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data tc.CacheGroupsResponse
+ if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return nil, reqInf, err
+ }
+
+ return data.Response, reqInf, nil
+}
+
+// GET a CacheGroup by the CacheGroup name
+func (to *Session) GetCacheGroupNullableByName(name string) ([]tc.CacheGroupNullable, ReqInf, error) {
+ url := fmt.Sprintf("%s?name=%s", API_v13_CacheGroups, url.QueryEscape(name))
+ resp, remoteAddr, err := to.request(http.MethodGet, url, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data tc.CacheGroupsNullableResponse
+ if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return nil, reqInf, err
+ }
+
+ return data.Response, reqInf, nil
+}
+
+// GET a CacheGroup by the CacheGroup name
+// Deprecated: use GetCachegroupNullableByName
+func (to *Session) GetCacheGroupByName(name string) ([]tc.CacheGroup, ReqInf, error) {
+ url := fmt.Sprintf("%s?name=%s", API_v13_CacheGroups, url.QueryEscape(name))
+ resp, remoteAddr, err := to.request(http.MethodGet, url, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data tc.CacheGroupsResponse
+ if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return nil, reqInf, err
+ }
+
+ return data.Response, reqInf, nil
+}
+
+// DELETE a CacheGroup by ID
+func (to *Session) DeleteCacheGroupByID(id int) (tc.Alerts, ReqInf, error) {
+ route := fmt.Sprintf("%s/%d", API_v13_CacheGroups, id)
+ resp, remoteAddr, err := to.request(http.MethodDelete, route, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ defer resp.Body.Close()
+ var alerts tc.Alerts
+ if err = json.NewDecoder(resp.Body).Decode(&alerts); err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ return alerts, reqInf, nil
+}
diff --git a/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/cachegroup_parameters.go b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/cachegroup_parameters.go
new file mode 100644
index 0000000..c42aef3
--- /dev/null
+++ b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/cachegroup_parameters.go
@@ -0,0 +1,109 @@
+/*
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package client
+
+import (
+ "encoding/json"
+ "fmt"
+ "net/http"
+
+ "github.com/apache/trafficcontrol/lib/go-tc"
+)
+
+const (
+ API_v14_CacheGroupParameters = "/api/1.4/cachegroupparameters"
+)
+
+// GetCacheGroupParameters Gets a Cache Group's Parameters
+func (to *Session) GetCacheGroupParameters(cacheGroupID int) ([]tc.CacheGroupParameter, ReqInf, error) {
+ route := fmt.Sprintf("%s/%d/parameters", API_v13_CacheGroups, cacheGroupID)
+ return to.getCacheGroupParameters(route, "")
+}
+
+// GetCacheGroupParametersByQueryParams Gets a Cache Group's Parameters with query parameters
+func (to *Session) GetCacheGroupParametersByQueryParams(cacheGroupID int, queryParams string) ([]tc.CacheGroupParameter, ReqInf, error) {
+ route := fmt.Sprintf("%s/%d/parameters", API_v13_CacheGroups, cacheGroupID)
+ return to.getCacheGroupParameters(route, queryParams)
+}
+
+// GetCacheGroupUnassignedParameters Gets a Cache Group's Unassigned Parameters
+func (to *Session) GetCacheGroupUnassignedParameters(cacheGroupID int) ([]tc.CacheGroupParameter, ReqInf, error) {
+ route := fmt.Sprintf("%s/%d/unassigned_parameters", API_v13_CacheGroups, cacheGroupID)
+ return to.getCacheGroupParameters(route, "")
+}
+
+// GetCacheGroupParametersByQueryParams Gets a Cache Group's Unassigned Parameters with query parameters
+func (to *Session) GetCacheGroupUnassignedParametersByQueryParams(cacheGroupID int, queryParams string) ([]tc.CacheGroupParameter, ReqInf, error) {
+ route := fmt.Sprintf("%s/%d/unassigned_parameters", API_v13_CacheGroups, cacheGroupID)
+ return to.getCacheGroupParameters(route, queryParams)
+}
+
+func (to *Session) getCacheGroupParameters(route, queryParams string) ([]tc.CacheGroupParameter, ReqInf, error) {
+ r := fmt.Sprintf("%s%s", route, queryParams)
+ resp, remoteAddr, err := to.request(http.MethodGet, r, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data tc.CacheGroupParametersResponse
+ if err = json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return nil, reqInf, err
+ }
+ return data.Response, reqInf, nil
+}
+
+// DeleteCacheGroupParameter Deassociates a Parameter with a Cache Group
+func (to *Session) DeleteCacheGroupParameter(cacheGroupID, parameterID int) (tc.Alerts, ReqInf, error) {
+ route := fmt.Sprintf("%s/%d/%d", API_v14_CacheGroupParameters, cacheGroupID, parameterID)
+ resp, remoteAddr, err := to.request(http.MethodDelete, route, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var alerts tc.Alerts
+ if err = json.NewDecoder(resp.Body).Decode(&alerts); err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ return alerts, reqInf, nil
+}
+
+// CreateCacheGroupParameter Associates a Parameter with a Cache Group
+func (to *Session) CreateCacheGroupParameter(cacheGroupID, parameterID int) (*tc.CacheGroupParametersPostResponse, ReqInf, error) {
+ cacheGroupParameterReq := tc.CacheGroupParameterRequest{
+ CacheGroupID: cacheGroupID,
+ ParameterID: parameterID,
+ }
+ reqBody, err := json.Marshal(cacheGroupParameterReq)
+ if err != nil {
+ return nil, ReqInf{}, err
+ }
+ resp, remoteAddr, err := to.request(http.MethodPost, API_v14_CacheGroupParameters, reqBody)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data tc.CacheGroupParametersPostResponse
+ if err = json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return nil, reqInf, err
+ }
+ return &data, reqInf, nil
+}
diff --git a/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/cdn.go b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/cdn.go
new file mode 100644
index 0000000..4503a78
--- /dev/null
+++ b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/cdn.go
@@ -0,0 +1,176 @@
+/*
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package client
+
+import (
+ "encoding/json"
+ "fmt"
+ "net"
+ "net/http"
+ "net/url"
+
+ "github.com/apache/trafficcontrol/lib/go-tc"
+)
+
+const (
+ API_v13_CDNs = "/api/1.3/cdns"
+)
+
+// Create a CDN
+func (to *Session) CreateCDN(cdn tc.CDN) (tc.Alerts, ReqInf, error) {
+
+ var remoteAddr net.Addr
+ reqBody, err := json.Marshal(cdn)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ resp, remoteAddr, err := to.request(http.MethodPost, API_v13_CDNs, reqBody)
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ defer resp.Body.Close()
+ var alerts tc.Alerts
+ err = json.NewDecoder(resp.Body).Decode(&alerts)
+ return alerts, reqInf, nil
+}
+
+// Update a CDN by ID
+func (to *Session) UpdateCDNByID(id int, cdn tc.CDN) (tc.Alerts, ReqInf, error) {
+
+ var remoteAddr net.Addr
+ reqBody, err := json.Marshal(cdn)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ route := fmt.Sprintf("%s/%d", API_v13_CDNs, id)
+ resp, remoteAddr, err := to.request(http.MethodPut, route, reqBody)
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ defer resp.Body.Close()
+ var alerts tc.Alerts
+ err = json.NewDecoder(resp.Body).Decode(&alerts)
+ return alerts, reqInf, nil
+}
+
+// Returns a list of CDNs
+func (to *Session) GetCDNs() ([]tc.CDN, ReqInf, error) {
+ resp, remoteAddr, err := to.request(http.MethodGet, API_v13_CDNs, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data tc.CDNsResponse
+ err = json.NewDecoder(resp.Body).Decode(&data)
+ return data.Response, reqInf, nil
+}
+
+// GET a CDN by the CDN ID
+func (to *Session) GetCDNByID(id int) ([]tc.CDN, ReqInf, error) {
+ route := fmt.Sprintf("%s/%d", API_v13_CDNs, id)
+ resp, remoteAddr, err := to.request(http.MethodGet, route, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data tc.CDNsResponse
+ if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return nil, reqInf, err
+ }
+
+ return data.Response, reqInf, nil
+}
+
+// GET a CDN by the CDN name
+func (to *Session) GetCDNByName(name string) ([]tc.CDN, ReqInf, error) {
+ url := fmt.Sprintf("%s?name=%s", API_v13_CDNs, url.QueryEscape(name))
+ resp, remoteAddr, err := to.request(http.MethodGet, url, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data tc.CDNsResponse
+ if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return nil, reqInf, err
+ }
+
+ return data.Response, reqInf, nil
+}
+
+// DELETE a CDN by ID
+func (to *Session) DeleteCDNByID(id int) (tc.Alerts, ReqInf, error) {
+ route := fmt.Sprintf("%s/%d", API_v13_CDNs, id)
+ resp, remoteAddr, err := to.request(http.MethodDelete, route, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ defer resp.Body.Close()
+ var alerts tc.Alerts
+ err = json.NewDecoder(resp.Body).Decode(&alerts)
+ return alerts, reqInf, nil
+}
+
+func (to *Session) GetCDNSSLKeys(name string) ([]tc.CDNSSLKeys, ReqInf, error) {
+ url := fmt.Sprintf("%s/name/%s/sslkeys", API_v13_CDNs, name)
+ resp, remoteAddr, err := to.request(http.MethodGet, url, nil)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return nil, reqInf, err
+ }
+ defer resp.Body.Close()
+
+ var data tc.CDNSSLKeysResponse
+ if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
+ return nil, reqInf, err
+ }
+
+ return data.Response, reqInf, nil
+}
+
+// Deprecated: use GetCDNs.
+func (to *Session) CDNs() ([]tc.CDN, error) {
+ cdns, _, err := to.GetCDNs()
+ return cdns, err
+}
+
+// CDNName gets an array of CDNs
+// Deprecated: use GetCDNByName
+func (to *Session) CDNName(name string) ([]tc.CDN, error) {
+ n, _, err := to.GetCDNByName(name)
+ return n, err
+}
+
+// CDNName gets an array of CDNs
+// Deprecated: use GetCDNByName
+func (to *Session) GetCDNName(name string) ([]tc.CDN, error) {
+ n, _, err := to.GetCDNByName(name)
+ return n, err
+}
+
+// Deprecated: use GetCDNSSLKeys
+func (to *Session) CDNSSLKeys(name string) ([]tc.CDNSSLKeys, error) {
+ ks, _, err := to.GetCDNSSLKeys(name)
+ return ks, err
+}
diff --git a/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/cdn_domains.go b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/cdn_domains.go
new file mode 100644
index 0000000..5e73db9
--- /dev/null
+++ b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/cdn_domains.go
@@ -0,0 +1,29 @@
+package client
+
+import (
+ "github.com/apache/trafficcontrol/lib/go-tc"
+)
+
+/*
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+func (to *Session) GetDomains() ([]tc.Domain, ReqInf, error) {
+ var data tc.DomainsResponse
+ inf, err := get(to, apiBase+"/cdns/domains", &data)
+ if err != nil {
+ return nil, inf, err
+ }
+ return data.Response, inf, nil
+}
diff --git a/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/cdnfederations.go b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/cdnfederations.go
new file mode 100644
index 0000000..e9f04b1
--- /dev/null
+++ b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/cdnfederations.go
@@ -0,0 +1,73 @@
+/*
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package client
+
+import (
+ "encoding/json"
+ "fmt"
+
+ "github.com/apache/trafficcontrol/lib/go-tc"
+)
+
+/* Internally, the CDNName is only used in the GET method. The CDNName
+ * seems to primarily be used to differentiate between `/federations` and
+ * `/cdns/:name/federations`. Although the behavior is odd, it is kept to
+ * keep the same behavior from perl. */
+
+func (to *Session) CreateCDNFederationByName(f tc.CDNFederation, CDNName string) (*tc.CreateCDNFederationResponse, ReqInf, error) {
+ jsonReq, err := json.Marshal(f)
+ if err != nil { //There is no remoteAddr for ReqInf at this point
+ return nil, ReqInf{CacheHitStatus: CacheHitStatusMiss}, err
+ }
+
+ data := tc.CreateCDNFederationResponse{}
+ url := fmt.Sprintf("%s/cdns/%s/federations", apiBase, CDNName)
+ inf, err := makeReq(to, "POST", url, jsonReq, &data)
+ return &data, inf, err
+}
+
+func (to *Session) GetCDNFederationsByName(CDNName string) (*tc.CDNFederationResponse, ReqInf, error) {
+ data := tc.CDNFederationResponse{}
+ url := fmt.Sprintf("%s/cdns/%s/federations", apiBase, CDNName)
+ inf, err := get(to, url, &data)
+ return &data, inf, err
+}
+
+func (to *Session) GetCDNFederationsByID(CDNName string, ID int) (*tc.CDNFederationResponse, ReqInf, error) {
+ data := tc.CDNFederationResponse{}
+ url := fmt.Sprintf("%s/cdns/%s/federations/%d", apiBase, CDNName, ID)
+ inf, err := get(to, url, &data)
+ return &data, inf, err
+}
+
+func (to *Session) UpdateCDNFederationsByID(f tc.CDNFederation, CDNName string, ID int) (*tc.UpdateCDNFederationResponse, ReqInf, error) {
+ jsonReq, err := json.Marshal(f)
+ if err != nil { //There is no remoteAddr for ReqInf at this point
+ return nil, ReqInf{CacheHitStatus: CacheHitStatusMiss}, err
+ }
+
+ data := tc.UpdateCDNFederationResponse{}
+ url := fmt.Sprintf("%s/cdns/%s/federations/%d", apiBase, CDNName, ID)
+ inf, err := makeReq(to, "PUT", url, jsonReq, &data)
+ return &data, inf, err
+}
+
+func (to *Session) DeleteCDNFederationByID(CDNName string, ID int) (*tc.DeleteCDNFederationResponse, ReqInf, error) {
+ data := tc.DeleteCDNFederationResponse{}
+ url := fmt.Sprintf("%s/cdns/%s/federations/%d", apiBase, CDNName, ID)
+ inf, err := makeReq(to, "DELETE", url, nil, &data)
+ return &data, inf, err
+}
diff --git a/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/coordinate.go b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/coordinate.go
new file mode 100644
index 0000000..ab209bd
--- /dev/null
+++ b/traffic_ops/ort/vendor/github.com/apache/trafficcontrol/traffic_ops/client/coordinate.go
@@ -0,0 +1,155 @@
+/*
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+package client
+
+import (
+ "encoding/json"
+ "errors"
+ "fmt"
+ "net"
+ "net/http"
+ "strconv"
+
+ "github.com/apache/trafficcontrol/lib/go-tc"
+)
+
+const (
+ API_V13_Coordinates = "/api/1.3/coordinates"
+)
+
+// Create a Coordinate
+func (to *Session) CreateCoordinate(coordinate tc.Coordinate) (tc.Alerts, ReqInf, error) {
+
+ var remoteAddr net.Addr
+ reqBody, err := json.Marshal(coordinate)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ resp, remoteAddr, err := to.request(http.MethodPost, API_V13_Coordinates, reqBody)
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ defer resp.Body.Close()
+ var alerts tc.Alerts
+ err = json.NewDecoder(resp.Body).Decode(&alerts)
+ return alerts, reqInf, nil
+}
+
+// Update a Coordinate by ID
+func (to *Session) UpdateCoordinateByID(id int, coordinate tc.Coordinate) (tc.Alerts, ReqInf, error) {
+
+ var remoteAddr net.Addr
+ reqBody, err := json.Marshal(coordinate)
+ reqInf := ReqInf{CacheHitStatus: CacheHitStatusMiss, RemoteAddr: remoteAddr}
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ route := fmt.Sprintf("%s?id=%d", API_V13_Coordinates, id)
+ resp, remoteAddr, err := to.request(http.MethodPut, route, reqBody)
+ if err != nil {
+ return tc.Alerts{}, reqInf, err
+ }
+ defer resp.Body.Close()
+ var alerts tc.Alerts
+ err = json.NewDecoder(resp.Body).Decode(&alerts)
... 6109 lines suppressed ...