You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficcontrol.apache.org by zr...@apache.org on 2021/08/30 17:06:13 UTC
[trafficcontrol] branch 6.0.x updated (3a4b570 -> 1c4ec19)
This is an automated email from the ASF dual-hosted git repository.
zrhoffman pushed a change to branch 6.0.x
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git.
from 3a4b570 added rel="noreferrer" to the anchor where '<a href="{{::url}}" target="_blank">' is used. (#6119)
new ed76701 Update CHANGELOG.md and SECURITY.md for 6.0.0 release
new 201032b Use tag 6.0.x for the builder images
new 8f5d0bc Fixed Federations IMS (#6122)
new 4381e9c TP tests alert logging (#6120)
new 1f210d5 Fix wording for package uninstall (#6131)
new 2e3151b Run goose~schema_migrations trigger after UPDATE, not INSERT (#6134)
new 1e6a3c7 Skip check for the past dbversion's migration existing if it is squashed (#6128)
new edf9093 t3cutil: core dump fix for TOClient SetURL interface (#6138)
new aa3fa79 Migrate Profiles test from E2E to integrate test suite TP (#6071)
new 66971a0 Fix roles hard code for TP integration test (#6139)
new 77a79b5 Use PostgreSQL 9.6-compatible syntax in create_tables.sql and migrations (#6143)
new 1c4ec19 Make TM recover from TO being unavailable at startup (#6146)
The 12 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/actions/todb-tests/entrypoint.sh | 11 +-
.github/actions/tp-integration-tests/cdn.json | 10 +-
.github/actions/tp-integration-tests/entrypoint.sh | 90 ++++-----
CHANGELOG.md | 6 +-
SECURITY.md | 3 +-
cache-config/t3c-apply/torequest/torequest.go | 2 +-
cache-config/t3cutil/toreq/client.go | 3 +-
infrastructure/docker/build/docker-compose-opt.yml | 2 +-
infrastructure/docker/build/docker-compose.yml | 20 +-
traffic_monitor/handler/handler.go | 2 +-
traffic_monitor/manager/opsconfig.go | 4 +-
traffic_monitor/towrap/towrap.go | 144 +++++++++-----
.../{datareq/version.go => towrap/towrap_test.go} | 21 +-
traffic_ops/app/db/admin.go | 59 ++++--
.../app/db/migrations/00000000000000_init.up.sql | 4 +-
.../2021061100000000_ds_tls_versions.down.sql | 6 +-
.../2021061100000000_ds_tls_versions.up.sql | 10 +-
traffic_ops/app/db/trafficvault/create_tables.sql | 10 +-
traffic_ops/testing/api/v4/cdnfederations_test.go | 33 ++++
traffic_ops/testing/api/v4/federations_test.go | 21 +-
.../cdnfederation/cdnfederations.go | 4 +
.../federations/allfederations.go | 54 ++---
.../traffic_ops_golang/federations/federations.go | 67 +++----
traffic_ops_db/test/docker/Dockerfile-db-admin | 38 +++-
traffic_ops_db/test/docker/run-db-test.sh | 2 +-
traffic_portal/test/integration/CommonUtils/API.ts | 219 ++++++++++++++-------
.../test/integration/CommonUtils/index.ts | 4 +-
.../test/integration/CommonUtils/utils.ts | 133 +++++++++++++
.../test/integration/Data/prerequisites.ts | 96 +++++++--
traffic_portal/test/integration/Data/profiles.ts | 32 +++
.../integration/PageObjects/ProfilesPage.po.ts | 39 ++--
traffic_portal/test/integration/config.model.ts | 165 ++++++++++++++++
traffic_portal/test/integration/config.ts | 21 +-
traffic_portal/test/integration/specs/ASNs.spec.ts | 3 +-
.../test/integration/specs/Coordinates.spec.ts | 3 +-
.../integration/specs/DeliveryServices.spec.ts | 5 +-
.../test/integration/specs/Divisions.spec.ts | 4 +-
traffic_portal/test/integration/specs/Jobs.spec.ts | 3 +-
.../test/integration/specs/Origins.spec.ts | 3 +-
.../test/integration/specs/Parameters.spec.ts | 5 +-
.../test/integration/specs/PhysLocations.spec.ts | 3 +-
.../test/integration/specs/Profiles.spec.ts | 82 +++++---
.../test/integration/specs/Regions.spec.ts | 3 +-
.../specs/ServerServerCapabilities.spec.ts | 3 +-
.../test/integration/specs/Servers.spec.ts | 3 +-
.../integration/specs/ServiceCategories.spec.ts | 3 +-
.../test/integration/specs/Statuses.spec.ts | 3 +-
.../test/integration/specs/Topologies.spec.ts | 6 +-
.../test/integration/specs/Types.spec.ts | 3 +-
49 files changed, 1072 insertions(+), 398 deletions(-)
copy traffic_monitor/{datareq/version.go => towrap/towrap_test.go} (65%)
copy cache-config/t3c-check-refs/test-files/libexec/cachekey.so => traffic_portal/test/integration/CommonUtils/index.ts (94%)
create mode 100644 traffic_portal/test/integration/CommonUtils/utils.ts
create mode 100644 traffic_portal/test/integration/config.model.ts
[trafficcontrol] 04/12: TP tests alert logging (#6120)
Posted by zr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zrhoffman pushed a commit to branch 6.0.x
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 4381e9c88d615afc8fceb7efff5d9fa1d6db4bce
Author: ocket8888 <oc...@apache.org>
AuthorDate: Fri Aug 20 11:15:17 2021 -0600
TP tests alert logging (#6120)
* Move `hasProperty` to its own module and provide type-specific property checking
* Add a data model for global testing configuration that can define log levels
* Make API take a configuration parameter and export an instance globally
* Add common logging of Alerts to all axios requests in the API class
* Update API usage to use new global export
* Fix implicit 'any' type detection in old TS versions
* Put TO logs in artifacts, clear up console output
This also fixes assumptions that asynchronous background jobs will have
reached a certain point by the time the foreground reaches a certain
point, and adds debug output to the TO logs. Plus uses `npm` scripts
instead of raw commands.
* Move Riak logs into artifacts, out of console
* Fix not restoring directory state after starting TP, moving open file descriptors
* upload artifacts on all failures, not just test failures
* Fix docker logs leaking stderr
* log full URL
* Wait for TO to start before continuing
* fix passing API requests through TP instead of straight to TO
* fix extra path separator in url building
* Fix incorrect info log, remove commented-out code
* Simplify checking property existence and type with new hasProperty call signatures
* Remove redundant option
* change debug logs to stdout
(cherry picked from commit 30c0feb6904de74517dd927df4c721f342105e57)
---
.github/actions/tp-integration-tests/cdn.json | 10 +-
.github/actions/tp-integration-tests/entrypoint.sh | 90 ++++-----
traffic_portal/test/integration/CommonUtils/API.ts | 211 ++++++++++++++-------
.../test/integration/CommonUtils/index.ts | 21 ++
.../test/integration/CommonUtils/utils.ts | 133 +++++++++++++
traffic_portal/test/integration/config.model.ts | 165 ++++++++++++++++
traffic_portal/test/integration/config.ts | 21 +-
traffic_portal/test/integration/specs/ASNs.spec.ts | 3 +-
.../test/integration/specs/Coordinates.spec.ts | 3 +-
.../integration/specs/DeliveryServices.spec.ts | 5 +-
.../test/integration/specs/Divisions.spec.ts | 4 +-
traffic_portal/test/integration/specs/Jobs.spec.ts | 3 +-
.../test/integration/specs/Origins.spec.ts | 3 +-
.../test/integration/specs/Parameters.spec.ts | 5 +-
.../test/integration/specs/PhysLocations.spec.ts | 3 +-
.../test/integration/specs/Profiles.spec.ts | 3 +-
.../test/integration/specs/Regions.spec.ts | 3 +-
.../specs/ServerServerCapabilities.spec.ts | 3 +-
.../test/integration/specs/Servers.spec.ts | 3 +-
.../integration/specs/ServiceCategories.spec.ts | 3 +-
.../test/integration/specs/Statuses.spec.ts | 3 +-
.../test/integration/specs/Topologies.spec.ts | 6 +-
.../test/integration/specs/Types.spec.ts | 3 +-
23 files changed, 554 insertions(+), 153 deletions(-)
diff --git a/.github/actions/tp-integration-tests/cdn.json b/.github/actions/tp-integration-tests/cdn.json
index d91f8be..8f1d100 100644
--- a/.github/actions/tp-integration-tests/cdn.json
+++ b/.github/actions/tp-integration-tests/cdn.json
@@ -13,11 +13,11 @@
"traffic_ops_golang": {
"insecure": true,
"port": "6443",
- "log_location_error": "error.log",
- "log_location_warning": "warning.log",
- "log_location_info": "info.log",
- "log_location_debug": null,
- "log_location_event": "event.log",
+ "log_location_error": "stderr",
+ "log_location_warning": "stderr",
+ "log_location_info": "stdout",
+ "log_location_debug": "stdout",
+ "log_location_event": "stdout",
"max_db_connections": 20,
"db_conn_max_lifetime_seconds": 60,
"db_query_timeout_seconds": 20,
diff --git a/.github/actions/tp-integration-tests/entrypoint.sh b/.github/actions/tp-integration-tests/entrypoint.sh
index afd3b8f..6e6ca2d 100755
--- a/.github/actions/tp-integration-tests/entrypoint.sh
+++ b/.github/actions/tp-integration-tests/entrypoint.sh
@@ -15,7 +15,35 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-trap 'echo "Error on line ${LINENO} of ${0}"; exit 1' ERR
+
+onFail() {
+ echo "Error on line ${1} of ${2}" >&2;
+ if ! [[ -d Reports ]]; then
+ mkdir Reports;
+ fi
+ if [[ -f tv.log ]]; then
+ cp tv.log Reports/traffic_vault.docker.log;
+ fi
+ docker logs "$trafficvault" > Reports/traffic_vault.log 2>&1;
+ if [[ -f tp.log ]]; then
+ mv tp.log Reports/forever.log
+ fi
+ if [[ -f access.log ]]; then
+ mv access.log Reports/tp-access.log
+ fi
+ if [[ -f out.log ]]; then
+ mv out.log Reports/node.log
+ fi
+ docker logs $CHROMIUM_CONTAINER > Reports/chromium.log 2>&1;
+ docker logs $HUB_CONTAINER > Reports/hub.log 2>&1;
+ if [[ -f "${REPO_DIR}/traffic_ops/traffic_ops_golang" ]]; then
+ cp "${REPO_DIR}/traffic_ops/traffic_ops_golang" Reports/to.log;
+ fi
+ echo "Detailed logs produced info Reports artifact"
+ exit 1
+}
+
+trap 'onFail "${LINENO}" "${0}"' ERR
set -o errexit -o nounset -o pipefail
hub_fqdn="http://localhost:4444/wd/hub/status"
@@ -23,7 +51,7 @@ to_fqdn="https://localhost:6443"
tp_fqdn="https://172.18.0.1:8443"
if ! curl -Lvsk "${hub_fqdn}" >/dev/null 2>&1; then
- echo "Selenium not started on ${hub_fqdn}"
+ echo "Selenium not started on ${hub_fqdn}" >&2;
exit 1
fi
@@ -81,19 +109,6 @@ QUERY
sudo useradd trafops
-gray_bg="$(printf '%s%s' $'\x1B' '[100m')";
-red_bg="$(printf '%s%s' $'\x1B' '[41m')";
-yellow_bg="$(printf '%s%s' $'\x1B' '[43m')";
-black_fg="$(printf '%s%s' $'\x1B' '[30m')";
-color_and_prefix() {
- color="$1";
- shift;
- prefix="$1";
- normal_bg="$(printf '%s%s' $'\x1B' '[49m')";
- normal_fg="$(printf '%s%s' $'\x1B' '[39m')";
- sed "s/^/${color}${black_fg}${prefix}: /" | sed "s/$/${normal_bg}${normal_fg}/";
-}
-
ciab_dir="${GITHUB_WORKSPACE}/infrastructure/cdn-in-a-box";
trafficvault=trafficvault;
start_traffic_vault() {
@@ -127,9 +142,9 @@ start_traffic_vault() {
--publish=8087:8087 \
--rm \
"$trafficvault" \
- /usr/lib/riak/riak-cluster.sh;
+ /usr/lib/riak/riak-cluster.sh
}
-start_traffic_vault &
+start_traffic_vault >tv.log 2>&1 &
sudo apt-get install -y --no-install-recommends gettext \
ruby ruby-dev libc-dev curl \
@@ -154,7 +169,7 @@ if [[ ! -e "$REPO_DIR" ]]; then
fi
to_build() {
- cd "${REPO_DIR}/traffic_ops/traffic_ops_golang"
+ pushd "${REPO_DIR}/traffic_ops/traffic_ops_golang"
if [[ ! -d "${GITHUB_WORKSPACE}/vendor/golang.org" ]]; then
go mod vendor
fi
@@ -167,56 +182,42 @@ to_build() {
export $(<"${ciab_dir}/variables.env" sed '/^#/d') # defines TV_ADMIN_USER/PASSWORD
envsubst <"${resources}/riak.json" >riak.conf
- truncate --size=0 warning.log error.log event.log info.log
+ truncate -s0 out.log
- ./traffic_ops_golang --cfg ./cdn.conf --dbcfg ./database.conf -riakcfg riak.conf &
- tail -f warning.log 2>&1 | color_and_prefix "${yellow_bg}" 'Traffic Ops WARN' &
- tail -f error.log 2>&1 | color_and_prefix "${red_bg}" 'Traffic Ops ERR' &
- tail -f event.log 2>&1 | color_and_prefix "${gray_bg}" 'Traffic Ops EVT' &
+ ./traffic_ops_golang --cfg ./cdn.conf --dbcfg ./database.conf -riakcfg riak.conf >out.log 2>&1 &
+ popd
}
tp_build() {
- cd "${REPO_DIR}/traffic_portal"
+ pushd "${REPO_DIR}/traffic_portal"
npm ci
bower install
grunt dist
cp "${resources}/config.js" ./conf/
touch tp.log access.log out.log err.log
- sudo forever --minUptime 5000 --spinSleepTime 2000 -f -o out.log start server.js &
- tail -f err.log 2>&1 | color_and_prefix "${red_bg}" "Node Err" &
+ sudo forever --minUptime 5000 --spinSleepTime 2000 -f start server.js >out.log 2>&1 &
+ popd
}
-(to_build) &
-(tp_build) &
-
-onFail() {
- docker logs "$trafficvault" > Reports/traffic_vault.log
- mv tp.log Reports/forever.log
- mv access.log Reports/tp-access.log
- mv out.log Reports/node.log
- docker logs $CHROMIUM_CONTAINER > Reports/chromium.log
- docker logs $HUB_CONTAINER > Reports/hub.log
- echo "Detailed logs produced info Reports artifact"
- exit 1
-}
+to_build
+tp_build
cd "${REPO_DIR}/traffic_portal/test/integration"
npm ci
-PATH=$(pwd)/node_modules/.bin/:$PATH
-webdriver-manager update --gecko false --versions.chrome "LATEST_RELEASE_$CHROMIUM_VER"
+./node_modules/.bin/webdriver-manager update --gecko false --versions.chrome "LATEST_RELEASE_$CHROMIUM_VER"
jq " .capabilities.chromeOptions.args = [
\"--headless\",
\"--no-sandbox\",
\"--disable-gpu\",
\"--ignore-certificate-errors\"
- ] | .params.apiUrl = \"${tp_fqdn}/api/4.0\" | .params.baseUrl =\"${tp_fqdn}\"
+ ] | .params.apiUrl = \"${to_fqdn}/api/4.0\" | .params.baseUrl =\"${tp_fqdn}\"
| .capabilities[\"goog:chromeOptions\"].w3c = false | .capabilities.chromeOptions.w3c = false" \
config.json > config.json.tmp && mv config.json.tmp config.json
-tsc
+npm run build
# Wait for tp/to build
timeout 5m bash <<TMOUT
@@ -226,5 +227,4 @@ timeout 5m bash <<TMOUT
done
TMOUT
-trap - ERR
-protractor ./GeneratedCode/config.js --params.baseUrl="${tp_fqdn}" --params.apiUrl="${to_fqdn}/api/4.0" || onFail
+npm test -- --params.baseUrl="${tp_fqdn}" --params.apiUrl="${to_fqdn}/api/4.0"
diff --git a/traffic_portal/test/integration/CommonUtils/API.ts b/traffic_portal/test/integration/CommonUtils/API.ts
index 9b18768..2a626a3 100644
--- a/traffic_portal/test/integration/CommonUtils/API.ts
+++ b/traffic_portal/test/integration/CommonUtils/API.ts
@@ -21,10 +21,12 @@
import { Agent } from "https";
import axios from 'axios';
-import type {AxiosResponse} from "axios";
+import type {AxiosResponse, AxiosError} from "axios";
import randomIpv6 from "random-ipv6";
-import { config, randomize } from '../config';
+import { hasProperty } from "./utils";
+import { randomize } from '../config';
+import { AlertLevel, isAlert, logAlert, TestingConfig } from "../config.model";
interface GetRequest {
queryKey: string;
@@ -50,33 +52,61 @@ export interface APIData {
}
/**
- * hasProperty checks, generically, whether some variable passed as `o` has the
- * property `k`.
+ * Checks if an object is an AxiosError, usually useful in `try`/`catch` blocks
+ * around axios calls.
*
- * @example
- * hasProperty({}, "id"); // returns false
- * hasProperty({id: 8}); // returns true
- * hasProperty({id: undefined}); // returns true
- *
- * @param o The object to check.
- * @param k The key for which to check in the object.
- * @returns Whether or not `o` has the property `k`.
- * @throws {Error} when the type check fails.
+ * @param e The object to check.
+ * @returns Whether or not `e` is an AxiosError.
*/
- export function hasProperty<T extends object, K extends PropertyKey, S>(o: T, k: K): o is T & Record<K, S | unknown> {
- return Object.prototype.hasOwnProperty.call(o, k);
+function isAxiosError(e: unknown): e is AxiosError {
+ if (typeof(e) !== "object" || e === null) {
+ return false;
+ }
+ if (!hasProperty(e, "isAxiosError", "boolean")) {
+ return false;
+ }
+ return e.isAxiosError;
}
export class API {
private cookie = "";
- private readonly config = config;
- constructor() {
+ /**
+ * This controls the alert levels that get logged - levels not in this set
+ * are not logged
+ */
+ private readonly alertLevels = new Set<AlertLevel>(["warning", "error", "info"]);
+ /**
+ * Stores login information for the admin-level user.
+ */
+ private readonly loginInfo: {
+ password: string;
+ username: string;
+ };
+ /**
+ * The URL base used for the Traffic Ops API.
+ *
+ * Trailing `/` is guaranteed.
+ *
+ * @example
+ * "https://localhost:6443/api/4.0/"
+ */
+ private readonly apiURL: string;
+
+ /**
+ * @param cfg The testing configuration.
+ */
+ constructor(cfg: TestingConfig) {
axios.defaults.headers.common['Accept'] = 'application/json'
axios.defaults.headers.common['Authorization'] = 'No-Auth'
axios.defaults.headers.common['Content-Type'] = 'application/json'
axios.defaults.httpsAgent = new Agent({ rejectUnauthorized: false })
+ if (cfg.alertLevels) {
+ this.alertLevels = new Set(cfg.alertLevels);
+ }
+ this.loginInfo = cfg.login;
+ this.apiURL = cfg.apiUrl.endsWith("/") ? cfg.apiUrl : `${cfg.apiUrl}/`;
}
/**
@@ -86,18 +116,90 @@ export class API {
* @throws {Error} when login fails, or when Traffic Ops doesn't return a cookie.
*/
public async Login(): Promise<AxiosResponse<unknown>> {
- const response = await axios({
- method: 'post',
- url: this.config.params.apiUrl + '/user/login',
- data: {
- u: this.config.params.login.username,
- p: this.config.params.login.password
- }
- });
- this.cookie = await response.headers["set-cookie"][0];
+ const data = {
+ p: this.loginInfo.password,
+ u: this.loginInfo.username,
+ }
+ const response = await this.getResponse("post", "/user/login", data);
+ const h = response.headers as object;
+ if (!hasProperty(h, "set-cookie", "Array") || h["set-cookie"].length < 1) {
+ throw new Error("Traffic Ops response did not set a cookie");
+ }
+ const cookie = await h["set-cookie"][0];
+ if (typeof(cookie) !== "string") {
+ throw new Error(`non-string cookie: ${cookie}`);
+ }
+ this.cookie = cookie;
return response
}
+ /**
+ * Retrieves a response from the API.
+ *
+ * Alerts will be logged if they are found - even if an error occurs and is
+ * thrown.
+ *
+ * @param method The request method to use.
+ * @param path The path to request, relative to the configured TO API URL.
+ * @returns The server's response.
+ * @throws {unknown} when the request fails for any reason. If an error
+ * response was returned from the API, it was logged, so there's no need to
+ * dig into the properties of these errors, really.
+ */
+ private async getResponse(method: "get" | "delete", path: string): Promise<AxiosResponse>;
+ /**
+ * Retrieves a response from the API.
+ *
+ * Alerts will be logged if they are found - even if an error occurs and is
+ * thrown.
+ *
+ * @param method The request method to use.
+ * @param path The path to request, relative to the configured TO API URL.
+ * @param data Data to send in the body of the POST request.
+ * @returns The server's response.
+ * @throws {unknown} when the request fails for any reason. If an error
+ * response was returned from the API, it was logged, so there's no need to
+ * dig into the properties of these errors, really.
+ */
+ private async getResponse(method: "post", path: string, data: unknown): Promise<AxiosResponse>;
+ private async getResponse(method: "post" | "get" | "delete", path: string, data?: unknown): Promise<AxiosResponse> {
+ if (method === "post" && data === undefined) {
+ throw new TypeError("request body must be given for POST requests");
+ }
+
+ const url = `${this.apiURL}${path.replace(/^\/+/g, "")}`;
+ const conf = {
+ method,
+ url,
+ headers: { Cookie: this.cookie },
+ data
+ }
+
+ let throwable;
+ let resp: AxiosResponse<unknown>;
+ try {
+ resp = await axios(conf);
+ } catch(e) {
+ if (!isAxiosError(e) || !e.response) {
+ console.debug("non-axios error or axios error with no response thrown");
+ throw e;
+ }
+ resp = e.response;
+ throwable = e;
+ }
+ if (typeof(resp.data) === "object" && resp.data !== null && hasProperty(resp.data, "alerts", "Array")) {
+ for (const a of resp.data.alerts) {
+ if (isAlert(a) && this.alertLevels.has(a.level)) {
+ logAlert(a, `${method.toUpperCase()} ${url} (${resp.status} ${resp.statusText}):`);
+ }
+ }
+ }
+ if (throwable) {
+ throw throwable;
+ }
+ return resp;
+ }
+
public async SendRequest<T extends IDData>(route: string, method: string, data: T): Promise<void> {
let response
this.Randomize(data)
@@ -117,19 +219,10 @@ export class API {
switch (method) {
case "post":
- response = await axios({
- method: method,
- url: this.config.params.apiUrl + route,
- headers: { Cookie: this.cookie},
- data: data
- });
+ response = await this.getResponse("post", route, data);
break;
case "get":
- response = await axios({
- method: method,
- url: this.config.params.apiUrl + route,
- headers: { Cookie: this.cookie},
- });
+ response = await this.getResponse("get", route);
break;
case "delete":
if (!data.route) {
@@ -147,11 +240,7 @@ export class API {
if((data.route).includes('/service_categories/')){
data.route = data.route + randomize
}
- response = await axios({
- method: method,
- url: this.config.params.apiUrl + data.route,
- headers: { Cookie: this.cookie},
- });
+ response = await this.getResponse("delete", data.route);
break;
default:
throw new Error(`unrecognized request method: '${method}'`);
@@ -171,11 +260,7 @@ export class API {
}
for (const request of data.getRequest) {
const query = `?${encodeURIComponent(request.queryKey)}=${encodeURIComponent(request.queryValue)}${randomize}`;
- const response = await axios({
- method: 'get',
- url: this.config.params.apiUrl + request.route + query,
- headers: { Cookie: this.cookie},
- });
+ const response = await this.getResponse("get", request.route + query)
if (response.status == 200) {
if(request.hasOwnProperty('isArray')){
@@ -193,7 +278,7 @@ export class API {
return null
}
- public Randomize(data: object): void {
+ public Randomize(data: object): void {
if (hasProperty(data, "fullName")) {
if (hasProperty(data, "email")) {
data.email = data.fullName + randomize + data.email;
@@ -231,23 +316,19 @@ export class API {
if(hasProperty(data, 'domainName')) {
data.domainName = data.domainName + randomize;
}
- if(hasProperty(data, 'nodes')){
- if (data.nodes instanceof Array) {
- data.nodes.map(i => {
- if (typeof(i) === "object" && i !== null && hasProperty(i, "cachegroup")) {
- i.cachegroup = i.cachegroup + randomize;
- }
- });
- }
+ if(hasProperty(data, 'nodes', "Array")){
+ data.nodes.map(i => {
+ if (typeof(i) === "object" && i !== null && hasProperty(i, "cachegroup")) {
+ i.cachegroup = i.cachegroup + randomize;
+ }
+ });
}
- if(hasProperty(data, 'interfaces')){
- if (data.interfaces instanceof Array) {
- const ipv6 = randomIpv6();
- for (const inf of data.interfaces) {
- if (typeof(inf) === "object" && inf !== null && hasProperty(inf, "ipAddresses") && inf.ipAddresses instanceof Array) {
- for (const ip of inf.ipAddresses) {
- ip.address = ipv6.toString();
- }
+ if(hasProperty(data, 'interfaces', "Array")){
+ const ipv6 = randomIpv6();
+ for (const inf of data.interfaces) {
+ if (typeof(inf) === "object" && inf !== null && hasProperty(inf, "ipAddresses", "Array")) {
+ for (const ip of inf.ipAddresses) {
+ (ip as Record<"address", string>).address = ipv6.toString();
}
}
}
@@ -273,7 +354,7 @@ export class API {
}
}
} else if (response.status == undefined) {
- throw new Error(`Error requesting ${this.config.params.apiUrl}: ${response}`);
+ throw new Error(`Error requesting ${this.apiURL}: ${response}`);
} else {
throw new Error(`Login failed: Response Status: '${response.statusText}'' Response Data: '${response.data}'`);
}
diff --git a/traffic_portal/test/integration/CommonUtils/index.ts b/traffic_portal/test/integration/CommonUtils/index.ts
new file mode 100644
index 0000000..1cd793c
--- /dev/null
+++ b/traffic_portal/test/integration/CommonUtils/index.ts
@@ -0,0 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+export * from "./API";
+export * from "./utils";
diff --git a/traffic_portal/test/integration/CommonUtils/utils.ts b/traffic_portal/test/integration/CommonUtils/utils.ts
new file mode 100644
index 0000000..5b3b6ac
--- /dev/null
+++ b/traffic_portal/test/integration/CommonUtils/utils.ts
@@ -0,0 +1,133 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/**
+ * hasProperty checks whether some variable passed as `o` has the string
+ * property `k`.
+ *
+ * @example
+ * hasProperty({}, "id", "string"); // returns false
+ * hasProperty({id: 8}, "id", "string"); // returns false
+ * hasProperty({id: undefined}, "id", "string"); // returns false
+ *
+ * @param o The object to check.
+ * @param k The key for which to check in the object.
+ * @param type Specifies that `o.k` should be a string.
+ * @returns Whether or not `o` has the string property `k`.
+ */
+export function hasProperty<T extends object, K extends PropertyKey>(o: T, k: K, type: "string"): o is T & Record<K, string>;
+/**
+ * hasProperty checks whether some variable passed as `o` has the number
+ * property `k`.
+ *
+ * @example
+ * hasProperty({}, "id", "number"); // returns false
+ * hasProperty({id: 8}, "id", "number"); // returns true
+ * hasProperty({id: undefined}, "id", "number"); // returns false
+ *
+ * @param o The object to check.
+ * @param k The key for which to check in the object.
+ * @param type Specifies that `o.k` should be a number.
+ * @returns Whether or not `o` has the number property `k`.
+ */
+export function hasProperty<T extends object, K extends PropertyKey>(o: T, k: K, type: "number"): o is T & Record<K, number>;
+/**
+ * hasProperty checks whether some variable passed as `o` has the boolean
+ * property `k`.
+ *
+ * @example
+ * hasProperty({}, "id", "boolean"); // returns false
+ * hasProperty({id: 8}, "id", "boolean"); // returns false
+ * hasProperty({id: undefined}, "id", "boolean"); // returns false
+ * hasProperty({id: true}, "id", "boolean"); // returns true
+ *
+ * @param o The object to check.
+ * @param k The key for which to check in the object.
+ * @param type Specifies that `o.k` should be a boolean.
+ * @returns Whether or not `o` has the boolean property `k`.
+ */
+export function hasProperty<T extends object, K extends PropertyKey>(o: T, k: K, type: "boolean"): o is T & Record<K, boolean>;
+/**
+ * hasProperty checks whether some variable passed as `o` has the Array property
+ * `k`.
+ *
+ * @example
+ * hasProperty({}, "id", "Array"); // returns false
+ * hasProperty({id: 8}, "id", "Array"); // returns false
+ * hasProperty({id: undefined}, "id", "Array"); // returns false
+ * hasProperty({id: []}, "id", "Array"); // returns true
+ * hasProperty({id: [undefined, null, -7, true]}, "id", "Array"); // returns true
+ *
+ * @param o The object to check.
+ * @param k The key for which to check in the object.
+ * @param type Specifies that `o.k` should be a (potentially non-homogenous)
+ * Array.
+ * @returns Whether or not `o` has the Array property `k`.
+ */
+export function hasProperty<T extends object, K extends PropertyKey>(o: T, k: K, type: "Array"): o is T & Record<K, Array<unknown>>;
+/**
+ * hasProperty checks, generically, whether some variable passed as `o` has the
+ * property `k`.
+ *
+ * @example
+ * hasProperty({}, "id"); // returns false
+ * hasProperty({id: 8}, "id"); // returns true
+ * hasProperty({id: undefined}, "id"); // returns true
+ *
+ * @param o The object to check.
+ * @param k The key for which to check in the object.
+ * @returns Whether or not `o` has the property `k`.
+ */
+export function hasProperty<T extends object, K extends PropertyKey>(o: T, k: K): o is T & Record<K, unknown>;
+/**
+ * hasProperty checks, generically, whether some variable passed as `o` has the
+ * property `k`.
+ *
+ * @example
+ * hasProperty({}, "id"); // returns false
+ * hasProperty({id: 8}, "id"); // returns true
+ * hasProperty({id: undefined}, "id"); // returns true
+ * hasProperty({id: 8}, "id", "number"); // returns true
+ * hasProperty({id: undefined}, "id", "number"); // returns false
+ * hasProperty({id: 8}, "id", "string"); // returns false
+ *
+ * @param o The object to check.
+ * @param k The key for which to check in the object.
+ * @param type Optionally specify a type to check for.
+ * @returns Whether or not `o` has the property `k`.
+ */
+export function hasProperty<T extends object, K extends PropertyKey, S>(o: T, k: K, type?: "string" | "number" | "boolean" | "Array"): o is T & Record<K, S> {
+ if (!Object.prototype.hasOwnProperty.call(o, k)) {
+ return false;
+ }
+ if (!type) {
+ return true;
+ }
+ const val = (o as Record<K, unknown>)[k];
+ switch (type) {
+ case "string":
+ return typeof(val) === "string";
+ case "number":
+ return typeof(val) === "number";
+ case "boolean":
+ return typeof(val) === "boolean";
+ case "Array":
+ return val instanceof Array;
+ }
+}
diff --git a/traffic_portal/test/integration/config.model.ts b/traffic_portal/test/integration/config.model.ts
new file mode 100644
index 0000000..9895820
--- /dev/null
+++ b/traffic_portal/test/integration/config.model.ts
@@ -0,0 +1,165 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import { hasProperty } from "./CommonUtils/utils";
+
+/** Possible levels of TO Alerts */
+export type AlertLevel = "success" | "info" | "warning" | "error";
+
+/**
+ * Checks whether an object is a valid Alert level.
+ *
+ * @param s The object to check.
+ * @returns Whether or not `s` is an AlertLevel.
+ */
+export function isAlertLevel(s: unknown): s is AlertLevel {
+ if (typeof(s) !== "string") {
+ return false;
+ }
+ switch (s) {
+ case "success":
+ case "info":
+ case "warning":
+ case "error":
+ return true;
+ }
+ return false;
+}
+
+/** TO API Alerts */
+export interface Alert {
+ level: AlertLevel;
+ text: string;
+}
+
+/**
+ * Checks whether an object is an Alert like the ones TO normally returns.
+ *
+ * @param a The object to check.
+ * @returns Whether or not `a` is an Alert.
+ */
+export function isAlert(a: unknown): a is Alert {
+ if (typeof(a) !== "object" || a === null) {
+ return false;
+ }
+ if (!hasProperty(a, "level") || !hasProperty(a, "text", "string")) {
+ return false;
+ }
+ return isAlertLevel(a.level);
+}
+
+/**
+ * Logs an alert to the appropriate console stream based on its `level`.
+ *
+ * @param a The Alert to log.
+ * @param prefix Optional prefix for the log message
+ */
+export function logAlert(a: Alert, prefix?: string): void {
+ let logfunc;
+ let pre = (prefix ?? "").trimStart();
+ switch (a.level) {
+ case "success":
+ logfunc = console.log;
+ pre = `SUCCESS: ${pre}`;
+ break;
+ case "info":
+ logfunc = console.info
+ pre = `INFO: ${pre}`;
+ break;
+ case "warning":
+ logfunc = console.warn
+ pre = `WARN: ${pre}`;
+ break;
+ case "error":
+ logfunc = console.error
+ pre = `ERROR: ${pre}`;
+ break;
+ }
+ logfunc(pre, a.text);
+}
+
+/** TestingConfig is the type of a testing configuration. */
+export interface TestingConfig {
+ /** This is login information for a user with admin-level permissions. */
+ readonly login: {
+ readonly password: string;
+ readonly username: string;
+ };
+ /** The URL at which the Traffic Ops API can be accessed. */
+ readonly apiUrl: string;
+ /** The URL at which Traffic Portal is served - root path. */
+ readonly baseUrl: string;
+ /** Logging alert levels that are enabled. */
+ readonly alertLevels?: Array<AlertLevel>;
+}
+
+/**
+ * Checks if a passed object is a valid testing configuration.
+ *
+ * @param c The object to check.
+ * @returns `true`, always. When the check fails, it throws an error that
+ * explains why.
+ */
+export function isTestingConfig(c: unknown): c is TestingConfig {
+ if (typeof(c) !== "object") {
+ throw new Error(`testing configuration must be an object, not a '${typeof(c)}'`);
+ }
+ if (c === null) {
+ throw new Error("testing configuration must be an object, not 'null'");
+ }
+
+ if (!hasProperty(c, "login") || typeof(c.login) !== "object" || c.login === null) {
+ throw new Error("missing or invalid 'login' property");
+ }
+ if (!hasProperty(c.login, "password", "string") || !hasProperty(c.login, "username", "string")) {
+ throw new Error("'login' property has missing and/or invalid 'password' and/or 'username' property(ies)");
+ }
+ if (c.login.username === "" || c.login.password === "") {
+ throw new Error("neither 'login.username' nor 'login.password' may be blank");
+ }
+ if (!hasProperty(c, "apiUrl", "string")) {
+ throw new Error("missing or invalid 'apiUrl' property");
+ }
+ try {
+ new URL(c.apiUrl);
+ } catch (e) {
+ throw new Error(`'apiUrl' is not a valid URL: ${c.apiUrl}`);
+ }
+ let baseURL;
+ if (!hasProperty(c, "baseUrl", "string")) {
+ throw new Error("missing or invalid 'baseUrl' property");
+ }
+ try {
+ baseURL = new URL(c.baseUrl);
+ } catch (e) {
+ throw new Error(`'baseUrl' is not a valid URL: ${c.baseUrl}`);
+ }
+ if (baseURL.pathname !== "/") {
+ throw new Error("'baseUrl' must be a root path");
+ }
+ if (!hasProperty(c, "alertLevels")) {
+ return true;
+ }
+ if (!(c.alertLevels instanceof Array)) {
+ throw new Error("'alertLevels' must be an array");
+ }
+ if (!c.alertLevels.every(isAlertLevel)) {
+ throw new Error(`invalid alert levels: ${c.alertLevels}`);
+ }
+ return true;
+}
diff --git a/traffic_portal/test/integration/config.ts b/traffic_portal/test/integration/config.ts
index 96ccc55..388d14b 100644
--- a/traffic_portal/test/integration/config.ts
+++ b/traffic_portal/test/integration/config.ts
@@ -23,9 +23,10 @@ import { Config, browser } from 'protractor';
import { JUnitXmlReporter } from 'jasmine-reporters';
import HtmlReporter from "protractor-beautiful-reporter";
-import { API } from './CommonUtils/API';
+import { API } from './CommonUtils';
import * as conf from "./config.json"
import { prerequisites } from "./Data";
+import { isTestingConfig } from "./config.model";
const downloadsPath = resolve('Downloads');
export const randomize = Math.random().toString(36).substring(3, 7);
@@ -37,6 +38,23 @@ if (config.capabilities) {
} else {
config.capabilities = {chromeOptions: {prefs: {download: {default_directory: downloadsPath}}}};
}
+
+if (!config.params) {
+ throw new Error("no testing parameters provided - cannot proceed");
+}
+
+try {
+ if (!isTestingConfig(config.params)) {
+ throw new Error();
+ }
+} catch (e) {
+ const msg = e instanceof Error ? e.message : String(e);
+ throw new Error(`invalid testing params: ${msg}`);
+}
+
+export const testingConfig = config.params;
+export const api = new API(testingConfig);
+
config.onPrepare = async function () {
await browser.waitForAngularEnabled(true);
await browser.driver.manage().window().maximize();
@@ -66,6 +84,5 @@ config.onPrepare = async function () {
}).getJasmine2Reporter());
}
- const api = new API();
await api.UseAPI(prerequisites);
}
diff --git a/traffic_portal/test/integration/specs/ASNs.spec.ts b/traffic_portal/test/integration/specs/ASNs.spec.ts
index a3761c8..3359931 100644
--- a/traffic_portal/test/integration/specs/ASNs.spec.ts
+++ b/traffic_portal/test/integration/specs/ASNs.spec.ts
@@ -20,11 +20,10 @@ import { browser } from 'protractor';
import { LoginPage } from '../PageObjects/LoginPage.po';
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { ASNsPage } from '../PageObjects/ASNs.po';
import { ASNs } from "../Data";
-const api = new API();
const loginPage = new LoginPage();
const topNavigation = new TopNavigationPage();
const asnsPage = new ASNsPage();
diff --git a/traffic_portal/test/integration/specs/Coordinates.spec.ts b/traffic_portal/test/integration/specs/Coordinates.spec.ts
index cb6c924..a617130 100644
--- a/traffic_portal/test/integration/specs/Coordinates.spec.ts
+++ b/traffic_portal/test/integration/specs/Coordinates.spec.ts
@@ -20,11 +20,10 @@ import { browser } from 'protractor';
import { LoginPage } from '../PageObjects/LoginPage.po'
import { CoordinatesPage } from '../PageObjects/CoordinatesPage.po';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
import { coordinates } from "../Data";
-const api = new API();
const loginPage = new LoginPage();
const topNavigation = new TopNavigationPage();
const coordinatesPage = new CoordinatesPage();
diff --git a/traffic_portal/test/integration/specs/DeliveryServices.spec.ts b/traffic_portal/test/integration/specs/DeliveryServices.spec.ts
index 2ba4a69..7c92742 100644
--- a/traffic_portal/test/integration/specs/DeliveryServices.spec.ts
+++ b/traffic_portal/test/integration/specs/DeliveryServices.spec.ts
@@ -21,10 +21,9 @@ import { browser } from 'protractor';
import { LoginPage } from '../PageObjects/LoginPage.po'
import { DeliveryServicePage } from '../PageObjects/DeliveryServicePage.po';
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { deliveryservices } from "../Data/deliveryservices";
-const api = new API();
const topNavigation = new TopNavigationPage();
const loginPage = new LoginPage();
const deliveryservicesPage = new DeliveryServicePage();
@@ -66,7 +65,7 @@ deliveryservices.tests.forEach(async deliveryservicesData => {
expect(await deliveryservicesPage.AssignServerToDeliveryService(assignserver)).toBe(true);
await deliveryservicesPage.OpenDeliveryServicePage();
}
-
+
)
})
deliveryservicesData.assignrequiredcapabilities.forEach(assignrc => {
diff --git a/traffic_portal/test/integration/specs/Divisions.spec.ts b/traffic_portal/test/integration/specs/Divisions.spec.ts
index 5028c4e..27f5342 100644
--- a/traffic_portal/test/integration/specs/Divisions.spec.ts
+++ b/traffic_portal/test/integration/specs/Divisions.spec.ts
@@ -20,11 +20,11 @@ import { browser } from 'protractor';
import { LoginPage } from '../PageObjects/LoginPage.po';
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { DivisionsPage } from '../PageObjects/Divisions.po';
import { divisions } from "../Data";
-const api = new API();
+
const loginPage = new LoginPage();
const topNavigation = new TopNavigationPage();
const divisionsPage = new DivisionsPage();
diff --git a/traffic_portal/test/integration/specs/Jobs.spec.ts b/traffic_portal/test/integration/specs/Jobs.spec.ts
index ba7b8c1..9a0973a 100644
--- a/traffic_portal/test/integration/specs/Jobs.spec.ts
+++ b/traffic_portal/test/integration/specs/Jobs.spec.ts
@@ -18,13 +18,12 @@
*/
import { browser } from 'protractor';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { LoginPage } from '../PageObjects/LoginPage.po';
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
import { JobsPage } from '../PageObjects/Jobs.po'
import { jobs } from '../Data/jobs';
-const api = new API();
const loginPage = new LoginPage();
const topNavigation = new TopNavigationPage();
const jobsPage = new JobsPage();
diff --git a/traffic_portal/test/integration/specs/Origins.spec.ts b/traffic_portal/test/integration/specs/Origins.spec.ts
index 20ead35..974be82 100644
--- a/traffic_portal/test/integration/specs/Origins.spec.ts
+++ b/traffic_portal/test/integration/specs/Origins.spec.ts
@@ -21,10 +21,9 @@ import { browser } from 'protractor';
import { LoginPage } from '../PageObjects/LoginPage.po'
import { OriginsPage } from '../PageObjects/OriginsPage.po';
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { origins } from "../Data";
-const api = new API();
const loginPage = new LoginPage();
const topNavigation = new TopNavigationPage();
const originsPage = new OriginsPage();
diff --git a/traffic_portal/test/integration/specs/Parameters.spec.ts b/traffic_portal/test/integration/specs/Parameters.spec.ts
index 95b8c59..04c6c6c 100644
--- a/traffic_portal/test/integration/specs/Parameters.spec.ts
+++ b/traffic_portal/test/integration/specs/Parameters.spec.ts
@@ -20,11 +20,10 @@ import { browser } from 'protractor';
import { LoginPage } from '../PageObjects/LoginPage.po'
import { ParametersPage } from '../PageObjects/ParametersPage.po';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
import { parameters } from "../Data";
-const api = new API();
const loginPage = new LoginPage();
const topNavigation = new TopNavigationPage();
const parametersPage = new ParametersPage();
@@ -63,7 +62,7 @@ parameters.tests.forEach(async parametersData => {
expect(await parametersPage.ToggleTableColumn(toggle.Name)).toBe(true);
await parametersPage.OpenParametersPage();
}
-
+
});
})
parametersData.add.forEach(add => {
diff --git a/traffic_portal/test/integration/specs/PhysLocations.spec.ts b/traffic_portal/test/integration/specs/PhysLocations.spec.ts
index 24bed71..c221fb8 100644
--- a/traffic_portal/test/integration/specs/PhysLocations.spec.ts
+++ b/traffic_portal/test/integration/specs/PhysLocations.spec.ts
@@ -20,11 +20,10 @@ import { browser } from 'protractor';
import { LoginPage } from '../PageObjects/LoginPage.po'
import { PhysLocationsPage } from '../PageObjects/PhysLocationsPage.po';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
import { physLocations } from "../Data";
-const api = new API();
const loginPage = new LoginPage();
const topNavigation = new TopNavigationPage();
const physlocationsPage = new PhysLocationsPage();
diff --git a/traffic_portal/test/integration/specs/Profiles.spec.ts b/traffic_portal/test/integration/specs/Profiles.spec.ts
index 5878b85..f415db0 100644
--- a/traffic_portal/test/integration/specs/Profiles.spec.ts
+++ b/traffic_portal/test/integration/specs/Profiles.spec.ts
@@ -21,10 +21,9 @@ import { browser } from 'protractor'
import { LoginPage } from '../PageObjects/LoginPage.po'
import { ProfilesPage } from '../PageObjects/ProfilesPage.po';
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { profiles } from "../Data";
-const api = new API();
const loginPage = new LoginPage();
const topNavigation = new TopNavigationPage();
const profilesPage = new ProfilesPage();
diff --git a/traffic_portal/test/integration/specs/Regions.spec.ts b/traffic_portal/test/integration/specs/Regions.spec.ts
index 4380c7a..82b768e 100644
--- a/traffic_portal/test/integration/specs/Regions.spec.ts
+++ b/traffic_portal/test/integration/specs/Regions.spec.ts
@@ -20,11 +20,10 @@ import { browser } from 'protractor';
import { LoginPage } from '../PageObjects/LoginPage.po';
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { RegionsPage } from '../PageObjects/RegionsPage.po';
import { regions } from "../Data";
-const api = new API();
const loginPage = new LoginPage();
const topNavigation = new TopNavigationPage();
const regionsPage = new RegionsPage();
diff --git a/traffic_portal/test/integration/specs/ServerServerCapabilities.spec.ts b/traffic_portal/test/integration/specs/ServerServerCapabilities.spec.ts
index c3efb2f..d21c6d9 100644
--- a/traffic_portal/test/integration/specs/ServerServerCapabilities.spec.ts
+++ b/traffic_portal/test/integration/specs/ServerServerCapabilities.spec.ts
@@ -22,10 +22,9 @@ import { LoginPage } from '../PageObjects/LoginPage.po'
import { ServerCapabilitiesPage } from '../PageObjects/ServerCapabilitiesPage.po';
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
import { ServersPage } from '../PageObjects/ServersPage.po';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { serverServerCapabilities } from "../Data";
-const api = new API();
const loginPage = new LoginPage();
const topNavigation = new TopNavigationPage();
const serverCapabilitiesPage = new ServerCapabilitiesPage();
diff --git a/traffic_portal/test/integration/specs/Servers.spec.ts b/traffic_portal/test/integration/specs/Servers.spec.ts
index 536a166..867397e 100644
--- a/traffic_portal/test/integration/specs/Servers.spec.ts
+++ b/traffic_portal/test/integration/specs/Servers.spec.ts
@@ -20,11 +20,10 @@ import { browser } from 'protractor';
import { LoginPage } from '../PageObjects/LoginPage.po'
import { ServersPage } from '../PageObjects/ServersPage.po';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
import { servers } from "../Data";
-const api = new API();
const loginPage = new LoginPage();
const topNavigation = new TopNavigationPage();
const serversPage = new ServersPage();
diff --git a/traffic_portal/test/integration/specs/ServiceCategories.spec.ts b/traffic_portal/test/integration/specs/ServiceCategories.spec.ts
index 131658d..fdbcd6b 100644
--- a/traffic_portal/test/integration/specs/ServiceCategories.spec.ts
+++ b/traffic_portal/test/integration/specs/ServiceCategories.spec.ts
@@ -20,11 +20,10 @@ import { browser } from 'protractor';
import { LoginPage } from '../PageObjects/LoginPage.po';
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { ServiceCategoriesPage } from '../PageObjects/ServiceCategories.po';
import { serviceCategories } from "../Data";
-const api = new API();
const loginPage = new LoginPage();
const topNavigation = new TopNavigationPage();
const serviceCategoriesPage = new ServiceCategoriesPage();
diff --git a/traffic_portal/test/integration/specs/Statuses.spec.ts b/traffic_portal/test/integration/specs/Statuses.spec.ts
index bb3ac1b..5f070c3 100644
--- a/traffic_portal/test/integration/specs/Statuses.spec.ts
+++ b/traffic_portal/test/integration/specs/Statuses.spec.ts
@@ -20,11 +20,10 @@ import { browser } from 'protractor';
import { LoginPage } from '../PageObjects/LoginPage.po';
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { StatusesPage } from '../PageObjects/Statuses.po'
import { statuses } from "../Data";
-const api = new API();
const loginPage = new LoginPage();
const topNavigation = new TopNavigationPage();
const statusesPage = new StatusesPage();
diff --git a/traffic_portal/test/integration/specs/Topologies.spec.ts b/traffic_portal/test/integration/specs/Topologies.spec.ts
index b9d9ccb..89678ab 100644
--- a/traffic_portal/test/integration/specs/Topologies.spec.ts
+++ b/traffic_portal/test/integration/specs/Topologies.spec.ts
@@ -20,11 +20,10 @@
import { browser } from 'protractor';
import { LoginPage } from '../PageObjects/LoginPage.po';
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { TopologiesPage } from '../PageObjects/TopologiesPage.po';
import { topologies } from "../Data/topologies";
-const api = new API();
const loginPage = new LoginPage();
const topologiesPage = new TopologiesPage();
const topNavigation = new TopNavigationPage();
@@ -56,7 +55,7 @@ topologies.tests.forEach(async topologiesData =>{
it('can logout', async function(){
expect(await topNavigation.Logout()).toBeTruthy();
})
-
+
})
})
})
@@ -66,4 +65,3 @@ describe('Clean Up API for Topologies Test', () => {
await api.UseAPI(topologies.cleanup);
});
});
-
diff --git a/traffic_portal/test/integration/specs/Types.spec.ts b/traffic_portal/test/integration/specs/Types.spec.ts
index 580635d..b9fb5d6 100644
--- a/traffic_portal/test/integration/specs/Types.spec.ts
+++ b/traffic_portal/test/integration/specs/Types.spec.ts
@@ -20,11 +20,10 @@ import { browser } from 'protractor';
import { LoginPage } from '../PageObjects/LoginPage.po';
import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
-import { API } from '../CommonUtils/API';
+import { api } from "../config";
import { TypesPage } from '../PageObjects/Types.po'
import { types } from "../Data";
-const api = new API();
const loginPage = new LoginPage();
const topNavigation = new TopNavigationPage();
const typesPage = new TypesPage();
[trafficcontrol] 08/12: t3cutil: core dump fix for TOClient SetURL
interface (#6138)
Posted by zr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zrhoffman pushed a commit to branch 6.0.x
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit edf90936cad8ad67a8212d9bb3aead4ad58947c9
Author: Brian Olsen <bn...@gmail.com>
AuthorDate: Wed Aug 25 11:42:01 2021 -0600
t3cutil: core dump fix for TOClient SetURL interface (#6138)
(cherry picked from commit 730af93470db44dbfacaab8a280b3325ce397577)
---
cache-config/t3cutil/toreq/client.go | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/cache-config/t3cutil/toreq/client.go b/cache-config/t3cutil/toreq/client.go
index c2af4e5..b73844c 100644
--- a/cache-config/t3cutil/toreq/client.go
+++ b/cache-config/t3cutil/toreq/client.go
@@ -58,8 +58,9 @@ func (cl *TOClient) URL() string {
func (cl *TOClient) SetURL(newURL string) {
if cl.c == nil {
cl.old.SetURL(newURL)
+ } else {
+ cl.c.URL = newURL
}
- cl.c.URL = newURL
}
// New logs into Traffic Ops, returning the TOClient which contains the logged-in client.
[trafficcontrol] 01/12: Update CHANGELOG.md and SECURITY.md for
6.0.0 release
Posted by zr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zrhoffman pushed a commit to branch 6.0.x
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit ed7670112f9473506a82e6dcee7442f43adc88d6
Author: Zach Hoffman <zr...@apache.org>
AuthorDate: Mon Aug 30 11:03:08 2021 -0600
Update CHANGELOG.md and SECURITY.md for 6.0.0 release
---
CHANGELOG.md | 4 ++--
SECURITY.md | 3 ++-
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index bd9cf83..41251a4 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -3,7 +3,7 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
-## [unreleased]
+## [6.0.0] - 2021-08-30
### Added
- [#4982](https://github.com/apache/trafficcontrol/issues/4982) Added the ability to support queueing updates by server type and profile
- [#5412](https://github.com/apache/trafficcontrol/issues/5412) Added last authenticated time to user API's (`GET /user/current, GET /users, GET /user?id=`) response payload
@@ -728,7 +728,7 @@ will be returned indicating that overlap exists.
### Changed
- Reformatted this CHANGELOG file to the keep-a-changelog format
-[unreleased]: https://github.com/apache/trafficcontrol/compare/RELEASE-5.0.0...HEAD
+[6.0.0]: https://github.com/apache/trafficcontrol/compare/RELEASE-6.0.0...RELEASE-5.0.0
[5.0.0]: https://github.com/apache/trafficcontrol/compare/RELEASE-4.1.0...RELEASE-5.0.0
[4.1.0]: https://github.com/apache/trafficcontrol/compare/RELEASE-4.0.0...RELEASE-4.1.0
[4.0.0]: https://github.com/apache/trafficcontrol/compare/RELEASE-3.0.0...RELEASE-4.0.0
diff --git a/SECURITY.md b/SECURITY.md
index b49304b..e5f0cfa 100644
--- a/SECURITY.md
+++ b/SECURITY.md
@@ -6,9 +6,10 @@
| ------- | ------------------ |
| < 4.0 | :x: |
| 4.0.x | :x: |
-| 4.1.x | :white_check_mark: |
+| 4.1.x | :x|
| 5.0.x | :x: |
| 5.1.x | :white_check_mark: |
+| 6.0.x | :white_check_mark: |
## Reporting a Vulnerability
Contact the [Apache Traffic Control Security Team](mailto:security@trafficcontrol.apache.org)
[trafficcontrol] 02/12: Use tag 6.0.x for the builder images
Posted by zr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zrhoffman pushed a commit to branch 6.0.x
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 201032b2603d4a35c5da4ef20e2d561048f47adf
Author: Zach Hoffman <zr...@apache.org>
AuthorDate: Mon Aug 30 11:03:37 2021 -0600
Use tag 6.0.x for the builder images
---
infrastructure/docker/build/docker-compose-opt.yml | 2 +-
infrastructure/docker/build/docker-compose.yml | 20 ++++++++++----------
2 files changed, 11 insertions(+), 11 deletions(-)
diff --git a/infrastructure/docker/build/docker-compose-opt.yml b/infrastructure/docker/build/docker-compose-opt.yml
index 894dc5d..a286fbd 100644
--- a/infrastructure/docker/build/docker-compose-opt.yml
+++ b/infrastructure/docker/build/docker-compose-opt.yml
@@ -27,7 +27,7 @@ services:
volumes:
- ../../../traffic_server/_tsb:/opt/tsb-ats:z
fakeorigin_build:
- image: apache/traffic_fakeorigin_builder:master
+ image: apache/traffic_fakeorigin_builder:6.0.x
build:
dockerfile: infrastructure/docker/build/Dockerfile-fakeOrigin
context: ../../..
diff --git a/infrastructure/docker/build/docker-compose.yml b/infrastructure/docker/build/docker-compose.yml
index 1a97284..73208aa 100644
--- a/infrastructure/docker/build/docker-compose.yml
+++ b/infrastructure/docker/build/docker-compose.yml
@@ -25,7 +25,7 @@ services:
command: ['-f', '/trafficcontrol/dist/weasel.txt', '/trafficcontrol']
source:
- image: apache/traffic_source_tarballer:master
+ image: apache/traffic_source_tarballer:6.0.x
build:
dockerfile: infrastructure/docker/build/Dockerfile-source
context: ../../..
@@ -35,7 +35,7 @@ services:
- ../../..:/trafficcontrol:z
traffic_monitor_build:
- image: apache/traffic_monitor_builder:master
+ image: apache/traffic_monitor_builder:6.0.x
build:
dockerfile: infrastructure/docker/build/Dockerfile-traffic_monitor
context: ../../..
@@ -45,7 +45,7 @@ services:
- ../../..:/trafficcontrol:z
traffic_ops_build:
- image: apache/traffic_ops_builder:master
+ image: apache/traffic_ops_builder:6.0.x
build:
dockerfile: infrastructure/docker/build/Dockerfile-traffic_ops
context: ../../..
@@ -55,7 +55,7 @@ services:
- ../../..:/trafficcontrol:z
cache-config_build:
- image: apache/trafficcontrol-cache-config_builder:master
+ image: apache/trafficcontrol-cache-config_builder:6.0.x
build:
dockerfile: infrastructure/docker/build/Dockerfile-cache-config
context: ../../..
@@ -65,7 +65,7 @@ services:
- ../../..:/trafficcontrol:z
traffic_portal_build:
- image: apache/traffic_portal_builder:master
+ image: apache/traffic_portal_builder:6.0.x
build:
dockerfile: infrastructure/docker/build/Dockerfile-traffic_portal
context: ../../..
@@ -76,7 +76,7 @@ services:
- ../../../.npm:/root/.npm:z
traffic_router_build:
- image: apache/traffic_router_builder:master
+ image: apache/traffic_router_builder:6.0.x
build:
dockerfile: infrastructure/docker/build/Dockerfile-traffic_router
context: ../../..
@@ -87,7 +87,7 @@ services:
- ../../../.m2:/root/.m2:z
traffic_stats_build:
- image: apache/traffic_stats_builder:master
+ image: apache/traffic_stats_builder:6.0.x
build:
dockerfile: infrastructure/docker/build/Dockerfile-traffic_stats
context: ../../..
@@ -97,7 +97,7 @@ services:
- ../../..:/trafficcontrol:z
grove_build:
- image: apache/traffic_grove_builder:master
+ image: apache/traffic_grove_builder:6.0.x
build:
dockerfile: infrastructure/docker/build/Dockerfile-grove
context: ../../..
@@ -107,7 +107,7 @@ services:
- ../../..:/trafficcontrol:z
grovetccfg_build:
- image: apache/traffic_grovetccfg_builder:master
+ image: apache/traffic_grovetccfg_builder:6.0.x
build:
dockerfile: infrastructure/docker/build/Dockerfile-grovetccfg
context: ../../..
@@ -117,7 +117,7 @@ services:
- ../../..:/trafficcontrol:z
docs:
- image: apache/traffic_docs_builder:master
+ image: apache/traffic_docs_builder:6.0.x
build:
dockerfile: infrastructure/docker/build/Dockerfile-docs
context: ../../..
[trafficcontrol] 06/12: Run goose~schema_migrations trigger after
UPDATE, not INSERT (#6134)
Posted by zr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zrhoffman pushed a commit to branch 6.0.x
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 2e3151b2b43da61a0edd2120c77e262c8b3ffe98
Author: Zach Hoffman <zr...@apache.org>
AuthorDate: Mon Aug 23 16:14:25 2021 -0600
Run goose~schema_migrations trigger after UPDATE, not INSERT (#6134)
(cherry picked from commit 2b7c52eaad01af415d4b8af50a9d864c0cc3574e)
---
traffic_ops/app/db/migrations/00000000000000_init.up.sql | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/traffic_ops/app/db/migrations/00000000000000_init.up.sql b/traffic_ops/app/db/migrations/00000000000000_init.up.sql
index 0628308..08bf05a 100644
--- a/traffic_ops/app/db/migrations/00000000000000_init.up.sql
+++ b/traffic_ops/app/db/migrations/00000000000000_init.up.sql
@@ -33,6 +33,6 @@ BEGIN
END$$;
CREATE TRIGGER fast_forward_schema_migrations_trigger
- AFTER INSERT
+ AFTER UPDATE
ON schema_migrations
EXECUTE PROCEDURE fast_forward_schema_migrations_version();
[trafficcontrol] 05/12: Fix wording for package uninstall (#6131)
Posted by zr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zrhoffman pushed a commit to branch 6.0.x
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 1f210d5ed80efcdf5542b61482e599900c89856b
Author: Evan Zelkowitz <ez...@apache.org>
AuthorDate: Fri Aug 20 17:17:31 2021 -0600
Fix wording for package uninstall (#6131)
(cherry picked from commit 02bc598cfd6185a24fa47abce7aa7b443d3a47aa)
---
cache-config/t3c-apply/torequest/torequest.go | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/cache-config/t3c-apply/torequest/torequest.go b/cache-config/t3c-apply/torequest/torequest.go
index e3b4be9..7763fbb 100644
--- a/cache-config/t3c-apply/torequest/torequest.go
+++ b/cache-config/t3c-apply/torequest/torequest.go
@@ -953,7 +953,7 @@ func (r *TrafficOpsReq) ProcessPackages() error {
// uninstall packages marked for removal
if len(install) > 0 && r.Cfg.InstallPackages {
for jj := range uninstall {
- log.Infof("Uninstalling %s\n", install[jj])
+ log.Infof("Uninstalling %s\n", uninstall[jj])
r, err := util.PackageAction("remove", uninstall[jj])
if err != nil {
return errors.New("Unable to uninstall " + uninstall[jj] + " : " + err.Error())
[trafficcontrol] 10/12: Fix roles hard code for TP integration test
(#6139)
Posted by zr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zrhoffman pushed a commit to branch 6.0.x
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 66971a0bfa8f6891ae2b1b00c93631c6dea59a9d
Author: dpham <ph...@yahoo.com>
AuthorDate: Thu Aug 26 16:16:28 2021 -0600
Fix roles hard code for TP integration test (#6139)
* Fix roles hard code
* Update randomize in API files
* Fix typo ?
* Fix reviewed
* Fix template string
* Fix template string compare
Co-authored-by: Pham, Dandy (Contractor) <Da...@comcast.com>
(cherry picked from commit 6bcca115a4ec3e10a2bbee5313981d5445f70647)
---
traffic_portal/test/integration/CommonUtils/API.ts | 8 +-
.../test/integration/Data/prerequisites.ts | 96 +++++++++++++++++++---
2 files changed, 90 insertions(+), 14 deletions(-)
diff --git a/traffic_portal/test/integration/CommonUtils/API.ts b/traffic_portal/test/integration/CommonUtils/API.ts
index 2a626a3..8e1a8eb 100644
--- a/traffic_portal/test/integration/CommonUtils/API.ts
+++ b/traffic_portal/test/integration/CommonUtils/API.ts
@@ -259,9 +259,13 @@ export class API {
return null;
}
for (const request of data.getRequest) {
- const query = `?${encodeURIComponent(request.queryKey)}=${encodeURIComponent(request.queryValue)}${randomize}`;
+ let query = `?${encodeURIComponent(request.queryKey)}=`;
+ if (request.queryValue === 'admin' || request.queryValue === 'operations' || request.queryValue === 'read-only'){
+ query += encodeURIComponent(request.queryValue);
+ }else{
+ query += encodeURIComponent(request.queryValue+randomize);
+ }
const response = await this.getResponse("get", request.route + query)
-
if (response.status == 200) {
if(request.hasOwnProperty('isArray')){
data[request.replace] = [await response.data.response[0].id];
diff --git a/traffic_portal/test/integration/Data/prerequisites.ts b/traffic_portal/test/integration/Data/prerequisites.ts
index 2d7523c..1afd3da 100644
--- a/traffic_portal/test/integration/Data/prerequisites.ts
+++ b/traffic_portal/test/integration/Data/prerequisites.ts
@@ -69,7 +69,7 @@ export const prerequisites = [
fullName: "TPAdmin",
username: "TPAdmin",
email: "@test.com",
- role: 1,
+ role: 0,
tenantId: 1,
localPasswd: "pa$$word",
confirmLocalPasswd: "pa$$word",
@@ -79,6 +79,12 @@ export const prerequisites = [
queryKey: "name",
queryValue: "tenantSame",
replace: "tenantId"
+ },
+ {
+ route: "/roles",
+ queryKey: "name",
+ queryValue: "admin",
+ replace: "role"
}
]
},
@@ -86,7 +92,7 @@ export const prerequisites = [
fullName: "TPOperator",
username: "TPOperator",
email: "@test.com",
- role: 2,
+ role: 0,
tenantId: 1,
localPasswd: "pa$$word",
confirmLocalPasswd: "pa$$word",
@@ -96,6 +102,12 @@ export const prerequisites = [
queryKey: "name",
queryValue: "tenantSame",
replace: "tenantId"
+ },
+ {
+ route: "/roles",
+ queryKey: "name",
+ queryValue: "operations",
+ replace: "role"
}
]
},
@@ -103,7 +115,7 @@ export const prerequisites = [
fullName: "TPReadOnly",
username: "TPReadOnly",
email: "@test.com",
- role: 3,
+ role: 0,
tenantId: 1,
localPasswd: "pa$$word",
confirmLocalPasswd: "pa$$word",
@@ -113,6 +125,12 @@ export const prerequisites = [
queryKey: "name",
queryValue: "tenantSame",
replace: "tenantId"
+ },
+ {
+ route: "/roles",
+ queryKey: "name",
+ queryValue: "read-only",
+ replace: "role"
}
]
},
@@ -120,7 +138,7 @@ export const prerequisites = [
fullName: "TPAdminDiff",
username: "TPAdminDiff",
email: "@test.com",
- role: 1,
+ role: 0,
tenantId: 1,
localPasswd: "pa$$word",
confirmLocalPasswd: "pa$$word",
@@ -130,6 +148,12 @@ export const prerequisites = [
queryKey: "name",
queryValue: "tenantDifferent",
replace: "tenantId"
+ },
+ {
+ route: "/roles",
+ queryKey: "name",
+ queryValue: "admin",
+ replace: "role"
}
]
},
@@ -137,7 +161,7 @@ export const prerequisites = [
fullName: "TPOperatorDiff",
username: "TPOperatorDiff",
email: "@test.com",
- role: 2,
+ role: 0,
tenantId: 1,
localPasswd: "pa$$word",
confirmLocalPasswd: "pa$$word",
@@ -147,6 +171,12 @@ export const prerequisites = [
queryKey: "name",
queryValue: "tenantDifferent",
replace: "tenantId"
+ },
+ {
+ route: "/roles",
+ queryKey: "name",
+ queryValue: "operations",
+ replace: "role"
}
]
},
@@ -154,7 +184,7 @@ export const prerequisites = [
fullName: "TPReadOnlyDiff",
username: "TPReadOnlyDiff",
email: "@test.com",
- role: 3,
+ role: 0,
tenantId: 1,
localPasswd: "pa$$word",
confirmLocalPasswd: "pa$$word",
@@ -164,6 +194,12 @@ export const prerequisites = [
queryKey: "name",
queryValue: "tenantDifferent",
replace: "tenantId"
+ },
+ {
+ route: "/roles",
+ queryKey: "name",
+ queryValue: "read-only",
+ replace: "role"
}
]
},
@@ -171,7 +207,7 @@ export const prerequisites = [
fullName: "TPAdminParent",
username: "TPAdminParent",
email: "@test.com",
- role: 1,
+ role: 0,
tenantId: 1,
localPasswd: "pa$$word",
confirmLocalPasswd: "pa$$word",
@@ -181,6 +217,12 @@ export const prerequisites = [
queryKey: "name",
queryValue: "tenantParent",
replace: "tenantId"
+ },
+ {
+ route: "/roles",
+ queryKey: "name",
+ queryValue: "admin",
+ replace: "role"
}
]
},
@@ -188,7 +230,7 @@ export const prerequisites = [
fullName: "TPOperatorParent",
username: "TPOperatorParent",
email: "@test.com",
- role: 2,
+ role: 0,
tenantId: 1,
localPasswd: "pa$$word",
confirmLocalPasswd: "pa$$word",
@@ -198,6 +240,12 @@ export const prerequisites = [
queryKey: "name",
queryValue: "tenantParent",
replace: "tenantId"
+ },
+ {
+ route: "/roles",
+ queryKey: "name",
+ queryValue: "operations",
+ replace: "role"
}
]
},
@@ -205,7 +253,7 @@ export const prerequisites = [
fullName: "TPReadOnlyParent",
username: "TPReadOnlyParent",
email: "@test.com",
- role: 3,
+ role: 0,
tenantId: 1,
localPasswd: "pa$$word",
confirmLocalPasswd: "pa$$word",
@@ -215,6 +263,12 @@ export const prerequisites = [
queryKey: "name",
queryValue: "tenantParent",
replace: "tenantId"
+ },
+ {
+ route: "/roles",
+ queryKey: "name",
+ queryValue: "read-only",
+ replace: "role"
}
]
},
@@ -222,7 +276,7 @@ export const prerequisites = [
fullName: "TPAdminChild",
username: "TPAdminChild",
email: "@test.com",
- role: 1,
+ role: 0,
tenantId: 1,
localPasswd: "pa$$word",
confirmLocalPasswd: "pa$$word",
@@ -232,6 +286,12 @@ export const prerequisites = [
queryKey: "name",
queryValue: "tenantChild",
replace: "tenantId"
+ },
+ {
+ route: "/roles",
+ queryKey: "name",
+ queryValue: "admin",
+ replace: "role"
}
]
},
@@ -239,7 +299,7 @@ export const prerequisites = [
fullName: "TPOperatorChild",
username: "TPOperatorChild",
email: "@test.com",
- role: 2,
+ role: 0,
tenantId: 1,
localPasswd: "pa$$word",
confirmLocalPasswd: "pa$$word",
@@ -249,6 +309,12 @@ export const prerequisites = [
queryKey: "name",
queryValue: "tenantChild",
replace: "tenantId"
+ },
+ {
+ route: "/roles",
+ queryKey: "name",
+ queryValue: "operations",
+ replace: "role"
}
]
},
@@ -256,7 +322,7 @@ export const prerequisites = [
fullName: "TPReadOnlyChild",
username: "TPReadOnlyChild",
email: "@test.com",
- role: 3,
+ role: 0,
tenantId: 1,
localPasswd: "pa$$word",
confirmLocalPasswd: "pa$$word",
@@ -266,6 +332,12 @@ export const prerequisites = [
queryKey: "name",
queryValue: "tenantChild",
replace: "tenantId"
+ },
+ {
+ route: "/roles",
+ queryKey: "name",
+ queryValue: "read-only",
+ replace: "role"
}
]
}
[trafficcontrol] 03/12: Fixed Federations IMS (#6122)
Posted by zr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zrhoffman pushed a commit to branch 6.0.x
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 8f5d0bc27527373a4002bba7f44a1b6018025de3
Author: mattjackson220 <33...@users.noreply.github.com>
AuthorDate: Fri Aug 20 09:19:53 2021 -0600
Fixed Federations IMS (#6122)
* Fixed Federations IMS
* updates per comments
(cherry picked from commit bfd9f91241c6024233d3af1db22163ce7e085be7)
---
CHANGELOG.md | 1 +
traffic_ops/testing/api/v4/cdnfederations_test.go | 33 +++++++++++
traffic_ops/testing/api/v4/federations_test.go | 21 ++++++-
.../cdnfederation/cdnfederations.go | 4 ++
.../federations/allfederations.go | 54 ++++++++---------
.../traffic_ops_golang/federations/federations.go | 67 ++++++++--------------
6 files changed, 109 insertions(+), 71 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 41251a4..010ddd9 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -100,6 +100,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
- [#6091](https://github.com/apache/trafficcontrol/issues/6091) - Fixed cache config of internal cache communication for https origins
- [#6066](https://github.com/apache/trafficcontrol/issues/6066) - Fixed missing/incorrect indices on some tables
- [#5576](https://github.com/apache/trafficcontrol/issues/5576) - Inconsistent Profile Name restrictions
+- Fixed Federations IMS so TR federations watcher will get updates.
### Changed
- Updated the Traffic Ops Python client to 3.0
diff --git a/traffic_ops/testing/api/v4/cdnfederations_test.go b/traffic_ops/testing/api/v4/cdnfederations_test.go
index 77fc4b7..f66efeb 100644
--- a/traffic_ops/testing/api/v4/cdnfederations_test.go
+++ b/traffic_ops/testing/api/v4/cdnfederations_test.go
@@ -35,6 +35,7 @@ func TestCDNFederations(t *testing.T) {
SortTestCDNFederations(t)
UpdateTestCDNFederations(t)
GetTestCDNFederations(t)
+ GetTestCDNFederationsIMS(t)
currentTime := time.Now().UTC().Add(-5 * time.Second)
time := currentTime.Format(time.RFC1123)
var header http.Header
@@ -196,6 +197,38 @@ func GetTestCDNFederations(t *testing.T) {
}
}
+func GetTestCDNFederationsIMS(t *testing.T) {
+ futureTime := time.Now().AddDate(0, 0, 1)
+ fmtFutureTime := futureTime.Format(time.RFC1123)
+ opts := client.NewRequestOptions()
+ opts.Header.Set(rfc.IfModifiedSince, fmtFutureTime)
+ for _, id := range fedIDs {
+ opts.QueryParameters.Set("id", strconv.Itoa(id))
+ data, reqInf, err := TOSession.GetCDNFederationsByName("foo", opts)
+ if err != nil {
+ t.Errorf("could not get federations: %v - alerts: %+v", err, data.Alerts)
+ }
+ if reqInf.StatusCode != http.StatusNotModified {
+ t.Fatalf("Expected 304 status code, got %v", reqInf.StatusCode)
+ }
+ }
+
+ pastTime := time.Now().AddDate(0, 0, -1)
+ fmtPastTime := pastTime.Format(time.RFC1123)
+ opts = client.NewRequestOptions()
+ opts.Header.Set(rfc.IfModifiedSince, fmtPastTime)
+ for _, id := range fedIDs {
+ opts.QueryParameters.Set("id", strconv.Itoa(id))
+ data, reqInf, err := TOSession.GetCDNFederationsByName("foo", opts)
+ if err != nil {
+ t.Errorf("could not get federations: %v - alerts: %+v", err, data.Alerts)
+ }
+ if reqInf.StatusCode != http.StatusOK {
+ t.Fatalf("Expected 200 status code, got %v", reqInf.StatusCode)
+ }
+ }
+}
+
func AssignTestFederationFederationResolvers(t *testing.T) {
// Setup
if len(fedIDs) < 2 {
diff --git a/traffic_ops/testing/api/v4/federations_test.go b/traffic_ops/testing/api/v4/federations_test.go
index c1958c6..2e60f7e 100644
--- a/traffic_ops/testing/api/v4/federations_test.go
+++ b/traffic_ops/testing/api/v4/federations_test.go
@@ -42,10 +42,10 @@ func TestFederations(t *testing.T) {
func GetTestFederationsIMS(t *testing.T) {
futureTime := time.Now().AddDate(0, 0, 1)
- time := futureTime.Format(time.RFC1123)
+ fmtFutureTime := futureTime.Format(time.RFC1123)
opts := client.NewRequestOptions()
- opts.Header.Set(rfc.IfModifiedSince, time)
+ opts.Header.Set(rfc.IfModifiedSince, fmtFutureTime)
if len(testData.Federations) == 0 {
t.Error("no federations test data")
}
@@ -57,6 +57,23 @@ func GetTestFederationsIMS(t *testing.T) {
if reqInf.StatusCode != http.StatusNotModified {
t.Fatalf("Expected 304 status code, got %v", reqInf.StatusCode)
}
+
+ pastTime := time.Now().AddDate(0, 0, -1)
+ fmtPastTime := pastTime.Format(time.RFC1123)
+
+ opts = client.NewRequestOptions()
+ opts.Header.Set(rfc.IfModifiedSince, fmtPastTime)
+ if len(testData.Federations) == 0 {
+ t.Error("no federations test data")
+ }
+
+ resp, reqInf, err = TOSession.AllFederations(opts)
+ if err != nil {
+ t.Fatalf("No error expected, but got: %v - alerts: %+v", err, resp.Alerts)
+ }
+ if reqInf.StatusCode != http.StatusOK {
+ t.Fatalf("Expected 200 status code, got %v", reqInf.StatusCode)
+ }
}
func GetTestFederations(t *testing.T) {
diff --git a/traffic_ops/traffic_ops_golang/cdnfederation/cdnfederations.go b/traffic_ops/traffic_ops_golang/cdnfederation/cdnfederations.go
index e34aec9..2607bcf 100644
--- a/traffic_ops/traffic_ops_golang/cdnfederation/cdnfederations.go
+++ b/traffic_ops/traffic_ops_golang/cdnfederation/cdnfederations.go
@@ -182,6 +182,10 @@ func (fed *TOCDNFederation) Read(h http.Header, useIMS bool) ([]interface{}, err
return nil, userErr, sysErr, errCode, nil
}
+ if errCode == http.StatusNotModified {
+ return []interface{}{}, nil, nil, http.StatusNotModified, maxTime
+ }
+
filteredFederations := []interface{}{}
for _, ifederation := range federations {
federation := ifederation.(*TOCDNFederation)
diff --git a/traffic_ops/traffic_ops_golang/federations/allfederations.go b/traffic_ops/traffic_ops_golang/federations/allfederations.go
index 20cfc08..035fa73 100644
--- a/traffic_ops/traffic_ops_golang/federations/allfederations.go
+++ b/traffic_ops/traffic_ops_golang/federations/allfederations.go
@@ -89,17 +89,7 @@ func GetAll(w http.ResponseWriter, r *http.Request) {
}
}
- fedsResolvers, err, code, maxTime := getFederationResolvers(inf.Tx.Tx, fedInfoIDs(feds), useIMS, r.Header)
- if code == http.StatusNotModified {
- if maxTime != nil && api.SetLastModifiedHeader(r, useIMS) {
- // RFC1123
- date := maxTime.Format("Mon, 02 Jan 2006 15:04:05 MST")
- w.Header().Add(rfc.LastModified, date)
- }
- w.WriteHeader(code)
- api.WriteResp(w, r, []tc.IAllFederation{})
- return
- }
+ fedsResolvers, err := getFederationResolvers(inf.Tx.Tx, fedInfoIDs(feds))
if err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("federations.Get getting federations resolvers: "+err.Error()))
return
@@ -125,13 +115,19 @@ FROM
ORDER BY
ds.xml_id
`
- imsQuery := `SELECT max(t) from (
- SELECT max(fds.last_updated) as t FROM
- federation_deliveryservice fds
- JOIN deliveryservice ds ON ds.id = fds.deliveryservice
- JOIN federation fd ON fd.id = fds.federation
- UNION ALL
- select max(last_updated) as t from last_deleted l where l.table_name='federation_deliveryservice') as res`
+ imsQuery := `SELECT Max(last_updated)
+ FROM (SELECT last_updated
+ FROM federation_deliveryservice fds
+ UNION ALL
+ SELECT last_updated
+ FROM federation_federation_resolver ffr
+ UNION ALL
+ SELECT last_updated
+ FROM federation fd
+ UNION ALL
+ SELECT Max(last_updated) AS t
+ FROM last_deleted l
+ WHERE l.table_name IN ( 'federation_deliveryservice', 'federation', 'federation_federation_resolver' )) AS res;`
if useIMS {
runSecond, maxTime = tryIfModifiedSinceQuery(header, tx, "", imsQuery)
@@ -181,14 +177,20 @@ ORDER BY
ds.xml_id
`
- imsQuery := `SELECT max(t) from (
- SELECT max(fds.last_updated) as t from federation_deliveryservice fds
- JOIN deliveryservice ds ON ds.id = fds.deliveryservice
- JOIN federation fd ON fd.id = fds.federation
- JOIN cdn on cdn.id = ds.cdn_id
- WHERE cdn.name = $1
- UNION ALL
- select max(last_updated) as t from last_deleted l where l.table_name='federation_deliveryservice') as res`
+ // TODO improve query to be CDN-specific
+ imsQuery := `SELECT Max(last_updated)
+ FROM (SELECT last_updated
+ FROM federation_deliveryservice fds
+ UNION ALL
+ SELECT last_updated
+ FROM federation_federation_resolver ffr
+ UNION ALL
+ SELECT last_updated
+ FROM federation fd
+ UNION ALL
+ SELECT Max(last_updated) AS t
+ FROM last_deleted l
+ WHERE l.table_name IN ( 'federation_deliveryservice', 'federation', 'federation_federation_resolver' )) AS res;`
if useIMS {
runSecond, maxTime = tryIfModifiedSinceQuery(header, tx, string(cdn), imsQuery)
diff --git a/traffic_ops/traffic_ops_golang/federations/federations.go b/traffic_ops/traffic_ops_golang/federations/federations.go
index 7d462c4..02acc73 100644
--- a/traffic_ops/traffic_ops_golang/federations/federations.go
+++ b/traffic_ops/traffic_ops_golang/federations/federations.go
@@ -103,12 +103,7 @@ func Get(w http.ResponseWriter, r *http.Request) {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("federations.Get getting federations: "+err.Error()))
return
}
- fedsResolvers, err, code, maxTime := getFederationResolvers(inf.Tx.Tx, fedInfoIDs(feds), useIMS, r.Header)
- if code == http.StatusNotModified {
- w.WriteHeader(code)
- api.WriteResp(w, r, []tc.IAllFederation{})
- return
- }
+ fedsResolvers, err := getFederationResolvers(inf.Tx.Tx, fedInfoIDs(feds))
if err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("federations.Get getting federations resolvers: "+err.Error()))
return
@@ -166,9 +161,7 @@ type FedResolverInfo struct {
}
// getFederationResolvers takes a slice of federation IDs, and returns a map[federationID]info.
-func getFederationResolvers(tx *sql.Tx, fedIDs []int, useIMS bool, header http.Header) (map[int][]FedResolverInfo, error, int, *time.Time) {
- var runSecond bool
- var maxTime time.Time
+func getFederationResolvers(tx *sql.Tx, fedIDs []int) (map[int][]FedResolverInfo, error) {
feds := map[int][]FedResolverInfo{}
qry := `
SELECT
@@ -182,29 +175,9 @@ FROM
WHERE
ffr.federation = ANY($1)
`
- imsQuery := `SELECT max(t) from (
- SELECT max(ffr.last_updated) as t FROM
- federation_federation_resolver ffr
- JOIN federation_resolver fr ON ffr.federation_resolver = fr.id
- JOIN type frt on fr.type = frt.id
-WHERE ffr.federation = ANY($1)
- UNION ALL
- select max(last_updated) as t from last_deleted l where l.table_name='federation_federation_resolver') as res`
-
- if useIMS {
- runSecond, maxTime = tryIfModifiedSinceQueryFederations(header, tx, pq.Array(fedIDs), imsQuery)
- if !runSecond {
- log.Debugln("IMS HIT")
- return feds, nil, http.StatusNotModified, &maxTime
- }
- log.Debugln("IMS MISS")
- } else {
- log.Debugln("Non IMS request")
- }
-
rows, err := tx.Query(qry, pq.Array(fedIDs))
if err != nil {
- return nil, errors.New("all federations resolvers querying: " + err.Error()), http.StatusInternalServerError, nil
+ return nil, errors.New("all federations resolvers querying: " + err.Error())
}
defer rows.Close()
@@ -213,12 +186,12 @@ WHERE ffr.federation = ANY($1)
f := FedResolverInfo{}
fType := ""
if err := rows.Scan(&fedID, &fType, &f.IP); err != nil {
- return nil, errors.New("all federations resolvers scanning: " + err.Error()), http.StatusInternalServerError, nil
+ return nil, errors.New("all federations resolvers scanning: " + err.Error())
}
f.Type = tc.FederationResolverTypeFromString(fType)
feds[fedID] = append(feds[fedID], f)
}
- return feds, nil, http.StatusOK, &maxTime
+ return feds, nil
}
func tryIfModifiedSinceQueryFederations(header http.Header, tx *sql.Tx, fedID interface{}, query string) (bool, time.Time) {
@@ -285,17 +258,25 @@ WHERE
ORDER BY
ds.xml_id
`
- imsQuery := `SELECT max(t) from (
- SELECT max(fds.last_updated) as t FROM
- federation_deliveryservice fds
- JOIN deliveryservice ds ON ds.id = fds.deliveryservice
- JOIN federation fd ON fd.id = fds.federation
- JOIN federation_tmuser fu on fu.federation = fd.id
- JOIN tm_user u on u.id = fu.tm_user
-WHERE
- u.username = $1
- UNION ALL
- select max(last_updated) as t from last_deleted l where l.table_name='federation_deliveryservice') as res`
+ imsQuery := `SELECT Max(t)
+ FROM ((SELECT Greatest(fdsmax, ffrmax, fdmax) AS t
+ FROM (SELECT Max(fds.last_updated) AS fdsmax,
+ Max(ffr.last_updated) AS ffrmax,
+ Max(fd.last_updated) AS fdmax
+ FROM federation_deliveryservice fds
+ JOIN federation_federation_resolver ffr
+ ON ffr.federation = fds.federation
+ JOIN federation fd
+ ON fd.id = fds.federation
+ JOIN federation_tmuser fu
+ ON fu.federation = fd.id
+ JOIN tm_user u
+ ON u.id = fu.tm_user
+ WHERE u.username = $1) AS t
+ UNION ALL
+ SELECT Max(last_updated) AS t
+ FROM last_deleted l
+ WHERE l.table_name IN ( 'federation_deliveryservice', 'federation', 'federation_federation_resolver' ))) AS res;`
if useIMS {
runSecond, maxTime = tryIfModifiedSinceQuery(header, tx, userName, imsQuery)
[trafficcontrol] 07/12: Skip check for the past dbversion's
migration existing if it is squashed (#6128)
Posted by zr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zrhoffman pushed a commit to branch 6.0.x
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 1e6a3c76a487a20e0e9d5226b03e612aa9071d8b
Author: Zach Hoffman <zr...@apache.org>
AuthorDate: Mon Aug 23 16:54:37 2021 -0600
Skip check for the past dbversion's migration existing if it is squashed (#6128)
* Make ConnectionString a global
* Get the migration version after migrating from Goose
* Skip check for the past dbversion's migration existing if it is squashed
* Install TO dependencies before installing TO in db-admin dockerfile
* TO DB tests bugfix: Specify the database name with `dropdb`
* Only fail on a non-latest migration being modified/added if that migration was changed in the PR or commit the TO DB tests GHA is running on
(cherry picked from commit 9fa6c299271ef06075eec711ff1ff6805093f92b)
---
.github/actions/todb-tests/entrypoint.sh | 11 ++++-
traffic_ops/app/db/admin.go | 59 ++++++++++++++++++++------
traffic_ops_db/test/docker/Dockerfile-db-admin | 38 ++++++++++++-----
traffic_ops_db/test/docker/run-db-test.sh | 2 +-
4 files changed, 84 insertions(+), 26 deletions(-)
diff --git a/.github/actions/todb-tests/entrypoint.sh b/.github/actions/todb-tests/entrypoint.sh
index 793c2c0..80702eb 100755
--- a/.github/actions/todb-tests/entrypoint.sh
+++ b/.github/actions/todb-tests/entrypoint.sh
@@ -18,6 +18,15 @@
set -e
+files_changed_in_pr() {
+ if [[ "$GITHUB_REF" == refs/pull/*/merge ]]; then
+ pr_number="$(<<<"$GITHUB_REF" grep -o '[0-9]\+')"
+ files_changed="$(curl "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${pr_number}/files" | jq -r .[].filename)"
+ else
+ files_changed="$(git diff-tree --no-commit-id --name-only -r "$GITHUB_SHA")"
+ fi
+}
+
CODE=0;
migration_dirs=(traffic_ops/app/db/migrations traffic_ops/app/db/trafficvault/migrations);
@@ -69,7 +78,7 @@ for migration_dir in ${migration_dirs[@]}; do
done
mtime_length=${#mtime_array[@]}
- if [[ $LATEST_FILE_TIME != ${mtime_array[$mtime_length-1]} ]]; then
+ if [[ $LATEST_FILE_TIME != ${mtime_array[$mtime_length-1]} ]] && <<<"$(files_changed_in_pr)" grep -q "^${LATEST_FILE}$"; then
echo "ERROR: latest added/modified file: $LATEST_FILE is not in the right order" >&2;
CODE=1;
fi
diff --git a/traffic_ops/app/db/admin.go b/traffic_ops/app/db/admin.go
index ce2a883..7f62287 100644
--- a/traffic_ops/app/db/admin.go
+++ b/traffic_ops/app/db/admin.go
@@ -34,7 +34,9 @@ import (
"github.com/apache/trafficcontrol/lib/go-log"
"github.com/golang-migrate/migrate/v4"
+ "github.com/golang-migrate/migrate/v4/database"
_ "github.com/golang-migrate/migrate/v4/database/postgres"
+ "github.com/golang-migrate/migrate/v4/source"
_ "github.com/golang-migrate/migrate/v4/source/file"
"gopkg.in/yaml.v2"
)
@@ -129,16 +131,17 @@ var (
DBVersionDirty bool
// globals that are parsed out of DBConfigFile and used in commands
- DBDriver string
- DBName string
- DBSuperUser = DefaultDBSuperUser
- DBUser string
- DBPassword string
- HostIP string
- HostPort string
- SSLMode string
- Migrate *migrate.Migrate
- MigrationName string
+ ConnectionString string
+ DBDriver string
+ DBName string
+ DBSuperUser = DefaultDBSuperUser
+ DBUser string
+ DBPassword string
+ HostIP string
+ HostPort string
+ SSLMode string
+ Migrate *migrate.Migrate
+ MigrationName string
)
func parseDBConfig() error {
@@ -330,13 +333,41 @@ func maybeMigrateFromGoose() bool {
if err := Migrate.Steps(1); err != nil {
die("Error migrating to Migrate from Goose: " + err.Error())
}
+ DBVersion, DBVersionDirty, _ = Migrate.Version()
return true
}
+// runFirstMigration is essentially Migrate.Migrate(FirstMigrationTimestamp) but without the obligatory Migrate.versionExists() call.
+// If calling Migrate.versionExists() is made optional, runFirstMigration() can be replaced.
+func runFirstMigration() error {
+ sourceDriver, sourceDriverErr := source.Open(DBMigrationsSource)
+ if sourceDriverErr != nil {
+ return fmt.Errorf("opening the migration source driver: " + sourceDriverErr.Error())
+ }
+ dbDriver, dbDriverErr := database.Open(ConnectionString)
+ if dbDriverErr != nil {
+ return fmt.Errorf("opening the dbdriver: " + dbDriverErr.Error())
+ }
+ firstMigration, firstMigrationName, migrationReadErr := sourceDriver.ReadUp(FirstMigrationTimestamp)
+ if migrationReadErr != nil {
+ return fmt.Errorf("reading migration %s: %s", firstMigrationName, migrationReadErr.Error())
+ }
+ if setDirtyVersionErr := dbDriver.SetVersion(int(FirstMigrationTimestamp), true); setDirtyVersionErr != nil {
+ return fmt.Errorf("setting the dirty version: %s", setDirtyVersionErr.Error())
+ }
+ if migrateErr := dbDriver.Run(firstMigration); migrateErr != nil {
+ return fmt.Errorf("running the migration: %s", migrateErr.Error())
+ }
+ if setVersionErr := dbDriver.SetVersion(int(FirstMigrationTimestamp), false); setVersionErr != nil {
+ return fmt.Errorf("setting the version after successfully running the migration: %s", setVersionErr.Error())
+ }
+ return nil
+}
+
func runMigrations() {
migratedFromGoose := initMigrate()
if !TrafficVault && DBVersion == LastSquashedMigrationTimestamp && !DBVersionDirty {
- if migrateErr := Migrate.Migrate(FirstMigrationTimestamp); migrateErr != nil {
+ if migrateErr := runFirstMigration(); migrateErr != nil {
die(fmt.Sprintf("Error migrating from DB version %d to %d: %s", LastSquashedMigrationTimestamp, FirstMigrationTimestamp, migrateErr.Error()))
}
}
@@ -569,11 +600,11 @@ func main() {
func initMigrate() bool {
var err error
- connectionString := fmt.Sprintf("%s://%s:%s@%s:%s/%s?sslmode=%s", DBDriver, DBUser, DBPassword, HostIP, HostPort, DBName, SSLMode)
+ ConnectionString = fmt.Sprintf("%s://%s:%s@%s:%s/%s?sslmode=%s", DBDriver, DBUser, DBPassword, HostIP, HostPort, DBName, SSLMode)
if TrafficVault {
- Migrate, err = migrate.New(TrafficVaultMigrationsSource, connectionString)
+ Migrate, err = migrate.New(TrafficVaultMigrationsSource, ConnectionString)
} else {
- Migrate, err = migrate.New(DBMigrationsSource, connectionString)
+ Migrate, err = migrate.New(DBMigrationsSource, ConnectionString)
}
if err != nil {
die("Starting Migrate: " + err.Error())
diff --git a/traffic_ops_db/test/docker/Dockerfile-db-admin b/traffic_ops_db/test/docker/Dockerfile-db-admin
index 6f60481..03745c9 100644
--- a/traffic_ops_db/test/docker/Dockerfile-db-admin
+++ b/traffic_ops_db/test/docker/Dockerfile-db-admin
@@ -23,25 +23,43 @@ FROM centos:7
ARG POSTGRES_VERSION=13.2
ENV POSTGRES_VERSION=$POSTGRES_VERSION
-# NOTE: temporary workaround for removal of golang packages from CentOS 7 base repo
RUN yum install -y \
- epel-release \
- centos-release-scl-rh \
- https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm \
+ epel-release \
+ centos-release-scl-rh \
+ https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm \
git && \
- yum -y install golang
+ yum -y install \
+ cpanminus \
+ expat-devel \
+ libcap \
+ libcurl-devel \
+ libidn-devel \
+ libpcap-devel \
+ mkisofs \
+ openssl-devel \
+ perl-core \
+ perl-Crypt-ScryptKDF \
+ perl-DBD-Pg \
+ perl-DBI \
+ perl-Digest-SHA1 \
+ perl-JSON \
+ perl-libwww-perl \
+ perl-TermReadKey \
+ perl-Test-CPAN-Meta \
+ perl-WWW-Curl \
+ postgresql13 \
+ postgresql13-devel &&\
+ yum clean all
# Override TRAFFIC_OPS_RPM arg to use a different one using --build-arg TRAFFIC_OPS_RPM=... Can be local file or http://...
ARG TRAFFIC_OPS_RPM=traffic_ops.rpm
ADD $TRAFFIC_OPS_RPM /
-RUN yum install -y \
- /$(basename $TRAFFIC_OPS_RPM) && \
- rm /$(basename $TRAFFIC_OPS_RPM) && \
- yum clean all
+RUN rpm -Uvh $(basename $TRAFFIC_OPS_RPM) && \
+ rm $(basename $TRAFFIC_OPS_RPM)
WORKDIR /opt/traffic_ops/app
-ADD run-db-test.sh \
+COPY run-db-test.sh \
db-config.sh \
/
diff --git a/traffic_ops_db/test/docker/run-db-test.sh b/traffic_ops_db/test/docker/run-db-test.sh
index ff476d6..3bbcb59 100755
--- a/traffic_ops_db/test/docker/run-db-test.sh
+++ b/traffic_ops_db/test/docker/run-db-test.sh
@@ -132,7 +132,7 @@ fi
# test full restoration of the initial DB dump
for d in $(get_db_dumps); do
echo "testing restoration of DB dump: $d"
- dropdb --echo --if-exists < "$d" > /dev/null || echo "Dropping DB ${DB_NAME} failed: $d"
+ dropdb --echo --if-exists "$DB_NAME" < "$d" > /dev/null || echo "Dropping DB ${DB_NAME} failed: $d"
createdb --echo < "$d" > /dev/null || echo "Creating DB ${DB_NAME} failed: $d"
pg_restore --verbose --clean --if-exists --exit-on-error -d "$DB_NAME" < "$d" > /dev/null || { echo "DB restoration failed: $d"; exit 1; }
done
[trafficcontrol] 11/12: Use PostgreSQL 9.6-compatible syntax in
create_tables.sql and migrations (#6143)
Posted by zr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zrhoffman pushed a commit to branch 6.0.x
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 77a79b55022ba1342573983c834d0e907ad5532c
Author: Zach Hoffman <zr...@apache.org>
AuthorDate: Thu Aug 26 17:31:35 2021 -0600
Use PostgreSQL 9.6-compatible syntax in create_tables.sql and migrations (#6143)
* Use PostgreSQL 9.6-compatible syntax in create_tables.sql and migrations
* Reference NEW and OLD in trigger functions
* Bugfix: Drop correct trigger name
* Add parentheses for PostgreSQL 9.6 compatibility
* Trigger FOR EACH ROW for NEW/OLD access
(cherry picked from commit 328250d0ab7672892e6e36e7b321086b8d811550)
---
traffic_ops/app/db/migrations/00000000000000_init.up.sql | 2 +-
.../db/migrations/2021061100000000_ds_tls_versions.down.sql | 6 +++---
.../app/db/migrations/2021061100000000_ds_tls_versions.up.sql | 10 ++++------
traffic_ops/app/db/trafficvault/create_tables.sql | 10 ++++------
4 files changed, 12 insertions(+), 16 deletions(-)
diff --git a/traffic_ops/app/db/migrations/00000000000000_init.up.sql b/traffic_ops/app/db/migrations/00000000000000_init.up.sql
index 08bf05a..7e8a022 100644
--- a/traffic_ops/app/db/migrations/00000000000000_init.up.sql
+++ b/traffic_ops/app/db/migrations/00000000000000_init.up.sql
@@ -28,7 +28,7 @@ BEGIN
RENAME TO goose_db_version_unused;
END IF;
DROP TRIGGER fast_forward_schema_migrations_trigger ON schema_migrations;
- DROP FUNCTION fast_forward_schema_migrations_version;
+ DROP FUNCTION fast_forward_schema_migrations_version();
RETURN NULL;
END$$;
diff --git a/traffic_ops/app/db/migrations/2021061100000000_ds_tls_versions.down.sql b/traffic_ops/app/db/migrations/2021061100000000_ds_tls_versions.down.sql
index a60944b..46ce8c5 100644
--- a/traffic_ops/app/db/migrations/2021061100000000_ds_tls_versions.down.sql
+++ b/traffic_ops/app/db/migrations/2021061100000000_ds_tls_versions.down.sql
@@ -23,8 +23,8 @@ SET
WHERE
original IS NOT NULL;
-DROP TRIGGER IF EXISTS update_ds_timestamp_on_tls_version_insertion_or_update ON public.deliveryservice_tls_version;
+DROP TRIGGER IF EXISTS update_ds_timestamp_on_tls_version_insertion ON public.deliveryservice_tls_version;
DROP TRIGGER IF EXISTS update_ds_timestamp_on_tls_version_delete ON public.deliveryservice_tls_version;
DROP TABLE IF EXISTS public.deliveryservice_tls_version;
-DROP FUNCTION IF EXISTS update_ds_timestamp_on_insert;
-DROP FUNCTION IF EXISTS update_ds_timestamp_on_delete;
+DROP FUNCTION IF EXISTS update_ds_timestamp_on_insert();
+DROP FUNCTION IF EXISTS update_ds_timestamp_on_delete();
diff --git a/traffic_ops/app/db/migrations/2021061100000000_ds_tls_versions.up.sql b/traffic_ops/app/db/migrations/2021061100000000_ds_tls_versions.up.sql
index 9382610..d49c6ba 100644
--- a/traffic_ops/app/db/migrations/2021061100000000_ds_tls_versions.up.sql
+++ b/traffic_ops/app/db/migrations/2021061100000000_ds_tls_versions.up.sql
@@ -25,7 +25,7 @@ BEGIN
SET last_updated=now()
WHERE id IN (
SELECT deliveryservice
- FROM new_table
+ FROM CAST(NEW AS deliveryservice_tls_version)
);
RETURN NULL;
END;
@@ -39,7 +39,7 @@ BEGIN
SET last_updated=now()
WHERE id IN (
SELECT deliveryservice
- FROM old_table
+ FROM CAST(OLD AS deliveryservice_tls_version)
);
RETURN NULL;
END;
@@ -47,13 +47,11 @@ $$ LANGUAGE plpgsql;
CREATE TRIGGER update_ds_timestamp_on_tls_version_insertion
AFTER INSERT ON public.deliveryservice_tls_version
- REFERENCING NEW TABLE AS new_table
- FOR EACH STATEMENT EXECUTE PROCEDURE update_ds_timestamp_on_insert();
+ FOR EACH ROW EXECUTE PROCEDURE update_ds_timestamp_on_insert();
CREATE TRIGGER update_ds_timestamp_on_tls_version_delete
AFTER DELETE ON public.deliveryservice_tls_version
- REFERENCING OLD TABLE AS old_table
- FOR EACH STATEMENT EXECUTE PROCEDURE update_ds_timestamp_on_delete();
+ FOR EACH ROW EXECUTE PROCEDURE update_ds_timestamp_on_delete();
UPDATE public.deliveryservice_request
SET
diff --git a/traffic_ops/app/db/trafficvault/create_tables.sql b/traffic_ops/app/db/trafficvault/create_tables.sql
index bc59b8f..2fcb97e 100644
--- a/traffic_ops/app/db/trafficvault/create_tables.sql
+++ b/traffic_ops/app/db/trafficvault/create_tables.sql
@@ -54,8 +54,6 @@ ALTER FUNCTION on_update_current_timestamp_last_updated() OWNER TO traffic_vault
SET default_tablespace = '';
-SET default_table_access_method = heap;
-
--
-- Name: dnssec; Type: TABLE; Schema: public; Owner: traffic_vault
--
@@ -169,28 +167,28 @@ CREATE INDEX sslkey_version_idx ON sslkey USING btree (version);
-- Name: dnssec dnssec_last_updated; Type: TRIGGER; Schema: public; Owner: traffic_vault
--
-CREATE TRIGGER dnssec_last_updated BEFORE UPDATE ON dnssec FOR EACH ROW EXECUTE FUNCTION on_update_current_timestamp_last_updated();
+CREATE TRIGGER dnssec_last_updated BEFORE UPDATE ON dnssec FOR EACH ROW EXECUTE PROCEDURE on_update_current_timestamp_last_updated();
--
-- Name: sslkey sslkey_last_updated; Type: TRIGGER; Schema: public; Owner: traffic_vault
--
-CREATE TRIGGER sslkey_last_updated BEFORE UPDATE ON sslkey FOR EACH ROW EXECUTE FUNCTION on_update_current_timestamp_last_updated();
+CREATE TRIGGER sslkey_last_updated BEFORE UPDATE ON sslkey FOR EACH ROW EXECUTE PROCEDURE on_update_current_timestamp_last_updated();
--
-- Name: uri_signing_key uri_signing_key_last_updated; Type: TRIGGER; Schema: public; Owner: traffic_vault
--
-CREATE TRIGGER uri_signing_key_last_updated BEFORE UPDATE ON uri_signing_key FOR EACH ROW EXECUTE FUNCTION on_update_current_timestamp_last_updated();
+CREATE TRIGGER uri_signing_key_last_updated BEFORE UPDATE ON uri_signing_key FOR EACH ROW EXECUTE PROCEDURE on_update_current_timestamp_last_updated();
--
-- Name: url_sig_key url_sig_key_last_updated; Type: TRIGGER; Schema: public; Owner: traffic_vault
--
-CREATE TRIGGER url_sig_key_last_updated BEFORE UPDATE ON url_sig_key FOR EACH ROW EXECUTE FUNCTION on_update_current_timestamp_last_updated();
+CREATE TRIGGER url_sig_key_last_updated BEFORE UPDATE ON url_sig_key FOR EACH ROW EXECUTE PROCEDURE on_update_current_timestamp_last_updated();
--
[trafficcontrol] 12/12: Make TM recover from TO being unavailable
at startup (#6146)
Posted by zr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zrhoffman pushed a commit to branch 6.0.x
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 1c4ec191dd43ccf63e15322630cd38ca2f5e4f60
Author: Rawlin Peters <ra...@apache.org>
AuthorDate: Mon Aug 30 09:32:11 2021 -0600
Make TM recover from TO being unavailable at startup (#6146)
* Make TM recover from TO being unavailable at startup
Always set non-nil TO sessions that can eventually authenticate
themselves when TO recovers. Additionally, instead of exclusively using
one major client version after starting up and logging in, always
attempt the latest major version and fall back to the legacy version in
case of error. This will allow TM to seamlessly transition between using
either major version no matter which order TM and TO are upgraded in.
Closes: #6129
* Organize imports, add UsingDummyTO back
(cherry picked from commit 16dda648671ad438fcb8b6b08eaf6fbeb2fa9589)
---
CHANGELOG.md | 1 +
traffic_monitor/handler/handler.go | 2 +-
traffic_monitor/manager/opsconfig.go | 4 +-
traffic_monitor/towrap/towrap.go | 144 ++++++++++++++-------
.../{handler/handler.go => towrap/towrap_test.go} | 33 ++---
5 files changed, 111 insertions(+), 73 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 010ddd9..34bf7fc 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -67,6 +67,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
### Fixed
- [#5690](https://github.com/apache/trafficcontrol/issues/5690) - Fixed github action for added/modified db migration file.
- [#2471](https://github.com/apache/trafficcontrol/issues/2471) - A PR check to ensure added db migration file is the latest.
+- [#6129](https://github.com/apache/trafficcontrol/issues/6129) - Traffic Monitor start doesn't recover when Traffic Ops is unavailable
- [#5609](https://github.com/apache/trafficcontrol/issues/5609) - Fixed GET /servercheck filter for an extra query param.
- [#5954](https://github.com/apache/trafficcontrol/issues/5954) - Traffic Ops HTTP response write errors are ignored
- [#6048](https://github.com/apache/trafficcontrol/issues/6048) - TM sometimes sets cachegroups to unavailable even though all caches are online
diff --git a/traffic_monitor/handler/handler.go b/traffic_monitor/handler/handler.go
index 88832e8..31b7d6d 100644
--- a/traffic_monitor/handler/handler.go
+++ b/traffic_monitor/handler/handler.go
@@ -40,7 +40,7 @@ type OpsConfig struct {
HttpsListener string `json:"httpsListener"`
CertFile string `json:"certFile"`
KeyFile string `json:"keyFile"`
- UsingDummyTO bool `json:"usingDummyTO"`
+ UsingDummyTO bool `json:"usingDummyTO"` // only used in the TM UI to indicate if TM started up with on-disk backup snapshots
}
type Handler interface {
diff --git a/traffic_monitor/manager/opsconfig.go b/traffic_monitor/manager/opsconfig.go
index 29fe954..0361825 100644
--- a/traffic_monitor/manager/opsconfig.go
+++ b/traffic_monitor/manager/opsconfig.go
@@ -39,7 +39,7 @@ import (
"github.com/apache/trafficcontrol/traffic_monitor/todata"
"github.com/apache/trafficcontrol/traffic_monitor/towrap"
- "github.com/json-iterator/go"
+ jsoniter "github.com/json-iterator/go"
)
// StartOpsConfigManager starts the ops config manager goroutine, returning the (threadsafe) variables which it sets.
@@ -172,8 +172,8 @@ func StartOpsConfigManager(
time.Sleep(duration)
if toSession.BackupFileExists() && (toLoginCount >= cfg.TrafficOpsDiskRetryMax) {
- log.Errorf("error instantiating Session with traffic_ops, backup disk files exist, creating empty traffic_ops session to read")
newOpsConfig.UsingDummyTO = true
+ log.Errorf("error instantiating authenticated session with Traffic Ops, backup disk files exist, continuing with unauthenticated session")
break
}
diff --git a/traffic_monitor/towrap/towrap.go b/traffic_monitor/towrap/towrap.go
index e0857e8..fb330a3 100644
--- a/traffic_monitor/towrap/towrap.go
+++ b/traffic_monitor/towrap/towrap.go
@@ -22,9 +22,12 @@ package towrap
*/
import (
+ "crypto/tls"
"errors"
"fmt"
"io/ioutil"
+ "net/http"
+ "net/http/cookiejar"
"net/url"
"os"
"strconv"
@@ -38,6 +41,7 @@ import (
client "github.com/apache/trafficcontrol/traffic_ops/v3-client"
jsoniter "github.com/json-iterator/go"
+ "golang.org/x/net/publicsuffix"
)
const localHostIP = "127.0.0.1"
@@ -193,7 +197,6 @@ type TrafficOpsSessionThreadsafe struct {
m *sync.Mutex
lastCRConfig ByteMapCache
crConfigHist CRConfigHistoryThreadsafe
- useLegacy bool
CRConfigBackupFile string
TMConfigBackupFile string
}
@@ -209,17 +212,13 @@ func NewTrafficOpsSessionThreadsafe(s *client.Session, ls *legacyClient.Session,
session: &s,
legacySession: &ls,
TMConfigBackupFile: cfg.TMConfigBackupFile,
- useLegacy: false,
}
}
// Initialized tells whether or not the TrafficOpsSessionThreadsafe has been
-// properly initialized (by calling 'Update').
+// properly initialized with non-nil sessions.
func (s TrafficOpsSessionThreadsafe) Initialized() bool {
- if s.useLegacy {
- return s.legacySession != nil && *s.legacySession != nil
- }
- return s.session != nil && *s.session != nil
+ return s.session != nil && *s.session != nil && s.legacySession != nil && *s.legacySession != nil
}
// Update updates the TrafficOpsSessionThreadsafe's connection information with
@@ -240,24 +239,70 @@ func (s *TrafficOpsSessionThreadsafe) Update(
s.m.Lock()
defer s.m.Unlock()
+ // always set unauthenticated sessions first which can eventually authenticate themselves when attempting requests
+ if err := s.setSession(url, username, password, insecure, userAgent, useCache, timeout); err != nil {
+ return err
+ }
+ if err := s.setLegacySession(url, username, password, insecure, userAgent, useCache, timeout); err != nil {
+ return err
+ }
+
session, _, err := client.LoginWithAgent(url, username, password, insecure, userAgent, useCache, timeout)
if err != nil {
- log.Errorf("Error logging in using up-to-date client: %v", err)
+ log.Errorf("logging in using up-to-date client: %v", err)
legacySession, _, err := legacyClient.LoginWithAgent(url, username, password, insecure, userAgent, useCache, timeout)
if err != nil || legacySession == nil {
err = fmt.Errorf("logging in using legacy client: %v", err)
return err
}
*s.legacySession = legacySession
- s.useLegacy = true
} else {
*s.session = session
- s.useLegacy = false
}
return nil
}
+// setSession sets the session for the up-to-date client without logging in.
+func (s *TrafficOpsSessionThreadsafe) setSession(url, username, password string, insecure bool, userAgent string, useCache bool, timeout time.Duration) error {
+ options := cookiejar.Options{
+ PublicSuffixList: publicsuffix.List,
+ }
+ jar, err := cookiejar.New(&options)
+ if err != nil {
+ return err
+ }
+ to := client.NewSession(username, password, url, userAgent, &http.Client{
+ Timeout: timeout,
+ Transport: &http.Transport{
+ TLSClientConfig: &tls.Config{InsecureSkipVerify: insecure},
+ },
+ Jar: jar,
+ }, useCache)
+ *s.session = to
+ return nil
+}
+
+// setSession sets the session for the legacy client without logging in.
+func (s *TrafficOpsSessionThreadsafe) setLegacySession(url, username, password string, insecure bool, userAgent string, useCache bool, timeout time.Duration) error {
+ options := cookiejar.Options{
+ PublicSuffixList: publicsuffix.List,
+ }
+ jar, err := cookiejar.New(&options)
+ if err != nil {
+ return err
+ }
+ to := legacyClient.NewSession(username, password, url, userAgent, &http.Client{
+ Timeout: timeout,
+ Transport: &http.Transport{
+ TLSClientConfig: &tls.Config{InsecureSkipVerify: insecure},
+ },
+ Jar: jar,
+ }, useCache)
+ *s.legacySession = to
+ return nil
+}
+
// getThreadsafeSession is used internally to get a copy of the session pointer,
// or nil if it doesn't exist. This should not be used outside
// TrafficOpsSessionThreadsafe, and never stored, because part of the purpose of
@@ -334,44 +379,47 @@ func (s TrafficOpsSessionThreadsafe) CRConfigRaw(cdn string) ([]byte, error) {
var remoteAddr string
var err error
var data []byte
+ var legacyReqInf legacyClient.ReqInf
- if s.useLegacy {
- ss := s.getLegacy()
- if ss == nil {
+ ss := s.get()
+ if ss == nil {
+ return nil, ErrNilSession
+ }
+ data, reqInf, err := ss.GetCRConfig(cdn)
+ if reqInf.RemoteAddr != nil {
+ remoteAddr = reqInf.RemoteAddr.String()
+ }
+ if err != nil {
+ log.Warnln("getting CRConfig from Traffic Ops using up-to-date client: " + err.Error() + ". Retrying with legacy client")
+ ls := s.getLegacy()
+ if ls == nil {
return nil, ErrNilSession
}
- b, reqInf, e := ss.GetCRConfig(cdn)
- err = e
- data = b
- if reqInf.RemoteAddr != nil {
- remoteAddr = reqInf.RemoteAddr.String()
+ data, legacyReqInf, err = ls.GetCRConfig(cdn)
+ if legacyReqInf.RemoteAddr != nil {
+ remoteAddr = legacyReqInf.RemoteAddr.String()
}
- } else {
- ss := s.get()
- if ss == nil {
- return nil, ErrNilSession
- }
- b, reqInf, e := ss.GetCRConfig(cdn)
- err = e
- data = b
- if reqInf.RemoteAddr != nil {
- remoteAddr = reqInf.RemoteAddr.String()
+ if err != nil {
+ log.Errorln("getting CRConfig from Traffic Ops using legacy client: " + err.Error() + ". Checking for backup")
}
}
if err == nil {
- ioutil.WriteFile(s.CRConfigBackupFile, data, 0644)
+ log.Infoln("successfully got CRConfig from Traffic Ops. Writing to backup file")
+ if wErr := ioutil.WriteFile(s.CRConfigBackupFile, data, 0644); wErr != nil {
+ log.Errorf("failed to write CRConfig backup file: %v", wErr)
+ }
} else {
if s.BackupFileExists() {
log.Errorln("using backup file for CRConfig snapshot due to error fetching CRConfig snapshot from Traffic Ops: " + err.Error())
data, err = ioutil.ReadFile(s.CRConfigBackupFile)
if err != nil {
- return nil, fmt.Errorf("file Read Error: %v", err)
+ return nil, fmt.Errorf("reading CRConfig backup file: %v", err)
}
remoteAddr = localHostIP
err = nil
} else {
- return nil, fmt.Errorf("Failed to get CRConfig from Traffic Ops (%v), and there is no backup file", err)
+ return nil, fmt.Errorf("failed to get CRConfig from Traffic Ops (%v), and there is no backup file", err)
}
}
@@ -383,11 +431,6 @@ func (s TrafficOpsSessionThreadsafe) CRConfigRaw(cdn string) ([]byte, error) {
}
defer s.crConfigHist.Add(hist)
- // TODO: per the above logic, I don't think this is possible
- if err != nil {
- return data, err
- }
-
crc := &tc.CRConfig{}
json := jsoniter.ConfigFastest
if err = json.Unmarshal(data, crc); err != nil {
@@ -438,6 +481,9 @@ func (s TrafficOpsSessionThreadsafe) fetchLegacyTMConfig(cdn string) (*tc.Traffi
}
m, _, e := ss.GetTrafficMonitorConfig(cdn)
+ if m == nil {
+ return nil, e
+ }
return m.Upgrade(), e
}
@@ -451,20 +497,20 @@ func (s TrafficOpsSessionThreadsafe) trafficMonitorConfigMapRaw(cdn string) (*tc
var configMap *tc.TrafficMonitorConfigMap
var err error
- if s.useLegacy {
+ config, err = s.fetchTMConfig(cdn)
+ if err != nil {
+ log.Warnln("getting Traffic Monitor config from Traffic Ops using up-to-date client: " + err.Error() + ". Retrying with legacy client")
config, err = s.fetchLegacyTMConfig(cdn)
- } else {
- config, err = s.fetchTMConfig(cdn)
- }
-
- if config == nil {
if err != nil {
- return nil, fmt.Errorf("getting Traffic Monitor configuration map: %v", err)
+ log.Errorln("getting Traffic Monitor config from Traffic Ops using legacy client: " + err.Error())
}
- return nil, errors.New("nil configMap after fetching")
}
if err == nil {
+ log.Infoln("successfully got Traffic Monitor config from Traffic Ops")
+ if config == nil {
+ return nil, fmt.Errorf("nil Traffic Monitor config after successful fetch")
+ }
configMap, err = tc.TrafficMonitorTransformToMap(config)
}
@@ -491,7 +537,9 @@ func (s TrafficOpsSessionThreadsafe) trafficMonitorConfigMapRaw(cdn string) (*tc
json := jsoniter.ConfigFastest
data, err := json.Marshal(*config)
if err == nil {
- ioutil.WriteFile(s.TMConfigBackupFile, data, 0644)
+ if wErr := ioutil.WriteFile(s.TMConfigBackupFile, data, 0644); wErr != nil {
+ log.Errorf("failed to write TM config backup file: %v", wErr)
+ }
}
return configMap, err
@@ -613,10 +661,10 @@ func (s TrafficOpsSessionThreadsafe) MonitorCDN(hostName string) (string, error)
var server tc.ServerV30
var err error
- if s.useLegacy {
+ server, err = s.fetchServerByHostname(hostName)
+ if err != nil {
+ log.Warnln("getting server by hostname '" + hostName + "' using up-to-date client: " + err.Error() + ". Retrying with legacy client")
server, err = s.fetchLegacyServerByHostname(hostName)
- } else {
- server, err = s.fetchServerByHostname(hostName)
}
if err != nil {
diff --git a/traffic_monitor/handler/handler.go b/traffic_monitor/towrap/towrap_test.go
similarity index 56%
copy from traffic_monitor/handler/handler.go
copy to traffic_monitor/towrap/towrap_test.go
index 88832e8..e8482f3 100644
--- a/traffic_monitor/handler/handler.go
+++ b/traffic_monitor/towrap/towrap_test.go
@@ -1,4 +1,4 @@
-package handler
+package towrap
/*
* Licensed to the Apache Software Foundation (ASF) under one
@@ -20,29 +20,18 @@ package handler
*/
import (
- "io"
+ "testing"
"time"
-)
-const (
- NOTIFY_NEVER = iota
- NOTIFY_CHANGE
- NOTIFY_ALWAYS
+ "github.com/apache/trafficcontrol/traffic_monitor/config"
)
-type OpsConfig struct {
- Username string `json:"username"`
- Password string `json:"password"`
- Url string `json:"url"`
- Insecure bool `json:"insecure"`
- CdnName string `json:"cdnName"`
- HttpListener string `json:"httpListener"`
- HttpsListener string `json:"httpsListener"`
- CertFile string `json:"certFile"`
- KeyFile string `json:"keyFile"`
- UsingDummyTO bool `json:"usingDummyTO"`
-}
-
-type Handler interface {
- Handle(string, io.Reader, string, time.Duration, time.Time, error, uint64, bool, interface{}, chan<- uint64)
+func TestTrafficOpsSessionThreadsafeUpdateSetsNonNilSessions(t *testing.T) {
+ s := NewTrafficOpsSessionThreadsafe(nil, nil, 5, config.Config{})
+ err := s.Update("", "", "", true, "", false, 10*time.Second)
+ if err == nil {
+ t.Error("expected an error, got nil")
+ } else if s.session == nil || *s.session == nil || s.legacySession == nil || *s.legacySession == nil {
+ t.Errorf("expected non-nil sessions after getting error from Update()")
+ }
}
[trafficcontrol] 09/12: Migrate Profiles test from E2E to integrate
test suite TP (#6071)
Posted by zr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zrhoffman pushed a commit to branch 6.0.x
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit aa3fa79988d89b089d5e306ae59f3146dc2d3a6d
Author: dpham <ph...@yahoo.com>
AuthorDate: Thu Aug 26 15:28:39 2021 -0600
Migrate Profiles test from E2E to integrate test suite TP (#6071)
* Add test
* Add function type
* Using aftereach and after all for profiles
* Retest
* Add before all and fix functions
* Fixing aftereach
* Fixing clean up and setup
* Remove after each and after all
* Retest after each
* Add new line
* Make the spec file clean
* Refix the structure
* Retest
* Change clean up back and add before all
* Test new structure
* Change back to normal remove beforeall
* Refix format
Co-authored-by: Pham, Dandy (Contractor) <Da...@comcast.com>
(cherry picked from commit 012960b74c495376ee442bfd2bb9e4174fb5e1da)
---
traffic_portal/test/integration/Data/profiles.ts | 32 +++++++++
.../integration/PageObjects/ProfilesPage.po.ts | 39 +++++++----
.../test/integration/specs/Profiles.spec.ts | 79 +++++++++++++++-------
3 files changed, 111 insertions(+), 39 deletions(-)
diff --git a/traffic_portal/test/integration/Data/profiles.ts b/traffic_portal/test/integration/Data/profiles.ts
index 61c1494..aebd6c5 100644
--- a/traffic_portal/test/integration/Data/profiles.ts
+++ b/traffic_portal/test/integration/Data/profiles.ts
@@ -107,6 +107,22 @@ export const profiles = {
password: "pa$$word"
}
],
+ check: [
+ {
+ description: "check CSV link from Profiles page",
+ Name: "Export as CSV"
+ }
+ ],
+ toggle:[
+ {
+ description: "hide first table column",
+ Name: "CDN"
+ },
+ {
+ description: "redisplay first table column",
+ Name: "CDN"
+ }
+ ],
add: [
{
description: "create a Profiles",
@@ -142,6 +158,22 @@ export const profiles = {
password: "pa$$word"
}
],
+ check: [
+ {
+ description: "check CSV link from Profiles page",
+ Name: "Export as CSV"
+ }
+ ],
+ toggle:[
+ {
+ description: "hide first table column",
+ Name: "CDN"
+ },
+ {
+ description: "redisplay first table column",
+ Name: "CDN"
+ }
+ ],
add: [
{
description: "create a Profiles",
diff --git a/traffic_portal/test/integration/PageObjects/ProfilesPage.po.ts b/traffic_portal/test/integration/PageObjects/ProfilesPage.po.ts
index fc186bf..bd9613f 100644
--- a/traffic_portal/test/integration/PageObjects/ProfilesPage.po.ts
+++ b/traffic_portal/test/integration/PageObjects/ProfilesPage.po.ts
@@ -59,22 +59,23 @@ export class ProfilesPage extends BasePage {
private txtCompareDropdown2 = element(by.name('compareDropdown2'));
private btnCompareSubmit = element(by.name('compareSubmit'));
private mnuCompareTable = element(by.id('profilesParamsCompareTable_wrapper'));
+ private btnTableColumn = element(by.className("caret"))
private randomize = randomize;
- async OpenProfilesPage() {
- let snp = new SideNavigationPage();
+ public async OpenProfilesPage() {
+ const snp = new SideNavigationPage();
await snp.NavigateToProfilesPage();
}
- async OpenConfigureMenu() {
- let snp = new SideNavigationPage();
+ public async OpenConfigureMenu() {
+ const snp = new SideNavigationPage();
await snp.ClickConfigureMenu();
}
public async CreateProfile(profile: CreateProfile): Promise<boolean> {
let result = false;
- let basePage = new BasePage();
- let snp = new SideNavigationPage();
+ const basePage = new BasePage();
+ const snp = new SideNavigationPage();
await snp.NavigateToProfilesPage();
await this.btnCreateNewProfile.click();
await this.txtName.sendKeys(profile.Name + this.randomize);
@@ -94,7 +95,7 @@ export class ProfilesPage extends BasePage {
}
public async SearchProfile(nameProfiles: string): Promise<boolean> {
- let snp = new SideNavigationPage();
+ const snp = new SideNavigationPage();
let name = nameProfiles + this.randomize;
await snp.NavigateToProfilesPage();
await this.txtSearch.clear();
@@ -106,9 +107,9 @@ export class ProfilesPage extends BasePage {
return false;
}
- async CompareProfile(profile1: string, profile2: string) {
+ public async CompareProfile(profile1: string, profile2: string) {
let result = false;
- let snp = new SideNavigationPage();
+ const snp = new SideNavigationPage();
await snp.NavigateToProfilesPage();
await this.btnMore.click();
await this.btnCompareProfile.click();
@@ -121,22 +122,22 @@ export class ProfilesPage extends BasePage {
}
}
- public async UpdateProfile(profile: UpdateProfile): Promise<boolean | undefined> {
- let basePage = new BasePage();
+ public async UpdateProfile(profile: UpdateProfile): Promise<boolean> {
+ const basePage = new BasePage();
switch (profile.description) {
case "update profile type":
await this.txtType.sendKeys(profile.Type);
await basePage.ClickUpdate();
break;
default:
- return undefined;
+ return false;
}
return await basePage.GetOutputMessage().then(value => profile.validationMessage === value);
}
public async DeleteProfile(profile: DeleteProfile): Promise<boolean> {
let result = false;
- let basePage = new BasePage();
+ const basePage = new BasePage();
await this.btnDelete.click();
await this.txtConfirmName.sendKeys(profile.Name + this.randomize);
await basePage.ClickDeletePermanently();
@@ -149,4 +150,16 @@ export class ProfilesPage extends BasePage {
})
return result;
}
+
+ public async CheckCSV(name: string): Promise<boolean> {
+ return element(by.cssContainingText("span", name)).isPresent();
+ }
+
+ public async ToggleTableColumn(name: string): Promise<boolean> {
+ await this.btnTableColumn.click();
+ const result = await element(by.cssContainingText("th", name)).isPresent();
+ await element(by.cssContainingText("label", name)).click();
+ await this.btnTableColumn.click();
+ return !result;
+ }
}
diff --git a/traffic_portal/test/integration/specs/Profiles.spec.ts b/traffic_portal/test/integration/specs/Profiles.spec.ts
index f415db0..02c7d4b 100644
--- a/traffic_portal/test/integration/specs/Profiles.spec.ts
+++ b/traffic_portal/test/integration/specs/Profiles.spec.ts
@@ -24,6 +24,7 @@ import { TopNavigationPage } from '../PageObjects/TopNavigationPage.po';
import { api } from "../config";
import { profiles } from "../Data";
+
const loginPage = new LoginPage();
const topNavigation = new TopNavigationPage();
const profilesPage = new ProfilesPage();
@@ -33,46 +34,72 @@ describe('Setup API for Profiles', () => {
await api.UseAPI(profiles.setup);
});
});
+
profiles.tests.forEach(async profilesData => {
profilesData.logins.forEach(login => {
describe(`Traffic Portal - Profiles - ${login.description}`, () => {
+ afterEach(async function () {
+ await profilesPage.OpenProfilesPage();
+ });
+ afterAll(async function () {
+ expect(await topNavigation.Logout()).toBeTruthy();
+ })
it('can login', async () => {
browser.get(browser.params.baseUrl);
await loginPage.Login(login);
expect(await loginPage.CheckUserName(login)).toBeTruthy();
- });
- it('can open profiles page', async () => {
await profilesPage.OpenConfigureMenu();
- await profilesPage.OpenProfilesPage();
});
- profilesData.add.forEach(add => {
- it(add.description, async () => {
- expect(await profilesPage.CreateProfile(add)).toBeTruthy();
- await profilesPage.OpenProfilesPage();
+ it('can perform check test suits', async () => {
+ profilesData.check.forEach(check => {
+ it(check.description, async () => {
+ expect(await profilesPage.CheckCSV(check.Name)).toBe(true);
+ });
});
- });
- profilesData.update.forEach(update => {
- it(update.description, async () => {
- await profilesPage.SearchProfile(update.Name);
- expect(await profilesPage.UpdateProfile(update)).toBeTruthy();
- await profilesPage.OpenProfilesPage();
+ })
+ it('can perform toggle test suits', async () => {
+ profilesData.toggle.forEach(toggle => {
+ it(toggle.description, async () => {
+ if (toggle.description.includes('hide')) {
+ expect(await profilesPage.ToggleTableColumn(toggle.Name)).toBe(false);
+ } else {
+ expect(await profilesPage.ToggleTableColumn(toggle.Name)).toBe(true);
+ }
+ });
+ })
+ })
+ it('can perform add test suits', async () => {
+ profilesData.add.forEach(add => {
+ it(add.description, async () => {
+ expect(await profilesPage.CreateProfile(add)).toBeTruthy();
+ });
});
- });
- profilesData.remove.forEach(remove => {
- it(remove.description, async () => {
- await profilesPage.SearchProfile(remove.Name);
- expect(await profilesPage.DeleteProfile(remove)).toBeTruthy();
- await profilesPage.OpenProfilesPage();
+ })
+ it('can perform update test suits', async () => {
+ profilesData.update.forEach(update => {
+ it(update.description, async () => {
+ await profilesPage.SearchProfile(update.Name);
+ expect(await profilesPage.UpdateProfile(update)).toBeTruthy();
+ });
});
- });
- it('can logout', async () => {
- expect(await topNavigation.Logout()).toBeTruthy();
- });
+ })
+ it('can perform remove test suits', async () => {
+ profilesData.remove.forEach(remove => {
+ it(remove.description, async () => {
+ await profilesPage.SearchProfile(remove.Name);
+ expect(await profilesPage.DeleteProfile(remove)).toBeTruthy();
+ });
+ });
+ })
});
});
});
+
describe('Clean up API for Profiles', () => {
- it('Cleanup', async () => {
- await api.UseAPI(profiles.cleanup);
- });
+ afterAll(async function () {
+ it('Cleanup', async () => {
+ await api.UseAPI(profiles.cleanup);
+ });
+ })
});
+