You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficcontrol.apache.org by oc...@apache.org on 2020/11/18 17:26:27 UTC
[trafficcontrol] 05/09: Remove unnecessary CDN in a Box waits
(#5283)
This is an automated email from the ASF dual-hosted git repository.
ocket8888 pushed a commit to branch 5.0.x
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 7f4b8ca79a05e605cfa631e9963e757e670225be
Author: Zach Hoffman <zr...@apache.org>
AuthorDate: Tue Nov 17 13:00:04 2020 -0700
Remove unnecessary CDN in a Box waits (#5283)
* Skip enroller initial data load when LOAD_TRAFFIC_OPS_DATA is not equal
to true
* Make DIG_IP_RETRY an inherited environment variable
* Enroll Traffic Vault after Riak finishes starting up
* Do not attempt to enroll Traffic Vault when running the Traffic Ops
integration tests GitHub action
* Add workflow-related files to Traffic Ops Go client/API integration
tests paths filters
* Add a period
* Specify LOAD_TRAFFIC_OPS_DATA and DIG_IP_RETRY for trafficops-perl service only
(cherry picked from commit 2b3e9cb71cf692e8531cb88f4354e9362f9a886b)
---
.github/actions/to-integration-tests/entrypoint.sh | 2 +-
.github/workflows/traffic ops.yml | 16 ++++++++++++----
docs/source/admin/quick_howto/ciab.rst | 4 ++++
docs/source/development/traffic_ops.rst | 3 +++
.../cdn-in-a-box/docker-compose.traffic-ops-test.yml | 7 ++++++-
.../cdn-in-a-box/docker-compose.traffic-portal-test.yml | 4 ++++
infrastructure/cdn-in-a-box/docker-compose.yml | 3 +++
.../cdn-in-a-box/traffic_ops/set-to-ips-from-dns.sh | 1 -
.../cdn-in-a-box/traffic_ops/trafficops-init.sh | 10 +++++++---
.../traffic_vault/poststart.d/02-add-search-schema.sh | 2 ++
infrastructure/cdn-in-a-box/traffic_vault/run.sh | 2 --
11 files changed, 42 insertions(+), 12 deletions(-)
diff --git a/.github/actions/to-integration-tests/entrypoint.sh b/.github/actions/to-integration-tests/entrypoint.sh
index d3c4ff2..443e7f4 100755
--- a/.github/actions/to-integration-tests/entrypoint.sh
+++ b/.github/actions/to-integration-tests/entrypoint.sh
@@ -72,7 +72,7 @@ start_traffic_vault() {
sed -i '0,/^update-ca-certificates/d' /etc/riak/prestart.d/00-config.sh;
# Do not try to source to-access.sh
- sed -i '/to-access\.sh/d' /etc/riak/{prestart.d,poststart.d}/*
+ sed -i '/to-access\.sh\|^to-enroll/d' /etc/riak/{prestart.d,poststart.d}/*
BASH_LINES
DOCKER_BUILDKIT=1 docker build "$ciab_dir" -f "${ciab_dir}/traffic_vault/Dockerfile" -t "$trafficvault" 2>&1 |
diff --git a/.github/workflows/traffic ops.yml b/.github/workflows/traffic ops.yml
index 80d674e..1a51795 100644
--- a/.github/workflows/traffic ops.yml
+++ b/.github/workflows/traffic ops.yml
@@ -20,17 +20,25 @@ name: Traffic Ops Go client/API integration tests
on:
push:
paths:
+ - .github/actions/todb-init/**
+ - .github/actions/to-integration-tests/**
+ - '.github/workflows/traffic ops.yml'
- GO_VERSION
- - traffic_ops/traffic_ops_golang/**.go
- - traffic_ops/testing/api/**.go
+ - infrastructure/cdn-in-a-box/traffic_vault/**
- traffic_ops/*client/**.go
+ - traffic_ops/testing/api/**.go
+ - traffic_ops/traffic_ops_golang/**.go
create:
pull_request:
paths:
+ - .github/actions/todb-init/**
+ - .github/actions/to-integration-tests/**
+ - '.github/workflows/traffic ops.yml'
- GO_VERSION
- - traffic_ops/traffic_ops_golang/**.go
- - traffic_ops/testing/api/**.go
+ - infrastructure/cdn-in-a-box/traffic_vault/**
- traffic_ops/*client/**.go
+ - traffic_ops/testing/api/**.go
+ - traffic_ops/traffic_ops_golang/**.go
types: [opened, reopened, ready_for_review, synchronize]
jobs:
diff --git a/docs/source/admin/quick_howto/ciab.rst b/docs/source/admin/quick_howto/ciab.rst
index 0f135ca..6a100d2 100644
--- a/docs/source/admin/quick_howto/ciab.rst
+++ b/docs/source/admin/quick_howto/ciab.rst
@@ -140,6 +140,8 @@ There also exist TP and TO integration tests containers. Both of these container
sudo docker-compose -f docker-compose.traffic-ops-test.yml up
+.. note:: If all CDN in a Box containers are started at once (example: ``docker-compose -f docker-compose.yml -f docker-compose.traffic-ops-test.yml up integration``), the :ref:`Enroller <ciab-enroller>` initial data load is skipped to prevent data conflicts with the :ref:`Traffic Ops API tests fixtures <dev-traffic-ops-fixtures>`.
+
variables.env
"""""""""""""
.. literalinclude:: ../../../../infrastructure/cdn-in-a-box/variables.env
@@ -226,6 +228,8 @@ Advanced Usage
==============
This section will be amended as functionality is added to the CDN in a Box project.
+.. _ciab-enroller:
+
The Enroller
------------
The "enroller" began as an efficient way for Traffic Ops to be populated with data as CDN in a Box starts up. It connects to Traffic Ops as the "admin" user and processes files places in the docker volume shared between the containers. The enroller watches each directory within the ``/shared/enroller`` directory for new :file:`{filename}.json` files to be created there. These files must follow the format outlined in the API guide for the ``POST`` method for each data type, (e.g. for a ` [...]
diff --git a/docs/source/development/traffic_ops.rst b/docs/source/development/traffic_ops.rst
index bd3b7b6..38c9bb8 100644
--- a/docs/source/development/traffic_ops.rst
+++ b/docs/source/development/traffic_ops.rst
@@ -336,6 +336,9 @@ The integration tests are run using :manpage:`go-test(1)`, with two configuratio
Specify the path to the `Test Configuration File`_. If not specified, it will attempt to read a file named ``traffic-ops-test.conf`` in the working directory.
.. seealso:: `Configuring the Integration Tests`_ for a detailed explanation of the format of this configuration file.
+
+.. _dev-traffic-ops-fixtures:
+
.. option:: --fixtures FIXTURES
Specify the path to a file containing static data for the tests to use. This should almost never be used, because many of the tests depend on the data having a certain content and structure. If not specified, it will attempt to read a file named ``tc-fixtures.json`` in the working directory.
diff --git a/infrastructure/cdn-in-a-box/docker-compose.traffic-ops-test.yml b/infrastructure/cdn-in-a-box/docker-compose.traffic-ops-test.yml
index 520df5f..2093e8a 100644
--- a/infrastructure/cdn-in-a-box/docker-compose.traffic-ops-test.yml
+++ b/infrastructure/cdn-in-a-box/docker-compose.traffic-ops-test.yml
@@ -38,8 +38,13 @@ services:
volumes:
- shared:/shared
- ../../junit:/junit
+
+ trafficops-perl:
+ environment:
+ DIG_IP_RETRY: ${DIG_IP_RETRY:-0}
+ LOAD_TRAFFIC_OPS_DATA: ${LOAD_TRAFFIC_OPS_DATA:-false}
+
volumes:
junit:
shared:
external: false
-
diff --git a/infrastructure/cdn-in-a-box/docker-compose.traffic-portal-test.yml b/infrastructure/cdn-in-a-box/docker-compose.traffic-portal-test.yml
index f285029..59ad76b 100644
--- a/infrastructure/cdn-in-a-box/docker-compose.traffic-portal-test.yml
+++ b/infrastructure/cdn-in-a-box/docker-compose.traffic-portal-test.yml
@@ -41,6 +41,10 @@ services:
- shared:/shared
- ../../junit:/junit
+ trafficops-perl:
+ environment:
+ DIG_IP_RETRY: ${DIG_IP_RETRY:-0}
+
volumes:
shared:
external: false
diff --git a/infrastructure/cdn-in-a-box/docker-compose.yml b/infrastructure/cdn-in-a-box/docker-compose.yml
index a78c52b..943ee4a 100644
--- a/infrastructure/cdn-in-a-box/docker-compose.yml
+++ b/infrastructure/cdn-in-a-box/docker-compose.yml
@@ -89,6 +89,9 @@ services:
domainname: infra.ciab.test
env_file:
- variables.env
+ environment:
+ DIG_IP_RETRY: ${DIG_IP_RETRY:-10}
+ LOAD_TRAFFIC_OPS_DATA: ${LOAD_TRAFFIC_OPS_DATA:-true}
hostname: trafficops-perl
image: trafficops-perl
# TODO: change to expose: "60443" to limit to containers
diff --git a/infrastructure/cdn-in-a-box/traffic_ops/set-to-ips-from-dns.sh b/infrastructure/cdn-in-a-box/traffic_ops/set-to-ips-from-dns.sh
index 1df6a8f..dc73ee9 100755
--- a/infrastructure/cdn-in-a-box/traffic_ops/set-to-ips-from-dns.sh
+++ b/infrastructure/cdn-in-a-box/traffic_ops/set-to-ips-from-dns.sh
@@ -40,7 +40,6 @@ service_ips="${gateway_ip}"
service_ip6s="${gateway_ip6}"
INTERFACE=$(ip link | awk '/\<UP\>/ && !/LOOPBACK/ {sub(/@.*/, "", $2); print $2}')
NETMASK=$(route | awk -v INTERFACE=$INTERFACE '$8 ~ INTERFACE && $1 !~ "default" {print $3}')
-DIG_IP_RETRY=10
for service_name in $service_names; do
service_fqdn="${service_name}.${service_domain}"
diff --git a/infrastructure/cdn-in-a-box/traffic_ops/trafficops-init.sh b/infrastructure/cdn-in-a-box/traffic_ops/trafficops-init.sh
index b0dc2b4..11deb9b 100755
--- a/infrastructure/cdn-in-a-box/traffic_ops/trafficops-init.sh
+++ b/infrastructure/cdn-in-a-box/traffic_ops/trafficops-init.sh
@@ -141,7 +141,11 @@ traffic_router_zonemanager_timeout() {
mv "$modified_crconfig" $crconfig_path;
}
-traffic_router_zonemanager_timeout
+if [[ "$LOAD_TRAFFIC_OPS_DATA" == true ]]; then
+ traffic_router_zonemanager_timeout
-# Load required data at the top level
-load_data_from /traffic_ops_data
+ # Load required data at the top level
+ load_data_from /traffic_ops_data
+else
+ touch "$ENROLLER_DIR/initial-load-done"
+fi
diff --git a/infrastructure/cdn-in-a-box/traffic_vault/poststart.d/02-add-search-schema.sh b/infrastructure/cdn-in-a-box/traffic_vault/poststart.d/02-add-search-schema.sh
index 512f479..dcf2037 100644
--- a/infrastructure/cdn-in-a-box/traffic_vault/poststart.d/02-add-search-schema.sh
+++ b/infrastructure/cdn-in-a-box/traffic_vault/poststart.d/02-add-search-schema.sh
@@ -21,3 +21,5 @@ curl -kvs -XPUT -H 'Content-Type:application/xml' "https://$TV_ADMIN_USER:$TV_AD
curl -kvs -XPUT -H 'Content-Type:application/json' "https://$TV_ADMIN_USER:$TV_ADMIN_PASSWORD@$TV_FQDN:$TV_HTTPS_PORT/search/index/sslkeys" -d '{"schema":"sslkeys"}'
curl -kvs -XPUT -H 'Content-Type:application/json' "https://$TV_ADMIN_USER:$TV_ADMIN_PASSWORD@$TV_FQDN:$TV_HTTPS_PORT/buckets/ssl/props" -d'{"props":{"search_index":"sslkeys"}}'
+
+to-enroll "tv" ALL "" "8088" "8088" || (while true; do echo "enroll failed."; sleep 3 ; done)
diff --git a/infrastructure/cdn-in-a-box/traffic_vault/run.sh b/infrastructure/cdn-in-a-box/traffic_vault/run.sh
index a7667e0..5585d0c 100755
--- a/infrastructure/cdn-in-a-box/traffic_vault/run.sh
+++ b/infrastructure/cdn-in-a-box/traffic_vault/run.sh
@@ -26,6 +26,4 @@ TO_URL=https://${TO_FQDN}:${TO_PORT}
TO_USER=$TV_USER
TO_PASSWORD=$TV_PASSWORD
-to-enroll "tv" ALL "" "8088" "8088" || (while true; do echo "enroll failed."; sleep 3 ; done)
-
${RIAK_HOME}/riak-cluster.sh