You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficcontrol.apache.org by GitBox <gi...@apache.org> on 2018/12/28 16:51:33 UTC
[GitHub] dangogh closed pull request #3169: CIAB: Fix Automatic
snapshot/queue-updates on startup.
dangogh closed pull request #3169: CIAB: Fix Automatic snapshot/queue-updates on startup.
URL: https://github.com/apache/trafficcontrol/pull/3169
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docs/source/admin/quick_howto/ciab.rst b/docs/source/admin/quick_howto/ciab.rst
index 770e4ff5c..03d79774d 100644
--- a/docs/source/admin/quick_howto/ciab.rst
+++ b/docs/source/admin/quick_howto/ciab.rst
@@ -171,6 +171,14 @@ The "enroller" provides an efficient way for Traffic Ops to be populated with da
The enroller runs within CDN in a Box using the ``-dir <dir>`` switch which provides the above behavior. It can also be run using the ``-http :<port>`` switch to instead have it listen on the indicated port. In this case, it accepts only POST requests with the JSON provided using the POST JSON method, e.g. ``curl -X POST https://enroller/api/1.4/regions -d @newregion.json``. CDN in a Box does not currently use this method, but may be modified in the future to avoid using the shared volume approach.
+Auto Snapshot/Queue-Updates
+---------------------------
+An automatic snapshot of the current Traffic Ops CDN configuration/toplogy will be performed once the "enroller" has finished loading all of the data and a minimum number of servers have been enrolled. To enable this feature, set the boolean ``AUTO_SNAPQUEUE_ENABLED`` to ``true`` [7]_. The snapshot and queue-updates actions will not be performed until all servers in ``AUTO_SNAPQUEUE_SERVERS`` (comma-delimited string) have been enrolled. The current enrolled servers will be polled every ``AUTO_SNAPQUEUE_POLL_INTERVAL`` seconds, and each action (snapshot and queue-updates) will be delayed ``AUTO_SNAPQUEUE_ACTION_WAIT`` seconds [8]_.
+
+.. [7] Automatic Snapshot/Queue-Updates is enabled by default in ``infrastructure/cdn-in-a-box/variables.env``.
+.. [8] Server poll interval and delay action wait are defaulted to a value of 2 seconds.
+
+
Mock Origin Service
-------------------
The default "origin" service container provides a basic static file HTTP server as the central respository for content. Additional files can be added to the origin root content directory located at ``infrastructure/cdn-in-a-box/origin/content``. To request content directly from the origin directly and bypass the CDN:
diff --git a/infrastructure/cdn-in-a-box/traffic_ops/config.sh b/infrastructure/cdn-in-a-box/traffic_ops/config.sh
index b310d0111..a653d09f7 100755
--- a/infrastructure/cdn-in-a-box/traffic_ops/config.sh
+++ b/infrastructure/cdn-in-a-box/traffic_ops/config.sh
@@ -90,7 +90,8 @@ cat <<-EOF >/opt/traffic_ops/app/conf/cdn.conf
"max_db_connections": 20,
"backend_max_connections": {
"mojolicious": 4
- }
+ },
+ "crconfig_snapshot_use_client_request_host": true
},
"cors" : {
"access_control_allow_origin" : "*"
diff --git a/infrastructure/cdn-in-a-box/traffic_ops/run-go.sh b/infrastructure/cdn-in-a-box/traffic_ops/run-go.sh
index 1aa8094c0..5914cebff 100755
--- a/infrastructure/cdn-in-a-box/traffic_ops/run-go.sh
+++ b/infrastructure/cdn-in-a-box/traffic_ops/run-go.sh
@@ -90,17 +90,38 @@ while true; do
sleep 2
done
-# Snapshot the CDN
-until [ $(to-get '/CRConfig-Snapshots/CDN-in-a-Box/CRConfig.json' 2>/dev/null | jq -c -e '.config|length') -gt 0 ] ; do
- sleep 2
- echo "Do Snapshot for CDN-in-a-Box...";
- to-put 'api/1.3/cdns/2/snapshot'
+### Automatic Queue/Snapshot ###
+while [[ "$AUTO_SNAPQUEUE_ENABLED" = true ]] ; do
+ # AUTO_SNAPQUEUE_SERVERS should be a comma delimited list of expected docker service names to be enrolled - see varibles.env
+ expected_servers_json=$(echo "$AUTO_SNAPQUEUE_SERVERS" | tr ',' '\n' | jq -R . | jq -M -c -e -s '.|sort')
+ expected_servers_list=$(jq -r -n --argjson expected "$expected_servers_json" '$expected|join(",")')
+ expected_servers_total=$(jq -r -n --argjson expected "$expected_servers_json" '$expected|length')
+
+ current_servers_json=$(to-get 'api/1.4/servers' 2>/dev/null | jq -c -e '[.response[] | .xmppId] | sort')
+ [ -z "$current_servers_json" ] && current_servers_json='[]'
+ current_servers_list=$(jq -r -n --argjson current "$current_servers_json" '$current|join(",")')
+ current_servers_total=$(jq -r -n --argjson current "$current_servers_json" '$current|length')
+
+ remain_servers_json=$(jq -n --argjson expected "$expected_servers_json" --argjson current "$current_servers_json" '$expected-$current')
+ remain_servers_list=$(jq -r -n --argjson remain "$remain_servers_json" '$remain|join(",")')
+ remain_servers_total=$(jq -r -n --argjson remain "$remain_servers_json" '$remain|length')
+
+ echo "AUTO-SNAPQUEUE - Expected Servers ($expected_servers_total): $expected_servers_list"
+ echo "AUTO-SNAPQUEUE - Current Servers ($current_servers_total): $current_servers_list"
+ echo "AUTO-SNAPQUEUE - Remain Servers ($remain_servers_total): $remain_servers_list"
+
+ if ((remain_servers_total == 0)) ; then
+ echo "AUTO-SNAPQUEUE - All expected servers enrolled."
+ sleep $AUTO_SNAPQUEUE_ACTION_WAIT
+ echo "AUTO-SNAPQUEUE - Do automatic snapshot..."
+ to-put 'api/1.3/cdns/2/snapshot'
+ sleep $AUTO_SNAPQUEUE_ACTION_WAIT
+ echo "AUTO-SNAPQUEUE - Do queue updates..."
+ to-post 'api/1.3/cdns/2/queue_update' '{"action":"queue"}'
+ break
+ fi
+
+ sleep $AUTO_SNAPQUEUE_POLL_INTERVAL
done
-# Queue Updates
-sleep 1
-to-post 'api/1.3/cdns/2/queue_update' '{"action":"queue"}'
-
-### Workaround: End DeliveryService and Edge association
-
exec tail -f /dev/null
diff --git a/infrastructure/cdn-in-a-box/traffic_ops/to-access.sh b/infrastructure/cdn-in-a-box/traffic_ops/to-access.sh
index 5ce8ccbc9..e06055738 100644
--- a/infrastructure/cdn-in-a-box/traffic_ops/to-access.sh
+++ b/infrastructure/cdn-in-a-box/traffic_ops/to-access.sh
@@ -111,7 +111,7 @@ to-post() {
fi
to-auth && \
curl $CURLAUTH $CURLOPTS --cookie "$COOKIEJAR" -X POST $data "$TO_URL/$1"
- [[ -n $t ]] && rm "$t"
+ [[ -n $t ]] && rm -f "$t"
}
to-put() {
diff --git a/infrastructure/cdn-in-a-box/traffic_router/run.sh b/infrastructure/cdn-in-a-box/traffic_router/run.sh
index 88f1918f4..e8112ac01 100755
--- a/infrastructure/cdn-in-a-box/traffic_router/run.sh
+++ b/infrastructure/cdn-in-a-box/traffic_router/run.sh
@@ -92,15 +92,15 @@ echo "traffic_ops.password=$TO_ADMIN_PASSWORD" >> $TO_PROPERTIES
echo "traffic_monitor.bootstrap.hosts=$TM_FQDN:$TM_PORT;" >> $TM_PROPERTIES
echo "traffic_monitor.properties.reload.period=60000" >> $TM_PROPERTIES
+# Enroll Traffic Router
+to-enroll tr || (while true; do echo "enroll failed."; sleep 3 ; done)
+
# Wait for traffic monitor
until nc $TM_FQDN $TM_PORT </dev/null >/dev/null 2>&1; do
echo "Waiting for Traffic Monitor to start..."
sleep 3
done
-# Enroll Traffic Router
-to-enroll tr || (while true; do echo "enroll failed."; sleep 3 ; done)
-
touch $LOGFILE $ACCESSLOG
tail -F $CATALINA_OUT $CATALINA_LOG $LOGFILE $ACCESSLOG &
diff --git a/infrastructure/cdn-in-a-box/variables.env b/infrastructure/cdn-in-a-box/variables.env
index b425c7e86..796577921 100644
--- a/infrastructure/cdn-in-a-box/variables.env
+++ b/infrastructure/cdn-in-a-box/variables.env
@@ -83,3 +83,7 @@ TV_INT_PORT=8087
TV_HTTP_PORT=8098
TV_HTTPS_PORT=8088
ENROLLER_DIR=/shared/enroller
+AUTO_SNAPQUEUE_ENABLED=true
+AUTO_SNAPQUEUE_SERVERS=trafficops,trafficops-perl,trafficmonitor,trafficrouter,trafficvault,edge,mid
+AUTO_SNAPQUEUE_POLL_INTERVAL=2
+AUTO_SNAPQUEUE_ACTION_WAIT=2
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services