You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficcontrol.apache.org by GitBox <gi...@apache.org> on 2020/11/18 00:24:42 UTC
[GitHub] [trafficcontrol] jrushford opened a new pull request #5298: DRAFT: Add ort integration testing.
jrushford opened a new pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298
## What does this PR (Pull Request) do?
This PR adds integration testing to the TrafficOps ORT packages.
- [x] This PR is not related to any Issue
## Which Traffic Control components are affected by this PR?
## What is the best way to verify this PR?
See traffic_ops_ort/testing/README.md for running these tests.
## The following criteria are ALL met by this PR
- [X] This PR includes tests OR I have explained why tests are unnecessary
- [X] This PR includes documentation OR I have explained why documentation is unnecessary
- [ ] This PR includes an update to CHANGELOG.md OR such an update is not necessary
- [ ] This PR includes any and all required license headers
- [ ] This PR **DOES NOT FIX A SERIOUS SECURITY VULNERABILITY** (see [the Apache Software Foundation's security guidelines](https://www.apache.org/security/) for details)
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564937078
##########
File path: traffic_ops_ort/testing/docker/traffic_ops/run.sh
##########
@@ -0,0 +1,306 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Script for running the Dockerfile for Traffic Ops.
+# The Dockerfile sets up a Docker image which can be used for any new Traffic Ops container;
+# This script, which should be run when the container is run (it's the ENTRYPOINT), will configure the container.
+#
+# The following environment variables must be set, ordinarily by `docker run -e` arguments:
+# DB_SERVER
+# DB_PORT
+# DB_ROOT_PASS
+# DB_USER
+# DB_USER_PASS
+# DB_NAME
+# TO_ADMIN_USER
+# TO_ADMIN_PASS
+# CERT_COUNTRY
+# CERT_STATE
+# CERT_CITY
+# CERT_COMPANY
+# TO_DOMAIN
+# TRAFFIC_VAULT_PASS
+
+# Check that env vars are set
+envvars=( DB_SERVER DB_PORT DB_ROOT_PASS DB_USER DB_USER_PASS TO_ADMIN_USER TO_ADMIN_PASS CERT_COUNTRY CERT_STATE CERT_CITY CERT_COMPANY TO_DOMAIN)
+for v in $envvars
+do
+ if [[ -z $$v ]]; then echo "$v is unset"; exit 1; fi
+done
+
+start() {
+ service traffic_ops start
+ exec tail -f /var/log/traffic_ops/traffic_ops.log
+}
+
+init() {
+ local postinstall_input_file="postinstall-input.json"
+ cat > "$postinstall_input_file" <<- ENDOFMESSAGE
+{
+ "/opt/traffic_ops/app/conf/production/database.conf":[
+ {
+ "Database type":"Pg",
+ "config_var":"type"
+ },
+ {
+ "Database name":"$DB_NAME",
+ "config_var":"dbname"
+ },
+ {
+ "Database server hostname IP or FQDN":"$DB_SERVER",
+ "config_var":"hostname"
+ },
+ {
+ "Database port number":"$DB_PORT",
+ "config_var":"port"
+ },
+ {
+ "Traffic Ops database user":"$DB_USER",
+ "config_var":"user"
+ },
+ {
+ "Traffic Ops database password":"$DB_USER_PASS",
+ "config_var":"password",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/app/db/dbconf.yml":[
+ {
+ "Database server root (admin) user":"postgres",
+ "config_var":"pgUser"
+ },
+ {
+ "Database server admin password":"$DB_ROOT_PASS",
+ "config_var":"pgPassword",
+ "hidden":"1"
+ },
+ {
+ "Download Maxmind Database?":"yes",
+ "config_var":"maxmind"
+ }
+ ],
+ "/opt/traffic_ops/app/conf/cdn.conf":[
+ {
+ "Generate a new secret?":"yes",
+ "config_var":"genSecret"
+ },
+ {
+ "Port to serve on?": "443",
+ "config_var": "port"
+ },
+ {
+ "Number of workers?": "12",
+ "config_var":"workers"
+ },
+ {
+ "Traffic Ops url?": "https://$TO_HOSTNAME",
+ "config_var": "base_url"
+ },
+ {
+ "Number of secrets to keep?":"1",
+ "config_var":"keepSecrets"
+ }
+ ],
+ "/opt/traffic_ops/app/conf/ldap.conf":[
+ {
+ "Do you want to set up LDAP?":"no",
+ "config_var":"setupLdap"
+ },
+ {
+ "LDAP server hostname":"",
+ "config_var":"host"
+ },
+ {
+ "LDAP Admin DN":"",
+ "config_var":"admin_dn"
+ },
+ {
+ "LDAP Admin Password":"",
+ "config_var":"admin_pass",
+ "hidden":"1"
+ },
+ {
+ "LDAP Search Base":"",
+ "config_var":"search_base"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/users.json":[
+ {
+ "Administration username for Traffic Ops":"$TO_ADMIN_USER",
+ "config_var":"tmAdminUser"
+ },
+ {
+ "Password for the admin user":"$TO_ADMIN_PASS",
+ "config_var":"tmAdminPw",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/install/data/profiles/":[
+ {
+ "Add custom profiles?":"no",
+ "config_var":"custom_profiles"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/openssl_configuration.json":[
+ {
+ "Do you want to generate a certificate?":"yes",
+ "config_var":"genCert"
+ },
+ {
+ "Country Name (2 letter code)":"$CERT_COUNTRY",
+ "config_var":"country"
+ },
+ {
+ "State or Province Name (full name)":"$CERT_STATE",
+ "config_var":"state"
+ },
+ {
+ "Locality Name (eg, city)":"$CERT_CITY",
+ "config_var":"locality"
+ },
+ {
+ "Organization Name (eg, company)":"$CERT_COMPANY",
+ "config_var":"company"
+ },
+ {
+ "Organizational Unit Name (eg, section)":"",
+ "config_var":"org_unit"
+ },
+ {
+ "Common Name (eg, your name or your server's hostname)":"$TO_HOSTNAME",
+ "config_var":"common_name"
+ },
+ {
+ "RSA Passphrase":"passphrase",
+ "config_var":"rsaPassword",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/profiles.json":[
+ {
+ "Traffic Ops url":"https://$TO_HOSTNAME",
+ "config_var":"tm.url"
+ },
+ {
+ "Human-readable CDN Name. (No whitespace, please)":"cdn",
+ "config_var":"cdn_name"
+ },
+ {
+ "Health Polling Interval (milliseconds)":"8000",
+ "config_var":"health_polling_int"
+ },
+ {
+ "DNS sub-domain for which your CDN is authoritative":"$TO_HOSTNAME.$TO_DOMAIN",
+ "config_var":"dns_subdomain"
+ },
+ {
+ "TLD SOA admin":"traffic_ops",
+ "config_var":"soa_admin"
+ },
+ {
+ "TrafficServer Drive Prefix":"/dev/ram",
+ "config_var":"driver_prefix"
+ },
+ {
+ "TrafficServer RAM Drive Prefix":"/dev/ram",
+ "config_var":"ram_drive_prefix"
+ },
+ {
+ "TrafficServer RAM Drive Letters (comma separated)":"1",
+ "config_var":"ram_drive_letters"
+ },
+ {
+ "Health Threshold Load Average":"25",
+ "config_var":"health_thresh_load_avg"
+ },
+ {
+ "Health Threshold Available Bandwidth in Kbps":"1750000",
+ "config_var":"health_thresh_kbps"
+ },
+ {
+ "Traffic Server Health Connection Timeout (milliseconds)":"2000",
+ "config_var":"health_connect_timeout"
+ }
+ ]
+}
+ENDOFMESSAGE
+
+echo -e "{\n\t\"user\": \"riakuser\",\n\t\"password\": \"$RIAK_USER_PASS\",\n\t\"MaxTLSVersion\": \"1.1\",\n\t\"tlsConfig\": {\n\t\t\"insecureSkipVerify\": true\n\t}\n}" > /opt/traffic_ops/app/conf/production/riak.conf
Review comment:
changed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r563848733
##########
File path: traffic_ops_ort/testing/README.md
##########
@@ -0,0 +1,102 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Traffic Ops ORT Tests
+
+The ORT tests are used to validate the ORT tools used with a
+release of Traffic Ops. The tests insure that the ORT tools
+are able to communicate with Traffic Ops, install required
+packages and generate the correct ATS configuration files on
+a Trafficserver cache.
+
+The first thing you need do is to provide the Traffic Ops and
+Traffic Ops ORT RPM files for the build you wish to test using
+this framework. This test environment provides the necessary
+Docker containers used to support and execute the tests when
+given the necessary RPM's. If you choose to not use the provided
+Docker containers you will need to provide the following resources:
+
+ - A running Traffic Ops with the installed release to be tested
+ - A running Postgres SQL database which is loaded with the proper
+ test data for the Traffic Ops release.
+ - An Apache Traffic Server host that has the installed release of
+ ORT to be tested against the release of Traffic Ops.
+ - A yum server configured to provide the test rpm's herein.
+ - A Traffic vault server.
+
+# Directory layout
+
+ - trafficcontrol/traffic_ops_ort/testing/docker: has all the
+ necessary files for running the test Docker containers.
+ - trafficcontrol/traffic_ops_ort/testing/ort-tests: this directory.
+ contains all the go files used to run the ORT tests.
+
+# Setup.
+
+ 1. Build the Traffic Ops and Traffic Ops ORT RPM's that you wish
+ to test. See the top level 'build' directory for building
+ instructions.
+ 2. Copy the Traffic Ops RPM to docker/trafffic_ops/traffic_ops.rpm
+ (NOTE: Use the file name 'traffic_ops.rpm')
+ 3. Copy the Traffic Ops ORT rpm to docker/ats_edge/traffic_ops_ort.rpm
+ (NOTE: Use the file name 'traffic_ops_ort.rpm'
+ 4. Copy the recent copy of riak-2.2.3-1.el7.centos.x86_64.rpm to
+ docker/traffic_vault/riak-2.2.3-1.el7.centos.x86_64.rpm
+ 5. Copy an Apache Trafficserver rpm to
+ docker/yumserver/test-rpms/trafficserver-$VERSION.$COMMIT-HASH.el7.x86_64.rpm
+ You will need to edit and adjust the trafficserver package value in
+ ort-tests/tc-fixtures.json to match the $VERSION.$COMMIT-HASH used in the name
+ of your RPM. Search for '8.0.8-19.77cb23a' in the ort-tests/tc-fixtures.json
+ and change the value to match the RPM version you choose to use.
+ For example, the current value in tc-fixtures.json is '8.0.8-19.77cb23a' and
+ therefore the rpm file expected in 'docker/yumserver/test-rpms' is
+ 'trafficserver-8.0.8-19.77cb23a.el7.x86_64.rpm'.
+ 6. The container Docker files have the usernames and passwords used in the various
+ containers ie, postgresql db, traffic_ops, and traffic_ops_ort. The usernames
+ and passwords passed to the 't3c' executable in in the
+ ort-tests/conf/docker-edge-cache.conf file. Make sure that the usernames/passwords
+ in the Docker files match those in the t3c configuration file.
+ An example ort-tests/conf/edge-cache.conf file is provided should you choose to
+ use your own Traffic Ops and Postgresql environment.
+ 7. Build the Docker images and start the containers:
+ ```
+ # cd trafficcontrol/traffic_ops_ort/testing/docker
+ # docker-compose build
+ # docker-compose up -d
+ ```
+ After some time, test results should be available at
+ 'ort-tests/test.log'
+
+ If you wish to run the tests manually use 'docker ps' to obtain the container id for
+ the ats_edge host and then:
+
+ ```
+ # docker exec -it $ats_edge_container_id /bin/bash -l
+ # cd /ort-tests
+ # go test -cfg=conf/docker-edge-cache.conf
+ ```
Review comment:
@ocket8888 this is fixed to.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford edited a comment on pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford edited a comment on pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#issuecomment-767804464
> Go files that are meant to run when you use `go test` should have the `_test.go` suffix
@ocket8888 Are you talking about the go files in testing/ort-tests/tcdata? I don't want them run as tests. I'm using them to load test fixtures data and when they fail to load some portion of the fixtures data I'll know about it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#issuecomment-768591327
> Tests don't appear to pass: https://pastebin.com/xFtQq1eh
@ocket8888 Where was that ATS RPM from? Did you grab one from the CDN repo? Here is the error, that RPM has an unsatisfied dependency on libmagic. Try an RPM from the CDN repo such as, trafficserver-8.1.0-41.f25b59e.el7.x86_64.rpm
There is a lot in that log but, the problem:
`ERROR: t3c.go:122: 2021-01-27T21:15:05.25734678Z: Error processing packages: Unable to install trafficserver-8.0.3-1.el7.x86_64 : Error executing '/usr/bin/yum': Error: Package: trafficserver-8.0.3-1.el7.x86_64 (traffic-control)
Requires: libMagickCore.so.5()(64bit)
Error: Package: trafficserver-8.0.3-1.el7.x86_64 (traffic-control)
Requires: libMagick++.so.5()(64bit)`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564937293
##########
File path: traffic_ops_ort/testing/docker/yumserver/Dockerfile
##########
@@ -0,0 +1,35 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Traffic Server container images
+# as Edges for Traffic Control 1.4
+# Based on CentOS 6.6
+############################################################
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y httpd
+RUN yum install -y createrepo
+RUN yum install -y yum-utils
Review comment:
@ocket8888 resolved
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564829650
##########
File path: traffic_ops_ort/testing/docker/yumserver/Dockerfile
##########
@@ -0,0 +1,35 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Traffic Server container images
+# as Edges for Traffic Control 1.4
+# Based on CentOS 6.6
+############################################################
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y httpd
+RUN yum install -y createrepo
+RUN yum install -y yum-utils
Review comment:
Installing each package in a separate layer leaders to inconsistent behavior. Depending on what modifications are made to the Dockerfile in the future, some layers will be cached and some will not, meaning that the image is no longer well-defined by a build date and a list of packages.
[Dockerfile best practices](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#apt-get) dictate grouping the list of packages to be installed into a single layer, preferably starting with a `yum -y update`, which will ensure that all packages listed are installed (see: "cache busting").
> it works, this is fine.
A big part of why a reviewer's input matters is that this code will eventually break and need to be maintained, and in a shared project, no contributor can claim with certainty that they will be the sole one to fix that code in the future. Requests to decrease the maintenance cost of a code fragment should be heeded.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564823605
##########
File path: traffic_ops_ort/testing/docker/traffic_ops/run.sh
##########
@@ -0,0 +1,306 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Script for running the Dockerfile for Traffic Ops.
+# The Dockerfile sets up a Docker image which can be used for any new Traffic Ops container;
+# This script, which should be run when the container is run (it's the ENTRYPOINT), will configure the container.
+#
+# The following environment variables must be set, ordinarily by `docker run -e` arguments:
+# DB_SERVER
+# DB_PORT
+# DB_ROOT_PASS
+# DB_USER
+# DB_USER_PASS
+# DB_NAME
+# TO_ADMIN_USER
+# TO_ADMIN_PASS
+# CERT_COUNTRY
+# CERT_STATE
+# CERT_CITY
+# CERT_COMPANY
+# TO_DOMAIN
+# TRAFFIC_VAULT_PASS
+
+# Check that env vars are set
+envvars=( DB_SERVER DB_PORT DB_ROOT_PASS DB_USER DB_USER_PASS TO_ADMIN_USER TO_ADMIN_PASS CERT_COUNTRY CERT_STATE CERT_CITY CERT_COMPANY TO_DOMAIN)
+for v in $envvars
+do
+ if [[ -z $$v ]]; then echo "$v is unset"; exit 1; fi
+done
+
+start() {
+ service traffic_ops start
+ exec tail -f /var/log/traffic_ops/traffic_ops.log
+}
+
+init() {
+ local postinstall_input_file="postinstall-input.json"
+ cat > "$postinstall_input_file" <<- ENDOFMESSAGE
+{
+ "/opt/traffic_ops/app/conf/production/database.conf":[
+ {
+ "Database type":"Pg",
+ "config_var":"type"
+ },
+ {
+ "Database name":"$DB_NAME",
+ "config_var":"dbname"
+ },
+ {
+ "Database server hostname IP or FQDN":"$DB_SERVER",
+ "config_var":"hostname"
+ },
+ {
+ "Database port number":"$DB_PORT",
+ "config_var":"port"
+ },
+ {
+ "Traffic Ops database user":"$DB_USER",
+ "config_var":"user"
+ },
+ {
+ "Traffic Ops database password":"$DB_USER_PASS",
+ "config_var":"password",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/app/db/dbconf.yml":[
+ {
+ "Database server root (admin) user":"postgres",
+ "config_var":"pgUser"
+ },
+ {
+ "Database server admin password":"$DB_ROOT_PASS",
+ "config_var":"pgPassword",
+ "hidden":"1"
+ },
+ {
+ "Download Maxmind Database?":"yes",
+ "config_var":"maxmind"
+ }
+ ],
+ "/opt/traffic_ops/app/conf/cdn.conf":[
+ {
+ "Generate a new secret?":"yes",
+ "config_var":"genSecret"
+ },
+ {
+ "Port to serve on?": "443",
+ "config_var": "port"
+ },
+ {
+ "Number of workers?": "12",
+ "config_var":"workers"
+ },
+ {
+ "Traffic Ops url?": "https://$TO_HOSTNAME",
+ "config_var": "base_url"
+ },
+ {
+ "Number of secrets to keep?":"1",
+ "config_var":"keepSecrets"
+ }
+ ],
+ "/opt/traffic_ops/app/conf/ldap.conf":[
+ {
+ "Do you want to set up LDAP?":"no",
+ "config_var":"setupLdap"
+ },
+ {
+ "LDAP server hostname":"",
+ "config_var":"host"
+ },
+ {
+ "LDAP Admin DN":"",
+ "config_var":"admin_dn"
+ },
+ {
+ "LDAP Admin Password":"",
+ "config_var":"admin_pass",
+ "hidden":"1"
+ },
+ {
+ "LDAP Search Base":"",
+ "config_var":"search_base"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/users.json":[
+ {
+ "Administration username for Traffic Ops":"$TO_ADMIN_USER",
+ "config_var":"tmAdminUser"
+ },
+ {
+ "Password for the admin user":"$TO_ADMIN_PASS",
+ "config_var":"tmAdminPw",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/install/data/profiles/":[
+ {
+ "Add custom profiles?":"no",
+ "config_var":"custom_profiles"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/openssl_configuration.json":[
+ {
+ "Do you want to generate a certificate?":"yes",
+ "config_var":"genCert"
+ },
+ {
+ "Country Name (2 letter code)":"$CERT_COUNTRY",
+ "config_var":"country"
+ },
+ {
+ "State or Province Name (full name)":"$CERT_STATE",
+ "config_var":"state"
+ },
+ {
+ "Locality Name (eg, city)":"$CERT_CITY",
+ "config_var":"locality"
+ },
+ {
+ "Organization Name (eg, company)":"$CERT_COMPANY",
+ "config_var":"company"
+ },
+ {
+ "Organizational Unit Name (eg, section)":"",
+ "config_var":"org_unit"
+ },
+ {
+ "Common Name (eg, your name or your server's hostname)":"$TO_HOSTNAME",
+ "config_var":"common_name"
+ },
+ {
+ "RSA Passphrase":"passphrase",
+ "config_var":"rsaPassword",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/profiles.json":[
+ {
+ "Traffic Ops url":"https://$TO_HOSTNAME",
+ "config_var":"tm.url"
+ },
+ {
+ "Human-readable CDN Name. (No whitespace, please)":"cdn",
+ "config_var":"cdn_name"
+ },
+ {
+ "Health Polling Interval (milliseconds)":"8000",
+ "config_var":"health_polling_int"
+ },
+ {
+ "DNS sub-domain for which your CDN is authoritative":"$TO_HOSTNAME.$TO_DOMAIN",
+ "config_var":"dns_subdomain"
+ },
+ {
+ "TLD SOA admin":"traffic_ops",
+ "config_var":"soa_admin"
+ },
+ {
+ "TrafficServer Drive Prefix":"/dev/ram",
+ "config_var":"driver_prefix"
+ },
+ {
+ "TrafficServer RAM Drive Prefix":"/dev/ram",
+ "config_var":"ram_drive_prefix"
+ },
+ {
+ "TrafficServer RAM Drive Letters (comma separated)":"1",
+ "config_var":"ram_drive_letters"
+ },
+ {
+ "Health Threshold Load Average":"25",
+ "config_var":"health_thresh_load_avg"
+ },
+ {
+ "Health Threshold Available Bandwidth in Kbps":"1750000",
+ "config_var":"health_thresh_kbps"
+ },
+ {
+ "Traffic Server Health Connection Timeout (milliseconds)":"2000",
+ "config_var":"health_connect_timeout"
+ }
+ ]
+}
+ENDOFMESSAGE
+
+echo -e "{\n\t\"user\": \"riakuser\",\n\t\"password\": \"$RIAK_USER_PASS\",\n\t\"MaxTLSVersion\": \"1.1\",\n\t\"tlsConfig\": {\n\t\t\"insecureSkipVerify\": true\n\t}\n}" > /opt/traffic_ops/app/conf/production/riak.conf
+
+ # TODO determine if term, user are necessary
+ export TERM=xterm && export USER=root && /opt/traffic_ops/install/bin/postinstall -cfile "$postinstall_input_file"
+
+ # Only listen on IPv4, not IPv6, because Docker doesn't provide a v6 interface by default. See http://mojolicious.org/perldoc/Mojo/Server/Daemon#listen
+ sed -i -e 's#https://\[::\]#https://127\.0\.0\.1#' /opt/traffic_ops/app/conf/cdn.conf
+ service traffic_ops restart
+
+ TRAFFIC_OPS_URI="https://localhost"
+
+ TMP_TO_COOKIE="$(curl -v -s -k -X POST --data '{ "u":"'"$TO_ADMIN_USER"'", "p":"'"$TO_ADMIN_PASS"'" }' $TRAFFIC_OPS_URI/api/1.2/user/login 2>&1 | grep 'Set-Cookie' | sed -e 's/.*mojolicious=\(.*\); expires.*/\1/')"
+ echo "Got cookie: $TMP_TO_COOKIE"
+
+ TMP_DOMAIN=$TO_DOMAIN
+ sed -i -- "s/{{.Domain}}/$TMP_DOMAIN/g" /profile.origin.traffic_ops
+ echo "Got domain: $TMP_DOMAIN"
+
+ echo "Importing origin"
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" -F "filename=profile.origin.traffic_ops" -F "profile_to_import=@/profile.origin.traffic_ops" $TRAFFIC_OPS_URI/profile/doImport
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "division.name=East" $TRAFFIC_OPS_URI/division/create
+ TMP_DIVISION_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/region/add | grep --color=never -oE "<option value=\"[0-9]+\">East</option>" | grep --color=never -oE "[0-9]+")"
+ echo "Got division ID: $TMP_DIVISION_ID"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "region.name=Eastish" --data-urlencode "region.division_id=$TMP_DIVISION_ID" $TRAFFIC_OPS_URI/region/create
+ TMP_REGION_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/regions.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="Eastish"]; print match[0]')"
+ echo "Got region ID: $TMP_REGION_ID"
+
+ TMP_CACHEGROUP_TYPE="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/types.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="MID_LOC"]; print match[0]')"
+ echo "Got cachegroup type ID: $TMP_CACHEGROUP_TYPE"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "cg_data.name=mid-east" --data-urlencode "cg_data.short_name=east" --data-urlencode "cg_data.latitude=0" --data-urlencode "cg_data.longitude=0" --data-urlencode "cg_data.parent_cachegroup_id=-1" --data-urlencode "cg_data.type=$TMP_CACHEGROUP_TYPE" $TRAFFIC_OPS_URI/cachegroup/create
+ TMP_CACHEGROUP_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/cachegroups.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="mid-east"]; print match[0]')"
+ echo "Got cachegroup ID: $TMP_CACHEGROUP_ID"
+
+ TMP_CACHEGROUP_EDGE_TYPE="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/types.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="EDGE_LOC"]; print match[0]')"
+ echo "Got cachegroup type ID: $TMP_CACHEGROUP_EDGE_TYPE"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "cg_data.name=edge-east" --data-urlencode "cg_data.short_name=eeast" --data-urlencode "cg_data.latitude=0" --data-urlencode "cg_data.longitude=0" --data-urlencode "cg_data.parent_cachegroup_id=$TMP_CACHEGROUP_ID" --data-urlencode "cg_data.type=$TMP_CACHEGROUP_EDGE_TYPE" $TRAFFIC_OPS_URI/cachegroup/create
+ TMP_CACHEGROUP_EDGE_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/cachegroups.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="edge-east"]; print match[0]')"
+ echo "Got cachegroup edge ID: $TMP_CACHEGROUP_EDGE_ID"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "location.name=plocation-nyc-1" --data-urlencode "location.short_name=nyc" --data-urlencode "location.address=1 Main Street" --data-urlencode "location.city=nyc" --data-urlencode "location.state=NY" --data-urlencode "location.zip=12345" --data-urlencode "location.poc=" --data-urlencode "location.phone=" --data-urlencode "location.email=no@no.no" --data-urlencode "location.comments=" --data-urlencode "location.region=$TMP_REGION_ID" $TRAFFIC_OPS_URI/phys_location/create
Review comment:
@ocket8888 so where's that at, to-access.sh? This I re-used from @rob05c infrastructure/docker/run.sh.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564823951
##########
File path: traffic_ops_ort/testing/docker/traffic_vault/Dockerfile
##########
@@ -0,0 +1,63 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Riak container images
+# as Traffic Vault for Traffic Control 1.6.0
+# Based on CentOS 6.6
+############################################################
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+ARG TV_ADMIN_PASS
+
+# Install openssl
+RUN yum install -y openssl
+
+# Install the initscripts
+RUN yum -y install initscripts
+
+# Install curl which is used to configure the riak search schema.
+RUN yum -y install curl
+
+# On CentOS/RedHat/Fedora (recommended)
+RUN yum install -y java-1.8.0-openjdk java-1.8.0-openjdk-devel
Review comment:
I think it's fine, it's clear to me what's is being installed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r563800572
##########
File path: traffic_ops_ort/testing/README.md
##########
@@ -0,0 +1,102 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Traffic Ops ORT Tests
+
+The ORT tests are used to validate the ORT tools used with a
+release of Traffic Ops. The tests insure that the ORT tools
+are able to communicate with Traffic Ops, install required
+packages and generate the correct ATS configuration files on
+a Trafficserver cache.
+
+The first thing you need do is to provide the Traffic Ops and
+Traffic Ops ORT RPM files for the build you wish to test using
+this framework. This test environment provides the necessary
+Docker containers used to support and execute the tests when
+given the necessary RPM's. If you choose to not use the provided
+Docker containers you will need to provide the following resources:
+
+ - A running Traffic Ops with the installed release to be tested
+ - A running Postgres SQL database which is loaded with the proper
+ test data for the Traffic Ops release.
+ - An Apache Traffic Server host that has the installed release of
+ ORT to be tested against the release of Traffic Ops.
+ - A yum server configured to provide the test rpm's herein.
+ - A Traffic vault server.
+
+# Directory layout
+
+ - trafficcontrol/traffic_ops_ort/testing/docker: has all the
+ necessary files for running the test Docker containers.
+ - trafficcontrol/traffic_ops_ort/testing/ort-tests: this directory.
+ contains all the go files used to run the ORT tests.
+
+# Setup.
+
+ 1. Build the Traffic Ops and Traffic Ops ORT RPM's that you wish
+ to test. See the top level 'build' directory for building
+ instructions.
+ 2. Copy the Traffic Ops RPM to docker/trafffic_ops/traffic_ops.rpm
+ (NOTE: Use the file name 'traffic_ops.rpm')
+ 3. Copy the Traffic Ops ORT rpm to docker/ats_edge/traffic_ops_ort.rpm
+ (NOTE: Use the file name 'traffic_ops_ort.rpm'
+ 4. Copy the recent copy of riak-2.2.3-1.el7.centos.x86_64.rpm to
+ docker/traffic_vault/riak-2.2.3-1.el7.centos.x86_64.rpm
+ 5. Copy an Apache Trafficserver rpm to
+ docker/yumserver/test-rpms/trafficserver-$VERSION.$COMMIT-HASH.el7.x86_64.rpm
+ You will need to edit and adjust the trafficserver package value in
+ ort-tests/tc-fixtures.json to match the $VERSION.$COMMIT-HASH used in the name
+ of your RPM. Search for '8.0.8-19.77cb23a' in the ort-tests/tc-fixtures.json
+ and change the value to match the RPM version you choose to use.
+ For example, the current value in tc-fixtures.json is '8.0.8-19.77cb23a' and
+ therefore the rpm file expected in 'docker/yumserver/test-rpms' is
+ 'trafficserver-8.0.8-19.77cb23a.el7.x86_64.rpm'.
+ 6. The container Docker files have the usernames and passwords used in the various
+ containers ie, postgresql db, traffic_ops, and traffic_ops_ort. The usernames
+ and passwords passed to the 't3c' executable in in the
+ ort-tests/conf/docker-edge-cache.conf file. Make sure that the usernames/passwords
+ in the Docker files match those in the t3c configuration file.
+ An example ort-tests/conf/edge-cache.conf file is provided should you choose to
+ use your own Traffic Ops and Postgresql environment.
+ 7. Build the Docker images and start the containers:
+ ```
+ # cd trafficcontrol/traffic_ops_ort/testing/docker
+ # docker-compose build
+ # docker-compose up -d
+ ```
Review comment:
@ocket8888 Fixed, removed the '#'
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564824198
##########
File path: traffic_ops_ort/testing/docker/traffic_vault/functions
##########
@@ -0,0 +1,707 @@
+# -*-Shell-script-*-
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# functions This file contains functions to be used by most or all
+# shell scripts in the /etc/init.d directory.
+#
+
+TEXTDOMAIN=initscripts
+
+# Make sure umask is sane
+umask 022
+
+# Set up a default search path.
+PATH="/sbin:/usr/sbin:/bin:/usr/bin"
+export PATH
+
+if [ $PPID -ne 1 -a -z "$SYSTEMCTL_SKIP_REDIRECT" ] && \
+ [ -d /run/systemd/system ] ; then
+ case "$0" in
+ /etc/init.d/*|/etc/rc.d/init.d/*)
+ _use_systemctl=1
+ ;;
+ esac
+fi
+
+systemctl_redirect () {
+ local s
+ local prog=${1##*/}
+ local command=$2
+ local options=""
+
+ case "$command" in
+ start)
+ s=$"Starting $prog (via systemctl): "
+ ;;
+ stop)
+ s=$"Stopping $prog (via systemctl): "
+ ;;
+ reload|try-reload)
+ s=$"Reloading $prog configuration (via systemctl): "
+ ;;
+ restart|try-restart|condrestart)
+ s=$"Restarting $prog (via systemctl): "
Review comment:
fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564818353
##########
File path: traffic_ops_ort/testing/docker/yumserver/Dockerfile
##########
@@ -0,0 +1,35 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Traffic Server container images
+# as Edges for Traffic Control 1.4
+# Based on CentOS 6.6
+############################################################
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y httpd
+RUN yum install -y createrepo
+RUN yum install -y yum-utils
Review comment:
@ocket8888 it works, this is fine.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r565376233
##########
File path: traffic_ops_ort/testing/docker/traffic_ops/run.sh
##########
@@ -0,0 +1,306 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Script for running the Dockerfile for Traffic Ops.
+# The Dockerfile sets up a Docker image which can be used for any new Traffic Ops container;
+# This script, which should be run when the container is run (it's the ENTRYPOINT), will configure the container.
+#
+# The following environment variables must be set, ordinarily by `docker run -e` arguments:
+# DB_SERVER
+# DB_PORT
+# DB_ROOT_PASS
+# DB_USER
+# DB_USER_PASS
+# DB_NAME
+# TO_ADMIN_USER
+# TO_ADMIN_PASS
+# CERT_COUNTRY
+# CERT_STATE
+# CERT_CITY
+# CERT_COMPANY
+# TO_DOMAIN
+# TRAFFIC_VAULT_PASS
+
+# Check that env vars are set
+envvars=( DB_SERVER DB_PORT DB_ROOT_PASS DB_USER DB_USER_PASS TO_ADMIN_USER TO_ADMIN_PASS CERT_COUNTRY CERT_STATE CERT_CITY CERT_COMPANY TO_DOMAIN)
+for v in $envvars
+do
+ if [[ -z $$v ]]; then echo "$v is unset"; exit 1; fi
+done
+
+start() {
+ service traffic_ops start
+ exec tail -f /var/log/traffic_ops/traffic_ops.log
+}
+
+init() {
+ local postinstall_input_file="postinstall-input.json"
+ cat > "$postinstall_input_file" <<- ENDOFMESSAGE
+{
+ "/opt/traffic_ops/app/conf/production/database.conf":[
+ {
+ "Database type":"Pg",
+ "config_var":"type"
+ },
+ {
+ "Database name":"$DB_NAME",
+ "config_var":"dbname"
+ },
+ {
+ "Database server hostname IP or FQDN":"$DB_SERVER",
+ "config_var":"hostname"
+ },
+ {
+ "Database port number":"$DB_PORT",
+ "config_var":"port"
+ },
+ {
+ "Traffic Ops database user":"$DB_USER",
+ "config_var":"user"
+ },
+ {
+ "Traffic Ops database password":"$DB_USER_PASS",
+ "config_var":"password",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/app/db/dbconf.yml":[
+ {
+ "Database server root (admin) user":"postgres",
+ "config_var":"pgUser"
+ },
+ {
+ "Database server admin password":"$DB_ROOT_PASS",
+ "config_var":"pgPassword",
+ "hidden":"1"
+ },
+ {
+ "Download Maxmind Database?":"yes",
+ "config_var":"maxmind"
+ }
+ ],
+ "/opt/traffic_ops/app/conf/cdn.conf":[
+ {
+ "Generate a new secret?":"yes",
+ "config_var":"genSecret"
+ },
+ {
+ "Port to serve on?": "443",
+ "config_var": "port"
+ },
+ {
+ "Number of workers?": "12",
+ "config_var":"workers"
+ },
+ {
+ "Traffic Ops url?": "https://$TO_HOSTNAME",
+ "config_var": "base_url"
+ },
+ {
+ "Number of secrets to keep?":"1",
+ "config_var":"keepSecrets"
+ }
+ ],
+ "/opt/traffic_ops/app/conf/ldap.conf":[
+ {
+ "Do you want to set up LDAP?":"no",
+ "config_var":"setupLdap"
+ },
+ {
+ "LDAP server hostname":"",
+ "config_var":"host"
+ },
+ {
+ "LDAP Admin DN":"",
+ "config_var":"admin_dn"
+ },
+ {
+ "LDAP Admin Password":"",
+ "config_var":"admin_pass",
+ "hidden":"1"
+ },
+ {
+ "LDAP Search Base":"",
+ "config_var":"search_base"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/users.json":[
+ {
+ "Administration username for Traffic Ops":"$TO_ADMIN_USER",
+ "config_var":"tmAdminUser"
+ },
+ {
+ "Password for the admin user":"$TO_ADMIN_PASS",
+ "config_var":"tmAdminPw",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/install/data/profiles/":[
+ {
+ "Add custom profiles?":"no",
+ "config_var":"custom_profiles"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/openssl_configuration.json":[
+ {
+ "Do you want to generate a certificate?":"yes",
+ "config_var":"genCert"
+ },
+ {
+ "Country Name (2 letter code)":"$CERT_COUNTRY",
+ "config_var":"country"
+ },
+ {
+ "State or Province Name (full name)":"$CERT_STATE",
+ "config_var":"state"
+ },
+ {
+ "Locality Name (eg, city)":"$CERT_CITY",
+ "config_var":"locality"
+ },
+ {
+ "Organization Name (eg, company)":"$CERT_COMPANY",
+ "config_var":"company"
+ },
+ {
+ "Organizational Unit Name (eg, section)":"",
+ "config_var":"org_unit"
+ },
+ {
+ "Common Name (eg, your name or your server's hostname)":"$TO_HOSTNAME",
+ "config_var":"common_name"
+ },
+ {
+ "RSA Passphrase":"passphrase",
+ "config_var":"rsaPassword",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/profiles.json":[
+ {
+ "Traffic Ops url":"https://$TO_HOSTNAME",
+ "config_var":"tm.url"
+ },
+ {
+ "Human-readable CDN Name. (No whitespace, please)":"cdn",
+ "config_var":"cdn_name"
+ },
+ {
+ "Health Polling Interval (milliseconds)":"8000",
+ "config_var":"health_polling_int"
+ },
+ {
+ "DNS sub-domain for which your CDN is authoritative":"$TO_HOSTNAME.$TO_DOMAIN",
+ "config_var":"dns_subdomain"
+ },
+ {
+ "TLD SOA admin":"traffic_ops",
+ "config_var":"soa_admin"
+ },
+ {
+ "TrafficServer Drive Prefix":"/dev/ram",
+ "config_var":"driver_prefix"
+ },
+ {
+ "TrafficServer RAM Drive Prefix":"/dev/ram",
+ "config_var":"ram_drive_prefix"
+ },
+ {
+ "TrafficServer RAM Drive Letters (comma separated)":"1",
+ "config_var":"ram_drive_letters"
+ },
+ {
+ "Health Threshold Load Average":"25",
+ "config_var":"health_thresh_load_avg"
+ },
+ {
+ "Health Threshold Available Bandwidth in Kbps":"1750000",
+ "config_var":"health_thresh_kbps"
+ },
+ {
+ "Traffic Server Health Connection Timeout (milliseconds)":"2000",
+ "config_var":"health_connect_timeout"
+ }
+ ]
+}
+ENDOFMESSAGE
+
+echo -e "{\n\t\"user\": \"riakuser\",\n\t\"password\": \"$RIAK_USER_PASS\",\n\t\"MaxTLSVersion\": \"1.1\",\n\t\"tlsConfig\": {\n\t\t\"insecureSkipVerify\": true\n\t}\n}" > /opt/traffic_ops/app/conf/production/riak.conf
+
+ # TODO determine if term, user are necessary
+ export TERM=xterm && export USER=root && /opt/traffic_ops/install/bin/postinstall -cfile "$postinstall_input_file"
+
+ # Only listen on IPv4, not IPv6, because Docker doesn't provide a v6 interface by default. See http://mojolicious.org/perldoc/Mojo/Server/Daemon#listen
+ sed -i -e 's#https://\[::\]#https://127\.0\.0\.1#' /opt/traffic_ops/app/conf/cdn.conf
+ service traffic_ops restart
+
+ TRAFFIC_OPS_URI="https://localhost"
+
+ TMP_TO_COOKIE="$(curl -v -s -k -X POST --data '{ "u":"'"$TO_ADMIN_USER"'", "p":"'"$TO_ADMIN_PASS"'" }' $TRAFFIC_OPS_URI/api/1.2/user/login 2>&1 | grep 'Set-Cookie' | sed -e 's/.*mojolicious=\(.*\); expires.*/\1/')"
+ echo "Got cookie: $TMP_TO_COOKIE"
+
+ TMP_DOMAIN=$TO_DOMAIN
+ sed -i -- "s/{{.Domain}}/$TMP_DOMAIN/g" /profile.origin.traffic_ops
+ echo "Got domain: $TMP_DOMAIN"
+
+ echo "Importing origin"
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" -F "filename=profile.origin.traffic_ops" -F "profile_to_import=@/profile.origin.traffic_ops" $TRAFFIC_OPS_URI/profile/doImport
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "division.name=East" $TRAFFIC_OPS_URI/division/create
+ TMP_DIVISION_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/region/add | grep --color=never -oE "<option value=\"[0-9]+\">East</option>" | grep --color=never -oE "[0-9]+")"
+ echo "Got division ID: $TMP_DIVISION_ID"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "region.name=Eastish" --data-urlencode "region.division_id=$TMP_DIVISION_ID" $TRAFFIC_OPS_URI/region/create
+ TMP_REGION_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/regions.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="Eastish"]; print match[0]')"
+ echo "Got region ID: $TMP_REGION_ID"
+
+ TMP_CACHEGROUP_TYPE="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/types.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="MID_LOC"]; print match[0]')"
+ echo "Got cachegroup type ID: $TMP_CACHEGROUP_TYPE"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "cg_data.name=mid-east" --data-urlencode "cg_data.short_name=east" --data-urlencode "cg_data.latitude=0" --data-urlencode "cg_data.longitude=0" --data-urlencode "cg_data.parent_cachegroup_id=-1" --data-urlencode "cg_data.type=$TMP_CACHEGROUP_TYPE" $TRAFFIC_OPS_URI/cachegroup/create
+ TMP_CACHEGROUP_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/cachegroups.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="mid-east"]; print match[0]')"
+ echo "Got cachegroup ID: $TMP_CACHEGROUP_ID"
+
+ TMP_CACHEGROUP_EDGE_TYPE="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/types.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="EDGE_LOC"]; print match[0]')"
+ echo "Got cachegroup type ID: $TMP_CACHEGROUP_EDGE_TYPE"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "cg_data.name=edge-east" --data-urlencode "cg_data.short_name=eeast" --data-urlencode "cg_data.latitude=0" --data-urlencode "cg_data.longitude=0" --data-urlencode "cg_data.parent_cachegroup_id=$TMP_CACHEGROUP_ID" --data-urlencode "cg_data.type=$TMP_CACHEGROUP_EDGE_TYPE" $TRAFFIC_OPS_URI/cachegroup/create
+ TMP_CACHEGROUP_EDGE_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/cachegroups.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="edge-east"]; print match[0]')"
+ echo "Got cachegroup edge ID: $TMP_CACHEGROUP_EDGE_ID"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "location.name=plocation-nyc-1" --data-urlencode "location.short_name=nyc" --data-urlencode "location.address=1 Main Street" --data-urlencode "location.city=nyc" --data-urlencode "location.state=NY" --data-urlencode "location.zip=12345" --data-urlencode "location.poc=" --data-urlencode "location.phone=" --data-urlencode "location.email=no@no.no" --data-urlencode "location.comments=" --data-urlencode "location.region=$TMP_REGION_ID" $TRAFFIC_OPS_URI/phys_location/create
Review comment:
@rob05c @ocket8888 none of that stuff is really needed, all data is loaded from the tc-fixtures.json
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
rob05c commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564829572
##########
File path: traffic_ops_ort/testing/docker/traffic_ops/run.sh
##########
@@ -0,0 +1,306 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Script for running the Dockerfile for Traffic Ops.
+# The Dockerfile sets up a Docker image which can be used for any new Traffic Ops container;
+# This script, which should be run when the container is run (it's the ENTRYPOINT), will configure the container.
+#
+# The following environment variables must be set, ordinarily by `docker run -e` arguments:
+# DB_SERVER
+# DB_PORT
+# DB_ROOT_PASS
+# DB_USER
+# DB_USER_PASS
+# DB_NAME
+# TO_ADMIN_USER
+# TO_ADMIN_PASS
+# CERT_COUNTRY
+# CERT_STATE
+# CERT_CITY
+# CERT_COMPANY
+# TO_DOMAIN
+# TRAFFIC_VAULT_PASS
+
+# Check that env vars are set
+envvars=( DB_SERVER DB_PORT DB_ROOT_PASS DB_USER DB_USER_PASS TO_ADMIN_USER TO_ADMIN_PASS CERT_COUNTRY CERT_STATE CERT_CITY CERT_COMPANY TO_DOMAIN)
+for v in $envvars
+do
+ if [[ -z $$v ]]; then echo "$v is unset"; exit 1; fi
+done
+
+start() {
+ service traffic_ops start
+ exec tail -f /var/log/traffic_ops/traffic_ops.log
+}
+
+init() {
+ local postinstall_input_file="postinstall-input.json"
+ cat > "$postinstall_input_file" <<- ENDOFMESSAGE
+{
+ "/opt/traffic_ops/app/conf/production/database.conf":[
+ {
+ "Database type":"Pg",
+ "config_var":"type"
+ },
+ {
+ "Database name":"$DB_NAME",
+ "config_var":"dbname"
+ },
+ {
+ "Database server hostname IP or FQDN":"$DB_SERVER",
+ "config_var":"hostname"
+ },
+ {
+ "Database port number":"$DB_PORT",
+ "config_var":"port"
+ },
+ {
+ "Traffic Ops database user":"$DB_USER",
+ "config_var":"user"
+ },
+ {
+ "Traffic Ops database password":"$DB_USER_PASS",
+ "config_var":"password",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/app/db/dbconf.yml":[
+ {
+ "Database server root (admin) user":"postgres",
+ "config_var":"pgUser"
+ },
+ {
+ "Database server admin password":"$DB_ROOT_PASS",
+ "config_var":"pgPassword",
+ "hidden":"1"
+ },
+ {
+ "Download Maxmind Database?":"yes",
+ "config_var":"maxmind"
+ }
+ ],
+ "/opt/traffic_ops/app/conf/cdn.conf":[
+ {
+ "Generate a new secret?":"yes",
+ "config_var":"genSecret"
+ },
+ {
+ "Port to serve on?": "443",
+ "config_var": "port"
+ },
+ {
+ "Number of workers?": "12",
+ "config_var":"workers"
+ },
+ {
+ "Traffic Ops url?": "https://$TO_HOSTNAME",
+ "config_var": "base_url"
+ },
+ {
+ "Number of secrets to keep?":"1",
+ "config_var":"keepSecrets"
+ }
+ ],
+ "/opt/traffic_ops/app/conf/ldap.conf":[
+ {
+ "Do you want to set up LDAP?":"no",
+ "config_var":"setupLdap"
+ },
+ {
+ "LDAP server hostname":"",
+ "config_var":"host"
+ },
+ {
+ "LDAP Admin DN":"",
+ "config_var":"admin_dn"
+ },
+ {
+ "LDAP Admin Password":"",
+ "config_var":"admin_pass",
+ "hidden":"1"
+ },
+ {
+ "LDAP Search Base":"",
+ "config_var":"search_base"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/users.json":[
+ {
+ "Administration username for Traffic Ops":"$TO_ADMIN_USER",
+ "config_var":"tmAdminUser"
+ },
+ {
+ "Password for the admin user":"$TO_ADMIN_PASS",
+ "config_var":"tmAdminPw",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/install/data/profiles/":[
+ {
+ "Add custom profiles?":"no",
+ "config_var":"custom_profiles"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/openssl_configuration.json":[
+ {
+ "Do you want to generate a certificate?":"yes",
+ "config_var":"genCert"
+ },
+ {
+ "Country Name (2 letter code)":"$CERT_COUNTRY",
+ "config_var":"country"
+ },
+ {
+ "State or Province Name (full name)":"$CERT_STATE",
+ "config_var":"state"
+ },
+ {
+ "Locality Name (eg, city)":"$CERT_CITY",
+ "config_var":"locality"
+ },
+ {
+ "Organization Name (eg, company)":"$CERT_COMPANY",
+ "config_var":"company"
+ },
+ {
+ "Organizational Unit Name (eg, section)":"",
+ "config_var":"org_unit"
+ },
+ {
+ "Common Name (eg, your name or your server's hostname)":"$TO_HOSTNAME",
+ "config_var":"common_name"
+ },
+ {
+ "RSA Passphrase":"passphrase",
+ "config_var":"rsaPassword",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/profiles.json":[
+ {
+ "Traffic Ops url":"https://$TO_HOSTNAME",
+ "config_var":"tm.url"
+ },
+ {
+ "Human-readable CDN Name. (No whitespace, please)":"cdn",
+ "config_var":"cdn_name"
+ },
+ {
+ "Health Polling Interval (milliseconds)":"8000",
+ "config_var":"health_polling_int"
+ },
+ {
+ "DNS sub-domain for which your CDN is authoritative":"$TO_HOSTNAME.$TO_DOMAIN",
+ "config_var":"dns_subdomain"
+ },
+ {
+ "TLD SOA admin":"traffic_ops",
+ "config_var":"soa_admin"
+ },
+ {
+ "TrafficServer Drive Prefix":"/dev/ram",
+ "config_var":"driver_prefix"
+ },
+ {
+ "TrafficServer RAM Drive Prefix":"/dev/ram",
+ "config_var":"ram_drive_prefix"
+ },
+ {
+ "TrafficServer RAM Drive Letters (comma separated)":"1",
+ "config_var":"ram_drive_letters"
+ },
+ {
+ "Health Threshold Load Average":"25",
+ "config_var":"health_thresh_load_avg"
+ },
+ {
+ "Health Threshold Available Bandwidth in Kbps":"1750000",
+ "config_var":"health_thresh_kbps"
+ },
+ {
+ "Traffic Server Health Connection Timeout (milliseconds)":"2000",
+ "config_var":"health_connect_timeout"
+ }
+ ]
+}
+ENDOFMESSAGE
+
+echo -e "{\n\t\"user\": \"riakuser\",\n\t\"password\": \"$RIAK_USER_PASS\",\n\t\"MaxTLSVersion\": \"1.1\",\n\t\"tlsConfig\": {\n\t\t\"insecureSkipVerify\": true\n\t}\n}" > /opt/traffic_ops/app/conf/production/riak.conf
+
+ # TODO determine if term, user are necessary
+ export TERM=xterm && export USER=root && /opt/traffic_ops/install/bin/postinstall -cfile "$postinstall_input_file"
+
+ # Only listen on IPv4, not IPv6, because Docker doesn't provide a v6 interface by default. See http://mojolicious.org/perldoc/Mojo/Server/Daemon#listen
+ sed -i -e 's#https://\[::\]#https://127\.0\.0\.1#' /opt/traffic_ops/app/conf/cdn.conf
+ service traffic_ops restart
+
+ TRAFFIC_OPS_URI="https://localhost"
+
+ TMP_TO_COOKIE="$(curl -v -s -k -X POST --data '{ "u":"'"$TO_ADMIN_USER"'", "p":"'"$TO_ADMIN_PASS"'" }' $TRAFFIC_OPS_URI/api/1.2/user/login 2>&1 | grep 'Set-Cookie' | sed -e 's/.*mojolicious=\(.*\); expires.*/\1/')"
+ echo "Got cookie: $TMP_TO_COOKIE"
+
+ TMP_DOMAIN=$TO_DOMAIN
+ sed -i -- "s/{{.Domain}}/$TMP_DOMAIN/g" /profile.origin.traffic_ops
+ echo "Got domain: $TMP_DOMAIN"
+
+ echo "Importing origin"
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" -F "filename=profile.origin.traffic_ops" -F "profile_to_import=@/profile.origin.traffic_ops" $TRAFFIC_OPS_URI/profile/doImport
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "division.name=East" $TRAFFIC_OPS_URI/division/create
+ TMP_DIVISION_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/region/add | grep --color=never -oE "<option value=\"[0-9]+\">East</option>" | grep --color=never -oE "[0-9]+")"
+ echo "Got division ID: $TMP_DIVISION_ID"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "region.name=Eastish" --data-urlencode "region.division_id=$TMP_DIVISION_ID" $TRAFFIC_OPS_URI/region/create
+ TMP_REGION_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/regions.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="Eastish"]; print match[0]')"
+ echo "Got region ID: $TMP_REGION_ID"
+
+ TMP_CACHEGROUP_TYPE="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/types.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="MID_LOC"]; print match[0]')"
+ echo "Got cachegroup type ID: $TMP_CACHEGROUP_TYPE"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "cg_data.name=mid-east" --data-urlencode "cg_data.short_name=east" --data-urlencode "cg_data.latitude=0" --data-urlencode "cg_data.longitude=0" --data-urlencode "cg_data.parent_cachegroup_id=-1" --data-urlencode "cg_data.type=$TMP_CACHEGROUP_TYPE" $TRAFFIC_OPS_URI/cachegroup/create
+ TMP_CACHEGROUP_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/cachegroups.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="mid-east"]; print match[0]')"
+ echo "Got cachegroup ID: $TMP_CACHEGROUP_ID"
+
+ TMP_CACHEGROUP_EDGE_TYPE="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/types.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="EDGE_LOC"]; print match[0]')"
+ echo "Got cachegroup type ID: $TMP_CACHEGROUP_EDGE_TYPE"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "cg_data.name=edge-east" --data-urlencode "cg_data.short_name=eeast" --data-urlencode "cg_data.latitude=0" --data-urlencode "cg_data.longitude=0" --data-urlencode "cg_data.parent_cachegroup_id=$TMP_CACHEGROUP_ID" --data-urlencode "cg_data.type=$TMP_CACHEGROUP_EDGE_TYPE" $TRAFFIC_OPS_URI/cachegroup/create
+ TMP_CACHEGROUP_EDGE_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/cachegroups.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="edge-east"]; print match[0]')"
+ echo "Got cachegroup edge ID: $TMP_CACHEGROUP_EDGE_ID"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "location.name=plocation-nyc-1" --data-urlencode "location.short_name=nyc" --data-urlencode "location.address=1 Main Street" --data-urlencode "location.city=nyc" --data-urlencode "location.state=NY" --data-urlencode "location.zip=12345" --data-urlencode "location.poc=" --data-urlencode "location.phone=" --data-urlencode "location.email=no@no.no" --data-urlencode "location.comments=" --data-urlencode "location.region=$TMP_REGION_ID" $TRAFFIC_OPS_URI/phys_location/create
Review comment:
I would vote for standard `curl` over the "to-access" scripts. `curl` is a ubiquitous tool, it's something people already know, versus having to learn a new script when they're new to the project. It'd be understandable if the standard was unreasonably difficult, or the script was drastically easier to use. But it isn't, `curl -b $mc` is easy.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r556508949
##########
File path: traffic_ops_ort/testing/README.md
##########
@@ -0,0 +1,102 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Traffic Ops ORT Tests
+
+The ORT tests are used to validate the ORT tools used with a
+release of Traffic Ops. The tests insure that the ORT tools
Review comment:
nit: *en*sure, not *in*sure
##########
File path: traffic_ops_ort/testing/README.md
##########
@@ -0,0 +1,102 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Traffic Ops ORT Tests
+
+The ORT tests are used to validate the ORT tools used with a
+release of Traffic Ops. The tests insure that the ORT tools
+are able to communicate with Traffic Ops, install required
+packages and generate the correct ATS configuration files on
+a Trafficserver cache.
+
+The first thing you need do is to provide the Traffic Ops and
+Traffic Ops ORT RPM files for the build you wish to test using
+this framework. This test environment provides the necessary
+Docker containers used to support and execute the tests when
+given the necessary RPM's. If you choose to not use the provided
+Docker containers you will need to provide the following resources:
+
+ - A running Traffic Ops with the installed release to be tested
+ - A running Postgres SQL database which is loaded with the proper
+ test data for the Traffic Ops release.
+ - An Apache Traffic Server host that has the installed release of
+ ORT to be tested against the release of Traffic Ops.
+ - A yum server configured to provide the test rpm's herein.
+ - A Traffic vault server.
+
+# Directory layout
+
+ - trafficcontrol/traffic_ops_ort/testing/docker: has all the
+ necessary files for running the test Docker containers.
+ - trafficcontrol/traffic_ops_ort/testing/ort-tests: this directory.
+ contains all the go files used to run the ORT tests.
+
+# Setup.
+
+ 1. Build the Traffic Ops and Traffic Ops ORT RPM's that you wish
+ to test. See the top level 'build' directory for building
+ instructions.
+ 2. Copy the Traffic Ops RPM to docker/trafffic_ops/traffic_ops.rpm
+ (NOTE: Use the file name 'traffic_ops.rpm')
+ 3. Copy the Traffic Ops ORT rpm to docker/ats_edge/traffic_ops_ort.rpm
+ (NOTE: Use the file name 'traffic_ops_ort.rpm'
+ 4. Copy the recent copy of riak-2.2.3-1.el7.centos.x86_64.rpm to
+ docker/traffic_vault/riak-2.2.3-1.el7.centos.x86_64.rpm
+ 5. Copy an Apache Trafficserver rpm to
+ docker/yumserver/test-rpms/trafficserver-$VERSION.$COMMIT-HASH.el7.x86_64.rpm
+ You will need to edit and adjust the trafficserver package value in
+ ort-tests/tc-fixtures.json to match the $VERSION.$COMMIT-HASH used in the name
+ of your RPM. Search for '8.0.8-19.77cb23a' in the ort-tests/tc-fixtures.json
+ and change the value to match the RPM version you choose to use.
+ For example, the current value in tc-fixtures.json is '8.0.8-19.77cb23a' and
+ therefore the rpm file expected in 'docker/yumserver/test-rpms' is
+ 'trafficserver-8.0.8-19.77cb23a.el7.x86_64.rpm'.
+ 6. The container Docker files have the usernames and passwords used in the various
+ containers ie, postgresql db, traffic_ops, and traffic_ops_ort. The usernames
+ and passwords passed to the 't3c' executable in in the
+ ort-tests/conf/docker-edge-cache.conf file. Make sure that the usernames/passwords
+ in the Docker files match those in the t3c configuration file.
+ An example ort-tests/conf/edge-cache.conf file is provided should you choose to
+ use your own Traffic Ops and Postgresql environment.
+ 7. Build the Docker images and start the containers:
+ ```
+ # cd trafficcontrol/traffic_ops_ort/testing/docker
+ # docker-compose build
+ # docker-compose up -d
+ ```
Review comment:
the `#` prompt sorta makes this look like they're comments. You can also get syntax highlighting with a specifier like `shell`:
```shell
cd trafficcontrol/traffic_ops_ort/testing/docker
docker-compose build
docker-compose up -d
```
##########
File path: traffic_ops_ort/testing/docker/yumserver/test-rpms/README.md
##########
@@ -0,0 +1 @@
+# test rpms
Review comment:
Missing license header
##########
File path: traffic_ops_ort/testing/README.md
##########
@@ -0,0 +1,102 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Traffic Ops ORT Tests
+
+The ORT tests are used to validate the ORT tools used with a
+release of Traffic Ops. The tests insure that the ORT tools
+are able to communicate with Traffic Ops, install required
+packages and generate the correct ATS configuration files on
+a Trafficserver cache.
+
+The first thing you need do is to provide the Traffic Ops and
+Traffic Ops ORT RPM files for the build you wish to test using
+this framework. This test environment provides the necessary
+Docker containers used to support and execute the tests when
+given the necessary RPM's. If you choose to not use the provided
+Docker containers you will need to provide the following resources:
+
+ - A running Traffic Ops with the installed release to be tested
+ - A running Postgres SQL database which is loaded with the proper
+ test data for the Traffic Ops release.
+ - An Apache Traffic Server host that has the installed release of
+ ORT to be tested against the release of Traffic Ops.
+ - A yum server configured to provide the test rpm's herein.
+ - A Traffic vault server.
+
+# Directory layout
+
+ - trafficcontrol/traffic_ops_ort/testing/docker: has all the
+ necessary files for running the test Docker containers.
+ - trafficcontrol/traffic_ops_ort/testing/ort-tests: this directory.
+ contains all the go files used to run the ORT tests.
+
+# Setup.
+
+ 1. Build the Traffic Ops and Traffic Ops ORT RPM's that you wish
+ to test. See the top level 'build' directory for building
+ instructions.
+ 2. Copy the Traffic Ops RPM to docker/trafffic_ops/traffic_ops.rpm
+ (NOTE: Use the file name 'traffic_ops.rpm')
+ 3. Copy the Traffic Ops ORT rpm to docker/ats_edge/traffic_ops_ort.rpm
+ (NOTE: Use the file name 'traffic_ops_ort.rpm'
+ 4. Copy the recent copy of riak-2.2.3-1.el7.centos.x86_64.rpm to
+ docker/traffic_vault/riak-2.2.3-1.el7.centos.x86_64.rpm
+ 5. Copy an Apache Trafficserver rpm to
+ docker/yumserver/test-rpms/trafficserver-$VERSION.$COMMIT-HASH.el7.x86_64.rpm
+ You will need to edit and adjust the trafficserver package value in
+ ort-tests/tc-fixtures.json to match the $VERSION.$COMMIT-HASH used in the name
+ of your RPM. Search for '8.0.8-19.77cb23a' in the ort-tests/tc-fixtures.json
+ and change the value to match the RPM version you choose to use.
+ For example, the current value in tc-fixtures.json is '8.0.8-19.77cb23a' and
+ therefore the rpm file expected in 'docker/yumserver/test-rpms' is
+ 'trafficserver-8.0.8-19.77cb23a.el7.x86_64.rpm'.
+ 6. The container Docker files have the usernames and passwords used in the various
+ containers ie, postgresql db, traffic_ops, and traffic_ops_ort. The usernames
+ and passwords passed to the 't3c' executable in in the
+ ort-tests/conf/docker-edge-cache.conf file. Make sure that the usernames/passwords
+ in the Docker files match those in the t3c configuration file.
+ An example ort-tests/conf/edge-cache.conf file is provided should you choose to
+ use your own Traffic Ops and Postgresql environment.
+ 7. Build the Docker images and start the containers:
+ ```
+ # cd trafficcontrol/traffic_ops_ort/testing/docker
+ # docker-compose build
+ # docker-compose up -d
+ ```
+ After some time, test results should be available at
+ 'ort-tests/test.log'
+
+ If you wish to run the tests manually use 'docker ps' to obtain the container id for
+ the ats_edge host and then:
+
+ ```
+ # docker exec -it $ats_edge_container_id /bin/bash -l
+ # cd /ort-tests
+ # go test -cfg=conf/docker-edge-cache.conf
+ ```
Review comment:
Same as above but also it looks like you're missing some indentation here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564817922
##########
File path: traffic_ops_ort/testing/docker/traffic_vault/functions
##########
@@ -0,0 +1,707 @@
+# -*-Shell-script-*-
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# functions This file contains functions to be used by most or all
+# shell scripts in the /etc/init.d directory.
+#
+
+TEXTDOMAIN=initscripts
+
+# Make sure umask is sane
+umask 022
+
+# Set up a default search path.
+PATH="/sbin:/usr/sbin:/bin:/usr/bin"
+export PATH
+
+if [ $PPID -ne 1 -a -z "$SYSTEMCTL_SKIP_REDIRECT" ] && \
+ [ -d /run/systemd/system ] ; then
+ case "$0" in
+ /etc/init.d/*|/etc/rc.d/init.d/*)
+ _use_systemctl=1
+ ;;
+ esac
+fi
+
+systemctl_redirect () {
+ local s
+ local prog=${1##*/}
+ local command=$2
+ local options=""
+
+ case "$command" in
+ start)
+ s=$"Starting $prog (via systemctl): "
+ ;;
+ stop)
+ s=$"Stopping $prog (via systemctl): "
+ ;;
+ reload|try-reload)
+ s=$"Reloading $prog configuration (via systemctl): "
+ ;;
+ restart|try-restart|condrestart)
+ s=$"Restarting $prog (via systemctl): "
Review comment:
@ocket8888 yeah, I copied that from somewhere and didn't notice the '$'. I'll fix it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford edited a comment on pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford edited a comment on pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#issuecomment-767805861
> Before I look through all the rest of this, did you consider re-using the CiaB architecture at all? It seems like there might be some duplicated pieces here, esp. for Riak
@ocket8888 I'm not going to use the CiaB Dockerfiles as I don't need the enrollee. As I mentioned to you when work on the automation piece I might redo the RIAK container to use the image from CiaB RIAK equivalent.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564848237
##########
File path: traffic_ops_ort/testing/docker/yumserver/Dockerfile
##########
@@ -0,0 +1,35 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Traffic Server container images
+# as Edges for Traffic Control 1.4
+# Based on CentOS 6.6
+############################################################
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y httpd
+RUN yum install -y createrepo
+RUN yum install -y yum-utils
Review comment:
Thanks for the Dockerfile best practices @zrhoffman but you can save the lecture!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564900980
##########
File path: traffic_ops_ort/testing/docker/yumserver/Dockerfile
##########
@@ -0,0 +1,35 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Traffic Server container images
+# as Edges for Traffic Control 1.4
+# Based on CentOS 6.6
+############################################################
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y httpd
+RUN yum install -y createrepo
+RUN yum install -y yum-utils
Review comment:
I thank you and my hard drive thanks you
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564885469
##########
File path: traffic_ops_ort/testing/docker/ort_test/Dockerfile
##########
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Traffic Server container images
+# as Edges for Traffic Control 1.4
+# Based on CentOS 6.6
+############################################################
+
+# For cache, you may either use (RAM or disk) block devices or disk directories
+# To use RAM block devices, pass them as /dev/ram0 and /dev/ram1 via `docker run --device`
+# To use disk directories, simply don't pass devices, and the container will configure Traffic Server for directories
+
+# Block devices may be created on the native machine with, for example, `modprobe brd`.
+# The recommended minimum size for each block devices is 1G.
+# For example, `sudo modprobe brd rd_size=1048576 rd_nr=2`
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y epel-release # needed for perl-Crypt-ScryptKDF
+RUN yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
+RUN yum install -y initscripts
+RUN yum install -y postgresql96.x86_64
+RUN yum install -y git
+RUN yum install -y gcc
+RUN yum install -y lua-5.1.4-15.el7
+RUN yum install -y lua-devel-5.1.4-15.el7
Review comment:
well at any rate it's being discussed below
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
rob05c commented on pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#issuecomment-759783934
@jrushford I Approved, and it looks like everything is passing except Weasel.
Looks like this line needs changed: https://github.com/apache/trafficcontrol/blob/master/LICENSE#L385
and the line following it, to remove `/traffic_ops`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r563807782
##########
File path: traffic_ops_ort/testing/docker/yumserver/test-rpms/README.md
##########
@@ -0,0 +1 @@
+# test rpms
Review comment:
@ocket8888 this is fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford edited a comment on pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford edited a comment on pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#issuecomment-768591327
> Tests don't appear to pass: https://pastebin.com/xFtQq1eh
@ocket8888 Where was that ATS RPM from? Did you grab one from the CDN repo? Here is the error, that RPM has an unsatisfied dependency on libmagick. Try an RPM from the CDN repo such as, trafficserver-8.1.0-41.f25b59e.el7.x86_64.rpm
There is a lot in that log but, the problem:
`ERROR: t3c.go:122: 2021-01-27T21:15:05.25734678Z: Error processing packages: Unable to install trafficserver-8.0.3-1.el7.x86_64 : Error executing '/usr/bin/yum': Error: Package: trafficserver-8.0.3-1.el7.x86_64 (traffic-control)
Requires: libMagickCore.so.5()(64bit)
Error: Package: trafficserver-8.0.3-1.el7.x86_64 (traffic-control)
Requires: libMagick++.so.5()(64bit)`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564856952
##########
File path: traffic_ops_ort/testing/README.md
##########
@@ -0,0 +1,102 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Traffic Ops ORT Tests
+
+The ORT tests are used to validate the ORT tools used with a
+release of Traffic Ops. The tests ensure that the ORT tools
+are able to communicate with Traffic Ops, install required
+packages and generate the correct ATS configuration files on
+a Trafficserver cache.
+
+The first thing you need do is to provide the Traffic Ops and
+Traffic Ops ORT RPM files for the build you wish to test using
+this framework. This test environment provides the necessary
+Docker containers used to support and execute the tests when
+given the necessary RPM's. If you choose to not use the provided
+Docker containers you will need to provide the following resources:
+
+ - A running Traffic Ops with the installed release to be tested
+ - A running Postgres SQL database which is loaded with the proper
+ test data for the Traffic Ops release.
+ - An Apache Traffic Server host that has the installed release of
+ ORT to be tested against the release of Traffic Ops.
+ - A yum server configured to provide the test rpm's herein.
+ - A Traffic vault server.
+
+# Directory layout
+
+ - trafficcontrol/traffic_ops_ort/testing/docker: has all the
+ necessary files for running the test Docker containers.
+ - trafficcontrol/traffic_ops_ort/testing/ort-tests: this directory.
+ contains all the go files used to run the ORT tests.
+
+# Setup.
+
+ 1. Build the Traffic Ops and Traffic Ops ORT RPM's that you wish
+ to test. See the top level 'build' directory for building
+ instructions.
+ 2. Copy the Traffic Ops RPM to docker/trafffic_ops/traffic_ops.rpm
+ (NOTE: Use the file name 'traffic_ops.rpm')
+ 3. Copy the Traffic Ops ORT rpm to docker/ort_test/traffic_ops_ort.rpm
+ (NOTE: Use the file name 'traffic_ops_ort.rpm'
+ 4. Copy the recent copy of riak-2.2.3-1.el7.centos.x86_64.rpm to
+ docker/traffic_vault/riak-2.2.3-1.el7.centos.x86_64.rpm
+ 5. Copy an Apache Trafficserver rpm to
+ docker/yumserver/test-rpms/trafficserver-$VERSION.$COMMIT-HASH.el7.x86_64.rpm
+ You will need to edit and adjust the trafficserver package value in
+ ort-tests/tc-fixtures.json to match the $VERSION.$COMMIT-HASH used in the name
+ of your RPM. Search for '8.0.8-19.77cb23a' in the ort-tests/tc-fixtures.json
+ and change the value to match the RPM version you choose to use.
+ For example, the current value in tc-fixtures.json is '8.0.8-19.77cb23a' and
+ therefore the rpm file expected in 'docker/yumserver/test-rpms' is
+ 'trafficserver-8.0.8-19.77cb23a.el7.x86_64.rpm'.
Review comment:
The text "8.0.8-19.77cb23a" does not appear in `tc-fixtures.json`
```shell-session
ocket8888@system76  testing  ➦ 63ba71cf1  grep "8.0.8" ./ort-tests/tc-fixtures.json
✘ ocket8888@system76  testing  ➦ 63ba71cf1 
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 merged pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
ocket8888 merged pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564890996
##########
File path: traffic_ops_ort/testing/docker/yumserver/Dockerfile
##########
@@ -0,0 +1,35 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Traffic Server container images
+# as Edges for Traffic Control 1.4
+# Based on CentOS 6.6
+############################################################
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y httpd
+RUN yum install -y createrepo
+RUN yum install -y yum-utils
Review comment:
I'll change it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564885262
##########
File path: traffic_ops_ort/testing/docker/traffic_ops/run.sh
##########
@@ -0,0 +1,306 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Script for running the Dockerfile for Traffic Ops.
+# The Dockerfile sets up a Docker image which can be used for any new Traffic Ops container;
+# This script, which should be run when the container is run (it's the ENTRYPOINT), will configure the container.
+#
+# The following environment variables must be set, ordinarily by `docker run -e` arguments:
+# DB_SERVER
+# DB_PORT
+# DB_ROOT_PASS
+# DB_USER
+# DB_USER_PASS
+# DB_NAME
+# TO_ADMIN_USER
+# TO_ADMIN_PASS
+# CERT_COUNTRY
+# CERT_STATE
+# CERT_CITY
+# CERT_COMPANY
+# TO_DOMAIN
+# TRAFFIC_VAULT_PASS
+
+# Check that env vars are set
+envvars=( DB_SERVER DB_PORT DB_ROOT_PASS DB_USER DB_USER_PASS TO_ADMIN_USER TO_ADMIN_PASS CERT_COUNTRY CERT_STATE CERT_CITY CERT_COMPANY TO_DOMAIN)
+for v in $envvars
+do
+ if [[ -z $$v ]]; then echo "$v is unset"; exit 1; fi
+done
+
+start() {
+ service traffic_ops start
+ exec tail -f /var/log/traffic_ops/traffic_ops.log
+}
+
+init() {
+ local postinstall_input_file="postinstall-input.json"
+ cat > "$postinstall_input_file" <<- ENDOFMESSAGE
+{
+ "/opt/traffic_ops/app/conf/production/database.conf":[
+ {
+ "Database type":"Pg",
+ "config_var":"type"
+ },
+ {
+ "Database name":"$DB_NAME",
+ "config_var":"dbname"
+ },
+ {
+ "Database server hostname IP or FQDN":"$DB_SERVER",
+ "config_var":"hostname"
+ },
+ {
+ "Database port number":"$DB_PORT",
+ "config_var":"port"
+ },
+ {
+ "Traffic Ops database user":"$DB_USER",
+ "config_var":"user"
+ },
+ {
+ "Traffic Ops database password":"$DB_USER_PASS",
+ "config_var":"password",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/app/db/dbconf.yml":[
+ {
+ "Database server root (admin) user":"postgres",
+ "config_var":"pgUser"
+ },
+ {
+ "Database server admin password":"$DB_ROOT_PASS",
+ "config_var":"pgPassword",
+ "hidden":"1"
+ },
+ {
+ "Download Maxmind Database?":"yes",
+ "config_var":"maxmind"
+ }
+ ],
+ "/opt/traffic_ops/app/conf/cdn.conf":[
+ {
+ "Generate a new secret?":"yes",
+ "config_var":"genSecret"
+ },
+ {
+ "Port to serve on?": "443",
+ "config_var": "port"
+ },
+ {
+ "Number of workers?": "12",
+ "config_var":"workers"
+ },
+ {
+ "Traffic Ops url?": "https://$TO_HOSTNAME",
+ "config_var": "base_url"
+ },
+ {
+ "Number of secrets to keep?":"1",
+ "config_var":"keepSecrets"
+ }
+ ],
+ "/opt/traffic_ops/app/conf/ldap.conf":[
+ {
+ "Do you want to set up LDAP?":"no",
+ "config_var":"setupLdap"
+ },
+ {
+ "LDAP server hostname":"",
+ "config_var":"host"
+ },
+ {
+ "LDAP Admin DN":"",
+ "config_var":"admin_dn"
+ },
+ {
+ "LDAP Admin Password":"",
+ "config_var":"admin_pass",
+ "hidden":"1"
+ },
+ {
+ "LDAP Search Base":"",
+ "config_var":"search_base"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/users.json":[
+ {
+ "Administration username for Traffic Ops":"$TO_ADMIN_USER",
+ "config_var":"tmAdminUser"
+ },
+ {
+ "Password for the admin user":"$TO_ADMIN_PASS",
+ "config_var":"tmAdminPw",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/install/data/profiles/":[
+ {
+ "Add custom profiles?":"no",
+ "config_var":"custom_profiles"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/openssl_configuration.json":[
+ {
+ "Do you want to generate a certificate?":"yes",
+ "config_var":"genCert"
+ },
+ {
+ "Country Name (2 letter code)":"$CERT_COUNTRY",
+ "config_var":"country"
+ },
+ {
+ "State or Province Name (full name)":"$CERT_STATE",
+ "config_var":"state"
+ },
+ {
+ "Locality Name (eg, city)":"$CERT_CITY",
+ "config_var":"locality"
+ },
+ {
+ "Organization Name (eg, company)":"$CERT_COMPANY",
+ "config_var":"company"
+ },
+ {
+ "Organizational Unit Name (eg, section)":"",
+ "config_var":"org_unit"
+ },
+ {
+ "Common Name (eg, your name or your server's hostname)":"$TO_HOSTNAME",
+ "config_var":"common_name"
+ },
+ {
+ "RSA Passphrase":"passphrase",
+ "config_var":"rsaPassword",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/profiles.json":[
+ {
+ "Traffic Ops url":"https://$TO_HOSTNAME",
+ "config_var":"tm.url"
+ },
+ {
+ "Human-readable CDN Name. (No whitespace, please)":"cdn",
+ "config_var":"cdn_name"
+ },
+ {
+ "Health Polling Interval (milliseconds)":"8000",
+ "config_var":"health_polling_int"
+ },
+ {
+ "DNS sub-domain for which your CDN is authoritative":"$TO_HOSTNAME.$TO_DOMAIN",
+ "config_var":"dns_subdomain"
+ },
+ {
+ "TLD SOA admin":"traffic_ops",
+ "config_var":"soa_admin"
+ },
+ {
+ "TrafficServer Drive Prefix":"/dev/ram",
+ "config_var":"driver_prefix"
+ },
+ {
+ "TrafficServer RAM Drive Prefix":"/dev/ram",
+ "config_var":"ram_drive_prefix"
+ },
+ {
+ "TrafficServer RAM Drive Letters (comma separated)":"1",
+ "config_var":"ram_drive_letters"
+ },
+ {
+ "Health Threshold Load Average":"25",
+ "config_var":"health_thresh_load_avg"
+ },
+ {
+ "Health Threshold Available Bandwidth in Kbps":"1750000",
+ "config_var":"health_thresh_kbps"
+ },
+ {
+ "Traffic Server Health Connection Timeout (milliseconds)":"2000",
+ "config_var":"health_connect_timeout"
+ }
+ ]
+}
+ENDOFMESSAGE
+
+echo -e "{\n\t\"user\": \"riakuser\",\n\t\"password\": \"$RIAK_USER_PASS\",\n\t\"MaxTLSVersion\": \"1.1\",\n\t\"tlsConfig\": {\n\t\t\"insecureSkipVerify\": true\n\t}\n}" > /opt/traffic_ops/app/conf/production/riak.conf
+
+ # TODO determine if term, user are necessary
+ export TERM=xterm && export USER=root && /opt/traffic_ops/install/bin/postinstall -cfile "$postinstall_input_file"
+
+ # Only listen on IPv4, not IPv6, because Docker doesn't provide a v6 interface by default. See http://mojolicious.org/perldoc/Mojo/Server/Daemon#listen
+ sed -i -e 's#https://\[::\]#https://127\.0\.0\.1#' /opt/traffic_ops/app/conf/cdn.conf
+ service traffic_ops restart
+
+ TRAFFIC_OPS_URI="https://localhost"
+
+ TMP_TO_COOKIE="$(curl -v -s -k -X POST --data '{ "u":"'"$TO_ADMIN_USER"'", "p":"'"$TO_ADMIN_PASS"'" }' $TRAFFIC_OPS_URI/api/1.2/user/login 2>&1 | grep 'Set-Cookie' | sed -e 's/.*mojolicious=\(.*\); expires.*/\1/')"
+ echo "Got cookie: $TMP_TO_COOKIE"
+
+ TMP_DOMAIN=$TO_DOMAIN
+ sed -i -- "s/{{.Domain}}/$TMP_DOMAIN/g" /profile.origin.traffic_ops
+ echo "Got domain: $TMP_DOMAIN"
+
+ echo "Importing origin"
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" -F "filename=profile.origin.traffic_ops" -F "profile_to_import=@/profile.origin.traffic_ops" $TRAFFIC_OPS_URI/profile/doImport
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "division.name=East" $TRAFFIC_OPS_URI/division/create
+ TMP_DIVISION_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/region/add | grep --color=never -oE "<option value=\"[0-9]+\">East</option>" | grep --color=never -oE "[0-9]+")"
+ echo "Got division ID: $TMP_DIVISION_ID"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "region.name=Eastish" --data-urlencode "region.division_id=$TMP_DIVISION_ID" $TRAFFIC_OPS_URI/region/create
+ TMP_REGION_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/regions.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="Eastish"]; print match[0]')"
+ echo "Got region ID: $TMP_REGION_ID"
+
+ TMP_CACHEGROUP_TYPE="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/types.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="MID_LOC"]; print match[0]')"
+ echo "Got cachegroup type ID: $TMP_CACHEGROUP_TYPE"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "cg_data.name=mid-east" --data-urlencode "cg_data.short_name=east" --data-urlencode "cg_data.latitude=0" --data-urlencode "cg_data.longitude=0" --data-urlencode "cg_data.parent_cachegroup_id=-1" --data-urlencode "cg_data.type=$TMP_CACHEGROUP_TYPE" $TRAFFIC_OPS_URI/cachegroup/create
+ TMP_CACHEGROUP_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/cachegroups.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="mid-east"]; print match[0]')"
+ echo "Got cachegroup ID: $TMP_CACHEGROUP_ID"
+
+ TMP_CACHEGROUP_EDGE_TYPE="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/types.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="EDGE_LOC"]; print match[0]')"
+ echo "Got cachegroup type ID: $TMP_CACHEGROUP_EDGE_TYPE"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "cg_data.name=edge-east" --data-urlencode "cg_data.short_name=eeast" --data-urlencode "cg_data.latitude=0" --data-urlencode "cg_data.longitude=0" --data-urlencode "cg_data.parent_cachegroup_id=$TMP_CACHEGROUP_ID" --data-urlencode "cg_data.type=$TMP_CACHEGROUP_EDGE_TYPE" $TRAFFIC_OPS_URI/cachegroup/create
+ TMP_CACHEGROUP_EDGE_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/cachegroups.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="edge-east"]; print match[0]')"
+ echo "Got cachegroup edge ID: $TMP_CACHEGROUP_EDGE_ID"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "location.name=plocation-nyc-1" --data-urlencode "location.short_name=nyc" --data-urlencode "location.address=1 Main Street" --data-urlencode "location.city=nyc" --data-urlencode "location.state=NY" --data-urlencode "location.zip=12345" --data-urlencode "location.poc=" --data-urlencode "location.phone=" --data-urlencode "location.email=no@no.no" --data-urlencode "location.comments=" --data-urlencode "location.region=$TMP_REGION_ID" $TRAFFIC_OPS_URI/phys_location/create
Review comment:
@jrushford It's linked there in the comment, just click on it and it'll show you where it is.
It's not really the cookie that's bothering me, it's the `--data-urlencode` for every field. But come to think of it that doesn't necessarily require`to-access.sh` either, you can pass the JSON either way - which should've been obvious since I helped write `to-access.sh` and it uses `curl` anyway.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564902756
##########
File path: traffic_ops_ort/testing/docker/yumserver/Dockerfile
##########
@@ -0,0 +1,35 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Traffic Server container images
+# as Edges for Traffic Control 1.4
+# Based on CentOS 6.6
+############################################################
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y httpd
+RUN yum install -y createrepo
+RUN yum install -y yum-utils
Review comment:
interesting, run a git blame on infrastructure/docker/traffic_ops/run.sh
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#issuecomment-767804464
> Go files that are meant to run when you use `go test` should have the `_test.go` suffix
Are you talking about the go files in testing/ort-tests/tcdata? I don't want them run as tests. I'm using them to load test fixtures data and when they fail to load some portion of the fixtures data I'll know about it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564902756
##########
File path: traffic_ops_ort/testing/docker/yumserver/Dockerfile
##########
@@ -0,0 +1,35 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Traffic Server container images
+# as Edges for Traffic Control 1.4
+# Based on CentOS 6.6
+############################################################
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y httpd
+RUN yum install -y createrepo
+RUN yum install -y yum-utils
Review comment:
interesting, run a git blame on infrastructure/docker/traffic_ops/Dockerfile
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford edited a comment on pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford edited a comment on pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#issuecomment-768591327
> Tests don't appear to pass: https://pastebin.com/xFtQq1eh
@ocket8888 Where was that ATS RPM from? Did you grab one from the CDN repo? Here is the error, that RPM has an unsatisfied dependency on libmagic. Try an RPM from the CDN repo such as, trafficserver-8.1.0-41.f25b59e.el7.x86_64.rpm
There is a lot in that log but, the problem:
`ERROR: t3c.go:122: 2021-01-27T21:15:05.25734678Z: Error processing packages: Unable to install trafficserver-8.0.3-1.el7.x86_64 : Error executing '/usr/bin/yum': Error: Package: trafficserver-8.0.3-1.el7.x86_64 (traffic-control)
Requires: libMagickCore.so.5()(64bit)
Error: Package: trafficserver-8.0.3-1.el7.x86_64 (traffic-control)
Requires: libMagick++.so.5()(64bit)`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#issuecomment-767805861
> Before I look through all the rest of this, did you consider re-using the CiaB architecture at all? It seems like there might be some duplicated pieces here, esp. for Riak
I'm not going to use the CiaB Dockerfiles as I don't need the enrollee. As I mentioned to you when work on the automation piece I might redo the RIAK container to use the image from CiaB RIAK equivalent.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 merged pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
ocket8888 merged pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#issuecomment-768583790
Tests don't appear to pass: https://pastebin.com/xFtQq1eh
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564937438
##########
File path: traffic_ops_ort/testing/README.md
##########
@@ -0,0 +1,102 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Traffic Ops ORT Tests
+
+The ORT tests are used to validate the ORT tools used with a
+release of Traffic Ops. The tests ensure that the ORT tools
+are able to communicate with Traffic Ops, install required
+packages and generate the correct ATS configuration files on
+a Trafficserver cache.
+
+The first thing you need do is to provide the Traffic Ops and
+Traffic Ops ORT RPM files for the build you wish to test using
+this framework. This test environment provides the necessary
+Docker containers used to support and execute the tests when
+given the necessary RPM's. If you choose to not use the provided
+Docker containers you will need to provide the following resources:
+
+ - A running Traffic Ops with the installed release to be tested
+ - A running Postgres SQL database which is loaded with the proper
+ test data for the Traffic Ops release.
+ - An Apache Traffic Server host that has the installed release of
+ ORT to be tested against the release of Traffic Ops.
+ - A yum server configured to provide the test rpm's herein.
+ - A Traffic vault server.
+
+# Directory layout
+
+ - trafficcontrol/traffic_ops_ort/testing/docker: has all the
+ necessary files for running the test Docker containers.
+ - trafficcontrol/traffic_ops_ort/testing/ort-tests: this directory.
+ contains all the go files used to run the ORT tests.
+
+# Setup.
+
+ 1. Build the Traffic Ops and Traffic Ops ORT RPM's that you wish
+ to test. See the top level 'build' directory for building
+ instructions.
+ 2. Copy the Traffic Ops RPM to docker/trafffic_ops/traffic_ops.rpm
+ (NOTE: Use the file name 'traffic_ops.rpm')
+ 3. Copy the Traffic Ops ORT rpm to docker/ort_test/traffic_ops_ort.rpm
+ (NOTE: Use the file name 'traffic_ops_ort.rpm'
+ 4. Copy the recent copy of riak-2.2.3-1.el7.centos.x86_64.rpm to
+ docker/traffic_vault/riak-2.2.3-1.el7.centos.x86_64.rpm
+ 5. Copy an Apache Trafficserver rpm to
+ docker/yumserver/test-rpms/trafficserver-$VERSION.$COMMIT-HASH.el7.x86_64.rpm
+ You will need to edit and adjust the trafficserver package value in
+ ort-tests/tc-fixtures.json to match the $VERSION.$COMMIT-HASH used in the name
+ of your RPM. Search for '8.0.8-19.77cb23a' in the ort-tests/tc-fixtures.json
+ and change the value to match the RPM version you choose to use.
+ For example, the current value in tc-fixtures.json is '8.0.8-19.77cb23a' and
+ therefore the rpm file expected in 'docker/yumserver/test-rpms' is
+ 'trafficserver-8.0.8-19.77cb23a.el7.x86_64.rpm'.
Review comment:
@ocket8888 resolved.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564812933
##########
File path: traffic_ops_ort/testing/docker/ort_test/Dockerfile
##########
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Traffic Server container images
+# as Edges for Traffic Control 1.4
+# Based on CentOS 6.6
+############################################################
+
+# For cache, you may either use (RAM or disk) block devices or disk directories
+# To use RAM block devices, pass them as /dev/ram0 and /dev/ram1 via `docker run --device`
+# To use disk directories, simply don't pass devices, and the container will configure Traffic Server for directories
+
+# Block devices may be created on the native machine with, for example, `modprobe brd`.
+# The recommended minimum size for each block devices is 1G.
+# For example, `sudo modprobe brd rd_size=1048576 rd_nr=2`
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y epel-release # needed for perl-Crypt-ScryptKDF
+RUN yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
+RUN yum install -y initscripts
+RUN yum install -y postgresql96.x86_64
+RUN yum install -y git
+RUN yum install -y gcc
+RUN yum install -y lua-5.1.4-15.el7
+RUN yum install -y lua-devel-5.1.4-15.el7
Review comment:
@ocket8888 it works fine like it is an it's easy to see what's loaded.
##########
File path: traffic_ops_ort/testing/docker/traffic_ops/Dockerfile
##########
@@ -0,0 +1,75 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+############################################################
+# Dockerfile to build Traffic Ops container images
+# Based on CentOS 7.2
+############################################################
+
+# Example Build and Run:
+# docker network create cdnet
+# docker build --rm --tag traffic_ops:1.7.0 --build-arg=RPM=http://traffic-control-cdn.net/downloads/1.7.0/RELEASE-1.7.0/traffic_ops-1.7.0-3908.5b77f60f.x86_64.rpm traffic_ops
+#
+# docker run --name my-traffic-ops-mysql --hostname my-traffic-ops-mysql --net cdnet --env MYSQL_ROOT_PASSWORD=secretrootpass --detach mysql:5.5
+#
+# docker run --name my-traffic-ops --hostname my-traffic-ops --net cdnet --publish 443:443 --env MYSQL_IP=my-traffic-ops-mysql --env MYSQL_PORT=3306 --env MYSQL_ROOT_PASS=secretrootpass --env MYSQL_TRAFFIC_OPS_PASS=supersecretpassword --env ADMIN_USER=superroot --env ADMIN_PASS=supersecreterpassward --env CERT_COUNTRY=US --env CERT_STATE=Colorado --env CERT_CITY=Denver --env CERT_COMPANY=NotComcast --env TRAFFIC_VAULT_PASS=marginallylesssecret --env DOMAIN=cdnet --detach traffic_ops:1.5.1
+
+FROM centos/systemd
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
+RUN yum install -y initscripts # needed for service command
+RUN yum install -y epel-release # needed for perl-Crypt-ScryptKDF
+RUN yum install -y perl-Crypt-ScryptKDF
+RUN yum install -y perl cpanminus perl-Test-CPAN-Meta
+RUN cpanm Carton
+RUN yum install -y perl-DBIx-Connector
Review comment:
@ocket8888 it works fine like it is an it's easy to see what's loaded.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564707441
##########
File path: traffic_ops_ort/testing/docker/traffic_ops/Dockerfile
##########
@@ -0,0 +1,75 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+############################################################
+# Dockerfile to build Traffic Ops container images
+# Based on CentOS 7.2
+############################################################
+
+# Example Build and Run:
+# docker network create cdnet
+# docker build --rm --tag traffic_ops:1.7.0 --build-arg=RPM=http://traffic-control-cdn.net/downloads/1.7.0/RELEASE-1.7.0/traffic_ops-1.7.0-3908.5b77f60f.x86_64.rpm traffic_ops
+#
+# docker run --name my-traffic-ops-mysql --hostname my-traffic-ops-mysql --net cdnet --env MYSQL_ROOT_PASSWORD=secretrootpass --detach mysql:5.5
+#
+# docker run --name my-traffic-ops --hostname my-traffic-ops --net cdnet --publish 443:443 --env MYSQL_IP=my-traffic-ops-mysql --env MYSQL_PORT=3306 --env MYSQL_ROOT_PASS=secretrootpass --env MYSQL_TRAFFIC_OPS_PASS=supersecretpassword --env ADMIN_USER=superroot --env ADMIN_PASS=supersecreterpassward --env CERT_COUNTRY=US --env CERT_STATE=Colorado --env CERT_CITY=Denver --env CERT_COMPANY=NotComcast --env TRAFFIC_VAULT_PASS=marginallylesssecret --env DOMAIN=cdnet --detach traffic_ops:1.5.1
+
+FROM centos/systemd
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
+RUN yum install -y initscripts # needed for service command
+RUN yum install -y epel-release # needed for perl-Crypt-ScryptKDF
+RUN yum install -y perl-Crypt-ScryptKDF
+RUN yum install -y perl cpanminus perl-Test-CPAN-Meta
+RUN cpanm Carton
+RUN yum install -y perl-DBIx-Connector
Review comment:
This could be two `yum` commands, the stuff that doesn't need `epel-release` and `epel-release` itself, followed by everything else.
##########
File path: traffic_ops_ort/testing/docker/ort_test/Dockerfile
##########
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Traffic Server container images
+# as Edges for Traffic Control 1.4
+# Based on CentOS 6.6
+############################################################
+
+# For cache, you may either use (RAM or disk) block devices or disk directories
+# To use RAM block devices, pass them as /dev/ram0 and /dev/ram1 via `docker run --device`
+# To use disk directories, simply don't pass devices, and the container will configure Traffic Server for directories
+
+# Block devices may be created on the native machine with, for example, `modprobe brd`.
+# The recommended minimum size for each block devices is 1G.
+# For example, `sudo modprobe brd rd_size=1048576 rd_nr=2`
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y epel-release # needed for perl-Crypt-ScryptKDF
+RUN yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
+RUN yum install -y initscripts
+RUN yum install -y postgresql96.x86_64
+RUN yum install -y git
+RUN yum install -y gcc
+RUN yum install -y lua-5.1.4-15.el7
+RUN yum install -y lua-devel-5.1.4-15.el7
Review comment:
These could all be a single command to prevent excessive caching.
##########
File path: traffic_ops_ort/testing/docker/traffic_vault/functions
##########
@@ -0,0 +1,707 @@
+# -*-Shell-script-*-
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# functions This file contains functions to be used by most or all
+# shell scripts in the /etc/init.d directory.
+#
+
+TEXTDOMAIN=initscripts
+
+# Make sure umask is sane
+umask 022
+
+# Set up a default search path.
+PATH="/sbin:/usr/sbin:/bin:/usr/bin"
+export PATH
+
+if [ $PPID -ne 1 -a -z "$SYSTEMCTL_SKIP_REDIRECT" ] && \
+ [ -d /run/systemd/system ] ; then
+ case "$0" in
+ /etc/init.d/*|/etc/rc.d/init.d/*)
+ _use_systemctl=1
+ ;;
+ esac
+fi
+
+systemctl_redirect () {
+ local s
+ local prog=${1##*/}
+ local command=$2
+ local options=""
+
+ case "$command" in
+ start)
+ s=$"Starting $prog (via systemctl): "
+ ;;
+ stop)
+ s=$"Stopping $prog (via systemctl): "
+ ;;
+ reload|try-reload)
+ s=$"Reloading $prog configuration (via systemctl): "
+ ;;
+ restart|try-restart|condrestart)
+ s=$"Restarting $prog (via systemctl): "
Review comment:
These `$`s are being added literally to `s`, which I'm not sure if you intended/care about. e.g.
```shell
prog="bash"
s=$"Starting $prog (via systemctl): "
echo "$s" # Outputs '$Starting bash (via systemctl): '
```
##########
File path: traffic_ops_ort/testing/docker/yumserver/Dockerfile
##########
@@ -0,0 +1,35 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Traffic Server container images
+# as Edges for Traffic Control 1.4
+# Based on CentOS 6.6
+############################################################
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y httpd
+RUN yum install -y createrepo
+RUN yum install -y yum-utils
Review comment:
These could all be a single `yum` command
##########
File path: traffic_ops_ort/testing/docker/traffic_ops/run.sh
##########
@@ -0,0 +1,306 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Script for running the Dockerfile for Traffic Ops.
+# The Dockerfile sets up a Docker image which can be used for any new Traffic Ops container;
+# This script, which should be run when the container is run (it's the ENTRYPOINT), will configure the container.
+#
+# The following environment variables must be set, ordinarily by `docker run -e` arguments:
+# DB_SERVER
+# DB_PORT
+# DB_ROOT_PASS
+# DB_USER
+# DB_USER_PASS
+# DB_NAME
+# TO_ADMIN_USER
+# TO_ADMIN_PASS
+# CERT_COUNTRY
+# CERT_STATE
+# CERT_CITY
+# CERT_COMPANY
+# TO_DOMAIN
+# TRAFFIC_VAULT_PASS
+
+# Check that env vars are set
+envvars=( DB_SERVER DB_PORT DB_ROOT_PASS DB_USER DB_USER_PASS TO_ADMIN_USER TO_ADMIN_PASS CERT_COUNTRY CERT_STATE CERT_CITY CERT_COMPANY TO_DOMAIN)
+for v in $envvars
+do
+ if [[ -z $$v ]]; then echo "$v is unset"; exit 1; fi
+done
+
+start() {
+ service traffic_ops start
+ exec tail -f /var/log/traffic_ops/traffic_ops.log
+}
+
+init() {
+ local postinstall_input_file="postinstall-input.json"
+ cat > "$postinstall_input_file" <<- ENDOFMESSAGE
+{
+ "/opt/traffic_ops/app/conf/production/database.conf":[
+ {
+ "Database type":"Pg",
+ "config_var":"type"
+ },
+ {
+ "Database name":"$DB_NAME",
+ "config_var":"dbname"
+ },
+ {
+ "Database server hostname IP or FQDN":"$DB_SERVER",
+ "config_var":"hostname"
+ },
+ {
+ "Database port number":"$DB_PORT",
+ "config_var":"port"
+ },
+ {
+ "Traffic Ops database user":"$DB_USER",
+ "config_var":"user"
+ },
+ {
+ "Traffic Ops database password":"$DB_USER_PASS",
+ "config_var":"password",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/app/db/dbconf.yml":[
+ {
+ "Database server root (admin) user":"postgres",
+ "config_var":"pgUser"
+ },
+ {
+ "Database server admin password":"$DB_ROOT_PASS",
+ "config_var":"pgPassword",
+ "hidden":"1"
+ },
+ {
+ "Download Maxmind Database?":"yes",
+ "config_var":"maxmind"
+ }
+ ],
+ "/opt/traffic_ops/app/conf/cdn.conf":[
+ {
+ "Generate a new secret?":"yes",
+ "config_var":"genSecret"
+ },
+ {
+ "Port to serve on?": "443",
+ "config_var": "port"
+ },
+ {
+ "Number of workers?": "12",
+ "config_var":"workers"
+ },
+ {
+ "Traffic Ops url?": "https://$TO_HOSTNAME",
+ "config_var": "base_url"
+ },
+ {
+ "Number of secrets to keep?":"1",
+ "config_var":"keepSecrets"
+ }
+ ],
+ "/opt/traffic_ops/app/conf/ldap.conf":[
+ {
+ "Do you want to set up LDAP?":"no",
+ "config_var":"setupLdap"
+ },
+ {
+ "LDAP server hostname":"",
+ "config_var":"host"
+ },
+ {
+ "LDAP Admin DN":"",
+ "config_var":"admin_dn"
+ },
+ {
+ "LDAP Admin Password":"",
+ "config_var":"admin_pass",
+ "hidden":"1"
+ },
+ {
+ "LDAP Search Base":"",
+ "config_var":"search_base"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/users.json":[
+ {
+ "Administration username for Traffic Ops":"$TO_ADMIN_USER",
+ "config_var":"tmAdminUser"
+ },
+ {
+ "Password for the admin user":"$TO_ADMIN_PASS",
+ "config_var":"tmAdminPw",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/install/data/profiles/":[
+ {
+ "Add custom profiles?":"no",
+ "config_var":"custom_profiles"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/openssl_configuration.json":[
+ {
+ "Do you want to generate a certificate?":"yes",
+ "config_var":"genCert"
+ },
+ {
+ "Country Name (2 letter code)":"$CERT_COUNTRY",
+ "config_var":"country"
+ },
+ {
+ "State or Province Name (full name)":"$CERT_STATE",
+ "config_var":"state"
+ },
+ {
+ "Locality Name (eg, city)":"$CERT_CITY",
+ "config_var":"locality"
+ },
+ {
+ "Organization Name (eg, company)":"$CERT_COMPANY",
+ "config_var":"company"
+ },
+ {
+ "Organizational Unit Name (eg, section)":"",
+ "config_var":"org_unit"
+ },
+ {
+ "Common Name (eg, your name or your server's hostname)":"$TO_HOSTNAME",
+ "config_var":"common_name"
+ },
+ {
+ "RSA Passphrase":"passphrase",
+ "config_var":"rsaPassword",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/profiles.json":[
+ {
+ "Traffic Ops url":"https://$TO_HOSTNAME",
+ "config_var":"tm.url"
+ },
+ {
+ "Human-readable CDN Name. (No whitespace, please)":"cdn",
+ "config_var":"cdn_name"
+ },
+ {
+ "Health Polling Interval (milliseconds)":"8000",
+ "config_var":"health_polling_int"
+ },
+ {
+ "DNS sub-domain for which your CDN is authoritative":"$TO_HOSTNAME.$TO_DOMAIN",
+ "config_var":"dns_subdomain"
+ },
+ {
+ "TLD SOA admin":"traffic_ops",
+ "config_var":"soa_admin"
+ },
+ {
+ "TrafficServer Drive Prefix":"/dev/ram",
+ "config_var":"driver_prefix"
+ },
+ {
+ "TrafficServer RAM Drive Prefix":"/dev/ram",
+ "config_var":"ram_drive_prefix"
+ },
+ {
+ "TrafficServer RAM Drive Letters (comma separated)":"1",
+ "config_var":"ram_drive_letters"
+ },
+ {
+ "Health Threshold Load Average":"25",
+ "config_var":"health_thresh_load_avg"
+ },
+ {
+ "Health Threshold Available Bandwidth in Kbps":"1750000",
+ "config_var":"health_thresh_kbps"
+ },
+ {
+ "Traffic Server Health Connection Timeout (milliseconds)":"2000",
+ "config_var":"health_connect_timeout"
+ }
+ ]
+}
+ENDOFMESSAGE
+
+echo -e "{\n\t\"user\": \"riakuser\",\n\t\"password\": \"$RIAK_USER_PASS\",\n\t\"MaxTLSVersion\": \"1.1\",\n\t\"tlsConfig\": {\n\t\t\"insecureSkipVerify\": true\n\t}\n}" > /opt/traffic_ops/app/conf/production/riak.conf
Review comment:
This should probably just be a heredoc like you did above
##########
File path: traffic_ops_ort/testing/docker/traffic_vault/Dockerfile
##########
@@ -0,0 +1,63 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Riak container images
+# as Traffic Vault for Traffic Control 1.6.0
+# Based on CentOS 6.6
+############################################################
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+ARG TV_ADMIN_PASS
+
+# Install openssl
+RUN yum install -y openssl
+
+# Install the initscripts
+RUN yum -y install initscripts
+
+# Install curl which is used to configure the riak search schema.
+RUN yum -y install curl
+
+# On CentOS/RedHat/Fedora (recommended)
+RUN yum install -y java-1.8.0-openjdk java-1.8.0-openjdk-devel
Review comment:
These could all be a single `yum` command
##########
File path: traffic_ops_ort/testing/docker/traffic_ops/run.sh
##########
@@ -0,0 +1,306 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Script for running the Dockerfile for Traffic Ops.
+# The Dockerfile sets up a Docker image which can be used for any new Traffic Ops container;
+# This script, which should be run when the container is run (it's the ENTRYPOINT), will configure the container.
+#
+# The following environment variables must be set, ordinarily by `docker run -e` arguments:
+# DB_SERVER
+# DB_PORT
+# DB_ROOT_PASS
+# DB_USER
+# DB_USER_PASS
+# DB_NAME
+# TO_ADMIN_USER
+# TO_ADMIN_PASS
+# CERT_COUNTRY
+# CERT_STATE
+# CERT_CITY
+# CERT_COMPANY
+# TO_DOMAIN
+# TRAFFIC_VAULT_PASS
+
+# Check that env vars are set
+envvars=( DB_SERVER DB_PORT DB_ROOT_PASS DB_USER DB_USER_PASS TO_ADMIN_USER TO_ADMIN_PASS CERT_COUNTRY CERT_STATE CERT_CITY CERT_COMPANY TO_DOMAIN)
+for v in $envvars
+do
+ if [[ -z $$v ]]; then echo "$v is unset"; exit 1; fi
+done
+
+start() {
+ service traffic_ops start
+ exec tail -f /var/log/traffic_ops/traffic_ops.log
+}
+
+init() {
+ local postinstall_input_file="postinstall-input.json"
+ cat > "$postinstall_input_file" <<- ENDOFMESSAGE
+{
+ "/opt/traffic_ops/app/conf/production/database.conf":[
+ {
+ "Database type":"Pg",
+ "config_var":"type"
+ },
+ {
+ "Database name":"$DB_NAME",
+ "config_var":"dbname"
+ },
+ {
+ "Database server hostname IP or FQDN":"$DB_SERVER",
+ "config_var":"hostname"
+ },
+ {
+ "Database port number":"$DB_PORT",
+ "config_var":"port"
+ },
+ {
+ "Traffic Ops database user":"$DB_USER",
+ "config_var":"user"
+ },
+ {
+ "Traffic Ops database password":"$DB_USER_PASS",
+ "config_var":"password",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/app/db/dbconf.yml":[
+ {
+ "Database server root (admin) user":"postgres",
+ "config_var":"pgUser"
+ },
+ {
+ "Database server admin password":"$DB_ROOT_PASS",
+ "config_var":"pgPassword",
+ "hidden":"1"
+ },
+ {
+ "Download Maxmind Database?":"yes",
+ "config_var":"maxmind"
+ }
+ ],
+ "/opt/traffic_ops/app/conf/cdn.conf":[
+ {
+ "Generate a new secret?":"yes",
+ "config_var":"genSecret"
+ },
+ {
+ "Port to serve on?": "443",
+ "config_var": "port"
+ },
+ {
+ "Number of workers?": "12",
+ "config_var":"workers"
+ },
+ {
+ "Traffic Ops url?": "https://$TO_HOSTNAME",
+ "config_var": "base_url"
+ },
+ {
+ "Number of secrets to keep?":"1",
+ "config_var":"keepSecrets"
+ }
+ ],
+ "/opt/traffic_ops/app/conf/ldap.conf":[
+ {
+ "Do you want to set up LDAP?":"no",
+ "config_var":"setupLdap"
+ },
+ {
+ "LDAP server hostname":"",
+ "config_var":"host"
+ },
+ {
+ "LDAP Admin DN":"",
+ "config_var":"admin_dn"
+ },
+ {
+ "LDAP Admin Password":"",
+ "config_var":"admin_pass",
+ "hidden":"1"
+ },
+ {
+ "LDAP Search Base":"",
+ "config_var":"search_base"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/users.json":[
+ {
+ "Administration username for Traffic Ops":"$TO_ADMIN_USER",
+ "config_var":"tmAdminUser"
+ },
+ {
+ "Password for the admin user":"$TO_ADMIN_PASS",
+ "config_var":"tmAdminPw",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/install/data/profiles/":[
+ {
+ "Add custom profiles?":"no",
+ "config_var":"custom_profiles"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/openssl_configuration.json":[
+ {
+ "Do you want to generate a certificate?":"yes",
+ "config_var":"genCert"
+ },
+ {
+ "Country Name (2 letter code)":"$CERT_COUNTRY",
+ "config_var":"country"
+ },
+ {
+ "State or Province Name (full name)":"$CERT_STATE",
+ "config_var":"state"
+ },
+ {
+ "Locality Name (eg, city)":"$CERT_CITY",
+ "config_var":"locality"
+ },
+ {
+ "Organization Name (eg, company)":"$CERT_COMPANY",
+ "config_var":"company"
+ },
+ {
+ "Organizational Unit Name (eg, section)":"",
+ "config_var":"org_unit"
+ },
+ {
+ "Common Name (eg, your name or your server's hostname)":"$TO_HOSTNAME",
+ "config_var":"common_name"
+ },
+ {
+ "RSA Passphrase":"passphrase",
+ "config_var":"rsaPassword",
+ "hidden":"1"
+ }
+ ],
+ "/opt/traffic_ops/install/data/json/profiles.json":[
+ {
+ "Traffic Ops url":"https://$TO_HOSTNAME",
+ "config_var":"tm.url"
+ },
+ {
+ "Human-readable CDN Name. (No whitespace, please)":"cdn",
+ "config_var":"cdn_name"
+ },
+ {
+ "Health Polling Interval (milliseconds)":"8000",
+ "config_var":"health_polling_int"
+ },
+ {
+ "DNS sub-domain for which your CDN is authoritative":"$TO_HOSTNAME.$TO_DOMAIN",
+ "config_var":"dns_subdomain"
+ },
+ {
+ "TLD SOA admin":"traffic_ops",
+ "config_var":"soa_admin"
+ },
+ {
+ "TrafficServer Drive Prefix":"/dev/ram",
+ "config_var":"driver_prefix"
+ },
+ {
+ "TrafficServer RAM Drive Prefix":"/dev/ram",
+ "config_var":"ram_drive_prefix"
+ },
+ {
+ "TrafficServer RAM Drive Letters (comma separated)":"1",
+ "config_var":"ram_drive_letters"
+ },
+ {
+ "Health Threshold Load Average":"25",
+ "config_var":"health_thresh_load_avg"
+ },
+ {
+ "Health Threshold Available Bandwidth in Kbps":"1750000",
+ "config_var":"health_thresh_kbps"
+ },
+ {
+ "Traffic Server Health Connection Timeout (milliseconds)":"2000",
+ "config_var":"health_connect_timeout"
+ }
+ ]
+}
+ENDOFMESSAGE
+
+echo -e "{\n\t\"user\": \"riakuser\",\n\t\"password\": \"$RIAK_USER_PASS\",\n\t\"MaxTLSVersion\": \"1.1\",\n\t\"tlsConfig\": {\n\t\t\"insecureSkipVerify\": true\n\t}\n}" > /opt/traffic_ops/app/conf/production/riak.conf
+
+ # TODO determine if term, user are necessary
+ export TERM=xterm && export USER=root && /opt/traffic_ops/install/bin/postinstall -cfile "$postinstall_input_file"
+
+ # Only listen on IPv4, not IPv6, because Docker doesn't provide a v6 interface by default. See http://mojolicious.org/perldoc/Mojo/Server/Daemon#listen
+ sed -i -e 's#https://\[::\]#https://127\.0\.0\.1#' /opt/traffic_ops/app/conf/cdn.conf
+ service traffic_ops restart
+
+ TRAFFIC_OPS_URI="https://localhost"
+
+ TMP_TO_COOKIE="$(curl -v -s -k -X POST --data '{ "u":"'"$TO_ADMIN_USER"'", "p":"'"$TO_ADMIN_PASS"'" }' $TRAFFIC_OPS_URI/api/1.2/user/login 2>&1 | grep 'Set-Cookie' | sed -e 's/.*mojolicious=\(.*\); expires.*/\1/')"
+ echo "Got cookie: $TMP_TO_COOKIE"
+
+ TMP_DOMAIN=$TO_DOMAIN
+ sed -i -- "s/{{.Domain}}/$TMP_DOMAIN/g" /profile.origin.traffic_ops
+ echo "Got domain: $TMP_DOMAIN"
+
+ echo "Importing origin"
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" -F "filename=profile.origin.traffic_ops" -F "profile_to_import=@/profile.origin.traffic_ops" $TRAFFIC_OPS_URI/profile/doImport
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "division.name=East" $TRAFFIC_OPS_URI/division/create
+ TMP_DIVISION_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/region/add | grep --color=never -oE "<option value=\"[0-9]+\">East</option>" | grep --color=never -oE "[0-9]+")"
+ echo "Got division ID: $TMP_DIVISION_ID"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "region.name=Eastish" --data-urlencode "region.division_id=$TMP_DIVISION_ID" $TRAFFIC_OPS_URI/region/create
+ TMP_REGION_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/regions.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="Eastish"]; print match[0]')"
+ echo "Got region ID: $TMP_REGION_ID"
+
+ TMP_CACHEGROUP_TYPE="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/types.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="MID_LOC"]; print match[0]')"
+ echo "Got cachegroup type ID: $TMP_CACHEGROUP_TYPE"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "cg_data.name=mid-east" --data-urlencode "cg_data.short_name=east" --data-urlencode "cg_data.latitude=0" --data-urlencode "cg_data.longitude=0" --data-urlencode "cg_data.parent_cachegroup_id=-1" --data-urlencode "cg_data.type=$TMP_CACHEGROUP_TYPE" $TRAFFIC_OPS_URI/cachegroup/create
+ TMP_CACHEGROUP_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/cachegroups.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="mid-east"]; print match[0]')"
+ echo "Got cachegroup ID: $TMP_CACHEGROUP_ID"
+
+ TMP_CACHEGROUP_EDGE_TYPE="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/types.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="EDGE_LOC"]; print match[0]')"
+ echo "Got cachegroup type ID: $TMP_CACHEGROUP_EDGE_TYPE"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "cg_data.name=edge-east" --data-urlencode "cg_data.short_name=eeast" --data-urlencode "cg_data.latitude=0" --data-urlencode "cg_data.longitude=0" --data-urlencode "cg_data.parent_cachegroup_id=$TMP_CACHEGROUP_ID" --data-urlencode "cg_data.type=$TMP_CACHEGROUP_EDGE_TYPE" $TRAFFIC_OPS_URI/cachegroup/create
+ TMP_CACHEGROUP_EDGE_ID="$(curl -s -k -X GET -H "Cookie: mojolicious=$TMP_TO_COOKIE" $TRAFFIC_OPS_URI/api/1.2/cachegroups.json | python -c 'import json,sys;obj=json.load(sys.stdin);match=[x["id"] for x in obj["response"] if x["name"]=="edge-east"]; print match[0]')"
+ echo "Got cachegroup edge ID: $TMP_CACHEGROUP_EDGE_ID"
+
+ curl -v -k -X POST -H "Cookie: mojolicious=$TMP_TO_COOKIE" --data-urlencode "location.name=plocation-nyc-1" --data-urlencode "location.short_name=nyc" --data-urlencode "location.address=1 Main Street" --data-urlencode "location.city=nyc" --data-urlencode "location.state=NY" --data-urlencode "location.zip=12345" --data-urlencode "location.poc=" --data-urlencode "location.phone=" --data-urlencode "location.email=no@no.no" --data-urlencode "location.comments=" --data-urlencode "location.region=$TMP_REGION_ID" $TRAFFIC_OPS_URI/phys_location/create
Review comment:
I really think this could've benefited from using the [`to-access.sh`](https://github.com/apache/trafficcontrol/blob/master/infrastructure/cdn-in-a-box/traffic_ops/to-access.sh) script we built for this purpose - or the Python client which provides similar scripts but is a fair bit heavier since it requires a Python interpreter.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] jrushford commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
jrushford commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r563800572
##########
File path: traffic_ops_ort/testing/README.md
##########
@@ -0,0 +1,102 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Traffic Ops ORT Tests
+
+The ORT tests are used to validate the ORT tools used with a
+release of Traffic Ops. The tests insure that the ORT tools
+are able to communicate with Traffic Ops, install required
+packages and generate the correct ATS configuration files on
+a Trafficserver cache.
+
+The first thing you need do is to provide the Traffic Ops and
+Traffic Ops ORT RPM files for the build you wish to test using
+this framework. This test environment provides the necessary
+Docker containers used to support and execute the tests when
+given the necessary RPM's. If you choose to not use the provided
+Docker containers you will need to provide the following resources:
+
+ - A running Traffic Ops with the installed release to be tested
+ - A running Postgres SQL database which is loaded with the proper
+ test data for the Traffic Ops release.
+ - An Apache Traffic Server host that has the installed release of
+ ORT to be tested against the release of Traffic Ops.
+ - A yum server configured to provide the test rpm's herein.
+ - A Traffic vault server.
+
+# Directory layout
+
+ - trafficcontrol/traffic_ops_ort/testing/docker: has all the
+ necessary files for running the test Docker containers.
+ - trafficcontrol/traffic_ops_ort/testing/ort-tests: this directory.
+ contains all the go files used to run the ORT tests.
+
+# Setup.
+
+ 1. Build the Traffic Ops and Traffic Ops ORT RPM's that you wish
+ to test. See the top level 'build' directory for building
+ instructions.
+ 2. Copy the Traffic Ops RPM to docker/trafffic_ops/traffic_ops.rpm
+ (NOTE: Use the file name 'traffic_ops.rpm')
+ 3. Copy the Traffic Ops ORT rpm to docker/ats_edge/traffic_ops_ort.rpm
+ (NOTE: Use the file name 'traffic_ops_ort.rpm'
+ 4. Copy the recent copy of riak-2.2.3-1.el7.centos.x86_64.rpm to
+ docker/traffic_vault/riak-2.2.3-1.el7.centos.x86_64.rpm
+ 5. Copy an Apache Trafficserver rpm to
+ docker/yumserver/test-rpms/trafficserver-$VERSION.$COMMIT-HASH.el7.x86_64.rpm
+ You will need to edit and adjust the trafficserver package value in
+ ort-tests/tc-fixtures.json to match the $VERSION.$COMMIT-HASH used in the name
+ of your RPM. Search for '8.0.8-19.77cb23a' in the ort-tests/tc-fixtures.json
+ and change the value to match the RPM version you choose to use.
+ For example, the current value in tc-fixtures.json is '8.0.8-19.77cb23a' and
+ therefore the rpm file expected in 'docker/yumserver/test-rpms' is
+ 'trafficserver-8.0.8-19.77cb23a.el7.x86_64.rpm'.
+ 6. The container Docker files have the usernames and passwords used in the various
+ containers ie, postgresql db, traffic_ops, and traffic_ops_ort. The usernames
+ and passwords passed to the 't3c' executable in in the
+ ort-tests/conf/docker-edge-cache.conf file. Make sure that the usernames/passwords
+ in the Docker files match those in the t3c configuration file.
+ An example ort-tests/conf/edge-cache.conf file is provided should you choose to
+ use your own Traffic Ops and Postgresql environment.
+ 7. Build the Docker images and start the containers:
+ ```
+ # cd trafficcontrol/traffic_ops_ort/testing/docker
+ # docker-compose build
+ # docker-compose up -d
+ ```
Review comment:
@ocket8888 Fixed, removed the '#'
##########
File path: traffic_ops_ort/testing/docker/yumserver/test-rpms/README.md
##########
@@ -0,0 +1 @@
+# test rpms
Review comment:
@ocket8888 this is fixed.
##########
File path: traffic_ops_ort/testing/README.md
##########
@@ -0,0 +1,102 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Traffic Ops ORT Tests
+
+The ORT tests are used to validate the ORT tools used with a
+release of Traffic Ops. The tests insure that the ORT tools
+are able to communicate with Traffic Ops, install required
+packages and generate the correct ATS configuration files on
+a Trafficserver cache.
+
+The first thing you need do is to provide the Traffic Ops and
+Traffic Ops ORT RPM files for the build you wish to test using
+this framework. This test environment provides the necessary
+Docker containers used to support and execute the tests when
+given the necessary RPM's. If you choose to not use the provided
+Docker containers you will need to provide the following resources:
+
+ - A running Traffic Ops with the installed release to be tested
+ - A running Postgres SQL database which is loaded with the proper
+ test data for the Traffic Ops release.
+ - An Apache Traffic Server host that has the installed release of
+ ORT to be tested against the release of Traffic Ops.
+ - A yum server configured to provide the test rpm's herein.
+ - A Traffic vault server.
+
+# Directory layout
+
+ - trafficcontrol/traffic_ops_ort/testing/docker: has all the
+ necessary files for running the test Docker containers.
+ - trafficcontrol/traffic_ops_ort/testing/ort-tests: this directory.
+ contains all the go files used to run the ORT tests.
+
+# Setup.
+
+ 1. Build the Traffic Ops and Traffic Ops ORT RPM's that you wish
+ to test. See the top level 'build' directory for building
+ instructions.
+ 2. Copy the Traffic Ops RPM to docker/trafffic_ops/traffic_ops.rpm
+ (NOTE: Use the file name 'traffic_ops.rpm')
+ 3. Copy the Traffic Ops ORT rpm to docker/ats_edge/traffic_ops_ort.rpm
+ (NOTE: Use the file name 'traffic_ops_ort.rpm'
+ 4. Copy the recent copy of riak-2.2.3-1.el7.centos.x86_64.rpm to
+ docker/traffic_vault/riak-2.2.3-1.el7.centos.x86_64.rpm
+ 5. Copy an Apache Trafficserver rpm to
+ docker/yumserver/test-rpms/trafficserver-$VERSION.$COMMIT-HASH.el7.x86_64.rpm
+ You will need to edit and adjust the trafficserver package value in
+ ort-tests/tc-fixtures.json to match the $VERSION.$COMMIT-HASH used in the name
+ of your RPM. Search for '8.0.8-19.77cb23a' in the ort-tests/tc-fixtures.json
+ and change the value to match the RPM version you choose to use.
+ For example, the current value in tc-fixtures.json is '8.0.8-19.77cb23a' and
+ therefore the rpm file expected in 'docker/yumserver/test-rpms' is
+ 'trafficserver-8.0.8-19.77cb23a.el7.x86_64.rpm'.
+ 6. The container Docker files have the usernames and passwords used in the various
+ containers ie, postgresql db, traffic_ops, and traffic_ops_ort. The usernames
+ and passwords passed to the 't3c' executable in in the
+ ort-tests/conf/docker-edge-cache.conf file. Make sure that the usernames/passwords
+ in the Docker files match those in the t3c configuration file.
+ An example ort-tests/conf/edge-cache.conf file is provided should you choose to
+ use your own Traffic Ops and Postgresql environment.
+ 7. Build the Docker images and start the containers:
+ ```
+ # cd trafficcontrol/traffic_ops_ort/testing/docker
+ # docker-compose build
+ # docker-compose up -d
+ ```
+ After some time, test results should be available at
+ 'ort-tests/test.log'
+
+ If you wish to run the tests manually use 'docker ps' to obtain the container id for
+ the ats_edge host and then:
+
+ ```
+ # docker exec -it $ats_edge_container_id /bin/bash -l
+ # cd /ort-tests
+ # go test -cfg=conf/docker-edge-cache.conf
+ ```
Review comment:
@ocket8888 this is fixed to.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564907929
##########
File path: traffic_ops_ort/testing/docker/yumserver/Dockerfile
##########
@@ -0,0 +1,35 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Traffic Server container images
+# as Edges for Traffic Control 1.4
+# Based on CentOS 6.6
+############################################################
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y httpd
+RUN yum install -y createrepo
+RUN yum install -y yum-utils
Review comment:
What am I looking for?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #5298: Add ort integration testing.
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #5298:
URL: https://github.com/apache/trafficcontrol/pull/5298#discussion_r564883331
##########
File path: traffic_ops_ort/testing/docker/yumserver/Dockerfile
##########
@@ -0,0 +1,35 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+############################################################
+# Dockerfile to build Traffic Server container images
+# as Edges for Traffic Control 1.4
+# Based on CentOS 6.6
+############################################################
+
+FROM centos:7
+MAINTAINER dev@trafficcontrol.apache.org
+
+RUN yum install -y httpd
+RUN yum install -y createrepo
+RUN yum install -y yum-utils
Review comment:
Testing by doing `docker system prune -af` then building, then collapse all those `yum` commands and doing it again shows that in separate lines it adds almost 2 minutes to the build time (which really isn't much on the scale it's on) and nearly 2GB of used disk space
- Separate lines: 14:40 build time, 4.8GB cache size on disk
- Combined: 12:48 build time, 3.1GB cache size on disk
So not collapsing them increases disk usage by 64%
But if you really don't want to fix it, I'll just do it in another PR. No point holding up this PR any longer for something that'll take 3 minutes to fix.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org