You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficcontrol.apache.org by GitBox <gi...@apache.org> on 2018/11/01 17:46:41 UTC
[GitHub] dangogh closed pull request #2965: Config compare test
dangogh closed pull request #2965: Config compare test
URL: https://github.com/apache/trafficcontrol/pull/2965
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docs/source/tools/compare.rst b/docs/source/tools/compare.rst
index 003e4167f..165d7c692 100644
--- a/docs/source/tools/compare.rst
+++ b/docs/source/tools/compare.rst
@@ -82,6 +82,8 @@ usage: genConfigRoutes.py [-h] [-k] [-v] InstanceA InstanceB LoginA [LoginB]
.. note:: If you're using a CDN-in-a-Box environment for testing, it's likely that you'll need the ``-k``/``--insecure`` option if you're outside the Docker network
+.. _compare-genConfigRoutes-positional_parameters:
+
.. table:: Positional Parameters (In Order)
+-----------+---------------------------------------------------------------------------------------------------------------------------------------+
@@ -97,6 +99,8 @@ usage: genConfigRoutes.py [-h] [-k] [-v] InstanceA InstanceB LoginA [LoginB]
| | If not given, LoginA will be re-used for the second connection (default: None) |
+-----------+---------------------------------------------------------------------------------------------------------------------------------------+
+.. note:: The full behaviour of the ``LoginB`` parameter described in :ref:`compare-genConfigRoutes-positional_parameters` is such that supports not only a fully missing authentication credentials pair, but also a blank string for each of the pair members. This means that you can, for instance, give it the testing pair 'admin:' to use the testing username 'admin', but default to the password used by the reference user. Or, another example is that passing ``LoginB`` as simple ``:`` will result in the same behaviour as not passing it all.
+
The genConfigRoutes.py script will output list of unique API routes (relative to the desired Traffic Ops URL) that point to generated configuration files for a sample set of servers common to both Traffic Ops instances. The results are printed to stdout, making the output perfect for piping directly into ``compare`` like so:
.. code-block:: shell
@@ -104,3 +108,29 @@ The genConfigRoutes.py script will output list of unique API routes (relative to
./genConfigRoutes.py https://trafficopsA.example.test https://trafficopsB.example.test username:password | ./compare
\... assuming the proper environment variables have been set for ``compare``.
+
+Usage with Docker
+=================
+A Dockerfile is provided to run tests on a pair of instances given the configuration environment variables necessary. This will generate configuration file routes using ``genConfigRoutes.py``, and add them to whatever is already contained in ``traffic_ops/testing/compare/testroutes.txt``, then run the ``compare`` tool on the final API route list. Build artifacts (i.e. anything output files created by the `compare` tool) are placed in the `/artifacts/` directory on the container. To retrieve these results, the use of a volume is recommended. The build context *must* be at the root of the Traffic Control repository, as the tools have dependencies on the Traffic Control clients. In order to use the container, the following environment variables must be defined for the container at runtime:
+
+TO_URL
+ The URL of the reference Traffic Ops instance
+TO_USER
+ The username to authenticate with the reference Traffic Ops instance
+TO_PASSWORD
+ The password to authenticate with the reference Traffic Ops instance
+TEST_URL
+ The URL of the testing Traffic Ops instance
+TEST_USER
+ The username to authenticate with the testing Traffic Ops instance
+TEST_PASSWORD
+ The password to authenticate with the testing Traffic Ops instance
+
+.. code-block:: shell
+ :caption: Sample Script to Build and Run
+ sudo docker build . -f traffic_ops/testing/compare/Dockerfile -t compare:latest
+ sudo docker run -v $PWD/artifacts:/artifacts -e TO_URL="$TO_URL" -e TEST_URL="$TEST_URL" -e TO_USER="admin" -e TO_PASSWORD="twelve" -e TEST_USER="admin" -e TEST_PASSWORD="twelve" compare:latest
+
+.. note:: The above code example assumes that the environment variables ``TO_URL`` and ``TEST_URL`` refer to the URL of the reference Traffic Ops instance and the URL of the test Traffic Ops instance, respectively (including port numbers). It also uses credentials suitable for logging into a stock :ref:`ciab` instance.
+
+.. note:: Unlike using the ``genRoutesConfig.py`` script and/or the ``compare`` on their own, *all* of the variables must be defined, even if they are duplicates.
diff --git a/traffic_control/clients/python/common/restapi.py b/traffic_control/clients/python/common/restapi.py
index 177e66e57..4fe55184b 100644
--- a/traffic_control/clients/python/common/restapi.py
+++ b/traffic_control/clients/python/common/restapi.py
@@ -36,7 +36,11 @@
# Python 2 to Python 3 Compatibility
import requests.compat as compat
from builtins import str
-from future.utils import iteritems
+
+try:
+ from future.utils import iteritems
+except ImportError:
+ iteritems = lambda x: x.items()
logger = logging.getLogger(__name__)
diff --git a/traffic_ops/testing/compare/Dockerfile b/traffic_ops/testing/compare/Dockerfile
new file mode 100644
index 000000000..e11041734
--- /dev/null
+++ b/traffic_ops/testing/compare/Dockerfile
@@ -0,0 +1,48 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+############################################################
+# Dockerfile to build Docker images for testing via the 'compare' tool
+# Based on Alpine Linux
+############################################################
+
+FROM golang:alpine
+
+ENV GOPATH /go/
+RUN mkdir -p /go/src/github.com/apache/trafficcontrol/traffic_control/clients/python\
+ /go/src/github.com/apache/trafficcontrol/lib\
+ /go/src/github.com/apache/trafficcontrol/traffic_ops/testing/compare\
+ /go/src/github.com/apache/trafficcontrol/traffic_ops/vendor/github.com/kelseyhightower\
+ /artifacts
+
+RUN apk update
+RUN apk add python3 git
+RUN python3 -m ensurepip && python3 -m pip install --upgrade pip && python3 -m pip install requests munch
+
+ADD traffic_control/clients/python /go/src/github.com/apache/trafficcontrol/traffic_control/clients/python/
+ADD lib /go/src/github.com/apache/trafficcontrol/lib
+ADD vendor /go/src/github.com/apache/trafficcontrol/vendor
+ADD traffic_ops/vendor/github.com/kelseyhightower /go/src/github.com/apache/trafficcontrol/traffic_ops/vendor/github.com/kelseyhightower
+ADD traffic_ops/testing/compare /go/src/github.com/apache/trafficcontrol/traffic_ops/testing/compare
+
+WORKDIR /go/src/github.com/apache/trafficcontrol/traffic_ops/testing/compare/
+RUN go get -v ./...
+RUN go build .
+RUN cp testroutes.txt /artifacts/
+
+CMD ./genConfigRoutes.py -k $TO_URL $TEST_URL $TO_USER:$TO_PASSWORD $TEST_USER:$TEST_PASSWORD -l INFO 2>&1 >>/artifacts/testroutes.txt | tee /artifacts/genRoutesConfig.log &&\
+ ./compare --ref_url=$TO_URL --test_url=$TEST_URL --ref_user=$TO_USER --ref_passwd=$TO_PASSWORD --test_user=$TEST_USER --test_passwd=$TEST_PASSWORD -r /artifacts </artifacts/testroutes.txt
diff --git a/traffic_ops/testing/compare/README.md b/traffic_ops/testing/compare/README.md
index e765384a9..f7fc302fa 100644
--- a/traffic_ops/testing/compare/README.md
+++ b/traffic_ops/testing/compare/README.md
@@ -1,20 +1,20 @@
<!--
- Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
-->
The Compare Tool
@@ -92,6 +92,18 @@ The genConfigRoutes.py script will output list of unique API routes (relative to
[1] Theoretically, if you downloaded the Traffic Control repository properly (into `$GOPATH/src/github.com/apache/trafficcontrol`), this will already be satisfied.
+Usage With Docker
+-----------------
+A Dockerfile is provided to run tests on a pair of instances given the configuration environment variables necessary. This will generate configuration file routes using `genConfigRoutes.py`, and add them to whatever is already contained in [testroutes.txt](./testroutes.txt), then run the `compare` tool on the final API route list. Build artifacts (i.e. anything output files created by the `compare` tool) are placed in the `/artifacts/` directory on the container. To retrieve these results, the use of a volume is recommended. The build context _must_ be at the root of the Traffic Control repository, as the tools have dependencies on the Traffic Control clients.
+
+```bash
+sudo docker build . -f traffic_ops/testing/compare/Dockerfile -t compare:latest
+sudo docker run -v $PWD/artifacts:/artifacts -e TO_URL="$TO_URL" -e TEST_URL="$TEST_URL" -e TO_USER="admin" -e TO_PASSWORD="twelve" -e TEST_USER="admin" -e TEST_PASSWORD="twelve" compare:latest
+```
+
+!!! note
+ The above code example assumes that the environment variables `TO_URL` and `TEST_URL` refer to the URL of the reference Traffic Ops instance and the URL of the test Traffic Ops instance, respectively (including port numbers). It also uses credentials suitable for logging into a stock CDN-in-a-Box instance.
+
Further Information
-------------------
diff --git a/traffic_ops/testing/compare/genConfigRoutes.py b/traffic_ops/testing/compare/genConfigRoutes.py
index c4720e691..0725c8892 100755
--- a/traffic_ops/testing/compare/genConfigRoutes.py
+++ b/traffic_ops/testing/compare/genConfigRoutes.py
@@ -170,8 +170,25 @@ def main(kwargs:argparse.Namespace) -> int:
instanceA = kwargs.InstanceA
instanceB = kwargs.InstanceB
- loginA = kwargs.LoginA.split(':')
- loginB = kwargs.LoginB.split(':') if kwargs.LoginB else loginA
+
+ try:
+ loginA = kwargs.LoginA.split(':')
+ loginA = (loginA[0], ':'.join(loginA[1:]))
+ except (KeyError, IndexError) as e:
+ logging.critical("Bad username/password pair: '%s' (hint: try -h/--help)", kwargs.LoginA)
+ return 1
+
+ loginB = loginA
+
+ try:
+ if kwargs.LoginB:
+ loginB = kwargs.LoginB.split(':')
+ loginB = (loginB[0], ':'.join(loginB[1:]))
+ loginB = (loginB[0] if loginB[0] else loginA[0], loginB[1] if loginB[1] else loginA[1])
+ except (KeyError, IndexError) as e:
+ logging.critical("Bad username/password pair: '%s' (hint: try -h/--help)", kwargs.LoginB)
+ return 1
+
verify = not kwargs.insecure
# Peel off all schemas
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services