You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficcontrol.apache.org by da...@apache.org on 2018/11/01 17:46:44 UTC
[trafficcontrol] branch master updated (8c4396b -> cc3ca2e)
This is an automated email from the ASF dual-hosted git repository.
dangogh pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git.
from 8c4396b Add TO client update ds nullable
new 8312f6f Added test Dockerfile
new 0704cdb Fixed a problem with restapi.py using Python2 functions
new 94008d6 Updated Documentation for the compare tool and genRoutesConfig.py script to include Docker method of use
new 4317ac7 Fixed a bug where people with ':' in their username and/or password would fail to authenticate
new cc3ca2e genRoutesConfig.py now supports defaulting blank testing username/password to reference username/password
The 5 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/source/tools/compare.rst | 30 +++++++++++++++
traffic_control/clients/python/common/restapi.py | 6 ++-
traffic_ops/testing/compare/Dockerfile | 48 ++++++++++++++++++++++++
traffic_ops/testing/compare/README.md | 44 ++++++++++++++--------
traffic_ops/testing/compare/genConfigRoutes.py | 21 ++++++++++-
5 files changed, 130 insertions(+), 19 deletions(-)
create mode 100644 traffic_ops/testing/compare/Dockerfile
[trafficcontrol] 04/05: Fixed a bug where people with ':' in their
username and/or password would fail to authenticate
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dangogh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 4317ac78b3d53dd2a89960fd721d0508cc5f41ee
Author: ocket8888 <oc...@gmail.com>
AuthorDate: Thu Nov 1 10:42:57 2018 -0600
Fixed a bug where people with ':' in their username and/or password would fail to authenticate
---
traffic_ops/testing/compare/genConfigRoutes.py | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/traffic_ops/testing/compare/genConfigRoutes.py b/traffic_ops/testing/compare/genConfigRoutes.py
index c4720e6..628f770 100755
--- a/traffic_ops/testing/compare/genConfigRoutes.py
+++ b/traffic_ops/testing/compare/genConfigRoutes.py
@@ -171,7 +171,13 @@ def main(kwargs:argparse.Namespace) -> int:
instanceA = kwargs.InstanceA
instanceB = kwargs.InstanceB
loginA = kwargs.LoginA.split(':')
- loginB = kwargs.LoginB.split(':') if kwargs.LoginB else loginA
+ loginA = (loginA[0], ':'.join(loginA[1:]))
+ loginB = loginA
+ if kwargs.LoginB:
+ loginB = kwargs.LoginB.split(':')
+ loginB = (loginB[0], ':'.join(loginB[1:]))
+
+
verify = not kwargs.insecure
# Peel off all schemas
[trafficcontrol] 02/05: Fixed a problem with restapi.py using
Python2 functions
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dangogh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 0704cdbcb6e81f5890b5ef116b7a3728a4c255bb
Author: ocket8888 <oc...@gmail.com>
AuthorDate: Mon Oct 29 11:41:05 2018 -0600
Fixed a problem with restapi.py using Python2 functions
---
traffic_control/clients/python/common/restapi.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/traffic_control/clients/python/common/restapi.py b/traffic_control/clients/python/common/restapi.py
index 177e66e..4fe5518 100644
--- a/traffic_control/clients/python/common/restapi.py
+++ b/traffic_control/clients/python/common/restapi.py
@@ -36,7 +36,11 @@ import common.utils as utils
# Python 2 to Python 3 Compatibility
import requests.compat as compat
from builtins import str
-from future.utils import iteritems
+
+try:
+ from future.utils import iteritems
+except ImportError:
+ iteritems = lambda x: x.items()
logger = logging.getLogger(__name__)
[trafficcontrol] 03/05: Updated Documentation for the compare tool
and genRoutesConfig.py script to include Docker method of use
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dangogh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 94008d6c0b090bd1c140fda3744f29a2c4a1d9cb
Author: ocket8888 <oc...@gmail.com>
AuthorDate: Mon Oct 29 13:12:09 2018 -0600
Updated Documentation for the compare tool and genRoutesConfig.py script to include Docker method of use
---
docs/source/tools/compare.rst | 26 +++++++++++++++++++++
traffic_ops/testing/compare/README.md | 44 ++++++++++++++++++++++-------------
2 files changed, 54 insertions(+), 16 deletions(-)
diff --git a/docs/source/tools/compare.rst b/docs/source/tools/compare.rst
index 003e416..a1e9e72 100644
--- a/docs/source/tools/compare.rst
+++ b/docs/source/tools/compare.rst
@@ -104,3 +104,29 @@ The genConfigRoutes.py script will output list of unique API routes (relative to
./genConfigRoutes.py https://trafficopsA.example.test https://trafficopsB.example.test username:password | ./compare
\... assuming the proper environment variables have been set for ``compare``.
+
+Usage with Docker
+=================
+A Dockerfile is provided to run tests on a pair of instances given the configuration environment variables necessary. This will generate configuration file routes using ``genConfigRoutes.py``, and add them to whatever is already contained in ``traffic_ops/testing/compare/testroutes.txt``, then run the ``compare`` tool on the final API route list. Build artifacts (i.e. anything output files created by the `compare` tool) are placed in the `/artifacts/` directory on the container. To retri [...]
+
+TO_URL
+ The URL of the reference Traffic Ops instance
+TO_USER
+ The username to authenticate with the reference Traffic Ops instance
+TO_PASSWORD
+ The password to authenticate with the reference Traffic Ops instance
+TEST_URL
+ The URL of the testing Traffic Ops instance
+TEST_USER
+ The username to authenticate with the testing Traffic Ops instance
+TEST_PASSWORD
+ The password to authenticate with the testing Traffic Ops instance
+
+.. code-block:: shell
+ :caption: Sample Script to Build and Run
+ sudo docker build . -f traffic_ops/testing/compare/Dockerfile -t compare:latest
+ sudo docker run -v $PWD/artifacts:/artifacts -e TO_URL="$TO_URL" -e TEST_URL="$TEST_URL" -e TO_USER="admin" -e TO_PASSWORD="twelve" -e TEST_USER="admin" -e TEST_PASSWORD="twelve" compare:latest
+
+.. note:: The above code example assumes that the environment variables ``TO_URL`` and ``TEST_URL`` refer to the URL of the reference Traffic Ops instance and the URL of the test Traffic Ops instance, respectively (including port numbers). It also uses credentials suitable for logging into a stock :ref:`ciab` instance.
+
+.. note:: Unlike using the ``genRoutesConfig.py`` script and/or the ``compare`` on their own, *all* of the variables must be defined, even if they are duplicates.
diff --git a/traffic_ops/testing/compare/README.md b/traffic_ops/testing/compare/README.md
index e765384..f7fc302 100644
--- a/traffic_ops/testing/compare/README.md
+++ b/traffic_ops/testing/compare/README.md
@@ -1,20 +1,20 @@
<!--
- Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
-->
The Compare Tool
@@ -92,6 +92,18 @@ The genConfigRoutes.py script will output list of unique API routes (relative to
[1] Theoretically, if you downloaded the Traffic Control repository properly (into `$GOPATH/src/github.com/apache/trafficcontrol`), this will already be satisfied.
+Usage With Docker
+-----------------
+A Dockerfile is provided to run tests on a pair of instances given the configuration environment variables necessary. This will generate configuration file routes using `genConfigRoutes.py`, and add them to whatever is already contained in [testroutes.txt](./testroutes.txt), then run the `compare` tool on the final API route list. Build artifacts (i.e. anything output files created by the `compare` tool) are placed in the `/artifacts/` directory on the container. To retrieve these result [...]
+
+```bash
+sudo docker build . -f traffic_ops/testing/compare/Dockerfile -t compare:latest
+sudo docker run -v $PWD/artifacts:/artifacts -e TO_URL="$TO_URL" -e TEST_URL="$TEST_URL" -e TO_USER="admin" -e TO_PASSWORD="twelve" -e TEST_USER="admin" -e TEST_PASSWORD="twelve" compare:latest
+```
+
+!!! note
+ The above code example assumes that the environment variables `TO_URL` and `TEST_URL` refer to the URL of the reference Traffic Ops instance and the URL of the test Traffic Ops instance, respectively (including port numbers). It also uses credentials suitable for logging into a stock CDN-in-a-Box instance.
+
Further Information
-------------------
[trafficcontrol] 01/05: Added test Dockerfile
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dangogh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 8312f6f7b612438578789b01597b1330cd192b64
Author: ocket8888 <oc...@gmail.com>
AuthorDate: Mon Oct 29 11:24:28 2018 -0600
Added test Dockerfile
---
traffic_ops/testing/compare/Dockerfile | 48 ++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
diff --git a/traffic_ops/testing/compare/Dockerfile b/traffic_ops/testing/compare/Dockerfile
new file mode 100644
index 0000000..e110417
--- /dev/null
+++ b/traffic_ops/testing/compare/Dockerfile
@@ -0,0 +1,48 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+############################################################
+# Dockerfile to build Docker images for testing via the 'compare' tool
+# Based on Alpine Linux
+############################################################
+
+FROM golang:alpine
+
+ENV GOPATH /go/
+RUN mkdir -p /go/src/github.com/apache/trafficcontrol/traffic_control/clients/python\
+ /go/src/github.com/apache/trafficcontrol/lib\
+ /go/src/github.com/apache/trafficcontrol/traffic_ops/testing/compare\
+ /go/src/github.com/apache/trafficcontrol/traffic_ops/vendor/github.com/kelseyhightower\
+ /artifacts
+
+RUN apk update
+RUN apk add python3 git
+RUN python3 -m ensurepip && python3 -m pip install --upgrade pip && python3 -m pip install requests munch
+
+ADD traffic_control/clients/python /go/src/github.com/apache/trafficcontrol/traffic_control/clients/python/
+ADD lib /go/src/github.com/apache/trafficcontrol/lib
+ADD vendor /go/src/github.com/apache/trafficcontrol/vendor
+ADD traffic_ops/vendor/github.com/kelseyhightower /go/src/github.com/apache/trafficcontrol/traffic_ops/vendor/github.com/kelseyhightower
+ADD traffic_ops/testing/compare /go/src/github.com/apache/trafficcontrol/traffic_ops/testing/compare
+
+WORKDIR /go/src/github.com/apache/trafficcontrol/traffic_ops/testing/compare/
+RUN go get -v ./...
+RUN go build .
+RUN cp testroutes.txt /artifacts/
+
+CMD ./genConfigRoutes.py -k $TO_URL $TEST_URL $TO_USER:$TO_PASSWORD $TEST_USER:$TEST_PASSWORD -l INFO 2>&1 >>/artifacts/testroutes.txt | tee /artifacts/genRoutesConfig.log &&\
+ ./compare --ref_url=$TO_URL --test_url=$TEST_URL --ref_user=$TO_USER --ref_passwd=$TO_PASSWORD --test_user=$TEST_USER --test_passwd=$TEST_PASSWORD -r /artifacts </artifacts/testroutes.txt
[trafficcontrol] 05/05: genRoutesConfig.py now supports defaulting
blank testing username/password to reference username/password
Posted by da...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dangogh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit cc3ca2e08987ed53391ff0f62521ababbd9fe54a
Author: ocket8888 <oc...@gmail.com>
AuthorDate: Thu Nov 1 10:58:35 2018 -0600
genRoutesConfig.py now supports defaulting blank testing username/password to reference username/password
This is as opposed to the existing behaviour which would use the reference username/password _pair_ if the testing pair was not given at all.
That means that you can, for instance, give it the testing pair 'admin:' to use the testing username 'admin', but default to the password used by the reference user
---
docs/source/tools/compare.rst | 4 ++++
traffic_ops/testing/compare/genConfigRoutes.py | 21 ++++++++++++++++-----
2 files changed, 20 insertions(+), 5 deletions(-)
diff --git a/docs/source/tools/compare.rst b/docs/source/tools/compare.rst
index a1e9e72..165d7c6 100644
--- a/docs/source/tools/compare.rst
+++ b/docs/source/tools/compare.rst
@@ -82,6 +82,8 @@ usage: genConfigRoutes.py [-h] [-k] [-v] InstanceA InstanceB LoginA [LoginB]
.. note:: If you're using a CDN-in-a-Box environment for testing, it's likely that you'll need the ``-k``/``--insecure`` option if you're outside the Docker network
+.. _compare-genConfigRoutes-positional_parameters:
+
.. table:: Positional Parameters (In Order)
+-----------+---------------------------------------------------------------------------------------------------------------------------------------+
@@ -97,6 +99,8 @@ usage: genConfigRoutes.py [-h] [-k] [-v] InstanceA InstanceB LoginA [LoginB]
| | If not given, LoginA will be re-used for the second connection (default: None) |
+-----------+---------------------------------------------------------------------------------------------------------------------------------------+
+.. note:: The full behaviour of the ``LoginB`` parameter described in :ref:`compare-genConfigRoutes-positional_parameters` is such that supports not only a fully missing authentication credentials pair, but also a blank string for each of the pair members. This means that you can, for instance, give it the testing pair 'admin:' to use the testing username 'admin', but default to the password used by the reference user. Or, another example is that passing ``LoginB`` as simple ``:`` will r [...]
+
The genConfigRoutes.py script will output list of unique API routes (relative to the desired Traffic Ops URL) that point to generated configuration files for a sample set of servers common to both Traffic Ops instances. The results are printed to stdout, making the output perfect for piping directly into ``compare`` like so:
.. code-block:: shell
diff --git a/traffic_ops/testing/compare/genConfigRoutes.py b/traffic_ops/testing/compare/genConfigRoutes.py
index 628f770..0725c88 100755
--- a/traffic_ops/testing/compare/genConfigRoutes.py
+++ b/traffic_ops/testing/compare/genConfigRoutes.py
@@ -170,13 +170,24 @@ def main(kwargs:argparse.Namespace) -> int:
instanceA = kwargs.InstanceA
instanceB = kwargs.InstanceB
- loginA = kwargs.LoginA.split(':')
- loginA = (loginA[0], ':'.join(loginA[1:]))
+
+ try:
+ loginA = kwargs.LoginA.split(':')
+ loginA = (loginA[0], ':'.join(loginA[1:]))
+ except (KeyError, IndexError) as e:
+ logging.critical("Bad username/password pair: '%s' (hint: try -h/--help)", kwargs.LoginA)
+ return 1
+
loginB = loginA
- if kwargs.LoginB:
- loginB = kwargs.LoginB.split(':')
- loginB = (loginB[0], ':'.join(loginB[1:]))
+ try:
+ if kwargs.LoginB:
+ loginB = kwargs.LoginB.split(':')
+ loginB = (loginB[0], ':'.join(loginB[1:]))
+ loginB = (loginB[0] if loginB[0] else loginA[0], loginB[1] if loginB[1] else loginA[1])
+ except (KeyError, IndexError) as e:
+ logging.critical("Bad username/password pair: '%s' (hint: try -h/--help)", kwargs.LoginB)
+ return 1
verify = not kwargs.insecure