You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficcontrol.apache.org by da...@apache.org on 2018/10/26 17:38:09 UTC
[trafficcontrol] 04/04: Added documentation for the compare tool
and route generator script
This is an automated email from the ASF dual-hosted git repository.
dangogh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
commit 64e7ffa6cf220474a14cc92920d52e3fd577ce36
Author: ocket8888 <oc...@gmail.com>
AuthorDate: Mon Oct 22 16:22:19 2018 -0600
Added documentation for the compare tool and route generator script
---
docs/source/index.rst | 9 +++
docs/source/tools/compare.rst | 106 +++++++++++++++++++++++++
docs/source/tools/index.rst | 26 ++++++
traffic_ops/testing/compare/README.md | 90 +++++++++++++++------
traffic_ops/testing/compare/genConfigRoutes.py | 2 +-
5 files changed, 209 insertions(+), 24 deletions(-)
diff --git a/docs/source/index.rst b/docs/source/index.rst
index 753d0e4..8bbb31c 100644
--- a/docs/source/index.rst
+++ b/docs/source/index.rst
@@ -69,6 +69,15 @@ A guide to external RESTful APIs for Traffic Ops
api/index
+Tools
+=====
+A living list of tools for testing, interacting with, and developing for the Traffic Control CDN
+
+.. toctree::
+ :maxdepth: 3
+
+ tools/index
+
FAQ
===
diff --git a/docs/source/tools/compare.rst b/docs/source/tools/compare.rst
new file mode 100644
index 0000000..003e416
--- /dev/null
+++ b/docs/source/tools/compare.rst
@@ -0,0 +1,106 @@
+..
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+..
+
+.. _compare-tool:
+
+****************
+The Compare Tool
+****************
+The ``compare`` tool is used to compare the output of a set of :ref:`Traffic Ops API <to-api>` endpoints between two running instances of Traffic Ops. The idea is that two different versions of Traffic Ops with the same data will have differences in the output of their API endpoints *if and only if* either the change was intentional, or a new bug was introduced in the newer version. Typically, this isn't really true, due to rapidly changing data structures like timestamps in the API outp [...]
+
+Location and Dependencies
+=========================
+The ``compare`` tool is written in Go, and can be found from within the Traffic Control repository at ``traffic_ops/testing/compare/``. The main file of interest is ``compare.go``, which contains the main routine and logic for checking endpoints. To build the executable, run ``go build .`` from within the ``traffic_ops/testing/compare/`` directory. Alternatively, run the file without storing a built binary by using ``go run <path to compare.go>``. In order to build/run the ``compare`` to [...]
+
+* github.com/apache/trafficcontrol/lib/go-tc\ [1]_
+* github.com/kelseyhightower/envconfig
+* golang.org/x/net/publicsuffix
+
+The directory containing the ``compare`` tool also contains an executable Python 3 script named ``genConfigRoutes.py``. This script can be used to scrape the two Traffic Ops instances for API routes that resolve to generated configuration files for mid-tier and edge-tier cache servers, which can then be fed directly into the ``compare`` tool via a file or pipe. While the script itself has no actual dependencies, it *must* be run from within the full Traffic Control repository, as it impo [...]
+
+.. TODO: ^ make that last statement not a dirty lie ^
+
+.. [1] Theoretically, if you downloaded the Traffic Control repository properly (into ``$GOPATH/src/github.com/apache/trafficcontrol``), this will already be satisfied.
+
+Usage
+=====
+
+``compare``
+-----------
+Usage: compare [-hsV] [-f value] [--ref_passwd value] [--ref_url value] [--ref_user value] [-r value] [--test_passwd value] [--test_url value] [--test_user value] [parameters ...]
+
+--ref_passwd=value The password for logging into the reference Traffic Ops instance (overrides TO_PASSWORD environment variable)
+--ref_url=value The URL for the reference Traffic Ops instance (overrides TO_URL environment variable)
+--ref_user=value The username for logging into the reference Traffic Ops instance (overrides TO_USER environment variable)
+--test_passwd=value The password for logging into the testing Traffic Ops instance (overrides TEST_PASSWORD environment variable)
+--test_url=value The URL for the testing Traffic Ops instance (overrides TEST_URL environment variable)
+--test_user=value The username for logging into the testing Traffic Ops instance (overrides TEST_USER environment variable)
+-f, --file=value File listing routes to test (will read from stdin if not given)
+-h, --help Print usage information and exit
+-r, --results_path=value Directory where results will be written
+-s, --snapshot Perform comparison of all CDN's snapshotted CRConfigs
+-V, --version Print version information and exit
+
+The typical way to use ``compare`` is to first specify some environment variables:
+
+TO_URL
+ The URL of the reference Traffic Ops instance
+TO_USER
+ The username to authenticate with the reference Traffic Ops instance
+TO_PASSWORD
+ The password to authenticate with the reference Traffic Ops instance
+TEST_URL
+ The URL of the testing Traffic Ops instance
+TEST_USER
+ The username to authenticate with the testing Traffic Ops instance
+TEST_PASSWORD
+ The password to authenticate with the testing Traffic Ops instance
+
+These can be overridden by command line switches as described above. If a username and/or password is not given for the testing instance (either via environment variables or on the command line), it/they will be assumed to be the same as the one/those specified for the reference instance.
+
+genConfigRoutes.py
+------------------
+usage: genConfigRoutes.py [-h] [-k] [-v] InstanceA InstanceB LoginA [LoginB]
+
+-h, --help show this help message and exit
+-k, --insecure Do not verify SSL certificate signatures against *either* Traffic Ops instance (default: False)
+-l LOG_LEVEL, --log_level LOG_LEVEL Sets the Python log level, one of 'DEBUG', 'INFO', 'WARN', 'ERROR', or 'CRITICAL'
+-q, --quiet Suppresses all logging output - even for critical errors
+-v, --version Print version information and exit
+
+.. note:: If you're using a CDN-in-a-Box environment for testing, it's likely that you'll need the ``-k``/``--insecure`` option if you're outside the Docker network
+
+.. table:: Positional Parameters (In Order)
+
+ +-----------+---------------------------------------------------------------------------------------------------------------------------------------+
+ | Name | Description |
+ +===========+=======================================================================================================================================+
+ | InstanceA | The full URL of the first Traffic Ops instance |
+ +-----------+---------------------------------------------------------------------------------------------------------------------------------------+
+ | InstanceB | The full URL of the second Traffic Ops instance |
+ +-----------+---------------------------------------------------------------------------------------------------------------------------------------+
+ | LoginA | A login string containing credentials for logging into the first Traffic Ops instance (InstanceA) in the format 'username:password'. |
+ +-----------+---------------------------------------------------------------------------------------------------------------------------------------+
+ | LoginB | A login string containing credentials for logging into the second Traffic Ops instance (InstanceB) in the format 'username:password'. |
+ | | If not given, LoginA will be re-used for the second connection (default: None) |
+ +-----------+---------------------------------------------------------------------------------------------------------------------------------------+
+
+The genConfigRoutes.py script will output list of unique API routes (relative to the desired Traffic Ops URL) that point to generated configuration files for a sample set of servers common to both Traffic Ops instances. The results are printed to stdout, making the output perfect for piping directly into ``compare`` like so:
+
+.. code-block:: shell
+
+ ./genConfigRoutes.py https://trafficopsA.example.test https://trafficopsB.example.test username:password | ./compare
+
+\... assuming the proper environment variables have been set for ``compare``.
diff --git a/docs/source/tools/index.rst b/docs/source/tools/index.rst
new file mode 100644
index 0000000..b58bbe6
--- /dev/null
+++ b/docs/source/tools/index.rst
@@ -0,0 +1,26 @@
+..
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+..
+
+.. _tools:
+
+*****
+Tools
+*****
+This is a living list of tools used to interact with, test, and develop for the Traffic Control CDN.
+
+.. toctree::
+ :maxdepth: 2
+
+ compare
diff --git a/traffic_ops/testing/compare/README.md b/traffic_ops/testing/compare/README.md
index 1674034..e765384 100644
--- a/traffic_ops/testing/compare/README.md
+++ b/traffic_ops/testing/compare/README.md
@@ -17,38 +17,82 @@
under the License.
-->
-# Traffic Ops Compare
+The Compare Tool
+================
+The `compare` tool is used to compare the output of a set of [Traffic Ops API](https://traffic-control-cdn.readthedocs.io/en/latest/api/) endpoints between two running instances of Traffic Ops. The idea is that two different versions of Traffic Ops with the same data will have differences in the output of their API endpoints *if and only if* either the change was intentional, or a new bug was introduced in the newer version. Typically, this isn't really true, due to rapidly changing data [...]
-Use this tool to compare API output between two instances of Traffic Ops API.
-It logs in to each instance and then processes all given endpoints. Any that get
-different results are reported and written to files in the output directory
-(default ./results).
+Dependencies
+------------
+- github.com/apache/trafficcontrol/lib/go-tc[1]
+- github.com/kelseyhightower/envconfig
+- golang.org/x/net/publicsuffix
-## Requirements
+Usage
+-----
-Two Traffic Ops instances with login credentials (possibly different). The following
-environment variables must be set:
+### `compare`
-- `TO_URL` -- the *reference* Traffic Ops (e.g. production version)
-- `TO_USER` -- the username for `TO_URL`
-- `TO_PASSWORD` -- the password for `TO_URL`
+Usage: compare \[-hsV\] \[-f value\] \[--ref\_passwd value\] \[--ref\_url value\] \[--ref\_user value\] \[-r value\] \[--test\_passwd value\] \[--test\_url value\] \[--test\_user value\] \[parameters ...\]
-- `TEST_URL` -- the *test* Traffic Ops (e.g. development version)
+--ref\_passwd=value The password for logging into the reference Traffic Ops instance (overrides TO\_PASSWORD environment variable)
+--ref\_url=value The URL for the reference Traffic Ops instance (overrides TO\_URL environment variable)
+--ref\_user=value The username for logging into the reference Traffic Ops instance (overrides TO\_USER environment variable)
+--test\_passwd=value The password for logging into the testing Traffic Ops instance (overrides TEST\_PASSWORD environment variable)
+--test\_url=value The URL for the testing Traffic Ops instance (overrides TEST\_URL environment variable)
+--test\_user=value The username for logging into the testing Traffic Ops instance (overrides TEST\_USER environment variable)
+-f, --file=value File listing routes to test (will read from stdin if not given)
+-h, --help Print usage information and exit
+-r, --results\_path=value Directory where results will be written
+-s, --snapshot Perform comparison of all CDN's snapshotted CRConfigs
+-V, --version Print version information and exit
-These are optional:
+The typical way to use `compare` is to first specify some environment variables:
-- `TEST_USER` -- the username for `TO_URL` (default -- same as `TO_USER`)
-- `TEST_PASSWORD` -- the password for `TO_URL` (default -- same as `TO_PASSWORD`)
+TO\_URL
+The URL of the reference Traffic Ops instance
-## Usage
+TO\_USER
+The username to authenticate with the reference Traffic Ops instance
+TO\_PASSWORD
+The password to authenticate with the reference Traffic Ops instance
+
+TEST\_URL
+The URL of the testing Traffic Ops instance
+
+TEST\_USER
+The username to authenticate with the testing Traffic Ops instance
+
+TEST\_PASSWORD
+The password to authenticate with the testing Traffic Ops instance
+
+These can be overridden by command line switches as described above. If a username and/or password is not given for the testing instance (either via environment variables or on the command line), it/they will be assumed to be the same as the one/those specified for the reference instance.
+
+### genConfigRoutes.py
+
+usage: genConfigRoutes.py \[-h\] \[-k\] \[-v\] InstanceA InstanceB LoginA \[LoginB\]
+
+-h, --help show this help message and exit
+-k, --insecure Do not verify SSL certificate signatures against *either* Traffic Ops instance (default: False)
+-l LOG_LEVEL, --log_level LOG_LEVEL Sets the Python log level, one of 'DEBUG', 'INFO', 'WARN', 'ERROR', or 'CRITICAL' (default: CRITICAL)
+-q, --quiet Suppresses all logging output - even for critical errors
+-v, --version Print version information and exit
+
+> **note**
+>
+> If you're using a CDN-in-a-Box environment for testing, it's likely that you'll need the `-k`/`--insecure` option if you're outside the Docker network
+
+The genConfigRoutes.py script will output list of unique API routes (relative to the desired Traffic Ops URL) that point to generated configuration files for a sample set of servers common to both Traffic Ops instances. The results are printed to stdout, making the output perfect for piping directly into `compare` like so:
+
+``` sourceCode
+./genConfigRoutes.py https://trafficopsA.example.test https://trafficopsB.example.test username:password | ./compare
```
- go run compare.go [-results <dir>] [-route <API route] [-file <file of routes>] [-snapshot]
-```
-Options:
+... assuming the proper environment variables have been set for `compare`.
+
+[1] Theoretically, if you downloaded the Traffic Control repository properly (into `$GOPATH/src/github.com/apache/trafficcontrol`), this will already be satisfied.
+
-- `-results <dir>` -- directory to write difference results
-- `-route <route>` -- a specific route to compare
-- `-file <file>` -- file containing routes to check (-route takes precedence)
-- `-snapshot` -- compare snapshot for each CDN
+Further Information
+-------------------
+For more information, please see [the official documentation](https://traffic-control-cdn.readthedocs.io/en/latest/tools/compare.html)
diff --git a/traffic_ops/testing/compare/genConfigRoutes.py b/traffic_ops/testing/compare/genConfigRoutes.py
index 1542230..c4720e6 100755
--- a/traffic_ops/testing/compare/genConfigRoutes.py
+++ b/traffic_ops/testing/compare/genConfigRoutes.py
@@ -267,7 +267,7 @@ if __name__ == '__main__':
version="%(prog)s v"+__version__)
parser.add_argument("-l", "--log_level",
help="Sets the Python log level, one of 'DEBUG', 'INFO', 'WARN', 'ERROR', "\
- "OR 'CRITICAL'",
+ "or 'CRITICAL'",
type=str,
default="INFO")
parser.add_argument("-q", "--quiet",