[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6358: Traffic Router Ultimate Test Harness blueprint

[GitBox <gi...@apache.org>]

rawlinp commented on a change in pull request #6358:
URL: https://github.com/apache/trafficcontrol/pull/6358#discussion_r765334389



##########
File path: blueprints/tr-load-tests.md
##########
@@ -0,0 +1,296 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Traffic Router Ultimate Test Harness <!-- a concise title for the new feature this blueprint will describe -->
+
+## Problem Description
+<!--
+*What* is being asked for?
+*Why* is this necessary?
+*How* will this be used?
+-->
+As the entrypoint of an Apache Traffic Control CDN, Traffic Router is the most exposed, most critical component, and it must be able to route traffic quickly and at a very high rate. Although Traffic Router has met this requirement in the past, if Traffic Router undergoes no performance tests as it evolves, there is no guarantee that its performance will not decline with time.
+
+Important times to run performance tests:
+
+* When traffic volume to your CDN changes significantly
+* When making harware changes to the server hosting the Traffic Router instance
+* When upgrading Traffic Router to a new Apache Traffic Control version
+* When developing or maintaining a Traffic Router feature that could have an impact on any aspect of Traffic Router performance, including before, during, and after pull request review
+* When a commit that modifies Traffic Router is pushed to a GitHub branch, for all Apache Traffic Control branches
+
+A load test for Delivery Services exists in the project at [`/test/router`](https://github.com/apache/trafficcontrol/tree/RELEASE-6.0.1/test/router), but
+- It has not been maintained over time, currently does not work
+- Only tests HTTP Delivery Services
+- Does not support testing Coverage Zone Maps
+- Does not use the TO Client Library for requests to Traffic Ops
+- Prompts only for the number of requests to make to Delivery Services, not a length of time to run the test
+- Does not fail if some minimum threshold of requests per second is not met
+- Is not configurable in other ways, such as length of paths generated for HTTP requests to Delivery Service, which client IP address to use
+
+## Proposed Change
+<!--
+*How* will this be implemented (at a high level)?
+-->
+The Traffic Router Ultimate Test Harness will include an end-to-end performance test suite verify that the features of Traffic Router meet expected performance thresholds, as well as additional end-to-end tests of other Traffic Router features.
+
+The TR Ultimate Test Harness may extend [`/test/router`](https://github.com/apache/trafficcontrol/tree/RELEASE-6.0.1/test/router) where possible, but it should not limit itself for that secondary goal.
+
+### Traffic Portal Impact
+<!--
+*How* will this impact Traffic Portal?
+What new UI changes will be required?
+Will entirely new pages/views be necessary?
+Will a new field be added to an existing form?
+How will the user interact with the new UI changes?
+-->
+No Traffic Portal impact is anticipated.
+
+### Traffic Ops Impact
+<!--
+*How* will this impact Traffic Ops (at a high level)?
+-->
+The Operations-level TR Ultimate Test Harness user will update the status of any Traffic Router that is currently undergoing load testing to `"OFFLINE"` in Traffic Ops. It should also update the status of that Traffic Router to its previous status once the load testing is complete, regardless of whether the test failed or succeeded.

Review comment:
       There seems to be a minor discrepancy between this line and L260, which states that an operator may want to set the TR to OFFLINE manually. Which line is correct? Also, I'm not positive, but I _think_ setting the TR under test to OFFLINE would affect its responses in that it would exclude itself from any responses.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org