You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficcontrol.apache.org by GitBox <gi...@apache.org> on 2020/10/13 15:42:50 UTC
[GitHub] [trafficcontrol] zrhoffman opened a new pull request #5144: Prohibit adding a cache group with no servers to a topology
zrhoffman opened a new pull request #5144:
URL: https://github.com/apache/trafficcontrol/pull/5144
<!--
************ STOP!! ************
If this Pull Request is intended to fix a security vulnerability, DO NOT submit it! Instead, contact
the Apache Software Foundation Security Team at security@trafficcontrol.apache.org and follow the
guidelines at https://www.apache.org/security/ regarding vulnerability disclosure.
-->
## What does this PR (Pull Request) do?
<!-- Explain the changes you made here. If this fixes an Issue, identify it by
replacing the text in the checkbox item with the Issue number e.g.
- [x] This PR fixes #9001 OR is not related to any Issue
^ This will automatically close Issue number 9001 when the Pull Request is
merged (The '#' is important).
Be sure you check the box properly, see the "The following criteria are ALL
met by this PR" section for details.
-->
- [x] This PR fixes #5143<!-- You can check for an issue here: https://github.com/apache/trafficcontrol/issues -->
- This PR adds validation to require that each cachegroup in a topology contains at least 1 server.
## Which Traffic Control components are affected by this PR?
<!-- Please delete all components from this list that are NOT affected by this
Pull Request. Also, feel free to add the name of a tool or script that is
affected but not on the list.
Additionally, if this Pull Request does NOT affect documentation, please
explain why documentation is not required. -->
- Traffic Ops
## What is the best way to verify this PR?
<!-- Please include here ALL the steps necessary to test your Pull Request. If
it includes tests (and most should), outline here the steps needed to run the
tests. If not, lay out the manual testing procedure and please explain why
tests are unnecessary for this Pull Request. -->
- Run the Traffic Ops API tests
- Try to create a topology that contains a cachegroup with no servers in it, make sure the POST fails
- Try to update a topology that contains a cachegroup with no servers in it, make sure the PUT fails
## The following criteria are ALL met by this PR
<!-- Check the boxes to signify that the associated statement is true. To
"check a box", replace the space inside of the square brackets with an 'x'.
e.g.
- [ x] <- Wrong
- [x ] <- Wrong
- [] <- Wrong
- [*] <- Wrong
- [x] <- Correct!
-->
- [x] This PR includes tests
- [x] This PR includes documentation
- [x] This PR includes an update to CHANGELOG.md
- [x] This PR includes any and all required license headers
- [x] This PR does not include a database migration
- [x] This PR **DOES NOT FIX A SERIOUS SECURITY VULNERABILITY** (see [the Apache Software Foundation's security guidelines](https://www.apache.org/security/) for details)
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on pull request #5144: Prohibit adding a cache group with no servers to a topology
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on pull request #5144:
URL: https://github.com/apache/trafficcontrol/pull/5144#issuecomment-708005548
Got CDN in a Box working in 10424bb410. I'm making this PR a draft until the API tests pass.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp merged pull request #5144: Prohibit adding a cache group with no servers to a topology
Posted by GitBox <gi...@apache.org>.
rawlinp merged pull request #5144:
URL: https://github.com/apache/trafficcontrol/pull/5144
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #5144: Prohibit adding a cache group with no servers to a topology
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #5144:
URL: https://github.com/apache/trafficcontrol/pull/5144#discussion_r504839135
##########
File path: traffic_ops/traffic_ops_golang/topology/topologies.go
##########
@@ -563,6 +614,23 @@ JOIN topology_cachegroup tc on t.name = tc.topology
return query
}
+func selectEmptyCacheGroupsQuery() string {
+ // language=SQL
+ query := `
+ SELECT c."name", COUNT((
+ SELECT s2.id FROM server s2
+ JOIN "type" t ON s."type" = t.id
+ WHERE s2.id = s.id
+ )) server_count
+ FROM cachegroup c
+ LEFT JOIN "server" s ON c.id = s.cachegroup
+ WHERE c."name" = ANY(CAST(:cachegroup_names AS TEXT[]))
+ GROUP BY c."name"
+ ORDER BY server_count
Review comment:
Due to the `LEFT JOIN` making `COUNT(*)` include `NULL`s, it would need to be something like `COUNT(*) FILTER (WHERE s.id IS NOT NULL)`.
Simplified in d5a2ffced
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #5144: Prohibit adding a cache group with no servers to a topology
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #5144:
URL: https://github.com/apache/trafficcontrol/pull/5144#discussion_r504821645
##########
File path: CHANGELOG.md
##########
@@ -21,6 +21,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
- Traffic Ops: Added validation to prohibit assigning caches to topology-based delivery services
- Traffic Ops: Added validation to prohibit removing a capability from a server if no other server in the same cachegroup can satisfy the required capabilities of the delivery services assigned to it via topologies.
- Traffic Ops: Added validation to ensure that at least one server per cachegroup in a delivery service's topology has the delivery service's required capabilities.
+ - Traffic Ops: Added validation to ensure that at least one server exists in each cachegroup.
Review comment:
Specified in d6417bfa6
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #5144: Prohibit adding a cache group with no servers to a topology
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #5144:
URL: https://github.com/apache/trafficcontrol/pull/5144#discussion_r504839135
##########
File path: traffic_ops/traffic_ops_golang/topology/topologies.go
##########
@@ -563,6 +614,23 @@ JOIN topology_cachegroup tc on t.name = tc.topology
return query
}
+func selectEmptyCacheGroupsQuery() string {
+ // language=SQL
+ query := `
+ SELECT c."name", COUNT((
+ SELECT s2.id FROM server s2
+ JOIN "type" t ON s."type" = t.id
+ WHERE s2.id = s.id
+ )) server_count
+ FROM cachegroup c
+ LEFT JOIN "server" s ON c.id = s.cachegroup
+ WHERE c."name" = ANY(CAST(:cachegroup_names AS TEXT[]))
+ GROUP BY c."name"
+ ORDER BY server_count
Review comment:
Due to the `LEFT JOIN` making `COUNT(*)` include `NULL`s, it would need to be something like `COUNT(*) FILTER (WHERE s.id IS NOT NULL)`.
Simplified in 118863a5e
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #5144: Prohibit adding a cache group with no servers to a topology
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #5144:
URL: https://github.com/apache/trafficcontrol/pull/5144#discussion_r504821754
##########
File path: traffic_ops/traffic_ops_golang/topology/topologies.go
##########
@@ -146,13 +149,61 @@ func (topology *TOTopology) Validate() error {
return util.JoinErrs(tovalidate.ToErrors(rules))
}
+func (topology *TOTopology) checkForEmptyCacheGroups() error {
+ var (
+ cacheGroupNames = make([]string, len(topology.Nodes))
+ baseError = errors.New("unable to check for cachegroups with no servers")
+ systemError = "checking for cachegroups with no servers: %s"
+ parameters = map[string]interface{}{}
+ query = selectEmptyCacheGroupsQuery()
+ )
+ for index, node := range topology.Nodes {
+ cacheGroupNames[index] = node.Cachegroup
+ }
+ parameters["cachegroup_names"] = pq.Array(cacheGroupNames)
+
+ rows, err := topology.ReqInfo.Tx.NamedQuery(query, parameters)
+ if err != nil {
+ log.Errorf(systemError, err.Error())
+ return baseError
+ }
+
+ var (
+ serverCount int
+ cacheGroup string
+ cacheGroups []string
+ )
+ defer log.Close(rows, "unable to close DB connection when checking for cachegroups with no servers")
+ for rows.Next() {
+ if err := rows.Scan(&cacheGroup, &serverCount); err != nil {
+ log.Errorf(systemError, err.Error())
+ return baseError
+ }
+ if serverCount != 0 {
+ break
+ }
+ cacheGroups = append(cacheGroups, cacheGroup)
+ }
+
+ if cacheGroups != nil {
Review comment:
Rewrote condition in 9f7b44cf5
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #5144: Prohibit adding a cache group with no servers to a topology
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #5144:
URL: https://github.com/apache/trafficcontrol/pull/5144#discussion_r504792069
##########
File path: CHANGELOG.md
##########
@@ -21,6 +21,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
- Traffic Ops: Added validation to prohibit assigning caches to topology-based delivery services
- Traffic Ops: Added validation to prohibit removing a capability from a server if no other server in the same cachegroup can satisfy the required capabilities of the delivery services assigned to it via topologies.
- Traffic Ops: Added validation to ensure that at least one server per cachegroup in a delivery service's topology has the delivery service's required capabilities.
+ - Traffic Ops: Added validation to ensure that at least one server exists in each cachegroup.
Review comment:
"... that that is used in a Topology"
##########
File path: traffic_ops/traffic_ops_golang/topology/topologies.go
##########
@@ -563,6 +614,23 @@ JOIN topology_cachegroup tc on t.name = tc.topology
return query
}
+func selectEmptyCacheGroupsQuery() string {
+ // language=SQL
+ query := `
+ SELECT c."name", COUNT((
+ SELECT s2.id FROM server s2
+ JOIN "type" t ON s."type" = t.id
+ WHERE s2.id = s.id
+ )) server_count
+ FROM cachegroup c
+ LEFT JOIN "server" s ON c.id = s.cachegroup
+ WHERE c."name" = ANY(CAST(:cachegroup_names AS TEXT[]))
+ GROUP BY c."name"
+ ORDER BY server_count
Review comment:
Can this simply be
```
SELECT c."name", COUNT(*) AS server_count
FROM cachegroup c
LEFT JOIN "server" s ON c.id = s.cachegroup
WHERE c."name" = ANY(CAST(:cachegroup_names AS TEXT[]))
GROUP BY c."name"
ORDER BY server_count;
```
instead?
##########
File path: traffic_ops/traffic_ops_golang/topology/topologies.go
##########
@@ -146,13 +149,61 @@ func (topology *TOTopology) Validate() error {
return util.JoinErrs(tovalidate.ToErrors(rules))
}
+func (topology *TOTopology) checkForEmptyCacheGroups() error {
+ var (
+ cacheGroupNames = make([]string, len(topology.Nodes))
+ baseError = errors.New("unable to check for cachegroups with no servers")
+ systemError = "checking for cachegroups with no servers: %s"
+ parameters = map[string]interface{}{}
+ query = selectEmptyCacheGroupsQuery()
+ )
+ for index, node := range topology.Nodes {
+ cacheGroupNames[index] = node.Cachegroup
+ }
+ parameters["cachegroup_names"] = pq.Array(cacheGroupNames)
+
+ rows, err := topology.ReqInfo.Tx.NamedQuery(query, parameters)
+ if err != nil {
+ log.Errorf(systemError, err.Error())
+ return baseError
+ }
+
+ var (
+ serverCount int
+ cacheGroup string
+ cacheGroups []string
+ )
+ defer log.Close(rows, "unable to close DB connection when checking for cachegroups with no servers")
+ for rows.Next() {
+ if err := rows.Scan(&cacheGroup, &serverCount); err != nil {
+ log.Errorf(systemError, err.Error())
+ return baseError
+ }
+ if serverCount != 0 {
+ break
+ }
+ cacheGroups = append(cacheGroups, cacheGroup)
+ }
+
+ if cacheGroups != nil {
Review comment:
nit: might read better as `len(cacheGroups) > 0`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on pull request #5144: Prohibit adding a cache group with no servers to a topology
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on pull request #5144:
URL: https://github.com/apache/trafficcontrol/pull/5144#issuecomment-708093117
Removed draft status, PR is ready for review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org