You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficcontrol.apache.org by GitBox <gi...@apache.org> on 2021/08/10 18:35:36 UTC
[GitHub] [trafficcontrol] rimashah25 opened a new pull request #6095: Layered Profile Blueprint
rimashah25 opened a new pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095
<!--
Thank you for contributing! Please be sure to read our contribution guidelines: https://github.com/apache/trafficcontrol/blob/master/CONTRIBUTING.md
If this closes or relates to an existing issue, please reference it using one of the following:
Closes: #ISSUE
Related: #ISSUE
If this PR fixes a security vulnerability, DO NOT submit! Instead, contact
the Apache Traffic Control Security Team at security@trafficcontrol.apache.org and follow the
guidelines at https://apache.org/security regarding vulnerability disclosure.
-->
-----------------------------------------------
<!-- **^ Add meaningful description above** -->
This PR is not related to any issue. This is a blueprint to propose layered profiles a simple way to assign profiles to Delivery Services and Servers.
## Which Traffic Control components are affected by this PR?
<!-- Please delete all components from this list that are NOT affected by this PR.
Feel free to add the name of a tool or script that is affected but not on the list.
-->
- Documentation
## What is the best way to verify this PR?
<!-- Please include here ALL the steps necessary to test your PR.
If your PR has tests (and most should), provide the steps needed to run the tests.
If not, please provide step-by-step instructions to test the PR manually and explain why your PR does not need tests. -->
Go through the blueprint and make sure it looks correct.
## If this is a bugfix, which Traffic Control versions contained the bug?
<!-- Delete this section if the PR is not a bugfix, or if the bug is only in the master branch.
Examples:
- 5.1.2
- 5.1.3 (RC1)
-->
- master
## PR submission checklist
- [x] This PR has tests <!-- If not, please delete this text and explain why this PR does not need tests. -->
- [x] This PR has documentation <!-- If not, please delete this text and explain why this PR does not need documentation. -->
- [x] This PR has a CHANGELOG.md entry <!-- A fix for a bug from an ATC release, an improvement, or a new feature should have a changelog entry. -->
- [x] This PR **DOES NOT FIX A SERIOUS SECURITY VULNERABILITY** (see [the Apache Software Foundation's security guidelines](https://apache.org/security) for details)
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r687933682
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
Review comment:
That will increase the complexity of checking for request preconditions. We also have join tables (actually just one, that I know of) that use triggers instead of their own `last_updated`, which I think is worth using moving forward because of how easy it makes things.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688135483
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
Review comment:
This seems more necessary for Server Profiles than it does for Delivery Service Profiles. The latter usually contain much fewer parameters (< 10 usually) and do not typically vary on things like hardware, CDN, datacenter, etc., like servers might.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688578048
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
Review comment:
Ah, I see that now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-898800486
On other hand, I'm with Rawlin on Servers and Delivery Services being our two most requested objects/endpoints. Maybe not in terms of "things that use the API" but almost certainly in terms of "requests that hit the API". It's also very troubling that if even one server uses a layered Profile `GET /servers` would fail completely. Could the API instead feign ignorance of things with layered Profiles in lower API versions than the one that supports it? like if you do `GET /api/4.0/servers` it returns a list of all servers that _don't_ have layered profiles, and maybe a warning alert that tells you some were omitted (maybe even which, by hostname or ID) and why. PUTs can still fail for layered-profile servers, POSTs can tell the truth if you try to create one that doesn't appear in the results of a GET - that the server actually exists but you're not seeing it because your API version isn't high enough to support its structure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] shamrickus commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
shamrickus commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r687908399
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
Review comment:
We have last updated on pretty much every joining table like this (`topology_cachegroup_parents`, `server_server_capability`, `federation_federation_resolver`, etc.). For consistency I'd think we should keep `last_updated`.
I don't actually know what we use them for but I imagine for "auditing"/debugging
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r770937516
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,296 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+- **_Note_**: All fields in the response structure for API 4.0/ 5.0 except `profileId`, `profileDesc` and `profile`, remain same.
+ - In API 4.0/5.0 `profile` changes to `profiles` and in API 5.0 `profileId` and `profileDesc` are no longer part of response structure
Review comment:
I hought the agreed upon direction was to make the API output a list of length one for a full major version, then move to an arbitrary length list only once all API versions that use `profile`/`profileName`/`profileId` have been _fully removed_? Because the question was raised of how to handle backwards comparability for those old APIs. Since APIv4 is unstable, API version 3 is not yet deprecated, so it can't be removed until a full major ATC version passes where APIv4 is stable and APIv3 is deprecated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rimashah25 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rimashah25 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r767912297
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 5.0+",
+ "level":"error"
+ }
+]}
+```
+
+#### API 6.0 GET
Review comment:
umm.. why?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r768052145
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 5.0+",
+ "level":"error"
+ }
+]}
+```
+
+#### API 6.0 GET
Review comment:
I believe the plan was to make the `profileId` -> `profiles: []` data type change in API 4.0 since it's currently unstable. That is one of the 3ish breaking changes we currently had planned for the unstable API 4.0, meaning we wouldn't need an API 5.0 for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r770938304
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,296 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+- **_Note_**: All fields in the response structure for API 4.0/ 5.0 except `profileId`, `profileDesc` and `profile`, remain same.
+ - In API 4.0/5.0 `profile` changes to `profiles` and in API 5.0 `profileId` and `profileDesc` are no longer part of response structure
Review comment:
We definitely can and should just get rid of `profileId` and `profileDesc` immediately in APIv4, though
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-905683044
Currently we restrict servers such that no two may share a Profile and a service IP address - how will that constraint change when servers have multiple Profiles? Will they be disallowed to share a service IP and _any_ of their Profiles? Any of their Profiles in the same order? Only the exact same sequence of Profiles? Or should the constraint be lifted (which would be my vote immediately without thinking through the implications - that could be a bad idea, idk)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688561286
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
Review comment:
I don't think it would ever be necessary in practice, even if it is more consistent. The same goes for Origins, they have a profile, yet we shouldn't make them layered because they only really have one or two possible parameters that I know of. It would be a large chunk of work for very little value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688138133
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
Review comment:
Since profile names are unique, can we use them instead of profile IDs in the DB?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] mitchell852 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
mitchell852 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r690709274
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
Review comment:
i agree. layered profiles should be limited to server profiles (where it is really needed). trust me. lots of work would be required to support ds profiles (and DSRs) for little benefit.
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
Review comment:
see my other comment. IMHO layered profiles should be limited to server profiles (which is actually needed due to the enormous amount of params). trying to do this for delivery service profiles would be a large amount of work (see TP DS and DSR view changes) for little benefit.
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
Review comment:
I would strongly, strongly suggest limiting this effort to server profiles....
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r687866989
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
Review comment:
Is it necessary to specify both the IDs and the Names? Seems like that will cause a couple of annoying edge case checking that could be skipped if they're only identified once (and I would vote that it be by name).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-898594701
> So instead of breaking getting things with profiles, we break changing the profiles or assigning things to them? I'm not seeing how that's a win, they both seem about the same.
It's a win because we wouldn't be breaking one of our most-used APIs entirely for users of prior API versions. Instead, we're only breaking profile-param assignments, _when the profile is layered_, which you're not even supposed to do. So it "breaks" something that you're not supposed to do, rather than breaking something that you _are_ supposed to do (updating a server).
Additionally, I would argue that placing the ordering of the profiles on the server object itself is bad, because, in general, all servers with the same characteristics are going to share the same order of layered profiles. My moving the order into the profile object itself, you don't have to specify the order on every single server you're using it for. Instead, you only have to get the order right once, and you can't accidentally change that ordering from the servers page in the UI.
As an operator, using the "profile list" idea, I still have to look at what profiles are in the list and figure out which profile to update. Using the "profile of profiles" idea, it could be presented exactly the same in the UI (as a list of profiles), so there wouldn't be any extra steps or pages to click through. You have to design your "profile of profiles", which would be the same amount of work as defining the order of the "profile list", except you don't have to worry about the ordering when assigning it to servers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-897960910
> One other thing I just thought of: how is backwards compatibility handled for objects with layered Profiles? If I have a server with:
I put a lot of thought into this in my initial proposal. I came up with some options, but they're _really_ hacky.
Honestly, I think the best solution is probably to return a 4xx code from /servers and /deliveryservices if a user has assigned more than 1 Profile. And to warn users when they do, that this will break old clients. Which is _mostly_ ok in practice, it _mostly_ means people just have to upgrade all ATC components and clients before they start using Layered Profiles.
I don't suggest that lightly. IMO breaking backward compatibility is one of the worst things we can do, in terms of usability, maintaining users, I could go on. But I honestly think the alternative is probably worse here.
See https://cwiki.apache.org/confluence/display/TC/Layered+Profiles for some of my compatibility ideas.
> I assume it'd never choose a middle Profile and would always choose consistently, but should we use the "bottom" Profile
We can't return any single real profile. If someone assigned multiple Profiles, those Parameters, in that exact order, are necessary. Anything that looks at a single profile for params will be broken. If you look at that wiki link, most of my ideas are around creating a "fake" profile. But IDs and names make that hard, and you still can't edit it, so POST/PUT still return error codes. Unless you did even more hacks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rimashah25 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rimashah25 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r767913836
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 5.0+",
+ "level":"error"
+ }
+]}
+```
+
+#### API 6.0 GET
+- JSON **response** with the proposed change will look as follows:
+ `/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 6.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+return following **response**
+```JSON
+{ "alerts": [
+ {
+ "text": "Server created/updated",
+ "level": "success"
+ }
+],
+ "response": {
+ "cachegroup": "CDN_in_a_Box_Mid",
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "id": 5,
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ "profileDesc": "Mid Cache - Apache Traffic Server, Amiga 123 Machine, CDN Foo",
+ "profileIds": [10, 11, 15],
+ ⋮
+ }
+}
+```
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------------| ---------------------| ----------- | ---------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| layeredProfiles | text | required | the profile names associated with a server |
Review comment:
You are right, i'll change it the description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rimashah25 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rimashah25 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r771544280
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,296 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+- **_Note_**: All fields in the response structure for API 4.0/ 5.0 except `profileId`, `profileDesc` and `profile`, remain same.
+ - In API 4.0/5.0 `profile` changes to `profiles` and in API 5.0 `profileId` and `profileDesc` are no longer part of response structure
Review comment:
This is really confusing me. I thought the plan was to implement the solution across two API end points to maintain backward compatibility. Now, I am reading that API 4.0 is unstable and we can totally implement this in one API call with a breaking change. The whole reason I have multiple comments on this blueprint was because I mentioned that the API was going to be a breaking change and may not maintain backward compatibility.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] mitchell852 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
mitchell852 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r706378455
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
Review comment:
@rob05c @rimashah25 ^^ you ok with that? limit this to server profiles for our first pass?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r759519801
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
Review comment:
I'd just like to note, we just had a production outage on a service a week ago because someone copied a Delivery Service Profile, which contained cache key settings, resulting in 2 services inadvertently using the same cache key. An outage that had me personally working all Friday evening to fix.
That's exactly the kind of problem Layered Profiles is designed to mitigate.
Layered Profiles doesn't explicitly prevent anything, but if the DSes had profiles like "GENERIC_MSO" ,"GENERIC_RETRY_SETTINGS", and "DS_FOO," it would've made the issue less likely to occur, just like it does for Servers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r687934605
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+"fk_deliveryservice" FOREIGN KEY (deliveryservice) REFERENCES deliveryservice(id)
+```
+
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile | bigint | | not null |
+ order | bigint | | not null |
+ last_updated | timestamp with time zone | | not null | now()
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+### ORT Impact
+New set of profiles will be created and the profile-parameter relationship will change.
+
+### Traffic Monitor Impact
+No impact
+
+### Traffic Router Impact
+No impact
+
+### Traffic Stats Impact
+No impact
+
+### Traffic Vault Impact
+No impact
+
+### Documentation Impact
+All existing endpoints will need to be updated, along with the documentation explaining `layered_profile`.
+
+### Testing Impact
+Client/API integration tests should be updated to verify the `layered_profile` functionality on existing API `/deliveryservices`, `/servers` endpoints.
+
+### Performance Impact
+We do not anticipate any performance impact with layered_profile approach.
+
+### Security Impact
+We do not anticipate any impact on security.
+
+### Upgrade Impact
+- A Database Migration to:
+ - drop profile column in existing deliveryservice and server table
+ - insert existing server and DS profiles along with their order into the new tables(deliveryservice_profiles, server_profiles)
+- The new capability can just be added to the `seeds.sql` file.
Review comment:
But you said there's no new endpoint like e.g. `deliveryservices/{{ID}}/profiles`, so shouldn't the ability to modify DS/server Profiles be covered by `ds-write` and `server-write`, which today already include the ability to modify Profile assignments to those objects?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688146514
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
Review comment:
We should probably require that all profiles assigned to a given server all have the same values of:
- type
- cdn
- routing_disabled
If any of those differ within the same server, it's probably a mistake.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-898550802
> it seems like using some kind of tool outside of TO to do the profile layering is sounding better and better
-1, It fundamentally doesn't solve the problem. The goal here is to make operating ATC/TO/TP easier. Right now, our Profiles are an unmaintainable mess, they're so huge, and 95% duplicated, the Parameters are constantly wrong and nobody knows it. Doing some kind of outside "generation" doesn't solve the problem. If anything, it makes it worse, because now operators aren't allowed to actually use TO/TP to modify things, so operating becomes even harder. Or if they do, the generation is out of sync, making it even more dangerous and error-prone.
> since it seems like we never want to stop adding to the monolith
I'm a pretty big proponent of getting rid of the monolith. But TO is our data store. The schema we have today is not operationally workable, and that's what Layered Profiles fixes. Generating what amounts to a denormalized blob and shoving that in the database is even more difficult and error-prone to operate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688179266
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
Review comment:
Is routing_disabled used? I thought that was legacy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 edited a comment on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 edited a comment on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-902206714
Okay, rather than "our two most used" they are merely "two of our most used". That's 35,000 failed requests if we break those endpoints.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] TaylorCFrey commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
TaylorCFrey commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r750409153
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
Review comment:
Seems reasonable to keep this focused to server profiles for the first iteration (assuming these are separable)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688566040
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
Review comment:
I just saw the prior two comments. I think having `last_updated` on the join tables as well as the main tables would be necessary if we have APIs that just return the "join" data. E.g. if we had an API that returned all the server_server_capability data, it would make sense to have a last_updated on that.
For _this_ feature, I'm not sure it would even be necessary to add a trigger though, because a PUT is going to update all the server fields anyways, including the new profile list, which will trigger the last_updated trigger on the server table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688214599
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
Review comment:
+1 on names. We might need to make sure the DB field cascades if the name changes, but otherwise it should be fine.
IMO the advantages almost always significantly outweigh the disadvantages for using a real candidate key (which may be compound) instead of an artificial ID.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r687871971
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+"fk_deliveryservice" FOREIGN KEY (deliveryservice) REFERENCES deliveryservice(id)
Review comment:
`profile` should probably also be a foreign key into `profile`, right?
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
Review comment:
This should probably use a Primary Key containing not just profile, or two delivery services would not be able to share any single Profile. Seems like it needs `PRIMARY KEY (profile, deliveryservice, order)`, so that no single Delivery Service may have the same Profile in the same position.
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+"fk_deliveryservice" FOREIGN KEY (deliveryservice) REFERENCES deliveryservice(id)
+```
+
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile | bigint | | not null |
+ order | bigint | | not null |
+ last_updated | timestamp with time zone | | not null | now()
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile)
Review comment:
Same as above RE: Primary Key
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
Review comment:
Also
- `/deliveryservice_requests`
- `/deliveryservice_requests/{{ID}}/assign`
- `/deliveryservice_requests/{{ID}}/status`
- `/deliveryservices/{{ID}}/safe`
- `/deliveryservices/{{ID}}/servers`
- `/deliveryservices/{{ID}}/servers/eligible`
- `/deliveryservices/{{XMLID}}/servers`
- `/servers/details`
- `/servers/{{ID}}/deliveryservices`
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
Review comment:
Is it necessary to specify both the IDs and the Names? Seems like that will cause a bunch of annoying edge case checking that could be skipped if they're only identified once (and I would vote that it be by name).
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
Review comment:
Does this actually need a `last_updated`? When would that be used or displayed? Could we instead just include a trigger to change the `last_updated` of affected Delivery Services when changes are made in this table?
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+"fk_deliveryservice" FOREIGN KEY (deliveryservice) REFERENCES deliveryservice(id)
+```
+
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile | bigint | | not null |
+ order | bigint | | not null |
+ last_updated | timestamp with time zone | | not null | now()
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+### ORT Impact
+New set of profiles will be created and the profile-parameter relationship will change.
+
+### Traffic Monitor Impact
+No impact
+
+### Traffic Router Impact
+No impact
+
+### Traffic Stats Impact
+No impact
+
+### Traffic Vault Impact
+No impact
+
+### Documentation Impact
+All existing endpoints will need to be updated, along with the documentation explaining `layered_profile`.
+
+### Testing Impact
+Client/API integration tests should be updated to verify the `layered_profile` functionality on existing API `/deliveryservices`, `/servers` endpoints.
+
+### Performance Impact
+We do not anticipate any performance impact with layered_profile approach.
+
+### Security Impact
+We do not anticipate any impact on security.
+
+### Upgrade Impact
+- A Database Migration to:
+ - drop profile column in existing deliveryservice and server table
+ - insert existing server and DS profiles along with their order into the new tables(deliveryservice_profiles, server_profiles)
+- The new capability can just be added to the `seeds.sql` file.
Review comment:
What new ~~Permission~~ capability?
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
Review comment:
Changes to DSR views will also be necessary to properly display changes proposed for swapping around Profiles - it may be necessary to make explicit note of the changes rather than just showing current vs proposed, because if the lists get long enough it could get very difficult to manually figure out the actual difference.
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+"fk_deliveryservice" FOREIGN KEY (deliveryservice) REFERENCES deliveryservice(id)
+```
+
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile | bigint | | not null |
+ order | bigint | | not null |
+ last_updated | timestamp with time zone | | not null | now()
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
Review comment:
same as above RE: `profile` foreign key
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
Review comment:
Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles. Should it allow specifying multiple Profiles in the filter?
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+"fk_deliveryservice" FOREIGN KEY (deliveryservice) REFERENCES deliveryservice(id)
+```
+
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile | bigint | | not null |
+ order | bigint | | not null |
+ last_updated | timestamp with time zone | | not null | now()
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+### ORT Impact
+New set of profiles will be created and the profile-parameter relationship will change.
+
+### Traffic Monitor Impact
+No impact
+
+### Traffic Router Impact
+No impact
Review comment:
These will probably be affected because the Snapshots and Monitoring Configurations for a CDN include Profile and Parameter information for the servers, Traffic Monitors, and Traffic Routers that they contain.
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+"fk_deliveryservice" FOREIGN KEY (deliveryservice) REFERENCES deliveryservice(id)
+```
+
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile | bigint | | not null |
+ order | bigint | | not null |
+ last_updated | timestamp with time zone | | not null | now()
Review comment:
Same as above RE: `last_updated`
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+"fk_deliveryservice" FOREIGN KEY (deliveryservice) REFERENCES deliveryservice(id)
+```
+
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile | bigint | | not null |
+ order | bigint | | not null |
+ last_updated | timestamp with time zone | | not null | now()
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+### ORT Impact
+New set of profiles will be created and the profile-parameter relationship will change.
+
+### Traffic Monitor Impact
+No impact
+
+### Traffic Router Impact
+No impact
+
+### Traffic Stats Impact
+No impact
+
+### Traffic Vault Impact
+No impact
+
+### Documentation Impact
+All existing endpoints will need to be updated, along with the documentation explaining `layered_profile`.
Review comment:
> ... along with the documentation explaining `layered_profile`.
That can probably go in the Profiles and Parameters overview section
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+"fk_deliveryservice" FOREIGN KEY (deliveryservice) REFERENCES deliveryservice(id)
+```
+
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile | bigint | | not null |
+ order | bigint | | not null |
+ last_updated | timestamp with time zone | | not null | now()
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+### ORT Impact
+New set of profiles will be created and the profile-parameter relationship will change.
+
+### Traffic Monitor Impact
+No impact
+
+### Traffic Router Impact
+No impact
+
+### Traffic Stats Impact
+No impact
+
+### Traffic Vault Impact
+No impact
+
+### Documentation Impact
+All existing endpoints will need to be updated, along with the documentation explaining `layered_profile`.
+
+### Testing Impact
+Client/API integration tests should be updated to verify the `layered_profile` functionality on existing API `/deliveryservices`, `/servers` endpoints.
Review comment:
and DSRs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rimashah25 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rimashah25 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r767907731
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 5.0+",
+ "level":"error"
+ }
+]}
+```
Review comment:
Corrected the error message.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rimashah25 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rimashah25 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r760363420
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+"fk_deliveryservice" FOREIGN KEY (deliveryservice) REFERENCES deliveryservice(id)
+```
+
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile | bigint | | not null |
+ order | bigint | | not null |
+ last_updated | timestamp with time zone | | not null | now()
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+### ORT Impact
+New set of profiles will be created and the profile-parameter relationship will change.
+
+### Traffic Monitor Impact
+No impact
+
+### Traffic Router Impact
+No impact
+
+### Traffic Stats Impact
+No impact
+
+### Traffic Vault Impact
+No impact
+
+### Documentation Impact
+All existing endpoints will need to be updated, along with the documentation explaining `layered_profile`.
+
+### Testing Impact
+Client/API integration tests should be updated to verify the `layered_profile` functionality on existing API `/deliveryservices`, `/servers` endpoints.
Review comment:
No longer part of this blueprint.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r759519801
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
Review comment:
I'd just like to note, we just had a production outage on a service a week ago because a Delivery Service Profile was copied, which contained cache key settings, resulting in 2 services inadvertently using the same cache key. An outage that had me personally working all Friday evening to fix.
That's exactly the kind of problem Layered Profiles is designed to mitigate.
Layered Profiles doesn't explicitly prevent anything, but if the DSes had profiles like "GENERIC_MSO" ,"GENERIC_RETRY_SETTINGS", and "DS_FOO," it would've made the issue less likely to occur, just like it does for Servers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r759522863
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
Review comment:
Not asking to change the decision here. I understand the DSR difficulties, and the consensus seems clear. Just wanted to add that incident for context.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688215177
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
Review comment:
+1 sounds like a usability improvement to me. I suspect there are a lot of API conveniences we could add like that. E.g. being able to get all Parameters associated with multiple Profiles would also be really convenient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] mitchell852 commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
mitchell852 commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-902130933
how about this? if you try to update a server that has a layered profile (1+ profiles) with a non-5.x endpoint, ignore the profile value on PUT (don't make the profile change) and return alert.type=warning and alert.message='server updated but server has a layered profile and changes to a layered profile must be made using api 5.x'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-898561848
> we could just update the profiles API to produce a "layered" profile. For example:
>``` "layeredProfiles": [
"abc",
"def"
]
```
I'm not seeing how that solves the backward-compatibiliity issue? That would still break existing clients
> So it only breaks backwards compatibility in terms of assigning parameters
So instead of breaking getting things with profiles, we break changing the profiles or assigning things to them? I'm not seeing how that's a win, they both seem about the same.
> A server would still only have one profile.
Except that we redefined what a "profile" is, and it can now be a "profile" or a "profile of profiles."
Maybe it's just me, but that seems a lot harder to understand and work with. Breaking change aside, the idea that a server/ds has a list of Profiles applied in order seems pretty straightforward. Where the idea that a server/ds has a profile, that can be a single or a "profile of profiles" seems to add another layer of abstraction and indirection, seems like it fundamentally adds more to think about. Where "a list" isn't fundamentally another layer of abstraction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688136833
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
Review comment:
+1 on identifying by name only, although I don't think this feature is necessary for delivery service profiles.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] mitchell852 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
mitchell852 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r690709274
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
Review comment:
i agree. layered profiles should be limited to server profiles (where it is really needed). trust me. lots of work would be required to support ds profiles (and DSRs) for little benefit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-898024415
> Should the API disallow multiple uses of the same Profile within the list for a single server or Delivery Service (or potentially Origin)?
Yes. Because it doesn't ever make sense. Since Parameters are applied one profile at a time, including the same profile twice will just override any of the same parameters in-between the two, and have the same effect as only having that profile in the later position.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-902818727
> We need to find a happy medium here. How about this:
I'm ok with most of those bullet points, but I have concerns with
> pre-5.0 GETs have profileId as the 1st profile in the list
causing production issues when operational scripts get partial data and apply it, potentially missing critical Parameters.
I'd like to propose an alternative, that I think addresses both our concerns:
1. In API 5.0, we change the `/servers` object `profile` and `profileId` fields to be arrays, but a PUT or POST with multiple values returns a 400 error. The UI may or may not display a list (which can only add 1), the client implements handling multiple, and the API is documented to potentially return multiple and how their Parameters must be applied.
2. In API 6.0 (or possibly 5.0 in the following TC major version wherein API 4.0 is removed), we actually implement the feature, and multiple profiles can be assigned, are displayed in the UI, etc.
The disadvantage is that it will take us a little bit longer to get the feature.
But the advantage is that it solves both our concerns: clients on supported APIs are never given 4xx errors even after the feature is in-use, and we never return or apply partial data that could produce wrong results in Production stuff.
I'd also note, we can still work on the development. We can add things like database multi-profile support, TP UI things, etc that just won't be "turned on."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-897954269
One other thing I just thought of: how is backwards compatibility handled for objects with layered Profiles? If I have a server with:
```json
{
"id": 7,
"profiles": ["foo", "bar"]
}
```
visible in APIv5, then if I request `GET /api/4.0/servers?id=7` do I see
```json
{
"id": 7,
"profile": "foo"
}
```
or
```json
{
"id": 7,
"profile": "bar"
}
```
? I assume it'd never choose a middle Profile and would always choose consistently, but should we use the "bottom" Profile `foo` because as the base Profile it's assumed to contain the bulk of the configuration and changed relatively the least? Or should it choose the "top" Profile `bar` because that's the one that is never overridden, and all of its Parameters definitely apply?
If I update a server changing its Profile through API v4 or earlier when it has been configured with multiple Profiles, does that change only the entry that would be chosen for display in a GET response, or does it replace all Profiles with the single one specified? That is, if I have
```json
{
"id": 7,
"profiles": ["foo", "bar"]
}
```
and do a `PUT /api/4.0/servers?id=7` with
```json
{
"id": 7,
"profile": "testquest"
}
```
in the body and later do a `GET /api/5.0/servers?id=7`, do I see
```json
{
"id": 7,
"profiles": ["foo", "testquest"]
}
```
or
```json
{
"id": 7,
"profiles": ["testquest", "bar"]
}
```
or
```json
{
"id": 7,
"profiles": ["testquest"]
}
```
?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r770965092
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,296 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+- **_Note_**: All fields in the response structure for API 4.0/ 5.0 except `profileId`, `profileDesc` and `profile`, remain same.
+ - In API 4.0/5.0 `profile` changes to `profiles` and in API 5.0 `profileId` and `profileDesc` are no longer part of response structure
Review comment:
The type changing from `[string]` to `[string, ...string[]]` isn't breaking at the parse level, but it is very different conceptually. Wouldn't people's scripts that look up a server's Profile be broken at that point? That is they'd silently give back incorrect results since they'd only expect to have to look at a single Profile? I guess on a certain level as long as this is communicated properly that'd be their fault, though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r770885677
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,328 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 4.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "response": [{
+ "id": 5,
+ ⋮
+ "profiles": ["MID"],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "response": [{
+ "profiles": ["MID"]
+ ⋮,
+ }
Review comment:
... and here. This doesn't break rendering (except syntax highlighting, discussed below), but it breaks parsing in the diff view on GitHub. Because the object is never closed properly, the diff highlighting doesn't capture the triple-grave accent as the end of a JSON code block, and tries to recover JSON highlighting throughout the document, resulting in:
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,328 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 4.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "response": [{
+ "id": 5,
+ ⋮
+ "profiles": ["MID"],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "response": [{
+ "profiles": ["MID"]
+ ⋮,
+ }
+}
+```
+
+#### API 4.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profiles": ["MID", "AMIGA_123", "CDN_FOO"],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
Review comment:
These vertical dots are breaking syntax highlighting of these code blocks, because they aren't valid JSON. I'd just leave them out, just only talk about the `profiles` field and make a small note before all the examples that fields that aren't changing as a result of the changes proposed by this blueprint are being left out of payloads.
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,328 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 4.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "response": [{
+ "id": 5,
+ ⋮
+ "profiles": ["MID"],
+ ⋮
+ }
Review comment:
You're missing a `]` here...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688142298
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+"fk_deliveryservice" FOREIGN KEY (deliveryservice) REFERENCES deliveryservice(id)
+```
+
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile | bigint | | not null |
+ order | bigint | | not null |
+ last_updated | timestamp with time zone | | not null | now()
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+### ORT Impact
+New set of profiles will be created and the profile-parameter relationship will change.
+
+### Traffic Monitor Impact
+No impact
+
+### Traffic Router Impact
+No impact
Review comment:
That is correct. Snapshot generation (both crconfig and monitoring) would be affected by this. Even though that is more of a TO impact, I'd still call it out in these sections.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688217070
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
Review comment:
I agree it's definitely more useful for Servers. But it's still convenient for DSes, and it's also more consistent to make everything with a Profile be Layerable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-902206714
Okay, rather than "our two most used" they are merely "two of our most used".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r767940132
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,330 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "response": [{
+ "id": 5,
+ ⋮
+ "profiles": ["MID"],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "response": [{
+ "profiles": ["MID"]
+ ⋮,
+ }
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profiles": ["MID", "AMIGA_123", "CDN_FOO"],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profiles": ["MID", "AMIGA_123", "CDN_FOO"],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 6.0",
+ "level":"error"
+ }
+]}
+```
+
+#### API 6.0 GET
+- JSON **response** with the proposed change will look as follows:
+ `/servers?id=5`
+```JSON
+{
+ "response": [{
+ "id": 5,
+ ⋮
+ "profiles": ["MID", "AMIGA_123", "CDN_FOO"],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "response": [{
+ "profiles": ["MID", "AMIGA_123", "CDN_FOO"],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 6.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profiles": ["MID", "AMIGA_123", "CDN_FOO"],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profiles": ["MID", "AMIGA_123", "CDN_FOO"],
+ ⋮
+}
+```
+
+return following **response**
+```JSON
+{ "alerts": [
+ {
+ "text": "Server created/updated",
+ "level": "success"
+ }
+],
+ "response": {
+ "cachegroup": "CDN_in_a_Box_Mid",
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "id": 5,
+ "profiles": ["MID", "AMIGA_123", "CDN_FOO"],
+ ⋮
+ }
+}
+```
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------------| ---------------------| ----------- | ---------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profiles | text | required | the profile names associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+
+**API constraints**
+- In API 5.0, GET /servers object profile and profileId fields to be arrays, but a PUT or POST with multiple values returns a 400 error.
+ The UI may or may not display a list (which can only add 1), the client implements handling multiple, and the API is documented to potentially return multiple and how their Parameters must be applied.
+- In API 6.0 (or possibly 5.0 in the following TC major version wherein API 4.0 is removed), this feature is actually implemented, and multiple profiles can be assigned, are displayed in the UI, etc.
+
+The only disadvantage to splitting it across mulitple version is a little delay to get the feature deployed.
+But the advantage is that it solves both concerns:
+- clients on supported APIs are never given 4xx errors even after the feature is in-use
+- we never return or apply partial data that could produce wrong results in production
+
+- Add `Profiles` key to API endpoints for Server objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile_name | text | | not null |
+ order | bigint | | not null |
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile_name, server, order)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
+ "fk_server" FOREIGN KEY (profile_name) REFERENCES profile(name)
+```
+
+All profiles assigned to a given server will have the same values of:
+- type
+- cdn
+- routing_disabled
+If any of those differ within the same server, it's probably a mistake.
+
+### ORT Impact
+New set of profiles will be created and the profile-parameter relationship will change.
+
+### Traffic Monitor Impact
+No impact
+
+### Traffic Router Impact
+These changes will affect the Snapshot generation (both crconfig and monitoring). Even though that is more of a TO impact.
+Reason being that Snapshots and Monitoring Configurations for a CDN include Profile and Parameter information for the servers, Traffic Monitors, and Traffic Routersz.
+
+
+### Traffic Stats Impact
+No impact
+
+### Traffic Vault Impact
+No impact
+
+### Documentation Impact
+All existing endpoints will need to be updated, along with the documentation explaining `layered_profile`.
+
+### Testing Impact
+Client/API integration tests should be updated to verify the `layered_profile` functionality on existing API `/servers` endpoints.
+
+### Performance Impact
+We do not anticipate any performance impact with layered_profile approach.
+
+### Security Impact
+We do not anticipate any impact on security.
+
+### Upgrade Impact
+- A Database Migration to:
+ - drop profile column in existing server table
+ - insert existing server and DS profiles along with their order into the new table(server_profiles)
Review comment:
I believe "and DS" should be removed here, since DS Layered Profiles was removed from the feature
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r767938991
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,330 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
Review comment:
`servers/details` needs added to the list.
https://traffic-control-cdn.readthedocs.io/en/latest/api/v4/servers_details.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] TaylorCFrey commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
TaylorCFrey commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r750412887
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
Review comment:
I'm trying to understand the "layered" approach to this. And perhaps it's a misunderstanding of the current implementation.
Do we currently only have the ability of a single profile to be assigned to a server? Does this new approach provide the ability to assign multiple profiles to a server? How does the concept of layering then get applied? That suggests there is some sort of hierarchy or order that must be enforced, since they layers are then added on top of each other.
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
Review comment:
+1 here as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r772468737
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,296 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+- **_Note_**: All fields in the response structure for API 4.0/ 5.0 except `profileId`, `profileDesc` and `profile`, remain same.
+ - In API 4.0/5.0 `profile` changes to `profiles` and in API 5.0 `profileId` and `profileDesc` are no longer part of response structure
Review comment:
I apologize. This is related to a mailing list proposal that I proposed back in late August after this blueprint was first submitted. It was [started here](https://lists.apache.org/thread/cth1xmmp5w85mkd1zoosvk55d1f622lv) and [continued here](https://lists.apache.org/thread/34v2v819yx658nls0gm5zdpslk22ozvn). Long story short: the TO API 4.0 is still considered "unstable" now. In other words, we are free to break it until such a time that we agree to consider it "stable," after which we will not make any more breaking changes to v4.0.
Before the unstable v4.0 API proposal was agreed upon, this blueprint was valid in stating that the v5.0 API would contain the new breaking changes. However, those breaking changes can now just be made in v4.0 instead, saving us from having to add yet another major TO API version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r770979868
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,296 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+- **_Note_**: All fields in the response structure for API 4.0/ 5.0 except `profileId`, `profileDesc` and `profile`, remain same.
+ - In API 4.0/5.0 `profile` changes to `profiles` and in API 5.0 `profileId` and `profileDesc` are no longer part of response structure
Review comment:
that actually makes more sense, now that I'm thinking about it. Done my way, we'd just be putting off the same problem until the APIv4/v5 transition point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688576177
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
Review comment:
That's true, our current implementation runs update queries even if nothing changes - because it doesn't check for changes. It's probably unnecessary unless postgres makes some kind of optimizations that would somehow skip unchanging updates and the associated triggers (which would surprise me). I wouldn't be opposed to adding the trigger anyway, but given that our strategy for updates isn't likely to change in this way any time soon, I agree the trigger sounds pretty optional.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-898626316
> If every combination of that requires a profile-of-profiles, now we're back to the combinatorial explosion of having to create and assign unique profiles for every 95%-identical thing with a few small changes.
That might be the case, but the combinatorial explosion doesn't also include parameters in the mess, which is the real issue here, since only the "legacy" profiles have parameters. You're still only editing the parameters of "legacy" profiles. Plus, since the ordering is important, it makes sense to not have the possibility of different orderings on every server in the same "set" by having the ordering be defined only once in the profile object.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688135724
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
Review comment:
I would argue that this feature isn't really necessary for DS profiles, just server profiles.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] dneuman64 commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
dneuman64 commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-896261522
LGTM, I assume that if users still want to use a single monolith profile that will still be supported? This is important for migration purposes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688143891
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+"fk_deliveryservice" FOREIGN KEY (deliveryservice) REFERENCES deliveryservice(id)
+```
+
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile | bigint | | not null |
+ order | bigint | | not null |
+ last_updated | timestamp with time zone | | not null | now()
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+### ORT Impact
+New set of profiles will be created and the profile-parameter relationship will change.
+
+### Traffic Monitor Impact
+No impact
+
+### Traffic Router Impact
+No impact
+
+### Traffic Stats Impact
+No impact
+
+### Traffic Vault Impact
+No impact
+
+### Documentation Impact
+All existing endpoints will need to be updated, along with the documentation explaining `layered_profile`.
+
+### Testing Impact
+Client/API integration tests should be updated to verify the `layered_profile` functionality on existing API `/deliveryservices`, `/servers` endpoints.
+
+### Performance Impact
+We do not anticipate any performance impact with layered_profile approach.
+
+### Security Impact
+We do not anticipate any impact on security.
+
+### Upgrade Impact
+- A Database Migration to:
+ - drop profile column in existing deliveryservice and server table
+ - insert existing server and DS profiles along with their order into the new tables(deliveryservice_profiles, server_profiles)
+- The new capability can just be added to the `seeds.sql` file.
+
+### Operations Impact
+The profile-parameter relationship will change based on new sets of profile and Operations will have to learn on how to assign profile order to a delivery service and/or server.
+
+### Developer Impact
+Developers will most likely need to use layered_profile, so they'll need to be familiar with the process
+of adding, sorting, deleting, debugging and working with layered_profiles.
Review comment:
I think what will be most important for developers to know is that wherever you need to look up any parameters assigned to a profile for a given server, you will need to look up _all_ profiles assigned to the server and then process all the parameters in order to get the "final" view of the profile.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rimashah25 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rimashah25 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r767912297
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 5.0+",
+ "level":"error"
+ }
+]}
+```
+
+#### API 6.0 GET
Review comment:
umm.. why? it is because we are already at 6.0.1?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688558780
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
Review comment:
It does appear to be used still at least in generating the crconfig, which is what it was for.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-898798143
I'm not really a big fan of those two approaches either. The second one where you redefine what a Profile is somewhat more promising, but I share Rob's reservations about it. It'd be nice if there was some way to just deprecate the old endpoints and introduce entirely new ones, so that they could operate totally independently, but that's not only a lot more work but difficult (for me, anyway) to decide how to design.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688178802
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
Review comment:
RE the conversation between Rob and I, that may not be the case, actually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688140069
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
Review comment:
compatibility -> compatible
Also, this makes it sound like prior API versions won't be backwards-compatible, which they have to be. Maybe it should be worded like this:
> Prior API versions will continue to work as-is, with the profileId and profileName being the 1st profile in the list, whereas v5.0 will present the entire list of profile names.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-914450659
> Currently we restrict servers such that no two may share a Profile and a service IP address - how will that constraint change when servers have multiple Profiles? Will they be disallowed to share a service IP and any of their Profiles? Any of their Profiles in the same order? Only the exact same sequence of Profiles? Or should the constraint be lifted (which would be my vote immediately without thinking through the implications - that could be a bad idea, idk)?
I've said for years, the key should be `(Host, domain, port)` or `(ip, port)`; probably both.
It'll probably take some testing to make sure everything is ok with that, but that _should_ be ok, and is really the right answer anyway.
> should the constraint be lifted
I don't think it should be removed entirely, I think we really need some kind of constraint to prevent duplicate servers. Namely the real unique network identifier: the IP-port tuple and/or hostname-port tuple.
Alternatively, if we don't want to incur the dev cost of that right now, we could probably make a database or API constraint (ideally both) for profiles-in-same-order+IP. It's probably a pretty ugly db constraint, but I think it's possible.
I don't have a strong opinion either way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rimashah25 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rimashah25 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r687911828
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+"fk_deliveryservice" FOREIGN KEY (deliveryservice) REFERENCES deliveryservice(id)
+```
+
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile | bigint | | not null |
+ order | bigint | | not null |
+ last_updated | timestamp with time zone | | not null | now()
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+### ORT Impact
+New set of profiles will be created and the profile-parameter relationship will change.
+
+### Traffic Monitor Impact
+No impact
+
+### Traffic Router Impact
+No impact
+
+### Traffic Stats Impact
+No impact
+
+### Traffic Vault Impact
+No impact
+
+### Documentation Impact
+All existing endpoints will need to be updated, along with the documentation explaining `layered_profile`.
+
+### Testing Impact
+Client/API integration tests should be updated to verify the `layered_profile` functionality on existing API `/deliveryservices`, `/servers` endpoints.
+
+### Performance Impact
+We do not anticipate any performance impact with layered_profile approach.
+
+### Security Impact
+We do not anticipate any impact on security.
+
+### Upgrade Impact
+- A Database Migration to:
+ - drop profile column in existing deliveryservice and server table
+ - insert existing server and DS profiles along with their order into the new tables(deliveryservice_profiles, server_profiles)
+- The new capability can just be added to the `seeds.sql` file.
Review comment:
Adding two new tables (deliveryservices_profiles, server_profiles).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rimashah25 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rimashah25 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r753421509
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
Review comment:
Yes and updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r760495771
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 5.0+",
+ "level":"error"
+ }
+]}
+```
+
+#### API 6.0 GET
+- JSON **response** with the proposed change will look as follows:
+ `/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 6.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+return following **response**
+```JSON
+{ "alerts": [
+ {
+ "text": "Server created/updated",
+ "level": "success"
+ }
+],
+ "response": {
+ "cachegroup": "CDN_in_a_Box_Mid",
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "id": 5,
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ "profileDesc": "Mid Cache - Apache Traffic Server, Amiga 123 Machine, CDN Foo",
+ "profileIds": [10, 11, 15],
Review comment:
+1, I think `profileDesc` is unnecessary on the server object. TP doesn't even display it, since you can click on the profile link to see its description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rimashah25 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rimashah25 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r767910828
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 5.0+",
+ "level":"error"
+ }
+]}
+```
+
+#### API 6.0 GET
+- JSON **response** with the proposed change will look as follows:
+ `/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 6.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+return following **response**
+```JSON
+{ "alerts": [
+ {
+ "text": "Server created/updated",
+ "level": "success"
+ }
+],
+ "response": {
+ "cachegroup": "CDN_in_a_Box_Mid",
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "id": 5,
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ "profileDesc": "Mid Cache - Apache Traffic Server, Amiga 123 Machine, CDN Foo",
+ "profileIds": [10, 11, 15],
Review comment:
corrected the response.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-898540943
Catching up on the conversation about backwards-compatibility, it seems like using some kind of tool _outside_ of TO to do the profile layering is sounding better and better. Or, since it seems like we never want to stop adding to the monolith, we could just update the profiles API to produce a "layered" profile. For example:
```
POST /api/4.1/profiles
{
"name": "my-new-abc-def-profile",
"type": "ATS_PROFILE",
"description": "my new layered profile that is composed of the abc and def profiles",
"cdn": "mycdn",
"routingDisabled": false,
"layeredProfiles": [
"abc",
"def"
]
}
{
"response": {
"name": "my-new-abc-def-profile",
"type": "ATS_PROFILE",
"id": 123,
"description": "my new layered profile that is composed of the abc and def profiles",
"cdn": "mycdn",
"routingDisabled": false,
"layeredProfiles": [
"abc",
"def",
]
...
}
}
```
This new profile would then just be the same as any other profile except that you can't assign parameters directly to it, you can only assign parameters to its individual layers. So it only breaks backwards compatibility in terms of assigning parameters to profiles when they're layered, rather than breaking our bread and butter servers API. A server would still only have one profile.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r687871227
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
Review comment:
This should probably use a Primary Key containing not just profile, or two Delivery Services would not be able to share any single Profile. Seems like it needs `PRIMARY KEY (profile, deliveryservice, order)`, so that no single Delivery Service may have the same Profile in the same position.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-898000264
> ... if there were multiple, now they just wiped them out, destroying data.
sure, but that's what you'd expect to happen, same as if you did a PUT with `"profiles": ["a single profile"]`.
It's just that these are probably two of the three most-used endpoints in the API. I'm not sure just a warning alert is enough when you're breaking probably nearly every single client of the API in existence already by the time you see it. That should definitely come with a confirmation dialog or something in TP, though I don't think there's an equivalent for raw API calls.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688566040
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
Review comment:
I just saw the prior two comments. I think having `last_updated` on the join tables as well as the main tables would be necessary if we have APIs that just return the "join" data? E.g. if we had an API that returned all the server_server_capability data, it would make sense to have a last_updated on that.
For _this_ feature, I'm not sure it would even be necessary to add a trigger though, because a PUT is going to update all the server fields anyways, including the new profile list, which will trigger the last_updated trigger on the server table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-902156747
> I'm with Rawlin on Servers and Delivery Services being our two most requested objects/endpoints
Do you have data to back that up?
I'm looking at our production requests for these endpoints over an arbitrary period, and I see about 10,000 profile requests, about 15,000 deliveryservice requests, about 125,000 parameter requests, and about 20,000 server requests. The data I'm seeing suggests none of these 4 endpoints are greatly more than the others, though parameters seem to be approaching order-of-magnitude.
Moreover, I maintain it also makes conceptual sense. My intuition is that most large scripts will need all 4, and that small scripts likely need profiles and parameters about as often as delivery services and servers. The data I see supports that.
> If every combination of that requires a profile-of-profiles, now we're back to the combinatorial explosion of having to create and assign unique profiles for every 95%-identical thing with a few small changes.
> > That might be the case, but the combinatorial explosion doesn't also include parameters in the mess, which is the real issue here
Er, that's not the issue this solves. Parameters aren't duplicated today. You can't create two Parameters with the same name, config file, and value; they're required to be unique. The problem Layered Profiles solves is the Profile combinatorial explosion, it doesn't affect the Parameters that will exist at all.
And again, a Profile-of-Profiles doesn't solve that, because that same combinatorial explosion of ATS8_CDN-Y_DATACENTER-FOO_HACK-2 still exists.
We do have a lot of Parameters, and we could come up with ideas to fix that. But this doesn't, that's a completely different problem.
> Defining the order in the profile object rather than the server also reduces the chances of making a mistake when creating a new server.
And increases the chance of making a mistake when creating the Profile-of-Profiles, and when assigning it when there are so many very similar ones. The chance of operator error, like the endpoints on old APIs, are just different not better or worse.
> Could the API instead feign ignorance of things with layered Profiles in lower API versions than the one that supports it? like if you do GET /api/4.0/servers it returns a list of all servers that don't have layered profiles,
That would be extremely dangerous. If an operator has a script that needs to get all profiles, or any subset of profiles, they almost certainly actually need all profiles. And arbitrarily excluding some, even with a warning that will frequently be ignored, will result in whatever they're doing with the data not applying to many servers or delivery services. This solution seems to me like it will make operators do the wrong thing 99% of the time.
> ignore the profile value on PUT (don't make the profile change) and return alert.type=warning and alert.message='server updated but server
That is also extremely dangerous and error-prone. If someone posted something, they almost certainly expect it to be applied completely or not at all. Worse, people shouldn't ignore warnings, but they frequently do, especially in scripts.
Applying part of what a user said to apply is far, far more dangerous than returning a real error and telling the user they need to upgrade their client.
> breaking prior versions seems like a non-starter.
I don't understand your position here. As above, the two proposed solutions do break prior versions. They just pretend not to. Which is exponentially worse.
But the original design _doesn't_ break prior versions, unless a user chooses to use the new feature.
It seems to me the original design breaks prior versions far less, and far less dangerously, than the proposed alternatives.
I mean, most new features inherently require new clients to work with. The only difference here is that choosing to use the feature requires upgrading all old clients, not just the old clients that need to work with the new feature.
Which I agree is unfortunate, but I don't see an alternative that doesn't require either a great deal of hackery, or fundamentally doesn't solve the problem this is solving, or else pretends not to fail but actually fails secretly and catastrophically.
Or else just not doing this, and continuing to live with our enormous number of often-wrong Profiles, which also seems worse.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rimashah25 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rimashah25 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688045414
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+"fk_deliveryservice" FOREIGN KEY (deliveryservice) REFERENCES deliveryservice(id)
+```
+
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile | bigint | | not null |
+ order | bigint | | not null |
+ last_updated | timestamp with time zone | | not null | now()
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+### ORT Impact
+New set of profiles will be created and the profile-parameter relationship will change.
+
+### Traffic Monitor Impact
+No impact
+
+### Traffic Router Impact
+No impact
+
+### Traffic Stats Impact
+No impact
+
+### Traffic Vault Impact
+No impact
+
+### Documentation Impact
+All existing endpoints will need to be updated, along with the documentation explaining `layered_profile`.
+
+### Testing Impact
+Client/API integration tests should be updated to verify the `layered_profile` functionality on existing API `/deliveryservices`, `/servers` endpoints.
+
+### Performance Impact
+We do not anticipate any performance impact with layered_profile approach.
+
+### Security Impact
+We do not anticipate any impact on security.
+
+### Upgrade Impact
+- A Database Migration to:
+ - drop profile column in existing deliveryservice and server table
+ - insert existing server and DS profiles along with their order into the new tables(deliveryservice_profiles, server_profiles)
+- The new capability can just be added to the `seeds.sql` file.
Review comment:
umm.. seeds.sql includes changes to database.. how is there any connection to the API endpoints (which there are no new ones) and instead of the word capability (I can see the confusion), I will change it to tables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rimashah25 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rimashah25 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r767970995
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,330 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "response": [{
+ "id": 5,
+ ⋮
+ "profiles": ["MID"],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "response": [{
+ "profiles": ["MID"]
+ ⋮,
+ }
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profiles": ["MID", "AMIGA_123", "CDN_FOO"],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profiles": ["MID", "AMIGA_123", "CDN_FOO"],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 6.0",
+ "level":"error"
+ }
+]}
+```
+
+#### API 6.0 GET
+- JSON **response** with the proposed change will look as follows:
+ `/servers?id=5`
+```JSON
+{
+ "response": [{
+ "id": 5,
+ ⋮
+ "profiles": ["MID", "AMIGA_123", "CDN_FOO"],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "response": [{
+ "profiles": ["MID", "AMIGA_123", "CDN_FOO"],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 6.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profiles": ["MID", "AMIGA_123", "CDN_FOO"],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profiles": ["MID", "AMIGA_123", "CDN_FOO"],
+ ⋮
+}
+```
+
+return following **response**
+```JSON
+{ "alerts": [
+ {
+ "text": "Server created/updated",
+ "level": "success"
+ }
+],
+ "response": {
+ "cachegroup": "CDN_in_a_Box_Mid",
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "id": 5,
+ "profiles": ["MID", "AMIGA_123", "CDN_FOO"],
+ ⋮
+ }
+}
+```
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------------| ---------------------| ----------- | ---------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profiles | text | required | the profile names associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+
+**API constraints**
+- In API 5.0, GET /servers object profile and profileId fields to be arrays, but a PUT or POST with multiple values returns a 400 error.
+ The UI may or may not display a list (which can only add 1), the client implements handling multiple, and the API is documented to potentially return multiple and how their Parameters must be applied.
+- In API 6.0 (or possibly 5.0 in the following TC major version wherein API 4.0 is removed), this feature is actually implemented, and multiple profiles can be assigned, are displayed in the UI, etc.
+
+The only disadvantage to splitting it across mulitple version is a little delay to get the feature deployed.
+But the advantage is that it solves both concerns:
+- clients on supported APIs are never given 4xx errors even after the feature is in-use
+- we never return or apply partial data that could produce wrong results in production
+
+- Add `Profiles` key to API endpoints for Server objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile_name | text | | not null |
+ order | bigint | | not null |
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile_name, server, order)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
+ "fk_server" FOREIGN KEY (profile_name) REFERENCES profile(name)
+```
+
+All profiles assigned to a given server will have the same values of:
+- type
+- cdn
+- routing_disabled
+If any of those differ within the same server, it's probably a mistake.
+
+### ORT Impact
+New set of profiles will be created and the profile-parameter relationship will change.
+
+### Traffic Monitor Impact
+No impact
+
+### Traffic Router Impact
+These changes will affect the Snapshot generation (both crconfig and monitoring). Even though that is more of a TO impact.
+Reason being that Snapshots and Monitoring Configurations for a CDN include Profile and Parameter information for the servers, Traffic Monitors, and Traffic Routersz.
+
+
+### Traffic Stats Impact
+No impact
+
+### Traffic Vault Impact
+No impact
+
+### Documentation Impact
+All existing endpoints will need to be updated, along with the documentation explaining `layered_profile`.
+
+### Testing Impact
+Client/API integration tests should be updated to verify the `layered_profile` functionality on existing API `/servers` endpoints.
+
+### Performance Impact
+We do not anticipate any performance impact with layered_profile approach.
+
+### Security Impact
+We do not anticipate any impact on security.
+
+### Upgrade Impact
+- A Database Migration to:
+ - drop profile column in existing server table
+ - insert existing server and DS profiles along with their order into the new table(server_profiles)
Review comment:
Done.. good catch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] mitchell852 merged pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
mitchell852 merged pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r770930495
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,296 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+- **_Note_**: All fields in the response structure for API 4.0/ 5.0 except `profileId`, `profileDesc` and `profile`, remain same.
+ - In API 4.0/5.0 `profile` changes to `profiles` and in API 5.0 `profileId` and `profileDesc` are no longer part of response structure
Review comment:
TO API 4.0 is currently declared unstable, so we are free to make the breaking changes to TO API 4.0 without having to make further changes in TO API 5.0 (if there even ends up being one). Because of that, what this blueprint describes as API 5.0 vs API 4.0 should all just be done in 4.0. There is no need to even mention TO API 5.0 at this point.
That said, t3c may require backwards-compatibility support similar to what was done for the `/jobs` breaking changes in API 4.0. Basically, it would need to handle both the _original_ 4.0 server struct (prior to implementing this blueprint) and the _new_ 4.0 server struct. It should attempt to unmarshal into the _new_ 4.0 struct first, then fall back to unmarshalling into the _original_ 4.0 struct if it determines that TO is not yet providing the _new_ 4.0 struct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r770937516
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,296 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+- **_Note_**: All fields in the response structure for API 4.0/ 5.0 except `profileId`, `profileDesc` and `profile`, remain same.
+ - In API 4.0/5.0 `profile` changes to `profiles` and in API 5.0 `profileId` and `profileDesc` are no longer part of response structure
Review comment:
I thought the agreed-upon direction was to make the API output a list of length one for a full major version, then move to an arbitrary length list only once all API versions that use `profile`/`profileName`/`profileId` have been _fully removed_? Because the question was raised of how to handle backwards comparability for those old APIs. Since APIv4 is unstable, API version 3 is not yet deprecated, so it can't be removed until a full major ATC version passes where APIv4 is stable and APIv3 is deprecated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r770978433
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,296 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+- **_Note_**: All fields in the response structure for API 4.0/ 5.0 except `profileId`, `profileDesc` and `profile`, remain same.
+ - In API 4.0/5.0 `profile` changes to `profiles` and in API 5.0 `profileId` and `profileDesc` are no longer part of response structure
Review comment:
Once the data type is an array, any client/script that uses the array must be able to account for the fact that there can be more than 1 profile at that point. We might not _allow_ more than 1 profile when they start using it, but their code must assume layered profiles nonetheless. They can't/shouldn't rely on the fact that we'll only be allowing 1 profile until some point in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-902222968
We need to find a happy medium here. How about this:
- if a server has multiple profiles assigned:
- pre-5.0 GETs have `profileId` as the 1st profile in the list
- pre-5.0 PUTs that change the `profileId` will return a 4xx saying that the server has multiple profiles and must be changed via 5.0+
- if a server has one profile assigned:
- pre-5.0 GET and PUT behavior is the same as it is today (you are free to change the server's single profile)
That way, we're only breaking older clients that want to change profiles and only when those servers _have multiple profiles_.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-898602378
> It's a win because we wouldn't be breaking one of our most-used APIs
I'm not sure "most-used" is true. In terms of operation, basically everything non-trivial needs Profiles, Parameters, DeliveryServices, and Servers. For operational scripts that are trivial, I'm not sure there are more little scripts that just need the DS or Server, than there are little scripts that just need Profiles or Parameters.
> Additionally, I would argue that placing the ordering of the profiles on the server object itself is bad, because, in general, all servers with the same characteristics are going to share the same order of layered profiles. My moving the order into the profile object itself, you don't have to specify the order on every single server you're using it for. Instead, you only have to get the order right once, and you can't accidentally change that ordering from the servers page in the UI.
That seems like a disadvantage, not an advantage. One of the primary use cases Layered Profiles solves is the 95% identical Profiles, that just have a few small changes. Layered Profiles allows e.g. `["ATS9", "DATACENTER-FOO", "my-one-off-hack", "CDN-X"']`. If every combination of that requires a profile-of-profiles, now we're back to the combinatorial explosion of having to create and assign unique profiles for every 95%-identical thing with a few small changes. Sure, we "isolated" the Parameters on `my-one-off-hack` which is kind of nice; but you still have to create, assign, and manage the Profiles `["ATS8", "DATACENTER-FOO", "my-one-off-hack", "CDN-X"']` and `["ATS9", "DATACENTER-BAR", "my-one-off-hack", "CDN-X"']` and `["ATS8", "DATACENTER-BAR", "my-one-off-hack", "CDN-X"']` and `["ATS9", "DATACENTER-FOO", "my-one-off-hack", "CDN-Y"']` and `["ATS8", "DATACENTER-FOO", "my-one-off-hack", "CDN-Y"']` and `["ATS8", "DATACENTER-BAR", "my-one-off-hack", "CDN-Y"']` and `["ATS8", "DAT
ACENTER-BAR", "my-one-off-hack", "CDN-Y"']`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c edited a comment on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c edited a comment on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-897960910
> One other thing I just thought of: how is backwards compatibility handled for objects with layered Profiles? If I have a server with:
I put a lot of thought into this in my initial proposal. I came up with some options, but they're _really_ hacky.
Honestly, I think the best solution is probably to return a 4xx code from previous API versions for /servers and /deliveryservices if a user has assigned more than 1 Profile. And to warn users when they do, that this will break old clients. Which is _mostly_ ok in practice, it _mostly_ means people just have to upgrade all ATC components and clients before they start using Layered Profiles.
I don't suggest that lightly. IMO breaking backward compatibility is one of the worst things we can do, in terms of usability, maintaining users, I could go on. But I honestly think the alternative is probably worse here.
See https://cwiki.apache.org/confluence/display/TC/Layered+Profiles for some of my compatibility ideas.
> I assume it'd never choose a middle Profile and would always choose consistently, but should we use the "bottom" Profile
We can't return any single real profile. If someone assigned multiple Profiles, those Parameters, in that exact order, are necessary. Anything that looks at a single profile for params will be broken. If you look at that wiki link, most of my ideas are around creating a "fake" profile. But IDs and names make that hard, and you still can't edit it, so POST/PUT still return error codes. Unless you did even more hacks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c edited a comment on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c edited a comment on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-914450659
> Currently we restrict servers such that no two may share a Profile and a service IP address - how will that constraint change when servers have multiple Profiles? Will they be disallowed to share a service IP and any of their Profiles? Any of their Profiles in the same order? Only the exact same sequence of Profiles? Or should the constraint be lifted (which would be my vote immediately without thinking through the implications - that could be a bad idea, idk)?
I've said for years, the key should be `(Host, domain, port)` or `(ip, port)`; probably both.
It'll probably take some testing to make sure everything is ok with that, but that _should_ be ok, and is really the right answer anyway.
> should the constraint be lifted
I don't think it should be removed entirely, I think we really need some kind of constraint to prevent duplicate servers. Namely the real unique network identifier: the IP-port tuple and/or fqdn-port tuple.
Alternatively, if we don't want to incur the dev cost of that right now, we could probably make a database or API constraint (ideally both) for profiles-in-same-order+IP. It's probably a pretty ugly db constraint, but I think it's possible.
I don't have a strong opinion either way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] mitchell852 edited a comment on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
mitchell852 edited a comment on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-902130933
how about this? if you try to update a server that has a layered profile (1+ profiles) with a non-5.x endpoint, ignore the profile value on PUT (don't make the profile change) and return alert.type=warning and alert.message='server updated but server has a layered profile and changes to a layered profile must be made using api 5.x'
breaking prior versions seems like a non-starter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-902133297
That sounds like a reasonable approach to me, @mitchell852.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688178802
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
Review comment:
RE the conversation between Rob and me, that may not be the case, actually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688141224
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
Review comment:
+1 on a trigger to just update `last_updated` on the "main" table
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-898628493
Defining the order in the profile object rather than the server also reduces the chances of making a mistake when creating a new server. Instead of having to replicate the list exactly on the new server to match another server in the same set, you just have to assign it the same profile. Less button clicks and no chance of messing up the order seems like a win for me. Plus, we've been using example profile lists of just a few profiles each. The chance of making a mistake in the ordering increases with the size of the list.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] mitchell852 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
mitchell852 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r690711123
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
Review comment:
see my other comment. IMHO layered profiles should be limited to server profiles (which is actually needed due to the enormous amount of params). trying to do this for delivery service profiles would be a large amount of work (see TP DS and DSR view changes) for little benefit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-902826209
That works for me, but for this:
> In API 5.0, we change the /servers object profile and profileId fields to be arrays
it might be nice to just have a single array of profile names, since they have to be unique anyways, and get rid of profile IDs on the server altogether. I believe the TO API generally has both name-based and ID-based profile parameter routes to use.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-897994998
> What if we returned a null Profile when there are multiple? That's still breaking, but somewhat less so
That makes clients work who don't need the profile/params, but my fear is it's even worse if they do. Especially DSes, which are allowed to not have profiles. But even Servers might not have error checking, that they were guaranteed not to need before.
I feel like forcibly breaking and making people upgrade is much safer here. If we don't, they can operate on that data, and do any number of wrong and broken things, potentially catastrophically breaking their CDN or deleting data. And they won't even know, and it could be a nightmare to track down.
Returning an HTTP error code makes it as easy and obvious as possible to fix, and doesn't potentially irrevocably destroy data, or create an impossible-to-debug scenario.
> I'd also vote that a PUT or POST from API versions 4 and earlier do what you'd expect - set the server's Profile to the single specified Profile.
Same issue: if there were multiple, now they just wiped them out, destroying data. I feel like the valid use cases of legitimately wanting to replace multiple profiles with a single are extremely unlikely.
Again, if there are 0 or 1 assigned, letting API <4.0 set it to a different single value is totally fine. But if it has 2+ Profiles, it seems highly unlikely to be correct, and potentially catastrophic. I think the only safe thing to do is return an error code and disallow the entire request or modification.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 edited a comment on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 edited a comment on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-902206714
Okay, rather than "our two most used" they are merely "two of our most used". You're right, I had no data to back that up, but it looks like the reality still shows how important these endpoints are. That's 35,000 failed requests if we break those endpoints.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-902215631
> That would be extremely dangerous. If an operator has a script that needs to get all profiles, or any subset of profiles, they almost certainly actually need all profiles... This solution seems to me like it will make operators do the wrong thing 99% of the time.
Well their alternative is to get none at all, doing nothing 100% of the time, and a script that no longer works, though. By "profiles" I assume you mean "servers", right? This shouldn't require any actual changes to Profiles, as I understand the blueprint.
Not that those are our only two choices, but I see them as at least equally unacceptable.
> Applying part of what a user said to apply is far, far more dangerous than returning a real error and telling the user they need to upgrade their client.
I agree with this; it seems like if you just ignore the Profile then the changes requested were not actually successfully made, and saying otherwise is just lying to the client. Although we have
> _"When an object is replaced, the response body MUST contain a representation of the object after replacement."_
in our API guidelines, in practice I don't think most endpoints actually respect that to the letter, and I'm sure most clients don't even look at the response.
> But the original design _doesn't_ break prior versions, unless a user chooses to use the new feature.
That is true, it's not like upgrading to a TO version that supports layered Profiles is going to immediately cause breakages. I think my main concern is just that the bar for breaking tens of thousands of requests after that upgrade is extremely low. Is there any way we can make it harder to make a change that's going to shatter the API? Like, idk, forcing an `admin` user to make the first Profile layering, before anyone else can? I'm not tied to that particular idea, but just some kind of gatekeeping so that everyone associated is ready for the change and everyone is **_very_** sure of that readiness.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rob05c edited a comment on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rob05c edited a comment on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-898561848
> we could just update the profiles API to produce a "layered" profile. For example:
> "layeredProfiles": [
> "abc",
> "def"
> ]
I'm not seeing how that solves the backward-compatibiliity issue? That would still break existing clients
> So it only breaks backwards compatibility in terms of assigning parameters
So instead of breaking getting things with profiles, we break changing the profiles or assigning things to them? I'm not seeing how that's a win, they both seem about the same.
> A server would still only have one profile.
Except that we redefined what a "profile" is, and it can now be a "profile" or a "profile of profiles."
Maybe it's just me, but that seems a lot harder to understand and work with. Breaking change aside, the idea that a server/ds has a list of Profiles applied in order seems pretty straightforward. Where the idea that a server/ds has a profile, that can be a single or a "profile of profiles" seems to add another layer of abstraction and indirection, seems like it fundamentally adds more to think about. Where "a list" isn't fundamentally another layer of abstraction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-897990524
What if we returned a `null` Profile when there are multiple? That's still breaking, but somewhat less so. It means that eg. my script that does `toget servers | jq '.response[]|.id` still gets me all the server IDs instead of failing. In that case, I'd also vote that a PUT or POST from API versions 4 and earlier do what you'd expect - set the server's Profile to the single specified Profile.
That's harder for Delivery Services, where `profile: "null"` is already totally legal, which makes it much more of a lie.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 edited a comment on pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 edited a comment on pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#issuecomment-897990524
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r760481942
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
Review comment:
nit but I don't think this was supposed to be a part of the list
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
Review comment:
this doesn't look right to me. Servers don't currently have a `requested` property, and I don't think the name of the property is supposed to be `layeredProfiles` - shouldn't it just be `profiles`? There are no "un-layered" Profiles on servers from which to distinguish them.
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 5.0+",
+ "level":"error"
+ }
+]}
+```
+
+#### API 6.0 GET
Review comment:
I think this section should be 5.0? And all of my comments above apply in turn to each endpoint here.
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
Review comment:
Hadn't we settled on using Profile Names instead of IDs? The way they are presented and identified should be the same in responses and requests
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
Review comment:
same as above RE: `requested` and `layeredProfiles`
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
Review comment:
Same as above RE: names vs IDs and requests and responses matching
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
Review comment:
Also:
- `/deliveryservices/{{ID}}/servers`
- `/deliveryservices/{{ID}}/servers/eligible`
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 5.0+",
+ "level":"error"
+ }
+]}
+```
+
+#### API 6.0 GET
+- JSON **response** with the proposed change will look as follows:
+ `/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 6.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+return following **response**
+```JSON
+{ "alerts": [
+ {
+ "text": "Server created/updated",
+ "level": "success"
+ }
+],
+ "response": {
+ "cachegroup": "CDN_in_a_Box_Mid",
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "id": 5,
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ "profileDesc": "Mid Cache - Apache Traffic Server, Amiga 123 Machine, CDN Foo",
+ "profileIds": [10, 11, 15],
+ ⋮
+ }
+}
+```
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------------| ---------------------| ----------- | ---------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| layeredProfiles | text | required | the profile names associated with a server |
Review comment:
This is really just the name of a single Profile, right? That's why it has `order`, and why we need a table at all, I thought.
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 5.0+",
+ "level":"error"
+ }
+]}
+```
+
+#### API 6.0 GET
+- JSON **response** with the proposed change will look as follows:
+ `/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 6.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+return following **response**
+```JSON
+{ "alerts": [
+ {
+ "text": "Server created/updated",
+ "level": "success"
+ }
+],
+ "response": {
+ "cachegroup": "CDN_in_a_Box_Mid",
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "id": 5,
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ "profileDesc": "Mid Cache - Apache Traffic Server, Amiga 123 Machine, CDN Foo",
+ "profileIds": [10, 11, 15],
+ ⋮
+ }
+}
+```
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------------| ---------------------| ----------- | ---------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| layeredProfiles | text | required | the profile names associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+
+**API constraints**
+- In API 5.0, GET /servers object profile and profileId fields to be arrays, but a PUT or POST with multiple values returns a 400 error.
+ The UI may or may not display a list (which can only add 1), the client implements handling multiple, and the API is documented to potentially return multiple and how their Parameters must be applied.
+- In API 6.0 (or possibly 5.0 in the following TC major version wherein API 4.0 is removed), this feature is actually implemented, and multiple profiles can be assigned, are displayed in the UI, etc.
+
+The only disadvantage to splitting it across mulitple version is a little delay to get the feature deployed.
+But the advantage is that it solves both concerns:
+- clients on supported APIs are never given 4xx errors even after the feature is in-use
+- we never return or apply partial data that could produce wrong results in production
+
+- Add `Profiles` key to API endpoints for Server objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile_names | text | | not null |
+ order | bigint | | not null |
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile_names, server, order)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
+ "fk_server" FOREIGN KEY (profile_names) REFERENCES profile(name)
+```
+
+All profiles assigned to a given server will have the same values of:
+- type
+- cdn
+- routing_disabled
Review comment:
This one sounds hard to enforce. Imagine I have a server that has a bunch of Profiles that are all `routingDisabled` set to `true`. What if I change one to `false`? Does Profile update validation prevent that? If it doesn't, my cache server cannot be updated unless I revert it. If it does, then I need to remove all Profiles from the server, update them all, and reassign in order to make the change. Both are cumbersome. I would say don't bother.
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 5.0+",
+ "level":"error"
+ }
+]}
+```
+
+#### API 6.0 GET
+- JSON **response** with the proposed change will look as follows:
+ `/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 6.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+return following **response**
+```JSON
+{ "alerts": [
+ {
+ "text": "Server created/updated",
+ "level": "success"
+ }
+],
+ "response": {
+ "cachegroup": "CDN_in_a_Box_Mid",
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "id": 5,
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ "profileDesc": "Mid Cache - Apache Traffic Server, Amiga 123 Machine, CDN Foo",
+ "profileIds": [10, 11, 15],
Review comment:
I don't think we wanted to identify Profiles twice, right? Also, for my money, we can just drop `profileDesc`. Figuring out how to generate it is too difficult, and I don't know what use it could possibly have.
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 5.0+",
+ "level":"error"
+ }
+]}
+```
Review comment:
This part confuses me - it's a response to a 5.0 request, but it's saying that the current API version is insufficient and the request should use API v5.0+?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r750436372
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
Review comment:
> Do we currently only have the ability of a single profile to be assigned to a server?
Yes
> Does this new approach provide the ability to assign multiple profiles to a server?
Yes
> How does the concept of layering then get applied?
The proposal is that the Profiles of a server are in precedence order, so that the Parameters of later entries overrule earlier ones.
> That suggests there is some sort of hierarchy or order that must be enforced, since they layers are then added on top of each other.
Yes, exactly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r688058058
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a delivery service/server, as well as the profile each parameter came from. A new page, linked from DS and Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for both Delivery Services and Servers, on the existing DS and Server pages.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for delivery services and server.
+- `/deliveryservices/ GET, POST`
+- `/deliveryservices/{id} PUT, DELETE`
+- `/server/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing delivery services and servers endpoints.
+- JSON **response** with the proposed change will look as follows:
+
+`/deliveryservices?id=100`
+```JSON
+{
+ "id": 100,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ]
+ ⋮
+ }
+}
+```
+
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ }
+}
+```
+
+JSON **request** with the proposed change will look as follows:
+
+`/deliveryservices`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1234,
+ 5678,
+ 9012
+ ],
+ "profileNames": [
+ "EDGE",
+ "ATS8",
+ "TOP"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "profileIds": [
+ 1357,
+ 2468,
+ 2356
+ ],
+ "profileNames": [
+ "MID",
+ "TOP",
+ "PRIMARY"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+The following table describes the top level `layered_profile` object for delivery services:
+
+| field | type | optionality | description |
+| ------------------ | --------------| ----------- | ---------------------------------------------------------------|
+| deliveryservice | bigint | required | the delivery service id associated with a given profile |
+| profile | bigint | required | the profile id associated with a delivery service |
+| order | bigint | required | the order in which a profile is applied to a delivery service |
+| lastUpdated | bigint | required | the last time this delivery service was updated |
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------- | ---------------------| ----------- | ---------------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| profile | bigint | required | the profile id associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+| lastUpdated | bigint | required | the last time this server was updated |
+
+**API constraints**
+- TO REST APIs will not be backward compatibility and this feature will be a major version (v5.0) change.
+- Add `Profiles` key to API endpoints for Server and DeliveryService objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/deliveryservices` and `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Delivery Services or Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+- A new Database Table `deliveryservice_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.deliveryservice_profiles"
+ Column | Type | Collation | Nullable | Default
+----------------+--------------------------+-----------+----------+--------
+deliveryservice | bigint | | not null |
+profile | bigint | | not null |
+order | bigint | | not null |
+last_updated | timestamp with time zone | | not null | now()
+Indexes:
+"pk_deliveryservice_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+"fk_deliveryservice" FOREIGN KEY (deliveryservice) REFERENCES deliveryservice(id)
+```
+
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile | bigint | | not null |
+ order | bigint | | not null |
+ last_updated | timestamp with time zone | | not null | now()
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+### ORT Impact
+New set of profiles will be created and the profile-parameter relationship will change.
+
+### Traffic Monitor Impact
+No impact
+
+### Traffic Router Impact
+No impact
+
+### Traffic Stats Impact
+No impact
+
+### Traffic Vault Impact
+No impact
+
+### Documentation Impact
+All existing endpoints will need to be updated, along with the documentation explaining `layered_profile`.
+
+### Testing Impact
+Client/API integration tests should be updated to verify the `layered_profile` functionality on existing API `/deliveryservices`, `/servers` endpoints.
+
+### Performance Impact
+We do not anticipate any performance impact with layered_profile approach.
+
+### Security Impact
+We do not anticipate any impact on security.
+
+### Upgrade Impact
+- A Database Migration to:
+ - drop profile column in existing deliveryservice and server table
+ - insert existing server and DS profiles along with their order into the new tables(deliveryservice_profiles, server_profiles)
+- The new capability can just be added to the `seeds.sql` file.
Review comment:
I think the new tables need to go into migrations, not `seeds.sql`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rimashah25 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rimashah25 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r768803679
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 5.0+",
+ "level":"error"
+ }
+]}
+```
+
+#### API 6.0 GET
+- JSON **response** with the proposed change will look as follows:
+ `/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 6.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+return following **response**
+```JSON
+{ "alerts": [
+ {
+ "text": "Server created/updated",
+ "level": "success"
+ }
+],
+ "response": {
+ "cachegroup": "CDN_in_a_Box_Mid",
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "id": 5,
+ "layeredProfiles": [
+ "MID",
+ "AMIGA_123",
+ "CDN_FOO"
+ ],
+ "profileDesc": "Mid Cache - Apache Traffic Server, Amiga 123 Machine, CDN Foo",
+ "profileIds": [10, 11, 15],
+ ⋮
+ }
+}
+```
+
+The following table describes the top level `layered_profile` object for servers:
+
+| field | type | optionality | description |
+| ----------------| ---------------------| ----------- | ---------------------------------------------------------|
+| server | bigint | required | the server id associated with a given profile |
+| layeredProfiles | text | required | the profile names associated with a server |
+| order | bigint | required | the order in which a profile is applied to a server |
+
+**API constraints**
+- In API 5.0, GET /servers object profile and profileId fields to be arrays, but a PUT or POST with multiple values returns a 400 error.
+ The UI may or may not display a list (which can only add 1), the client implements handling multiple, and the API is documented to potentially return multiple and how their Parameters must be applied.
+- In API 6.0 (or possibly 5.0 in the following TC major version wherein API 4.0 is removed), this feature is actually implemented, and multiple profiles can be assigned, are displayed in the UI, etc.
+
+The only disadvantage to splitting it across mulitple version is a little delay to get the feature deployed.
+But the advantage is that it solves both concerns:
+- clients on supported APIs are never given 4xx errors even after the feature is in-use
+- we never return or apply partial data that could produce wrong results in production
+
+- Add `Profiles` key to API endpoints for Server objects (and ProfileNames, ProfileIDs)
+
+#### Client Impact
+- Existing Go client methods will be updated for the `/servers` endpoints in order to write TO API tests for the exising endpoints.
+- All functions which get Parameters on Servers must be changed to get the Parameters of all assigned Profiles in order.
+
+#### Data Model / Database Impact
+
+- A new Database Table `server_profiles` as described below, will be created:
+```text
+ Table "traffic_ops.server_profiles"
+ Column | Type | Collation | Nullable | Default
+---------------+--------------------------+-----------+----------+--------
+ server | bigint | | not null |
+ profile_names | text | | not null |
+ order | bigint | | not null |
+Indexes:
+ "pk_server_profile" PRIMARY KEY(profile_names, server, order)
+Foreign-key constraints:
+ "fk_server" FOREIGN KEY (server) REFERENCES server(id)
+ "fk_server" FOREIGN KEY (profile_names) REFERENCES profile(name)
+```
+
+All profiles assigned to a given server will have the same values of:
+- type
+- cdn
+- routing_disabled
Review comment:
Brennan explained above comment over chat.
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+`PUT /servers/5`
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
+ ⋮
+}
+```
+
+returns a **400** **response** with
+
+```JSON
+{
+ "alerts":[
+ {
+ "text":"cannot associate multiple profiles to a server with current API version but can be done via 5.0+",
+ "level":"error"
+ }
+]}
+```
+
+#### API 6.0 GET
Review comment:
Brennan explained above comment over chat.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rimashah25 commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rimashah25 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r767907135
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,347 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Traffic Ops will need to add the logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+
+#### API 5.0 GET
+- JSON **response** with the proposed change will look as follows:
+`/servers?id=5`
+```JSON
+{
+ "id": 5,
+ ⋮
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ }
+}
+```
+
+`/servers`
+```JSON
+{
+ "requested": {
+ "layeredProfiles": [
+ "MID"
+ ],
+ ⋮
+ },
+ ⋮
+}
+```
+
+#### API 5.0 POST/PUT
+JSON **request** with the proposed change will look as follows:
+
+`POST /servers `
+```JSON
+{
+ "cachegroupId": 6,
+ "cdnId": 2,
+ ⋮
+ "interfaces": [
+ ⋮
+ ],
+ ⋮
+ "profileId": [10, 11, 15],
Review comment:
Yes, I missed correcting that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #6095: Server Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
rawlinp commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r770957612
##########
File path: blueprints/layered-profile-server.md
##########
@@ -0,0 +1,296 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied.
+
+Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage.
+
+With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order:
+1. EDGE
+2. AMIGA_123
+3. CDN_FOO
+4. DATACENTER_ABC
+5. ATS_714
+6. Custom_Hack_42
+
+### Traffic Portal Impact
+
+1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg:
+
+| Name |Config File | Value | Profile |
+|--------------------------------------------------|--------------------------|--------------------------------------|-----------|
+| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE |
+| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 |
+| error_url | url_sig_myotherds.config | 403 | EDGE |
+| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 |
+
+2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages.
+3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles.
+
+### Traffic Ops Impact
+
+- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server.
+
+- `/servers/ GET, POST`
+- `/servers/{id} PUT, DELETE`
+- `/servers/details`
+- `/deliveryservices/{{ID}}/servers`
+- `/deliveryservices/{{ID}}/servers/eligible`
+
+#### REST API Impact
+
+- No new endpoints are required
+
+**Existing Endpoints**
+
+- Modify JSON request and response for existing servers endpoints.
+- **_Note_**: All fields in the response structure for API 4.0/ 5.0 except `profileId`, `profileDesc` and `profile`, remain same.
+ - In API 4.0/5.0 `profile` changes to `profiles` and in API 5.0 `profileId` and `profileDesc` are no longer part of response structure
Review comment:
Yes, we should permit a server to only have a single profile until API v3 has been deprecated and removed. In theory, if we stabilized 4.0 (with these changes) and deprecated 3.x in ATC 6.1, we could remove 3.x and allow servers to have more than 1 profile in the next release (e.g. 7.0). That wouldn't require a new 5.0 API version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] mitchell852 commented on a change in pull request #6095: Layered Profile Blueprint
Posted by GitBox <gi...@apache.org>.
mitchell852 commented on a change in pull request #6095:
URL: https://github.com/apache/trafficcontrol/pull/6095#discussion_r690713236
##########
File path: blueprints/layered-profile.md
##########
@@ -0,0 +1,274 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Layered Profile
+
+## Problem Description
+
+Profiles are unwieldy and dangerous for Operations.
+
+Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are
+1. Edges
+2. Amiga 123 machines
+3. In the Foo CDN
+4. In the ABC datacenter
+5. Running ATS 7.14 RC0
+6. Who knows what 42 means?
+
+Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories:
+
+EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29
+
+- The number of Profiles quickly becomes a combinatorial explosion
+- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter.
+- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one?
+
+## Proposed Change
+
+Layered Profiles allow assigning multiple profiles, in order, to both Delivery Services and Servers. If multiple profiles have a parameter with the same name and config File, the parameter from the last profile in the ordering is applied.
Review comment:
I would strongly, strongly suggest limiting this effort to server profiles....
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org