You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficcontrol.apache.org by GitBox <gi...@apache.org> on 2022/05/31 20:56:54 UTC
[GitHub] [trafficcontrol] srijeet0406 opened a new pull request, #6872: Traffic Ops Service Oriented Architecture blueprint
srijeet0406 opened a new pull request, #6872:
URL: https://github.com/apache/trafficcontrol/pull/6872
<!--
Thank you for contributing! Please be sure to read our contribution guidelines: https://github.com/apache/trafficcontrol/blob/master/CONTRIBUTING.md
If this closes or relates to an existing issue, please reference it using one of the following:
Closes: #ISSUE
Related: #ISSUE
If this PR fixes a security vulnerability, DO NOT submit! Instead, contact
the Apache Traffic Control Security Team at security@trafficcontrol.apache.org and follow the
guidelines at https://apache.org/security regarding vulnerability disclosure.
-->
This PR is not related to any issue.
<!-- **^ Add meaningful description above** --><hr/>
## Which Traffic Control components are affected by this PR?
<!-- Please delete all components from this list that are NOT affected by this PR.
Feel free to add the name of a tool or script that is affected but not on the list.
-->
- Documentation
## What is the best way to verify this PR?
<!-- Please include here ALL the steps necessary to test your PR.
If your PR has tests (and most should), provide the steps needed to run the tests.
If not, please provide step-by-step instructions to test the PR manually and explain why your PR does not need tests. -->
n/a
## If this is a bugfix, which Traffic Control versions contained the bug?
<!-- Delete this section if the PR is not a bugfix, or if the bug is only in the master branch.
Examples:
- 5.1.2
- 5.1.3 (RC1)
-->
- master
## PR submission checklist
- [ ] This PR has tests <!-- If not, please delete this text and explain why this PR does not need tests. -->
- [x] This PR has documentation <!-- If not, please delete this text and explain why this PR does not need documentation. -->
- [ ] This PR has a CHANGELOG.md entry <!-- A fix for a bug from an ATC release, an improvement, or a new feature should have a changelog entry. -->
- [x] This PR **DOES NOT FIX A SERIOUS SECURITY VULNERABILITY** (see [the Apache Software Foundation's security guidelines](https://apache.org/security) for details)
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a diff in pull request #6872: Traffic Ops Service Oriented Architecture blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on code in PR #6872:
URL: https://github.com/apache/trafficcontrol/pull/6872#discussion_r887094719
##########
blueprints/to_soa.md:
##########
@@ -0,0 +1,159 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Service Oriented Architecture for Traffic Ops
+
+## Problem Description
+Traffic Ops (TO), in its current form, can be best described as a monolithic application.
+All the code and logic for TO resides in one single application. Hence, making even a minor
+change to one area of TO requires a redeploy of the entire TO application. This is not ideal,
+as it adds the additional burden of deploying a huge application, for a small change. In order to
+move faster and in a more seamless manner, TO needs to be refactored so that we can redeploy/ restart
+one part of the application without affecting the other parts.
+
+Rewriting the entire TO application will be a highly time and resource intensive effort. However, we could
+refactor TO, to convert it into a Service Oriented architecture (SOA) style product. This would allow us to
+decouple new features and functionality from the monolith while also providing the opportunity to re-factor out
+existing features in the future.
+
+## Proposed Change
+The goal is to transition TO from a monolithic application to a more service-oriented architecture.
+What this means is that TO will be more of a "collection of services". New features and functionality will be
+added as stand-alone services with stand-alone testing, deployment and scalability. TO, the way it is today, will serve
+as `TO-core`, and any new service will be added as a "backend service", fulfilling the requirements of that service, and nothing else.
+The user would still send a request to the TO API. TO, upon receiving this request, would check in its configured "backend routes". If
+a match is encountered, TO would act as a reverse proxy for this service. This means that TO would simply forward the request to the backend
+service, get the response from the backend service and send it back to the user. From the user's perspective, this should be totally transparent
+and seamless. Once TO starts supporting the SOA pattern, we would be able to scale the individual services independently of TO core.
+
+
+### Traffic Portal Impact
+Any new Traffic Portal (TP) changes that the backend service requires, will be required to be added as modules to the current TP architecture.
+As for the TO refactor, no TP changes are required.
+
+### Traffic Ops Impact
+TO will need to read a new configuration file(supplied with the `backendcfg` option) on startup. This file will list the routes that will be served by the backend services, some route
Review Comment:
Then is the `--backendcfg` long-option optional? Seems like it should be. Is there a short flag synonym for it e.g. `-b`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] srijeet0406 commented on a diff in pull request #6872: Traffic Ops Service Oriented Architecture blueprint
Posted by GitBox <gi...@apache.org>.
srijeet0406 commented on code in PR #6872:
URL: https://github.com/apache/trafficcontrol/pull/6872#discussion_r886972611
##########
blueprints/to_soa.md:
##########
@@ -0,0 +1,159 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Service Oriented Architecture for Traffic Ops
+
+## Problem Description
+Traffic Ops (TO), in its current form, can be best described as a monolithic application.
+All the code and logic for TO resides in one single application. Hence, making even a minor
+change to one area of TO requires a redeploy of the entire TO application. This is not ideal,
+as it adds the additional burden of deploying a huge application, for a small change. In order to
+move faster and in a more seamless manner, TO needs to be refactored so that we can redeploy/ restart
+one part of the application without affecting the other parts.
+
+Rewriting the entire TO application will be a highly time and resource intensive effort. However, we could
+refactor TO, to convert it into a Service Oriented architecture (SOA) style product. This would allow us to
+decouple new features and functionality from the monolith while also providing the opportunity to re-factor out
+existing features in the future.
+
+## Proposed Change
+The goal is to transition TO from a monolithic application to a more service-oriented architecture.
+What this means is that TO will be more of a "collection of services". New features and functionality will be
+added as stand-alone services with stand-alone testing, deployment and scalability. TO, the way it is today, will serve
+as `TO-core`, and any new service will be added as a "backend service", fulfilling the requirements of that service, and nothing else.
+The user would still send a request to the TO API. TO, upon receiving this request, would check in its configured "backend routes". If
+a match is encountered, TO would act as a reverse proxy for this service. This means that TO would simply forward the request to the backend
+service, get the response from the backend service and send it back to the user. From the user's perspective, this should be totally transparent
+and seamless. Once TO starts supporting the SOA pattern, we would be able to scale the individual services independently of TO core.
+
+
+### Traffic Portal Impact
+Any new Traffic Portal (TP) changes that the backend service requires, will be required to be added as modules to the current TP architecture.
+As for the TO refactor, no TP changes are required.
+
+### Traffic Ops Impact
+TO will need to read a new configuration file(supplied with the `backendcfg` option) on startup. This file will list the routes that will be served by the backend services, some route
Review Comment:
It doesn't need to be, but I didn't want to overload the current TO config any more. It's already pretty big and has lots of stuff in it as it is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] srijeet0406 commented on a diff in pull request #6872: Traffic Ops Service Oriented Architecture blueprint
Posted by GitBox <gi...@apache.org>.
srijeet0406 commented on code in PR #6872:
URL: https://github.com/apache/trafficcontrol/pull/6872#discussion_r886974936
##########
blueprints/to_soa.md:
##########
@@ -0,0 +1,159 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Service Oriented Architecture for Traffic Ops
+
+## Problem Description
+Traffic Ops (TO), in its current form, can be best described as a monolithic application.
+All the code and logic for TO resides in one single application. Hence, making even a minor
+change to one area of TO requires a redeploy of the entire TO application. This is not ideal,
+as it adds the additional burden of deploying a huge application, for a small change. In order to
+move faster and in a more seamless manner, TO needs to be refactored so that we can redeploy/ restart
+one part of the application without affecting the other parts.
+
+Rewriting the entire TO application will be a highly time and resource intensive effort. However, we could
+refactor TO, to convert it into a Service Oriented architecture (SOA) style product. This would allow us to
+decouple new features and functionality from the monolith while also providing the opportunity to re-factor out
+existing features in the future.
+
+## Proposed Change
+The goal is to transition TO from a monolithic application to a more service-oriented architecture.
+What this means is that TO will be more of a "collection of services". New features and functionality will be
+added as stand-alone services with stand-alone testing, deployment and scalability. TO, the way it is today, will serve
+as `TO-core`, and any new service will be added as a "backend service", fulfilling the requirements of that service, and nothing else.
+The user would still send a request to the TO API. TO, upon receiving this request, would check in its configured "backend routes". If
+a match is encountered, TO would act as a reverse proxy for this service. This means that TO would simply forward the request to the backend
+service, get the response from the backend service and send it back to the user. From the user's perspective, this should be totally transparent
+and seamless. Once TO starts supporting the SOA pattern, we would be able to scale the individual services independently of TO core.
+
+
+### Traffic Portal Impact
+Any new Traffic Portal (TP) changes that the backend service requires, will be required to be added as modules to the current TP architecture.
+As for the TO refactor, no TP changes are required.
+
+### Traffic Ops Impact
+TO will need to read a new configuration file(supplied with the `backendcfg` option) on startup. This file will list the routes that will be served by the backend services, some route
+specific metadata (permissions, route ID, etc.) and where the incoming requests on those paths should be reverse proxied to. Any change to
+this configuration file will require a reload of the new configuration values on TO's side. This can be achieved by sending a `SIGHUP` signal
+to the TO process.
+
+The contents of an example backend configuration file are listed below:
+
+```json
+{
+ // List of routes for which TO needs to act as a reverse proxy
+ "routes": [
+ {
+ // The path to match on
+ "path": "^/api/4.0/foo/?$",
+ // The HTTP method for this path that TO should support
+ "method": "GET",
+ // List of hosts where the backend service is running, and where TO needs to forward the incoming request to
+ "hosts": [
+ {
+ // Protocol to follow while forwarding the requests
+ "protocol": "https",
+ // Host name where the backend service is running
+ "hostname": "to.backend.net",
+ // Port where the backend service is running on the above mentioned host
+ "port": 8444
+ },
+ {
+ "protocol": "https",
+ "hostname": "to.backend.net",
+ "port": 8445
+ }
+ ],
+ // A boolean specifying whether or not TO should verify the backend server's certificate chain and host name. This is not recommended for production use.
+ "insecure": false,
+ // Permissions required to access the route
+ "permissions": [
+ "FOO:READ"
+ ],
+ // ID for the route
+ "routeId": 123456,
+ // Extra options to configure how TO forwards these requests
+ "opts": {
+ // Currently, TO just supports a round robin algorithm to forward the requests.
+ "alg": "roundrobin"
+ }
+ }
+ ]
+}
+```
+
+#### REST API Impact
+n/a
+
+#### Client Impact
+Clients will still send requests and receive responses (for the newly configured endpoints) from TO, so nothing will change from the clients' perspective.
+
+#### Data Model / Database Impact
+There is no impact on the existing TO database structure/ schema. The backend services are free to have their own databases and
+serve data from there. As before, only TO will be allowed to access the Traffic Ops Database (TODB).
+
+### t3c Impact
+n/a
+
+### Traffic Monitor Impact
+n/a
+
+### Traffic Router Impact
+n/a
+
+### Traffic Stats Impact
+n/a
+
+### Traffic Vault Impact
+n/a
+
+### Documentation Impact
+Documentation will have to be added for the new configuration option for TO.
+
+### Testing Impact
+The backend services will be responsible for writing their own test suites.
+
+### Performance Impact
+We don't foresee any performance impact on TO as a result of this architecture change.
+
+### Security Impact
+SSL certificates will be required for HTTPS protocol to be used between TO and the services. These can be implemented however is preferred by the
+backend service. The overhead will include certificate generation and signing, certificate deployment to services' servers, certificate management
+and renewal. Client certificates are an option for the future but will require further development.
+
+### Upgrade Impact
+If a change has to go in to one of the backend services, a redeploy/ restart of TO will no longer be necessary. The backend service will, however,
+need to be re deployed. After that, if there is no change to the backend configuration file, no action will be required on the TO side. Else, as explained
+earlier, a SIGHUP signal will need to be sent to TO to reload the new configuration values.
+
+### Operations Impact
+The added backend services will add some overhead to operations. However, this architecture will also reduce the load on operations from the TO side, since
+every code change now won't force redeploying TO as a whole.
+
+### Developer Impact
+If a new endpoint (or refactor of an existing one) is deemed as a fit candidate for a backend service, the developer will have to undertake the tasks of writing
+this new service (which can be a simple server serving these endpoints), and the tests associated with it.
Review Comment:
So the plan is that anything that has the ability to be in the TO core, i.e., the functionality of the new API route can affect more than one backend service, goes in the TO core. If the feature has is an "add-on", it gets added as a backend service. This is mainly for organizations' internal business logic, but there is nothing stopping the OSS community from using thie feature as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] srijeet0406 commented on a diff in pull request #6872: Traffic Ops Service Oriented Architecture blueprint
Posted by GitBox <gi...@apache.org>.
srijeet0406 commented on code in PR #6872:
URL: https://github.com/apache/trafficcontrol/pull/6872#discussion_r887235486
##########
blueprints/to_soa.md:
##########
@@ -0,0 +1,159 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Service Oriented Architecture for Traffic Ops
+
+## Problem Description
+Traffic Ops (TO), in its current form, can be best described as a monolithic application.
+All the code and logic for TO resides in one single application. Hence, making even a minor
+change to one area of TO requires a redeploy of the entire TO application. This is not ideal,
+as it adds the additional burden of deploying a huge application, for a small change. In order to
+move faster and in a more seamless manner, TO needs to be refactored so that we can redeploy/ restart
+one part of the application without affecting the other parts.
+
+Rewriting the entire TO application will be a highly time and resource intensive effort. However, we could
+refactor TO, to convert it into a Service Oriented architecture (SOA) style product. This would allow us to
+decouple new features and functionality from the monolith while also providing the opportunity to re-factor out
+existing features in the future.
+
+## Proposed Change
+The goal is to transition TO from a monolithic application to a more service-oriented architecture.
+What this means is that TO will be more of a "collection of services". New features and functionality will be
+added as stand-alone services with stand-alone testing, deployment and scalability. TO, the way it is today, will serve
+as `TO-core`, and any new service will be added as a "backend service", fulfilling the requirements of that service, and nothing else.
+The user would still send a request to the TO API. TO, upon receiving this request, would check in its configured "backend routes". If
+a match is encountered, TO would act as a reverse proxy for this service. This means that TO would simply forward the request to the backend
+service, get the response from the backend service and send it back to the user. From the user's perspective, this should be totally transparent
+and seamless. Once TO starts supporting the SOA pattern, we would be able to scale the individual services independently of TO core.
+
+
+### Traffic Portal Impact
+Any new Traffic Portal (TP) changes that the backend service requires, will be required to be added as modules to the current TP architecture.
+As for the TO refactor, no TP changes are required.
+
+### Traffic Ops Impact
+TO will need to read a new configuration file(supplied with the `backendcfg` option) on startup. This file will list the routes that will be served by the backend services, some route
+specific metadata (permissions, route ID, etc.) and where the incoming requests on those paths should be reverse proxied to. Any change to
+this configuration file will require a reload of the new configuration values on TO's side. This can be achieved by sending a `SIGHUP` signal
+to the TO process.
+
+The contents of an example backend configuration file are listed below:
+
+```json
+{
+ // List of routes for which TO needs to act as a reverse proxy
+ "routes": [
+ {
+ // The path to match on
+ "path": "^/api/4.0/foo/?$",
+ // The HTTP method for this path that TO should support
+ "method": "GET",
+ // List of hosts where the backend service is running, and where TO needs to forward the incoming request to
+ "hosts": [
+ {
+ // Protocol to follow while forwarding the requests
+ "protocol": "https",
+ // Host name where the backend service is running
+ "hostname": "to.backend.net",
+ // Port where the backend service is running on the above mentioned host
+ "port": 8444
+ },
+ {
+ "protocol": "https",
+ "hostname": "to.backend.net",
+ "port": 8445
+ }
+ ],
+ // A boolean specifying whether or not TO should verify the backend server's certificate chain and host name. This is not recommended for production use.
+ "insecure": false,
+ // Permissions required to access the route
+ "permissions": [
+ "FOO:READ"
+ ],
+ // ID for the route
+ "routeId": 123456,
+ // Extra options to configure how TO forwards these requests
+ "opts": {
+ // Currently, TO just supports a round robin algorithm to forward the requests.
+ "alg": "roundrobin"
+ }
+ }
+ ]
+}
+```
+
+#### REST API Impact
+n/a
+
+#### Client Impact
+Clients will still send requests and receive responses (for the newly configured endpoints) from TO, so nothing will change from the clients' perspective.
+
+#### Data Model / Database Impact
+There is no impact on the existing TO database structure/ schema. The backend services are free to have their own databases and
+serve data from there. As before, only TO will be allowed to access the Traffic Ops Database (TODB).
+
+### t3c Impact
+n/a
+
+### Traffic Monitor Impact
+n/a
+
+### Traffic Router Impact
+n/a
+
+### Traffic Stats Impact
+n/a
+
+### Traffic Vault Impact
+n/a
+
+### Documentation Impact
+Documentation will have to be added for the new configuration option for TO.
+
+### Testing Impact
+The backend services will be responsible for writing their own test suites.
+
+### Performance Impact
+We don't foresee any performance impact on TO as a result of this architecture change.
+
+### Security Impact
+SSL certificates will be required for HTTPS protocol to be used between TO and the services. These can be implemented however is preferred by the
+backend service. The overhead will include certificate generation and signing, certificate deployment to services' servers, certificate management
+and renewal. Client certificates are an option for the future but will require further development.
+
+### Upgrade Impact
+If a change has to go in to one of the backend services, a redeploy/ restart of TO will no longer be necessary. The backend service will, however,
+need to be re deployed. After that, if there is no change to the backend configuration file, no action will be required on the TO side. Else, as explained
+earlier, a SIGHUP signal will need to be sent to TO to reload the new configuration values.
+
+### Operations Impact
+The added backend services will add some overhead to operations. However, this architecture will also reduce the load on operations from the TO side, since
+every code change now won't force redeploying TO as a whole.
+
+### Developer Impact
+If a new endpoint (or refactor of an existing one) is deemed as a fit candidate for a backend service, the developer will have to undertake the tasks of writing
+this new service (which can be a simple server serving these endpoints), and the tests associated with it.
Review Comment:
#6810 will definitely use this pattern.
The existing plugin system stays as is, but we could, as a future effort, move those plugins to be backend services. That way, if a change goes into one of the plugins, the entire TO application doesn't need to be upgraded and redeployed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a diff in pull request #6872: Traffic Ops Service Oriented Architecture blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on code in PR #6872:
URL: https://github.com/apache/trafficcontrol/pull/6872#discussion_r886928833
##########
blueprints/to_soa.md:
##########
@@ -0,0 +1,159 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Service Oriented Architecture for Traffic Ops
+
+## Problem Description
+Traffic Ops (TO), in its current form, can be best described as a monolithic application.
+All the code and logic for TO resides in one single application. Hence, making even a minor
+change to one area of TO requires a redeploy of the entire TO application. This is not ideal,
+as it adds the additional burden of deploying a huge application, for a small change. In order to
+move faster and in a more seamless manner, TO needs to be refactored so that we can redeploy/ restart
+one part of the application without affecting the other parts.
+
+Rewriting the entire TO application will be a highly time and resource intensive effort. However, we could
+refactor TO, to convert it into a Service Oriented architecture (SOA) style product. This would allow us to
+decouple new features and functionality from the monolith while also providing the opportunity to re-factor out
+existing features in the future.
+
+## Proposed Change
+The goal is to transition TO from a monolithic application to a more service-oriented architecture.
+What this means is that TO will be more of a "collection of services". New features and functionality will be
+added as stand-alone services with stand-alone testing, deployment and scalability. TO, the way it is today, will serve
+as `TO-core`, and any new service will be added as a "backend service", fulfilling the requirements of that service, and nothing else.
+The user would still send a request to the TO API. TO, upon receiving this request, would check in its configured "backend routes". If
+a match is encountered, TO would act as a reverse proxy for this service. This means that TO would simply forward the request to the backend
+service, get the response from the backend service and send it back to the user. From the user's perspective, this should be totally transparent
+and seamless. Once TO starts supporting the SOA pattern, we would be able to scale the individual services independently of TO core.
+
+
+### Traffic Portal Impact
+Any new Traffic Portal (TP) changes that the backend service requires, will be required to be added as modules to the current TP architecture.
+As for the TO refactor, no TP changes are required.
+
+### Traffic Ops Impact
+TO will need to read a new configuration file(supplied with the `backendcfg` option) on startup. This file will list the routes that will be served by the backend services, some route
+specific metadata (permissions, route ID, etc.) and where the incoming requests on those paths should be reverse proxied to. Any change to
+this configuration file will require a reload of the new configuration values on TO's side. This can be achieved by sending a `SIGHUP` signal
+to the TO process.
+
+The contents of an example backend configuration file are listed below:
+
+```json
Review Comment:
To include comments in highlighted json, tag the code block with `jsonc` instead of `json`.
# Example
## `json`
```json
{
// This is a syntax error
"test": "quest"
}
```
## `jsonc`
```jsonc
{
// This is fine
"test": "quest"
}
```
##########
blueprints/to_soa.md:
##########
@@ -0,0 +1,159 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Service Oriented Architecture for Traffic Ops
+
+## Problem Description
+Traffic Ops (TO), in its current form, can be best described as a monolithic application.
+All the code and logic for TO resides in one single application. Hence, making even a minor
+change to one area of TO requires a redeploy of the entire TO application. This is not ideal,
+as it adds the additional burden of deploying a huge application, for a small change. In order to
+move faster and in a more seamless manner, TO needs to be refactored so that we can redeploy/ restart
+one part of the application without affecting the other parts.
+
+Rewriting the entire TO application will be a highly time and resource intensive effort. However, we could
+refactor TO, to convert it into a Service Oriented architecture (SOA) style product. This would allow us to
+decouple new features and functionality from the monolith while also providing the opportunity to re-factor out
+existing features in the future.
+
+## Proposed Change
+The goal is to transition TO from a monolithic application to a more service-oriented architecture.
+What this means is that TO will be more of a "collection of services". New features and functionality will be
+added as stand-alone services with stand-alone testing, deployment and scalability. TO, the way it is today, will serve
+as `TO-core`, and any new service will be added as a "backend service", fulfilling the requirements of that service, and nothing else.
+The user would still send a request to the TO API. TO, upon receiving this request, would check in its configured "backend routes". If
+a match is encountered, TO would act as a reverse proxy for this service. This means that TO would simply forward the request to the backend
+service, get the response from the backend service and send it back to the user. From the user's perspective, this should be totally transparent
+and seamless. Once TO starts supporting the SOA pattern, we would be able to scale the individual services independently of TO core.
+
+
+### Traffic Portal Impact
+Any new Traffic Portal (TP) changes that the backend service requires, will be required to be added as modules to the current TP architecture.
+As for the TO refactor, no TP changes are required.
+
+### Traffic Ops Impact
+TO will need to read a new configuration file(supplied with the `backendcfg` option) on startup. This file will list the routes that will be served by the backend services, some route
+specific metadata (permissions, route ID, etc.) and where the incoming requests on those paths should be reverse proxied to. Any change to
+this configuration file will require a reload of the new configuration values on TO's side. This can be achieved by sending a `SIGHUP` signal
+to the TO process.
+
+The contents of an example backend configuration file are listed below:
+
+```json
+{
+ // List of routes for which TO needs to act as a reverse proxy
+ "routes": [
+ {
+ // The path to match on
+ "path": "^/api/4.0/foo/?$",
+ // The HTTP method for this path that TO should support
+ "method": "GET",
+ // List of hosts where the backend service is running, and where TO needs to forward the incoming request to
+ "hosts": [
+ {
+ // Protocol to follow while forwarding the requests
+ "protocol": "https",
+ // Host name where the backend service is running
+ "hostname": "to.backend.net",
+ // Port where the backend service is running on the above mentioned host
+ "port": 8444
+ },
+ {
+ "protocol": "https",
+ "hostname": "to.backend.net",
+ "port": 8445
+ }
+ ],
+ // A boolean specifying whether or not TO should verify the backend server's certificate chain and host name. This is not recommended for production use.
+ "insecure": false,
+ // Permissions required to access the route
+ "permissions": [
+ "FOO:READ"
+ ],
+ // ID for the route
+ "routeId": 123456,
+ // Extra options to configure how TO forwards these requests
+ "opts": {
+ // Currently, TO just supports a round robin algorithm to forward the requests.
+ "alg": "roundrobin"
+ }
+ }
+ ]
+}
+```
+
+#### REST API Impact
+n/a
+
+#### Client Impact
+Clients will still send requests and receive responses (for the newly configured endpoints) from TO, so nothing will change from the clients' perspective.
+
+#### Data Model / Database Impact
+There is no impact on the existing TO database structure/ schema. The backend services are free to have their own databases and
+serve data from there. As before, only TO will be allowed to access the Traffic Ops Database (TODB).
+
+### t3c Impact
+n/a
+
+### Traffic Monitor Impact
+n/a
+
+### Traffic Router Impact
+n/a
+
+### Traffic Stats Impact
+n/a
+
+### Traffic Vault Impact
+n/a
+
+### Documentation Impact
+Documentation will have to be added for the new configuration option for TO.
+
+### Testing Impact
+The backend services will be responsible for writing their own test suites.
+
+### Performance Impact
+We don't foresee any performance impact on TO as a result of this architecture change.
+
+### Security Impact
+SSL certificates will be required for HTTPS protocol to be used between TO and the services. These can be implemented however is preferred by the
+backend service. The overhead will include certificate generation and signing, certificate deployment to services' servers, certificate management
+and renewal. Client certificates are an option for the future but will require further development.
+
+### Upgrade Impact
+If a change has to go in to one of the backend services, a redeploy/ restart of TO will no longer be necessary. The backend service will, however,
+need to be re deployed. After that, if there is no change to the backend configuration file, no action will be required on the TO side. Else, as explained
+earlier, a SIGHUP signal will need to be sent to TO to reload the new configuration values.
+
+### Operations Impact
+The added backend services will add some overhead to operations. However, this architecture will also reduce the load on operations from the TO side, since
+every code change now won't force redeploying TO as a whole.
+
+### Developer Impact
+If a new endpoint (or refactor of an existing one) is deemed as a fit candidate for a backend service, the developer will have to undertake the tasks of writing
+this new service (which can be a simple server serving these endpoints), and the tests associated with it.
Review Comment:
With regard to future API development, is the expectation that new API endpoints always be "back-end" services that will live in the repository? Or is this only meant to be used by organizations for internal business logic and purposes? Or some combination of the two?
##########
blueprints/to_soa.md:
##########
@@ -0,0 +1,159 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Service Oriented Architecture for Traffic Ops
+
+## Problem Description
+Traffic Ops (TO), in its current form, can be best described as a monolithic application.
+All the code and logic for TO resides in one single application. Hence, making even a minor
+change to one area of TO requires a redeploy of the entire TO application. This is not ideal,
+as it adds the additional burden of deploying a huge application, for a small change. In order to
+move faster and in a more seamless manner, TO needs to be refactored so that we can redeploy/ restart
+one part of the application without affecting the other parts.
+
+Rewriting the entire TO application will be a highly time and resource intensive effort. However, we could
+refactor TO, to convert it into a Service Oriented architecture (SOA) style product. This would allow us to
+decouple new features and functionality from the monolith while also providing the opportunity to re-factor out
+existing features in the future.
+
+## Proposed Change
+The goal is to transition TO from a monolithic application to a more service-oriented architecture.
+What this means is that TO will be more of a "collection of services". New features and functionality will be
+added as stand-alone services with stand-alone testing, deployment and scalability. TO, the way it is today, will serve
+as `TO-core`, and any new service will be added as a "backend service", fulfilling the requirements of that service, and nothing else.
+The user would still send a request to the TO API. TO, upon receiving this request, would check in its configured "backend routes". If
+a match is encountered, TO would act as a reverse proxy for this service. This means that TO would simply forward the request to the backend
+service, get the response from the backend service and send it back to the user. From the user's perspective, this should be totally transparent
+and seamless. Once TO starts supporting the SOA pattern, we would be able to scale the individual services independently of TO core.
+
+
+### Traffic Portal Impact
+Any new Traffic Portal (TP) changes that the backend service requires, will be required to be added as modules to the current TP architecture.
+As for the TO refactor, no TP changes are required.
+
+### Traffic Ops Impact
+TO will need to read a new configuration file(supplied with the `backendcfg` option) on startup. This file will list the routes that will be served by the backend services, some route
Review Comment:
Does it need to be a new configuration file? Can't this just go in the standard TO config?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] srijeet0406 commented on a diff in pull request #6872: Traffic Ops Service Oriented Architecture blueprint
Posted by GitBox <gi...@apache.org>.
srijeet0406 commented on code in PR #6872:
URL: https://github.com/apache/trafficcontrol/pull/6872#discussion_r887258543
##########
blueprints/to_soa.md:
##########
@@ -0,0 +1,159 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Service Oriented Architecture for Traffic Ops
+
+## Problem Description
+Traffic Ops (TO), in its current form, can be best described as a monolithic application.
+All the code and logic for TO resides in one single application. Hence, making even a minor
+change to one area of TO requires a redeploy of the entire TO application. This is not ideal,
+as it adds the additional burden of deploying a huge application, for a small change. In order to
+move faster and in a more seamless manner, TO needs to be refactored so that we can redeploy/ restart
+one part of the application without affecting the other parts.
+
+Rewriting the entire TO application will be a highly time and resource intensive effort. However, we could
+refactor TO, to convert it into a Service Oriented architecture (SOA) style product. This would allow us to
+decouple new features and functionality from the monolith while also providing the opportunity to re-factor out
+existing features in the future.
+
+## Proposed Change
+The goal is to transition TO from a monolithic application to a more service-oriented architecture.
+What this means is that TO will be more of a "collection of services". New features and functionality will be
+added as stand-alone services with stand-alone testing, deployment and scalability. TO, the way it is today, will serve
+as `TO-core`, and any new service will be added as a "backend service", fulfilling the requirements of that service, and nothing else.
+The user would still send a request to the TO API. TO, upon receiving this request, would check in its configured "backend routes". If
+a match is encountered, TO would act as a reverse proxy for this service. This means that TO would simply forward the request to the backend
+service, get the response from the backend service and send it back to the user. From the user's perspective, this should be totally transparent
+and seamless. Once TO starts supporting the SOA pattern, we would be able to scale the individual services independently of TO core.
+
+
+### Traffic Portal Impact
+Any new Traffic Portal (TP) changes that the backend service requires, will be required to be added as modules to the current TP architecture.
+As for the TO refactor, no TP changes are required.
+
+### Traffic Ops Impact
+TO will need to read a new configuration file(supplied with the `backendcfg` option) on startup. This file will list the routes that will be served by the backend services, some route
+specific metadata (permissions, route ID, etc.) and where the incoming requests on those paths should be reverse proxied to. Any change to
+this configuration file will require a reload of the new configuration values on TO's side. This can be achieved by sending a `SIGHUP` signal
+to the TO process.
+
+The contents of an example backend configuration file are listed below:
+
+```json
+{
+ // List of routes for which TO needs to act as a reverse proxy
+ "routes": [
+ {
+ // The path to match on
+ "path": "^/api/4.0/foo/?$",
+ // The HTTP method for this path that TO should support
+ "method": "GET",
+ // List of hosts where the backend service is running, and where TO needs to forward the incoming request to
+ "hosts": [
+ {
+ // Protocol to follow while forwarding the requests
+ "protocol": "https",
+ // Host name where the backend service is running
+ "hostname": "to.backend.net",
+ // Port where the backend service is running on the above mentioned host
+ "port": 8444
+ },
+ {
+ "protocol": "https",
+ "hostname": "to.backend.net",
+ "port": 8445
+ }
+ ],
+ // A boolean specifying whether or not TO should verify the backend server's certificate chain and host name. This is not recommended for production use.
+ "insecure": false,
+ // Permissions required to access the route
+ "permissions": [
+ "FOO:READ"
+ ],
+ // ID for the route
+ "routeId": 123456,
+ // Extra options to configure how TO forwards these requests
+ "opts": {
+ // Currently, TO just supports a round robin algorithm to forward the requests.
+ "alg": "roundrobin"
+ }
+ }
+ ]
+}
+```
+
+#### REST API Impact
+n/a
+
+#### Client Impact
+Clients will still send requests and receive responses (for the newly configured endpoints) from TO, so nothing will change from the clients' perspective.
+
+#### Data Model / Database Impact
+There is no impact on the existing TO database structure/ schema. The backend services are free to have their own databases and
+serve data from there. As before, only TO will be allowed to access the Traffic Ops Database (TODB).
+
+### t3c Impact
+n/a
+
+### Traffic Monitor Impact
+n/a
+
+### Traffic Router Impact
+n/a
+
+### Traffic Stats Impact
+n/a
+
+### Traffic Vault Impact
+n/a
+
+### Documentation Impact
+Documentation will have to be added for the new configuration option for TO.
+
+### Testing Impact
+The backend services will be responsible for writing their own test suites.
+
+### Performance Impact
+We don't foresee any performance impact on TO as a result of this architecture change.
+
+### Security Impact
+SSL certificates will be required for HTTPS protocol to be used between TO and the services. These can be implemented however is preferred by the
+backend service. The overhead will include certificate generation and signing, certificate deployment to services' servers, certificate management
+and renewal. Client certificates are an option for the future but will require further development.
+
+### Upgrade Impact
+If a change has to go in to one of the backend services, a redeploy/ restart of TO will no longer be necessary. The backend service will, however,
+need to be re deployed. After that, if there is no change to the backend configuration file, no action will be required on the TO side. Else, as explained
+earlier, a SIGHUP signal will need to be sent to TO to reload the new configuration values.
+
+### Operations Impact
+The added backend services will add some overhead to operations. However, this architecture will also reduce the load on operations from the TO side, since
+every code change now won't force redeploying TO as a whole.
+
+### Developer Impact
+If a new endpoint (or refactor of an existing one) is deemed as a fit candidate for a backend service, the developer will have to undertake the tasks of writing
+this new service (which can be a simple server serving these endpoints), and the tests associated with it.
Review Comment:
You're right. The idea is not to deprecate the plugin system. We'll have the plugins as they are, going forward. This architecture will be used by backend services that have their own databases and don't need to access the TODB.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a diff in pull request #6872: Traffic Ops Service Oriented Architecture blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on code in PR #6872:
URL: https://github.com/apache/trafficcontrol/pull/6872#discussion_r887253794
##########
blueprints/to_soa.md:
##########
@@ -0,0 +1,159 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Service Oriented Architecture for Traffic Ops
+
+## Problem Description
+Traffic Ops (TO), in its current form, can be best described as a monolithic application.
+All the code and logic for TO resides in one single application. Hence, making even a minor
+change to one area of TO requires a redeploy of the entire TO application. This is not ideal,
+as it adds the additional burden of deploying a huge application, for a small change. In order to
+move faster and in a more seamless manner, TO needs to be refactored so that we can redeploy/ restart
+one part of the application without affecting the other parts.
+
+Rewriting the entire TO application will be a highly time and resource intensive effort. However, we could
+refactor TO, to convert it into a Service Oriented architecture (SOA) style product. This would allow us to
+decouple new features and functionality from the monolith while also providing the opportunity to re-factor out
+existing features in the future.
+
+## Proposed Change
+The goal is to transition TO from a monolithic application to a more service-oriented architecture.
+What this means is that TO will be more of a "collection of services". New features and functionality will be
+added as stand-alone services with stand-alone testing, deployment and scalability. TO, the way it is today, will serve
+as `TO-core`, and any new service will be added as a "backend service", fulfilling the requirements of that service, and nothing else.
+The user would still send a request to the TO API. TO, upon receiving this request, would check in its configured "backend routes". If
+a match is encountered, TO would act as a reverse proxy for this service. This means that TO would simply forward the request to the backend
+service, get the response from the backend service and send it back to the user. From the user's perspective, this should be totally transparent
+and seamless. Once TO starts supporting the SOA pattern, we would be able to scale the individual services independently of TO core.
+
+
+### Traffic Portal Impact
+Any new Traffic Portal (TP) changes that the backend service requires, will be required to be added as modules to the current TP architecture.
+As for the TO refactor, no TP changes are required.
+
+### Traffic Ops Impact
+TO will need to read a new configuration file(supplied with the `backendcfg` option) on startup. This file will list the routes that will be served by the backend services, some route
Review Comment:
if `dbCfg` and `riakcfg` don't have short-form flags, then I wouldn't bother. They can all have them added at some point if anyone cares enough to do it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a diff in pull request #6872: Traffic Ops Service Oriented Architecture blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on code in PR #6872:
URL: https://github.com/apache/trafficcontrol/pull/6872#discussion_r887340563
##########
blueprints/to_soa.md:
##########
@@ -0,0 +1,159 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Service Oriented Architecture for Traffic Ops
+
+## Problem Description
+Traffic Ops (TO), in its current form, can be best described as a monolithic application.
+All the code and logic for TO resides in one single application. Hence, making even a minor
+change to one area of TO requires a redeploy of the entire TO application. This is not ideal,
+as it adds the additional burden of deploying a huge application, for a small change. In order to
+move faster and in a more seamless manner, TO needs to be refactored so that we can redeploy/ restart
+one part of the application without affecting the other parts.
+
+Rewriting the entire TO application will be a highly time and resource intensive effort. However, we could
+refactor TO, to convert it into a Service Oriented architecture (SOA) style product. This would allow us to
+decouple new features and functionality from the monolith while also providing the opportunity to re-factor out
+existing features in the future.
+
+## Proposed Change
+The goal is to transition TO from a monolithic application to a more service-oriented architecture.
+What this means is that TO will be more of a "collection of services". New features and functionality will be
+added as stand-alone services with stand-alone testing, deployment and scalability. TO, the way it is today, will serve
+as `TO-core`, and any new service will be added as a "backend service", fulfilling the requirements of that service, and nothing else.
+The user would still send a request to the TO API. TO, upon receiving this request, would check in its configured "backend routes". If
+a match is encountered, TO would act as a reverse proxy for this service. This means that TO would simply forward the request to the backend
+service, get the response from the backend service and send it back to the user. From the user's perspective, this should be totally transparent
+and seamless. Once TO starts supporting the SOA pattern, we would be able to scale the individual services independently of TO core.
+
+
+### Traffic Portal Impact
+Any new Traffic Portal (TP) changes that the backend service requires, will be required to be added as modules to the current TP architecture.
+As for the TO refactor, no TP changes are required.
+
+### Traffic Ops Impact
+TO will need to read a new configuration file(supplied with the `backendcfg` option) on startup. This file will list the routes that will be served by the backend services, some route
+specific metadata (permissions, route ID, etc.) and where the incoming requests on those paths should be reverse proxied to. Any change to
+this configuration file will require a reload of the new configuration values on TO's side. This can be achieved by sending a `SIGHUP` signal
+to the TO process.
+
+The contents of an example backend configuration file are listed below:
+
+```json
+{
+ // List of routes for which TO needs to act as a reverse proxy
+ "routes": [
+ {
+ // The path to match on
+ "path": "^/api/4.0/foo/?$",
+ // The HTTP method for this path that TO should support
+ "method": "GET",
+ // List of hosts where the backend service is running, and where TO needs to forward the incoming request to
+ "hosts": [
+ {
+ // Protocol to follow while forwarding the requests
+ "protocol": "https",
+ // Host name where the backend service is running
+ "hostname": "to.backend.net",
+ // Port where the backend service is running on the above mentioned host
+ "port": 8444
+ },
+ {
+ "protocol": "https",
+ "hostname": "to.backend.net",
+ "port": 8445
+ }
+ ],
+ // A boolean specifying whether or not TO should verify the backend server's certificate chain and host name. This is not recommended for production use.
+ "insecure": false,
+ // Permissions required to access the route
+ "permissions": [
+ "FOO:READ"
+ ],
+ // ID for the route
+ "routeId": 123456,
+ // Extra options to configure how TO forwards these requests
+ "opts": {
+ // Currently, TO just supports a round robin algorithm to forward the requests.
+ "alg": "roundrobin"
+ }
+ }
+ ]
+}
+```
+
+#### REST API Impact
+n/a
+
+#### Client Impact
+Clients will still send requests and receive responses (for the newly configured endpoints) from TO, so nothing will change from the clients' perspective.
+
+#### Data Model / Database Impact
+There is no impact on the existing TO database structure/ schema. The backend services are free to have their own databases and
+serve data from there. As before, only TO will be allowed to access the Traffic Ops Database (TODB).
+
+### t3c Impact
+n/a
+
+### Traffic Monitor Impact
+n/a
+
+### Traffic Router Impact
+n/a
+
+### Traffic Stats Impact
+n/a
+
+### Traffic Vault Impact
+n/a
+
+### Documentation Impact
+Documentation will have to be added for the new configuration option for TO.
+
+### Testing Impact
+The backend services will be responsible for writing their own test suites.
+
+### Performance Impact
+We don't foresee any performance impact on TO as a result of this architecture change.
+
+### Security Impact
+SSL certificates will be required for HTTPS protocol to be used between TO and the services. These can be implemented however is preferred by the
+backend service. The overhead will include certificate generation and signing, certificate deployment to services' servers, certificate management
+and renewal. Client certificates are an option for the future but will require further development.
+
+### Upgrade Impact
+If a change has to go in to one of the backend services, a redeploy/ restart of TO will no longer be necessary. The backend service will, however,
+need to be re deployed. After that, if there is no change to the backend configuration file, no action will be required on the TO side. Else, as explained
+earlier, a SIGHUP signal will need to be sent to TO to reload the new configuration values.
+
+### Operations Impact
+The added backend services will add some overhead to operations. However, this architecture will also reduce the load on operations from the TO side, since
+every code change now won't force redeploying TO as a whole.
+
+### Developer Impact
+If a new endpoint (or refactor of an existing one) is deemed as a fit candidate for a backend service, the developer will have to undertake the tasks of writing
+this new service (which can be a simple server serving these endpoints), and the tests associated with it.
Review Comment:
Okay, that's probably fine
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a diff in pull request #6872: Traffic Ops Service Oriented Architecture blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on code in PR #6872:
URL: https://github.com/apache/trafficcontrol/pull/6872#discussion_r887255796
##########
blueprints/to_soa.md:
##########
@@ -0,0 +1,159 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Service Oriented Architecture for Traffic Ops
+
+## Problem Description
+Traffic Ops (TO), in its current form, can be best described as a monolithic application.
+All the code and logic for TO resides in one single application. Hence, making even a minor
+change to one area of TO requires a redeploy of the entire TO application. This is not ideal,
+as it adds the additional burden of deploying a huge application, for a small change. In order to
+move faster and in a more seamless manner, TO needs to be refactored so that we can redeploy/ restart
+one part of the application without affecting the other parts.
+
+Rewriting the entire TO application will be a highly time and resource intensive effort. However, we could
+refactor TO, to convert it into a Service Oriented architecture (SOA) style product. This would allow us to
+decouple new features and functionality from the monolith while also providing the opportunity to re-factor out
+existing features in the future.
+
+## Proposed Change
+The goal is to transition TO from a monolithic application to a more service-oriented architecture.
+What this means is that TO will be more of a "collection of services". New features and functionality will be
+added as stand-alone services with stand-alone testing, deployment and scalability. TO, the way it is today, will serve
+as `TO-core`, and any new service will be added as a "backend service", fulfilling the requirements of that service, and nothing else.
+The user would still send a request to the TO API. TO, upon receiving this request, would check in its configured "backend routes". If
+a match is encountered, TO would act as a reverse proxy for this service. This means that TO would simply forward the request to the backend
+service, get the response from the backend service and send it back to the user. From the user's perspective, this should be totally transparent
+and seamless. Once TO starts supporting the SOA pattern, we would be able to scale the individual services independently of TO core.
+
+
+### Traffic Portal Impact
+Any new Traffic Portal (TP) changes that the backend service requires, will be required to be added as modules to the current TP architecture.
+As for the TO refactor, no TP changes are required.
+
+### Traffic Ops Impact
+TO will need to read a new configuration file(supplied with the `backendcfg` option) on startup. This file will list the routes that will be served by the backend services, some route
+specific metadata (permissions, route ID, etc.) and where the incoming requests on those paths should be reverse proxied to. Any change to
+this configuration file will require a reload of the new configuration values on TO's side. This can be achieved by sending a `SIGHUP` signal
+to the TO process.
+
+The contents of an example backend configuration file are listed below:
+
+```json
+{
+ // List of routes for which TO needs to act as a reverse proxy
+ "routes": [
+ {
+ // The path to match on
+ "path": "^/api/4.0/foo/?$",
+ // The HTTP method for this path that TO should support
+ "method": "GET",
+ // List of hosts where the backend service is running, and where TO needs to forward the incoming request to
+ "hosts": [
+ {
+ // Protocol to follow while forwarding the requests
+ "protocol": "https",
+ // Host name where the backend service is running
+ "hostname": "to.backend.net",
+ // Port where the backend service is running on the above mentioned host
+ "port": 8444
+ },
+ {
+ "protocol": "https",
+ "hostname": "to.backend.net",
+ "port": 8445
+ }
+ ],
+ // A boolean specifying whether or not TO should verify the backend server's certificate chain and host name. This is not recommended for production use.
+ "insecure": false,
+ // Permissions required to access the route
+ "permissions": [
+ "FOO:READ"
+ ],
+ // ID for the route
+ "routeId": 123456,
+ // Extra options to configure how TO forwards these requests
+ "opts": {
+ // Currently, TO just supports a round robin algorithm to forward the requests.
+ "alg": "roundrobin"
+ }
+ }
+ ]
+}
+```
+
+#### REST API Impact
+n/a
+
+#### Client Impact
+Clients will still send requests and receive responses (for the newly configured endpoints) from TO, so nothing will change from the clients' perspective.
+
+#### Data Model / Database Impact
+There is no impact on the existing TO database structure/ schema. The backend services are free to have their own databases and
+serve data from there. As before, only TO will be allowed to access the Traffic Ops Database (TODB).
+
+### t3c Impact
+n/a
+
+### Traffic Monitor Impact
+n/a
+
+### Traffic Router Impact
+n/a
+
+### Traffic Stats Impact
+n/a
+
+### Traffic Vault Impact
+n/a
+
+### Documentation Impact
+Documentation will have to be added for the new configuration option for TO.
+
+### Testing Impact
+The backend services will be responsible for writing their own test suites.
+
+### Performance Impact
+We don't foresee any performance impact on TO as a result of this architecture change.
+
+### Security Impact
+SSL certificates will be required for HTTPS protocol to be used between TO and the services. These can be implemented however is preferred by the
+backend service. The overhead will include certificate generation and signing, certificate deployment to services' servers, certificate management
+and renewal. Client certificates are an option for the future but will require further development.
+
+### Upgrade Impact
+If a change has to go in to one of the backend services, a redeploy/ restart of TO will no longer be necessary. The backend service will, however,
+need to be re deployed. After that, if there is no change to the backend configuration file, no action will be required on the TO side. Else, as explained
+earlier, a SIGHUP signal will need to be sent to TO to reload the new configuration values.
+
+### Operations Impact
+The added backend services will add some overhead to operations. However, this architecture will also reduce the load on operations from the TO side, since
+every code change now won't force redeploying TO as a whole.
+
+### Developer Impact
+If a new endpoint (or refactor of an existing one) is deemed as a fit candidate for a backend service, the developer will have to undertake the tasks of writing
+this new service (which can be a simple server serving these endpoints), and the tests associated with it.
Review Comment:
Plugins, though, have access to the Traffic Ops database, so they currently can't all be implemented as back-end services as described in this blueprint because:
> As before, only TO will be allowed to access the Traffic Ops Database
So if the intention is to deprecate the existing plugin system, then should services have access to the TODB?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] mattjackson220 merged pull request #6872: Traffic Ops Service Oriented Architecture blueprint
Posted by GitBox <gi...@apache.org>.
mattjackson220 merged PR #6872:
URL: https://github.com/apache/trafficcontrol/pull/6872
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a diff in pull request #6872: Traffic Ops Service Oriented Architecture blueprint
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on code in PR #6872:
URL: https://github.com/apache/trafficcontrol/pull/6872#discussion_r887097608
##########
blueprints/to_soa.md:
##########
@@ -0,0 +1,159 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Service Oriented Architecture for Traffic Ops
+
+## Problem Description
+Traffic Ops (TO), in its current form, can be best described as a monolithic application.
+All the code and logic for TO resides in one single application. Hence, making even a minor
+change to one area of TO requires a redeploy of the entire TO application. This is not ideal,
+as it adds the additional burden of deploying a huge application, for a small change. In order to
+move faster and in a more seamless manner, TO needs to be refactored so that we can redeploy/ restart
+one part of the application without affecting the other parts.
+
+Rewriting the entire TO application will be a highly time and resource intensive effort. However, we could
+refactor TO, to convert it into a Service Oriented architecture (SOA) style product. This would allow us to
+decouple new features and functionality from the monolith while also providing the opportunity to re-factor out
+existing features in the future.
+
+## Proposed Change
+The goal is to transition TO from a monolithic application to a more service-oriented architecture.
+What this means is that TO will be more of a "collection of services". New features and functionality will be
+added as stand-alone services with stand-alone testing, deployment and scalability. TO, the way it is today, will serve
+as `TO-core`, and any new service will be added as a "backend service", fulfilling the requirements of that service, and nothing else.
+The user would still send a request to the TO API. TO, upon receiving this request, would check in its configured "backend routes". If
+a match is encountered, TO would act as a reverse proxy for this service. This means that TO would simply forward the request to the backend
+service, get the response from the backend service and send it back to the user. From the user's perspective, this should be totally transparent
+and seamless. Once TO starts supporting the SOA pattern, we would be able to scale the individual services independently of TO core.
+
+
+### Traffic Portal Impact
+Any new Traffic Portal (TP) changes that the backend service requires, will be required to be added as modules to the current TP architecture.
+As for the TO refactor, no TP changes are required.
+
+### Traffic Ops Impact
+TO will need to read a new configuration file(supplied with the `backendcfg` option) on startup. This file will list the routes that will be served by the backend services, some route
+specific metadata (permissions, route ID, etc.) and where the incoming requests on those paths should be reverse proxied to. Any change to
+this configuration file will require a reload of the new configuration values on TO's side. This can be achieved by sending a `SIGHUP` signal
+to the TO process.
+
+The contents of an example backend configuration file are listed below:
+
+```json
+{
+ // List of routes for which TO needs to act as a reverse proxy
+ "routes": [
+ {
+ // The path to match on
+ "path": "^/api/4.0/foo/?$",
+ // The HTTP method for this path that TO should support
+ "method": "GET",
+ // List of hosts where the backend service is running, and where TO needs to forward the incoming request to
+ "hosts": [
+ {
+ // Protocol to follow while forwarding the requests
+ "protocol": "https",
+ // Host name where the backend service is running
+ "hostname": "to.backend.net",
+ // Port where the backend service is running on the above mentioned host
+ "port": 8444
+ },
+ {
+ "protocol": "https",
+ "hostname": "to.backend.net",
+ "port": 8445
+ }
+ ],
+ // A boolean specifying whether or not TO should verify the backend server's certificate chain and host name. This is not recommended for production use.
+ "insecure": false,
+ // Permissions required to access the route
+ "permissions": [
+ "FOO:READ"
+ ],
+ // ID for the route
+ "routeId": 123456,
+ // Extra options to configure how TO forwards these requests
+ "opts": {
+ // Currently, TO just supports a round robin algorithm to forward the requests.
+ "alg": "roundrobin"
+ }
+ }
+ ]
+}
+```
+
+#### REST API Impact
+n/a
+
+#### Client Impact
+Clients will still send requests and receive responses (for the newly configured endpoints) from TO, so nothing will change from the clients' perspective.
+
+#### Data Model / Database Impact
+There is no impact on the existing TO database structure/ schema. The backend services are free to have their own databases and
+serve data from there. As before, only TO will be allowed to access the Traffic Ops Database (TODB).
+
+### t3c Impact
+n/a
+
+### Traffic Monitor Impact
+n/a
+
+### Traffic Router Impact
+n/a
+
+### Traffic Stats Impact
+n/a
+
+### Traffic Vault Impact
+n/a
+
+### Documentation Impact
+Documentation will have to be added for the new configuration option for TO.
+
+### Testing Impact
+The backend services will be responsible for writing their own test suites.
+
+### Performance Impact
+We don't foresee any performance impact on TO as a result of this architecture change.
+
+### Security Impact
+SSL certificates will be required for HTTPS protocol to be used between TO and the services. These can be implemented however is preferred by the
+backend service. The overhead will include certificate generation and signing, certificate deployment to services' servers, certificate management
+and renewal. Client certificates are an option for the future but will require further development.
+
+### Upgrade Impact
+If a change has to go in to one of the backend services, a redeploy/ restart of TO will no longer be necessary. The backend service will, however,
+need to be re deployed. After that, if there is no change to the backend configuration file, no action will be required on the TO side. Else, as explained
+earlier, a SIGHUP signal will need to be sent to TO to reload the new configuration values.
+
+### Operations Impact
+The added backend services will add some overhead to operations. However, this architecture will also reduce the load on operations from the TO side, since
+every code change now won't force redeploying TO as a whole.
+
+### Developer Impact
+If a new endpoint (or refactor of an existing one) is deemed as a fit candidate for a backend service, the developer will have to undertake the tasks of writing
+this new service (which can be a simple server serving these endpoints), and the tests associated with it.
Review Comment:
Seems like #6810 would want to make use of it, at least.
Is this meant to replace the existing plugin system, or would those still have a purpose moving forward?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] srijeet0406 commented on a diff in pull request #6872: Traffic Ops Service Oriented Architecture blueprint
Posted by GitBox <gi...@apache.org>.
srijeet0406 commented on code in PR #6872:
URL: https://github.com/apache/trafficcontrol/pull/6872#discussion_r887229928
##########
blueprints/to_soa.md:
##########
@@ -0,0 +1,159 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Service Oriented Architecture for Traffic Ops
+
+## Problem Description
+Traffic Ops (TO), in its current form, can be best described as a monolithic application.
+All the code and logic for TO resides in one single application. Hence, making even a minor
+change to one area of TO requires a redeploy of the entire TO application. This is not ideal,
+as it adds the additional burden of deploying a huge application, for a small change. In order to
+move faster and in a more seamless manner, TO needs to be refactored so that we can redeploy/ restart
+one part of the application without affecting the other parts.
+
+Rewriting the entire TO application will be a highly time and resource intensive effort. However, we could
+refactor TO, to convert it into a Service Oriented architecture (SOA) style product. This would allow us to
+decouple new features and functionality from the monolith while also providing the opportunity to re-factor out
+existing features in the future.
+
+## Proposed Change
+The goal is to transition TO from a monolithic application to a more service-oriented architecture.
+What this means is that TO will be more of a "collection of services". New features and functionality will be
+added as stand-alone services with stand-alone testing, deployment and scalability. TO, the way it is today, will serve
+as `TO-core`, and any new service will be added as a "backend service", fulfilling the requirements of that service, and nothing else.
+The user would still send a request to the TO API. TO, upon receiving this request, would check in its configured "backend routes". If
+a match is encountered, TO would act as a reverse proxy for this service. This means that TO would simply forward the request to the backend
+service, get the response from the backend service and send it back to the user. From the user's perspective, this should be totally transparent
+and seamless. Once TO starts supporting the SOA pattern, we would be able to scale the individual services independently of TO core.
+
+
+### Traffic Portal Impact
+Any new Traffic Portal (TP) changes that the backend service requires, will be required to be added as modules to the current TP architecture.
+As for the TO refactor, no TP changes are required.
+
+### Traffic Ops Impact
+TO will need to read a new configuration file(supplied with the `backendcfg` option) on startup. This file will list the routes that will be served by the backend services, some route
Review Comment:
Yes, this is an optional argument. I've addressed this in the latest commit. I haven't provided for a short flag synonym for this (I was just trying to mimic the `dbCfg` and `riakCfg` options). That said, there is nothing stopping us from adding a short hand option for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficcontrol.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org