You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@trafficcontrol.apache.org by GitBox <gi...@apache.org> on 2020/03/18 17:56:29 UTC

[GitHub] [trafficcontrol] ocket8888 opened a new pull request #4521: Multi-interface Servers blueprint

ocket8888 opened a new pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521
 
 
   <!--
   ************ STOP!! ************
   If this Pull Request is intended to fix a security vulnerability, DO NOT submit it! Instead, contact
   the Apache Software Foundation Security Team at security@trafficcontrol.apache.org and follow the
   guidelines at https://www.apache.org/security/ regarding vulnerability disclosure.
   -->
   ## What does this PR (Pull Request) do?
   <!-- Explain the changes you made here. If this fixes an Issue, identify it by
   replacing the text in the checkbox item with the Issue number e.g.
   
   - [x] This PR fixes #9001 OR is not related to any Issue
   
   ^ This will automatically close Issue number 9001 when the Pull Request is
   merged (The '#' is important).
   
   Be sure you check the box properly, see the "The following criteria are ALL
   met by this PR" section for details.
   -->
   
   - [ ] This PR fixes #REPLACE_ME OR is not related to any Issue <!-- You can check for an issue here: https://github.com/apache/trafficcontrol/issues -->
   
   
   ## Which Traffic Control components are affected by this PR?
   <!-- Please delete all components from this list that are NOT affected by this
   Pull Request. Also, feel free to add the name of a tool or script that is
   affected but not on the list.
   
   Additionally, if this Pull Request does NOT affect documentation, please
   explain why documentation is not required. -->
   
   - CDN in a Box
   - Documentation
   - Grove
   - Traffic Control Client <!-- Please specify which; e.g. 'Python', 'Go', 'Java' -->
   - Traffic Monitor
   - Traffic Ops
   - Traffic Ops ORT
   - Traffic Portal
   - Traffic Router
   - Traffic Stats
   - Traffic Vault
   
   ## What is the best way to verify this PR?
   <!-- Please include here ALL the steps necessary to test your Pull Request. If
   it includes tests (and most should), outline here the steps needed to run the
   tests. If not, lay out the manual testing procedure and please explain why
   tests are unnecessary for this Pull Request. -->
   
   ## If this is a bug fix, what versions of Traffic Control are affected?
   <!-- If this PR fixes a bug, please list here all of the affected versions - to
   the best of your knowledge. It's also pretty helpful to include a commit hash
   of where 'master' is at the time this PR is opened (if it affects master),
   because what 'master' means will change over time. For example, if this PR
   fixes a bug that's present in master (at commit hash '2697ebac'), in v3.0.0,
   and in the current 3.0.1 Release candidate (e.g. RC1), then this list would
   look like:
   
   - master (2697ebac)
   - 3.0.0
   - 3.0.1 (RC1)
   
   If you don't know what other versions might have this bug, AND don't know how
   to find the commit hash of 'master', then feel free to leave this section
   blank (or, preferably, delete it entirely).
    -->
   
   
   ## The following criteria are ALL met by this PR
   <!-- Check the boxes to signify that the associated statement is true. To
   "check a box", replace the space inside of the square brackets with an 'x'.
   e.g.
   
   - [ x] <- Wrong
   - [x ] <- Wrong
   - [] <- Wrong
   - [*] <- Wrong
   - [x] <- Correct!
   
   -->
   
   - [ ] This PR includes tests OR I have explained why tests are unnecessary
   - [ ] This PR includes documentation OR I have explained why documentation is unnecessary
   - [ ] This PR includes an update to CHANGELOG.md OR such an update is not necessary
   - [ ] This PR includes any and all required license headers
   - [ ] This PR ensures that database migration sequence is correct OR this PR does not include a database migration
   - [ ] This PR **DOES NOT FIX A SERIOUS SECURITY VULNERABILITY** (see [the Apache Software Foundation's security guidelines](https://www.apache.org/security/) for details)
   
   
   ## Additional Information
   <!-- If you would like to include any additional information on the PR for
   potential reviewers please put it here.
   
   Some examples of this would be:
   
   - Before and after screenshots/gifs of the Traffic Portal if it is affected
   - Links to other dependent Pull Requests
   - References to relevant context (e.g. new/updates to dependent libraries,
   mailing list records, blueprints)
   
   Feel free to leave this section blank (or, preferably, delete it entirely).
   -->
   
   <!--
   Licensed to the Apache Software Foundation (ASF) under one
   or more contributor license agreements.  See the NOTICE file
   distributed with this work for additional information
   regarding copyright ownership.  The ASF licenses this file
   to you under the Apache License, Version 2.0 (the
   "License"); you may not use this file except in compliance
   with the License.  You may obtain a copy of the License at
   
       http://www.apache.org/licenses/LICENSE-2.0
   
   Unless required by applicable law or agreed to in writing,
   software distributed under the License is distributed on an
   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
   KIND, either express or implied.  See the License for the
   specific language governing permissions and limitations
   under the License.
   -->
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r396794343
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
 
 Review comment:
   That's probably better, but to be honest I'm still not sure what MTU actually is or if 1500 and 9000 are the only values that ever can or will be possible for network interface devices. I was sort of hoping someone would call me out on that assertion if it was wrong.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r394739703
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
 
 Review comment:
   arbitrarily chosen?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r397229941
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
 
 Review comment:
   You distinguish those using the "monitored" field.  If an interface is monitored it is a "public" interface, if it is not monitored it is a "private" interface.  

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 edited a comment on issue #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 edited a comment on issue #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#issuecomment-601233207
 
 
   Could you add a blurb under the TP impact regarding the need to update the ISO generation form to handle multiple server interfaces when selecting the "Copy Server Attributes From" drop down? Or i guess you'd prefer a PR into this PR, right? i can do that.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r394739703
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
 
 Review comment:
   arbitrarily chosen? that doesn't sound right.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398728946
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,829 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * If it is an IPv6 address
+	 */
+	address: string;
+
+	/**
+	 * The netmask used by the address.
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * The maximum allowed bandwidth for this interface to be considered "healthy"
+	 * by Traffic Monitor.
+	 *
+	 * This has no effect if `monitor` is not true.
+	 * Values are in kb/s.
+	 * The value `0` means "no limit".
+	 */
+	maxBandwidth: bigint;
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: number | null;
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          bigint;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+            Table "traffic_ops.interface"
+     Column    |  Type   | Collation | Nullable | Default
+---------------+---------+-----------+----------+---------
+ name          | text    |           | not null |
+ server        | bigint  |           | not null |
+ monitor       | boolean |           | not null |
+ mtu           | integer |           |          | 1500
+ max_bandwidth | integer |           | not null | 0
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Check constraints:
+	"interface_mtu_check" CHECK (mtu > 1280)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+               Table "traffic_ops.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
 
 Review comment:
   With a foreign key constraint. The primary key of an interface is `(name, server)`, so `(interface, server)` is a foreign key into interfaces.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 edited a comment on issue #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 edited a comment on issue #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#issuecomment-604491881
 
 
   Is there any impact to:
   
   https://tp.domain.net/#!/cache-checks and the backing api endpoint - GET /servercheck
   https://tp.domain.net/#!/cache-stats and the backing api endpoint - GET /caches/stats
   
   And these api endpoints as well:
   
   GET /cdns/health
   GET /cdns/{name}/health
   GET /deliveryservices/{id}/health

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 commented on issue #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 commented on issue #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#issuecomment-604491881
 
 
   Is there any impact to:
   
   https://tp.domain.net/#!/cache-checks and the backing api endpoint(s)
   https://tp.domain.net/#!/cache-stats and the backing api endpoint(s)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r404164345
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,829 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * If it is an IPv6 address
+	 */
+	address: string;
+
+	/**
+	 * The netmask used by the address.
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * The maximum allowed bandwidth for this interface to be considered "healthy"
+	 * by Traffic Monitor.
+	 *
+	 * This has no effect if `monitor` is not true.
+	 * Values are in kb/s.
+	 * The value `0` means "no limit".
+	 */
+	maxBandwidth: bigint;
 
 Review comment:
   Oh, I see. This isn't what you guys think it is. This is the equivalent of the `health.threshold.availableBandwidthInKbps` Parameter today.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r396861690
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
 
 Review comment:
   How do you represent a server with two public and two private interface (used for different private networks)?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

rawlinp commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398749247

##########
File path: blueprints/multi-interface-servers.md
##########
@@ -0,0 +1,829 @@
+
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+ /**
+ * The actual IP (v4 or v6) address which is used by an interface.
+ * If it is an IPv6 address
+ */
+ address: string;
+
+ /**
+ * The netmask used by the address.
+
+ /**
+ * The IP (v4 or v6) address of the gateway used by this IP address.
+ */
+ gateway: string;
+
+ /**
+ * Tells whether or not this address of this interface is the server's
+ * "service" address.
+ * At least one address of EXACTLY ONE interface MUST have this set to
+ * 'true' for a server.
+ */
+ serviceAddress: boolean;

Review comment:
I could see supporting multiple service addresses per cache in the future, but I think we want to limit the scope of this feature to one service address per cache for the time being. Supporting that would mean we'd just have to drop that constraint and implement that in Traffic Router and ATS config (if necessary), right? So should need little API/DB rework.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [trafficcontrol] jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398212025
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
 
 Review comment:
   I'd be happy with something along the lines of adding:
   ```
   Any unsupported 3rd-party client code will need to be updated to
   leverage this new functionality in connection with these changes to core API/datamodel 
   concepts going forward.
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r394588892
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,657 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+
+```
+
+This should be sufficient to capture the model changes.
+
+#### API
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
 
 Review comment:
   just wondering. is `serviceAddress` applicable to all interfaces? for the interface that is included in the TR snapshot (the "active" interface), this has meaning but does it have meaning for the other interfaces?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] limited commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

limited commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398696596
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,829 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * If it is an IPv6 address
+	 */
+	address: string;
+
+	/**
+	 * The netmask used by the address.
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * The maximum allowed bandwidth for this interface to be considered "healthy"
+	 * by Traffic Monitor.
+	 *
+	 * This has no effect if `monitor` is not true.
+	 * Values are in kb/s.
+	 * The value `0` means "no limit".
+	 */
+	maxBandwidth: bigint;
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: number | null;
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          bigint;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+            Table "traffic_ops.interface"
+     Column    |  Type   | Collation | Nullable | Default
+---------------+---------+-----------+----------+---------
+ name          | text    |           | not null |
+ server        | bigint  |           | not null |
+ monitor       | boolean |           | not null |
+ mtu           | integer |           |          | 1500
+ max_bandwidth | integer |           | not null | 0
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Check constraints:
+	"interface_mtu_check" CHECK (mtu > 1280)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+               Table "traffic_ops.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
 
 Review comment:
   How do we guarantee consistency between interface.server and ip_address.server/ip_address.interface?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r395415004
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
+
+### Traffic Portal Impact
+The server edit/display/creation page will need to be updated to remove all of
+the properties of servers which no longer exist because they've been moved into
+the "interfaces" section, and a new set of interface-related input fields with
+controls for dynamically adding to them should be added.
+
+A sample screenshot is shown below.
+
+![](img/multi-interface-ui-sample.png "Mockup of proposed interface controls")
+
+If desired, the markup used for that sample may be found in
+[Appendix A](#appendix-a)
+
+Additionally, the properties of servers no longer present on a server object
+will need to be removed from table columns (and `interfaces` should not be
+added).
+
+### Traffic Monitor Impact
+Traffic Monitor will need to update its parsing of `/monitoring` payloads to
+account for the new structure, and will need to poll all listed interfaces for
+health (instead of just one per cache server as was done previously).
+
+The cache
+server should be marked unavailable - or "down" - if and only if the interface
+proving the service addresses is considered unhealthy.
+
+## Documentation Impact
+Documentation for the affected endpoints will need to be updated. Beyond that,
+the "Health Protocol" section and perhaps the "Traffic Monitor - Using" section
+should be updated to describe the behavior of multiple monitored interfaces.
+
+## Testing Impact
+A bunch of client/API integration tests should be written to verify the
+constraints outlined in [API](#sec:api).
+
+## Performance Impact
+The proposed method of updating servers to change their interfaces is by
+submitting a PUT request to `/servers/{{ID}}` like normal. Traffic Ops will then
+delete **all** associated interfaces before inserting the ones supplied in the
+PUT payload. This will occur even if the interfaces are not actually changing in
+values. So a single server update now requires a minimum of two extra database
+calls.
+
+This probably isn't a significant performance concern, but at the time of this
+writing no profiling has been done.
+
+## Security Impact
+Setting an improper interface can allow users with read-only permissions to read
+arbitrary files named 'speed' or 'operstate' that the 'ats' user has permission
+to read on the cache server's file system. Of course, this was already possible,
+and improper values can only be entered by users with at least "operator"
+permissions. Now, though, an arbitrary number of these files could be exposed
+per Snapshot, whereas previously it was limited to one file per cache server per
+Snapshot.
+
+Improper configuration has the same result as improper configuration of
+`interfaceName` currently, which is to say it can cause a cache server to be
+marked unavailable.
+
+Multiple new inputs will need to be sanitized, as always.
+
+> _"Will new SQL queries properly use parameter binding?"_
+
+The answer to this question should absolutely never be "no," can't imagine a
+world wherein that would need to change for this - or any other - particular
+feature.
+
+## Upgrade Impact
+There will be at least one, but possibly two, database migrations. The changes
+to Traffic Monitor ought to be made in such a way that it can parse either the
+old `/monitoring` payloads or the new ones. Then Traffic Monitor should be
+upgraded before Traffic Ops, and no additional, special steps need be taken to
+upgrade.
 
 Review comment:
   Well, the way this will work is this: if these changes get into the 2.0 API release, you can upgrade TO first because TM will still be using the 1.x API.
   
   And actually doing anything else would be a violation of semantic versioning, because we're removing fields not just adding them. So if this doesn't make it into API 2.0 it'd need to go in 3.0.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r404171954

Review comment:
Yes, this is being set in the database so that TM knows what the max_bw is per interface. We will still use the `health.threshold.kbps` parameter for the overall cache bandwidth threshold. The idea here is that TM needs to mark a cache unhealth if the overall max bw is reached OR the max bw for an interface is reached.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r394739192
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
 
 Review comment:
   camel case?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] rawlinp commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

rawlinp commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r397340024
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
 
 Review comment:
   Per https://www.ietf.org/rfc/rfc2460.txt we should probably constrain it to _at least_ 1280 or higher, but TP should probably still show 1500 and 9000 as the available options until we know we need different sizes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r396925229
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
 
 Review comment:
   There doesn't appear to be a spec for "super jumbo frames", but maybe I should add the "baby giant frames" value (1600).
   
   Also, what're they gonna do when "super jumbo" *is* standardized and they want something even bigger? "Giga Frames"? "Maximum Over-Frames"?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r397217997
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
 
 Review comment:
   There are many different values that you _could__ set for MTU.  1500 and 9000 are the most common, there's a 4500ish one that is pretty common too.  I  DO NOT think this should be an enum.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398726919
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,829 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * If it is an IPv6 address
+	 */
+	address: string;
+
+	/**
+	 * The netmask used by the address.
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
 
 Review comment:
   Because Traffic Router needs to be able to put the cache server in a hash ring.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398726919

Review comment:
Because Traffic Router needs to be able to put the cache server in a hash ring. It's also worth noting that a cache server today can only have one IPv4 and one IPv6 service address.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on issue #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on issue #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#issuecomment-610138270
 
 
   > Is there any impact to:
   > https://tp.domain.net/#!/cache-checks and the backing api endpoint - `GET /servercheck`
   > https://tp.domain.net/#!/cache-stats and the backing api endpoint - `GET /caches/stats`
   > And these api endpoints as well:
   > `GET /cdns/health`
   > `GET /cdns/{name}/health`
   > `GET /deliveryservices/{id}/health`
   
   - `/servercheck` - No. That endpoint does not interact with the Monitor at all.
   - `/caches/stats` - This _does_ pull some stats from the Monitor, but the specific ones for which it's looking should be totally unaffected, as they are in relation to the operation of ATS and not the machine's network interfaces.
   - `/cdns/health`/`/cdns/{{name}}/health` - The output of this is generated by tabulating which cache servers are "healthy". The determination of what "healthy" means will change, but not the ability to or structure of doing so. That is, the output value may change for a particular CDN may change for that endpoint, but the endpoint itself need not change.
   - `/deliveryservices/{{ID}}/health` - Same as the CDN-based route.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398727915
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,829 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * If it is an IPv6 address
+	 */
+	address: string;
+
+	/**
+	 * The netmask used by the address.
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * The maximum allowed bandwidth for this interface to be considered "healthy"
+	 * by Traffic Monitor.
+	 *
+	 * This has no effect if `monitor` is not true.
+	 * Values are in kb/s.
+	 * The value `0` means "no limit".
+	 */
+	maxBandwidth: bigint;
 
 Review comment:
   Discover this? This will be set on the object when it's created.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r396793164
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
 
 Review comment:
   I agree, but that's out of scope for this blueprint.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r395316242
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
+
+### Traffic Portal Impact
+The server edit/display/creation page will need to be updated to remove all of
+the properties of servers which no longer exist because they've been moved into
+the "interfaces" section, and a new set of interface-related input fields with
+controls for dynamically adding to them should be added.
+
+A sample screenshot is shown below.
+
+![](img/multi-interface-ui-sample.png "Mockup of proposed interface controls")
+
+If desired, the markup used for that sample may be found in
+[Appendix A](#appendix-a)
+
+Additionally, the properties of servers no longer present on a server object
+will need to be removed from table columns (and `interfaces` should not be
+added).
+
+### Traffic Monitor Impact
+Traffic Monitor will need to update its parsing of `/monitoring` payloads to
+account for the new structure, and will need to poll all listed interfaces for
+health (instead of just one per cache server as was done previously).
+
+The cache
+server should be marked unavailable - or "down" - if and only if the interface
+proving the service addresses is considered unhealthy.
 
 Review comment:
   There might also be stat aggregation changes to consider. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r404391053
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,829 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * If it is an IPv6 address
+	 */
+	address: string;
+
+	/**
+	 * The netmask used by the address.
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
 
 Review comment:
   Because that's a typescript block.
   
   But the root of the matter was I just wasn't sure if he was suggesting that
   ```json
   {
     "address": "0.0.0.0",
     "gateway": null,
     "serviceAddress": false
   }
   ```
   be acceptable or
   ```json
   {
     "address": "0.0.0.0",
     "serviceAddress": false
   }
   ```
   be acceptable. And I prefer the former.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r394588892
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,657 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+
+```
+
+This should be sufficient to capture the model changes.
+
+#### API
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
 
 Review comment:
   just wondering. is `serviceAddress` applicable to all interfaces? for the interface that is included in the TR snapshot (the "active" interface), this has meaning but does it have meaning for the other interfaces? Or is this being used to determine the "active" interface (the one that shows up in the snapshot)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on issue #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on issue #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#issuecomment-600959404
 
 
   > ... changes to TM UI...
   
   None are necessary. The TM UI has no special handling regarding _why_ a server is unavailable, it only knows if it that _is_ the case.
   
   > ... TM cacheStats ...
   
   idk what that is - the API endpoint that fetches it? I hadn't considered the TM API, might wanna look into that.
   
   > ... how TM determines health based on stats of multiple interfaces ...
   
   This may not be desirable, but the current impact description says:
   
   > _"The cache server should be marked unavailable - or 'down' - if and only if the interface proving the service addresses is considered unhealthy."_
   
   So it is addressed, at least.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398730794

Review comment:
A "single shared IP address" is implemented by having a service address that is load-balanced to the other interfaces on the cache server side, and is the primary motivation behind this feature.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r395307201
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
 
 Review comment:
   maybe interfaceName here?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r394759337
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
 
 Review comment:
   snake_case is what we typically use, because PostgreSQL (SQL standard?) case-folds everything you don't mark as an explicit identifier with double-quotes (<kbd>"</kbd>) to lowercase. Which is what happened here. That's just an example name, though, the only thing that's meant to specify is table structure, the column and table names are all arbitrary (provided you know what they're for).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r395305524

##########
File path: blueprints/multi-interface-servers.md
##########
@@ -0,0 +1,666 @@
+
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+ /**
+ * The actual IP (v4 or v6) address which is used by an interface.
+ * Any "mask" should be present here as a CIDR-notation suffix.
+ */
+ address: string;
+
+ /**
+ * The IP (v4 or v6) address of the gateway used by this IP address.
+ */
+ gateway: string;
+
+ /**
+ * Tells whether or not this address of this interface is the server's
+ * "service" address.
+ * At least one address of EXACTLY ONE interface MUST have this set to
+ * 'true' for a server.
+ */
+ serviceAddress: boolean;
+}
+
+interface Interface {

Review comment:
I think we should add a "maximumThroughput" or "maxSpeed" field here which can by used by TM to monitor the maximum bandwidth we want to allow per interface

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [trafficcontrol] jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r402534747
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,829 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * If it is an IPv6 address
+	 */
+	address: string;
+
+	/**
+	 * The netmask used by the address.
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * The maximum allowed bandwidth for this interface to be considered "healthy"
+	 * by Traffic Monitor.
+	 *
+	 * This has no effect if `monitor` is not true.
+	 * Values are in kb/s.
+	 * The value `0` means "no limit".
+	 */
+	maxBandwidth: bigint;
 
 Review comment:
   Even though it could help, I'm not a fan of moving this to TODB because it's making more state and more opportunity for humans to do it wrong.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r397305008
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
 
 Review comment:
   Well then should we be restricting it at all? Maybe just >= 1500? Because currently TP will only let you set 1500 or 9000, which is why I chose those restrictions in the first place, but if it could theoretically be set to anything then maybe TP should just set a reasonable default but not restrict allowable values there, or in the API or database?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r396796617
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
 
 Review comment:
   As far as "purpose" fields... ¯\\\_(ツ)\_/¯
   
   could be useful, but I worry that they'd fill up with mostly junk like "interface eth0". Maybe if it was optional, and definitely with a character limit. Like 60 characters, or something.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] limited commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

limited commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398694430
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,829 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * If it is an IPv6 address
+	 */
+	address: string;
+
+	/**
+	 * The netmask used by the address.
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
 
 Review comment:
   gateway should be optional, not all IPs have an associated gateway. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r404451894
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,829 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * If it is an IPv6 address
+	 */
+	address: string;
+
+	/**
+	 * The netmask used by the address.
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
 
 Review comment:
   Gateways are now allowed to be `null`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r395917328
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
+
+### Traffic Portal Impact
+The server edit/display/creation page will need to be updated to remove all of
+the properties of servers which no longer exist because they've been moved into
+the "interfaces" section, and a new set of interface-related input fields with
+controls for dynamically adding to them should be added.
+
+A sample screenshot is shown below.
+
+![](img/multi-interface-ui-sample.png "Mockup of proposed interface controls")
+
+If desired, the markup used for that sample may be found in
+[Appendix A](#appendix-a)
+
+Additionally, the properties of servers no longer present on a server object
+will need to be removed from table columns (and `interfaces` should not be
+added).
+
+### Traffic Monitor Impact
+Traffic Monitor will need to update its parsing of `/monitoring` payloads to
+account for the new structure, and will need to poll all listed interfaces for
+health (instead of just one per cache server as was done previously).
+
+The cache
+server should be marked unavailable - or "down" - if and only if the interface
+proving the service addresses is considered unhealthy.
+
+## Documentation Impact
+Documentation for the affected endpoints will need to be updated. Beyond that,
+the "Health Protocol" section and perhaps the "Traffic Monitor - Using" section
+should be updated to describe the behavior of multiple monitored interfaces.
+
+## Testing Impact
+A bunch of client/API integration tests should be written to verify the
+constraints outlined in [API](#sec:api).
+
+## Performance Impact
+The proposed method of updating servers to change their interfaces is by
+submitting a PUT request to `/servers/{{ID}}` like normal. Traffic Ops will then
+delete **all** associated interfaces before inserting the ones supplied in the
+PUT payload. This will occur even if the interfaces are not actually changing in
+values. So a single server update now requires a minimum of two extra database
+calls.
+
+This probably isn't a significant performance concern, but at the time of this
+writing no profiling has been done.
+
+## Security Impact
+Setting an improper interface can allow users with read-only permissions to read
+arbitrary files named 'speed' or 'operstate' that the 'ats' user has permission
+to read on the cache server's file system. Of course, this was already possible,
+and improper values can only be entered by users with at least "operator"
+permissions. Now, though, an arbitrary number of these files could be exposed
+per Snapshot, whereas previously it was limited to one file per cache server per
+Snapshot.
+
+Improper configuration has the same result as improper configuration of
+`interfaceName` currently, which is to say it can cause a cache server to be
+marked unavailable.
+
+Multiple new inputs will need to be sanitized, as always.
+
+> _"Will new SQL queries properly use parameter binding?"_
+
+The answer to this question should absolutely never be "no," can't imagine a
+world wherein that would need to change for this - or any other - particular
+feature.
+
+## Upgrade Impact
+There will be at least one, but possibly two, database migrations. The changes
+to Traffic Monitor ought to be made in such a way that it can parse either the
+old `/monitoring` payloads or the new ones. Then Traffic Monitor should be
+upgraded before Traffic Ops, and no additional, special steps need be taken to
+upgrade.
 
 Review comment:
   Right

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r394739703
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
 
 Review comment:
   arbitrarily?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r404428525

Review comment:
So you want to move the concept of the bandwidth limit partially out of profiles/parameters. It feels a bit odd to split the implementation like that, but ok. Instead of fixed caps, could it be either percentage or fixed reservation? That would allow us to fix the loophole with bonded interfaces in various link state mixes down the road.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [trafficcontrol] jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r396678662
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
 
 Review comment:
   Where?  Should there be an purpose field on an interface?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r395079794
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
 
 Review comment:
   gotcha and sorry, i meant snake. got my snakes and camels screwed up. :)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 merged pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 merged pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r396682959
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
 
 Review comment:
   Anyone not using one of the native clients will have to change their client code.  Maybe I'm misreading this though.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398726158

Review comment:
"Optional" means something very specific in TypeScript. It means that the property can be missing from an object, and I'd prefer allowing things to be `null`. Just want to be clear what you're suggesting.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r394693974
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,657 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+
+```
+
+This should be sufficient to capture the model changes.
+
+#### API
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
 
 Review comment:
   From a different point in the blueprint:
   
   > _"A server object should be accepted (based only on its `interfaces` property) if it has at least one interface, every interface has at least one IP address, exactly one interface has at least one IP address that is a service address, and no two IP addresses of the same family/version are simultaneously marked as 'service addresses'."_
   
   So to directly answer your question, only one interface on a server object is allowed to have ipAddresses that are "service addresses". The meaning that field has for other, non-"active" interfaces is that its value being `true` indicates an erroneous configuration, because only the "active" interface has "service addresses", and in fact the "active" interface, as you put it, is defined by having said "service addresses".

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r395314798
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
 
 Review comment:
   Is this true?  We will have struct changes but not code changes?  Are those different?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r394588892
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,657 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+
+```
+
+This should be sufficient to capture the model changes.
+
+#### API
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
 
 Review comment:
   just wondering. is `serviceAddress` applicable to all interfaces? for the interface that is included in the TR snapshot, this has meaning but does it have meaning for the other interfaces?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r396790574
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
+
+### Traffic Portal Impact
+The server edit/display/creation page will need to be updated to remove all of
+the properties of servers which no longer exist because they've been moved into
+the "interfaces" section, and a new set of interface-related input fields with
+controls for dynamically adding to them should be added.
+
+A sample screenshot is shown below.
+
+![](img/multi-interface-ui-sample.png "Mockup of proposed interface controls")
+
+If desired, the markup used for that sample may be found in
+[Appendix A](#appendix-a)
+
+Additionally, the properties of servers no longer present on a server object
+will need to be removed from table columns (and `interfaces` should not be
+added).
+
+### Traffic Monitor Impact
+Traffic Monitor will need to update its parsing of `/monitoring` payloads to
+account for the new structure, and will need to poll all listed interfaces for
+health (instead of just one per cache server as was done previously).
+
+The cache
+server should be marked unavailable - or "down" - if and only if the interface
+proving the service addresses is considered unhealthy.
 
 Review comment:
   So interfaces now have `maxBandwidth` specified, but "stat aggregation changes" remain unaddressed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 commented on issue #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 commented on issue #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#issuecomment-601233207
 
 
   Could you add a blurb under the TP impact regarding the need to update the ISO generation form to handle multiple server interfaces when selecting the "Copy Server Attributes From" drop down?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398239338
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
 
 Review comment:
   But that's implicit in the versioning process? This doesn't contain a proposal to break the versioning promise. If it doesn't make it into 2.0 before release it'll need to go into 3.0 because this is a breaking API change. Clients always need to check what's changed between major releases. A blueprint is a statement of intent to do some work on the project, nothing in it should be directed at nor even directly relevant to third parties. They can just look at the changelog.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r404453115

Review comment:
What loophole?

Also, I think adding support for percentages would entail a different body of work and ought to be considered separately from implementing the multiple-interface servers as outlined in this blueprint.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r395307374
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
 
 Review comment:
   Again we should add maxThroughput or maxSpeed

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r397221072
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
 
 Review comment:
   +100.  Maybe this is a good push to use the client we provide and maintain as part of the project?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r396679725
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
 
 Review comment:
   if we're going to only support two values and null, why use an integer type over an enum or domain (with appropriate check constraint)?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 edited a comment on issue #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 edited a comment on issue #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#issuecomment-604491881
 
 
   Is there any impact to:
   
   https://tp.domain.net/#!/cache-checks and the backing api endpoint(s)
   https://tp.domain.net/#!/cache-stats and the backing api endpoint(s)
   
   And these api endpoints:
   
   GET /cdns/health
   GET /cdns/{name}/health
   GET /deliveryservices/{id}/health

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r404173079

Review comment:
This is optional in the scope of database optionality. I don't know what Typescript has to do with it. In other words, this field should be nullable.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r394759817
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
 
 Review comment:
   It's not. The intention was to convey that a service-address-containing interface may or may not have an ipv4 or ipv6 address, and the mechanism for choosing one isn't specified in that section. But it needs to be worded better.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r395080165
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
+
+### Traffic Portal Impact
+The server edit/display/creation page will need to be updated to remove all of
+the properties of servers which no longer exist because they've been moved into
+the "interfaces" section, and a new set of interface-related input fields with
+controls for dynamically adding to them should be added.
+
+A sample screenshot is shown below.
+
+![](img/multi-interface-ui-sample.png "Mockup of proposed interface controls")
+
+If desired, the markup used for that sample may be found in
+[Appendix A](#appendix-a)
+
+Additionally, the properties of servers no longer present on a server object
 
 Review comment:
   > but you could have one or two of those - can you handle sorting by collection properties in TP tables?
   
   maybe?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r395308605
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
 
 Review comment:
   I think this should be interfaceName

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r396860375
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
 
 Review comment:
   MTU is Maximum Transmission Unit and has to do with the size of Layer 2 Ethernet frames.  1500 is more or less the default, but if you can get 9000 beginning to end you should. https://en.wikipedia.org/wiki/Jumbo_frame
   
   Super jumbo frames technically exist, but I've not run across them except in router configs.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398723283
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,829 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * If it is an IPv6 address
+	 */
+	address: string;
+
+	/**
+	 * The netmask used by the address.
 
 Review comment:
   I don't think so. That comment was left in from an earlier iteration on accident.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r396790042
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
 
 Review comment:
   This section is only about the client code maintained as a part of ATC. Any other clients will need to be aware of API changes across versions and adjust their code accordingly, just like always.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r396794901
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
 
 Review comment:
   It would need to be associated with the username and password information currently on the server object, which is why the decision was made to leave it be, as no other interface would ever use those fields.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r395325091
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
+
+### Traffic Portal Impact
+The server edit/display/creation page will need to be updated to remove all of
+the properties of servers which no longer exist because they've been moved into
+the "interfaces" section, and a new set of interface-related input fields with
+controls for dynamically adding to them should be added.
+
+A sample screenshot is shown below.
+
+![](img/multi-interface-ui-sample.png "Mockup of proposed interface controls")
+
+If desired, the markup used for that sample may be found in
+[Appendix A](#appendix-a)
+
+Additionally, the properties of servers no longer present on a server object
+will need to be removed from table columns (and `interfaces` should not be
+added).
+
+### Traffic Monitor Impact
+Traffic Monitor will need to update its parsing of `/monitoring` payloads to
+account for the new structure, and will need to poll all listed interfaces for
+health (instead of just one per cache server as was done previously).
+
+The cache
+server should be marked unavailable - or "down" - if and only if the interface
+proving the service addresses is considered unhealthy.
+
+## Documentation Impact
+Documentation for the affected endpoints will need to be updated. Beyond that,
+the "Health Protocol" section and perhaps the "Traffic Monitor - Using" section
+should be updated to describe the behavior of multiple monitored interfaces.
+
+## Testing Impact
+A bunch of client/API integration tests should be written to verify the
+constraints outlined in [API](#sec:api).
+
+## Performance Impact
+The proposed method of updating servers to change their interfaces is by
+submitting a PUT request to `/servers/{{ID}}` like normal. Traffic Ops will then
+delete **all** associated interfaces before inserting the ones supplied in the
+PUT payload. This will occur even if the interfaces are not actually changing in
+values. So a single server update now requires a minimum of two extra database
+calls.
+
+This probably isn't a significant performance concern, but at the time of this
+writing no profiling has been done.
+
+## Security Impact
+Setting an improper interface can allow users with read-only permissions to read
+arbitrary files named 'speed' or 'operstate' that the 'ats' user has permission
+to read on the cache server's file system. Of course, this was already possible,
+and improper values can only be entered by users with at least "operator"
+permissions. Now, though, an arbitrary number of these files could be exposed
+per Snapshot, whereas previously it was limited to one file per cache server per
+Snapshot.
+
+Improper configuration has the same result as improper configuration of
+`interfaceName` currently, which is to say it can cause a cache server to be
+marked unavailable.
+
+Multiple new inputs will need to be sanitized, as always.
+
+> _"Will new SQL queries properly use parameter binding?"_
+
+The answer to this question should absolutely never be "no," can't imagine a
+world wherein that would need to change for this - or any other - particular
+feature.
+
+## Upgrade Impact
+There will be at least one, but possibly two, database migrations. The changes
+to Traffic Monitor ought to be made in such a way that it can parse either the
+old `/monitoring` payloads or the new ones. Then Traffic Monitor should be
+upgraded before Traffic Ops, and no additional, special steps need be taken to
+upgrade.
 
 Review comment:
   What's with the deployment order dependency here? Are we anticipating breaking the old interfaces such that we have to deploy TM first?  I usually think that we want to deploy TO first so I want to be sure to understand why that would change.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r396924317
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
 
 Review comment:
   There's no distinction between "public" and "private" interfaces, if that's what your asking

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r396676787
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
 
 Review comment:
   We might consider changing the nomenclature around iLo to OOB as that's an HP specific term for an out-of-band management interface.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r397564971
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
 
 Review comment:
   curl support can't be broken unless we stop doing HTTP, tho

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r402531848
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,829 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * If it is an IPv6 address
+	 */
+	address: string;
+
+	/**
+	 * The netmask used by the address.
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * The maximum allowed bandwidth for this interface to be considered "healthy"
+	 * by Traffic Monitor.
+	 *
+	 * This has no effect if `monitor` is not true.
+	 * Values are in kb/s.
+	 * The value `0` means "no limit".
+	 */
+	maxBandwidth: bigint;
 
 Review comment:
   There is a minor bug #2500 where it's impossible for astats to figure it out, and a gap where link aggregate member statuses aren't taken into account.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] limited commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

limited commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398692587

Review comment:
Having a single shared IP address is one potential solution. I think just as likely would be different IPs per interface. If at all possible, it would be great if this feature could support that as well

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [trafficcontrol] jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r397335664
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
 
 Review comment:
   Disagree, but I guess my only real objection here is wording.  We use the term client when we mean atc supported sdk.  We need to always maintain compatibility with curl so that we can better triage breakage in the API, or support simple use cases in languages not supported by ATC.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] limited commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

limited commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398694595
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,829 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * If it is an IPv6 address
+	 */
+	address: string;
+
+	/**
+	 * The netmask used by the address.
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
 
 Review comment:
   Why restrict it to exactly one serviceAddress per cache? 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r394743538
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
+
+### Traffic Portal Impact
+The server edit/display/creation page will need to be updated to remove all of
+the properties of servers which no longer exist because they've been moved into
+the "interfaces" section, and a new set of interface-related input fields with
+controls for dynamically adding to them should be added.
+
+A sample screenshot is shown below.
+
+![](img/multi-interface-ui-sample.png "Mockup of proposed interface controls")
+
+If desired, the markup used for that sample may be found in
+[Appendix A](#appendix-a)
+
+Additionally, the properties of servers no longer present on a server object
 
 Review comment:
   It might be beneficial however to show server.interface.ipAddress.address where serviceAddress=true however in the servers table for each server. I could see that being a property you might want to see when viewing a collection of servers. Something to think about.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398207409
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
 
 Review comment:
   I'm referring to the promise that the API itself follows the API versioning rules, and not just our native clients (sdk).  Additionally, acknowledgement that unsupported clients leveraging the API directly must change as this will break the API going forward.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r395316492
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
+
+### Traffic Portal Impact
+The server edit/display/creation page will need to be updated to remove all of
+the properties of servers which no longer exist because they've been moved into
+the "interfaces" section, and a new set of interface-related input fields with
+controls for dynamically adding to them should be added.
+
+A sample screenshot is shown below.
+
+![](img/multi-interface-ui-sample.png "Mockup of proposed interface controls")
+
+If desired, the markup used for that sample may be found in
+[Appendix A](#appendix-a)
+
+Additionally, the properties of servers no longer present on a server object
+will need to be removed from table columns (and `interfaces` should not be
+added).
+
+### Traffic Monitor Impact
+Traffic Monitor will need to update its parsing of `/monitoring` payloads to
+account for the new structure, and will need to poll all listed interfaces for
+health (instead of just one per cache server as was done previously).
+
+The cache
+server should be marked unavailable - or "down" - if and only if the interface
+proving the service addresses is considered unhealthy.
+
+## Documentation Impact
+Documentation for the affected endpoints will need to be updated. Beyond that,
+the "Health Protocol" section and perhaps the "Traffic Monitor - Using" section
+should be updated to describe the behavior of multiple monitored interfaces.
+
+## Testing Impact
+A bunch of client/API integration tests should be written to verify the
+constraints outlined in [API](#sec:api).
 
 Review comment:
   In addition to TM traffic monitor integration tests (with caches and TR).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] limited commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

limited commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398694107
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,829 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * If it is an IPv6 address
+	 */
+	address: string;
+
+	/**
+	 * The netmask used by the address.
 
 Review comment:
   is netmask needed?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r395853116
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
+
+### Traffic Portal Impact
+The server edit/display/creation page will need to be updated to remove all of
+the properties of servers which no longer exist because they've been moved into
+the "interfaces" section, and a new set of interface-related input fields with
+controls for dynamically adding to them should be added.
+
+A sample screenshot is shown below.
+
+![](img/multi-interface-ui-sample.png "Mockup of proposed interface controls")
+
+If desired, the markup used for that sample may be found in
+[Appendix A](#appendix-a)
+
+Additionally, the properties of servers no longer present on a server object
+will need to be removed from table columns (and `interfaces` should not be
+added).
+
+### Traffic Monitor Impact
+Traffic Monitor will need to update its parsing of `/monitoring` payloads to
+account for the new structure, and will need to poll all listed interfaces for
+health (instead of just one per cache server as was done previously).
+
+The cache
+server should be marked unavailable - or "down" - if and only if the interface
+proving the service addresses is considered unhealthy.
+
+## Documentation Impact
+Documentation for the affected endpoints will need to be updated. Beyond that,
+the "Health Protocol" section and perhaps the "Traffic Monitor - Using" section
+should be updated to describe the behavior of multiple monitored interfaces.
+
+## Testing Impact
+A bunch of client/API integration tests should be written to verify the
+constraints outlined in [API](#sec:api).
+
+## Performance Impact
+The proposed method of updating servers to change their interfaces is by
+submitting a PUT request to `/servers/{{ID}}` like normal. Traffic Ops will then
+delete **all** associated interfaces before inserting the ones supplied in the
+PUT payload. This will occur even if the interfaces are not actually changing in
+values. So a single server update now requires a minimum of two extra database
+calls.
+
+This probably isn't a significant performance concern, but at the time of this
+writing no profiling has been done.
+
+## Security Impact
+Setting an improper interface can allow users with read-only permissions to read
+arbitrary files named 'speed' or 'operstate' that the 'ats' user has permission
+to read on the cache server's file system. Of course, this was already possible,
+and improper values can only be entered by users with at least "operator"
+permissions. Now, though, an arbitrary number of these files could be exposed
+per Snapshot, whereas previously it was limited to one file per cache server per
+Snapshot.
+
+Improper configuration has the same result as improper configuration of
+`interfaceName` currently, which is to say it can cause a cache server to be
+marked unavailable.
+
+Multiple new inputs will need to be sanitized, as always.
+
+> _"Will new SQL queries properly use parameter binding?"_
+
+The answer to this question should absolutely never be "no," can't imagine a
+world wherein that would need to change for this - or any other - particular
+feature.
+
+## Upgrade Impact
+There will be at least one, but possibly two, database migrations. The changes
+to Traffic Monitor ought to be made in such a way that it can parse either the
+old `/monitoring` payloads or the new ones. Then Traffic Monitor should be
+upgraded before Traffic Ops, and no additional, special steps need be taken to
+upgrade.
 
 Review comment:
   OK, so if we bump the API to 3.0 with this new API then we don't have an issue, right?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r394760292
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
+
+### Traffic Portal Impact
+The server edit/display/creation page will need to be updated to remove all of
+the properties of servers which no longer exist because they've been moved into
+the "interfaces" section, and a new set of interface-related input fields with
+controls for dynamically adding to them should be added.
+
+A sample screenshot is shown below.
+
+![](img/multi-interface-ui-sample.png "Mockup of proposed interface controls")
+
+If desired, the markup used for that sample may be found in
+[Appendix A](#appendix-a)
+
+Additionally, the properties of servers no longer present on a server object
 
 Review comment:
   but you could have one or two of those - can you handle sorting by collection properties in TP tables?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398239518
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
 
 Review comment:
   I went with >1280

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r404977756

Review comment:
Looking into that, it seems like this really ought to just be solved by having the stats system on the Cache Server read the interface's bonded directory and calculating the totals from its slave interfaces. Which seems like a bug with that/those plugin(s) that should be remedied there regardless of how many interfaces exist on cache server objects in TO.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [trafficcontrol] jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r396684665
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,670 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
 
 Review comment:
   This would mean you could also model out-of-band management the same way as mgmtIpAddress etc.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

dneuman64 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r395315924
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
+
+### Traffic Portal Impact
+The server edit/display/creation page will need to be updated to remove all of
+the properties of servers which no longer exist because they've been moved into
+the "interfaces" section, and a new set of interface-related input fields with
+controls for dynamically adding to them should be added.
+
+A sample screenshot is shown below.
+
+![](img/multi-interface-ui-sample.png "Mockup of proposed interface controls")
+
+If desired, the markup used for that sample may be found in
+[Appendix A](#appendix-a)
+
+Additionally, the properties of servers no longer present on a server object
+will need to be removed from table columns (and `interfaces` should not be
+added).
+
+### Traffic Monitor Impact
+Traffic Monitor will need to update its parsing of `/monitoring` payloads to
+account for the new structure, and will need to poll all listed interfaces for
+health (instead of just one per cache server as was done previously).
+
+The cache
+server should be marked unavailable - or "down" - if and only if the interface
+proving the service addresses is considered unhealthy.
 
 Review comment:
   This is not true.  We can mark a cache as unavailable if our interfaces go over their defined thresholds (for connections, bandwidth, etc).  This could be any of the monitored interfaces or the service interface

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

jhg03a commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r402528794

Review comment:
That number hasn't been present in the database before as it's derived by the astats plugin to ATS based on which interface name you ask for.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

With regards,
Apache Git Services

[GitHub] [trafficcontrol] mitchell852 edited a comment on issue #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

mitchell852 edited a comment on issue #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#issuecomment-604491881
 
 
   Is there any impact to:
   
   https://tp.domain.net/#!/cache-checks and the backing api endpoint - GET /servercheck
   https://tp.domain.net/#!/cache-stats and the backing api endpoint - GET /caches/stats
   
   And these api endpoints:
   
   GET /cdns/health
   GET /cdns/{name}/health
   GET /deliveryservices/{id}/health

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

ocket8888 commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r395375823
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,666 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * Any "mask" should be present here as a CIDR-notation suffix.
+	 */
+	address: string;
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * The name of the interface device on the server e.g. eth0.
+	 */
+	name: string;
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * Whether or not Traffic Monitor should monitor this particular interface.
+	 */
+	monitor: boolean;
+	/**
+	 * The interface's Maximum Transmission Unit.
+	 * If this is 'null' it is assumed that the interface's MTU is not known/is
+	 * irrelevant.
+	 */
+	mtu: 1500 | 9000 | null;
+}
+```
+We don't have a real data model for ATC, so what follows is an approximately
+defined representation of a "server" that is valid within certain contexts. The
+modifications made to it for the purposes of the changes herein proposed should
+be considered valid in those same contexts - whatever they may be.
+
+```typescript
+interface Server {
+
+	cachegroup?:  string | null;
+	cachegroupId: number;
+	cdnId:        number;
+	cdnName?:     string | null;
+	domainName:   string;
+	guid?:        string | null;
+	hostName:     string;
+	httpsPort?:   number | null;
+	id?:          number;
+
+	// ILO things aren't being moved in because they also require a
+	// username/password that aren't used by any other interface type.
+	iloIpAddress?: string | null;
+	iloIpGateway?: string | null;
+	iloIpNetmask?: string | null;
+	iloPassword?:  string | null;
+	iloUsername?:  string | null;
+
+	/**
+	 * New field containing additional interfaces.
+	 * It is illegal for a server to not have at least one interface.
+	 */
+	interfaces: Array<Interface> & {0: Interface};
+	lastUpdated?:  string;
+
+	// notice not here: mgmtIpAddress, mgmtIpGateway, mgmtIpNetmask
+	// these will be moved into the interfaces property
+
+	offlineReason:   string;
+	physLocation?:   string | null;
+	physLocationId:  number;
+	profile?:        string | null;
+	profileDesc?:    string | null;
+	profileId:       number;
+	revalPending?:   boolean;
+	rack?:           string | null;
+	routerHostName?: string | null;
+	routerPortName?: string | null;
+	status?:         string | null;
+	statusId:        number;
+	tcpPort:         number;
+	type?:           string | null;
+	typeId:          number;
+	updPending?:     boolean;
+	xmppId?:         string | null;
+	xmppPassword?:   string | null;
+}
+```
+
+`/monitoring` payloads and CDN Snapshots are purposely un-versioned and
+un-modeled, so this section will make no attempts to define the model changes to
+those objects.
+
+## Impacted Components
+The only components that should need to be modified to accommodate this feature
+are Traffic Ops, Traffic Portal, and Traffic Monitor.
+
+### Traffic Ops Impact
+
+#### Database
+A new table will need to be introduced to contain interface information:
+`interface`, which is described below.
+```text
+              Table "public.interface"
+ Column  |  Type   | Collation | Nullable | Default
+---------+---------+-----------+----------+---------
+ name    | text    |           | not null |
+ server  | bigint  |           | not null |
+ monitor | boolean |           | not null |
+ mtu     | integer |           |          | 1500
+Indexes:
+    "interface_pkey" PRIMARY KEY, btree (name, server)
+Foreign-key constraints:
+    "interface_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+Another new table will need to be created to contain address information, which
+is described below.
+```text
+                 Table "public.ip_address"
+     Column     |  Type   | Collation | Nullable | Default
+----------------+---------+-----------+----------+---------
+ address        | inet    |           | not null |
+ gateway        | inet    |           | not null |
+ serviceaddress | boolean |           | not null |
+ server         | bigint  |           | not null |
+ interface      | text    |           | not null |
+Indexes:
+    "ip_address_pkey" PRIMARY KEY, btree (address, interface, server)
+Check constraints:
+    "ip_address_gateway_check" CHECK (masklen(gateway) = 0)
+Foreign-key constraints:
+    "ip_address_interface_fkey" FOREIGN KEY (interface, server) REFERENCES interface(name, server)
+    "ip_address_server_fkey" FOREIGN KEY (server) REFERENCES server(id)
+```
+
+This should be sufficient to capture the model changes. In addition to creating
+these tables, a migration will need to be written to convert currently stored
+server information to utilize these new tables.
+
+#### API
+<a name="sec:api"></a>
+The affected endpoints will be:
+
+##### `/cdns/{{name}}/configs/monitoring`
+The current payload includes a `trafficServers` property that contains an array
+of cache server representations to be monitored by a Traffic Monitor within the
+path-specified CDN. An example of one such array member is given below.
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfacename": "eth0",
+	"ip6": "dead::babe",
+	"ip": "192.168.0.0",
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+This will need to change to reflect the new set of interfaces, which for the
+above example with no additional interfaces would look like:
+```json
+{
+	"cacheGroup": "Ciab_Edge",
+	"fqdn": "edge.infra.ciab.test",
+	"hashId": "edge",
+	"hostName": "edge",
+	"interfaces": [
+		{
+			"name": "eth0",
+			"ipAddresses": [
+				{
+					"address": "192.168.0.0/32",
+					"serviceAddress": true
+				},
+				{
+					"address": "dead::babe/128",
+					"serviceAddress": true
+				}
+			]
+		}
+	],
+	"port": 80,
+	"profile": "ATS_EDGE",
+	"status": "REPORTED",
+	"type": "EDGE"
+}
+```
+Note that only interfaces that have "monitor" set to `true` should appear at all
+in the payloads of this endpoint. Further note that neither gateways nor
+<abbr title="Maximum Transmission Unit">MTU</abbr> are reported.
+
+##### `/cdns/{{name}}/snapshot`
+This is a semantic change only - payloads remain structurally unchanged. The
+reported `ip`, `ip6` and `interfaceName` fields for elements of the
+`contentServers` array property should be an arbitrarily chosen IPv4 service
+address, an arbitrarily chosen IPv6 service address, and the interface name of
+that/those chosen address(es).
+
+##### `/cdns/{{name}}/snapshot/new`
+This is a semantic change only - payloads remain structurally unchanged. The
+changes to this endpoint are the same as those described for
+`/cdns/{{name}}/snapshot` above.
+
+##### `/servers`/`/servers/{{ID}}`
+An example of one element of an output array for this endpoint at the time of
+this writing is shown below.
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaceMtu": 1500,
+	"interfaceName": "eth0",
+	"ip6Address": "fc01:9400:1000:8::120",
+	"ip6Gateway": "fc01:9400:1000:8::1",
+	"ipAddress": "172.16.239.120",
+	"ipGateway": "172.16.239.1",
+	"ipNetmask": "255.255.255.0",
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"mgmtIpAddress": "",
+	"mgmtIpGateway": "",
+	"mgmtIpNetmask": "",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+	"ipIsService": true,
+	"ip6IsService": true
+}
+```
+After this feature's inclusion, the same element would look like this:
+```json
+{
+	"cachegroup": "CDN_in_a_Box_Mid",
+	"cachegroupId": 6,
+	"cdnId": 2,
+	"cdnName": "CDN-in-a-Box",
+	"domainName": "infra.ciab.test",
+	"guid": null,
+	"hostName": "mid",
+	"httpsPort": 443,
+	"id": 10,
+	"iloIpAddress": "",
+	"iloIpGateway": "",
+	"iloIpNetmask": "",
+	"iloPassword": "",
+	"iloUsername": "",
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "172.16.239.120/24",
+					"gateway": "172.16.239.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "fc01:9400:1000:8::120/128",
+					"gateway": "fc01:9400:1000:8::1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	],
+	"lastUpdated": "2018-12-05 18:45:05+00",
+	"offlineReason": "",
+	"physLocation": "Apachecon North America 2018",
+	"physLocationId": 1,
+	"profile": "ATS_MID_TIER_CACHE",
+	"profileDesc": "Mid Cache - Apache Traffic Server",
+	"profileId": 10,
+	"rack": "",
+	"revalPending": false,
+	"routerHostName": "",
+	"routerPortName": "",
+	"status": "REPORTED",
+	"statusId": 3,
+	"tcpPort": 80,
+	"type": "MID",
+	"typeId": 12,
+	"updPending": false,
+	"xmppId": "mid",
+	"xmppPasswd": "",
+}
+```
+Request payloads for `PUT` and `POST` methods will need to change similarly. A
+server object should be accepted (based only on its `interfaces` property) if
+it has at least one interface, every interface has at least one IP address,
+exactly one interface has at least one IP address that is a service address, and
+no two IP addresses of the same family/version are simultaneously marked as
+'service addresses'. And, of course, the property's values must be syntactically
+and semantically valid.
+
+Below are some examples of invalid `interfaces` property arrays, to which
+Traffic Ops MUST respond with a `400 Bad Request` and an appropriate Alert:
+
+Empty array
+```json
+{"interfaces": []}
+```
+Null property
+```json
+{"interfaces": null}
+```
+Missing property
+```json
+{}
+```
+Empty `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Null `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": null,
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Missing `ipAddresses` sub-property
+```json
+{
+	"interfaces": [
+		{
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+No service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": false
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+Too many interfaces with service addresses
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		},
+		{
+			"ipAddresses": [
+				{
+					"address": "f1d0::f00d/128",
+					"gateway": "dead::beef",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "wnlp0"
+		}
+	]
+}
+```
+Too many service addresses of the same family/version
+```json
+{
+	"interfaces": [
+		{
+			"ipAddresses": [
+				{
+					"address": "0.0.0.1/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				},
+				{
+					"address": "0.0.0.2/32",
+					"gateway": "10.0.0.1",
+					"serviceAddress": true
+				}
+			],
+			"monitor": true,
+			"mtu": 1500,
+			"name": "eth0"
+		}
+	]
+}
+```
+##### `/servers/details`
+This endpoint is subject to the same changes described above for
+`/servers`/`/servers/{{ID}}` - though obviously the full output objects will
+differ.
+
+#### Client Impact
+The structures output by various clients will change, but no client code changes
+should be necessary.
 
 Review comment:
   ish. The actual code in `traffic_ops/client` won't need to change. The structs all live under `/lib` so they can be shared between components.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [trafficcontrol] limited commented on a change in pull request #4521: Multi-interface Servers blueprint

Posted by GitBox <gi...@apache.org>.

limited commented on a change in pull request #4521: Multi-interface Servers blueprint
URL: https://github.com/apache/trafficcontrol/pull/4521#discussion_r398694914
 
 

 ##########
 File path: blueprints/multi-interface-servers.md
 ##########
 @@ -0,0 +1,829 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# Additional Server Interfaces
+
+## Problem Description
+Multiple network interfaces may be used by cache servers to service requests.
+However, these cannot be expressed as parts of the current "server" object.
+Specifically, assuming each interface has its own IP address, then each would
+need to be treated as a separate cache server. This means that getting clients
+to use one when the other is unavailable requires manual intervention.
+
+## Proposed Change
+Instead, cache servers should be configurable with numerous network interfaces
+monitored by Traffic Monitor that are all associated with the server, each
+having its own associated network information e.g. IP Address, but Traffic
+Router should only be aware of the "main" interface on which the cache server
+listens for connections.
+
+The specific use-case outlined above, obviously, entails some management be done
+on the cache servers to actually support multiple interfaces answering to the
+same "main" IP address and this should be considered outside of
+<abbr title="Apache Traffic Control">ATC</abbr>'s scope of operation.
+
+## Data Model Impact
+The proposed structure of each of these interfaces is given below as a pair of
+TypeScript<sup>[1](#typescript)</sup> interfaces:
+```typescript
+interface IPAddress {
+	/**
+	 * The actual IP (v4 or v6) address which is used by an interface.
+	 * If it is an IPv6 address
+	 */
+	address: string;
+
+	/**
+	 * The netmask used by the address.
+
+	/**
+	 * The IP (v4 or v6) address of the gateway used by this IP address.
+	 */
+	gateway: string;
+
+	/**
+	 * Tells whether or not this address of this interface is the server's
+	 * "service" address.
+	 * At least one address of EXACTLY ONE interface MUST have this set to
+	 * 'true' for a server.
+	 */
+	serviceAddress: boolean;
+}
+
+interface Interface {
+	/**
+	 * These will be all of the IPv4/IPv6 addresses assigned to the interface,
+	 * including gateways and "masks".
+	 * It is illegal for an interface to not have at least one associated IP
+	 * address.
+	 */
+	ipAddresses: Array<IPAddress> & {0: IPAddress};
+	/**
+	 * The maximum allowed bandwidth for this interface to be considered "healthy"
+	 * by Traffic Monitor.
+	 *
+	 * This has no effect if `monitor` is not true.
+	 * Values are in kb/s.
+	 * The value `0` means "no limit".
+	 */
+	maxBandwidth: bigint;
 
 Review comment:
   Will we continue to discover this from the interface stats?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services