You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficcontrol.apache.org by "Fieck, Brennan" <Br...@comcast.com> on 2018/12/17 17:52:07 UTC
Origins assigned to Multipe Delivery Services produces indeterminate
parent.config
As some of you may be aware, `parent.config` files generated by Traffic Ops can vary wildly when an origin is assigned to multiple Delivery Services. This results in undefined behavior. I'm told that the conflict only happens when two Delivery Services with different "topology requirements" use the same origin, whatever that means (content routing type?). Regardless, the issue should be addressed. The obvious solution is to put in place a database constraint that prevents an origin from being assigned to more that one Delivery Service with API checks in place that would provide helpful error messages when an attempt is made to violate the constraint. However, would that mess with things like Multi-Site Origin? Or is it just not viable for some other reason? If it is a good solution, I'm prepared to work on a fix that utilizes it.
Re: [EXTERNAL] Origins assigned to Multipe Delivery Services produces
indeterminate parent.config
Posted by "Gelinas, Derek" <De...@comcast.com>.
It should not affect multi site origin. Those origins have to be created individually as servers in the server table, and are used as parents not origin fields. The origin field is still set in the delivery service itself. As long as the origin fqdn is unique it should not affect MSO.
> On Dec 17, 2018, at 12:59 PM, Fieck, Brennan <Br...@comcast.com> wrote:
>
> As some of you may be aware, `parent.config` files generated by Traffic Ops can vary wildly when an origin is assigned to multiple Delivery Services. This results in undefined behavior. I'm told that the conflict only happens when two Delivery Services with different "topology requirements" use the same origin, whatever that means (content routing type?). Regardless, the issue should be addressed. The obvious solution is to put in place a database constraint that prevents an origin from being assigned to more that one Delivery Service with API checks in place that would provide helpful error messages when an attempt is made to violate the constraint. However, would that mess with things like Multi-Site Origin? Or is it just not viable for some other reason? If it is a good solution, I'm prepared to work on a fix that utilizes it.
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Jeremy Mitchell <mi...@gmail.com>.
thanks for the example, rob.
On Tue, Dec 18, 2018 at 1:10 PM Robert Butts <ro...@apache.org> wrote:
> Here's a SQL query, to find duplicate origins on different delivery
> services:
>
> ```
> WITH duplicate_origins as (
> SELECT fqdn FROM origin
> where is_primary
> GROUP BY fqdn
> HAVING COUNT(*) > 1
> )
> SELECT o.fqdn, ds.xml_id AS ds_name
> FROM origin o
> JOIN duplicate_origins du on du.fqdn = o.fqdn
> JOIN deliveryservice ds ON ds.id = o.deliveryservice
> WHERE o.is_primary
> ORDER BY fqdn;
> ```
>
>
> On Tue, Dec 18, 2018 at 1:09 PM Robert Butts <ro...@apache.org> wrote:
>
> > >can you give an example of what parent.config looks like when 2 ds's
> > share an origin and have different a different topology?
> >
> > Answering because I encountered this directly, when rewriting
> > parent.config.
> >
> > For example: Suppose you have one Delivery Service:
> > XML_ID: foo
> > Type: HTPT_LIVE_NATL
> > Query String Handling: 1 - ignore in cache key, and pass up
> > Origin Server Base URL: http://foo.example.net
> >
> > And another Delivery Service:
> > XML_ID: bar
> > Type: HTPT_LIVE_NATL
> > Query String Handling: 1 - ignore in cache key, and pass up
> > Origin Server Base URL: http://foo.example.net
> >
> > ATS only supports unique `dest_domain` entries in parent.config.
> > Therefore, the parent.config generated for a server assigned to both of
> > these Delivery Services with either be:
> >
> > dest_domain=foo.example.net port=80 go_direct=true
> >
> > Or
> >
> > dest_domain=foo.example.net port=80 go_direct=false qstring=consider
> >
> > Right now, it's arbitrary which Perl Traffic Ops inserts, and Perl
> > provides no warning or error of any kind (the pending Go parent.config PR
> > logs an error).
> >
> > Whichever is arbitrarily inserted, the resulting remaps for the other
> > delivery service will be wrong. Either `foo` requests will drop the query
> > string when they shouldn't, and go to the mid when they shouldn't; or
> `bar`
> > requests will use the query string and skip the mid when it shouldn't.
> >
> >
> > Does that make sense? The only correct solution, is to somehow prevent
> > different DSes having the same origin, and tell tenants they must use
> > CNAMEs if they need.
> >
> > This isn't a bug in Traffic Control. ATS fundamentally doesn't support
> > multiple remap rules with the same parent FQDN with different
> > configurations. Hence, Traffic Control needs to prohibit that.
> >
> >
> > On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <mi...@gmail.com>
> > wrote:
> >
> >> brennan,
> >>
> >> can you give an example of what parent.config looks like when 2 ds's
> share
> >> an origin and have different a different topology?
> >>
> >> jeremy
> >>
> >> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
> >> Brennan_Fieck@comcast.com>
> >> wrote:
> >>
> >> > To be clear, the "Warning" I'm talking about would happen at startup,
> >> but
> >> > I'd like a UI-only constraint to come with that to disallow using the
> >> API
> >> > to bind the same origin to multiple Delivery Services with varying
> >> > topography requirements. It wouldn't change the existing data, but
> >> prevent
> >> > users from creating more bad data.
> >> >
> >> > "warning" doesn't really sufficiently describe that, my bad.
> >> > ________________________________________
> >> > From: Fieck, Brennan <Br...@comcast.com>
> >> > Sent: Tuesday, December 18, 2018 11:24 AM
> >> > To: dev@trafficcontrol.apache.org
> >> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> >> Services
> >> > produces indeterminate parent.config
> >> >
> >> > Well the cost of fixing this bug is a constraint on the data. Unless
> we
> >> > make it a UI-only constraint - which I'm personally against - there
> >> must be
> >> > some point in the future where ATC cannot reasonably be expected to
> work
> >> > with data that violates that constraint. The question is when that
> >> should
> >> > occur, which should likely happen at a minor version release. Minor
> not
> >> > major because it doesn't involve a change in data structures, merely
> >> > relationships between them - in my opinion that's a minor version
> change
> >> > but that's definitely up for debate. With several release candidates
> for
> >> > 3.0.0 that _doesn't_ include this restriction already in the wild, I
> >> > wouldn't recommend putting it in there. That means to fix the bug as
> >> soon
> >> > as possible it should go in 3.1.0 which should be the target of
> "master"
> >> > after the 3.0.0 release is cut from it.
> >> >
> >> > So I'd recommend immediately implementing the constraint in master
> with
> >> a
> >> > refusal to upgrade with bad data, and backport a warning about the
> >> future
> >> > behavior into 3.0.0 or as part of a 3.0.1 provided we had more changes
> >> that
> >> > would warrant a micro version bump.
> >> > ________________________________________
> >> > From: Gray, Jonathan <Jo...@comcast.com>
> >> > Sent: Tuesday, December 18, 2018 9:34 AM
> >> > To: dev@trafficcontrol.apache.org
> >> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> >> Services
> >> > produces indeterminate parent.config
> >> >
> >> > -1 Holding an ATC upgrade hostage to data cleanup seems like a bad
> idea.
> >> > The issue isn't great, but it's also not new. We should allow teams
> to
> >> fix
> >> > their data at their normal paces if it doesn't create significant
> >> overhead
> >> > or an inherant blocker for new functionality or correction of other
> >> major
> >> > problems imho.
> >> >
> >> > Jonathan G
> >> >
> >> >
> >> > On 12/18/18, 9:28 AM, "Fieck, Brennan" <Br...@comcast.com>
> >> wrote:
> >> >
> >> > Another option is we could detect collisions at startup and simply
> >> > refuse to continue with the upgrade until the data is fixed. That
> would
> >> > allow people using the now-unsupported data format to continue to use
> >> their
> >> > old versions of Traffic Ops without wrecking their database, but also
> >> > provide an incentive to clean up the data.
> >> > ________________________________________
> >> > From: Gray, Jonathan <Jo...@comcast.com>
> >> > Sent: Tuesday, December 18, 2018 5:12 AM
> >> > To: dev@trafficcontrol.apache.org
> >> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> >> > Services produces indeterminate parent.config
> >> >
> >> > I'm generally a fan of constrain your data in your database, but
> not
> >> > necessarily exclusively. I see this as a one-way cleanup/conversion
> so
> >> it
> >> > doesn't need to be configurable; otherwise you have to ask the
> question
> >> > what happens if someone turns it off. That said, something in the UI
> >> layer
> >> > would be nice to prevent spending significant quantities of time
> >> building a
> >> > complex DS only to have it fail to post for reasons that could have
> been
> >> > known earlier.
> >> >
> >> > The way my brain works in this case:
> >> > If !unique_constraint_exists_query()
> >> > If has_duplicates_query()
> >> > show_warning()
> >> > else
> >> > add_unique_constraint()
> >> >
> >> > to which the API and UI configuration could also make use of
> >> > unique_constraint_exists_query() to drive additional layer constraints
> >> if
> >> > desired.
> >> >
> >> > Jonathan G
> >> >
> >> > On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com>
> >> wrote:
> >> >
> >> > That is an interesting idea...detect at TO startup whether or
> >> not
> >> > there are duplicate origins and operate in a "prevent
> duplicate
> >> > origins" state if no duplicates are found or "prevent
> >> conflicting
> >> > DS
> >> > topologies" state if duplicates are found? So once operators
> >> have
> >> > replaced all the duplicate origins with CNAMEs, TO will
> >> essentially
> >> > operate in a "prohibit all duplicate origins" state. That
> would
> >> > probably make for a simpler transition, but I'd want to remove
> >> that
> >> > logic in a following release that strictly prohibits duplicate
> >> > origins
> >> > (assuming that the community agrees we should prohibit
> duplicate
> >> > origins altogether).
> >> >
> >> > As for DB constraints vs UI, I was thinking those DS-type
> >> > constraints
> >> > I pointed out would live in the API. It would basically be
> added
> >> > validation in the deliveryservices POST/PUT endpoint that
> checks
> >> > the
> >> > DB for existing DSes that conflict with the requested DS.
> >> >
> >> > - Rawlin
> >> >
> >> > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> >> > <Jo...@comcast.com> wrote:
> >> > >
> >> > > These kinds of conditions should be detectable with a
> >> > sufficiently advanced SQL query. Is it possible to add the constraint
> >> if
> >> > it passes and emit a warning during TO startup otherwise? That would
> >> let
> >> > you know the condition exists at startup but not getting in your way
> and
> >> > keep you out of trouble once you've cleaned up. We made a mistake
> early
> >> > on, but this would acknowledge it was bad and encourage it to be fixed
> >> at
> >> > the speed of operations teams. Also this puts the constraint in the
> >> > database rather than the UI which is really where the contention is
> for
> >> > usability.
> >> > >
> >> > > Jonathan G
> >> > >
> >> > >
> >> > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> >> rawlin.peters@gmail.com>
> >> > wrote:
> >> > >
> >> > > We occasionally discuss this issue but haven't tackled
> it
> >> > yet. I think
> >> > > the main issue is just that duplicate origins have been
> >> > allowed since
> >> > > the beginning, and now everyone's Traffic Ops could be
> >> > littered with
> >> > > duplicate origins. Also, depending on the config of the
> >> > duplicate
> >> > > delivery services, the origins might not be in conflict
> at
> >> > all (if
> >> > > they don't have different topology constraints). I would
> >> > love for us
> >> > > to just add a uniqueness constraint, but there would
> need
> >> to
> >> > be a fair
> >> > > amount of warning to the community before doing so and
> >> might
> >> > > invalidate a significant amount of valid use cases.
> >> > Operators would
> >> > > need time to make DNS CNAME records for the duplicate
> >> > origins and
> >> > > update their DSes to use the different CNAMEs.
> >> > >
> >> > > I think as a good first step to eliminating the use of
> >> > duplicate
> >> > > origins altogether, we should identify which "topology
> >> > constraints"
> >> > > actually cause conflicting config when used with
> duplicate
> >> > origins and
> >> > > prevent creating DSes with duplicate origins _if it
> would
> >> > cause a
> >> > > conflict with an existing DS that uses the same origin_.
> >> > >
> >> > > For instance, I believe an HTTP and DNS-type DS can live
> >> > happily
> >> > > side-by-side using the same origin (probably need
> >> different
> >> > > routing_names?), but scenarios like HTTP and HTTP_LIVE,
> or
> >> > DNS and
> >> > > HTTP_NO_CACHE sharing the same origin will cause
> conflicts
> >> > for sure.
> >> > > So maybe we can start by making sure the DS types
> "match"
> >> > when using
> >> > > the same origin:
> >> > > HTTP + DNS: possibly good, if they have different
> routing
> >> > names?
> >> > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> >> > > HTTP_NO_CACHE + [any other type]: bad
> >> > > HTTP_LIVE + HTTP: bad
> >> > > etc.
> >> > >
> >> > > There are most likely other conflict scenarios that
> don't
> >> > involve the
> >> > > DS types, but I think this would be a good start. In the
> >> > future with
> >> > > Delivery Service Topologies (aka Flexible Cachegroups
> aka
> >> > Bring Your
> >> > > Own Topology), we might be able to prohibit assigning a
> DS
> >> > to a
> >> > > Topology if the DS's origin is already used by another
> DS
> >> in
> >> > a
> >> > > different Topology.
> >> > >
> >> > > - Rawlin
> >> > >
> >> > > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> >> > > <Br...@comcast.com> wrote:
> >> > > >
> >> > > > As some of you may be aware, `parent.config` files
> >> > generated by Traffic Ops can vary wildly when an origin is assigned to
> >> > multiple Delivery Services. This results in undefined behavior. I'm
> told
> >> > that the conflict only happens when two Delivery Services with
> different
> >> > "topology requirements" use the same origin, whatever that means
> >> (content
> >> > routing type?). Regardless, the issue should be addressed. The obvious
> >> > solution is to put in place a database constraint that prevents an
> >> origin
> >> > from being assigned to more that one Delivery Service with API checks
> in
> >> > place that would provide helpful error messages when an attempt is
> made
> >> to
> >> > violate the constraint. However, would that mess with things like
> >> > Multi-Site Origin? Or is it just not viable for some other reason? If
> >> it is
> >> > a good solution, I'm prepared to work on a fix that utilizes it.
> >> > >
> >> > >
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
> >
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Robert Butts <ro...@apache.org>.
Here's a SQL query, to find duplicate origins on different delivery
services:
```
WITH duplicate_origins as (
SELECT fqdn FROM origin
where is_primary
GROUP BY fqdn
HAVING COUNT(*) > 1
)
SELECT o.fqdn, ds.xml_id AS ds_name
FROM origin o
JOIN duplicate_origins du on du.fqdn = o.fqdn
JOIN deliveryservice ds ON ds.id = o.deliveryservice
WHERE o.is_primary
ORDER BY fqdn;
```
On Tue, Dec 18, 2018 at 1:09 PM Robert Butts <ro...@apache.org> wrote:
> >can you give an example of what parent.config looks like when 2 ds's
> share an origin and have different a different topology?
>
> Answering because I encountered this directly, when rewriting
> parent.config.
>
> For example: Suppose you have one Delivery Service:
> XML_ID: foo
> Type: HTPT_LIVE_NATL
> Query String Handling: 1 - ignore in cache key, and pass up
> Origin Server Base URL: http://foo.example.net
>
> And another Delivery Service:
> XML_ID: bar
> Type: HTPT_LIVE_NATL
> Query String Handling: 1 - ignore in cache key, and pass up
> Origin Server Base URL: http://foo.example.net
>
> ATS only supports unique `dest_domain` entries in parent.config.
> Therefore, the parent.config generated for a server assigned to both of
> these Delivery Services with either be:
>
> dest_domain=foo.example.net port=80 go_direct=true
>
> Or
>
> dest_domain=foo.example.net port=80 go_direct=false qstring=consider
>
> Right now, it's arbitrary which Perl Traffic Ops inserts, and Perl
> provides no warning or error of any kind (the pending Go parent.config PR
> logs an error).
>
> Whichever is arbitrarily inserted, the resulting remaps for the other
> delivery service will be wrong. Either `foo` requests will drop the query
> string when they shouldn't, and go to the mid when they shouldn't; or `bar`
> requests will use the query string and skip the mid when it shouldn't.
>
>
> Does that make sense? The only correct solution, is to somehow prevent
> different DSes having the same origin, and tell tenants they must use
> CNAMEs if they need.
>
> This isn't a bug in Traffic Control. ATS fundamentally doesn't support
> multiple remap rules with the same parent FQDN with different
> configurations. Hence, Traffic Control needs to prohibit that.
>
>
> On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
>> brennan,
>>
>> can you give an example of what parent.config looks like when 2 ds's share
>> an origin and have different a different topology?
>>
>> jeremy
>>
>> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
>> Brennan_Fieck@comcast.com>
>> wrote:
>>
>> > To be clear, the "Warning" I'm talking about would happen at startup,
>> but
>> > I'd like a UI-only constraint to come with that to disallow using the
>> API
>> > to bind the same origin to multiple Delivery Services with varying
>> > topography requirements. It wouldn't change the existing data, but
>> prevent
>> > users from creating more bad data.
>> >
>> > "warning" doesn't really sufficiently describe that, my bad.
>> > ________________________________________
>> > From: Fieck, Brennan <Br...@comcast.com>
>> > Sent: Tuesday, December 18, 2018 11:24 AM
>> > To: dev@trafficcontrol.apache.org
>> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>> Services
>> > produces indeterminate parent.config
>> >
>> > Well the cost of fixing this bug is a constraint on the data. Unless we
>> > make it a UI-only constraint - which I'm personally against - there
>> must be
>> > some point in the future where ATC cannot reasonably be expected to work
>> > with data that violates that constraint. The question is when that
>> should
>> > occur, which should likely happen at a minor version release. Minor not
>> > major because it doesn't involve a change in data structures, merely
>> > relationships between them - in my opinion that's a minor version change
>> > but that's definitely up for debate. With several release candidates for
>> > 3.0.0 that _doesn't_ include this restriction already in the wild, I
>> > wouldn't recommend putting it in there. That means to fix the bug as
>> soon
>> > as possible it should go in 3.1.0 which should be the target of "master"
>> > after the 3.0.0 release is cut from it.
>> >
>> > So I'd recommend immediately implementing the constraint in master with
>> a
>> > refusal to upgrade with bad data, and backport a warning about the
>> future
>> > behavior into 3.0.0 or as part of a 3.0.1 provided we had more changes
>> that
>> > would warrant a micro version bump.
>> > ________________________________________
>> > From: Gray, Jonathan <Jo...@comcast.com>
>> > Sent: Tuesday, December 18, 2018 9:34 AM
>> > To: dev@trafficcontrol.apache.org
>> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>> Services
>> > produces indeterminate parent.config
>> >
>> > -1 Holding an ATC upgrade hostage to data cleanup seems like a bad idea.
>> > The issue isn't great, but it's also not new. We should allow teams to
>> fix
>> > their data at their normal paces if it doesn't create significant
>> overhead
>> > or an inherant blocker for new functionality or correction of other
>> major
>> > problems imho.
>> >
>> > Jonathan G
>> >
>> >
>> > On 12/18/18, 9:28 AM, "Fieck, Brennan" <Br...@comcast.com>
>> wrote:
>> >
>> > Another option is we could detect collisions at startup and simply
>> > refuse to continue with the upgrade until the data is fixed. That would
>> > allow people using the now-unsupported data format to continue to use
>> their
>> > old versions of Traffic Ops without wrecking their database, but also
>> > provide an incentive to clean up the data.
>> > ________________________________________
>> > From: Gray, Jonathan <Jo...@comcast.com>
>> > Sent: Tuesday, December 18, 2018 5:12 AM
>> > To: dev@trafficcontrol.apache.org
>> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>> > Services produces indeterminate parent.config
>> >
>> > I'm generally a fan of constrain your data in your database, but not
>> > necessarily exclusively. I see this as a one-way cleanup/conversion so
>> it
>> > doesn't need to be configurable; otherwise you have to ask the question
>> > what happens if someone turns it off. That said, something in the UI
>> layer
>> > would be nice to prevent spending significant quantities of time
>> building a
>> > complex DS only to have it fail to post for reasons that could have been
>> > known earlier.
>> >
>> > The way my brain works in this case:
>> > If !unique_constraint_exists_query()
>> > If has_duplicates_query()
>> > show_warning()
>> > else
>> > add_unique_constraint()
>> >
>> > to which the API and UI configuration could also make use of
>> > unique_constraint_exists_query() to drive additional layer constraints
>> if
>> > desired.
>> >
>> > Jonathan G
>> >
>> > On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com>
>> wrote:
>> >
>> > That is an interesting idea...detect at TO startup whether or
>> not
>> > there are duplicate origins and operate in a "prevent duplicate
>> > origins" state if no duplicates are found or "prevent
>> conflicting
>> > DS
>> > topologies" state if duplicates are found? So once operators
>> have
>> > replaced all the duplicate origins with CNAMEs, TO will
>> essentially
>> > operate in a "prohibit all duplicate origins" state. That would
>> > probably make for a simpler transition, but I'd want to remove
>> that
>> > logic in a following release that strictly prohibits duplicate
>> > origins
>> > (assuming that the community agrees we should prohibit duplicate
>> > origins altogether).
>> >
>> > As for DB constraints vs UI, I was thinking those DS-type
>> > constraints
>> > I pointed out would live in the API. It would basically be added
>> > validation in the deliveryservices POST/PUT endpoint that checks
>> > the
>> > DB for existing DSes that conflict with the requested DS.
>> >
>> > - Rawlin
>> >
>> > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
>> > <Jo...@comcast.com> wrote:
>> > >
>> > > These kinds of conditions should be detectable with a
>> > sufficiently advanced SQL query. Is it possible to add the constraint
>> if
>> > it passes and emit a warning during TO startup otherwise? That would
>> let
>> > you know the condition exists at startup but not getting in your way and
>> > keep you out of trouble once you've cleaned up. We made a mistake early
>> > on, but this would acknowledge it was bad and encourage it to be fixed
>> at
>> > the speed of operations teams. Also this puts the constraint in the
>> > database rather than the UI which is really where the contention is for
>> > usability.
>> > >
>> > > Jonathan G
>> > >
>> > >
>> > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
>> rawlin.peters@gmail.com>
>> > wrote:
>> > >
>> > > We occasionally discuss this issue but haven't tackled it
>> > yet. I think
>> > > the main issue is just that duplicate origins have been
>> > allowed since
>> > > the beginning, and now everyone's Traffic Ops could be
>> > littered with
>> > > duplicate origins. Also, depending on the config of the
>> > duplicate
>> > > delivery services, the origins might not be in conflict at
>> > all (if
>> > > they don't have different topology constraints). I would
>> > love for us
>> > > to just add a uniqueness constraint, but there would need
>> to
>> > be a fair
>> > > amount of warning to the community before doing so and
>> might
>> > > invalidate a significant amount of valid use cases.
>> > Operators would
>> > > need time to make DNS CNAME records for the duplicate
>> > origins and
>> > > update their DSes to use the different CNAMEs.
>> > >
>> > > I think as a good first step to eliminating the use of
>> > duplicate
>> > > origins altogether, we should identify which "topology
>> > constraints"
>> > > actually cause conflicting config when used with duplicate
>> > origins and
>> > > prevent creating DSes with duplicate origins _if it would
>> > cause a
>> > > conflict with an existing DS that uses the same origin_.
>> > >
>> > > For instance, I believe an HTTP and DNS-type DS can live
>> > happily
>> > > side-by-side using the same origin (probably need
>> different
>> > > routing_names?), but scenarios like HTTP and HTTP_LIVE, or
>> > DNS and
>> > > HTTP_NO_CACHE sharing the same origin will cause conflicts
>> > for sure.
>> > > So maybe we can start by making sure the DS types "match"
>> > when using
>> > > the same origin:
>> > > HTTP + DNS: possibly good, if they have different routing
>> > names?
>> > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
>> > > HTTP_NO_CACHE + [any other type]: bad
>> > > HTTP_LIVE + HTTP: bad
>> > > etc.
>> > >
>> > > There are most likely other conflict scenarios that don't
>> > involve the
>> > > DS types, but I think this would be a good start. In the
>> > future with
>> > > Delivery Service Topologies (aka Flexible Cachegroups aka
>> > Bring Your
>> > > Own Topology), we might be able to prohibit assigning a DS
>> > to a
>> > > Topology if the DS's origin is already used by another DS
>> in
>> > a
>> > > different Topology.
>> > >
>> > > - Rawlin
>> > >
>> > > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
>> > > <Br...@comcast.com> wrote:
>> > > >
>> > > > As some of you may be aware, `parent.config` files
>> > generated by Traffic Ops can vary wildly when an origin is assigned to
>> > multiple Delivery Services. This results in undefined behavior. I'm told
>> > that the conflict only happens when two Delivery Services with different
>> > "topology requirements" use the same origin, whatever that means
>> (content
>> > routing type?). Regardless, the issue should be addressed. The obvious
>> > solution is to put in place a database constraint that prevents an
>> origin
>> > from being assigned to more that one Delivery Service with API checks in
>> > place that would provide helpful error messages when an attempt is made
>> to
>> > violate the constraint. However, would that mess with things like
>> > Multi-Site Origin? Or is it just not viable for some other reason? If
>> it is
>> > a good solution, I'm prepared to work on a fix that utilizes it.
>> > >
>> > >
>> >
>> >
>> >
>> >
>> >
>>
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Jeremy Mitchell <mi...@gmail.com>.
seems like the simplest solution (or maybe i'm oversimplifying) is on DS
create/update if ds.origin is already being used by another ds, simply fail
create/update with a message like "origin already in use by another ds".
this will prevent more potentially "dirty" data from entering the system
but will not address the dirty data already in the db (if any). that will
require a manual cleanup effort.
On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek <De...@comcast.com>
wrote:
> The only situation in which they can share origins is if a) the origins
> are shared in an MSO configuration but still have different defined origin
> fields in the delivery service, or if they're assigned to completely
> different cachegroups. It's when two delivery services share the same
> edges that there's an issue, because you end up with parent.config issues.
> Actually you could even get away with it in mids as long as you weren't
> doing anything like MSO to it.
>
> Could get messy real fast, though. Best to just create a second FQDN.
>
> Derek
>
> On 12/18/18, 3:23 PM, "Fieck, Brennan" <Br...@comcast.com> wrote:
>
> So no two Delivery Services may share an origin *regardless of cache
> hierarchy* ? I've been told that DNS Delivery Services can have the same
> origin as HTTP Delivery Services because they obey the same cache
> hierarchy. You're saying that would still produce invalid output and/or is
> explicitly disallowed by ATS?
> ________________________________________
> From: Robert Butts <ro...@apache.org>
> Sent: Tuesday, December 18, 2018 1:09 PM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> Services produces indeterminate parent.config
>
> >can you give an example of what parent.config looks like when 2 ds's
> share
> an origin and have different a different topology?
>
> Answering because I encountered this directly, when rewriting
> parent.config.
>
> For example: Suppose you have one Delivery Service:
> XML_ID: foo
> Type: HTPT_LIVE_NATL
> Query String Handling: 1 - ignore in cache key, and pass up
> Origin Server Base URL: http://foo.example.net
>
> And another Delivery Service:
> XML_ID: bar
> Type: HTPT_LIVE_NATL
> Query String Handling: 1 - ignore in cache key, and pass up
> Origin Server Base URL: http://foo.example.net
>
> ATS only supports unique `dest_domain` entries in parent.config.
> Therefore,
> the parent.config generated for a server assigned to both of these
> Delivery
> Services with either be:
>
> dest_domain=foo.example.net port=80 go_direct=true
>
> Or
>
> dest_domain=foo.example.net port=80 go_direct=false qstring=consider
>
> Right now, it's arbitrary which Perl Traffic Ops inserts, and Perl
> provides
> no warning or error of any kind (the pending Go parent.config PR logs
> an
> error).
>
> Whichever is arbitrarily inserted, the resulting remaps for the other
> delivery service will be wrong. Either `foo` requests will drop the
> query
> string when they shouldn't, and go to the mid when they shouldn't; or
> `bar`
> requests will use the query string and skip the mid when it shouldn't.
>
>
> Does that make sense? The only correct solution, is to somehow prevent
> different DSes having the same origin, and tell tenants they must use
> CNAMEs if they need.
>
> This isn't a bug in Traffic Control. ATS fundamentally doesn't support
> multiple remap rules with the same parent FQDN with different
> configurations. Hence, Traffic Control needs to prohibit that.
>
>
> On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
> mitchell852@gmail.com>
> wrote:
>
> > brennan,
> >
> > can you give an example of what parent.config looks like when 2 ds's
> share
> > an origin and have different a different topology?
> >
> > jeremy
> >
> > On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
> Brennan_Fieck@comcast.com
> > >
> > wrote:
> >
> > > To be clear, the "Warning" I'm talking about would happen at
> startup, but
> > > I'd like a UI-only constraint to come with that to disallow using
> the API
> > > to bind the same origin to multiple Delivery Services with varying
> > > topography requirements. It wouldn't change the existing data, but
> > prevent
> > > users from creating more bad data.
> > >
> > > "warning" doesn't really sufficiently describe that, my bad.
> > > ________________________________________
> > > From: Fieck, Brennan <Br...@comcast.com>
> > > Sent: Tuesday, December 18, 2018 11:24 AM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> Services
> > > produces indeterminate parent.config
> > >
> > > Well the cost of fixing this bug is a constraint on the data.
> Unless we
> > > make it a UI-only constraint - which I'm personally against -
> there must
> > be
> > > some point in the future where ATC cannot reasonably be expected
> to work
> > > with data that violates that constraint. The question is when that
> should
> > > occur, which should likely happen at a minor version release.
> Minor not
> > > major because it doesn't involve a change in data structures,
> merely
> > > relationships between them - in my opinion that's a minor version
> change
> > > but that's definitely up for debate. With several release
> candidates for
> > > 3.0.0 that _doesn't_ include this restriction already in the wild,
> I
> > > wouldn't recommend putting it in there. That means to fix the bug
> as soon
> > > as possible it should go in 3.1.0 which should be the target of
> "master"
> > > after the 3.0.0 release is cut from it.
> > >
> > > So I'd recommend immediately implementing the constraint in master
> with a
> > > refusal to upgrade with bad data, and backport a warning about the
> future
> > > behavior into 3.0.0 or as part of a 3.0.1 provided we had more
> changes
> > that
> > > would warrant a micro version bump.
> > > ________________________________________
> > > From: Gray, Jonathan <Jo...@comcast.com>
> > > Sent: Tuesday, December 18, 2018 9:34 AM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> Services
> > > produces indeterminate parent.config
> > >
> > > -1 Holding an ATC upgrade hostage to data cleanup seems like a bad
> idea.
> > > The issue isn't great, but it's also not new. We should allow
> teams to
> > fix
> > > their data at their normal paces if it doesn't create significant
> > overhead
> > > or an inherant blocker for new functionality or correction of
> other major
> > > problems imho.
> > >
> > > Jonathan G
> > >
> > >
> > > On 12/18/18, 9:28 AM, "Fieck, Brennan" <Br...@comcast.com>
> > wrote:
> > >
> > > Another option is we could detect collisions at startup and
> simply
> > > refuse to continue with the upgrade until the data is fixed. That
> would
> > > allow people using the now-unsupported data format to continue to
> use
> > their
> > > old versions of Traffic Ops without wrecking their database, but
> also
> > > provide an incentive to clean up the data.
> > > ________________________________________
> > > From: Gray, Jonathan <Jo...@comcast.com>
> > > Sent: Tuesday, December 18, 2018 5:12 AM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> Delivery
> > > Services produces indeterminate parent.config
> > >
> > > I'm generally a fan of constrain your data in your database,
> but not
> > > necessarily exclusively. I see this as a one-way
> cleanup/conversion so
> > it
> > > doesn't need to be configurable; otherwise you have to ask the
> question
> > > what happens if someone turns it off. That said, something in the
> UI
> > layer
> > > would be nice to prevent spending significant quantities of time
> > building a
> > > complex DS only to have it fail to post for reasons that could
> have been
> > > known earlier.
> > >
> > > The way my brain works in this case:
> > > If !unique_constraint_exists_query()
> > > If has_duplicates_query()
> > > show_warning()
> > > else
> > > add_unique_constraint()
> > >
> > > to which the API and UI configuration could also make use of
> > > unique_constraint_exists_query() to drive additional layer
> constraints if
> > > desired.
> > >
> > > Jonathan G
> > >
> > > On 12/17/18, 1:11 PM, "Rawlin Peters" <rawlin.peters@gmail.com
> >
> > wrote:
> > >
> > > That is an interesting idea...detect at TO startup whether
> or not
> > > there are duplicate origins and operate in a "prevent
> duplicate
> > > origins" state if no duplicates are found or "prevent
> conflicting
> > > DS
> > > topologies" state if duplicates are found? So once
> operators have
> > > replaced all the duplicate origins with CNAMEs, TO will
> > essentially
> > > operate in a "prohibit all duplicate origins" state. That
> would
> > > probably make for a simpler transition, but I'd want to
> remove
> > that
> > > logic in a following release that strictly prohibits
> duplicate
> > > origins
> > > (assuming that the community agrees we should prohibit
> duplicate
> > > origins altogether).
> > >
> > > As for DB constraints vs UI, I was thinking those DS-type
> > > constraints
> > > I pointed out would live in the API. It would basically be
> added
> > > validation in the deliveryservices POST/PUT endpoint that
> checks
> > > the
> > > DB for existing DSes that conflict with the requested DS.
> > >
> > > - Rawlin
> > >
> > > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > > <Jo...@comcast.com> wrote:
> > > >
> > > > These kinds of conditions should be detectable with a
> > > sufficiently advanced SQL query. Is it possible to add the
> constraint if
> > > it passes and emit a warning during TO startup otherwise? That
> would let
> > > you know the condition exists at startup but not getting in your
> way and
> > > keep you out of trouble once you've cleaned up. We made a mistake
> early
> > > on, but this would acknowledge it was bad and encourage it to be
> fixed at
> > > the speed of operations teams. Also this puts the constraint in
> the
> > > database rather than the UI which is really where the contention
> is for
> > > usability.
> > > >
> > > > Jonathan G
> > > >
> > > >
> > > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> > rawlin.peters@gmail.com>
> > > wrote:
> > > >
> > > > We occasionally discuss this issue but haven't
> tackled it
> > > yet. I think
> > > > the main issue is just that duplicate origins have
> been
> > > allowed since
> > > > the beginning, and now everyone's Traffic Ops could
> be
> > > littered with
> > > > duplicate origins. Also, depending on the config of
> the
> > > duplicate
> > > > delivery services, the origins might not be in
> conflict at
> > > all (if
> > > > they don't have different topology constraints). I
> would
> > > love for us
> > > > to just add a uniqueness constraint, but there would
> need
> > to
> > > be a fair
> > > > amount of warning to the community before doing so
> and
> > might
> > > > invalidate a significant amount of valid use cases.
> > > Operators would
> > > > need time to make DNS CNAME records for the duplicate
> > > origins and
> > > > update their DSes to use the different CNAMEs.
> > > >
> > > > I think as a good first step to eliminating the use
> of
> > > duplicate
> > > > origins altogether, we should identify which
> "topology
> > > constraints"
> > > > actually cause conflicting config when used with
> duplicate
> > > origins and
> > > > prevent creating DSes with duplicate origins _if it
> would
> > > cause a
> > > > conflict with an existing DS that uses the same
> origin_.
> > > >
> > > > For instance, I believe an HTTP and DNS-type DS can
> live
> > > happily
> > > > side-by-side using the same origin (probably need
> different
> > > > routing_names?), but scenarios like HTTP and
> HTTP_LIVE, or
> > > DNS and
> > > > HTTP_NO_CACHE sharing the same origin will cause
> conflicts
> > > for sure.
> > > > So maybe we can start by making sure the DS types
> "match"
> > > when using
> > > > the same origin:
> > > > HTTP + DNS: possibly good, if they have different
> routing
> > > names?
> > > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > > HTTP_NO_CACHE + [any other type]: bad
> > > > HTTP_LIVE + HTTP: bad
> > > > etc.
> > > >
> > > > There are most likely other conflict scenarios that
> don't
> > > involve the
> > > > DS types, but I think this would be a good start. In
> the
> > > future with
> > > > Delivery Service Topologies (aka Flexible
> Cachegroups aka
> > > Bring Your
> > > > Own Topology), we might be able to prohibit
> assigning a DS
> > > to a
> > > > Topology if the DS's origin is already used by
> another DS
> > in
> > > a
> > > > different Topology.
> > > >
> > > > - Rawlin
> > > >
> > > > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> > > > <Br...@comcast.com> wrote:
> > > > >
> > > > > As some of you may be aware, `parent.config` files
> > > generated by Traffic Ops can vary wildly when an origin is
> assigned to
> > > multiple Delivery Services. This results in undefined behavior.
> I'm told
> > > that the conflict only happens when two Delivery Services with
> different
> > > "topology requirements" use the same origin, whatever that means
> (content
> > > routing type?). Regardless, the issue should be addressed. The
> obvious
> > > solution is to put in place a database constraint that prevents an
> origin
> > > from being assigned to more that one Delivery Service with API
> checks in
> > > place that would provide helpful error messages when an attempt is
> made
> > to
> > > violate the constraint. However, would that mess with things like
> > > Multi-Site Origin? Or is it just not viable for some other reason?
> If it
> > is
> > > a good solution, I'm prepared to work on a fix that utilizes it.
> > > >
> > > >
> > >
> > >
> > >
> > >
> > >
> >
>
>
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Gray, Jonathan" <Jo...@comcast.com>.
If I'm not mistaken there is also a general problem at the mid tier when it comes to logs since ATS is origin based so you lose granularity to know which DS at the mid tier is responsible.
Jonathan G
On 12/18/18, 1:34 PM, "Gelinas, Derek" <De...@comcast.com> wrote:
The only situation in which they can share origins is if a) the origins are shared in an MSO configuration but still have different defined origin fields in the delivery service, or if they're assigned to completely different cachegroups. It's when two delivery services share the same edges that there's an issue, because you end up with parent.config issues. Actually you could even get away with it in mids as long as you weren't doing anything like MSO to it.
Could get messy real fast, though. Best to just create a second FQDN.
Derek
On 12/18/18, 3:23 PM, "Fieck, Brennan" <Br...@comcast.com> wrote:
So no two Delivery Services may share an origin *regardless of cache hierarchy* ? I've been told that DNS Delivery Services can have the same origin as HTTP Delivery Services because they obey the same cache hierarchy. You're saying that would still produce invalid output and/or is explicitly disallowed by ATS?
________________________________________
From: Robert Butts <ro...@apache.org>
Sent: Tuesday, December 18, 2018 1:09 PM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
>can you give an example of what parent.config looks like when 2 ds's share
an origin and have different a different topology?
Answering because I encountered this directly, when rewriting parent.config.
For example: Suppose you have one Delivery Service:
XML_ID: foo
Type: HTPT_LIVE_NATL
Query String Handling: 1 - ignore in cache key, and pass up
Origin Server Base URL: http://foo.example.net
And another Delivery Service:
XML_ID: bar
Type: HTPT_LIVE_NATL
Query String Handling: 1 - ignore in cache key, and pass up
Origin Server Base URL: http://foo.example.net
ATS only supports unique `dest_domain` entries in parent.config. Therefore,
the parent.config generated for a server assigned to both of these Delivery
Services with either be:
dest_domain=foo.example.net port=80 go_direct=true
Or
dest_domain=foo.example.net port=80 go_direct=false qstring=consider
Right now, it's arbitrary which Perl Traffic Ops inserts, and Perl provides
no warning or error of any kind (the pending Go parent.config PR logs an
error).
Whichever is arbitrarily inserted, the resulting remaps for the other
delivery service will be wrong. Either `foo` requests will drop the query
string when they shouldn't, and go to the mid when they shouldn't; or `bar`
requests will use the query string and skip the mid when it shouldn't.
Does that make sense? The only correct solution, is to somehow prevent
different DSes having the same origin, and tell tenants they must use
CNAMEs if they need.
This isn't a bug in Traffic Control. ATS fundamentally doesn't support
multiple remap rules with the same parent FQDN with different
configurations. Hence, Traffic Control needs to prohibit that.
On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <mi...@gmail.com>
wrote:
> brennan,
>
> can you give an example of what parent.config looks like when 2 ds's share
> an origin and have different a different topology?
>
> jeremy
>
> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <Brennan_Fieck@comcast.com
> >
> wrote:
>
> > To be clear, the "Warning" I'm talking about would happen at startup, but
> > I'd like a UI-only constraint to come with that to disallow using the API
> > to bind the same origin to multiple Delivery Services with varying
> > topography requirements. It wouldn't change the existing data, but
> prevent
> > users from creating more bad data.
> >
> > "warning" doesn't really sufficiently describe that, my bad.
> > ________________________________________
> > From: Fieck, Brennan <Br...@comcast.com>
> > Sent: Tuesday, December 18, 2018 11:24 AM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > produces indeterminate parent.config
> >
> > Well the cost of fixing this bug is a constraint on the data. Unless we
> > make it a UI-only constraint - which I'm personally against - there must
> be
> > some point in the future where ATC cannot reasonably be expected to work
> > with data that violates that constraint. The question is when that should
> > occur, which should likely happen at a minor version release. Minor not
> > major because it doesn't involve a change in data structures, merely
> > relationships between them - in my opinion that's a minor version change
> > but that's definitely up for debate. With several release candidates for
> > 3.0.0 that _doesn't_ include this restriction already in the wild, I
> > wouldn't recommend putting it in there. That means to fix the bug as soon
> > as possible it should go in 3.1.0 which should be the target of "master"
> > after the 3.0.0 release is cut from it.
> >
> > So I'd recommend immediately implementing the constraint in master with a
> > refusal to upgrade with bad data, and backport a warning about the future
> > behavior into 3.0.0 or as part of a 3.0.1 provided we had more changes
> that
> > would warrant a micro version bump.
> > ________________________________________
> > From: Gray, Jonathan <Jo...@comcast.com>
> > Sent: Tuesday, December 18, 2018 9:34 AM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > produces indeterminate parent.config
> >
> > -1 Holding an ATC upgrade hostage to data cleanup seems like a bad idea.
> > The issue isn't great, but it's also not new. We should allow teams to
> fix
> > their data at their normal paces if it doesn't create significant
> overhead
> > or an inherant blocker for new functionality or correction of other major
> > problems imho.
> >
> > Jonathan G
> >
> >
> > On 12/18/18, 9:28 AM, "Fieck, Brennan" <Br...@comcast.com>
> wrote:
> >
> > Another option is we could detect collisions at startup and simply
> > refuse to continue with the upgrade until the data is fixed. That would
> > allow people using the now-unsupported data format to continue to use
> their
> > old versions of Traffic Ops without wrecking their database, but also
> > provide an incentive to clean up the data.
> > ________________________________________
> > From: Gray, Jonathan <Jo...@comcast.com>
> > Sent: Tuesday, December 18, 2018 5:12 AM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > Services produces indeterminate parent.config
> >
> > I'm generally a fan of constrain your data in your database, but not
> > necessarily exclusively. I see this as a one-way cleanup/conversion so
> it
> > doesn't need to be configurable; otherwise you have to ask the question
> > what happens if someone turns it off. That said, something in the UI
> layer
> > would be nice to prevent spending significant quantities of time
> building a
> > complex DS only to have it fail to post for reasons that could have been
> > known earlier.
> >
> > The way my brain works in this case:
> > If !unique_constraint_exists_query()
> > If has_duplicates_query()
> > show_warning()
> > else
> > add_unique_constraint()
> >
> > to which the API and UI configuration could also make use of
> > unique_constraint_exists_query() to drive additional layer constraints if
> > desired.
> >
> > Jonathan G
> >
> > On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com>
> wrote:
> >
> > That is an interesting idea...detect at TO startup whether or not
> > there are duplicate origins and operate in a "prevent duplicate
> > origins" state if no duplicates are found or "prevent conflicting
> > DS
> > topologies" state if duplicates are found? So once operators have
> > replaced all the duplicate origins with CNAMEs, TO will
> essentially
> > operate in a "prohibit all duplicate origins" state. That would
> > probably make for a simpler transition, but I'd want to remove
> that
> > logic in a following release that strictly prohibits duplicate
> > origins
> > (assuming that the community agrees we should prohibit duplicate
> > origins altogether).
> >
> > As for DB constraints vs UI, I was thinking those DS-type
> > constraints
> > I pointed out would live in the API. It would basically be added
> > validation in the deliveryservices POST/PUT endpoint that checks
> > the
> > DB for existing DSes that conflict with the requested DS.
> >
> > - Rawlin
> >
> > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > <Jo...@comcast.com> wrote:
> > >
> > > These kinds of conditions should be detectable with a
> > sufficiently advanced SQL query. Is it possible to add the constraint if
> > it passes and emit a warning during TO startup otherwise? That would let
> > you know the condition exists at startup but not getting in your way and
> > keep you out of trouble once you've cleaned up. We made a mistake early
> > on, but this would acknowledge it was bad and encourage it to be fixed at
> > the speed of operations teams. Also this puts the constraint in the
> > database rather than the UI which is really where the contention is for
> > usability.
> > >
> > > Jonathan G
> > >
> > >
> > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> rawlin.peters@gmail.com>
> > wrote:
> > >
> > > We occasionally discuss this issue but haven't tackled it
> > yet. I think
> > > the main issue is just that duplicate origins have been
> > allowed since
> > > the beginning, and now everyone's Traffic Ops could be
> > littered with
> > > duplicate origins. Also, depending on the config of the
> > duplicate
> > > delivery services, the origins might not be in conflict at
> > all (if
> > > they don't have different topology constraints). I would
> > love for us
> > > to just add a uniqueness constraint, but there would need
> to
> > be a fair
> > > amount of warning to the community before doing so and
> might
> > > invalidate a significant amount of valid use cases.
> > Operators would
> > > need time to make DNS CNAME records for the duplicate
> > origins and
> > > update their DSes to use the different CNAMEs.
> > >
> > > I think as a good first step to eliminating the use of
> > duplicate
> > > origins altogether, we should identify which "topology
> > constraints"
> > > actually cause conflicting config when used with duplicate
> > origins and
> > > prevent creating DSes with duplicate origins _if it would
> > cause a
> > > conflict with an existing DS that uses the same origin_.
> > >
> > > For instance, I believe an HTTP and DNS-type DS can live
> > happily
> > > side-by-side using the same origin (probably need different
> > > routing_names?), but scenarios like HTTP and HTTP_LIVE, or
> > DNS and
> > > HTTP_NO_CACHE sharing the same origin will cause conflicts
> > for sure.
> > > So maybe we can start by making sure the DS types "match"
> > when using
> > > the same origin:
> > > HTTP + DNS: possibly good, if they have different routing
> > names?
> > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > HTTP_NO_CACHE + [any other type]: bad
> > > HTTP_LIVE + HTTP: bad
> > > etc.
> > >
> > > There are most likely other conflict scenarios that don't
> > involve the
> > > DS types, but I think this would be a good start. In the
> > future with
> > > Delivery Service Topologies (aka Flexible Cachegroups aka
> > Bring Your
> > > Own Topology), we might be able to prohibit assigning a DS
> > to a
> > > Topology if the DS's origin is already used by another DS
> in
> > a
> > > different Topology.
> > >
> > > - Rawlin
> > >
> > > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> > > <Br...@comcast.com> wrote:
> > > >
> > > > As some of you may be aware, `parent.config` files
> > generated by Traffic Ops can vary wildly when an origin is assigned to
> > multiple Delivery Services. This results in undefined behavior. I'm told
> > that the conflict only happens when two Delivery Services with different
> > "topology requirements" use the same origin, whatever that means (content
> > routing type?). Regardless, the issue should be addressed. The obvious
> > solution is to put in place a database constraint that prevents an origin
> > from being assigned to more that one Delivery Service with API checks in
> > place that would provide helpful error messages when an attempt is made
> to
> > violate the constraint. However, would that mess with things like
> > Multi-Site Origin? Or is it just not viable for some other reason? If it
> is
> > a good solution, I'm prepared to work on a fix that utilizes it.
> > >
> > >
> >
> >
> >
> >
> >
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by John Rushford <jj...@gmail.com>.
This is the records.config setting: proxy.config.url_remap.pristine_host_hdr. The vhost
config on the origin would need to match the CNAME.
> On Jan 31, 2019, at 10:30 PM, John Rushford <jj...@gmail.com> wrote:
>
> origin virtual hosting relies on the Host header. So if the request from the client to the reverse
> proxy has a Host header and maintain_pristine_host_hdr is not enabled, remap
> will rewrite the Host header along with the request url. So in this case, remapping to
> a CNAME should work.
>
>> On Jan 29, 2019, at 6:02 PM, Steve Malenfant <sm...@gmail.com> wrote:
>>
>> CNAME might not necessary work if the origin is using virtual hosting. It
>> would need to be configured as well. Might work with some origin? But not
>> all of them.
>>
>> On Tue, Jan 29, 2019 at 7:40 PM Fieck, Brennan <Br...@comcast.com>
>> wrote:
>>
>>> The issue is inherent with the structure of ATS configuration files. If I
>>> understand correctly - someone correct me if I don't -
>>> you can't have more than one rule for a "primary destination" in the
>>> parent.config. Or, you can, but only one of them will
>>> wind up catching all of your matches. Which means that if you have
>>> multiple Delivery Services that use the same origin FQDN
>>> (dest_domain) then you get proper behavior out of ATS if and only if the
>>> resulting rules would be identical. I'd highly
>>> recommend that you not move forward using duplicate origin FQDNs on your
>>> Delivery Services, because it probably won't
>>> do what you want.
>>>
>>> I'm not sure what you mean about the cachekeys. The Delivery Service URLs
>>> should still be unique, so you shouldn't run
>>> into a collision in the cache keys.
>>>
>>> Origins are an object. They're also a cachegroup type. And a profile type.
>>> And a server type. And a URL field on a Delivery Service.
>>>
>>> The solution that's been used in the past to use the same actual origin
>>> machine to service multiple Delivery Services safely
>>> is to add a "Canonical Name" DNS record to the ATC internal zone for each
>>> Delivery Service. The idea is to have a unique name
>>> for each Delivery Service, but they all resolve to the same IP address.
>>> ________________________________________
>>> From: Steve Malenfant <sm...@gmail.com>
>>> Sent: Tuesday, January 29, 2019 4:32 PM
>>> To: dev@trafficcontrol.apache.org
>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
>>> produces indeterminate parent.config
>>>
>>> We use and will use the same origin for multiple DS. It’s an inherent part
>>> of the design of some part of our delivery.
>>>
>>> Wouldn’t be less of a problem using cachekeys for different delivery
>>> services?
>>>
>>> I would rather see origin as an object and not just a simple URL. I’ve seen
>>> this in the past in other systems.
>>>
>>> I don’t believe we can create multiple http services endpoint for a single
>>> DS like we can for DNS type delivery service. This would solve it for us.
>>>
>>> Steve
>>>
>>> On Thu, Jan 24, 2019 at 11:22 AM Gray, Jonathan <Jonathan_Gray@comcast.com
>>>>
>>> wrote:
>>>
>>>> Even that has a strong element of confusion when in a self-service
>>>> multi-tenancy world. Multiple tenants can use the same origins for
>>> things
>>>> like akami, aws, internal load balancers and any communication about the
>>>> origin already being used will be confusing.
>>>>
>>>> Jonathan G
>>>>
>>>>
>>>> On 1/23/19, 7:37 PM, "Gelinas, Derek" <De...@comcast.com>
>>> wrote:
>>>>
>>>> Perhaps we could solve both issues and just allow it to happen on DS
>>>> config but throw up a message when it is saved that says "hey we're going
>>>> to do this but it's really not without an element of risk and ats isn't
>>>> designed to work properly with multiple identical origins. We recommend
>>> you
>>>> use an alternative fqdn blah blah blah."
>>>>
>>>> Something along those lines, anyway.
>>>>
>>>> Derek
>>>>
>>>>> On Jan 23, 2019, at 7:18 PM, Rawlin Peters <
>>> rawlin.peters@gmail.com>
>>>> wrote:
>>>>>
>>>>> Simply prohibiting duplicate origin FQDNs has been vetoed now, so
>>> we
>>>>> can't really move forward with that unless the vetoers change their
>>>>> minds and remove their -1s.
>>>>>
>>>>> Currently that leaves us with checking at DS create/update whether
>>> or
>>>>> not it would conflict with a shared origin due to different DS
>>> types,
>>>>> qstring config, etc. I am -1 on that idea since I think it will be
>>>>> difficult to maintain and enumerate all the possible conflicting
>>>>> cases, will present confusing error messages to the user to which
>>> the
>>>>> solution would just be "create a CNAME record" anyways, and that
>>> the
>>>>> ATS parent.config should really be per-remap rather than "global".
>>> I
>>>>> know there has been some discussion in the ATS community about
>>> making
>>>>> parent.config per-remap, but I don't have much information there.
>>>>>
>>>>> That is to say, I don't think there is a great solution to this
>>>>> problem right now that we can all agree upon, but maybe our efforts
>>>>> would be better spent in the ATS community by making a per-remap
>>>>> parent.config. Then the duplicate origin problem in Traffic Control
>>>>> would go away.
>>>>>
>>>>> Another option could be to have some kind of generic rules engine
>>>> that
>>>>> would allow an organization to create some set of rules that DSes
>>>>> would have to abide by. For example, no duplicate origins, an
>>> origin
>>>>> can't be in a list of known malicious domains, you can't use the DS
>>>>> type "HTTP_NO_CACHE", maxDnsAnswers must be < 5, TTLs must be > 30
>>>> and
>>>>> < 6000, protocol must be HTTPS, intitialDispersion must be < 10,
>>> DSCP
>>>>> must be 40, etc. It seems like for something truly self-service
>>> you'd
>>>>> probably need to set up some rules for your users to abide by that
>>>>> might not necessarily apply to ALL organizations. Just a thought. I
>>>>> don't think the duplicate origins problem alone would warrant
>>>>> something like that, but if it also provided a solution for the
>>>>> self-service effort, maybe it would be worth it.
>>>>>
>>>>> - Rawlin
>>>>>
>>>>> On Wed, Jan 23, 2019 at 4:15 AM Gelinas, Derek
>>>>> <De...@comcast.com> wrote:
>>>>>>
>>>>>> That's the problem. We either need to be smart enough to recognize
>>>> that in our config and warn the user or just prevent it and suggest to
>>> the
>>>> user that they use a different fqdn.
>>>>>>
>>>>>> My vote is for preventing it at this point.
>>>>>>
>>>>>>> On Jan 23, 2019, at 12:50 AM, Gray, Jonathan <
>>>> Jonathan_Gray@comcast.com> wrote:
>>>>>>>
>>>>>>> It's not just differing types, it's anything that affects the
>>>> parent.config which also includes the, qstring configuration, MSO
>>>> parents/config, etc. as well.
>>>>>>>
>>>>>>> Jonathan G
>>>>>>>
>>>>>>>
>>>>>>> On 1/22/19, 2:33 PM, "Dave Neuman" <ne...@apache.org> wrote:
>>>>>>>
>>>>>>> -1 on the one DS to Origin limitation. I think there are legit
>>>> use cases
>>>>>>> for it.
>>>>>>> I would be +1 on the idea if we include type though. Two DSs
>>>> can share the
>>>>>>> same origin as long as they are of the same type (DNS, HTTP,
>>>> HTTP_LIVE),
>>>>>>> etc. You shouldn't be able to have two DSs with the same
>>> origin
>>>> but
>>>>>>> different types.
>>>>>>>
>>>>>>> On Mon, Jan 14, 2019 at 7:53 AM Fieck, Brennan <
>>>> Brennan_Fieck@comcast.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I don't see this as a complicated use case or a barrier to
>>> entry,
>>>> an
>>>>>>>> extremely basic introduction to ATC would only make use of one
>>> DS
>>>>>>>> (shameless CDN-in-a-Box plug).
>>>>>>>> It seems far more complicated for a user to be using many DSes
>>> to
>>>> serve a
>>>>>>>> single origin - we can easily make it clear in the docs that the
>>>> two have a
>>>>>>>> 1:1 relationship, with a footnote about CNAME records for users
>>>> with the
>>>>>>>> knowledge, resources, and need for such workarounds.
>>>>>>>> ________________________________________
>>>>>>>> From: Mark Torluemke <mt...@apache.org>
>>>>>>>> Sent: Friday, January 11, 2019 1:21 PM
>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>>>> Services
>>>>>>>> produces indeterminate parent.config
>>>>>>>>
>>>>>>>>>> 1. Prohibit creating new delivery services that would share an
>>>>>>>> existing origin and prohibit updating a delivery service to a
>>>> shared
>>>>>>>> origin
>>>>>>>> In case my position has been lost, I'm still -1 on this. :) IMO,
>>>> we
>>>>>>>> shouldn't let the few complicated use cases increase the
>>> barrier
>>>> to entry
>>>>>>>> (to using ATC) for the masses.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Jan 11, 2019 at 11:53 AM Rawlin Peters <
>>>> rawlin.peters@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Alright, I'm trying to sum up this discussion so far since it
>>>> seems
>>>>>>>>> like everyone went on vacation and didn't really get a chance
>>> to
>>>> wrap
>>>>>>>>> this one up:
>>>>>>>>> - duplicate origins cause undefined behavior
>>>>>>>>> - we need a way to migrate to a future that is free of
>>> duplicate
>>>>>>>>> origins in Traffic Control
>>>>>>>>> - we need a visual and easy way to determine if Traffic Ops
>>>> currently
>>>>>>>>> contains duplicate origins, so that operators are incentivized
>>>> to fix
>>>>>>>>> them rather than let it slide indefinitely
>>>>>>>>> - operators should have a fair amount of time to fix their
>>>> duplicate
>>>>>>>>> origins
>>>>>>>>>
>>>>>>>>> I believe this is what we've mostly agreed upon but haven't
>>>> clearly voted
>>>>>>>>> on:
>>>>>>>>>
>>>>>>>>> In release N we will:
>>>>>>>>> 1. Prohibit creating new delivery services that would share an
>>>>>>>>> existing origin and prohibit updating a delivery service to a
>>>> shared
>>>>>>>>> origin
>>>>>>>>> 2. Add some kind of visual indicator that duplicate origins
>>> are a
>>>>>>>>> problem that need to be fixed before release N+1; otherwise, an
>>>>>>>>> upgrade to N+1 will be prohibited.
>>>>>>>>>
>>>>>>>>> In release N+1 we will:
>>>>>>>>> 3. Include a DB migration that adds a uniqueness constraint on
>>>> origin
>>>>>>>>> FQDN, removing the API-level checks for that.
>>>>>>>>> 4. Prevent an upgrade to N+1 if duplicate origins are found
>>> (this
>>>>>>>>> might occur as a byproduct of step 3).
>>>>>>>>>
>>>>>>>>> I am +1 on this plan and believe this would hit on all the
>>>> summarized
>>>>>>>>> points above. Please provide a clear vote on this plan so that
>>>> we can
>>>>>>>>> dive deeper in the details (i.e. what release 'N' is, the best
>>>> visual
>>>>>>>>> indicator for step 2, and a friendly way to handle step 4).
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Rawlin
>>>>>>>>>
>>>>>>>>> On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <
>>>> mitchell852@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Not sure TP is the right place for a warning like "clean up
>>> this
>>>>>>>>>> 'duplicate' origin or your next upgrade will fail". Most users
>>>> of our
>>>>>>>>>> system will be like "not my problem".
>>>>>>>>>>
>>>>>>>>>> On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <
>>>>>>>>> Brennan_Fieck@comcast.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Probably. It would impact load times by a bit, but the page
>>>> for an
>>>>>>>>>>> individual object is not our bottleneck.
>>>>>>>>>>> ________________________________________
>>>>>>>>>>> From: Robert Butts <ro...@apache.org>
>>>>>>>>>>> Sent: Wednesday, December 19, 2018 11:50 AM
>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>> Delivery
>>>>>>>>> Services
>>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>>
>>>>>>>>>>>> - Including a warning on startup and an API constraint
>>>> preventing
>>>>>>>>> adding
>>>>>>>>>>> more bad data in the next 3.0.0 Release Candidate
>>>>>>>>>>>> - Adding a database constraint immediately into master that
>>>> won't
>>>>>>>> be
>>>>>>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
>>>>>>>>>>>
>>>>>>>>>>> +1
>>>>>>>>>>>
>>>>>>>>>>> I understand Jonathan's objection, but at some point, we have
>>>> to be
>>>>>>>>> able to
>>>>>>>>>>> move forward. This is a good compromise: deprecate, then
>>>> remove. That
>>>>>>>>> gives
>>>>>>>>>>> people a full major version to fix their data.
>>>>>>>>>>>
>>>>>>>>>>> I would be ideal if it were more than just a logged warning,
>>>> though.
>>>>>>>>> Can we
>>>>>>>>>>> add a big red banner in Traffic Portal, on the Delivery
>>>> Service page
>>>>>>>>> for
>>>>>>>>>>> any DS with a duplicate origin, telling users to fix it, and
>>>> that
>>>>>>>> they
>>>>>>>>>>> won't be able to upgrade to the next major version until it's
>>>> fixed?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <
>>>>>>>>> Brennan_Fieck@comcast.com
>>>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> So it seems like nobody has a problem with the "how" -
>>>> disallowing
>>>>>>>>>>>> duplicate origin FQDNs on Delivery Services - but we never
>>>> reached
>>>>>>>> a
>>>>>>>>>>>> consensus on "when".
>>>>>>>>>>>>
>>>>>>>>>>>> I stand by my previous proposal:
>>>>>>>>>>>> - Including a warning on startup and an API constraint
>>>> preventing
>>>>>>>>> adding
>>>>>>>>>>>> more bad data in the next 3.0.0 Release Candidate
>>>>>>>>>>>> - Adding a database constraint immediately into master that
>>>> won't
>>>>>>>> be
>>>>>>>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>> From: Rawlin Peters <ra...@gmail.com>
>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 4:59 PM
>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>> Delivery
>>>>>>>>> Services
>>>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>>>
>>>>>>>>>>>> Also, building more around DS types will make it even harder
>>>> to get
>>>>>>>>>>>> away from DS types in the future too, which I know is
>>>> something
>>>>>>>> we've
>>>>>>>>>>>> discussed on this mailing list before. It also adds to the
>>>> overhead
>>>>>>>>> of
>>>>>>>>>>>> Delivery Service Topologies, since a lot of the DS types
>>> won't
>>>>>>>>>>>> carryover into that world.
>>>>>>>>>>>>
>>>>>>>>>>>> - Rawlin
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
>>>>>>>>>>>> <Br...@comcast.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> +1.
>>>>>>>>>>>>> If there's a simple way to work around duplicate origins
>>>> being
>>>>>>>>>>>> prohibited,
>>>>>>>>>>>>> then we should rely on that instead of "enumerating all
>>> those
>>>>>>>>> possible
>>>>>>>>>>>> conflicting
>>>>>>>>>>>>> settings, which are not only highly complex and confusing,
>>>> but
>>>>>>>> also
>>>>>>>>>>>> further
>>>>>>>>>>>>> entrench us in only supporting ATS as a caching proxy
>>>> (hurting
>>>>>>>>> efforts
>>>>>>>>>>> to
>>>>>>>>>>>>> integrate e.g. Grove, nginx etc.)
>>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>>> From: Rawlin Peters <ra...@gmail.com>
>>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 2:20 PM
>>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>> Delivery
>>>>>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>>>>>
>>>>>>>>>>>>> There are a number of different DS settings at play that
>>> can
>>>>>>>>>>>>> potentially cause conflicts. The question is: do we want to
>>>> get
>>>>>>>>> into
>>>>>>>>>>>>> the business of enumerating all those possible conflicting
>>>>>>>>> settings or
>>>>>>>>>>>>> just simply prohibit duplicate origins altogether? I think
>>>> we can
>>>>>>>>> dig
>>>>>>>>>>>>> in and get that "sufficiently advanced sql query" to check
>>>> for
>>>>>>>>>>>>> conflicting origins, but is that something we want to carry
>>>> along
>>>>>>>>> for
>>>>>>>>>>>>> the foreseeable future? Aren't CNAMEs relatively cheaper
>>> than
>>>>>>>>>>>>> developing and maintaining that code and the mental
>>> overhead
>>>>>>>>> required
>>>>>>>>>>>>> in understanding why you're getting an error that says your
>>>>>>>>> requested
>>>>>>>>>>>>> DS would cause an origin conflict? I think at the point
>>>> you've
>>>>>>>>>>>>> requested a DS that would create a conflict, you've chosen
>>>> those
>>>>>>>>>>>>> settings for a reason and would probably prefer to just
>>>>>>>> create/use
>>>>>>>>> a
>>>>>>>>>>>>> CNAME in your new DS and keep the rest of your settings the
>>>> same.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thinking in terms of errors, I'm imagining:
>>>>>>>>>>>>> "cannot create delivery service: origin fqdn '
>>>> foo.example.com'
>>>>>>>>> already
>>>>>>>>>>>> in use"
>>>>>>>>>>>>> vs
>>>>>>>>>>>>> "cannot create delivery service: origin fqdn '
>>>> foo.example.com'
>>>>>>>>> already
>>>>>>>>>>>>> in use as type DNS_LIVE_NATNL, which is incompatible with
>>>> your
>>>>>>>>> chosen
>>>>>>>>>>>>> type of HTTP_NO_CACHE"
>>>>>>>>>>>>>
>>>>>>>>>>>>> At that point you'd probably say to yourself, "uh, I need
>>>>>>>>>>>>> HTTP_NO_CACHE, so what am I supposed to do now?"
>>>>>>>>>>>>>
>>>>>>>>>>>>> As a lazy developer I'm +1 on prohibiting duplicate origin
>>>> fqdns
>>>>>>>>>>>>> because the resulting code will be simpler, but I think
>>>>>>>> eliminating
>>>>>>>>>>>>> the mental overhead for operators could be worthwhile too.
>>>> If we
>>>>>>>>> can
>>>>>>>>>>>>> agree on an end state of prohibiting duplicate origins
>>>>>>>> altogether,
>>>>>>>>> we
>>>>>>>>>>>>> can start working on a design to smoothly transition us to
>>>> that
>>>>>>>>> point.
>>>>>>>>>>>>> Are we willing to live with "just CNAME your origin fqdn"
>>> as
>>>> the
>>>>>>>>>>>>> standard solution to duplicates?
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Rawlin
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
>>>>>>>>>>>>> <De...@comcast.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The only situation in which they can share origins is if
>>> a)
>>>> the
>>>>>>>>>>>> origins are shared in an MSO configuration but still have
>>>> different
>>>>>>>>>>> defined
>>>>>>>>>>>> origin fields in the delivery service, or if they're
>>> assigned
>>>> to
>>>>>>>>>>> completely
>>>>>>>>>>>> different cachegroups. It's when two delivery services
>>> share
>>>> the
>>>>>>>>> same
>>>>>>>>>>>> edges that there's an issue, because you end up with
>>>> parent.config
>>>>>>>>>>> issues.
>>>>>>>>>>>> Actually you could even get away with it in mids as long as
>>>> you
>>>>>>>>> weren't
>>>>>>>>>>>> doing anything like MSO to it.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Could get messy real fast, though. Best to just create a
>>>>>>>> second
>>>>>>>>>>> FQDN.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Derek
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 12/18/18, 3:23 PM, "Fieck, Brennan" <
>>>>>>>>> Brennan_Fieck@comcast.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So no two Delivery Services may share an origin
>>>> *regardless
>>>>>>>>> of
>>>>>>>>>>>> cache hierarchy* ? I've been told that DNS Delivery Services
>>>> can
>>>>>>>>> have the
>>>>>>>>>>>> same origin as HTTP Delivery Services because they obey the
>>>> same
>>>>>>>>> cache
>>>>>>>>>>>> hierarchy. You're saying that would still produce invalid
>>>> output
>>>>>>>>> and/or
>>>>>>>>>>> is
>>>>>>>>>>>> explicitly disallowed by ATS?
>>>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>>>> From: Robert Butts <ro...@apache.org>
>>>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 1:09 PM
>>>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>>>>>> Delivery
>>>>>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> can you give an example of what parent.config looks like
>>>>>>>>> when 2
>>>>>>>>>>>> ds's share
>>>>>>>>>>>>>> an origin and have different a different topology?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Answering because I encountered this directly, when
>>>>>>>> rewriting
>>>>>>>>>>>> parent.config.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> For example: Suppose you have one Delivery Service:
>>>>>>>>>>>>>> XML_ID: foo
>>>>>>>>>>>>>> Type: HTPT_LIVE_NATL
>>>>>>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass
>>>> up
>>>>>>>>>>>>>> Origin Server Base URL: http://foo.example.net
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> And another Delivery Service:
>>>>>>>>>>>>>> XML_ID: bar
>>>>>>>>>>>>>> Type: HTPT_LIVE_NATL
>>>>>>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass
>>>> up
>>>>>>>>>>>>>> Origin Server Base URL: http://foo.example.net
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ATS only supports unique `dest_domain` entries in
>>>>>>>>> parent.config.
>>>>>>>>>>>> Therefore,
>>>>>>>>>>>>>> the parent.config generated for a server assigned to
>>> both
>>>>>>>> of
>>>>>>>>>>> these
>>>>>>>>>>>> Delivery
>>>>>>>>>>>>>> Services with either be:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=true
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Or
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=false
>>>>>>>>>>>> qstring=consider
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Right now, it's arbitrary which Perl Traffic Ops
>>> inserts,
>>>>>>>> and
>>>>>>>>>>> Perl
>>>>>>>>>>>> provides
>>>>>>>>>>>>>> no warning or error of any kind (the pending Go
>>>>>>>>> parent.config PR
>>>>>>>>>>>> logs an
>>>>>>>>>>>>>> error).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Whichever is arbitrarily inserted, the resulting remaps
>>>> for
>>>>>>>>> the
>>>>>>>>>>>> other
>>>>>>>>>>>>>> delivery service will be wrong. Either `foo` requests
>>> will
>>>>>>>>> drop
>>>>>>>>>>>> the query
>>>>>>>>>>>>>> string when they shouldn't, and go to the mid when they
>>>>>>>>>>> shouldn't;
>>>>>>>>>>>> or `bar`
>>>>>>>>>>>>>> requests will use the query string and skip the mid when
>>>> it
>>>>>>>>>>>> shouldn't.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Does that make sense? The only correct solution, is to
>>>>>>>>> somehow
>>>>>>>>>>>> prevent
>>>>>>>>>>>>>> different DSes having the same origin, and tell tenants
>>>>>>>> they
>>>>>>>>> must
>>>>>>>>>>>> use
>>>>>>>>>>>>>> CNAMEs if they need.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This isn't a bug in Traffic Control. ATS fundamentally
>>>>>>>>> doesn't
>>>>>>>>>>>> support
>>>>>>>>>>>>>> multiple remap rules with the same parent FQDN with
>>>>>>>> different
>>>>>>>>>>>>>> configurations. Hence, Traffic Control needs to prohibit
>>>>>>>>> that.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
>>>>>>>>>>>> mitchell852@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> brennan,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> can you give an example of what parent.config looks like
>>>>>>>>> when 2
>>>>>>>>>>>> ds's share
>>>>>>>>>>>>>>> an origin and have different a different topology?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> jeremy
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
>>>>>>>>>>>> Brennan_Fieck@comcast.com
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To be clear, the "Warning" I'm talking about would
>>>>>>>>> happen at
>>>>>>>>>>>> startup, but
>>>>>>>>>>>>>>>> I'd like a UI-only constraint to come with that to
>>>>>>>>> disallow
>>>>>>>>>>>> using the API
>>>>>>>>>>>>>>>> to bind the same origin to multiple Delivery Services
>>>>>>>>> with
>>>>>>>>>>>> varying
>>>>>>>>>>>>>>>> topography requirements. It wouldn't change the
>>>>>>>> existing
>>>>>>>>>>> data,
>>>>>>>>>>>> but
>>>>>>>>>>>>>>> prevent
>>>>>>>>>>>>>>>> users from creating more bad data.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> "warning" doesn't really sufficiently describe that, my
>>>>>>>>> bad.
>>>>>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>>>>>> From: Fieck, Brennan <Br...@comcast.com>
>>>>>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 11:24 AM
>>>>>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>>>>>>>>> Delivery Services
>>>>>>>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Well the cost of fixing this bug is a constraint on the
>>>>>>>>> data.
>>>>>>>>>>>> Unless we
>>>>>>>>>>>>>>>> make it a UI-only constraint - which I'm personally
>>>>>>>>> against -
>>>>>>>>>>>> there must
>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>> some point in the future where ATC cannot reasonably be
>>>>>>>>>>>> expected to work
>>>>>>>>>>>>>>>> with data that violates that constraint. The question
>>>>>>>> is
>>>>>>>>> when
>>>>>>>>>>>> that should
>>>>>>>>>>>>>>>> occur, which should likely happen at a minor version
>>>>>>>>> release.
>>>>>>>>>>>> Minor not
>>>>>>>>>>>>>>>> major because it doesn't involve a change in data
>>>>>>>>> structures,
>>>>>>>>>>>> merely
>>>>>>>>>>>>>>>> relationships between them - in my opinion that's a
>>>>>>>> minor
>>>>>>>>>>>> version change
>>>>>>>>>>>>>>>> but that's definitely up for debate. With several
>>>>>>>> release
>>>>>>>>>>>> candidates for
>>>>>>>>>>>>>>>> 3.0.0 that _doesn't_ include this restriction already
>>>>>>>> in
>>>>>>>>> the
>>>>>>>>>>>> wild, I
>>>>>>>>>>>>>>>> wouldn't recommend putting it in there. That means to
>>>>>>>>> fix the
>>>>>>>>>>>> bug as soon
>>>>>>>>>>>>>>>> as possible it should go in 3.1.0 which should be the
>>>>>>>>> target
>>>>>>>>>>>> of "master"
>>>>>>>>>>>>>>>> after the 3.0.0 release is cut from it.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> So I'd recommend immediately implementing the
>>>>>>>> constraint
>>>>>>>>> in
>>>>>>>>>>>> master with a
>>>>>>>>>>>>>>>> refusal to upgrade with bad data, and backport a
>>>>>>>> warning
>>>>>>>>>>> about
>>>>>>>>>>>> the future
>>>>>>>>>>>>>>>> behavior into 3.0.0 or as part of a 3.0.1 provided we
>>>>>>>> had
>>>>>>>>>>> more
>>>>>>>>>>>> changes
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>> would warrant a micro version bump.
>>>>>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
>>>>>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 9:34 AM
>>>>>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>>>>>>>>> Delivery Services
>>>>>>>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -1 Holding an ATC upgrade hostage to data cleanup seems
>>>>>>>>> like
>>>>>>>>>>> a
>>>>>>>>>>>> bad idea.
>>>>>>>>>>>>>>>> The issue isn't great, but it's also not new. We
>>>>>>>> should
>>>>>>>>>>> allow
>>>>>>>>>>>> teams to
>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>> their data at their normal paces if it doesn't create
>>>>>>>>>>>> significant
>>>>>>>>>>>>>>> overhead
>>>>>>>>>>>>>>>> or an inherant blocker for new functionality or
>>>>>>>>> correction of
>>>>>>>>>>>> other major
>>>>>>>>>>>>>>>> problems imho.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Jonathan G
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 12/18/18, 9:28 AM, "Fieck, Brennan" <
>>>>>>>>>>>> Brennan_Fieck@comcast.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Another option is we could detect collisions at
>>>>>>>>> startup
>>>>>>>>>>>> and simply
>>>>>>>>>>>>>>>> refuse to continue with the upgrade until the data is
>>>>>>>>> fixed.
>>>>>>>>>>>> That would
>>>>>>>>>>>>>>>> allow people using the now-unsupported data format to
>>>>>>>>>>> continue
>>>>>>>>>>>> to use
>>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>>> old versions of Traffic Ops without wrecking their
>>>>>>>>> database,
>>>>>>>>>>>> but also
>>>>>>>>>>>>>>>> provide an incentive to clean up the data.
>>>>>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
>>>>>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 5:12 AM
>>>>>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to
>>>>>>>>> Multipe
>>>>>>>>>>>> Delivery
>>>>>>>>>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm generally a fan of constrain your data in your
>>>>>>>>>>>> database, but not
>>>>>>>>>>>>>>>> necessarily exclusively. I see this as a one-way
>>>>>>>>>>>> cleanup/conversion so
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>> doesn't need to be configurable; otherwise you have to
>>>>>>>>> ask
>>>>>>>>>>> the
>>>>>>>>>>>> question
>>>>>>>>>>>>>>>> what happens if someone turns it off. That said,
>>>>>>>>> something
>>>>>>>>>>> in
>>>>>>>>>>>> the UI
>>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>>> would be nice to prevent spending significant
>>>>>>>> quantities
>>>>>>>>> of
>>>>>>>>>>>> time
>>>>>>>>>>>>>>> building a
>>>>>>>>>>>>>>>> complex DS only to have it fail to post for reasons
>>>>>>>> that
>>>>>>>>>>> could
>>>>>>>>>>>> have been
>>>>>>>>>>>>>>>> known earlier.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The way my brain works in this case:
>>>>>>>>>>>>>>>> If !unique_constraint_exists_query()
>>>>>>>>>>>>>>>> If has_duplicates_query()
>>>>>>>>>>>>>>>> show_warning()
>>>>>>>>>>>>>>>> else
>>>>>>>>>>>>>>>> add_unique_constraint()
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> to which the API and UI configuration could also
>>>>>>>>> make use
>>>>>>>>>>>> of
>>>>>>>>>>>>>>>> unique_constraint_exists_query() to drive additional
>>>>>>>>> layer
>>>>>>>>>>>> constraints if
>>>>>>>>>>>>>>>> desired.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Jonathan G
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 12/17/18, 1:11 PM, "Rawlin Peters" <
>>>>>>>>>>>> rawlin.peters@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> That is an interesting idea...detect at TO
>>>>>>>>> startup
>>>>>>>>>>>> whether or not
>>>>>>>>>>>>>>>> there are duplicate origins and operate in a
>>>>>>>>> "prevent
>>>>>>>>>>>> duplicate
>>>>>>>>>>>>>>>> origins" state if no duplicates are found or
>>>>>>>>> "prevent
>>>>>>>>>>>> conflicting
>>>>>>>>>>>>>>>> DS
>>>>>>>>>>>>>>>> topologies" state if duplicates are found? So
>>>>>>>>> once
>>>>>>>>>>>> operators have
>>>>>>>>>>>>>>>> replaced all the duplicate origins with CNAMEs,
>>>>>>>>> TO
>>>>>>>>>>> will
>>>>>>>>>>>>>>> essentially
>>>>>>>>>>>>>>>> operate in a "prohibit all duplicate origins"
>>>>>>>>> state.
>>>>>>>>>>>> That would
>>>>>>>>>>>>>>>> probably make for a simpler transition, but I'd
>>>>>>>>> want
>>>>>>>>>>>> to remove
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>> logic in a following release that strictly
>>>>>>>>> prohibits
>>>>>>>>>>>> duplicate
>>>>>>>>>>>>>>>> origins
>>>>>>>>>>>>>>>> (assuming that the community agrees we should
>>>>>>>>>>> prohibit
>>>>>>>>>>>> duplicate
>>>>>>>>>>>>>>>> origins altogether).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> As for DB constraints vs UI, I was thinking
>>>>>>>> those
>>>>>>>>>>>> DS-type
>>>>>>>>>>>>>>>> constraints
>>>>>>>>>>>>>>>> I pointed out would live in the API. It would
>>>>>>>>>>>> basically be added
>>>>>>>>>>>>>>>> validation in the deliveryservices POST/PUT
>>>>>>>>> endpoint
>>>>>>>>>>>> that checks
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> DB for existing DSes that conflict with the
>>>>>>>>> requested
>>>>>>>>>>>> DS.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> - Rawlin
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
>>>>>>>>>>>>>>>> <Jo...@comcast.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> These kinds of conditions should be
>>>>>>>> detectable
>>>>>>>>>>> with a
>>>>>>>>>>>>>>>> sufficiently advanced SQL query. Is it possible to add
>>>>>>>>> the
>>>>>>>>>>>> constraint if
>>>>>>>>>>>>>>>> it passes and emit a warning during TO startup
>>>>>>>> otherwise?
>>>>>>>>>>>> That would let
>>>>>>>>>>>>>>>> you know the condition exists at startup but not
>>>>>>>> getting
>>>>>>>>> in
>>>>>>>>>>>> your way and
>>>>>>>>>>>>>>>> keep you out of trouble once you've cleaned up. We
>>>>>>>> made
>>>>>>>>> a
>>>>>>>>>>>> mistake early
>>>>>>>>>>>>>>>> on, but this would acknowledge it was bad and encourage
>>>>>>>>> it to
>>>>>>>>>>>> be fixed at
>>>>>>>>>>>>>>>> the speed of operations teams. Also this puts the
>>>>>>>>> constraint
>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>> database rather than the UI which is really where the
>>>>>>>>>>>> contention is for
>>>>>>>>>>>>>>>> usability.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Jonathan G
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 12/17/18, 11:38 AM, "Rawlin Peters" <
>>>>>>>>>>>>>>> rawlin.peters@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We occasionally discuss this issue but
>>>>>>>>> haven't
>>>>>>>>>>>> tackled it
>>>>>>>>>>>>>>>> yet. I think
>>>>>>>>>>>>>>>>> the main issue is just that duplicate
>>>>>>>>> origins
>>>>>>>>>>>> have been
>>>>>>>>>>>>>>>> allowed since
>>>>>>>>>>>>>>>>> the beginning, and now everyone's Traffic
>>>>>>>>> Ops
>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>> littered with
>>>>>>>>>>>>>>>>> duplicate origins. Also, depending on the
>>>>>>>>>>> config
>>>>>>>>>>>> of the
>>>>>>>>>>>>>>>> duplicate
>>>>>>>>>>>>>>>>> delivery services, the origins might not
>>>>>>>>> be in
>>>>>>>>>>>> conflict at
>>>>>>>>>>>>>>>> all (if
>>>>>>>>>>>>>>>>> they don't have different topology
>>>>>>>>>>> constraints).
>>>>>>>>>>>> I would
>>>>>>>>>>>>>>>> love for us
>>>>>>>>>>>>>>>>> to just add a uniqueness constraint, but
>>>>>>>>> there
>>>>>>>>>>>> would need
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> be a fair
>>>>>>>>>>>>>>>>> amount of warning to the community before
>>>>>>>>> doing
>>>>>>>>>>>> so and
>>>>>>>>>>>>>>> might
>>>>>>>>>>>>>>>>> invalidate a significant amount of valid
>>>>>>>>> use
>>>>>>>>>>>> cases.
>>>>>>>>>>>>>>>> Operators would
>>>>>>>>>>>>>>>>> need time to make DNS CNAME records for
>>>>>>>> the
>>>>>>>>>>>> duplicate
>>>>>>>>>>>>>>>> origins and
>>>>>>>>>>>>>>>>> update their DSes to use the different
>>>>>>>>> CNAMEs.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think as a good first step to
>>>>>>>>> eliminating the
>>>>>>>>>>>> use of
>>>>>>>>>>>>>>>> duplicate
>>>>>>>>>>>>>>>>> origins altogether, we should identify
>>>>>>>>> which
>>>>>>>>>>>> "topology
>>>>>>>>>>>>>>>> constraints"
>>>>>>>>>>>>>>>>> actually cause conflicting config when
>>>>>>>> used
>>>>>>>>>>> with
>>>>>>>>>>>> duplicate
>>>>>>>>>>>>>>>> origins and
>>>>>>>>>>>>>>>>> prevent creating DSes with duplicate
>>>>>>>>> origins
>>>>>>>>>>> _if
>>>>>>>>>>>> it would
>>>>>>>>>>>>>>>> cause a
>>>>>>>>>>>>>>>>> conflict with an existing DS that uses
>>>>>>>> the
>>>>>>>>> same
>>>>>>>>>>>> origin_.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> For instance, I believe an HTTP and
>>>>>>>>> DNS-type DS
>>>>>>>>>>>> can live
>>>>>>>>>>>>>>>> happily
>>>>>>>>>>>>>>>>> side-by-side using the same origin
>>>>>>>>> (probably
>>>>>>>>>>>> need different
>>>>>>>>>>>>>>>>> routing_names?), but scenarios like HTTP
>>>>>>>>> and
>>>>>>>>>>>> HTTP_LIVE, or
>>>>>>>>>>>>>>>> DNS and
>>>>>>>>>>>>>>>>> HTTP_NO_CACHE sharing the same origin
>>>>>>>> will
>>>>>>>>>>> cause
>>>>>>>>>>>> conflicts
>>>>>>>>>>>>>>>> for sure.
>>>>>>>>>>>>>>>>> So maybe we can start by making sure the
>>>>>>>> DS
>>>>>>>>>>>> types "match"
>>>>>>>>>>>>>>>> when using
>>>>>>>>>>>>>>>>> the same origin:
>>>>>>>>>>>>>>>>> HTTP + DNS: possibly good, if they have
>>>>>>>>>>>> different routing
>>>>>>>>>>>>>>>> names?
>>>>>>>>>>>>>>>>> HTTP_LIVE + HTTP_LIVE_NATNL: bad
>>>>>>>>>>>>>>>>> HTTP_NO_CACHE + [any other type]: bad
>>>>>>>>>>>>>>>>> HTTP_LIVE + HTTP: bad
>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> There are most likely other conflict
>>>>>>>>> scenarios
>>>>>>>>>>>> that don't
>>>>>>>>>>>>>>>> involve the
>>>>>>>>>>>>>>>>> DS types, but I think this would be a
>>>>>>>> good
>>>>>>>>>>>> start. In the
>>>>>>>>>>>>>>>> future with
>>>>>>>>>>>>>>>>> Delivery Service Topologies (aka Flexible
>>>>>>>>>>>> Cachegroups aka
>>>>>>>>>>>>>>>> Bring Your
>>>>>>>>>>>>>>>>> Own Topology), we might be able to
>>>>>>>> prohibit
>>>>>>>>>>>> assigning a DS
>>>>>>>>>>>>>>>> to a
>>>>>>>>>>>>>>>>> Topology if the DS's origin is already
>>>>>>>>> used by
>>>>>>>>>>>> another DS
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> different Topology.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> - Rawlin
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Dec 17, 2018 at 10:52 AM Fieck,
>>>>>>>>> Brennan
>>>>>>>>>>>>>>>>> <Br...@comcast.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> As some of you may be aware,
>>>>>>>>> `parent.config`
>>>>>>>>>>>> files
>>>>>>>>>>>>>>>> generated by Traffic Ops can vary wildly when an origin
>>>>>>>>> is
>>>>>>>>>>>> assigned to
>>>>>>>>>>>>>>>> multiple Delivery Services. This results in undefined
>>>>>>>>>>>> behavior. I'm told
>>>>>>>>>>>>>>>> that the conflict only happens when two Delivery
>>>>>>>> Services
>>>>>>>>>>> with
>>>>>>>>>>>> different
>>>>>>>>>>>>>>>> "topology requirements" use the same origin, whatever
>>>>>>>>> that
>>>>>>>>>>>> means (content
>>>>>>>>>>>>>>>> routing type?). Regardless, the issue should be
>>>>>>>>> addressed.
>>>>>>>>>>> The
>>>>>>>>>>>> obvious
>>>>>>>>>>>>>>>> solution is to put in place a database constraint that
>>>>>>>>>>>> prevents an origin
>>>>>>>>>>>>>>>> from being assigned to more that one Delivery Service
>>>>>>>>> with
>>>>>>>>>>> API
>>>>>>>>>>>> checks in
>>>>>>>>>>>>>>>> place that would provide helpful error messages when an
>>>>>>>>>>>> attempt is made
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> violate the constraint. However, would that mess with
>>>>>>>>> things
>>>>>>>>>>>> like
>>>>>>>>>>>>>>>> Multi-Site Origin? Or is it just not viable for some
>>>>>>>>> other
>>>>>>>>>>>> reason? If it
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>> a good solution, I'm prepared to work on a fix that
>>>>>>>>> utilizes
>>>>>>>>>>>> it.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>
>>>>
>>>>
>>>
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by John Rushford <jj...@gmail.com>.
origin virtual hosting relies on the Host header. So if the request from the client to the reverse
proxy has a Host header and maintain_pristine_host_hdr is not enabled, remap
will rewrite the Host header along with the request url. So in this case, remapping to
a CNAME should work.
> On Jan 29, 2019, at 6:02 PM, Steve Malenfant <sm...@gmail.com> wrote:
>
> CNAME might not necessary work if the origin is using virtual hosting. It
> would need to be configured as well. Might work with some origin? But not
> all of them.
>
> On Tue, Jan 29, 2019 at 7:40 PM Fieck, Brennan <Br...@comcast.com>
> wrote:
>
>> The issue is inherent with the structure of ATS configuration files. If I
>> understand correctly - someone correct me if I don't -
>> you can't have more than one rule for a "primary destination" in the
>> parent.config. Or, you can, but only one of them will
>> wind up catching all of your matches. Which means that if you have
>> multiple Delivery Services that use the same origin FQDN
>> (dest_domain) then you get proper behavior out of ATS if and only if the
>> resulting rules would be identical. I'd highly
>> recommend that you not move forward using duplicate origin FQDNs on your
>> Delivery Services, because it probably won't
>> do what you want.
>>
>> I'm not sure what you mean about the cachekeys. The Delivery Service URLs
>> should still be unique, so you shouldn't run
>> into a collision in the cache keys.
>>
>> Origins are an object. They're also a cachegroup type. And a profile type.
>> And a server type. And a URL field on a Delivery Service.
>>
>> The solution that's been used in the past to use the same actual origin
>> machine to service multiple Delivery Services safely
>> is to add a "Canonical Name" DNS record to the ATC internal zone for each
>> Delivery Service. The idea is to have a unique name
>> for each Delivery Service, but they all resolve to the same IP address.
>> ________________________________________
>> From: Steve Malenfant <sm...@gmail.com>
>> Sent: Tuesday, January 29, 2019 4:32 PM
>> To: dev@trafficcontrol.apache.org
>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
>> produces indeterminate parent.config
>>
>> We use and will use the same origin for multiple DS. It’s an inherent part
>> of the design of some part of our delivery.
>>
>> Wouldn’t be less of a problem using cachekeys for different delivery
>> services?
>>
>> I would rather see origin as an object and not just a simple URL. I’ve seen
>> this in the past in other systems.
>>
>> I don’t believe we can create multiple http services endpoint for a single
>> DS like we can for DNS type delivery service. This would solve it for us.
>>
>> Steve
>>
>> On Thu, Jan 24, 2019 at 11:22 AM Gray, Jonathan <Jonathan_Gray@comcast.com
>>>
>> wrote:
>>
>>> Even that has a strong element of confusion when in a self-service
>>> multi-tenancy world. Multiple tenants can use the same origins for
>> things
>>> like akami, aws, internal load balancers and any communication about the
>>> origin already being used will be confusing.
>>>
>>> Jonathan G
>>>
>>>
>>> On 1/23/19, 7:37 PM, "Gelinas, Derek" <De...@comcast.com>
>> wrote:
>>>
>>> Perhaps we could solve both issues and just allow it to happen on DS
>>> config but throw up a message when it is saved that says "hey we're going
>>> to do this but it's really not without an element of risk and ats isn't
>>> designed to work properly with multiple identical origins. We recommend
>> you
>>> use an alternative fqdn blah blah blah."
>>>
>>> Something along those lines, anyway.
>>>
>>> Derek
>>>
>>>> On Jan 23, 2019, at 7:18 PM, Rawlin Peters <
>> rawlin.peters@gmail.com>
>>> wrote:
>>>>
>>>> Simply prohibiting duplicate origin FQDNs has been vetoed now, so
>> we
>>>> can't really move forward with that unless the vetoers change their
>>>> minds and remove their -1s.
>>>>
>>>> Currently that leaves us with checking at DS create/update whether
>> or
>>>> not it would conflict with a shared origin due to different DS
>> types,
>>>> qstring config, etc. I am -1 on that idea since I think it will be
>>>> difficult to maintain and enumerate all the possible conflicting
>>>> cases, will present confusing error messages to the user to which
>> the
>>>> solution would just be "create a CNAME record" anyways, and that
>> the
>>>> ATS parent.config should really be per-remap rather than "global".
>> I
>>>> know there has been some discussion in the ATS community about
>> making
>>>> parent.config per-remap, but I don't have much information there.
>>>>
>>>> That is to say, I don't think there is a great solution to this
>>>> problem right now that we can all agree upon, but maybe our efforts
>>>> would be better spent in the ATS community by making a per-remap
>>>> parent.config. Then the duplicate origin problem in Traffic Control
>>>> would go away.
>>>>
>>>> Another option could be to have some kind of generic rules engine
>>> that
>>>> would allow an organization to create some set of rules that DSes
>>>> would have to abide by. For example, no duplicate origins, an
>> origin
>>>> can't be in a list of known malicious domains, you can't use the DS
>>>> type "HTTP_NO_CACHE", maxDnsAnswers must be < 5, TTLs must be > 30
>>> and
>>>> < 6000, protocol must be HTTPS, intitialDispersion must be < 10,
>> DSCP
>>>> must be 40, etc. It seems like for something truly self-service
>> you'd
>>>> probably need to set up some rules for your users to abide by that
>>>> might not necessarily apply to ALL organizations. Just a thought. I
>>>> don't think the duplicate origins problem alone would warrant
>>>> something like that, but if it also provided a solution for the
>>>> self-service effort, maybe it would be worth it.
>>>>
>>>> - Rawlin
>>>>
>>>> On Wed, Jan 23, 2019 at 4:15 AM Gelinas, Derek
>>>> <De...@comcast.com> wrote:
>>>>>
>>>>> That's the problem. We either need to be smart enough to recognize
>>> that in our config and warn the user or just prevent it and suggest to
>> the
>>> user that they use a different fqdn.
>>>>>
>>>>> My vote is for preventing it at this point.
>>>>>
>>>>>> On Jan 23, 2019, at 12:50 AM, Gray, Jonathan <
>>> Jonathan_Gray@comcast.com> wrote:
>>>>>>
>>>>>> It's not just differing types, it's anything that affects the
>>> parent.config which also includes the, qstring configuration, MSO
>>> parents/config, etc. as well.
>>>>>>
>>>>>> Jonathan G
>>>>>>
>>>>>>
>>>>>> On 1/22/19, 2:33 PM, "Dave Neuman" <ne...@apache.org> wrote:
>>>>>>
>>>>>> -1 on the one DS to Origin limitation. I think there are legit
>>> use cases
>>>>>> for it.
>>>>>> I would be +1 on the idea if we include type though. Two DSs
>>> can share the
>>>>>> same origin as long as they are of the same type (DNS, HTTP,
>>> HTTP_LIVE),
>>>>>> etc. You shouldn't be able to have two DSs with the same
>> origin
>>> but
>>>>>> different types.
>>>>>>
>>>>>> On Mon, Jan 14, 2019 at 7:53 AM Fieck, Brennan <
>>> Brennan_Fieck@comcast.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I don't see this as a complicated use case or a barrier to
>> entry,
>>> an
>>>>>>> extremely basic introduction to ATC would only make use of one
>> DS
>>>>>>> (shameless CDN-in-a-Box plug).
>>>>>>> It seems far more complicated for a user to be using many DSes
>> to
>>> serve a
>>>>>>> single origin - we can easily make it clear in the docs that the
>>> two have a
>>>>>>> 1:1 relationship, with a footnote about CNAME records for users
>>> with the
>>>>>>> knowledge, resources, and need for such workarounds.
>>>>>>> ________________________________________
>>>>>>> From: Mark Torluemke <mt...@apache.org>
>>>>>>> Sent: Friday, January 11, 2019 1:21 PM
>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>>> Services
>>>>>>> produces indeterminate parent.config
>>>>>>>
>>>>>>>>> 1. Prohibit creating new delivery services that would share an
>>>>>>> existing origin and prohibit updating a delivery service to a
>>> shared
>>>>>>> origin
>>>>>>> In case my position has been lost, I'm still -1 on this. :) IMO,
>>> we
>>>>>>> shouldn't let the few complicated use cases increase the
>> barrier
>>> to entry
>>>>>>> (to using ATC) for the masses.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jan 11, 2019 at 11:53 AM Rawlin Peters <
>>> rawlin.peters@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Alright, I'm trying to sum up this discussion so far since it
>>> seems
>>>>>>>> like everyone went on vacation and didn't really get a chance
>> to
>>> wrap
>>>>>>>> this one up:
>>>>>>>> - duplicate origins cause undefined behavior
>>>>>>>> - we need a way to migrate to a future that is free of
>> duplicate
>>>>>>>> origins in Traffic Control
>>>>>>>> - we need a visual and easy way to determine if Traffic Ops
>>> currently
>>>>>>>> contains duplicate origins, so that operators are incentivized
>>> to fix
>>>>>>>> them rather than let it slide indefinitely
>>>>>>>> - operators should have a fair amount of time to fix their
>>> duplicate
>>>>>>>> origins
>>>>>>>>
>>>>>>>> I believe this is what we've mostly agreed upon but haven't
>>> clearly voted
>>>>>>>> on:
>>>>>>>>
>>>>>>>> In release N we will:
>>>>>>>> 1. Prohibit creating new delivery services that would share an
>>>>>>>> existing origin and prohibit updating a delivery service to a
>>> shared
>>>>>>>> origin
>>>>>>>> 2. Add some kind of visual indicator that duplicate origins
>> are a
>>>>>>>> problem that need to be fixed before release N+1; otherwise, an
>>>>>>>> upgrade to N+1 will be prohibited.
>>>>>>>>
>>>>>>>> In release N+1 we will:
>>>>>>>> 3. Include a DB migration that adds a uniqueness constraint on
>>> origin
>>>>>>>> FQDN, removing the API-level checks for that.
>>>>>>>> 4. Prevent an upgrade to N+1 if duplicate origins are found
>> (this
>>>>>>>> might occur as a byproduct of step 3).
>>>>>>>>
>>>>>>>> I am +1 on this plan and believe this would hit on all the
>>> summarized
>>>>>>>> points above. Please provide a clear vote on this plan so that
>>> we can
>>>>>>>> dive deeper in the details (i.e. what release 'N' is, the best
>>> visual
>>>>>>>> indicator for step 2, and a friendly way to handle step 4).
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Rawlin
>>>>>>>>
>>>>>>>> On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <
>>> mitchell852@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Not sure TP is the right place for a warning like "clean up
>> this
>>>>>>>>> 'duplicate' origin or your next upgrade will fail". Most users
>>> of our
>>>>>>>>> system will be like "not my problem".
>>>>>>>>>
>>>>>>>>> On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <
>>>>>>>> Brennan_Fieck@comcast.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Probably. It would impact load times by a bit, but the page
>>> for an
>>>>>>>>>> individual object is not our bottleneck.
>>>>>>>>>> ________________________________________
>>>>>>>>>> From: Robert Butts <ro...@apache.org>
>>>>>>>>>> Sent: Wednesday, December 19, 2018 11:50 AM
>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>> Delivery
>>>>>>>> Services
>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>
>>>>>>>>>>> - Including a warning on startup and an API constraint
>>> preventing
>>>>>>>> adding
>>>>>>>>>> more bad data in the next 3.0.0 Release Candidate
>>>>>>>>>>> - Adding a database constraint immediately into master that
>>> won't
>>>>>>> be
>>>>>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
>>>>>>>>>>
>>>>>>>>>> +1
>>>>>>>>>>
>>>>>>>>>> I understand Jonathan's objection, but at some point, we have
>>> to be
>>>>>>>> able to
>>>>>>>>>> move forward. This is a good compromise: deprecate, then
>>> remove. That
>>>>>>>> gives
>>>>>>>>>> people a full major version to fix their data.
>>>>>>>>>>
>>>>>>>>>> I would be ideal if it were more than just a logged warning,
>>> though.
>>>>>>>> Can we
>>>>>>>>>> add a big red banner in Traffic Portal, on the Delivery
>>> Service page
>>>>>>>> for
>>>>>>>>>> any DS with a duplicate origin, telling users to fix it, and
>>> that
>>>>>>> they
>>>>>>>>>> won't be able to upgrade to the next major version until it's
>>> fixed?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <
>>>>>>>> Brennan_Fieck@comcast.com
>>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> So it seems like nobody has a problem with the "how" -
>>> disallowing
>>>>>>>>>>> duplicate origin FQDNs on Delivery Services - but we never
>>> reached
>>>>>>> a
>>>>>>>>>>> consensus on "when".
>>>>>>>>>>>
>>>>>>>>>>> I stand by my previous proposal:
>>>>>>>>>>> - Including a warning on startup and an API constraint
>>> preventing
>>>>>>>> adding
>>>>>>>>>>> more bad data in the next 3.0.0 Release Candidate
>>>>>>>>>>> - Adding a database constraint immediately into master that
>>> won't
>>>>>>> be
>>>>>>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
>>>>>>>>>>> ________________________________________
>>>>>>>>>>> From: Rawlin Peters <ra...@gmail.com>
>>>>>>>>>>> Sent: Tuesday, December 18, 2018 4:59 PM
>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>> Delivery
>>>>>>>> Services
>>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>>
>>>>>>>>>>> Also, building more around DS types will make it even harder
>>> to get
>>>>>>>>>>> away from DS types in the future too, which I know is
>>> something
>>>>>>> we've
>>>>>>>>>>> discussed on this mailing list before. It also adds to the
>>> overhead
>>>>>>>> of
>>>>>>>>>>> Delivery Service Topologies, since a lot of the DS types
>> won't
>>>>>>>>>>> carryover into that world.
>>>>>>>>>>>
>>>>>>>>>>> - Rawlin
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
>>>>>>>>>>> <Br...@comcast.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> +1.
>>>>>>>>>>>> If there's a simple way to work around duplicate origins
>>> being
>>>>>>>>>>> prohibited,
>>>>>>>>>>>> then we should rely on that instead of "enumerating all
>> those
>>>>>>>> possible
>>>>>>>>>>> conflicting
>>>>>>>>>>>> settings, which are not only highly complex and confusing,
>>> but
>>>>>>> also
>>>>>>>>>>> further
>>>>>>>>>>>> entrench us in only supporting ATS as a caching proxy
>>> (hurting
>>>>>>>> efforts
>>>>>>>>>> to
>>>>>>>>>>>> integrate e.g. Grove, nginx etc.)
>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>> From: Rawlin Peters <ra...@gmail.com>
>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 2:20 PM
>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>> Delivery
>>>>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>>>>
>>>>>>>>>>>> There are a number of different DS settings at play that
>> can
>>>>>>>>>>>> potentially cause conflicts. The question is: do we want to
>>> get
>>>>>>>> into
>>>>>>>>>>>> the business of enumerating all those possible conflicting
>>>>>>>> settings or
>>>>>>>>>>>> just simply prohibit duplicate origins altogether? I think
>>> we can
>>>>>>>> dig
>>>>>>>>>>>> in and get that "sufficiently advanced sql query" to check
>>> for
>>>>>>>>>>>> conflicting origins, but is that something we want to carry
>>> along
>>>>>>>> for
>>>>>>>>>>>> the foreseeable future? Aren't CNAMEs relatively cheaper
>> than
>>>>>>>>>>>> developing and maintaining that code and the mental
>> overhead
>>>>>>>> required
>>>>>>>>>>>> in understanding why you're getting an error that says your
>>>>>>>> requested
>>>>>>>>>>>> DS would cause an origin conflict? I think at the point
>>> you've
>>>>>>>>>>>> requested a DS that would create a conflict, you've chosen
>>> those
>>>>>>>>>>>> settings for a reason and would probably prefer to just
>>>>>>> create/use
>>>>>>>> a
>>>>>>>>>>>> CNAME in your new DS and keep the rest of your settings the
>>> same.
>>>>>>>>>>>>
>>>>>>>>>>>> Thinking in terms of errors, I'm imagining:
>>>>>>>>>>>> "cannot create delivery service: origin fqdn '
>>> foo.example.com'
>>>>>>>> already
>>>>>>>>>>> in use"
>>>>>>>>>>>> vs
>>>>>>>>>>>> "cannot create delivery service: origin fqdn '
>>> foo.example.com'
>>>>>>>> already
>>>>>>>>>>>> in use as type DNS_LIVE_NATNL, which is incompatible with
>>> your
>>>>>>>> chosen
>>>>>>>>>>>> type of HTTP_NO_CACHE"
>>>>>>>>>>>>
>>>>>>>>>>>> At that point you'd probably say to yourself, "uh, I need
>>>>>>>>>>>> HTTP_NO_CACHE, so what am I supposed to do now?"
>>>>>>>>>>>>
>>>>>>>>>>>> As a lazy developer I'm +1 on prohibiting duplicate origin
>>> fqdns
>>>>>>>>>>>> because the resulting code will be simpler, but I think
>>>>>>> eliminating
>>>>>>>>>>>> the mental overhead for operators could be worthwhile too.
>>> If we
>>>>>>>> can
>>>>>>>>>>>> agree on an end state of prohibiting duplicate origins
>>>>>>> altogether,
>>>>>>>> we
>>>>>>>>>>>> can start working on a design to smoothly transition us to
>>> that
>>>>>>>> point.
>>>>>>>>>>>> Are we willing to live with "just CNAME your origin fqdn"
>> as
>>> the
>>>>>>>>>>>> standard solution to duplicates?
>>>>>>>>>>>>
>>>>>>>>>>>> - Rawlin
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
>>>>>>>>>>>> <De...@comcast.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> The only situation in which they can share origins is if
>> a)
>>> the
>>>>>>>>>>> origins are shared in an MSO configuration but still have
>>> different
>>>>>>>>>> defined
>>>>>>>>>>> origin fields in the delivery service, or if they're
>> assigned
>>> to
>>>>>>>>>> completely
>>>>>>>>>>> different cachegroups. It's when two delivery services
>> share
>>> the
>>>>>>>> same
>>>>>>>>>>> edges that there's an issue, because you end up with
>>> parent.config
>>>>>>>>>> issues.
>>>>>>>>>>> Actually you could even get away with it in mids as long as
>>> you
>>>>>>>> weren't
>>>>>>>>>>> doing anything like MSO to it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Could get messy real fast, though. Best to just create a
>>>>>>> second
>>>>>>>>>> FQDN.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Derek
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 12/18/18, 3:23 PM, "Fieck, Brennan" <
>>>>>>>> Brennan_Fieck@comcast.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> So no two Delivery Services may share an origin
>>> *regardless
>>>>>>>> of
>>>>>>>>>>> cache hierarchy* ? I've been told that DNS Delivery Services
>>> can
>>>>>>>> have the
>>>>>>>>>>> same origin as HTTP Delivery Services because they obey the
>>> same
>>>>>>>> cache
>>>>>>>>>>> hierarchy. You're saying that would still produce invalid
>>> output
>>>>>>>> and/or
>>>>>>>>>> is
>>>>>>>>>>> explicitly disallowed by ATS?
>>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>>> From: Robert Butts <ro...@apache.org>
>>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 1:09 PM
>>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>>>>> Delivery
>>>>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>>>>>
>>>>>>>>>>>>>> can you give an example of what parent.config looks like
>>>>>>>> when 2
>>>>>>>>>>> ds's share
>>>>>>>>>>>>> an origin and have different a different topology?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Answering because I encountered this directly, when
>>>>>>> rewriting
>>>>>>>>>>> parent.config.
>>>>>>>>>>>>>
>>>>>>>>>>>>> For example: Suppose you have one Delivery Service:
>>>>>>>>>>>>> XML_ID: foo
>>>>>>>>>>>>> Type: HTPT_LIVE_NATL
>>>>>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass
>>> up
>>>>>>>>>>>>> Origin Server Base URL: http://foo.example.net
>>>>>>>>>>>>>
>>>>>>>>>>>>> And another Delivery Service:
>>>>>>>>>>>>> XML_ID: bar
>>>>>>>>>>>>> Type: HTPT_LIVE_NATL
>>>>>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass
>>> up
>>>>>>>>>>>>> Origin Server Base URL: http://foo.example.net
>>>>>>>>>>>>>
>>>>>>>>>>>>> ATS only supports unique `dest_domain` entries in
>>>>>>>> parent.config.
>>>>>>>>>>> Therefore,
>>>>>>>>>>>>> the parent.config generated for a server assigned to
>> both
>>>>>>> of
>>>>>>>>>> these
>>>>>>>>>>> Delivery
>>>>>>>>>>>>> Services with either be:
>>>>>>>>>>>>>
>>>>>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=true
>>>>>>>>>>>>>
>>>>>>>>>>>>> Or
>>>>>>>>>>>>>
>>>>>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=false
>>>>>>>>>>> qstring=consider
>>>>>>>>>>>>>
>>>>>>>>>>>>> Right now, it's arbitrary which Perl Traffic Ops
>> inserts,
>>>>>>> and
>>>>>>>>>> Perl
>>>>>>>>>>> provides
>>>>>>>>>>>>> no warning or error of any kind (the pending Go
>>>>>>>> parent.config PR
>>>>>>>>>>> logs an
>>>>>>>>>>>>> error).
>>>>>>>>>>>>>
>>>>>>>>>>>>> Whichever is arbitrarily inserted, the resulting remaps
>>> for
>>>>>>>> the
>>>>>>>>>>> other
>>>>>>>>>>>>> delivery service will be wrong. Either `foo` requests
>> will
>>>>>>>> drop
>>>>>>>>>>> the query
>>>>>>>>>>>>> string when they shouldn't, and go to the mid when they
>>>>>>>>>> shouldn't;
>>>>>>>>>>> or `bar`
>>>>>>>>>>>>> requests will use the query string and skip the mid when
>>> it
>>>>>>>>>>> shouldn't.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Does that make sense? The only correct solution, is to
>>>>>>>> somehow
>>>>>>>>>>> prevent
>>>>>>>>>>>>> different DSes having the same origin, and tell tenants
>>>>>>> they
>>>>>>>> must
>>>>>>>>>>> use
>>>>>>>>>>>>> CNAMEs if they need.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This isn't a bug in Traffic Control. ATS fundamentally
>>>>>>>> doesn't
>>>>>>>>>>> support
>>>>>>>>>>>>> multiple remap rules with the same parent FQDN with
>>>>>>> different
>>>>>>>>>>>>> configurations. Hence, Traffic Control needs to prohibit
>>>>>>>> that.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
>>>>>>>>>>> mitchell852@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> brennan,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> can you give an example of what parent.config looks like
>>>>>>>> when 2
>>>>>>>>>>> ds's share
>>>>>>>>>>>>>> an origin and have different a different topology?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> jeremy
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
>>>>>>>>>>> Brennan_Fieck@comcast.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To be clear, the "Warning" I'm talking about would
>>>>>>>> happen at
>>>>>>>>>>> startup, but
>>>>>>>>>>>>>>> I'd like a UI-only constraint to come with that to
>>>>>>>> disallow
>>>>>>>>>>> using the API
>>>>>>>>>>>>>>> to bind the same origin to multiple Delivery Services
>>>>>>>> with
>>>>>>>>>>> varying
>>>>>>>>>>>>>>> topography requirements. It wouldn't change the
>>>>>>> existing
>>>>>>>>>> data,
>>>>>>>>>>> but
>>>>>>>>>>>>>> prevent
>>>>>>>>>>>>>>> users from creating more bad data.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> "warning" doesn't really sufficiently describe that, my
>>>>>>>> bad.
>>>>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>>>>> From: Fieck, Brennan <Br...@comcast.com>
>>>>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 11:24 AM
>>>>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>>>>>>>> Delivery Services
>>>>>>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Well the cost of fixing this bug is a constraint on the
>>>>>>>> data.
>>>>>>>>>>> Unless we
>>>>>>>>>>>>>>> make it a UI-only constraint - which I'm personally
>>>>>>>> against -
>>>>>>>>>>> there must
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>> some point in the future where ATC cannot reasonably be
>>>>>>>>>>> expected to work
>>>>>>>>>>>>>>> with data that violates that constraint. The question
>>>>>>> is
>>>>>>>> when
>>>>>>>>>>> that should
>>>>>>>>>>>>>>> occur, which should likely happen at a minor version
>>>>>>>> release.
>>>>>>>>>>> Minor not
>>>>>>>>>>>>>>> major because it doesn't involve a change in data
>>>>>>>> structures,
>>>>>>>>>>> merely
>>>>>>>>>>>>>>> relationships between them - in my opinion that's a
>>>>>>> minor
>>>>>>>>>>> version change
>>>>>>>>>>>>>>> but that's definitely up for debate. With several
>>>>>>> release
>>>>>>>>>>> candidates for
>>>>>>>>>>>>>>> 3.0.0 that _doesn't_ include this restriction already
>>>>>>> in
>>>>>>>> the
>>>>>>>>>>> wild, I
>>>>>>>>>>>>>>> wouldn't recommend putting it in there. That means to
>>>>>>>> fix the
>>>>>>>>>>> bug as soon
>>>>>>>>>>>>>>> as possible it should go in 3.1.0 which should be the
>>>>>>>> target
>>>>>>>>>>> of "master"
>>>>>>>>>>>>>>> after the 3.0.0 release is cut from it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> So I'd recommend immediately implementing the
>>>>>>> constraint
>>>>>>>> in
>>>>>>>>>>> master with a
>>>>>>>>>>>>>>> refusal to upgrade with bad data, and backport a
>>>>>>> warning
>>>>>>>>>> about
>>>>>>>>>>> the future
>>>>>>>>>>>>>>> behavior into 3.0.0 or as part of a 3.0.1 provided we
>>>>>>> had
>>>>>>>>>> more
>>>>>>>>>>> changes
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> would warrant a micro version bump.
>>>>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
>>>>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 9:34 AM
>>>>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>>>>>>>> Delivery Services
>>>>>>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -1 Holding an ATC upgrade hostage to data cleanup seems
>>>>>>>> like
>>>>>>>>>> a
>>>>>>>>>>> bad idea.
>>>>>>>>>>>>>>> The issue isn't great, but it's also not new. We
>>>>>>> should
>>>>>>>>>> allow
>>>>>>>>>>> teams to
>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>> their data at their normal paces if it doesn't create
>>>>>>>>>>> significant
>>>>>>>>>>>>>> overhead
>>>>>>>>>>>>>>> or an inherant blocker for new functionality or
>>>>>>>> correction of
>>>>>>>>>>> other major
>>>>>>>>>>>>>>> problems imho.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Jonathan G
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 12/18/18, 9:28 AM, "Fieck, Brennan" <
>>>>>>>>>>> Brennan_Fieck@comcast.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Another option is we could detect collisions at
>>>>>>>> startup
>>>>>>>>>>> and simply
>>>>>>>>>>>>>>> refuse to continue with the upgrade until the data is
>>>>>>>> fixed.
>>>>>>>>>>> That would
>>>>>>>>>>>>>>> allow people using the now-unsupported data format to
>>>>>>>>>> continue
>>>>>>>>>>> to use
>>>>>>>>>>>>>> their
>>>>>>>>>>>>>>> old versions of Traffic Ops without wrecking their
>>>>>>>> database,
>>>>>>>>>>> but also
>>>>>>>>>>>>>>> provide an incentive to clean up the data.
>>>>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
>>>>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 5:12 AM
>>>>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to
>>>>>>>> Multipe
>>>>>>>>>>> Delivery
>>>>>>>>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm generally a fan of constrain your data in your
>>>>>>>>>>> database, but not
>>>>>>>>>>>>>>> necessarily exclusively. I see this as a one-way
>>>>>>>>>>> cleanup/conversion so
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> doesn't need to be configurable; otherwise you have to
>>>>>>>> ask
>>>>>>>>>> the
>>>>>>>>>>> question
>>>>>>>>>>>>>>> what happens if someone turns it off. That said,
>>>>>>>> something
>>>>>>>>>> in
>>>>>>>>>>> the UI
>>>>>>>>>>>>>> layer
>>>>>>>>>>>>>>> would be nice to prevent spending significant
>>>>>>> quantities
>>>>>>>> of
>>>>>>>>>>> time
>>>>>>>>>>>>>> building a
>>>>>>>>>>>>>>> complex DS only to have it fail to post for reasons
>>>>>>> that
>>>>>>>>>> could
>>>>>>>>>>> have been
>>>>>>>>>>>>>>> known earlier.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The way my brain works in this case:
>>>>>>>>>>>>>>> If !unique_constraint_exists_query()
>>>>>>>>>>>>>>> If has_duplicates_query()
>>>>>>>>>>>>>>> show_warning()
>>>>>>>>>>>>>>> else
>>>>>>>>>>>>>>> add_unique_constraint()
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> to which the API and UI configuration could also
>>>>>>>> make use
>>>>>>>>>>> of
>>>>>>>>>>>>>>> unique_constraint_exists_query() to drive additional
>>>>>>>> layer
>>>>>>>>>>> constraints if
>>>>>>>>>>>>>>> desired.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Jonathan G
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 12/17/18, 1:11 PM, "Rawlin Peters" <
>>>>>>>>>>> rawlin.peters@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> That is an interesting idea...detect at TO
>>>>>>>> startup
>>>>>>>>>>> whether or not
>>>>>>>>>>>>>>> there are duplicate origins and operate in a
>>>>>>>> "prevent
>>>>>>>>>>> duplicate
>>>>>>>>>>>>>>> origins" state if no duplicates are found or
>>>>>>>> "prevent
>>>>>>>>>>> conflicting
>>>>>>>>>>>>>>> DS
>>>>>>>>>>>>>>> topologies" state if duplicates are found? So
>>>>>>>> once
>>>>>>>>>>> operators have
>>>>>>>>>>>>>>> replaced all the duplicate origins with CNAMEs,
>>>>>>>> TO
>>>>>>>>>> will
>>>>>>>>>>>>>> essentially
>>>>>>>>>>>>>>> operate in a "prohibit all duplicate origins"
>>>>>>>> state.
>>>>>>>>>>> That would
>>>>>>>>>>>>>>> probably make for a simpler transition, but I'd
>>>>>>>> want
>>>>>>>>>>> to remove
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> logic in a following release that strictly
>>>>>>>> prohibits
>>>>>>>>>>> duplicate
>>>>>>>>>>>>>>> origins
>>>>>>>>>>>>>>> (assuming that the community agrees we should
>>>>>>>>>> prohibit
>>>>>>>>>>> duplicate
>>>>>>>>>>>>>>> origins altogether).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As for DB constraints vs UI, I was thinking
>>>>>>> those
>>>>>>>>>>> DS-type
>>>>>>>>>>>>>>> constraints
>>>>>>>>>>>>>>> I pointed out would live in the API. It would
>>>>>>>>>>> basically be added
>>>>>>>>>>>>>>> validation in the deliveryservices POST/PUT
>>>>>>>> endpoint
>>>>>>>>>>> that checks
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> DB for existing DSes that conflict with the
>>>>>>>> requested
>>>>>>>>>>> DS.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Rawlin
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
>>>>>>>>>>>>>>> <Jo...@comcast.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> These kinds of conditions should be
>>>>>>> detectable
>>>>>>>>>> with a
>>>>>>>>>>>>>>> sufficiently advanced SQL query. Is it possible to add
>>>>>>>> the
>>>>>>>>>>> constraint if
>>>>>>>>>>>>>>> it passes and emit a warning during TO startup
>>>>>>> otherwise?
>>>>>>>>>>> That would let
>>>>>>>>>>>>>>> you know the condition exists at startup but not
>>>>>>> getting
>>>>>>>> in
>>>>>>>>>>> your way and
>>>>>>>>>>>>>>> keep you out of trouble once you've cleaned up. We
>>>>>>> made
>>>>>>>> a
>>>>>>>>>>> mistake early
>>>>>>>>>>>>>>> on, but this would acknowledge it was bad and encourage
>>>>>>>> it to
>>>>>>>>>>> be fixed at
>>>>>>>>>>>>>>> the speed of operations teams. Also this puts the
>>>>>>>> constraint
>>>>>>>>>>> in the
>>>>>>>>>>>>>>> database rather than the UI which is really where the
>>>>>>>>>>> contention is for
>>>>>>>>>>>>>>> usability.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Jonathan G
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 12/17/18, 11:38 AM, "Rawlin Peters" <
>>>>>>>>>>>>>> rawlin.peters@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We occasionally discuss this issue but
>>>>>>>> haven't
>>>>>>>>>>> tackled it
>>>>>>>>>>>>>>> yet. I think
>>>>>>>>>>>>>>>> the main issue is just that duplicate
>>>>>>>> origins
>>>>>>>>>>> have been
>>>>>>>>>>>>>>> allowed since
>>>>>>>>>>>>>>>> the beginning, and now everyone's Traffic
>>>>>>>> Ops
>>>>>>>>>>> could be
>>>>>>>>>>>>>>> littered with
>>>>>>>>>>>>>>>> duplicate origins. Also, depending on the
>>>>>>>>>> config
>>>>>>>>>>> of the
>>>>>>>>>>>>>>> duplicate
>>>>>>>>>>>>>>>> delivery services, the origins might not
>>>>>>>> be in
>>>>>>>>>>> conflict at
>>>>>>>>>>>>>>> all (if
>>>>>>>>>>>>>>>> they don't have different topology
>>>>>>>>>> constraints).
>>>>>>>>>>> I would
>>>>>>>>>>>>>>> love for us
>>>>>>>>>>>>>>>> to just add a uniqueness constraint, but
>>>>>>>> there
>>>>>>>>>>> would need
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> be a fair
>>>>>>>>>>>>>>>> amount of warning to the community before
>>>>>>>> doing
>>>>>>>>>>> so and
>>>>>>>>>>>>>> might
>>>>>>>>>>>>>>>> invalidate a significant amount of valid
>>>>>>>> use
>>>>>>>>>>> cases.
>>>>>>>>>>>>>>> Operators would
>>>>>>>>>>>>>>>> need time to make DNS CNAME records for
>>>>>>> the
>>>>>>>>>>> duplicate
>>>>>>>>>>>>>>> origins and
>>>>>>>>>>>>>>>> update their DSes to use the different
>>>>>>>> CNAMEs.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I think as a good first step to
>>>>>>>> eliminating the
>>>>>>>>>>> use of
>>>>>>>>>>>>>>> duplicate
>>>>>>>>>>>>>>>> origins altogether, we should identify
>>>>>>>> which
>>>>>>>>>>> "topology
>>>>>>>>>>>>>>> constraints"
>>>>>>>>>>>>>>>> actually cause conflicting config when
>>>>>>> used
>>>>>>>>>> with
>>>>>>>>>>> duplicate
>>>>>>>>>>>>>>> origins and
>>>>>>>>>>>>>>>> prevent creating DSes with duplicate
>>>>>>>> origins
>>>>>>>>>> _if
>>>>>>>>>>> it would
>>>>>>>>>>>>>>> cause a
>>>>>>>>>>>>>>>> conflict with an existing DS that uses
>>>>>>> the
>>>>>>>> same
>>>>>>>>>>> origin_.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> For instance, I believe an HTTP and
>>>>>>>> DNS-type DS
>>>>>>>>>>> can live
>>>>>>>>>>>>>>> happily
>>>>>>>>>>>>>>>> side-by-side using the same origin
>>>>>>>> (probably
>>>>>>>>>>> need different
>>>>>>>>>>>>>>>> routing_names?), but scenarios like HTTP
>>>>>>>> and
>>>>>>>>>>> HTTP_LIVE, or
>>>>>>>>>>>>>>> DNS and
>>>>>>>>>>>>>>>> HTTP_NO_CACHE sharing the same origin
>>>>>>> will
>>>>>>>>>> cause
>>>>>>>>>>> conflicts
>>>>>>>>>>>>>>> for sure.
>>>>>>>>>>>>>>>> So maybe we can start by making sure the
>>>>>>> DS
>>>>>>>>>>> types "match"
>>>>>>>>>>>>>>> when using
>>>>>>>>>>>>>>>> the same origin:
>>>>>>>>>>>>>>>> HTTP + DNS: possibly good, if they have
>>>>>>>>>>> different routing
>>>>>>>>>>>>>>> names?
>>>>>>>>>>>>>>>> HTTP_LIVE + HTTP_LIVE_NATNL: bad
>>>>>>>>>>>>>>>> HTTP_NO_CACHE + [any other type]: bad
>>>>>>>>>>>>>>>> HTTP_LIVE + HTTP: bad
>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There are most likely other conflict
>>>>>>>> scenarios
>>>>>>>>>>> that don't
>>>>>>>>>>>>>>> involve the
>>>>>>>>>>>>>>>> DS types, but I think this would be a
>>>>>>> good
>>>>>>>>>>> start. In the
>>>>>>>>>>>>>>> future with
>>>>>>>>>>>>>>>> Delivery Service Topologies (aka Flexible
>>>>>>>>>>> Cachegroups aka
>>>>>>>>>>>>>>> Bring Your
>>>>>>>>>>>>>>>> Own Topology), we might be able to
>>>>>>> prohibit
>>>>>>>>>>> assigning a DS
>>>>>>>>>>>>>>> to a
>>>>>>>>>>>>>>>> Topology if the DS's origin is already
>>>>>>>> used by
>>>>>>>>>>> another DS
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> different Topology.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> - Rawlin
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Dec 17, 2018 at 10:52 AM Fieck,
>>>>>>>> Brennan
>>>>>>>>>>>>>>>> <Br...@comcast.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> As some of you may be aware,
>>>>>>>> `parent.config`
>>>>>>>>>>> files
>>>>>>>>>>>>>>> generated by Traffic Ops can vary wildly when an origin
>>>>>>>> is
>>>>>>>>>>> assigned to
>>>>>>>>>>>>>>> multiple Delivery Services. This results in undefined
>>>>>>>>>>> behavior. I'm told
>>>>>>>>>>>>>>> that the conflict only happens when two Delivery
>>>>>>> Services
>>>>>>>>>> with
>>>>>>>>>>> different
>>>>>>>>>>>>>>> "topology requirements" use the same origin, whatever
>>>>>>>> that
>>>>>>>>>>> means (content
>>>>>>>>>>>>>>> routing type?). Regardless, the issue should be
>>>>>>>> addressed.
>>>>>>>>>> The
>>>>>>>>>>> obvious
>>>>>>>>>>>>>>> solution is to put in place a database constraint that
>>>>>>>>>>> prevents an origin
>>>>>>>>>>>>>>> from being assigned to more that one Delivery Service
>>>>>>>> with
>>>>>>>>>> API
>>>>>>>>>>> checks in
>>>>>>>>>>>>>>> place that would provide helpful error messages when an
>>>>>>>>>>> attempt is made
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> violate the constraint. However, would that mess with
>>>>>>>> things
>>>>>>>>>>> like
>>>>>>>>>>>>>>> Multi-Site Origin? Or is it just not viable for some
>>>>>>>> other
>>>>>>>>>>> reason? If it
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> a good solution, I'm prepared to work on a fix that
>>>>>>>> utilizes
>>>>>>>>>>> it.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>
>>>
>>>
>>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Gray, Jonathan" <Jo...@comcast.com>.
A further clarification is that the contention I believe is on the "Base URL" delivery service field which gets converted to the primary parent in the parent.config unless you're using MSO maybe. Those duplicated base URLs would get conflated and in the case where any of the parent.config fields vary you would see inconsistent config file generation. So there are some cases where it's still fine as long as you know it's out there waiting to bite you later. If you're using vhosts I think that might solve the issue similarly to how the usage of CNames also fixes it by way of disambiguating the base url field. If you promise that all the parent.config fields are identical such as the ds type, qstring, etc; it will still work as it always has.
Jonathan G
On 1/29/19, 6:03 PM, "Steve Malenfant" <sm...@gmail.com> wrote:
CNAME might not necessary work if the origin is using virtual hosting. It
would need to be configured as well. Might work with some origin? But not
all of them.
On Tue, Jan 29, 2019 at 7:40 PM Fieck, Brennan <Br...@comcast.com>
wrote:
> The issue is inherent with the structure of ATS configuration files. If I
> understand correctly - someone correct me if I don't -
> you can't have more than one rule for a "primary destination" in the
> parent.config. Or, you can, but only one of them will
> wind up catching all of your matches. Which means that if you have
> multiple Delivery Services that use the same origin FQDN
> (dest_domain) then you get proper behavior out of ATS if and only if the
> resulting rules would be identical. I'd highly
> recommend that you not move forward using duplicate origin FQDNs on your
> Delivery Services, because it probably won't
> do what you want.
>
> I'm not sure what you mean about the cachekeys. The Delivery Service URLs
> should still be unique, so you shouldn't run
> into a collision in the cache keys.
>
> Origins are an object. They're also a cachegroup type. And a profile type.
> And a server type. And a URL field on a Delivery Service.
>
> The solution that's been used in the past to use the same actual origin
> machine to service multiple Delivery Services safely
> is to add a "Canonical Name" DNS record to the ATC internal zone for each
> Delivery Service. The idea is to have a unique name
> for each Delivery Service, but they all resolve to the same IP address.
> ________________________________________
> From: Steve Malenfant <sm...@gmail.com>
> Sent: Tuesday, January 29, 2019 4:32 PM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> produces indeterminate parent.config
>
> We use and will use the same origin for multiple DS. It’s an inherent part
> of the design of some part of our delivery.
>
> Wouldn’t be less of a problem using cachekeys for different delivery
> services?
>
> I would rather see origin as an object and not just a simple URL. I’ve seen
> this in the past in other systems.
>
> I don’t believe we can create multiple http services endpoint for a single
> DS like we can for DNS type delivery service. This would solve it for us.
>
> Steve
>
> On Thu, Jan 24, 2019 at 11:22 AM Gray, Jonathan <Jonathan_Gray@comcast.com
> >
> wrote:
>
> > Even that has a strong element of confusion when in a self-service
> > multi-tenancy world. Multiple tenants can use the same origins for
> things
> > like akami, aws, internal load balancers and any communication about the
> > origin already being used will be confusing.
> >
> > Jonathan G
> >
> >
> > On 1/23/19, 7:37 PM, "Gelinas, Derek" <De...@comcast.com>
> wrote:
> >
> > Perhaps we could solve both issues and just allow it to happen on DS
> > config but throw up a message when it is saved that says "hey we're going
> > to do this but it's really not without an element of risk and ats isn't
> > designed to work properly with multiple identical origins. We recommend
> you
> > use an alternative fqdn blah blah blah."
> >
> > Something along those lines, anyway.
> >
> > Derek
> >
> > > On Jan 23, 2019, at 7:18 PM, Rawlin Peters <
> rawlin.peters@gmail.com>
> > wrote:
> > >
> > > Simply prohibiting duplicate origin FQDNs has been vetoed now, so
> we
> > > can't really move forward with that unless the vetoers change their
> > > minds and remove their -1s.
> > >
> > > Currently that leaves us with checking at DS create/update whether
> or
> > > not it would conflict with a shared origin due to different DS
> types,
> > > qstring config, etc. I am -1 on that idea since I think it will be
> > > difficult to maintain and enumerate all the possible conflicting
> > > cases, will present confusing error messages to the user to which
> the
> > > solution would just be "create a CNAME record" anyways, and that
> the
> > > ATS parent.config should really be per-remap rather than "global".
> I
> > > know there has been some discussion in the ATS community about
> making
> > > parent.config per-remap, but I don't have much information there.
> > >
> > > That is to say, I don't think there is a great solution to this
> > > problem right now that we can all agree upon, but maybe our efforts
> > > would be better spent in the ATS community by making a per-remap
> > > parent.config. Then the duplicate origin problem in Traffic Control
> > > would go away.
> > >
> > > Another option could be to have some kind of generic rules engine
> > that
> > > would allow an organization to create some set of rules that DSes
> > > would have to abide by. For example, no duplicate origins, an
> origin
> > > can't be in a list of known malicious domains, you can't use the DS
> > > type "HTTP_NO_CACHE", maxDnsAnswers must be < 5, TTLs must be > 30
> > and
> > > < 6000, protocol must be HTTPS, intitialDispersion must be < 10,
> DSCP
> > > must be 40, etc. It seems like for something truly self-service
> you'd
> > > probably need to set up some rules for your users to abide by that
> > > might not necessarily apply to ALL organizations. Just a thought. I
> > > don't think the duplicate origins problem alone would warrant
> > > something like that, but if it also provided a solution for the
> > > self-service effort, maybe it would be worth it.
> > >
> > > - Rawlin
> > >
> > > On Wed, Jan 23, 2019 at 4:15 AM Gelinas, Derek
> > > <De...@comcast.com> wrote:
> > >>
> > >> That's the problem. We either need to be smart enough to recognize
> > that in our config and warn the user or just prevent it and suggest to
> the
> > user that they use a different fqdn.
> > >>
> > >> My vote is for preventing it at this point.
> > >>
> > >>> On Jan 23, 2019, at 12:50 AM, Gray, Jonathan <
> > Jonathan_Gray@comcast.com> wrote:
> > >>>
> > >>> It's not just differing types, it's anything that affects the
> > parent.config which also includes the, qstring configuration, MSO
> > parents/config, etc. as well.
> > >>>
> > >>> Jonathan G
> > >>>
> > >>>
> > >>> On 1/22/19, 2:33 PM, "Dave Neuman" <ne...@apache.org> wrote:
> > >>>
> > >>> -1 on the one DS to Origin limitation. I think there are legit
> > use cases
> > >>> for it.
> > >>> I would be +1 on the idea if we include type though. Two DSs
> > can share the
> > >>> same origin as long as they are of the same type (DNS, HTTP,
> > HTTP_LIVE),
> > >>> etc. You shouldn't be able to have two DSs with the same
> origin
> > but
> > >>> different types.
> > >>>
> > >>> On Mon, Jan 14, 2019 at 7:53 AM Fieck, Brennan <
> > Brennan_Fieck@comcast.com>
> > >>> wrote:
> > >>>
> > >>>> I don't see this as a complicated use case or a barrier to
> entry,
> > an
> > >>>> extremely basic introduction to ATC would only make use of one
> DS
> > >>>> (shameless CDN-in-a-Box plug).
> > >>>> It seems far more complicated for a user to be using many DSes
> to
> > serve a
> > >>>> single origin - we can easily make it clear in the docs that the
> > two have a
> > >>>> 1:1 relationship, with a footnote about CNAME records for users
> > with the
> > >>>> knowledge, resources, and need for such workarounds.
> > >>>> ________________________________________
> > >>>> From: Mark Torluemke <mt...@apache.org>
> > >>>> Sent: Friday, January 11, 2019 1:21 PM
> > >>>> To: dev@trafficcontrol.apache.org
> > >>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > Services
> > >>>> produces indeterminate parent.config
> > >>>>
> > >>>>>> 1. Prohibit creating new delivery services that would share an
> > >>>> existing origin and prohibit updating a delivery service to a
> > shared
> > >>>> origin
> > >>>> In case my position has been lost, I'm still -1 on this. :) IMO,
> > we
> > >>>> shouldn't let the few complicated use cases increase the
> barrier
> > to entry
> > >>>> (to using ATC) for the masses.
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Fri, Jan 11, 2019 at 11:53 AM Rawlin Peters <
> > rawlin.peters@gmail.com>
> > >>>> wrote:
> > >>>>
> > >>>>> Alright, I'm trying to sum up this discussion so far since it
> > seems
> > >>>>> like everyone went on vacation and didn't really get a chance
> to
> > wrap
> > >>>>> this one up:
> > >>>>> - duplicate origins cause undefined behavior
> > >>>>> - we need a way to migrate to a future that is free of
> duplicate
> > >>>>> origins in Traffic Control
> > >>>>> - we need a visual and easy way to determine if Traffic Ops
> > currently
> > >>>>> contains duplicate origins, so that operators are incentivized
> > to fix
> > >>>>> them rather than let it slide indefinitely
> > >>>>> - operators should have a fair amount of time to fix their
> > duplicate
> > >>>>> origins
> > >>>>>
> > >>>>> I believe this is what we've mostly agreed upon but haven't
> > clearly voted
> > >>>>> on:
> > >>>>>
> > >>>>> In release N we will:
> > >>>>> 1. Prohibit creating new delivery services that would share an
> > >>>>> existing origin and prohibit updating a delivery service to a
> > shared
> > >>>>> origin
> > >>>>> 2. Add some kind of visual indicator that duplicate origins
> are a
> > >>>>> problem that need to be fixed before release N+1; otherwise, an
> > >>>>> upgrade to N+1 will be prohibited.
> > >>>>>
> > >>>>> In release N+1 we will:
> > >>>>> 3. Include a DB migration that adds a uniqueness constraint on
> > origin
> > >>>>> FQDN, removing the API-level checks for that.
> > >>>>> 4. Prevent an upgrade to N+1 if duplicate origins are found
> (this
> > >>>>> might occur as a byproduct of step 3).
> > >>>>>
> > >>>>> I am +1 on this plan and believe this would hit on all the
> > summarized
> > >>>>> points above. Please provide a clear vote on this plan so that
> > we can
> > >>>>> dive deeper in the details (i.e. what release 'N' is, the best
> > visual
> > >>>>> indicator for step 2, and a friendly way to handle step 4).
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Rawlin
> > >>>>>
> > >>>>> On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <
> > mitchell852@gmail.com>
> > >>>>> wrote:
> > >>>>>>
> > >>>>>> Not sure TP is the right place for a warning like "clean up
> this
> > >>>>>> 'duplicate' origin or your next upgrade will fail". Most users
> > of our
> > >>>>>> system will be like "not my problem".
> > >>>>>>
> > >>>>>> On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <
> > >>>>> Brennan_Fieck@comcast.com>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Probably. It would impact load times by a bit, but the page
> > for an
> > >>>>>>> individual object is not our bottleneck.
> > >>>>>>> ________________________________________
> > >>>>>>> From: Robert Butts <ro...@apache.org>
> > >>>>>>> Sent: Wednesday, December 19, 2018 11:50 AM
> > >>>>>>> To: dev@trafficcontrol.apache.org
> > >>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > Delivery
> > >>>>> Services
> > >>>>>>> produces indeterminate parent.config
> > >>>>>>>
> > >>>>>>>> - Including a warning on startup and an API constraint
> > preventing
> > >>>>> adding
> > >>>>>>> more bad data in the next 3.0.0 Release Candidate
> > >>>>>>>> - Adding a database constraint immediately into master that
> > won't
> > >>>> be
> > >>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
> > >>>>>>>
> > >>>>>>> +1
> > >>>>>>>
> > >>>>>>> I understand Jonathan's objection, but at some point, we have
> > to be
> > >>>>> able to
> > >>>>>>> move forward. This is a good compromise: deprecate, then
> > remove. That
> > >>>>> gives
> > >>>>>>> people a full major version to fix their data.
> > >>>>>>>
> > >>>>>>> I would be ideal if it were more than just a logged warning,
> > though.
> > >>>>> Can we
> > >>>>>>> add a big red banner in Traffic Portal, on the Delivery
> > Service page
> > >>>>> for
> > >>>>>>> any DS with a duplicate origin, telling users to fix it, and
> > that
> > >>>> they
> > >>>>>>> won't be able to upgrade to the next major version until it's
> > fixed?
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <
> > >>>>> Brennan_Fieck@comcast.com
> > >>>>>>>>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> So it seems like nobody has a problem with the "how" -
> > disallowing
> > >>>>>>>> duplicate origin FQDNs on Delivery Services - but we never
> > reached
> > >>>> a
> > >>>>>>>> consensus on "when".
> > >>>>>>>>
> > >>>>>>>> I stand by my previous proposal:
> > >>>>>>>> - Including a warning on startup and an API constraint
> > preventing
> > >>>>> adding
> > >>>>>>>> more bad data in the next 3.0.0 Release Candidate
> > >>>>>>>> - Adding a database constraint immediately into master that
> > won't
> > >>>> be
> > >>>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
> > >>>>>>>> ________________________________________
> > >>>>>>>> From: Rawlin Peters <ra...@gmail.com>
> > >>>>>>>> Sent: Tuesday, December 18, 2018 4:59 PM
> > >>>>>>>> To: dev@trafficcontrol.apache.org
> > >>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > Delivery
> > >>>>> Services
> > >>>>>>>> produces indeterminate parent.config
> > >>>>>>>>
> > >>>>>>>> Also, building more around DS types will make it even harder
> > to get
> > >>>>>>>> away from DS types in the future too, which I know is
> > something
> > >>>> we've
> > >>>>>>>> discussed on this mailing list before. It also adds to the
> > overhead
> > >>>>> of
> > >>>>>>>> Delivery Service Topologies, since a lot of the DS types
> won't
> > >>>>>>>> carryover into that world.
> > >>>>>>>>
> > >>>>>>>> - Rawlin
> > >>>>>>>>
> > >>>>>>>> On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
> > >>>>>>>> <Br...@comcast.com> wrote:
> > >>>>>>>>>
> > >>>>>>>>> +1.
> > >>>>>>>>> If there's a simple way to work around duplicate origins
> > being
> > >>>>>>>> prohibited,
> > >>>>>>>>> then we should rely on that instead of "enumerating all
> those
> > >>>>> possible
> > >>>>>>>> conflicting
> > >>>>>>>>> settings, which are not only highly complex and confusing,
> > but
> > >>>> also
> > >>>>>>>> further
> > >>>>>>>>> entrench us in only supporting ATS as a caching proxy
> > (hurting
> > >>>>> efforts
> > >>>>>>> to
> > >>>>>>>>> integrate e.g. Grove, nginx etc.)
> > >>>>>>>>> ________________________________________
> > >>>>>>>>> From: Rawlin Peters <ra...@gmail.com>
> > >>>>>>>>> Sent: Tuesday, December 18, 2018 2:20 PM
> > >>>>>>>>> To: dev@trafficcontrol.apache.org
> > >>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > Delivery
> > >>>>>>>> Services produces indeterminate parent.config
> > >>>>>>>>>
> > >>>>>>>>> There are a number of different DS settings at play that
> can
> > >>>>>>>>> potentially cause conflicts. The question is: do we want to
> > get
> > >>>>> into
> > >>>>>>>>> the business of enumerating all those possible conflicting
> > >>>>> settings or
> > >>>>>>>>> just simply prohibit duplicate origins altogether? I think
> > we can
> > >>>>> dig
> > >>>>>>>>> in and get that "sufficiently advanced sql query" to check
> > for
> > >>>>>>>>> conflicting origins, but is that something we want to carry
> > along
> > >>>>> for
> > >>>>>>>>> the foreseeable future? Aren't CNAMEs relatively cheaper
> than
> > >>>>>>>>> developing and maintaining that code and the mental
> overhead
> > >>>>> required
> > >>>>>>>>> in understanding why you're getting an error that says your
> > >>>>> requested
> > >>>>>>>>> DS would cause an origin conflict? I think at the point
> > you've
> > >>>>>>>>> requested a DS that would create a conflict, you've chosen
> > those
> > >>>>>>>>> settings for a reason and would probably prefer to just
> > >>>> create/use
> > >>>>> a
> > >>>>>>>>> CNAME in your new DS and keep the rest of your settings the
> > same.
> > >>>>>>>>>
> > >>>>>>>>> Thinking in terms of errors, I'm imagining:
> > >>>>>>>>> "cannot create delivery service: origin fqdn '
> > foo.example.com'
> > >>>>> already
> > >>>>>>>> in use"
> > >>>>>>>>> vs
> > >>>>>>>>> "cannot create delivery service: origin fqdn '
> > foo.example.com'
> > >>>>> already
> > >>>>>>>>> in use as type DNS_LIVE_NATNL, which is incompatible with
> > your
> > >>>>> chosen
> > >>>>>>>>> type of HTTP_NO_CACHE"
> > >>>>>>>>>
> > >>>>>>>>> At that point you'd probably say to yourself, "uh, I need
> > >>>>>>>>> HTTP_NO_CACHE, so what am I supposed to do now?"
> > >>>>>>>>>
> > >>>>>>>>> As a lazy developer I'm +1 on prohibiting duplicate origin
> > fqdns
> > >>>>>>>>> because the resulting code will be simpler, but I think
> > >>>> eliminating
> > >>>>>>>>> the mental overhead for operators could be worthwhile too.
> > If we
> > >>>>> can
> > >>>>>>>>> agree on an end state of prohibiting duplicate origins
> > >>>> altogether,
> > >>>>> we
> > >>>>>>>>> can start working on a design to smoothly transition us to
> > that
> > >>>>> point.
> > >>>>>>>>> Are we willing to live with "just CNAME your origin fqdn"
> as
> > the
> > >>>>>>>>> standard solution to duplicates?
> > >>>>>>>>>
> > >>>>>>>>> - Rawlin
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
> > >>>>>>>>> <De...@comcast.com> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> The only situation in which they can share origins is if
> a)
> > the
> > >>>>>>>> origins are shared in an MSO configuration but still have
> > different
> > >>>>>>> defined
> > >>>>>>>> origin fields in the delivery service, or if they're
> assigned
> > to
> > >>>>>>> completely
> > >>>>>>>> different cachegroups. It's when two delivery services
> share
> > the
> > >>>>> same
> > >>>>>>>> edges that there's an issue, because you end up with
> > parent.config
> > >>>>>>> issues.
> > >>>>>>>> Actually you could even get away with it in mids as long as
> > you
> > >>>>> weren't
> > >>>>>>>> doing anything like MSO to it.
> > >>>>>>>>>>
> > >>>>>>>>>> Could get messy real fast, though. Best to just create a
> > >>>> second
> > >>>>>>> FQDN.
> > >>>>>>>>>>
> > >>>>>>>>>> Derek
> > >>>>>>>>>>
> > >>>>>>>>>> On 12/18/18, 3:23 PM, "Fieck, Brennan" <
> > >>>>> Brennan_Fieck@comcast.com>
> > >>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> So no two Delivery Services may share an origin
> > *regardless
> > >>>>> of
> > >>>>>>>> cache hierarchy* ? I've been told that DNS Delivery Services
> > can
> > >>>>> have the
> > >>>>>>>> same origin as HTTP Delivery Services because they obey the
> > same
> > >>>>> cache
> > >>>>>>>> hierarchy. You're saying that would still produce invalid
> > output
> > >>>>> and/or
> > >>>>>>> is
> > >>>>>>>> explicitly disallowed by ATS?
> > >>>>>>>>>> ________________________________________
> > >>>>>>>>>> From: Robert Butts <ro...@apache.org>
> > >>>>>>>>>> Sent: Tuesday, December 18, 2018 1:09 PM
> > >>>>>>>>>> To: dev@trafficcontrol.apache.org
> > >>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > >>>>> Delivery
> > >>>>>>>> Services produces indeterminate parent.config
> > >>>>>>>>>>
> > >>>>>>>>>>> can you give an example of what parent.config looks like
> > >>>>> when 2
> > >>>>>>>> ds's share
> > >>>>>>>>>> an origin and have different a different topology?
> > >>>>>>>>>>
> > >>>>>>>>>> Answering because I encountered this directly, when
> > >>>> rewriting
> > >>>>>>>> parent.config.
> > >>>>>>>>>>
> > >>>>>>>>>> For example: Suppose you have one Delivery Service:
> > >>>>>>>>>> XML_ID: foo
> > >>>>>>>>>> Type: HTPT_LIVE_NATL
> > >>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass
> > up
> > >>>>>>>>>> Origin Server Base URL: http://foo.example.net
> > >>>>>>>>>>
> > >>>>>>>>>> And another Delivery Service:
> > >>>>>>>>>> XML_ID: bar
> > >>>>>>>>>> Type: HTPT_LIVE_NATL
> > >>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass
> > up
> > >>>>>>>>>> Origin Server Base URL: http://foo.example.net
> > >>>>>>>>>>
> > >>>>>>>>>> ATS only supports unique `dest_domain` entries in
> > >>>>> parent.config.
> > >>>>>>>> Therefore,
> > >>>>>>>>>> the parent.config generated for a server assigned to
> both
> > >>>> of
> > >>>>>>> these
> > >>>>>>>> Delivery
> > >>>>>>>>>> Services with either be:
> > >>>>>>>>>>
> > >>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=true
> > >>>>>>>>>>
> > >>>>>>>>>> Or
> > >>>>>>>>>>
> > >>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=false
> > >>>>>>>> qstring=consider
> > >>>>>>>>>>
> > >>>>>>>>>> Right now, it's arbitrary which Perl Traffic Ops
> inserts,
> > >>>> and
> > >>>>>>> Perl
> > >>>>>>>> provides
> > >>>>>>>>>> no warning or error of any kind (the pending Go
> > >>>>> parent.config PR
> > >>>>>>>> logs an
> > >>>>>>>>>> error).
> > >>>>>>>>>>
> > >>>>>>>>>> Whichever is arbitrarily inserted, the resulting remaps
> > for
> > >>>>> the
> > >>>>>>>> other
> > >>>>>>>>>> delivery service will be wrong. Either `foo` requests
> will
> > >>>>> drop
> > >>>>>>>> the query
> > >>>>>>>>>> string when they shouldn't, and go to the mid when they
> > >>>>>>> shouldn't;
> > >>>>>>>> or `bar`
> > >>>>>>>>>> requests will use the query string and skip the mid when
> > it
> > >>>>>>>> shouldn't.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> Does that make sense? The only correct solution, is to
> > >>>>> somehow
> > >>>>>>>> prevent
> > >>>>>>>>>> different DSes having the same origin, and tell tenants
> > >>>> they
> > >>>>> must
> > >>>>>>>> use
> > >>>>>>>>>> CNAMEs if they need.
> > >>>>>>>>>>
> > >>>>>>>>>> This isn't a bug in Traffic Control. ATS fundamentally
> > >>>>> doesn't
> > >>>>>>>> support
> > >>>>>>>>>> multiple remap rules with the same parent FQDN with
> > >>>> different
> > >>>>>>>>>> configurations. Hence, Traffic Control needs to prohibit
> > >>>>> that.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
> > >>>>>>>> mitchell852@gmail.com>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> brennan,
> > >>>>>>>>>>>
> > >>>>>>>>>>> can you give an example of what parent.config looks like
> > >>>>> when 2
> > >>>>>>>> ds's share
> > >>>>>>>>>>> an origin and have different a different topology?
> > >>>>>>>>>>>
> > >>>>>>>>>>> jeremy
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
> > >>>>>>>> Brennan_Fieck@comcast.com
> > >>>>>>>>>>>>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> To be clear, the "Warning" I'm talking about would
> > >>>>> happen at
> > >>>>>>>> startup, but
> > >>>>>>>>>>>> I'd like a UI-only constraint to come with that to
> > >>>>> disallow
> > >>>>>>>> using the API
> > >>>>>>>>>>>> to bind the same origin to multiple Delivery Services
> > >>>>> with
> > >>>>>>>> varying
> > >>>>>>>>>>>> topography requirements. It wouldn't change the
> > >>>> existing
> > >>>>>>> data,
> > >>>>>>>> but
> > >>>>>>>>>>> prevent
> > >>>>>>>>>>>> users from creating more bad data.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> "warning" doesn't really sufficiently describe that, my
> > >>>>> bad.
> > >>>>>>>>>>>> ________________________________________
> > >>>>>>>>>>>> From: Fieck, Brennan <Br...@comcast.com>
> > >>>>>>>>>>>> Sent: Tuesday, December 18, 2018 11:24 AM
> > >>>>>>>>>>>> To: dev@trafficcontrol.apache.org
> > >>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > >>>>>>>> Delivery Services
> > >>>>>>>>>>>> produces indeterminate parent.config
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Well the cost of fixing this bug is a constraint on the
> > >>>>> data.
> > >>>>>>>> Unless we
> > >>>>>>>>>>>> make it a UI-only constraint - which I'm personally
> > >>>>> against -
> > >>>>>>>> there must
> > >>>>>>>>>>> be
> > >>>>>>>>>>>> some point in the future where ATC cannot reasonably be
> > >>>>>>>> expected to work
> > >>>>>>>>>>>> with data that violates that constraint. The question
> > >>>> is
> > >>>>> when
> > >>>>>>>> that should
> > >>>>>>>>>>>> occur, which should likely happen at a minor version
> > >>>>> release.
> > >>>>>>>> Minor not
> > >>>>>>>>>>>> major because it doesn't involve a change in data
> > >>>>> structures,
> > >>>>>>>> merely
> > >>>>>>>>>>>> relationships between them - in my opinion that's a
> > >>>> minor
> > >>>>>>>> version change
> > >>>>>>>>>>>> but that's definitely up for debate. With several
> > >>>> release
> > >>>>>>>> candidates for
> > >>>>>>>>>>>> 3.0.0 that _doesn't_ include this restriction already
> > >>>> in
> > >>>>> the
> > >>>>>>>> wild, I
> > >>>>>>>>>>>> wouldn't recommend putting it in there. That means to
> > >>>>> fix the
> > >>>>>>>> bug as soon
> > >>>>>>>>>>>> as possible it should go in 3.1.0 which should be the
> > >>>>> target
> > >>>>>>>> of "master"
> > >>>>>>>>>>>> after the 3.0.0 release is cut from it.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> So I'd recommend immediately implementing the
> > >>>> constraint
> > >>>>> in
> > >>>>>>>> master with a
> > >>>>>>>>>>>> refusal to upgrade with bad data, and backport a
> > >>>> warning
> > >>>>>>> about
> > >>>>>>>> the future
> > >>>>>>>>>>>> behavior into 3.0.0 or as part of a 3.0.1 provided we
> > >>>> had
> > >>>>>>> more
> > >>>>>>>> changes
> > >>>>>>>>>>> that
> > >>>>>>>>>>>> would warrant a micro version bump.
> > >>>>>>>>>>>> ________________________________________
> > >>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
> > >>>>>>>>>>>> Sent: Tuesday, December 18, 2018 9:34 AM
> > >>>>>>>>>>>> To: dev@trafficcontrol.apache.org
> > >>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > >>>>>>>> Delivery Services
> > >>>>>>>>>>>> produces indeterminate parent.config
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> -1 Holding an ATC upgrade hostage to data cleanup seems
> > >>>>> like
> > >>>>>>> a
> > >>>>>>>> bad idea.
> > >>>>>>>>>>>> The issue isn't great, but it's also not new. We
> > >>>> should
> > >>>>>>> allow
> > >>>>>>>> teams to
> > >>>>>>>>>>> fix
> > >>>>>>>>>>>> their data at their normal paces if it doesn't create
> > >>>>>>>> significant
> > >>>>>>>>>>> overhead
> > >>>>>>>>>>>> or an inherant blocker for new functionality or
> > >>>>> correction of
> > >>>>>>>> other major
> > >>>>>>>>>>>> problems imho.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Jonathan G
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On 12/18/18, 9:28 AM, "Fieck, Brennan" <
> > >>>>>>>> Brennan_Fieck@comcast.com>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Another option is we could detect collisions at
> > >>>>> startup
> > >>>>>>>> and simply
> > >>>>>>>>>>>> refuse to continue with the upgrade until the data is
> > >>>>> fixed.
> > >>>>>>>> That would
> > >>>>>>>>>>>> allow people using the now-unsupported data format to
> > >>>>>>> continue
> > >>>>>>>> to use
> > >>>>>>>>>>> their
> > >>>>>>>>>>>> old versions of Traffic Ops without wrecking their
> > >>>>> database,
> > >>>>>>>> but also
> > >>>>>>>>>>>> provide an incentive to clean up the data.
> > >>>>>>>>>>>> ________________________________________
> > >>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
> > >>>>>>>>>>>> Sent: Tuesday, December 18, 2018 5:12 AM
> > >>>>>>>>>>>> To: dev@trafficcontrol.apache.org
> > >>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to
> > >>>>> Multipe
> > >>>>>>>> Delivery
> > >>>>>>>>>>>> Services produces indeterminate parent.config
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I'm generally a fan of constrain your data in your
> > >>>>>>>> database, but not
> > >>>>>>>>>>>> necessarily exclusively. I see this as a one-way
> > >>>>>>>> cleanup/conversion so
> > >>>>>>>>>>> it
> > >>>>>>>>>>>> doesn't need to be configurable; otherwise you have to
> > >>>>> ask
> > >>>>>>> the
> > >>>>>>>> question
> > >>>>>>>>>>>> what happens if someone turns it off. That said,
> > >>>>> something
> > >>>>>>> in
> > >>>>>>>> the UI
> > >>>>>>>>>>> layer
> > >>>>>>>>>>>> would be nice to prevent spending significant
> > >>>> quantities
> > >>>>> of
> > >>>>>>>> time
> > >>>>>>>>>>> building a
> > >>>>>>>>>>>> complex DS only to have it fail to post for reasons
> > >>>> that
> > >>>>>>> could
> > >>>>>>>> have been
> > >>>>>>>>>>>> known earlier.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> The way my brain works in this case:
> > >>>>>>>>>>>> If !unique_constraint_exists_query()
> > >>>>>>>>>>>> If has_duplicates_query()
> > >>>>>>>>>>>> show_warning()
> > >>>>>>>>>>>> else
> > >>>>>>>>>>>> add_unique_constraint()
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> to which the API and UI configuration could also
> > >>>>> make use
> > >>>>>>>> of
> > >>>>>>>>>>>> unique_constraint_exists_query() to drive additional
> > >>>>> layer
> > >>>>>>>> constraints if
> > >>>>>>>>>>>> desired.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Jonathan G
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On 12/17/18, 1:11 PM, "Rawlin Peters" <
> > >>>>>>>> rawlin.peters@gmail.com>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> That is an interesting idea...detect at TO
> > >>>>> startup
> > >>>>>>>> whether or not
> > >>>>>>>>>>>> there are duplicate origins and operate in a
> > >>>>> "prevent
> > >>>>>>>> duplicate
> > >>>>>>>>>>>> origins" state if no duplicates are found or
> > >>>>> "prevent
> > >>>>>>>> conflicting
> > >>>>>>>>>>>> DS
> > >>>>>>>>>>>> topologies" state if duplicates are found? So
> > >>>>> once
> > >>>>>>>> operators have
> > >>>>>>>>>>>> replaced all the duplicate origins with CNAMEs,
> > >>>>> TO
> > >>>>>>> will
> > >>>>>>>>>>> essentially
> > >>>>>>>>>>>> operate in a "prohibit all duplicate origins"
> > >>>>> state.
> > >>>>>>>> That would
> > >>>>>>>>>>>> probably make for a simpler transition, but I'd
> > >>>>> want
> > >>>>>>>> to remove
> > >>>>>>>>>>> that
> > >>>>>>>>>>>> logic in a following release that strictly
> > >>>>> prohibits
> > >>>>>>>> duplicate
> > >>>>>>>>>>>> origins
> > >>>>>>>>>>>> (assuming that the community agrees we should
> > >>>>>>> prohibit
> > >>>>>>>> duplicate
> > >>>>>>>>>>>> origins altogether).
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> As for DB constraints vs UI, I was thinking
> > >>>> those
> > >>>>>>>> DS-type
> > >>>>>>>>>>>> constraints
> > >>>>>>>>>>>> I pointed out would live in the API. It would
> > >>>>>>>> basically be added
> > >>>>>>>>>>>> validation in the deliveryservices POST/PUT
> > >>>>> endpoint
> > >>>>>>>> that checks
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>> DB for existing DSes that conflict with the
> > >>>>> requested
> > >>>>>>>> DS.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> - Rawlin
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > >>>>>>>>>>>> <Jo...@comcast.com> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> These kinds of conditions should be
> > >>>> detectable
> > >>>>>>> with a
> > >>>>>>>>>>>> sufficiently advanced SQL query. Is it possible to add
> > >>>>> the
> > >>>>>>>> constraint if
> > >>>>>>>>>>>> it passes and emit a warning during TO startup
> > >>>> otherwise?
> > >>>>>>>> That would let
> > >>>>>>>>>>>> you know the condition exists at startup but not
> > >>>> getting
> > >>>>> in
> > >>>>>>>> your way and
> > >>>>>>>>>>>> keep you out of trouble once you've cleaned up. We
> > >>>> made
> > >>>>> a
> > >>>>>>>> mistake early
> > >>>>>>>>>>>> on, but this would acknowledge it was bad and encourage
> > >>>>> it to
> > >>>>>>>> be fixed at
> > >>>>>>>>>>>> the speed of operations teams. Also this puts the
> > >>>>> constraint
> > >>>>>>>> in the
> > >>>>>>>>>>>> database rather than the UI which is really where the
> > >>>>>>>> contention is for
> > >>>>>>>>>>>> usability.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Jonathan G
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On 12/17/18, 11:38 AM, "Rawlin Peters" <
> > >>>>>>>>>>> rawlin.peters@gmail.com>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> We occasionally discuss this issue but
> > >>>>> haven't
> > >>>>>>>> tackled it
> > >>>>>>>>>>>> yet. I think
> > >>>>>>>>>>>>> the main issue is just that duplicate
> > >>>>> origins
> > >>>>>>>> have been
> > >>>>>>>>>>>> allowed since
> > >>>>>>>>>>>>> the beginning, and now everyone's Traffic
> > >>>>> Ops
> > >>>>>>>> could be
> > >>>>>>>>>>>> littered with
> > >>>>>>>>>>>>> duplicate origins. Also, depending on the
> > >>>>>>> config
> > >>>>>>>> of the
> > >>>>>>>>>>>> duplicate
> > >>>>>>>>>>>>> delivery services, the origins might not
> > >>>>> be in
> > >>>>>>>> conflict at
> > >>>>>>>>>>>> all (if
> > >>>>>>>>>>>>> they don't have different topology
> > >>>>>>> constraints).
> > >>>>>>>> I would
> > >>>>>>>>>>>> love for us
> > >>>>>>>>>>>>> to just add a uniqueness constraint, but
> > >>>>> there
> > >>>>>>>> would need
> > >>>>>>>>>>> to
> > >>>>>>>>>>>> be a fair
> > >>>>>>>>>>>>> amount of warning to the community before
> > >>>>> doing
> > >>>>>>>> so and
> > >>>>>>>>>>> might
> > >>>>>>>>>>>>> invalidate a significant amount of valid
> > >>>>> use
> > >>>>>>>> cases.
> > >>>>>>>>>>>> Operators would
> > >>>>>>>>>>>>> need time to make DNS CNAME records for
> > >>>> the
> > >>>>>>>> duplicate
> > >>>>>>>>>>>> origins and
> > >>>>>>>>>>>>> update their DSes to use the different
> > >>>>> CNAMEs.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I think as a good first step to
> > >>>>> eliminating the
> > >>>>>>>> use of
> > >>>>>>>>>>>> duplicate
> > >>>>>>>>>>>>> origins altogether, we should identify
> > >>>>> which
> > >>>>>>>> "topology
> > >>>>>>>>>>>> constraints"
> > >>>>>>>>>>>>> actually cause conflicting config when
> > >>>> used
> > >>>>>>> with
> > >>>>>>>> duplicate
> > >>>>>>>>>>>> origins and
> > >>>>>>>>>>>>> prevent creating DSes with duplicate
> > >>>>> origins
> > >>>>>>> _if
> > >>>>>>>> it would
> > >>>>>>>>>>>> cause a
> > >>>>>>>>>>>>> conflict with an existing DS that uses
> > >>>> the
> > >>>>> same
> > >>>>>>>> origin_.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> For instance, I believe an HTTP and
> > >>>>> DNS-type DS
> > >>>>>>>> can live
> > >>>>>>>>>>>> happily
> > >>>>>>>>>>>>> side-by-side using the same origin
> > >>>>> (probably
> > >>>>>>>> need different
> > >>>>>>>>>>>>> routing_names?), but scenarios like HTTP
> > >>>>> and
> > >>>>>>>> HTTP_LIVE, or
> > >>>>>>>>>>>> DNS and
> > >>>>>>>>>>>>> HTTP_NO_CACHE sharing the same origin
> > >>>> will
> > >>>>>>> cause
> > >>>>>>>> conflicts
> > >>>>>>>>>>>> for sure.
> > >>>>>>>>>>>>> So maybe we can start by making sure the
> > >>>> DS
> > >>>>>>>> types "match"
> > >>>>>>>>>>>> when using
> > >>>>>>>>>>>>> the same origin:
> > >>>>>>>>>>>>> HTTP + DNS: possibly good, if they have
> > >>>>>>>> different routing
> > >>>>>>>>>>>> names?
> > >>>>>>>>>>>>> HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > >>>>>>>>>>>>> HTTP_NO_CACHE + [any other type]: bad
> > >>>>>>>>>>>>> HTTP_LIVE + HTTP: bad
> > >>>>>>>>>>>>> etc.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> There are most likely other conflict
> > >>>>> scenarios
> > >>>>>>>> that don't
> > >>>>>>>>>>>> involve the
> > >>>>>>>>>>>>> DS types, but I think this would be a
> > >>>> good
> > >>>>>>>> start. In the
> > >>>>>>>>>>>> future with
> > >>>>>>>>>>>>> Delivery Service Topologies (aka Flexible
> > >>>>>>>> Cachegroups aka
> > >>>>>>>>>>>> Bring Your
> > >>>>>>>>>>>>> Own Topology), we might be able to
> > >>>> prohibit
> > >>>>>>>> assigning a DS
> > >>>>>>>>>>>> to a
> > >>>>>>>>>>>>> Topology if the DS's origin is already
> > >>>>> used by
> > >>>>>>>> another DS
> > >>>>>>>>>>> in
> > >>>>>>>>>>>> a
> > >>>>>>>>>>>>> different Topology.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> - Rawlin
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Mon, Dec 17, 2018 at 10:52 AM Fieck,
> > >>>>> Brennan
> > >>>>>>>>>>>>> <Br...@comcast.com> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> As some of you may be aware,
> > >>>>> `parent.config`
> > >>>>>>>> files
> > >>>>>>>>>>>> generated by Traffic Ops can vary wildly when an origin
> > >>>>> is
> > >>>>>>>> assigned to
> > >>>>>>>>>>>> multiple Delivery Services. This results in undefined
> > >>>>>>>> behavior. I'm told
> > >>>>>>>>>>>> that the conflict only happens when two Delivery
> > >>>> Services
> > >>>>>>> with
> > >>>>>>>> different
> > >>>>>>>>>>>> "topology requirements" use the same origin, whatever
> > >>>>> that
> > >>>>>>>> means (content
> > >>>>>>>>>>>> routing type?). Regardless, the issue should be
> > >>>>> addressed.
> > >>>>>>> The
> > >>>>>>>> obvious
> > >>>>>>>>>>>> solution is to put in place a database constraint that
> > >>>>>>>> prevents an origin
> > >>>>>>>>>>>> from being assigned to more that one Delivery Service
> > >>>>> with
> > >>>>>>> API
> > >>>>>>>> checks in
> > >>>>>>>>>>>> place that would provide helpful error messages when an
> > >>>>>>>> attempt is made
> > >>>>>>>>>>> to
> > >>>>>>>>>>>> violate the constraint. However, would that mess with
> > >>>>> things
> > >>>>>>>> like
> > >>>>>>>>>>>> Multi-Site Origin? Or is it just not viable for some
> > >>>>> other
> > >>>>>>>> reason? If it
> > >>>>>>>>>>> is
> > >>>>>>>>>>>> a good solution, I'm prepared to work on a fix that
> > >>>>> utilizes
> > >>>>>>>> it.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> >
> >
> >
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Steve Malenfant <sm...@gmail.com>.
CNAME might not necessary work if the origin is using virtual hosting. It
would need to be configured as well. Might work with some origin? But not
all of them.
On Tue, Jan 29, 2019 at 7:40 PM Fieck, Brennan <Br...@comcast.com>
wrote:
> The issue is inherent with the structure of ATS configuration files. If I
> understand correctly - someone correct me if I don't -
> you can't have more than one rule for a "primary destination" in the
> parent.config. Or, you can, but only one of them will
> wind up catching all of your matches. Which means that if you have
> multiple Delivery Services that use the same origin FQDN
> (dest_domain) then you get proper behavior out of ATS if and only if the
> resulting rules would be identical. I'd highly
> recommend that you not move forward using duplicate origin FQDNs on your
> Delivery Services, because it probably won't
> do what you want.
>
> I'm not sure what you mean about the cachekeys. The Delivery Service URLs
> should still be unique, so you shouldn't run
> into a collision in the cache keys.
>
> Origins are an object. They're also a cachegroup type. And a profile type.
> And a server type. And a URL field on a Delivery Service.
>
> The solution that's been used in the past to use the same actual origin
> machine to service multiple Delivery Services safely
> is to add a "Canonical Name" DNS record to the ATC internal zone for each
> Delivery Service. The idea is to have a unique name
> for each Delivery Service, but they all resolve to the same IP address.
> ________________________________________
> From: Steve Malenfant <sm...@gmail.com>
> Sent: Tuesday, January 29, 2019 4:32 PM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> produces indeterminate parent.config
>
> We use and will use the same origin for multiple DS. It’s an inherent part
> of the design of some part of our delivery.
>
> Wouldn’t be less of a problem using cachekeys for different delivery
> services?
>
> I would rather see origin as an object and not just a simple URL. I’ve seen
> this in the past in other systems.
>
> I don’t believe we can create multiple http services endpoint for a single
> DS like we can for DNS type delivery service. This would solve it for us.
>
> Steve
>
> On Thu, Jan 24, 2019 at 11:22 AM Gray, Jonathan <Jonathan_Gray@comcast.com
> >
> wrote:
>
> > Even that has a strong element of confusion when in a self-service
> > multi-tenancy world. Multiple tenants can use the same origins for
> things
> > like akami, aws, internal load balancers and any communication about the
> > origin already being used will be confusing.
> >
> > Jonathan G
> >
> >
> > On 1/23/19, 7:37 PM, "Gelinas, Derek" <De...@comcast.com>
> wrote:
> >
> > Perhaps we could solve both issues and just allow it to happen on DS
> > config but throw up a message when it is saved that says "hey we're going
> > to do this but it's really not without an element of risk and ats isn't
> > designed to work properly with multiple identical origins. We recommend
> you
> > use an alternative fqdn blah blah blah."
> >
> > Something along those lines, anyway.
> >
> > Derek
> >
> > > On Jan 23, 2019, at 7:18 PM, Rawlin Peters <
> rawlin.peters@gmail.com>
> > wrote:
> > >
> > > Simply prohibiting duplicate origin FQDNs has been vetoed now, so
> we
> > > can't really move forward with that unless the vetoers change their
> > > minds and remove their -1s.
> > >
> > > Currently that leaves us with checking at DS create/update whether
> or
> > > not it would conflict with a shared origin due to different DS
> types,
> > > qstring config, etc. I am -1 on that idea since I think it will be
> > > difficult to maintain and enumerate all the possible conflicting
> > > cases, will present confusing error messages to the user to which
> the
> > > solution would just be "create a CNAME record" anyways, and that
> the
> > > ATS parent.config should really be per-remap rather than "global".
> I
> > > know there has been some discussion in the ATS community about
> making
> > > parent.config per-remap, but I don't have much information there.
> > >
> > > That is to say, I don't think there is a great solution to this
> > > problem right now that we can all agree upon, but maybe our efforts
> > > would be better spent in the ATS community by making a per-remap
> > > parent.config. Then the duplicate origin problem in Traffic Control
> > > would go away.
> > >
> > > Another option could be to have some kind of generic rules engine
> > that
> > > would allow an organization to create some set of rules that DSes
> > > would have to abide by. For example, no duplicate origins, an
> origin
> > > can't be in a list of known malicious domains, you can't use the DS
> > > type "HTTP_NO_CACHE", maxDnsAnswers must be < 5, TTLs must be > 30
> > and
> > > < 6000, protocol must be HTTPS, intitialDispersion must be < 10,
> DSCP
> > > must be 40, etc. It seems like for something truly self-service
> you'd
> > > probably need to set up some rules for your users to abide by that
> > > might not necessarily apply to ALL organizations. Just a thought. I
> > > don't think the duplicate origins problem alone would warrant
> > > something like that, but if it also provided a solution for the
> > > self-service effort, maybe it would be worth it.
> > >
> > > - Rawlin
> > >
> > > On Wed, Jan 23, 2019 at 4:15 AM Gelinas, Derek
> > > <De...@comcast.com> wrote:
> > >>
> > >> That's the problem. We either need to be smart enough to recognize
> > that in our config and warn the user or just prevent it and suggest to
> the
> > user that they use a different fqdn.
> > >>
> > >> My vote is for preventing it at this point.
> > >>
> > >>> On Jan 23, 2019, at 12:50 AM, Gray, Jonathan <
> > Jonathan_Gray@comcast.com> wrote:
> > >>>
> > >>> It's not just differing types, it's anything that affects the
> > parent.config which also includes the, qstring configuration, MSO
> > parents/config, etc. as well.
> > >>>
> > >>> Jonathan G
> > >>>
> > >>>
> > >>> On 1/22/19, 2:33 PM, "Dave Neuman" <ne...@apache.org> wrote:
> > >>>
> > >>> -1 on the one DS to Origin limitation. I think there are legit
> > use cases
> > >>> for it.
> > >>> I would be +1 on the idea if we include type though. Two DSs
> > can share the
> > >>> same origin as long as they are of the same type (DNS, HTTP,
> > HTTP_LIVE),
> > >>> etc. You shouldn't be able to have two DSs with the same
> origin
> > but
> > >>> different types.
> > >>>
> > >>> On Mon, Jan 14, 2019 at 7:53 AM Fieck, Brennan <
> > Brennan_Fieck@comcast.com>
> > >>> wrote:
> > >>>
> > >>>> I don't see this as a complicated use case or a barrier to
> entry,
> > an
> > >>>> extremely basic introduction to ATC would only make use of one
> DS
> > >>>> (shameless CDN-in-a-Box plug).
> > >>>> It seems far more complicated for a user to be using many DSes
> to
> > serve a
> > >>>> single origin - we can easily make it clear in the docs that the
> > two have a
> > >>>> 1:1 relationship, with a footnote about CNAME records for users
> > with the
> > >>>> knowledge, resources, and need for such workarounds.
> > >>>> ________________________________________
> > >>>> From: Mark Torluemke <mt...@apache.org>
> > >>>> Sent: Friday, January 11, 2019 1:21 PM
> > >>>> To: dev@trafficcontrol.apache.org
> > >>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > Services
> > >>>> produces indeterminate parent.config
> > >>>>
> > >>>>>> 1. Prohibit creating new delivery services that would share an
> > >>>> existing origin and prohibit updating a delivery service to a
> > shared
> > >>>> origin
> > >>>> In case my position has been lost, I'm still -1 on this. :) IMO,
> > we
> > >>>> shouldn't let the few complicated use cases increase the
> barrier
> > to entry
> > >>>> (to using ATC) for the masses.
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Fri, Jan 11, 2019 at 11:53 AM Rawlin Peters <
> > rawlin.peters@gmail.com>
> > >>>> wrote:
> > >>>>
> > >>>>> Alright, I'm trying to sum up this discussion so far since it
> > seems
> > >>>>> like everyone went on vacation and didn't really get a chance
> to
> > wrap
> > >>>>> this one up:
> > >>>>> - duplicate origins cause undefined behavior
> > >>>>> - we need a way to migrate to a future that is free of
> duplicate
> > >>>>> origins in Traffic Control
> > >>>>> - we need a visual and easy way to determine if Traffic Ops
> > currently
> > >>>>> contains duplicate origins, so that operators are incentivized
> > to fix
> > >>>>> them rather than let it slide indefinitely
> > >>>>> - operators should have a fair amount of time to fix their
> > duplicate
> > >>>>> origins
> > >>>>>
> > >>>>> I believe this is what we've mostly agreed upon but haven't
> > clearly voted
> > >>>>> on:
> > >>>>>
> > >>>>> In release N we will:
> > >>>>> 1. Prohibit creating new delivery services that would share an
> > >>>>> existing origin and prohibit updating a delivery service to a
> > shared
> > >>>>> origin
> > >>>>> 2. Add some kind of visual indicator that duplicate origins
> are a
> > >>>>> problem that need to be fixed before release N+1; otherwise, an
> > >>>>> upgrade to N+1 will be prohibited.
> > >>>>>
> > >>>>> In release N+1 we will:
> > >>>>> 3. Include a DB migration that adds a uniqueness constraint on
> > origin
> > >>>>> FQDN, removing the API-level checks for that.
> > >>>>> 4. Prevent an upgrade to N+1 if duplicate origins are found
> (this
> > >>>>> might occur as a byproduct of step 3).
> > >>>>>
> > >>>>> I am +1 on this plan and believe this would hit on all the
> > summarized
> > >>>>> points above. Please provide a clear vote on this plan so that
> > we can
> > >>>>> dive deeper in the details (i.e. what release 'N' is, the best
> > visual
> > >>>>> indicator for step 2, and a friendly way to handle step 4).
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Rawlin
> > >>>>>
> > >>>>> On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <
> > mitchell852@gmail.com>
> > >>>>> wrote:
> > >>>>>>
> > >>>>>> Not sure TP is the right place for a warning like "clean up
> this
> > >>>>>> 'duplicate' origin or your next upgrade will fail". Most users
> > of our
> > >>>>>> system will be like "not my problem".
> > >>>>>>
> > >>>>>> On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <
> > >>>>> Brennan_Fieck@comcast.com>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Probably. It would impact load times by a bit, but the page
> > for an
> > >>>>>>> individual object is not our bottleneck.
> > >>>>>>> ________________________________________
> > >>>>>>> From: Robert Butts <ro...@apache.org>
> > >>>>>>> Sent: Wednesday, December 19, 2018 11:50 AM
> > >>>>>>> To: dev@trafficcontrol.apache.org
> > >>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > Delivery
> > >>>>> Services
> > >>>>>>> produces indeterminate parent.config
> > >>>>>>>
> > >>>>>>>> - Including a warning on startup and an API constraint
> > preventing
> > >>>>> adding
> > >>>>>>> more bad data in the next 3.0.0 Release Candidate
> > >>>>>>>> - Adding a database constraint immediately into master that
> > won't
> > >>>> be
> > >>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
> > >>>>>>>
> > >>>>>>> +1
> > >>>>>>>
> > >>>>>>> I understand Jonathan's objection, but at some point, we have
> > to be
> > >>>>> able to
> > >>>>>>> move forward. This is a good compromise: deprecate, then
> > remove. That
> > >>>>> gives
> > >>>>>>> people a full major version to fix their data.
> > >>>>>>>
> > >>>>>>> I would be ideal if it were more than just a logged warning,
> > though.
> > >>>>> Can we
> > >>>>>>> add a big red banner in Traffic Portal, on the Delivery
> > Service page
> > >>>>> for
> > >>>>>>> any DS with a duplicate origin, telling users to fix it, and
> > that
> > >>>> they
> > >>>>>>> won't be able to upgrade to the next major version until it's
> > fixed?
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <
> > >>>>> Brennan_Fieck@comcast.com
> > >>>>>>>>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> So it seems like nobody has a problem with the "how" -
> > disallowing
> > >>>>>>>> duplicate origin FQDNs on Delivery Services - but we never
> > reached
> > >>>> a
> > >>>>>>>> consensus on "when".
> > >>>>>>>>
> > >>>>>>>> I stand by my previous proposal:
> > >>>>>>>> - Including a warning on startup and an API constraint
> > preventing
> > >>>>> adding
> > >>>>>>>> more bad data in the next 3.0.0 Release Candidate
> > >>>>>>>> - Adding a database constraint immediately into master that
> > won't
> > >>>> be
> > >>>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
> > >>>>>>>> ________________________________________
> > >>>>>>>> From: Rawlin Peters <ra...@gmail.com>
> > >>>>>>>> Sent: Tuesday, December 18, 2018 4:59 PM
> > >>>>>>>> To: dev@trafficcontrol.apache.org
> > >>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > Delivery
> > >>>>> Services
> > >>>>>>>> produces indeterminate parent.config
> > >>>>>>>>
> > >>>>>>>> Also, building more around DS types will make it even harder
> > to get
> > >>>>>>>> away from DS types in the future too, which I know is
> > something
> > >>>> we've
> > >>>>>>>> discussed on this mailing list before. It also adds to the
> > overhead
> > >>>>> of
> > >>>>>>>> Delivery Service Topologies, since a lot of the DS types
> won't
> > >>>>>>>> carryover into that world.
> > >>>>>>>>
> > >>>>>>>> - Rawlin
> > >>>>>>>>
> > >>>>>>>> On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
> > >>>>>>>> <Br...@comcast.com> wrote:
> > >>>>>>>>>
> > >>>>>>>>> +1.
> > >>>>>>>>> If there's a simple way to work around duplicate origins
> > being
> > >>>>>>>> prohibited,
> > >>>>>>>>> then we should rely on that instead of "enumerating all
> those
> > >>>>> possible
> > >>>>>>>> conflicting
> > >>>>>>>>> settings, which are not only highly complex and confusing,
> > but
> > >>>> also
> > >>>>>>>> further
> > >>>>>>>>> entrench us in only supporting ATS as a caching proxy
> > (hurting
> > >>>>> efforts
> > >>>>>>> to
> > >>>>>>>>> integrate e.g. Grove, nginx etc.)
> > >>>>>>>>> ________________________________________
> > >>>>>>>>> From: Rawlin Peters <ra...@gmail.com>
> > >>>>>>>>> Sent: Tuesday, December 18, 2018 2:20 PM
> > >>>>>>>>> To: dev@trafficcontrol.apache.org
> > >>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > Delivery
> > >>>>>>>> Services produces indeterminate parent.config
> > >>>>>>>>>
> > >>>>>>>>> There are a number of different DS settings at play that
> can
> > >>>>>>>>> potentially cause conflicts. The question is: do we want to
> > get
> > >>>>> into
> > >>>>>>>>> the business of enumerating all those possible conflicting
> > >>>>> settings or
> > >>>>>>>>> just simply prohibit duplicate origins altogether? I think
> > we can
> > >>>>> dig
> > >>>>>>>>> in and get that "sufficiently advanced sql query" to check
> > for
> > >>>>>>>>> conflicting origins, but is that something we want to carry
> > along
> > >>>>> for
> > >>>>>>>>> the foreseeable future? Aren't CNAMEs relatively cheaper
> than
> > >>>>>>>>> developing and maintaining that code and the mental
> overhead
> > >>>>> required
> > >>>>>>>>> in understanding why you're getting an error that says your
> > >>>>> requested
> > >>>>>>>>> DS would cause an origin conflict? I think at the point
> > you've
> > >>>>>>>>> requested a DS that would create a conflict, you've chosen
> > those
> > >>>>>>>>> settings for a reason and would probably prefer to just
> > >>>> create/use
> > >>>>> a
> > >>>>>>>>> CNAME in your new DS and keep the rest of your settings the
> > same.
> > >>>>>>>>>
> > >>>>>>>>> Thinking in terms of errors, I'm imagining:
> > >>>>>>>>> "cannot create delivery service: origin fqdn '
> > foo.example.com'
> > >>>>> already
> > >>>>>>>> in use"
> > >>>>>>>>> vs
> > >>>>>>>>> "cannot create delivery service: origin fqdn '
> > foo.example.com'
> > >>>>> already
> > >>>>>>>>> in use as type DNS_LIVE_NATNL, which is incompatible with
> > your
> > >>>>> chosen
> > >>>>>>>>> type of HTTP_NO_CACHE"
> > >>>>>>>>>
> > >>>>>>>>> At that point you'd probably say to yourself, "uh, I need
> > >>>>>>>>> HTTP_NO_CACHE, so what am I supposed to do now?"
> > >>>>>>>>>
> > >>>>>>>>> As a lazy developer I'm +1 on prohibiting duplicate origin
> > fqdns
> > >>>>>>>>> because the resulting code will be simpler, but I think
> > >>>> eliminating
> > >>>>>>>>> the mental overhead for operators could be worthwhile too.
> > If we
> > >>>>> can
> > >>>>>>>>> agree on an end state of prohibiting duplicate origins
> > >>>> altogether,
> > >>>>> we
> > >>>>>>>>> can start working on a design to smoothly transition us to
> > that
> > >>>>> point.
> > >>>>>>>>> Are we willing to live with "just CNAME your origin fqdn"
> as
> > the
> > >>>>>>>>> standard solution to duplicates?
> > >>>>>>>>>
> > >>>>>>>>> - Rawlin
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
> > >>>>>>>>> <De...@comcast.com> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> The only situation in which they can share origins is if
> a)
> > the
> > >>>>>>>> origins are shared in an MSO configuration but still have
> > different
> > >>>>>>> defined
> > >>>>>>>> origin fields in the delivery service, or if they're
> assigned
> > to
> > >>>>>>> completely
> > >>>>>>>> different cachegroups. It's when two delivery services
> share
> > the
> > >>>>> same
> > >>>>>>>> edges that there's an issue, because you end up with
> > parent.config
> > >>>>>>> issues.
> > >>>>>>>> Actually you could even get away with it in mids as long as
> > you
> > >>>>> weren't
> > >>>>>>>> doing anything like MSO to it.
> > >>>>>>>>>>
> > >>>>>>>>>> Could get messy real fast, though. Best to just create a
> > >>>> second
> > >>>>>>> FQDN.
> > >>>>>>>>>>
> > >>>>>>>>>> Derek
> > >>>>>>>>>>
> > >>>>>>>>>> On 12/18/18, 3:23 PM, "Fieck, Brennan" <
> > >>>>> Brennan_Fieck@comcast.com>
> > >>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> So no two Delivery Services may share an origin
> > *regardless
> > >>>>> of
> > >>>>>>>> cache hierarchy* ? I've been told that DNS Delivery Services
> > can
> > >>>>> have the
> > >>>>>>>> same origin as HTTP Delivery Services because they obey the
> > same
> > >>>>> cache
> > >>>>>>>> hierarchy. You're saying that would still produce invalid
> > output
> > >>>>> and/or
> > >>>>>>> is
> > >>>>>>>> explicitly disallowed by ATS?
> > >>>>>>>>>> ________________________________________
> > >>>>>>>>>> From: Robert Butts <ro...@apache.org>
> > >>>>>>>>>> Sent: Tuesday, December 18, 2018 1:09 PM
> > >>>>>>>>>> To: dev@trafficcontrol.apache.org
> > >>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > >>>>> Delivery
> > >>>>>>>> Services produces indeterminate parent.config
> > >>>>>>>>>>
> > >>>>>>>>>>> can you give an example of what parent.config looks like
> > >>>>> when 2
> > >>>>>>>> ds's share
> > >>>>>>>>>> an origin and have different a different topology?
> > >>>>>>>>>>
> > >>>>>>>>>> Answering because I encountered this directly, when
> > >>>> rewriting
> > >>>>>>>> parent.config.
> > >>>>>>>>>>
> > >>>>>>>>>> For example: Suppose you have one Delivery Service:
> > >>>>>>>>>> XML_ID: foo
> > >>>>>>>>>> Type: HTPT_LIVE_NATL
> > >>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass
> > up
> > >>>>>>>>>> Origin Server Base URL: http://foo.example.net
> > >>>>>>>>>>
> > >>>>>>>>>> And another Delivery Service:
> > >>>>>>>>>> XML_ID: bar
> > >>>>>>>>>> Type: HTPT_LIVE_NATL
> > >>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass
> > up
> > >>>>>>>>>> Origin Server Base URL: http://foo.example.net
> > >>>>>>>>>>
> > >>>>>>>>>> ATS only supports unique `dest_domain` entries in
> > >>>>> parent.config.
> > >>>>>>>> Therefore,
> > >>>>>>>>>> the parent.config generated for a server assigned to
> both
> > >>>> of
> > >>>>>>> these
> > >>>>>>>> Delivery
> > >>>>>>>>>> Services with either be:
> > >>>>>>>>>>
> > >>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=true
> > >>>>>>>>>>
> > >>>>>>>>>> Or
> > >>>>>>>>>>
> > >>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=false
> > >>>>>>>> qstring=consider
> > >>>>>>>>>>
> > >>>>>>>>>> Right now, it's arbitrary which Perl Traffic Ops
> inserts,
> > >>>> and
> > >>>>>>> Perl
> > >>>>>>>> provides
> > >>>>>>>>>> no warning or error of any kind (the pending Go
> > >>>>> parent.config PR
> > >>>>>>>> logs an
> > >>>>>>>>>> error).
> > >>>>>>>>>>
> > >>>>>>>>>> Whichever is arbitrarily inserted, the resulting remaps
> > for
> > >>>>> the
> > >>>>>>>> other
> > >>>>>>>>>> delivery service will be wrong. Either `foo` requests
> will
> > >>>>> drop
> > >>>>>>>> the query
> > >>>>>>>>>> string when they shouldn't, and go to the mid when they
> > >>>>>>> shouldn't;
> > >>>>>>>> or `bar`
> > >>>>>>>>>> requests will use the query string and skip the mid when
> > it
> > >>>>>>>> shouldn't.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> Does that make sense? The only correct solution, is to
> > >>>>> somehow
> > >>>>>>>> prevent
> > >>>>>>>>>> different DSes having the same origin, and tell tenants
> > >>>> they
> > >>>>> must
> > >>>>>>>> use
> > >>>>>>>>>> CNAMEs if they need.
> > >>>>>>>>>>
> > >>>>>>>>>> This isn't a bug in Traffic Control. ATS fundamentally
> > >>>>> doesn't
> > >>>>>>>> support
> > >>>>>>>>>> multiple remap rules with the same parent FQDN with
> > >>>> different
> > >>>>>>>>>> configurations. Hence, Traffic Control needs to prohibit
> > >>>>> that.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
> > >>>>>>>> mitchell852@gmail.com>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> brennan,
> > >>>>>>>>>>>
> > >>>>>>>>>>> can you give an example of what parent.config looks like
> > >>>>> when 2
> > >>>>>>>> ds's share
> > >>>>>>>>>>> an origin and have different a different topology?
> > >>>>>>>>>>>
> > >>>>>>>>>>> jeremy
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
> > >>>>>>>> Brennan_Fieck@comcast.com
> > >>>>>>>>>>>>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> To be clear, the "Warning" I'm talking about would
> > >>>>> happen at
> > >>>>>>>> startup, but
> > >>>>>>>>>>>> I'd like a UI-only constraint to come with that to
> > >>>>> disallow
> > >>>>>>>> using the API
> > >>>>>>>>>>>> to bind the same origin to multiple Delivery Services
> > >>>>> with
> > >>>>>>>> varying
> > >>>>>>>>>>>> topography requirements. It wouldn't change the
> > >>>> existing
> > >>>>>>> data,
> > >>>>>>>> but
> > >>>>>>>>>>> prevent
> > >>>>>>>>>>>> users from creating more bad data.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> "warning" doesn't really sufficiently describe that, my
> > >>>>> bad.
> > >>>>>>>>>>>> ________________________________________
> > >>>>>>>>>>>> From: Fieck, Brennan <Br...@comcast.com>
> > >>>>>>>>>>>> Sent: Tuesday, December 18, 2018 11:24 AM
> > >>>>>>>>>>>> To: dev@trafficcontrol.apache.org
> > >>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > >>>>>>>> Delivery Services
> > >>>>>>>>>>>> produces indeterminate parent.config
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Well the cost of fixing this bug is a constraint on the
> > >>>>> data.
> > >>>>>>>> Unless we
> > >>>>>>>>>>>> make it a UI-only constraint - which I'm personally
> > >>>>> against -
> > >>>>>>>> there must
> > >>>>>>>>>>> be
> > >>>>>>>>>>>> some point in the future where ATC cannot reasonably be
> > >>>>>>>> expected to work
> > >>>>>>>>>>>> with data that violates that constraint. The question
> > >>>> is
> > >>>>> when
> > >>>>>>>> that should
> > >>>>>>>>>>>> occur, which should likely happen at a minor version
> > >>>>> release.
> > >>>>>>>> Minor not
> > >>>>>>>>>>>> major because it doesn't involve a change in data
> > >>>>> structures,
> > >>>>>>>> merely
> > >>>>>>>>>>>> relationships between them - in my opinion that's a
> > >>>> minor
> > >>>>>>>> version change
> > >>>>>>>>>>>> but that's definitely up for debate. With several
> > >>>> release
> > >>>>>>>> candidates for
> > >>>>>>>>>>>> 3.0.0 that _doesn't_ include this restriction already
> > >>>> in
> > >>>>> the
> > >>>>>>>> wild, I
> > >>>>>>>>>>>> wouldn't recommend putting it in there. That means to
> > >>>>> fix the
> > >>>>>>>> bug as soon
> > >>>>>>>>>>>> as possible it should go in 3.1.0 which should be the
> > >>>>> target
> > >>>>>>>> of "master"
> > >>>>>>>>>>>> after the 3.0.0 release is cut from it.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> So I'd recommend immediately implementing the
> > >>>> constraint
> > >>>>> in
> > >>>>>>>> master with a
> > >>>>>>>>>>>> refusal to upgrade with bad data, and backport a
> > >>>> warning
> > >>>>>>> about
> > >>>>>>>> the future
> > >>>>>>>>>>>> behavior into 3.0.0 or as part of a 3.0.1 provided we
> > >>>> had
> > >>>>>>> more
> > >>>>>>>> changes
> > >>>>>>>>>>> that
> > >>>>>>>>>>>> would warrant a micro version bump.
> > >>>>>>>>>>>> ________________________________________
> > >>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
> > >>>>>>>>>>>> Sent: Tuesday, December 18, 2018 9:34 AM
> > >>>>>>>>>>>> To: dev@trafficcontrol.apache.org
> > >>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > >>>>>>>> Delivery Services
> > >>>>>>>>>>>> produces indeterminate parent.config
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> -1 Holding an ATC upgrade hostage to data cleanup seems
> > >>>>> like
> > >>>>>>> a
> > >>>>>>>> bad idea.
> > >>>>>>>>>>>> The issue isn't great, but it's also not new. We
> > >>>> should
> > >>>>>>> allow
> > >>>>>>>> teams to
> > >>>>>>>>>>> fix
> > >>>>>>>>>>>> their data at their normal paces if it doesn't create
> > >>>>>>>> significant
> > >>>>>>>>>>> overhead
> > >>>>>>>>>>>> or an inherant blocker for new functionality or
> > >>>>> correction of
> > >>>>>>>> other major
> > >>>>>>>>>>>> problems imho.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Jonathan G
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On 12/18/18, 9:28 AM, "Fieck, Brennan" <
> > >>>>>>>> Brennan_Fieck@comcast.com>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Another option is we could detect collisions at
> > >>>>> startup
> > >>>>>>>> and simply
> > >>>>>>>>>>>> refuse to continue with the upgrade until the data is
> > >>>>> fixed.
> > >>>>>>>> That would
> > >>>>>>>>>>>> allow people using the now-unsupported data format to
> > >>>>>>> continue
> > >>>>>>>> to use
> > >>>>>>>>>>> their
> > >>>>>>>>>>>> old versions of Traffic Ops without wrecking their
> > >>>>> database,
> > >>>>>>>> but also
> > >>>>>>>>>>>> provide an incentive to clean up the data.
> > >>>>>>>>>>>> ________________________________________
> > >>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
> > >>>>>>>>>>>> Sent: Tuesday, December 18, 2018 5:12 AM
> > >>>>>>>>>>>> To: dev@trafficcontrol.apache.org
> > >>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to
> > >>>>> Multipe
> > >>>>>>>> Delivery
> > >>>>>>>>>>>> Services produces indeterminate parent.config
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I'm generally a fan of constrain your data in your
> > >>>>>>>> database, but not
> > >>>>>>>>>>>> necessarily exclusively. I see this as a one-way
> > >>>>>>>> cleanup/conversion so
> > >>>>>>>>>>> it
> > >>>>>>>>>>>> doesn't need to be configurable; otherwise you have to
> > >>>>> ask
> > >>>>>>> the
> > >>>>>>>> question
> > >>>>>>>>>>>> what happens if someone turns it off. That said,
> > >>>>> something
> > >>>>>>> in
> > >>>>>>>> the UI
> > >>>>>>>>>>> layer
> > >>>>>>>>>>>> would be nice to prevent spending significant
> > >>>> quantities
> > >>>>> of
> > >>>>>>>> time
> > >>>>>>>>>>> building a
> > >>>>>>>>>>>> complex DS only to have it fail to post for reasons
> > >>>> that
> > >>>>>>> could
> > >>>>>>>> have been
> > >>>>>>>>>>>> known earlier.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> The way my brain works in this case:
> > >>>>>>>>>>>> If !unique_constraint_exists_query()
> > >>>>>>>>>>>> If has_duplicates_query()
> > >>>>>>>>>>>> show_warning()
> > >>>>>>>>>>>> else
> > >>>>>>>>>>>> add_unique_constraint()
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> to which the API and UI configuration could also
> > >>>>> make use
> > >>>>>>>> of
> > >>>>>>>>>>>> unique_constraint_exists_query() to drive additional
> > >>>>> layer
> > >>>>>>>> constraints if
> > >>>>>>>>>>>> desired.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Jonathan G
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On 12/17/18, 1:11 PM, "Rawlin Peters" <
> > >>>>>>>> rawlin.peters@gmail.com>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> That is an interesting idea...detect at TO
> > >>>>> startup
> > >>>>>>>> whether or not
> > >>>>>>>>>>>> there are duplicate origins and operate in a
> > >>>>> "prevent
> > >>>>>>>> duplicate
> > >>>>>>>>>>>> origins" state if no duplicates are found or
> > >>>>> "prevent
> > >>>>>>>> conflicting
> > >>>>>>>>>>>> DS
> > >>>>>>>>>>>> topologies" state if duplicates are found? So
> > >>>>> once
> > >>>>>>>> operators have
> > >>>>>>>>>>>> replaced all the duplicate origins with CNAMEs,
> > >>>>> TO
> > >>>>>>> will
> > >>>>>>>>>>> essentially
> > >>>>>>>>>>>> operate in a "prohibit all duplicate origins"
> > >>>>> state.
> > >>>>>>>> That would
> > >>>>>>>>>>>> probably make for a simpler transition, but I'd
> > >>>>> want
> > >>>>>>>> to remove
> > >>>>>>>>>>> that
> > >>>>>>>>>>>> logic in a following release that strictly
> > >>>>> prohibits
> > >>>>>>>> duplicate
> > >>>>>>>>>>>> origins
> > >>>>>>>>>>>> (assuming that the community agrees we should
> > >>>>>>> prohibit
> > >>>>>>>> duplicate
> > >>>>>>>>>>>> origins altogether).
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> As for DB constraints vs UI, I was thinking
> > >>>> those
> > >>>>>>>> DS-type
> > >>>>>>>>>>>> constraints
> > >>>>>>>>>>>> I pointed out would live in the API. It would
> > >>>>>>>> basically be added
> > >>>>>>>>>>>> validation in the deliveryservices POST/PUT
> > >>>>> endpoint
> > >>>>>>>> that checks
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>> DB for existing DSes that conflict with the
> > >>>>> requested
> > >>>>>>>> DS.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> - Rawlin
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > >>>>>>>>>>>> <Jo...@comcast.com> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> These kinds of conditions should be
> > >>>> detectable
> > >>>>>>> with a
> > >>>>>>>>>>>> sufficiently advanced SQL query. Is it possible to add
> > >>>>> the
> > >>>>>>>> constraint if
> > >>>>>>>>>>>> it passes and emit a warning during TO startup
> > >>>> otherwise?
> > >>>>>>>> That would let
> > >>>>>>>>>>>> you know the condition exists at startup but not
> > >>>> getting
> > >>>>> in
> > >>>>>>>> your way and
> > >>>>>>>>>>>> keep you out of trouble once you've cleaned up. We
> > >>>> made
> > >>>>> a
> > >>>>>>>> mistake early
> > >>>>>>>>>>>> on, but this would acknowledge it was bad and encourage
> > >>>>> it to
> > >>>>>>>> be fixed at
> > >>>>>>>>>>>> the speed of operations teams. Also this puts the
> > >>>>> constraint
> > >>>>>>>> in the
> > >>>>>>>>>>>> database rather than the UI which is really where the
> > >>>>>>>> contention is for
> > >>>>>>>>>>>> usability.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Jonathan G
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On 12/17/18, 11:38 AM, "Rawlin Peters" <
> > >>>>>>>>>>> rawlin.peters@gmail.com>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> We occasionally discuss this issue but
> > >>>>> haven't
> > >>>>>>>> tackled it
> > >>>>>>>>>>>> yet. I think
> > >>>>>>>>>>>>> the main issue is just that duplicate
> > >>>>> origins
> > >>>>>>>> have been
> > >>>>>>>>>>>> allowed since
> > >>>>>>>>>>>>> the beginning, and now everyone's Traffic
> > >>>>> Ops
> > >>>>>>>> could be
> > >>>>>>>>>>>> littered with
> > >>>>>>>>>>>>> duplicate origins. Also, depending on the
> > >>>>>>> config
> > >>>>>>>> of the
> > >>>>>>>>>>>> duplicate
> > >>>>>>>>>>>>> delivery services, the origins might not
> > >>>>> be in
> > >>>>>>>> conflict at
> > >>>>>>>>>>>> all (if
> > >>>>>>>>>>>>> they don't have different topology
> > >>>>>>> constraints).
> > >>>>>>>> I would
> > >>>>>>>>>>>> love for us
> > >>>>>>>>>>>>> to just add a uniqueness constraint, but
> > >>>>> there
> > >>>>>>>> would need
> > >>>>>>>>>>> to
> > >>>>>>>>>>>> be a fair
> > >>>>>>>>>>>>> amount of warning to the community before
> > >>>>> doing
> > >>>>>>>> so and
> > >>>>>>>>>>> might
> > >>>>>>>>>>>>> invalidate a significant amount of valid
> > >>>>> use
> > >>>>>>>> cases.
> > >>>>>>>>>>>> Operators would
> > >>>>>>>>>>>>> need time to make DNS CNAME records for
> > >>>> the
> > >>>>>>>> duplicate
> > >>>>>>>>>>>> origins and
> > >>>>>>>>>>>>> update their DSes to use the different
> > >>>>> CNAMEs.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I think as a good first step to
> > >>>>> eliminating the
> > >>>>>>>> use of
> > >>>>>>>>>>>> duplicate
> > >>>>>>>>>>>>> origins altogether, we should identify
> > >>>>> which
> > >>>>>>>> "topology
> > >>>>>>>>>>>> constraints"
> > >>>>>>>>>>>>> actually cause conflicting config when
> > >>>> used
> > >>>>>>> with
> > >>>>>>>> duplicate
> > >>>>>>>>>>>> origins and
> > >>>>>>>>>>>>> prevent creating DSes with duplicate
> > >>>>> origins
> > >>>>>>> _if
> > >>>>>>>> it would
> > >>>>>>>>>>>> cause a
> > >>>>>>>>>>>>> conflict with an existing DS that uses
> > >>>> the
> > >>>>> same
> > >>>>>>>> origin_.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> For instance, I believe an HTTP and
> > >>>>> DNS-type DS
> > >>>>>>>> can live
> > >>>>>>>>>>>> happily
> > >>>>>>>>>>>>> side-by-side using the same origin
> > >>>>> (probably
> > >>>>>>>> need different
> > >>>>>>>>>>>>> routing_names?), but scenarios like HTTP
> > >>>>> and
> > >>>>>>>> HTTP_LIVE, or
> > >>>>>>>>>>>> DNS and
> > >>>>>>>>>>>>> HTTP_NO_CACHE sharing the same origin
> > >>>> will
> > >>>>>>> cause
> > >>>>>>>> conflicts
> > >>>>>>>>>>>> for sure.
> > >>>>>>>>>>>>> So maybe we can start by making sure the
> > >>>> DS
> > >>>>>>>> types "match"
> > >>>>>>>>>>>> when using
> > >>>>>>>>>>>>> the same origin:
> > >>>>>>>>>>>>> HTTP + DNS: possibly good, if they have
> > >>>>>>>> different routing
> > >>>>>>>>>>>> names?
> > >>>>>>>>>>>>> HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > >>>>>>>>>>>>> HTTP_NO_CACHE + [any other type]: bad
> > >>>>>>>>>>>>> HTTP_LIVE + HTTP: bad
> > >>>>>>>>>>>>> etc.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> There are most likely other conflict
> > >>>>> scenarios
> > >>>>>>>> that don't
> > >>>>>>>>>>>> involve the
> > >>>>>>>>>>>>> DS types, but I think this would be a
> > >>>> good
> > >>>>>>>> start. In the
> > >>>>>>>>>>>> future with
> > >>>>>>>>>>>>> Delivery Service Topologies (aka Flexible
> > >>>>>>>> Cachegroups aka
> > >>>>>>>>>>>> Bring Your
> > >>>>>>>>>>>>> Own Topology), we might be able to
> > >>>> prohibit
> > >>>>>>>> assigning a DS
> > >>>>>>>>>>>> to a
> > >>>>>>>>>>>>> Topology if the DS's origin is already
> > >>>>> used by
> > >>>>>>>> another DS
> > >>>>>>>>>>> in
> > >>>>>>>>>>>> a
> > >>>>>>>>>>>>> different Topology.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> - Rawlin
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Mon, Dec 17, 2018 at 10:52 AM Fieck,
> > >>>>> Brennan
> > >>>>>>>>>>>>> <Br...@comcast.com> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> As some of you may be aware,
> > >>>>> `parent.config`
> > >>>>>>>> files
> > >>>>>>>>>>>> generated by Traffic Ops can vary wildly when an origin
> > >>>>> is
> > >>>>>>>> assigned to
> > >>>>>>>>>>>> multiple Delivery Services. This results in undefined
> > >>>>>>>> behavior. I'm told
> > >>>>>>>>>>>> that the conflict only happens when two Delivery
> > >>>> Services
> > >>>>>>> with
> > >>>>>>>> different
> > >>>>>>>>>>>> "topology requirements" use the same origin, whatever
> > >>>>> that
> > >>>>>>>> means (content
> > >>>>>>>>>>>> routing type?). Regardless, the issue should be
> > >>>>> addressed.
> > >>>>>>> The
> > >>>>>>>> obvious
> > >>>>>>>>>>>> solution is to put in place a database constraint that
> > >>>>>>>> prevents an origin
> > >>>>>>>>>>>> from being assigned to more that one Delivery Service
> > >>>>> with
> > >>>>>>> API
> > >>>>>>>> checks in
> > >>>>>>>>>>>> place that would provide helpful error messages when an
> > >>>>>>>> attempt is made
> > >>>>>>>>>>> to
> > >>>>>>>>>>>> violate the constraint. However, would that mess with
> > >>>>> things
> > >>>>>>>> like
> > >>>>>>>>>>>> Multi-Site Origin? Or is it just not viable for some
> > >>>>> other
> > >>>>>>>> reason? If it
> > >>>>>>>>>>> is
> > >>>>>>>>>>>> a good solution, I'm prepared to work on a fix that
> > >>>>> utilizes
> > >>>>>>>> it.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> >
> >
> >
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Fieck, Brennan" <Br...@comcast.com>.
The issue is inherent with the structure of ATS configuration files. If I understand correctly - someone correct me if I don't -
you can't have more than one rule for a "primary destination" in the parent.config. Or, you can, but only one of them will
wind up catching all of your matches. Which means that if you have multiple Delivery Services that use the same origin FQDN
(dest_domain) then you get proper behavior out of ATS if and only if the resulting rules would be identical. I'd highly
recommend that you not move forward using duplicate origin FQDNs on your Delivery Services, because it probably won't
do what you want.
I'm not sure what you mean about the cachekeys. The Delivery Service URLs should still be unique, so you shouldn't run
into a collision in the cache keys.
Origins are an object. They're also a cachegroup type. And a profile type. And a server type. And a URL field on a Delivery Service.
The solution that's been used in the past to use the same actual origin machine to service multiple Delivery Services safely
is to add a "Canonical Name" DNS record to the ATC internal zone for each Delivery Service. The idea is to have a unique name
for each Delivery Service, but they all resolve to the same IP address.
________________________________________
From: Steve Malenfant <sm...@gmail.com>
Sent: Tuesday, January 29, 2019 4:32 PM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
We use and will use the same origin for multiple DS. It’s an inherent part
of the design of some part of our delivery.
Wouldn’t be less of a problem using cachekeys for different delivery
services?
I would rather see origin as an object and not just a simple URL. I’ve seen
this in the past in other systems.
I don’t believe we can create multiple http services endpoint for a single
DS like we can for DNS type delivery service. This would solve it for us.
Steve
On Thu, Jan 24, 2019 at 11:22 AM Gray, Jonathan <Jo...@comcast.com>
wrote:
> Even that has a strong element of confusion when in a self-service
> multi-tenancy world. Multiple tenants can use the same origins for things
> like akami, aws, internal load balancers and any communication about the
> origin already being used will be confusing.
>
> Jonathan G
>
>
> On 1/23/19, 7:37 PM, "Gelinas, Derek" <De...@comcast.com> wrote:
>
> Perhaps we could solve both issues and just allow it to happen on DS
> config but throw up a message when it is saved that says "hey we're going
> to do this but it's really not without an element of risk and ats isn't
> designed to work properly with multiple identical origins. We recommend you
> use an alternative fqdn blah blah blah."
>
> Something along those lines, anyway.
>
> Derek
>
> > On Jan 23, 2019, at 7:18 PM, Rawlin Peters <ra...@gmail.com>
> wrote:
> >
> > Simply prohibiting duplicate origin FQDNs has been vetoed now, so we
> > can't really move forward with that unless the vetoers change their
> > minds and remove their -1s.
> >
> > Currently that leaves us with checking at DS create/update whether or
> > not it would conflict with a shared origin due to different DS types,
> > qstring config, etc. I am -1 on that idea since I think it will be
> > difficult to maintain and enumerate all the possible conflicting
> > cases, will present confusing error messages to the user to which the
> > solution would just be "create a CNAME record" anyways, and that the
> > ATS parent.config should really be per-remap rather than "global". I
> > know there has been some discussion in the ATS community about making
> > parent.config per-remap, but I don't have much information there.
> >
> > That is to say, I don't think there is a great solution to this
> > problem right now that we can all agree upon, but maybe our efforts
> > would be better spent in the ATS community by making a per-remap
> > parent.config. Then the duplicate origin problem in Traffic Control
> > would go away.
> >
> > Another option could be to have some kind of generic rules engine
> that
> > would allow an organization to create some set of rules that DSes
> > would have to abide by. For example, no duplicate origins, an origin
> > can't be in a list of known malicious domains, you can't use the DS
> > type "HTTP_NO_CACHE", maxDnsAnswers must be < 5, TTLs must be > 30
> and
> > < 6000, protocol must be HTTPS, intitialDispersion must be < 10, DSCP
> > must be 40, etc. It seems like for something truly self-service you'd
> > probably need to set up some rules for your users to abide by that
> > might not necessarily apply to ALL organizations. Just a thought. I
> > don't think the duplicate origins problem alone would warrant
> > something like that, but if it also provided a solution for the
> > self-service effort, maybe it would be worth it.
> >
> > - Rawlin
> >
> > On Wed, Jan 23, 2019 at 4:15 AM Gelinas, Derek
> > <De...@comcast.com> wrote:
> >>
> >> That's the problem. We either need to be smart enough to recognize
> that in our config and warn the user or just prevent it and suggest to the
> user that they use a different fqdn.
> >>
> >> My vote is for preventing it at this point.
> >>
> >>> On Jan 23, 2019, at 12:50 AM, Gray, Jonathan <
> Jonathan_Gray@comcast.com> wrote:
> >>>
> >>> It's not just differing types, it's anything that affects the
> parent.config which also includes the, qstring configuration, MSO
> parents/config, etc. as well.
> >>>
> >>> Jonathan G
> >>>
> >>>
> >>> On 1/22/19, 2:33 PM, "Dave Neuman" <ne...@apache.org> wrote:
> >>>
> >>> -1 on the one DS to Origin limitation. I think there are legit
> use cases
> >>> for it.
> >>> I would be +1 on the idea if we include type though. Two DSs
> can share the
> >>> same origin as long as they are of the same type (DNS, HTTP,
> HTTP_LIVE),
> >>> etc. You shouldn't be able to have two DSs with the same origin
> but
> >>> different types.
> >>>
> >>> On Mon, Jan 14, 2019 at 7:53 AM Fieck, Brennan <
> Brennan_Fieck@comcast.com>
> >>> wrote:
> >>>
> >>>> I don't see this as a complicated use case or a barrier to entry,
> an
> >>>> extremely basic introduction to ATC would only make use of one DS
> >>>> (shameless CDN-in-a-Box plug).
> >>>> It seems far more complicated for a user to be using many DSes to
> serve a
> >>>> single origin - we can easily make it clear in the docs that the
> two have a
> >>>> 1:1 relationship, with a footnote about CNAME records for users
> with the
> >>>> knowledge, resources, and need for such workarounds.
> >>>> ________________________________________
> >>>> From: Mark Torluemke <mt...@apache.org>
> >>>> Sent: Friday, January 11, 2019 1:21 PM
> >>>> To: dev@trafficcontrol.apache.org
> >>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> Services
> >>>> produces indeterminate parent.config
> >>>>
> >>>>>> 1. Prohibit creating new delivery services that would share an
> >>>> existing origin and prohibit updating a delivery service to a
> shared
> >>>> origin
> >>>> In case my position has been lost, I'm still -1 on this. :) IMO,
> we
> >>>> shouldn't let the few complicated use cases increase the barrier
> to entry
> >>>> (to using ATC) for the masses.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Fri, Jan 11, 2019 at 11:53 AM Rawlin Peters <
> rawlin.peters@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> Alright, I'm trying to sum up this discussion so far since it
> seems
> >>>>> like everyone went on vacation and didn't really get a chance to
> wrap
> >>>>> this one up:
> >>>>> - duplicate origins cause undefined behavior
> >>>>> - we need a way to migrate to a future that is free of duplicate
> >>>>> origins in Traffic Control
> >>>>> - we need a visual and easy way to determine if Traffic Ops
> currently
> >>>>> contains duplicate origins, so that operators are incentivized
> to fix
> >>>>> them rather than let it slide indefinitely
> >>>>> - operators should have a fair amount of time to fix their
> duplicate
> >>>>> origins
> >>>>>
> >>>>> I believe this is what we've mostly agreed upon but haven't
> clearly voted
> >>>>> on:
> >>>>>
> >>>>> In release N we will:
> >>>>> 1. Prohibit creating new delivery services that would share an
> >>>>> existing origin and prohibit updating a delivery service to a
> shared
> >>>>> origin
> >>>>> 2. Add some kind of visual indicator that duplicate origins are a
> >>>>> problem that need to be fixed before release N+1; otherwise, an
> >>>>> upgrade to N+1 will be prohibited.
> >>>>>
> >>>>> In release N+1 we will:
> >>>>> 3. Include a DB migration that adds a uniqueness constraint on
> origin
> >>>>> FQDN, removing the API-level checks for that.
> >>>>> 4. Prevent an upgrade to N+1 if duplicate origins are found (this
> >>>>> might occur as a byproduct of step 3).
> >>>>>
> >>>>> I am +1 on this plan and believe this would hit on all the
> summarized
> >>>>> points above. Please provide a clear vote on this plan so that
> we can
> >>>>> dive deeper in the details (i.e. what release 'N' is, the best
> visual
> >>>>> indicator for step 2, and a friendly way to handle step 4).
> >>>>>
> >>>>> Thanks,
> >>>>> Rawlin
> >>>>>
> >>>>> On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <
> mitchell852@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>> Not sure TP is the right place for a warning like "clean up this
> >>>>>> 'duplicate' origin or your next upgrade will fail". Most users
> of our
> >>>>>> system will be like "not my problem".
> >>>>>>
> >>>>>> On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <
> >>>>> Brennan_Fieck@comcast.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Probably. It would impact load times by a bit, but the page
> for an
> >>>>>>> individual object is not our bottleneck.
> >>>>>>> ________________________________________
> >>>>>>> From: Robert Butts <ro...@apache.org>
> >>>>>>> Sent: Wednesday, December 19, 2018 11:50 AM
> >>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> Delivery
> >>>>> Services
> >>>>>>> produces indeterminate parent.config
> >>>>>>>
> >>>>>>>> - Including a warning on startup and an API constraint
> preventing
> >>>>> adding
> >>>>>>> more bad data in the next 3.0.0 Release Candidate
> >>>>>>>> - Adding a database constraint immediately into master that
> won't
> >>>> be
> >>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
> >>>>>>>
> >>>>>>> +1
> >>>>>>>
> >>>>>>> I understand Jonathan's objection, but at some point, we have
> to be
> >>>>> able to
> >>>>>>> move forward. This is a good compromise: deprecate, then
> remove. That
> >>>>> gives
> >>>>>>> people a full major version to fix their data.
> >>>>>>>
> >>>>>>> I would be ideal if it were more than just a logged warning,
> though.
> >>>>> Can we
> >>>>>>> add a big red banner in Traffic Portal, on the Delivery
> Service page
> >>>>> for
> >>>>>>> any DS with a duplicate origin, telling users to fix it, and
> that
> >>>> they
> >>>>>>> won't be able to upgrade to the next major version until it's
> fixed?
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <
> >>>>> Brennan_Fieck@comcast.com
> >>>>>>>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> So it seems like nobody has a problem with the "how" -
> disallowing
> >>>>>>>> duplicate origin FQDNs on Delivery Services - but we never
> reached
> >>>> a
> >>>>>>>> consensus on "when".
> >>>>>>>>
> >>>>>>>> I stand by my previous proposal:
> >>>>>>>> - Including a warning on startup and an API constraint
> preventing
> >>>>> adding
> >>>>>>>> more bad data in the next 3.0.0 Release Candidate
> >>>>>>>> - Adding a database constraint immediately into master that
> won't
> >>>> be
> >>>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
> >>>>>>>> ________________________________________
> >>>>>>>> From: Rawlin Peters <ra...@gmail.com>
> >>>>>>>> Sent: Tuesday, December 18, 2018 4:59 PM
> >>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> Delivery
> >>>>> Services
> >>>>>>>> produces indeterminate parent.config
> >>>>>>>>
> >>>>>>>> Also, building more around DS types will make it even harder
> to get
> >>>>>>>> away from DS types in the future too, which I know is
> something
> >>>> we've
> >>>>>>>> discussed on this mailing list before. It also adds to the
> overhead
> >>>>> of
> >>>>>>>> Delivery Service Topologies, since a lot of the DS types won't
> >>>>>>>> carryover into that world.
> >>>>>>>>
> >>>>>>>> - Rawlin
> >>>>>>>>
> >>>>>>>> On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
> >>>>>>>> <Br...@comcast.com> wrote:
> >>>>>>>>>
> >>>>>>>>> +1.
> >>>>>>>>> If there's a simple way to work around duplicate origins
> being
> >>>>>>>> prohibited,
> >>>>>>>>> then we should rely on that instead of "enumerating all those
> >>>>> possible
> >>>>>>>> conflicting
> >>>>>>>>> settings, which are not only highly complex and confusing,
> but
> >>>> also
> >>>>>>>> further
> >>>>>>>>> entrench us in only supporting ATS as a caching proxy
> (hurting
> >>>>> efforts
> >>>>>>> to
> >>>>>>>>> integrate e.g. Grove, nginx etc.)
> >>>>>>>>> ________________________________________
> >>>>>>>>> From: Rawlin Peters <ra...@gmail.com>
> >>>>>>>>> Sent: Tuesday, December 18, 2018 2:20 PM
> >>>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> Delivery
> >>>>>>>> Services produces indeterminate parent.config
> >>>>>>>>>
> >>>>>>>>> There are a number of different DS settings at play that can
> >>>>>>>>> potentially cause conflicts. The question is: do we want to
> get
> >>>>> into
> >>>>>>>>> the business of enumerating all those possible conflicting
> >>>>> settings or
> >>>>>>>>> just simply prohibit duplicate origins altogether? I think
> we can
> >>>>> dig
> >>>>>>>>> in and get that "sufficiently advanced sql query" to check
> for
> >>>>>>>>> conflicting origins, but is that something we want to carry
> along
> >>>>> for
> >>>>>>>>> the foreseeable future? Aren't CNAMEs relatively cheaper than
> >>>>>>>>> developing and maintaining that code and the mental overhead
> >>>>> required
> >>>>>>>>> in understanding why you're getting an error that says your
> >>>>> requested
> >>>>>>>>> DS would cause an origin conflict? I think at the point
> you've
> >>>>>>>>> requested a DS that would create a conflict, you've chosen
> those
> >>>>>>>>> settings for a reason and would probably prefer to just
> >>>> create/use
> >>>>> a
> >>>>>>>>> CNAME in your new DS and keep the rest of your settings the
> same.
> >>>>>>>>>
> >>>>>>>>> Thinking in terms of errors, I'm imagining:
> >>>>>>>>> "cannot create delivery service: origin fqdn '
> foo.example.com'
> >>>>> already
> >>>>>>>> in use"
> >>>>>>>>> vs
> >>>>>>>>> "cannot create delivery service: origin fqdn '
> foo.example.com'
> >>>>> already
> >>>>>>>>> in use as type DNS_LIVE_NATNL, which is incompatible with
> your
> >>>>> chosen
> >>>>>>>>> type of HTTP_NO_CACHE"
> >>>>>>>>>
> >>>>>>>>> At that point you'd probably say to yourself, "uh, I need
> >>>>>>>>> HTTP_NO_CACHE, so what am I supposed to do now?"
> >>>>>>>>>
> >>>>>>>>> As a lazy developer I'm +1 on prohibiting duplicate origin
> fqdns
> >>>>>>>>> because the resulting code will be simpler, but I think
> >>>> eliminating
> >>>>>>>>> the mental overhead for operators could be worthwhile too.
> If we
> >>>>> can
> >>>>>>>>> agree on an end state of prohibiting duplicate origins
> >>>> altogether,
> >>>>> we
> >>>>>>>>> can start working on a design to smoothly transition us to
> that
> >>>>> point.
> >>>>>>>>> Are we willing to live with "just CNAME your origin fqdn" as
> the
> >>>>>>>>> standard solution to duplicates?
> >>>>>>>>>
> >>>>>>>>> - Rawlin
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
> >>>>>>>>> <De...@comcast.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>> The only situation in which they can share origins is if a)
> the
> >>>>>>>> origins are shared in an MSO configuration but still have
> different
> >>>>>>> defined
> >>>>>>>> origin fields in the delivery service, or if they're assigned
> to
> >>>>>>> completely
> >>>>>>>> different cachegroups. It's when two delivery services share
> the
> >>>>> same
> >>>>>>>> edges that there's an issue, because you end up with
> parent.config
> >>>>>>> issues.
> >>>>>>>> Actually you could even get away with it in mids as long as
> you
> >>>>> weren't
> >>>>>>>> doing anything like MSO to it.
> >>>>>>>>>>
> >>>>>>>>>> Could get messy real fast, though. Best to just create a
> >>>> second
> >>>>>>> FQDN.
> >>>>>>>>>>
> >>>>>>>>>> Derek
> >>>>>>>>>>
> >>>>>>>>>> On 12/18/18, 3:23 PM, "Fieck, Brennan" <
> >>>>> Brennan_Fieck@comcast.com>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> So no two Delivery Services may share an origin
> *regardless
> >>>>> of
> >>>>>>>> cache hierarchy* ? I've been told that DNS Delivery Services
> can
> >>>>> have the
> >>>>>>>> same origin as HTTP Delivery Services because they obey the
> same
> >>>>> cache
> >>>>>>>> hierarchy. You're saying that would still produce invalid
> output
> >>>>> and/or
> >>>>>>> is
> >>>>>>>> explicitly disallowed by ATS?
> >>>>>>>>>> ________________________________________
> >>>>>>>>>> From: Robert Butts <ro...@apache.org>
> >>>>>>>>>> Sent: Tuesday, December 18, 2018 1:09 PM
> >>>>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> >>>>> Delivery
> >>>>>>>> Services produces indeterminate parent.config
> >>>>>>>>>>
> >>>>>>>>>>> can you give an example of what parent.config looks like
> >>>>> when 2
> >>>>>>>> ds's share
> >>>>>>>>>> an origin and have different a different topology?
> >>>>>>>>>>
> >>>>>>>>>> Answering because I encountered this directly, when
> >>>> rewriting
> >>>>>>>> parent.config.
> >>>>>>>>>>
> >>>>>>>>>> For example: Suppose you have one Delivery Service:
> >>>>>>>>>> XML_ID: foo
> >>>>>>>>>> Type: HTPT_LIVE_NATL
> >>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass
> up
> >>>>>>>>>> Origin Server Base URL: http://foo.example.net
> >>>>>>>>>>
> >>>>>>>>>> And another Delivery Service:
> >>>>>>>>>> XML_ID: bar
> >>>>>>>>>> Type: HTPT_LIVE_NATL
> >>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass
> up
> >>>>>>>>>> Origin Server Base URL: http://foo.example.net
> >>>>>>>>>>
> >>>>>>>>>> ATS only supports unique `dest_domain` entries in
> >>>>> parent.config.
> >>>>>>>> Therefore,
> >>>>>>>>>> the parent.config generated for a server assigned to both
> >>>> of
> >>>>>>> these
> >>>>>>>> Delivery
> >>>>>>>>>> Services with either be:
> >>>>>>>>>>
> >>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=true
> >>>>>>>>>>
> >>>>>>>>>> Or
> >>>>>>>>>>
> >>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=false
> >>>>>>>> qstring=consider
> >>>>>>>>>>
> >>>>>>>>>> Right now, it's arbitrary which Perl Traffic Ops inserts,
> >>>> and
> >>>>>>> Perl
> >>>>>>>> provides
> >>>>>>>>>> no warning or error of any kind (the pending Go
> >>>>> parent.config PR
> >>>>>>>> logs an
> >>>>>>>>>> error).
> >>>>>>>>>>
> >>>>>>>>>> Whichever is arbitrarily inserted, the resulting remaps
> for
> >>>>> the
> >>>>>>>> other
> >>>>>>>>>> delivery service will be wrong. Either `foo` requests will
> >>>>> drop
> >>>>>>>> the query
> >>>>>>>>>> string when they shouldn't, and go to the mid when they
> >>>>>>> shouldn't;
> >>>>>>>> or `bar`
> >>>>>>>>>> requests will use the query string and skip the mid when
> it
> >>>>>>>> shouldn't.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Does that make sense? The only correct solution, is to
> >>>>> somehow
> >>>>>>>> prevent
> >>>>>>>>>> different DSes having the same origin, and tell tenants
> >>>> they
> >>>>> must
> >>>>>>>> use
> >>>>>>>>>> CNAMEs if they need.
> >>>>>>>>>>
> >>>>>>>>>> This isn't a bug in Traffic Control. ATS fundamentally
> >>>>> doesn't
> >>>>>>>> support
> >>>>>>>>>> multiple remap rules with the same parent FQDN with
> >>>> different
> >>>>>>>>>> configurations. Hence, Traffic Control needs to prohibit
> >>>>> that.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
> >>>>>>>> mitchell852@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> brennan,
> >>>>>>>>>>>
> >>>>>>>>>>> can you give an example of what parent.config looks like
> >>>>> when 2
> >>>>>>>> ds's share
> >>>>>>>>>>> an origin and have different a different topology?
> >>>>>>>>>>>
> >>>>>>>>>>> jeremy
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
> >>>>>>>> Brennan_Fieck@comcast.com
> >>>>>>>>>>>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> To be clear, the "Warning" I'm talking about would
> >>>>> happen at
> >>>>>>>> startup, but
> >>>>>>>>>>>> I'd like a UI-only constraint to come with that to
> >>>>> disallow
> >>>>>>>> using the API
> >>>>>>>>>>>> to bind the same origin to multiple Delivery Services
> >>>>> with
> >>>>>>>> varying
> >>>>>>>>>>>> topography requirements. It wouldn't change the
> >>>> existing
> >>>>>>> data,
> >>>>>>>> but
> >>>>>>>>>>> prevent
> >>>>>>>>>>>> users from creating more bad data.
> >>>>>>>>>>>>
> >>>>>>>>>>>> "warning" doesn't really sufficiently describe that, my
> >>>>> bad.
> >>>>>>>>>>>> ________________________________________
> >>>>>>>>>>>> From: Fieck, Brennan <Br...@comcast.com>
> >>>>>>>>>>>> Sent: Tuesday, December 18, 2018 11:24 AM
> >>>>>>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> >>>>>>>> Delivery Services
> >>>>>>>>>>>> produces indeterminate parent.config
> >>>>>>>>>>>>
> >>>>>>>>>>>> Well the cost of fixing this bug is a constraint on the
> >>>>> data.
> >>>>>>>> Unless we
> >>>>>>>>>>>> make it a UI-only constraint - which I'm personally
> >>>>> against -
> >>>>>>>> there must
> >>>>>>>>>>> be
> >>>>>>>>>>>> some point in the future where ATC cannot reasonably be
> >>>>>>>> expected to work
> >>>>>>>>>>>> with data that violates that constraint. The question
> >>>> is
> >>>>> when
> >>>>>>>> that should
> >>>>>>>>>>>> occur, which should likely happen at a minor version
> >>>>> release.
> >>>>>>>> Minor not
> >>>>>>>>>>>> major because it doesn't involve a change in data
> >>>>> structures,
> >>>>>>>> merely
> >>>>>>>>>>>> relationships between them - in my opinion that's a
> >>>> minor
> >>>>>>>> version change
> >>>>>>>>>>>> but that's definitely up for debate. With several
> >>>> release
> >>>>>>>> candidates for
> >>>>>>>>>>>> 3.0.0 that _doesn't_ include this restriction already
> >>>> in
> >>>>> the
> >>>>>>>> wild, I
> >>>>>>>>>>>> wouldn't recommend putting it in there. That means to
> >>>>> fix the
> >>>>>>>> bug as soon
> >>>>>>>>>>>> as possible it should go in 3.1.0 which should be the
> >>>>> target
> >>>>>>>> of "master"
> >>>>>>>>>>>> after the 3.0.0 release is cut from it.
> >>>>>>>>>>>>
> >>>>>>>>>>>> So I'd recommend immediately implementing the
> >>>> constraint
> >>>>> in
> >>>>>>>> master with a
> >>>>>>>>>>>> refusal to upgrade with bad data, and backport a
> >>>> warning
> >>>>>>> about
> >>>>>>>> the future
> >>>>>>>>>>>> behavior into 3.0.0 or as part of a 3.0.1 provided we
> >>>> had
> >>>>>>> more
> >>>>>>>> changes
> >>>>>>>>>>> that
> >>>>>>>>>>>> would warrant a micro version bump.
> >>>>>>>>>>>> ________________________________________
> >>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
> >>>>>>>>>>>> Sent: Tuesday, December 18, 2018 9:34 AM
> >>>>>>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> >>>>>>>> Delivery Services
> >>>>>>>>>>>> produces indeterminate parent.config
> >>>>>>>>>>>>
> >>>>>>>>>>>> -1 Holding an ATC upgrade hostage to data cleanup seems
> >>>>> like
> >>>>>>> a
> >>>>>>>> bad idea.
> >>>>>>>>>>>> The issue isn't great, but it's also not new. We
> >>>> should
> >>>>>>> allow
> >>>>>>>> teams to
> >>>>>>>>>>> fix
> >>>>>>>>>>>> their data at their normal paces if it doesn't create
> >>>>>>>> significant
> >>>>>>>>>>> overhead
> >>>>>>>>>>>> or an inherant blocker for new functionality or
> >>>>> correction of
> >>>>>>>> other major
> >>>>>>>>>>>> problems imho.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Jonathan G
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 12/18/18, 9:28 AM, "Fieck, Brennan" <
> >>>>>>>> Brennan_Fieck@comcast.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Another option is we could detect collisions at
> >>>>> startup
> >>>>>>>> and simply
> >>>>>>>>>>>> refuse to continue with the upgrade until the data is
> >>>>> fixed.
> >>>>>>>> That would
> >>>>>>>>>>>> allow people using the now-unsupported data format to
> >>>>>>> continue
> >>>>>>>> to use
> >>>>>>>>>>> their
> >>>>>>>>>>>> old versions of Traffic Ops without wrecking their
> >>>>> database,
> >>>>>>>> but also
> >>>>>>>>>>>> provide an incentive to clean up the data.
> >>>>>>>>>>>> ________________________________________
> >>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
> >>>>>>>>>>>> Sent: Tuesday, December 18, 2018 5:12 AM
> >>>>>>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to
> >>>>> Multipe
> >>>>>>>> Delivery
> >>>>>>>>>>>> Services produces indeterminate parent.config
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'm generally a fan of constrain your data in your
> >>>>>>>> database, but not
> >>>>>>>>>>>> necessarily exclusively. I see this as a one-way
> >>>>>>>> cleanup/conversion so
> >>>>>>>>>>> it
> >>>>>>>>>>>> doesn't need to be configurable; otherwise you have to
> >>>>> ask
> >>>>>>> the
> >>>>>>>> question
> >>>>>>>>>>>> what happens if someone turns it off. That said,
> >>>>> something
> >>>>>>> in
> >>>>>>>> the UI
> >>>>>>>>>>> layer
> >>>>>>>>>>>> would be nice to prevent spending significant
> >>>> quantities
> >>>>> of
> >>>>>>>> time
> >>>>>>>>>>> building a
> >>>>>>>>>>>> complex DS only to have it fail to post for reasons
> >>>> that
> >>>>>>> could
> >>>>>>>> have been
> >>>>>>>>>>>> known earlier.
> >>>>>>>>>>>>
> >>>>>>>>>>>> The way my brain works in this case:
> >>>>>>>>>>>> If !unique_constraint_exists_query()
> >>>>>>>>>>>> If has_duplicates_query()
> >>>>>>>>>>>> show_warning()
> >>>>>>>>>>>> else
> >>>>>>>>>>>> add_unique_constraint()
> >>>>>>>>>>>>
> >>>>>>>>>>>> to which the API and UI configuration could also
> >>>>> make use
> >>>>>>>> of
> >>>>>>>>>>>> unique_constraint_exists_query() to drive additional
> >>>>> layer
> >>>>>>>> constraints if
> >>>>>>>>>>>> desired.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Jonathan G
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 12/17/18, 1:11 PM, "Rawlin Peters" <
> >>>>>>>> rawlin.peters@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> That is an interesting idea...detect at TO
> >>>>> startup
> >>>>>>>> whether or not
> >>>>>>>>>>>> there are duplicate origins and operate in a
> >>>>> "prevent
> >>>>>>>> duplicate
> >>>>>>>>>>>> origins" state if no duplicates are found or
> >>>>> "prevent
> >>>>>>>> conflicting
> >>>>>>>>>>>> DS
> >>>>>>>>>>>> topologies" state if duplicates are found? So
> >>>>> once
> >>>>>>>> operators have
> >>>>>>>>>>>> replaced all the duplicate origins with CNAMEs,
> >>>>> TO
> >>>>>>> will
> >>>>>>>>>>> essentially
> >>>>>>>>>>>> operate in a "prohibit all duplicate origins"
> >>>>> state.
> >>>>>>>> That would
> >>>>>>>>>>>> probably make for a simpler transition, but I'd
> >>>>> want
> >>>>>>>> to remove
> >>>>>>>>>>> that
> >>>>>>>>>>>> logic in a following release that strictly
> >>>>> prohibits
> >>>>>>>> duplicate
> >>>>>>>>>>>> origins
> >>>>>>>>>>>> (assuming that the community agrees we should
> >>>>>>> prohibit
> >>>>>>>> duplicate
> >>>>>>>>>>>> origins altogether).
> >>>>>>>>>>>>
> >>>>>>>>>>>> As for DB constraints vs UI, I was thinking
> >>>> those
> >>>>>>>> DS-type
> >>>>>>>>>>>> constraints
> >>>>>>>>>>>> I pointed out would live in the API. It would
> >>>>>>>> basically be added
> >>>>>>>>>>>> validation in the deliveryservices POST/PUT
> >>>>> endpoint
> >>>>>>>> that checks
> >>>>>>>>>>>> the
> >>>>>>>>>>>> DB for existing DSes that conflict with the
> >>>>> requested
> >>>>>>>> DS.
> >>>>>>>>>>>>
> >>>>>>>>>>>> - Rawlin
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> >>>>>>>>>>>> <Jo...@comcast.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> These kinds of conditions should be
> >>>> detectable
> >>>>>>> with a
> >>>>>>>>>>>> sufficiently advanced SQL query. Is it possible to add
> >>>>> the
> >>>>>>>> constraint if
> >>>>>>>>>>>> it passes and emit a warning during TO startup
> >>>> otherwise?
> >>>>>>>> That would let
> >>>>>>>>>>>> you know the condition exists at startup but not
> >>>> getting
> >>>>> in
> >>>>>>>> your way and
> >>>>>>>>>>>> keep you out of trouble once you've cleaned up. We
> >>>> made
> >>>>> a
> >>>>>>>> mistake early
> >>>>>>>>>>>> on, but this would acknowledge it was bad and encourage
> >>>>> it to
> >>>>>>>> be fixed at
> >>>>>>>>>>>> the speed of operations teams. Also this puts the
> >>>>> constraint
> >>>>>>>> in the
> >>>>>>>>>>>> database rather than the UI which is really where the
> >>>>>>>> contention is for
> >>>>>>>>>>>> usability.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Jonathan G
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On 12/17/18, 11:38 AM, "Rawlin Peters" <
> >>>>>>>>>>> rawlin.peters@gmail.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> We occasionally discuss this issue but
> >>>>> haven't
> >>>>>>>> tackled it
> >>>>>>>>>>>> yet. I think
> >>>>>>>>>>>>> the main issue is just that duplicate
> >>>>> origins
> >>>>>>>> have been
> >>>>>>>>>>>> allowed since
> >>>>>>>>>>>>> the beginning, and now everyone's Traffic
> >>>>> Ops
> >>>>>>>> could be
> >>>>>>>>>>>> littered with
> >>>>>>>>>>>>> duplicate origins. Also, depending on the
> >>>>>>> config
> >>>>>>>> of the
> >>>>>>>>>>>> duplicate
> >>>>>>>>>>>>> delivery services, the origins might not
> >>>>> be in
> >>>>>>>> conflict at
> >>>>>>>>>>>> all (if
> >>>>>>>>>>>>> they don't have different topology
> >>>>>>> constraints).
> >>>>>>>> I would
> >>>>>>>>>>>> love for us
> >>>>>>>>>>>>> to just add a uniqueness constraint, but
> >>>>> there
> >>>>>>>> would need
> >>>>>>>>>>> to
> >>>>>>>>>>>> be a fair
> >>>>>>>>>>>>> amount of warning to the community before
> >>>>> doing
> >>>>>>>> so and
> >>>>>>>>>>> might
> >>>>>>>>>>>>> invalidate a significant amount of valid
> >>>>> use
> >>>>>>>> cases.
> >>>>>>>>>>>> Operators would
> >>>>>>>>>>>>> need time to make DNS CNAME records for
> >>>> the
> >>>>>>>> duplicate
> >>>>>>>>>>>> origins and
> >>>>>>>>>>>>> update their DSes to use the different
> >>>>> CNAMEs.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I think as a good first step to
> >>>>> eliminating the
> >>>>>>>> use of
> >>>>>>>>>>>> duplicate
> >>>>>>>>>>>>> origins altogether, we should identify
> >>>>> which
> >>>>>>>> "topology
> >>>>>>>>>>>> constraints"
> >>>>>>>>>>>>> actually cause conflicting config when
> >>>> used
> >>>>>>> with
> >>>>>>>> duplicate
> >>>>>>>>>>>> origins and
> >>>>>>>>>>>>> prevent creating DSes with duplicate
> >>>>> origins
> >>>>>>> _if
> >>>>>>>> it would
> >>>>>>>>>>>> cause a
> >>>>>>>>>>>>> conflict with an existing DS that uses
> >>>> the
> >>>>> same
> >>>>>>>> origin_.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> For instance, I believe an HTTP and
> >>>>> DNS-type DS
> >>>>>>>> can live
> >>>>>>>>>>>> happily
> >>>>>>>>>>>>> side-by-side using the same origin
> >>>>> (probably
> >>>>>>>> need different
> >>>>>>>>>>>>> routing_names?), but scenarios like HTTP
> >>>>> and
> >>>>>>>> HTTP_LIVE, or
> >>>>>>>>>>>> DNS and
> >>>>>>>>>>>>> HTTP_NO_CACHE sharing the same origin
> >>>> will
> >>>>>>> cause
> >>>>>>>> conflicts
> >>>>>>>>>>>> for sure.
> >>>>>>>>>>>>> So maybe we can start by making sure the
> >>>> DS
> >>>>>>>> types "match"
> >>>>>>>>>>>> when using
> >>>>>>>>>>>>> the same origin:
> >>>>>>>>>>>>> HTTP + DNS: possibly good, if they have
> >>>>>>>> different routing
> >>>>>>>>>>>> names?
> >>>>>>>>>>>>> HTTP_LIVE + HTTP_LIVE_NATNL: bad
> >>>>>>>>>>>>> HTTP_NO_CACHE + [any other type]: bad
> >>>>>>>>>>>>> HTTP_LIVE + HTTP: bad
> >>>>>>>>>>>>> etc.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> There are most likely other conflict
> >>>>> scenarios
> >>>>>>>> that don't
> >>>>>>>>>>>> involve the
> >>>>>>>>>>>>> DS types, but I think this would be a
> >>>> good
> >>>>>>>> start. In the
> >>>>>>>>>>>> future with
> >>>>>>>>>>>>> Delivery Service Topologies (aka Flexible
> >>>>>>>> Cachegroups aka
> >>>>>>>>>>>> Bring Your
> >>>>>>>>>>>>> Own Topology), we might be able to
> >>>> prohibit
> >>>>>>>> assigning a DS
> >>>>>>>>>>>> to a
> >>>>>>>>>>>>> Topology if the DS's origin is already
> >>>>> used by
> >>>>>>>> another DS
> >>>>>>>>>>> in
> >>>>>>>>>>>> a
> >>>>>>>>>>>>> different Topology.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> - Rawlin
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Dec 17, 2018 at 10:52 AM Fieck,
> >>>>> Brennan
> >>>>>>>>>>>>> <Br...@comcast.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> As some of you may be aware,
> >>>>> `parent.config`
> >>>>>>>> files
> >>>>>>>>>>>> generated by Traffic Ops can vary wildly when an origin
> >>>>> is
> >>>>>>>> assigned to
> >>>>>>>>>>>> multiple Delivery Services. This results in undefined
> >>>>>>>> behavior. I'm told
> >>>>>>>>>>>> that the conflict only happens when two Delivery
> >>>> Services
> >>>>>>> with
> >>>>>>>> different
> >>>>>>>>>>>> "topology requirements" use the same origin, whatever
> >>>>> that
> >>>>>>>> means (content
> >>>>>>>>>>>> routing type?). Regardless, the issue should be
> >>>>> addressed.
> >>>>>>> The
> >>>>>>>> obvious
> >>>>>>>>>>>> solution is to put in place a database constraint that
> >>>>>>>> prevents an origin
> >>>>>>>>>>>> from being assigned to more that one Delivery Service
> >>>>> with
> >>>>>>> API
> >>>>>>>> checks in
> >>>>>>>>>>>> place that would provide helpful error messages when an
> >>>>>>>> attempt is made
> >>>>>>>>>>> to
> >>>>>>>>>>>> violate the constraint. However, would that mess with
> >>>>> things
> >>>>>>>> like
> >>>>>>>>>>>> Multi-Site Origin? Or is it just not viable for some
> >>>>> other
> >>>>>>>> reason? If it
> >>>>>>>>>>> is
> >>>>>>>>>>>> a good solution, I'm prepared to work on a fix that
> >>>>> utilizes
> >>>>>>>> it.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>>
> >>>
>
>
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Steve Malenfant <sm...@gmail.com>.
We use and will use the same origin for multiple DS. It’s an inherent part
of the design of some part of our delivery.
Wouldn’t be less of a problem using cachekeys for different delivery
services?
I would rather see origin as an object and not just a simple URL. I’ve seen
this in the past in other systems.
I don’t believe we can create multiple http services endpoint for a single
DS like we can for DNS type delivery service. This would solve it for us.
Steve
On Thu, Jan 24, 2019 at 11:22 AM Gray, Jonathan <Jo...@comcast.com>
wrote:
> Even that has a strong element of confusion when in a self-service
> multi-tenancy world. Multiple tenants can use the same origins for things
> like akami, aws, internal load balancers and any communication about the
> origin already being used will be confusing.
>
> Jonathan G
>
>
> On 1/23/19, 7:37 PM, "Gelinas, Derek" <De...@comcast.com> wrote:
>
> Perhaps we could solve both issues and just allow it to happen on DS
> config but throw up a message when it is saved that says "hey we're going
> to do this but it's really not without an element of risk and ats isn't
> designed to work properly with multiple identical origins. We recommend you
> use an alternative fqdn blah blah blah."
>
> Something along those lines, anyway.
>
> Derek
>
> > On Jan 23, 2019, at 7:18 PM, Rawlin Peters <ra...@gmail.com>
> wrote:
> >
> > Simply prohibiting duplicate origin FQDNs has been vetoed now, so we
> > can't really move forward with that unless the vetoers change their
> > minds and remove their -1s.
> >
> > Currently that leaves us with checking at DS create/update whether or
> > not it would conflict with a shared origin due to different DS types,
> > qstring config, etc. I am -1 on that idea since I think it will be
> > difficult to maintain and enumerate all the possible conflicting
> > cases, will present confusing error messages to the user to which the
> > solution would just be "create a CNAME record" anyways, and that the
> > ATS parent.config should really be per-remap rather than "global". I
> > know there has been some discussion in the ATS community about making
> > parent.config per-remap, but I don't have much information there.
> >
> > That is to say, I don't think there is a great solution to this
> > problem right now that we can all agree upon, but maybe our efforts
> > would be better spent in the ATS community by making a per-remap
> > parent.config. Then the duplicate origin problem in Traffic Control
> > would go away.
> >
> > Another option could be to have some kind of generic rules engine
> that
> > would allow an organization to create some set of rules that DSes
> > would have to abide by. For example, no duplicate origins, an origin
> > can't be in a list of known malicious domains, you can't use the DS
> > type "HTTP_NO_CACHE", maxDnsAnswers must be < 5, TTLs must be > 30
> and
> > < 6000, protocol must be HTTPS, intitialDispersion must be < 10, DSCP
> > must be 40, etc. It seems like for something truly self-service you'd
> > probably need to set up some rules for your users to abide by that
> > might not necessarily apply to ALL organizations. Just a thought. I
> > don't think the duplicate origins problem alone would warrant
> > something like that, but if it also provided a solution for the
> > self-service effort, maybe it would be worth it.
> >
> > - Rawlin
> >
> > On Wed, Jan 23, 2019 at 4:15 AM Gelinas, Derek
> > <De...@comcast.com> wrote:
> >>
> >> That's the problem. We either need to be smart enough to recognize
> that in our config and warn the user or just prevent it and suggest to the
> user that they use a different fqdn.
> >>
> >> My vote is for preventing it at this point.
> >>
> >>> On Jan 23, 2019, at 12:50 AM, Gray, Jonathan <
> Jonathan_Gray@comcast.com> wrote:
> >>>
> >>> It's not just differing types, it's anything that affects the
> parent.config which also includes the, qstring configuration, MSO
> parents/config, etc. as well.
> >>>
> >>> Jonathan G
> >>>
> >>>
> >>> On 1/22/19, 2:33 PM, "Dave Neuman" <ne...@apache.org> wrote:
> >>>
> >>> -1 on the one DS to Origin limitation. I think there are legit
> use cases
> >>> for it.
> >>> I would be +1 on the idea if we include type though. Two DSs
> can share the
> >>> same origin as long as they are of the same type (DNS, HTTP,
> HTTP_LIVE),
> >>> etc. You shouldn't be able to have two DSs with the same origin
> but
> >>> different types.
> >>>
> >>> On Mon, Jan 14, 2019 at 7:53 AM Fieck, Brennan <
> Brennan_Fieck@comcast.com>
> >>> wrote:
> >>>
> >>>> I don't see this as a complicated use case or a barrier to entry,
> an
> >>>> extremely basic introduction to ATC would only make use of one DS
> >>>> (shameless CDN-in-a-Box plug).
> >>>> It seems far more complicated for a user to be using many DSes to
> serve a
> >>>> single origin - we can easily make it clear in the docs that the
> two have a
> >>>> 1:1 relationship, with a footnote about CNAME records for users
> with the
> >>>> knowledge, resources, and need for such workarounds.
> >>>> ________________________________________
> >>>> From: Mark Torluemke <mt...@apache.org>
> >>>> Sent: Friday, January 11, 2019 1:21 PM
> >>>> To: dev@trafficcontrol.apache.org
> >>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> Services
> >>>> produces indeterminate parent.config
> >>>>
> >>>>>> 1. Prohibit creating new delivery services that would share an
> >>>> existing origin and prohibit updating a delivery service to a
> shared
> >>>> origin
> >>>> In case my position has been lost, I'm still -1 on this. :) IMO,
> we
> >>>> shouldn't let the few complicated use cases increase the barrier
> to entry
> >>>> (to using ATC) for the masses.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Fri, Jan 11, 2019 at 11:53 AM Rawlin Peters <
> rawlin.peters@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> Alright, I'm trying to sum up this discussion so far since it
> seems
> >>>>> like everyone went on vacation and didn't really get a chance to
> wrap
> >>>>> this one up:
> >>>>> - duplicate origins cause undefined behavior
> >>>>> - we need a way to migrate to a future that is free of duplicate
> >>>>> origins in Traffic Control
> >>>>> - we need a visual and easy way to determine if Traffic Ops
> currently
> >>>>> contains duplicate origins, so that operators are incentivized
> to fix
> >>>>> them rather than let it slide indefinitely
> >>>>> - operators should have a fair amount of time to fix their
> duplicate
> >>>>> origins
> >>>>>
> >>>>> I believe this is what we've mostly agreed upon but haven't
> clearly voted
> >>>>> on:
> >>>>>
> >>>>> In release N we will:
> >>>>> 1. Prohibit creating new delivery services that would share an
> >>>>> existing origin and prohibit updating a delivery service to a
> shared
> >>>>> origin
> >>>>> 2. Add some kind of visual indicator that duplicate origins are a
> >>>>> problem that need to be fixed before release N+1; otherwise, an
> >>>>> upgrade to N+1 will be prohibited.
> >>>>>
> >>>>> In release N+1 we will:
> >>>>> 3. Include a DB migration that adds a uniqueness constraint on
> origin
> >>>>> FQDN, removing the API-level checks for that.
> >>>>> 4. Prevent an upgrade to N+1 if duplicate origins are found (this
> >>>>> might occur as a byproduct of step 3).
> >>>>>
> >>>>> I am +1 on this plan and believe this would hit on all the
> summarized
> >>>>> points above. Please provide a clear vote on this plan so that
> we can
> >>>>> dive deeper in the details (i.e. what release 'N' is, the best
> visual
> >>>>> indicator for step 2, and a friendly way to handle step 4).
> >>>>>
> >>>>> Thanks,
> >>>>> Rawlin
> >>>>>
> >>>>> On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <
> mitchell852@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>> Not sure TP is the right place for a warning like "clean up this
> >>>>>> 'duplicate' origin or your next upgrade will fail". Most users
> of our
> >>>>>> system will be like "not my problem".
> >>>>>>
> >>>>>> On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <
> >>>>> Brennan_Fieck@comcast.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Probably. It would impact load times by a bit, but the page
> for an
> >>>>>>> individual object is not our bottleneck.
> >>>>>>> ________________________________________
> >>>>>>> From: Robert Butts <ro...@apache.org>
> >>>>>>> Sent: Wednesday, December 19, 2018 11:50 AM
> >>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> Delivery
> >>>>> Services
> >>>>>>> produces indeterminate parent.config
> >>>>>>>
> >>>>>>>> - Including a warning on startup and an API constraint
> preventing
> >>>>> adding
> >>>>>>> more bad data in the next 3.0.0 Release Candidate
> >>>>>>>> - Adding a database constraint immediately into master that
> won't
> >>>> be
> >>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
> >>>>>>>
> >>>>>>> +1
> >>>>>>>
> >>>>>>> I understand Jonathan's objection, but at some point, we have
> to be
> >>>>> able to
> >>>>>>> move forward. This is a good compromise: deprecate, then
> remove. That
> >>>>> gives
> >>>>>>> people a full major version to fix their data.
> >>>>>>>
> >>>>>>> I would be ideal if it were more than just a logged warning,
> though.
> >>>>> Can we
> >>>>>>> add a big red banner in Traffic Portal, on the Delivery
> Service page
> >>>>> for
> >>>>>>> any DS with a duplicate origin, telling users to fix it, and
> that
> >>>> they
> >>>>>>> won't be able to upgrade to the next major version until it's
> fixed?
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <
> >>>>> Brennan_Fieck@comcast.com
> >>>>>>>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> So it seems like nobody has a problem with the "how" -
> disallowing
> >>>>>>>> duplicate origin FQDNs on Delivery Services - but we never
> reached
> >>>> a
> >>>>>>>> consensus on "when".
> >>>>>>>>
> >>>>>>>> I stand by my previous proposal:
> >>>>>>>> - Including a warning on startup and an API constraint
> preventing
> >>>>> adding
> >>>>>>>> more bad data in the next 3.0.0 Release Candidate
> >>>>>>>> - Adding a database constraint immediately into master that
> won't
> >>>> be
> >>>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
> >>>>>>>> ________________________________________
> >>>>>>>> From: Rawlin Peters <ra...@gmail.com>
> >>>>>>>> Sent: Tuesday, December 18, 2018 4:59 PM
> >>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> Delivery
> >>>>> Services
> >>>>>>>> produces indeterminate parent.config
> >>>>>>>>
> >>>>>>>> Also, building more around DS types will make it even harder
> to get
> >>>>>>>> away from DS types in the future too, which I know is
> something
> >>>> we've
> >>>>>>>> discussed on this mailing list before. It also adds to the
> overhead
> >>>>> of
> >>>>>>>> Delivery Service Topologies, since a lot of the DS types won't
> >>>>>>>> carryover into that world.
> >>>>>>>>
> >>>>>>>> - Rawlin
> >>>>>>>>
> >>>>>>>> On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
> >>>>>>>> <Br...@comcast.com> wrote:
> >>>>>>>>>
> >>>>>>>>> +1.
> >>>>>>>>> If there's a simple way to work around duplicate origins
> being
> >>>>>>>> prohibited,
> >>>>>>>>> then we should rely on that instead of "enumerating all those
> >>>>> possible
> >>>>>>>> conflicting
> >>>>>>>>> settings, which are not only highly complex and confusing,
> but
> >>>> also
> >>>>>>>> further
> >>>>>>>>> entrench us in only supporting ATS as a caching proxy
> (hurting
> >>>>> efforts
> >>>>>>> to
> >>>>>>>>> integrate e.g. Grove, nginx etc.)
> >>>>>>>>> ________________________________________
> >>>>>>>>> From: Rawlin Peters <ra...@gmail.com>
> >>>>>>>>> Sent: Tuesday, December 18, 2018 2:20 PM
> >>>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> Delivery
> >>>>>>>> Services produces indeterminate parent.config
> >>>>>>>>>
> >>>>>>>>> There are a number of different DS settings at play that can
> >>>>>>>>> potentially cause conflicts. The question is: do we want to
> get
> >>>>> into
> >>>>>>>>> the business of enumerating all those possible conflicting
> >>>>> settings or
> >>>>>>>>> just simply prohibit duplicate origins altogether? I think
> we can
> >>>>> dig
> >>>>>>>>> in and get that "sufficiently advanced sql query" to check
> for
> >>>>>>>>> conflicting origins, but is that something we want to carry
> along
> >>>>> for
> >>>>>>>>> the foreseeable future? Aren't CNAMEs relatively cheaper than
> >>>>>>>>> developing and maintaining that code and the mental overhead
> >>>>> required
> >>>>>>>>> in understanding why you're getting an error that says your
> >>>>> requested
> >>>>>>>>> DS would cause an origin conflict? I think at the point
> you've
> >>>>>>>>> requested a DS that would create a conflict, you've chosen
> those
> >>>>>>>>> settings for a reason and would probably prefer to just
> >>>> create/use
> >>>>> a
> >>>>>>>>> CNAME in your new DS and keep the rest of your settings the
> same.
> >>>>>>>>>
> >>>>>>>>> Thinking in terms of errors, I'm imagining:
> >>>>>>>>> "cannot create delivery service: origin fqdn '
> foo.example.com'
> >>>>> already
> >>>>>>>> in use"
> >>>>>>>>> vs
> >>>>>>>>> "cannot create delivery service: origin fqdn '
> foo.example.com'
> >>>>> already
> >>>>>>>>> in use as type DNS_LIVE_NATNL, which is incompatible with
> your
> >>>>> chosen
> >>>>>>>>> type of HTTP_NO_CACHE"
> >>>>>>>>>
> >>>>>>>>> At that point you'd probably say to yourself, "uh, I need
> >>>>>>>>> HTTP_NO_CACHE, so what am I supposed to do now?"
> >>>>>>>>>
> >>>>>>>>> As a lazy developer I'm +1 on prohibiting duplicate origin
> fqdns
> >>>>>>>>> because the resulting code will be simpler, but I think
> >>>> eliminating
> >>>>>>>>> the mental overhead for operators could be worthwhile too.
> If we
> >>>>> can
> >>>>>>>>> agree on an end state of prohibiting duplicate origins
> >>>> altogether,
> >>>>> we
> >>>>>>>>> can start working on a design to smoothly transition us to
> that
> >>>>> point.
> >>>>>>>>> Are we willing to live with "just CNAME your origin fqdn" as
> the
> >>>>>>>>> standard solution to duplicates?
> >>>>>>>>>
> >>>>>>>>> - Rawlin
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
> >>>>>>>>> <De...@comcast.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>> The only situation in which they can share origins is if a)
> the
> >>>>>>>> origins are shared in an MSO configuration but still have
> different
> >>>>>>> defined
> >>>>>>>> origin fields in the delivery service, or if they're assigned
> to
> >>>>>>> completely
> >>>>>>>> different cachegroups. It's when two delivery services share
> the
> >>>>> same
> >>>>>>>> edges that there's an issue, because you end up with
> parent.config
> >>>>>>> issues.
> >>>>>>>> Actually you could even get away with it in mids as long as
> you
> >>>>> weren't
> >>>>>>>> doing anything like MSO to it.
> >>>>>>>>>>
> >>>>>>>>>> Could get messy real fast, though. Best to just create a
> >>>> second
> >>>>>>> FQDN.
> >>>>>>>>>>
> >>>>>>>>>> Derek
> >>>>>>>>>>
> >>>>>>>>>> On 12/18/18, 3:23 PM, "Fieck, Brennan" <
> >>>>> Brennan_Fieck@comcast.com>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> So no two Delivery Services may share an origin
> *regardless
> >>>>> of
> >>>>>>>> cache hierarchy* ? I've been told that DNS Delivery Services
> can
> >>>>> have the
> >>>>>>>> same origin as HTTP Delivery Services because they obey the
> same
> >>>>> cache
> >>>>>>>> hierarchy. You're saying that would still produce invalid
> output
> >>>>> and/or
> >>>>>>> is
> >>>>>>>> explicitly disallowed by ATS?
> >>>>>>>>>> ________________________________________
> >>>>>>>>>> From: Robert Butts <ro...@apache.org>
> >>>>>>>>>> Sent: Tuesday, December 18, 2018 1:09 PM
> >>>>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> >>>>> Delivery
> >>>>>>>> Services produces indeterminate parent.config
> >>>>>>>>>>
> >>>>>>>>>>> can you give an example of what parent.config looks like
> >>>>> when 2
> >>>>>>>> ds's share
> >>>>>>>>>> an origin and have different a different topology?
> >>>>>>>>>>
> >>>>>>>>>> Answering because I encountered this directly, when
> >>>> rewriting
> >>>>>>>> parent.config.
> >>>>>>>>>>
> >>>>>>>>>> For example: Suppose you have one Delivery Service:
> >>>>>>>>>> XML_ID: foo
> >>>>>>>>>> Type: HTPT_LIVE_NATL
> >>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass
> up
> >>>>>>>>>> Origin Server Base URL: http://foo.example.net
> >>>>>>>>>>
> >>>>>>>>>> And another Delivery Service:
> >>>>>>>>>> XML_ID: bar
> >>>>>>>>>> Type: HTPT_LIVE_NATL
> >>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass
> up
> >>>>>>>>>> Origin Server Base URL: http://foo.example.net
> >>>>>>>>>>
> >>>>>>>>>> ATS only supports unique `dest_domain` entries in
> >>>>> parent.config.
> >>>>>>>> Therefore,
> >>>>>>>>>> the parent.config generated for a server assigned to both
> >>>> of
> >>>>>>> these
> >>>>>>>> Delivery
> >>>>>>>>>> Services with either be:
> >>>>>>>>>>
> >>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=true
> >>>>>>>>>>
> >>>>>>>>>> Or
> >>>>>>>>>>
> >>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=false
> >>>>>>>> qstring=consider
> >>>>>>>>>>
> >>>>>>>>>> Right now, it's arbitrary which Perl Traffic Ops inserts,
> >>>> and
> >>>>>>> Perl
> >>>>>>>> provides
> >>>>>>>>>> no warning or error of any kind (the pending Go
> >>>>> parent.config PR
> >>>>>>>> logs an
> >>>>>>>>>> error).
> >>>>>>>>>>
> >>>>>>>>>> Whichever is arbitrarily inserted, the resulting remaps
> for
> >>>>> the
> >>>>>>>> other
> >>>>>>>>>> delivery service will be wrong. Either `foo` requests will
> >>>>> drop
> >>>>>>>> the query
> >>>>>>>>>> string when they shouldn't, and go to the mid when they
> >>>>>>> shouldn't;
> >>>>>>>> or `bar`
> >>>>>>>>>> requests will use the query string and skip the mid when
> it
> >>>>>>>> shouldn't.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Does that make sense? The only correct solution, is to
> >>>>> somehow
> >>>>>>>> prevent
> >>>>>>>>>> different DSes having the same origin, and tell tenants
> >>>> they
> >>>>> must
> >>>>>>>> use
> >>>>>>>>>> CNAMEs if they need.
> >>>>>>>>>>
> >>>>>>>>>> This isn't a bug in Traffic Control. ATS fundamentally
> >>>>> doesn't
> >>>>>>>> support
> >>>>>>>>>> multiple remap rules with the same parent FQDN with
> >>>> different
> >>>>>>>>>> configurations. Hence, Traffic Control needs to prohibit
> >>>>> that.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
> >>>>>>>> mitchell852@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> brennan,
> >>>>>>>>>>>
> >>>>>>>>>>> can you give an example of what parent.config looks like
> >>>>> when 2
> >>>>>>>> ds's share
> >>>>>>>>>>> an origin and have different a different topology?
> >>>>>>>>>>>
> >>>>>>>>>>> jeremy
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
> >>>>>>>> Brennan_Fieck@comcast.com
> >>>>>>>>>>>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> To be clear, the "Warning" I'm talking about would
> >>>>> happen at
> >>>>>>>> startup, but
> >>>>>>>>>>>> I'd like a UI-only constraint to come with that to
> >>>>> disallow
> >>>>>>>> using the API
> >>>>>>>>>>>> to bind the same origin to multiple Delivery Services
> >>>>> with
> >>>>>>>> varying
> >>>>>>>>>>>> topography requirements. It wouldn't change the
> >>>> existing
> >>>>>>> data,
> >>>>>>>> but
> >>>>>>>>>>> prevent
> >>>>>>>>>>>> users from creating more bad data.
> >>>>>>>>>>>>
> >>>>>>>>>>>> "warning" doesn't really sufficiently describe that, my
> >>>>> bad.
> >>>>>>>>>>>> ________________________________________
> >>>>>>>>>>>> From: Fieck, Brennan <Br...@comcast.com>
> >>>>>>>>>>>> Sent: Tuesday, December 18, 2018 11:24 AM
> >>>>>>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> >>>>>>>> Delivery Services
> >>>>>>>>>>>> produces indeterminate parent.config
> >>>>>>>>>>>>
> >>>>>>>>>>>> Well the cost of fixing this bug is a constraint on the
> >>>>> data.
> >>>>>>>> Unless we
> >>>>>>>>>>>> make it a UI-only constraint - which I'm personally
> >>>>> against -
> >>>>>>>> there must
> >>>>>>>>>>> be
> >>>>>>>>>>>> some point in the future where ATC cannot reasonably be
> >>>>>>>> expected to work
> >>>>>>>>>>>> with data that violates that constraint. The question
> >>>> is
> >>>>> when
> >>>>>>>> that should
> >>>>>>>>>>>> occur, which should likely happen at a minor version
> >>>>> release.
> >>>>>>>> Minor not
> >>>>>>>>>>>> major because it doesn't involve a change in data
> >>>>> structures,
> >>>>>>>> merely
> >>>>>>>>>>>> relationships between them - in my opinion that's a
> >>>> minor
> >>>>>>>> version change
> >>>>>>>>>>>> but that's definitely up for debate. With several
> >>>> release
> >>>>>>>> candidates for
> >>>>>>>>>>>> 3.0.0 that _doesn't_ include this restriction already
> >>>> in
> >>>>> the
> >>>>>>>> wild, I
> >>>>>>>>>>>> wouldn't recommend putting it in there. That means to
> >>>>> fix the
> >>>>>>>> bug as soon
> >>>>>>>>>>>> as possible it should go in 3.1.0 which should be the
> >>>>> target
> >>>>>>>> of "master"
> >>>>>>>>>>>> after the 3.0.0 release is cut from it.
> >>>>>>>>>>>>
> >>>>>>>>>>>> So I'd recommend immediately implementing the
> >>>> constraint
> >>>>> in
> >>>>>>>> master with a
> >>>>>>>>>>>> refusal to upgrade with bad data, and backport a
> >>>> warning
> >>>>>>> about
> >>>>>>>> the future
> >>>>>>>>>>>> behavior into 3.0.0 or as part of a 3.0.1 provided we
> >>>> had
> >>>>>>> more
> >>>>>>>> changes
> >>>>>>>>>>> that
> >>>>>>>>>>>> would warrant a micro version bump.
> >>>>>>>>>>>> ________________________________________
> >>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
> >>>>>>>>>>>> Sent: Tuesday, December 18, 2018 9:34 AM
> >>>>>>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> >>>>>>>> Delivery Services
> >>>>>>>>>>>> produces indeterminate parent.config
> >>>>>>>>>>>>
> >>>>>>>>>>>> -1 Holding an ATC upgrade hostage to data cleanup seems
> >>>>> like
> >>>>>>> a
> >>>>>>>> bad idea.
> >>>>>>>>>>>> The issue isn't great, but it's also not new. We
> >>>> should
> >>>>>>> allow
> >>>>>>>> teams to
> >>>>>>>>>>> fix
> >>>>>>>>>>>> their data at their normal paces if it doesn't create
> >>>>>>>> significant
> >>>>>>>>>>> overhead
> >>>>>>>>>>>> or an inherant blocker for new functionality or
> >>>>> correction of
> >>>>>>>> other major
> >>>>>>>>>>>> problems imho.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Jonathan G
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 12/18/18, 9:28 AM, "Fieck, Brennan" <
> >>>>>>>> Brennan_Fieck@comcast.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Another option is we could detect collisions at
> >>>>> startup
> >>>>>>>> and simply
> >>>>>>>>>>>> refuse to continue with the upgrade until the data is
> >>>>> fixed.
> >>>>>>>> That would
> >>>>>>>>>>>> allow people using the now-unsupported data format to
> >>>>>>> continue
> >>>>>>>> to use
> >>>>>>>>>>> their
> >>>>>>>>>>>> old versions of Traffic Ops without wrecking their
> >>>>> database,
> >>>>>>>> but also
> >>>>>>>>>>>> provide an incentive to clean up the data.
> >>>>>>>>>>>> ________________________________________
> >>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
> >>>>>>>>>>>> Sent: Tuesday, December 18, 2018 5:12 AM
> >>>>>>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to
> >>>>> Multipe
> >>>>>>>> Delivery
> >>>>>>>>>>>> Services produces indeterminate parent.config
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'm generally a fan of constrain your data in your
> >>>>>>>> database, but not
> >>>>>>>>>>>> necessarily exclusively. I see this as a one-way
> >>>>>>>> cleanup/conversion so
> >>>>>>>>>>> it
> >>>>>>>>>>>> doesn't need to be configurable; otherwise you have to
> >>>>> ask
> >>>>>>> the
> >>>>>>>> question
> >>>>>>>>>>>> what happens if someone turns it off. That said,
> >>>>> something
> >>>>>>> in
> >>>>>>>> the UI
> >>>>>>>>>>> layer
> >>>>>>>>>>>> would be nice to prevent spending significant
> >>>> quantities
> >>>>> of
> >>>>>>>> time
> >>>>>>>>>>> building a
> >>>>>>>>>>>> complex DS only to have it fail to post for reasons
> >>>> that
> >>>>>>> could
> >>>>>>>> have been
> >>>>>>>>>>>> known earlier.
> >>>>>>>>>>>>
> >>>>>>>>>>>> The way my brain works in this case:
> >>>>>>>>>>>> If !unique_constraint_exists_query()
> >>>>>>>>>>>> If has_duplicates_query()
> >>>>>>>>>>>> show_warning()
> >>>>>>>>>>>> else
> >>>>>>>>>>>> add_unique_constraint()
> >>>>>>>>>>>>
> >>>>>>>>>>>> to which the API and UI configuration could also
> >>>>> make use
> >>>>>>>> of
> >>>>>>>>>>>> unique_constraint_exists_query() to drive additional
> >>>>> layer
> >>>>>>>> constraints if
> >>>>>>>>>>>> desired.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Jonathan G
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 12/17/18, 1:11 PM, "Rawlin Peters" <
> >>>>>>>> rawlin.peters@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> That is an interesting idea...detect at TO
> >>>>> startup
> >>>>>>>> whether or not
> >>>>>>>>>>>> there are duplicate origins and operate in a
> >>>>> "prevent
> >>>>>>>> duplicate
> >>>>>>>>>>>> origins" state if no duplicates are found or
> >>>>> "prevent
> >>>>>>>> conflicting
> >>>>>>>>>>>> DS
> >>>>>>>>>>>> topologies" state if duplicates are found? So
> >>>>> once
> >>>>>>>> operators have
> >>>>>>>>>>>> replaced all the duplicate origins with CNAMEs,
> >>>>> TO
> >>>>>>> will
> >>>>>>>>>>> essentially
> >>>>>>>>>>>> operate in a "prohibit all duplicate origins"
> >>>>> state.
> >>>>>>>> That would
> >>>>>>>>>>>> probably make for a simpler transition, but I'd
> >>>>> want
> >>>>>>>> to remove
> >>>>>>>>>>> that
> >>>>>>>>>>>> logic in a following release that strictly
> >>>>> prohibits
> >>>>>>>> duplicate
> >>>>>>>>>>>> origins
> >>>>>>>>>>>> (assuming that the community agrees we should
> >>>>>>> prohibit
> >>>>>>>> duplicate
> >>>>>>>>>>>> origins altogether).
> >>>>>>>>>>>>
> >>>>>>>>>>>> As for DB constraints vs UI, I was thinking
> >>>> those
> >>>>>>>> DS-type
> >>>>>>>>>>>> constraints
> >>>>>>>>>>>> I pointed out would live in the API. It would
> >>>>>>>> basically be added
> >>>>>>>>>>>> validation in the deliveryservices POST/PUT
> >>>>> endpoint
> >>>>>>>> that checks
> >>>>>>>>>>>> the
> >>>>>>>>>>>> DB for existing DSes that conflict with the
> >>>>> requested
> >>>>>>>> DS.
> >>>>>>>>>>>>
> >>>>>>>>>>>> - Rawlin
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> >>>>>>>>>>>> <Jo...@comcast.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> These kinds of conditions should be
> >>>> detectable
> >>>>>>> with a
> >>>>>>>>>>>> sufficiently advanced SQL query. Is it possible to add
> >>>>> the
> >>>>>>>> constraint if
> >>>>>>>>>>>> it passes and emit a warning during TO startup
> >>>> otherwise?
> >>>>>>>> That would let
> >>>>>>>>>>>> you know the condition exists at startup but not
> >>>> getting
> >>>>> in
> >>>>>>>> your way and
> >>>>>>>>>>>> keep you out of trouble once you've cleaned up. We
> >>>> made
> >>>>> a
> >>>>>>>> mistake early
> >>>>>>>>>>>> on, but this would acknowledge it was bad and encourage
> >>>>> it to
> >>>>>>>> be fixed at
> >>>>>>>>>>>> the speed of operations teams. Also this puts the
> >>>>> constraint
> >>>>>>>> in the
> >>>>>>>>>>>> database rather than the UI which is really where the
> >>>>>>>> contention is for
> >>>>>>>>>>>> usability.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Jonathan G
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On 12/17/18, 11:38 AM, "Rawlin Peters" <
> >>>>>>>>>>> rawlin.peters@gmail.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> We occasionally discuss this issue but
> >>>>> haven't
> >>>>>>>> tackled it
> >>>>>>>>>>>> yet. I think
> >>>>>>>>>>>>> the main issue is just that duplicate
> >>>>> origins
> >>>>>>>> have been
> >>>>>>>>>>>> allowed since
> >>>>>>>>>>>>> the beginning, and now everyone's Traffic
> >>>>> Ops
> >>>>>>>> could be
> >>>>>>>>>>>> littered with
> >>>>>>>>>>>>> duplicate origins. Also, depending on the
> >>>>>>> config
> >>>>>>>> of the
> >>>>>>>>>>>> duplicate
> >>>>>>>>>>>>> delivery services, the origins might not
> >>>>> be in
> >>>>>>>> conflict at
> >>>>>>>>>>>> all (if
> >>>>>>>>>>>>> they don't have different topology
> >>>>>>> constraints).
> >>>>>>>> I would
> >>>>>>>>>>>> love for us
> >>>>>>>>>>>>> to just add a uniqueness constraint, but
> >>>>> there
> >>>>>>>> would need
> >>>>>>>>>>> to
> >>>>>>>>>>>> be a fair
> >>>>>>>>>>>>> amount of warning to the community before
> >>>>> doing
> >>>>>>>> so and
> >>>>>>>>>>> might
> >>>>>>>>>>>>> invalidate a significant amount of valid
> >>>>> use
> >>>>>>>> cases.
> >>>>>>>>>>>> Operators would
> >>>>>>>>>>>>> need time to make DNS CNAME records for
> >>>> the
> >>>>>>>> duplicate
> >>>>>>>>>>>> origins and
> >>>>>>>>>>>>> update their DSes to use the different
> >>>>> CNAMEs.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I think as a good first step to
> >>>>> eliminating the
> >>>>>>>> use of
> >>>>>>>>>>>> duplicate
> >>>>>>>>>>>>> origins altogether, we should identify
> >>>>> which
> >>>>>>>> "topology
> >>>>>>>>>>>> constraints"
> >>>>>>>>>>>>> actually cause conflicting config when
> >>>> used
> >>>>>>> with
> >>>>>>>> duplicate
> >>>>>>>>>>>> origins and
> >>>>>>>>>>>>> prevent creating DSes with duplicate
> >>>>> origins
> >>>>>>> _if
> >>>>>>>> it would
> >>>>>>>>>>>> cause a
> >>>>>>>>>>>>> conflict with an existing DS that uses
> >>>> the
> >>>>> same
> >>>>>>>> origin_.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> For instance, I believe an HTTP and
> >>>>> DNS-type DS
> >>>>>>>> can live
> >>>>>>>>>>>> happily
> >>>>>>>>>>>>> side-by-side using the same origin
> >>>>> (probably
> >>>>>>>> need different
> >>>>>>>>>>>>> routing_names?), but scenarios like HTTP
> >>>>> and
> >>>>>>>> HTTP_LIVE, or
> >>>>>>>>>>>> DNS and
> >>>>>>>>>>>>> HTTP_NO_CACHE sharing the same origin
> >>>> will
> >>>>>>> cause
> >>>>>>>> conflicts
> >>>>>>>>>>>> for sure.
> >>>>>>>>>>>>> So maybe we can start by making sure the
> >>>> DS
> >>>>>>>> types "match"
> >>>>>>>>>>>> when using
> >>>>>>>>>>>>> the same origin:
> >>>>>>>>>>>>> HTTP + DNS: possibly good, if they have
> >>>>>>>> different routing
> >>>>>>>>>>>> names?
> >>>>>>>>>>>>> HTTP_LIVE + HTTP_LIVE_NATNL: bad
> >>>>>>>>>>>>> HTTP_NO_CACHE + [any other type]: bad
> >>>>>>>>>>>>> HTTP_LIVE + HTTP: bad
> >>>>>>>>>>>>> etc.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> There are most likely other conflict
> >>>>> scenarios
> >>>>>>>> that don't
> >>>>>>>>>>>> involve the
> >>>>>>>>>>>>> DS types, but I think this would be a
> >>>> good
> >>>>>>>> start. In the
> >>>>>>>>>>>> future with
> >>>>>>>>>>>>> Delivery Service Topologies (aka Flexible
> >>>>>>>> Cachegroups aka
> >>>>>>>>>>>> Bring Your
> >>>>>>>>>>>>> Own Topology), we might be able to
> >>>> prohibit
> >>>>>>>> assigning a DS
> >>>>>>>>>>>> to a
> >>>>>>>>>>>>> Topology if the DS's origin is already
> >>>>> used by
> >>>>>>>> another DS
> >>>>>>>>>>> in
> >>>>>>>>>>>> a
> >>>>>>>>>>>>> different Topology.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> - Rawlin
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Dec 17, 2018 at 10:52 AM Fieck,
> >>>>> Brennan
> >>>>>>>>>>>>> <Br...@comcast.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> As some of you may be aware,
> >>>>> `parent.config`
> >>>>>>>> files
> >>>>>>>>>>>> generated by Traffic Ops can vary wildly when an origin
> >>>>> is
> >>>>>>>> assigned to
> >>>>>>>>>>>> multiple Delivery Services. This results in undefined
> >>>>>>>> behavior. I'm told
> >>>>>>>>>>>> that the conflict only happens when two Delivery
> >>>> Services
> >>>>>>> with
> >>>>>>>> different
> >>>>>>>>>>>> "topology requirements" use the same origin, whatever
> >>>>> that
> >>>>>>>> means (content
> >>>>>>>>>>>> routing type?). Regardless, the issue should be
> >>>>> addressed.
> >>>>>>> The
> >>>>>>>> obvious
> >>>>>>>>>>>> solution is to put in place a database constraint that
> >>>>>>>> prevents an origin
> >>>>>>>>>>>> from being assigned to more that one Delivery Service
> >>>>> with
> >>>>>>> API
> >>>>>>>> checks in
> >>>>>>>>>>>> place that would provide helpful error messages when an
> >>>>>>>> attempt is made
> >>>>>>>>>>> to
> >>>>>>>>>>>> violate the constraint. However, would that mess with
> >>>>> things
> >>>>>>>> like
> >>>>>>>>>>>> Multi-Site Origin? Or is it just not viable for some
> >>>>> other
> >>>>>>>> reason? If it
> >>>>>>>>>>> is
> >>>>>>>>>>>> a good solution, I'm prepared to work on a fix that
> >>>>> utilizes
> >>>>>>>> it.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>>
> >>>
>
>
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Gray, Jonathan" <Jo...@comcast.com>.
Even that has a strong element of confusion when in a self-service multi-tenancy world. Multiple tenants can use the same origins for things like akami, aws, internal load balancers and any communication about the origin already being used will be confusing.
Jonathan G
On 1/23/19, 7:37 PM, "Gelinas, Derek" <De...@comcast.com> wrote:
Perhaps we could solve both issues and just allow it to happen on DS config but throw up a message when it is saved that says "hey we're going to do this but it's really not without an element of risk and ats isn't designed to work properly with multiple identical origins. We recommend you use an alternative fqdn blah blah blah."
Something along those lines, anyway.
Derek
> On Jan 23, 2019, at 7:18 PM, Rawlin Peters <ra...@gmail.com> wrote:
>
> Simply prohibiting duplicate origin FQDNs has been vetoed now, so we
> can't really move forward with that unless the vetoers change their
> minds and remove their -1s.
>
> Currently that leaves us with checking at DS create/update whether or
> not it would conflict with a shared origin due to different DS types,
> qstring config, etc. I am -1 on that idea since I think it will be
> difficult to maintain and enumerate all the possible conflicting
> cases, will present confusing error messages to the user to which the
> solution would just be "create a CNAME record" anyways, and that the
> ATS parent.config should really be per-remap rather than "global". I
> know there has been some discussion in the ATS community about making
> parent.config per-remap, but I don't have much information there.
>
> That is to say, I don't think there is a great solution to this
> problem right now that we can all agree upon, but maybe our efforts
> would be better spent in the ATS community by making a per-remap
> parent.config. Then the duplicate origin problem in Traffic Control
> would go away.
>
> Another option could be to have some kind of generic rules engine that
> would allow an organization to create some set of rules that DSes
> would have to abide by. For example, no duplicate origins, an origin
> can't be in a list of known malicious domains, you can't use the DS
> type "HTTP_NO_CACHE", maxDnsAnswers must be < 5, TTLs must be > 30 and
> < 6000, protocol must be HTTPS, intitialDispersion must be < 10, DSCP
> must be 40, etc. It seems like for something truly self-service you'd
> probably need to set up some rules for your users to abide by that
> might not necessarily apply to ALL organizations. Just a thought. I
> don't think the duplicate origins problem alone would warrant
> something like that, but if it also provided a solution for the
> self-service effort, maybe it would be worth it.
>
> - Rawlin
>
> On Wed, Jan 23, 2019 at 4:15 AM Gelinas, Derek
> <De...@comcast.com> wrote:
>>
>> That's the problem. We either need to be smart enough to recognize that in our config and warn the user or just prevent it and suggest to the user that they use a different fqdn.
>>
>> My vote is for preventing it at this point.
>>
>>> On Jan 23, 2019, at 12:50 AM, Gray, Jonathan <Jo...@comcast.com> wrote:
>>>
>>> It's not just differing types, it's anything that affects the parent.config which also includes the, qstring configuration, MSO parents/config, etc. as well.
>>>
>>> Jonathan G
>>>
>>>
>>> On 1/22/19, 2:33 PM, "Dave Neuman" <ne...@apache.org> wrote:
>>>
>>> -1 on the one DS to Origin limitation. I think there are legit use cases
>>> for it.
>>> I would be +1 on the idea if we include type though. Two DSs can share the
>>> same origin as long as they are of the same type (DNS, HTTP, HTTP_LIVE),
>>> etc. You shouldn't be able to have two DSs with the same origin but
>>> different types.
>>>
>>> On Mon, Jan 14, 2019 at 7:53 AM Fieck, Brennan <Br...@comcast.com>
>>> wrote:
>>>
>>>> I don't see this as a complicated use case or a barrier to entry, an
>>>> extremely basic introduction to ATC would only make use of one DS
>>>> (shameless CDN-in-a-Box plug).
>>>> It seems far more complicated for a user to be using many DSes to serve a
>>>> single origin - we can easily make it clear in the docs that the two have a
>>>> 1:1 relationship, with a footnote about CNAME records for users with the
>>>> knowledge, resources, and need for such workarounds.
>>>> ________________________________________
>>>> From: Mark Torluemke <mt...@apache.org>
>>>> Sent: Friday, January 11, 2019 1:21 PM
>>>> To: dev@trafficcontrol.apache.org
>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
>>>> produces indeterminate parent.config
>>>>
>>>>>> 1. Prohibit creating new delivery services that would share an
>>>> existing origin and prohibit updating a delivery service to a shared
>>>> origin
>>>> In case my position has been lost, I'm still -1 on this. :) IMO, we
>>>> shouldn't let the few complicated use cases increase the barrier to entry
>>>> (to using ATC) for the masses.
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Jan 11, 2019 at 11:53 AM Rawlin Peters <ra...@gmail.com>
>>>> wrote:
>>>>
>>>>> Alright, I'm trying to sum up this discussion so far since it seems
>>>>> like everyone went on vacation and didn't really get a chance to wrap
>>>>> this one up:
>>>>> - duplicate origins cause undefined behavior
>>>>> - we need a way to migrate to a future that is free of duplicate
>>>>> origins in Traffic Control
>>>>> - we need a visual and easy way to determine if Traffic Ops currently
>>>>> contains duplicate origins, so that operators are incentivized to fix
>>>>> them rather than let it slide indefinitely
>>>>> - operators should have a fair amount of time to fix their duplicate
>>>>> origins
>>>>>
>>>>> I believe this is what we've mostly agreed upon but haven't clearly voted
>>>>> on:
>>>>>
>>>>> In release N we will:
>>>>> 1. Prohibit creating new delivery services that would share an
>>>>> existing origin and prohibit updating a delivery service to a shared
>>>>> origin
>>>>> 2. Add some kind of visual indicator that duplicate origins are a
>>>>> problem that need to be fixed before release N+1; otherwise, an
>>>>> upgrade to N+1 will be prohibited.
>>>>>
>>>>> In release N+1 we will:
>>>>> 3. Include a DB migration that adds a uniqueness constraint on origin
>>>>> FQDN, removing the API-level checks for that.
>>>>> 4. Prevent an upgrade to N+1 if duplicate origins are found (this
>>>>> might occur as a byproduct of step 3).
>>>>>
>>>>> I am +1 on this plan and believe this would hit on all the summarized
>>>>> points above. Please provide a clear vote on this plan so that we can
>>>>> dive deeper in the details (i.e. what release 'N' is, the best visual
>>>>> indicator for step 2, and a friendly way to handle step 4).
>>>>>
>>>>> Thanks,
>>>>> Rawlin
>>>>>
>>>>> On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <mi...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Not sure TP is the right place for a warning like "clean up this
>>>>>> 'duplicate' origin or your next upgrade will fail". Most users of our
>>>>>> system will be like "not my problem".
>>>>>>
>>>>>> On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <
>>>>> Brennan_Fieck@comcast.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Probably. It would impact load times by a bit, but the page for an
>>>>>>> individual object is not our bottleneck.
>>>>>>> ________________________________________
>>>>>>> From: Robert Butts <ro...@apache.org>
>>>>>>> Sent: Wednesday, December 19, 2018 11:50 AM
>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>>>>> Services
>>>>>>> produces indeterminate parent.config
>>>>>>>
>>>>>>>> - Including a warning on startup and an API constraint preventing
>>>>> adding
>>>>>>> more bad data in the next 3.0.0 Release Candidate
>>>>>>>> - Adding a database constraint immediately into master that won't
>>>> be
>>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
>>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>> I understand Jonathan's objection, but at some point, we have to be
>>>>> able to
>>>>>>> move forward. This is a good compromise: deprecate, then remove. That
>>>>> gives
>>>>>>> people a full major version to fix their data.
>>>>>>>
>>>>>>> I would be ideal if it were more than just a logged warning, though.
>>>>> Can we
>>>>>>> add a big red banner in Traffic Portal, on the Delivery Service page
>>>>> for
>>>>>>> any DS with a duplicate origin, telling users to fix it, and that
>>>> they
>>>>>>> won't be able to upgrade to the next major version until it's fixed?
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <
>>>>> Brennan_Fieck@comcast.com
>>>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> So it seems like nobody has a problem with the "how" - disallowing
>>>>>>>> duplicate origin FQDNs on Delivery Services - but we never reached
>>>> a
>>>>>>>> consensus on "when".
>>>>>>>>
>>>>>>>> I stand by my previous proposal:
>>>>>>>> - Including a warning on startup and an API constraint preventing
>>>>> adding
>>>>>>>> more bad data in the next 3.0.0 Release Candidate
>>>>>>>> - Adding a database constraint immediately into master that won't
>>>> be
>>>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
>>>>>>>> ________________________________________
>>>>>>>> From: Rawlin Peters <ra...@gmail.com>
>>>>>>>> Sent: Tuesday, December 18, 2018 4:59 PM
>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>>>>> Services
>>>>>>>> produces indeterminate parent.config
>>>>>>>>
>>>>>>>> Also, building more around DS types will make it even harder to get
>>>>>>>> away from DS types in the future too, which I know is something
>>>> we've
>>>>>>>> discussed on this mailing list before. It also adds to the overhead
>>>>> of
>>>>>>>> Delivery Service Topologies, since a lot of the DS types won't
>>>>>>>> carryover into that world.
>>>>>>>>
>>>>>>>> - Rawlin
>>>>>>>>
>>>>>>>> On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
>>>>>>>> <Br...@comcast.com> wrote:
>>>>>>>>>
>>>>>>>>> +1.
>>>>>>>>> If there's a simple way to work around duplicate origins being
>>>>>>>> prohibited,
>>>>>>>>> then we should rely on that instead of "enumerating all those
>>>>> possible
>>>>>>>> conflicting
>>>>>>>>> settings, which are not only highly complex and confusing, but
>>>> also
>>>>>>>> further
>>>>>>>>> entrench us in only supporting ATS as a caching proxy (hurting
>>>>> efforts
>>>>>>> to
>>>>>>>>> integrate e.g. Grove, nginx etc.)
>>>>>>>>> ________________________________________
>>>>>>>>> From: Rawlin Peters <ra...@gmail.com>
>>>>>>>>> Sent: Tuesday, December 18, 2018 2:20 PM
>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>
>>>>>>>>> There are a number of different DS settings at play that can
>>>>>>>>> potentially cause conflicts. The question is: do we want to get
>>>>> into
>>>>>>>>> the business of enumerating all those possible conflicting
>>>>> settings or
>>>>>>>>> just simply prohibit duplicate origins altogether? I think we can
>>>>> dig
>>>>>>>>> in and get that "sufficiently advanced sql query" to check for
>>>>>>>>> conflicting origins, but is that something we want to carry along
>>>>> for
>>>>>>>>> the foreseeable future? Aren't CNAMEs relatively cheaper than
>>>>>>>>> developing and maintaining that code and the mental overhead
>>>>> required
>>>>>>>>> in understanding why you're getting an error that says your
>>>>> requested
>>>>>>>>> DS would cause an origin conflict? I think at the point you've
>>>>>>>>> requested a DS that would create a conflict, you've chosen those
>>>>>>>>> settings for a reason and would probably prefer to just
>>>> create/use
>>>>> a
>>>>>>>>> CNAME in your new DS and keep the rest of your settings the same.
>>>>>>>>>
>>>>>>>>> Thinking in terms of errors, I'm imagining:
>>>>>>>>> "cannot create delivery service: origin fqdn 'foo.example.com'
>>>>> already
>>>>>>>> in use"
>>>>>>>>> vs
>>>>>>>>> "cannot create delivery service: origin fqdn 'foo.example.com'
>>>>> already
>>>>>>>>> in use as type DNS_LIVE_NATNL, which is incompatible with your
>>>>> chosen
>>>>>>>>> type of HTTP_NO_CACHE"
>>>>>>>>>
>>>>>>>>> At that point you'd probably say to yourself, "uh, I need
>>>>>>>>> HTTP_NO_CACHE, so what am I supposed to do now?"
>>>>>>>>>
>>>>>>>>> As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
>>>>>>>>> because the resulting code will be simpler, but I think
>>>> eliminating
>>>>>>>>> the mental overhead for operators could be worthwhile too. If we
>>>>> can
>>>>>>>>> agree on an end state of prohibiting duplicate origins
>>>> altogether,
>>>>> we
>>>>>>>>> can start working on a design to smoothly transition us to that
>>>>> point.
>>>>>>>>> Are we willing to live with "just CNAME your origin fqdn" as the
>>>>>>>>> standard solution to duplicates?
>>>>>>>>>
>>>>>>>>> - Rawlin
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
>>>>>>>>> <De...@comcast.com> wrote:
>>>>>>>>>>
>>>>>>>>>> The only situation in which they can share origins is if a) the
>>>>>>>> origins are shared in an MSO configuration but still have different
>>>>>>> defined
>>>>>>>> origin fields in the delivery service, or if they're assigned to
>>>>>>> completely
>>>>>>>> different cachegroups. It's when two delivery services share the
>>>>> same
>>>>>>>> edges that there's an issue, because you end up with parent.config
>>>>>>> issues.
>>>>>>>> Actually you could even get away with it in mids as long as you
>>>>> weren't
>>>>>>>> doing anything like MSO to it.
>>>>>>>>>>
>>>>>>>>>> Could get messy real fast, though. Best to just create a
>>>> second
>>>>>>> FQDN.
>>>>>>>>>>
>>>>>>>>>> Derek
>>>>>>>>>>
>>>>>>>>>> On 12/18/18, 3:23 PM, "Fieck, Brennan" <
>>>>> Brennan_Fieck@comcast.com>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> So no two Delivery Services may share an origin *regardless
>>>>> of
>>>>>>>> cache hierarchy* ? I've been told that DNS Delivery Services can
>>>>> have the
>>>>>>>> same origin as HTTP Delivery Services because they obey the same
>>>>> cache
>>>>>>>> hierarchy. You're saying that would still produce invalid output
>>>>> and/or
>>>>>>> is
>>>>>>>> explicitly disallowed by ATS?
>>>>>>>>>> ________________________________________
>>>>>>>>>> From: Robert Butts <ro...@apache.org>
>>>>>>>>>> Sent: Tuesday, December 18, 2018 1:09 PM
>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>> Delivery
>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>>
>>>>>>>>>>> can you give an example of what parent.config looks like
>>>>> when 2
>>>>>>>> ds's share
>>>>>>>>>> an origin and have different a different topology?
>>>>>>>>>>
>>>>>>>>>> Answering because I encountered this directly, when
>>>> rewriting
>>>>>>>> parent.config.
>>>>>>>>>>
>>>>>>>>>> For example: Suppose you have one Delivery Service:
>>>>>>>>>> XML_ID: foo
>>>>>>>>>> Type: HTPT_LIVE_NATL
>>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass up
>>>>>>>>>> Origin Server Base URL: http://foo.example.net
>>>>>>>>>>
>>>>>>>>>> And another Delivery Service:
>>>>>>>>>> XML_ID: bar
>>>>>>>>>> Type: HTPT_LIVE_NATL
>>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass up
>>>>>>>>>> Origin Server Base URL: http://foo.example.net
>>>>>>>>>>
>>>>>>>>>> ATS only supports unique `dest_domain` entries in
>>>>> parent.config.
>>>>>>>> Therefore,
>>>>>>>>>> the parent.config generated for a server assigned to both
>>>> of
>>>>>>> these
>>>>>>>> Delivery
>>>>>>>>>> Services with either be:
>>>>>>>>>>
>>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=true
>>>>>>>>>>
>>>>>>>>>> Or
>>>>>>>>>>
>>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=false
>>>>>>>> qstring=consider
>>>>>>>>>>
>>>>>>>>>> Right now, it's arbitrary which Perl Traffic Ops inserts,
>>>> and
>>>>>>> Perl
>>>>>>>> provides
>>>>>>>>>> no warning or error of any kind (the pending Go
>>>>> parent.config PR
>>>>>>>> logs an
>>>>>>>>>> error).
>>>>>>>>>>
>>>>>>>>>> Whichever is arbitrarily inserted, the resulting remaps for
>>>>> the
>>>>>>>> other
>>>>>>>>>> delivery service will be wrong. Either `foo` requests will
>>>>> drop
>>>>>>>> the query
>>>>>>>>>> string when they shouldn't, and go to the mid when they
>>>>>>> shouldn't;
>>>>>>>> or `bar`
>>>>>>>>>> requests will use the query string and skip the mid when it
>>>>>>>> shouldn't.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Does that make sense? The only correct solution, is to
>>>>> somehow
>>>>>>>> prevent
>>>>>>>>>> different DSes having the same origin, and tell tenants
>>>> they
>>>>> must
>>>>>>>> use
>>>>>>>>>> CNAMEs if they need.
>>>>>>>>>>
>>>>>>>>>> This isn't a bug in Traffic Control. ATS fundamentally
>>>>> doesn't
>>>>>>>> support
>>>>>>>>>> multiple remap rules with the same parent FQDN with
>>>> different
>>>>>>>>>> configurations. Hence, Traffic Control needs to prohibit
>>>>> that.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
>>>>>>>> mitchell852@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> brennan,
>>>>>>>>>>>
>>>>>>>>>>> can you give an example of what parent.config looks like
>>>>> when 2
>>>>>>>> ds's share
>>>>>>>>>>> an origin and have different a different topology?
>>>>>>>>>>>
>>>>>>>>>>> jeremy
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
>>>>>>>> Brennan_Fieck@comcast.com
>>>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> To be clear, the "Warning" I'm talking about would
>>>>> happen at
>>>>>>>> startup, but
>>>>>>>>>>>> I'd like a UI-only constraint to come with that to
>>>>> disallow
>>>>>>>> using the API
>>>>>>>>>>>> to bind the same origin to multiple Delivery Services
>>>>> with
>>>>>>>> varying
>>>>>>>>>>>> topography requirements. It wouldn't change the
>>>> existing
>>>>>>> data,
>>>>>>>> but
>>>>>>>>>>> prevent
>>>>>>>>>>>> users from creating more bad data.
>>>>>>>>>>>>
>>>>>>>>>>>> "warning" doesn't really sufficiently describe that, my
>>>>> bad.
>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>> From: Fieck, Brennan <Br...@comcast.com>
>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 11:24 AM
>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>>>>> Delivery Services
>>>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>>>
>>>>>>>>>>>> Well the cost of fixing this bug is a constraint on the
>>>>> data.
>>>>>>>> Unless we
>>>>>>>>>>>> make it a UI-only constraint - which I'm personally
>>>>> against -
>>>>>>>> there must
>>>>>>>>>>> be
>>>>>>>>>>>> some point in the future where ATC cannot reasonably be
>>>>>>>> expected to work
>>>>>>>>>>>> with data that violates that constraint. The question
>>>> is
>>>>> when
>>>>>>>> that should
>>>>>>>>>>>> occur, which should likely happen at a minor version
>>>>> release.
>>>>>>>> Minor not
>>>>>>>>>>>> major because it doesn't involve a change in data
>>>>> structures,
>>>>>>>> merely
>>>>>>>>>>>> relationships between them - in my opinion that's a
>>>> minor
>>>>>>>> version change
>>>>>>>>>>>> but that's definitely up for debate. With several
>>>> release
>>>>>>>> candidates for
>>>>>>>>>>>> 3.0.0 that _doesn't_ include this restriction already
>>>> in
>>>>> the
>>>>>>>> wild, I
>>>>>>>>>>>> wouldn't recommend putting it in there. That means to
>>>>> fix the
>>>>>>>> bug as soon
>>>>>>>>>>>> as possible it should go in 3.1.0 which should be the
>>>>> target
>>>>>>>> of "master"
>>>>>>>>>>>> after the 3.0.0 release is cut from it.
>>>>>>>>>>>>
>>>>>>>>>>>> So I'd recommend immediately implementing the
>>>> constraint
>>>>> in
>>>>>>>> master with a
>>>>>>>>>>>> refusal to upgrade with bad data, and backport a
>>>> warning
>>>>>>> about
>>>>>>>> the future
>>>>>>>>>>>> behavior into 3.0.0 or as part of a 3.0.1 provided we
>>>> had
>>>>>>> more
>>>>>>>> changes
>>>>>>>>>>> that
>>>>>>>>>>>> would warrant a micro version bump.
>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 9:34 AM
>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>>>>> Delivery Services
>>>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>>>
>>>>>>>>>>>> -1 Holding an ATC upgrade hostage to data cleanup seems
>>>>> like
>>>>>>> a
>>>>>>>> bad idea.
>>>>>>>>>>>> The issue isn't great, but it's also not new. We
>>>> should
>>>>>>> allow
>>>>>>>> teams to
>>>>>>>>>>> fix
>>>>>>>>>>>> their data at their normal paces if it doesn't create
>>>>>>>> significant
>>>>>>>>>>> overhead
>>>>>>>>>>>> or an inherant blocker for new functionality or
>>>>> correction of
>>>>>>>> other major
>>>>>>>>>>>> problems imho.
>>>>>>>>>>>>
>>>>>>>>>>>> Jonathan G
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 12/18/18, 9:28 AM, "Fieck, Brennan" <
>>>>>>>> Brennan_Fieck@comcast.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Another option is we could detect collisions at
>>>>> startup
>>>>>>>> and simply
>>>>>>>>>>>> refuse to continue with the upgrade until the data is
>>>>> fixed.
>>>>>>>> That would
>>>>>>>>>>>> allow people using the now-unsupported data format to
>>>>>>> continue
>>>>>>>> to use
>>>>>>>>>>> their
>>>>>>>>>>>> old versions of Traffic Ops without wrecking their
>>>>> database,
>>>>>>>> but also
>>>>>>>>>>>> provide an incentive to clean up the data.
>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 5:12 AM
>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to
>>>>> Multipe
>>>>>>>> Delivery
>>>>>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>>>>
>>>>>>>>>>>> I'm generally a fan of constrain your data in your
>>>>>>>> database, but not
>>>>>>>>>>>> necessarily exclusively. I see this as a one-way
>>>>>>>> cleanup/conversion so
>>>>>>>>>>> it
>>>>>>>>>>>> doesn't need to be configurable; otherwise you have to
>>>>> ask
>>>>>>> the
>>>>>>>> question
>>>>>>>>>>>> what happens if someone turns it off. That said,
>>>>> something
>>>>>>> in
>>>>>>>> the UI
>>>>>>>>>>> layer
>>>>>>>>>>>> would be nice to prevent spending significant
>>>> quantities
>>>>> of
>>>>>>>> time
>>>>>>>>>>> building a
>>>>>>>>>>>> complex DS only to have it fail to post for reasons
>>>> that
>>>>>>> could
>>>>>>>> have been
>>>>>>>>>>>> known earlier.
>>>>>>>>>>>>
>>>>>>>>>>>> The way my brain works in this case:
>>>>>>>>>>>> If !unique_constraint_exists_query()
>>>>>>>>>>>> If has_duplicates_query()
>>>>>>>>>>>> show_warning()
>>>>>>>>>>>> else
>>>>>>>>>>>> add_unique_constraint()
>>>>>>>>>>>>
>>>>>>>>>>>> to which the API and UI configuration could also
>>>>> make use
>>>>>>>> of
>>>>>>>>>>>> unique_constraint_exists_query() to drive additional
>>>>> layer
>>>>>>>> constraints if
>>>>>>>>>>>> desired.
>>>>>>>>>>>>
>>>>>>>>>>>> Jonathan G
>>>>>>>>>>>>
>>>>>>>>>>>> On 12/17/18, 1:11 PM, "Rawlin Peters" <
>>>>>>>> rawlin.peters@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> That is an interesting idea...detect at TO
>>>>> startup
>>>>>>>> whether or not
>>>>>>>>>>>> there are duplicate origins and operate in a
>>>>> "prevent
>>>>>>>> duplicate
>>>>>>>>>>>> origins" state if no duplicates are found or
>>>>> "prevent
>>>>>>>> conflicting
>>>>>>>>>>>> DS
>>>>>>>>>>>> topologies" state if duplicates are found? So
>>>>> once
>>>>>>>> operators have
>>>>>>>>>>>> replaced all the duplicate origins with CNAMEs,
>>>>> TO
>>>>>>> will
>>>>>>>>>>> essentially
>>>>>>>>>>>> operate in a "prohibit all duplicate origins"
>>>>> state.
>>>>>>>> That would
>>>>>>>>>>>> probably make for a simpler transition, but I'd
>>>>> want
>>>>>>>> to remove
>>>>>>>>>>> that
>>>>>>>>>>>> logic in a following release that strictly
>>>>> prohibits
>>>>>>>> duplicate
>>>>>>>>>>>> origins
>>>>>>>>>>>> (assuming that the community agrees we should
>>>>>>> prohibit
>>>>>>>> duplicate
>>>>>>>>>>>> origins altogether).
>>>>>>>>>>>>
>>>>>>>>>>>> As for DB constraints vs UI, I was thinking
>>>> those
>>>>>>>> DS-type
>>>>>>>>>>>> constraints
>>>>>>>>>>>> I pointed out would live in the API. It would
>>>>>>>> basically be added
>>>>>>>>>>>> validation in the deliveryservices POST/PUT
>>>>> endpoint
>>>>>>>> that checks
>>>>>>>>>>>> the
>>>>>>>>>>>> DB for existing DSes that conflict with the
>>>>> requested
>>>>>>>> DS.
>>>>>>>>>>>>
>>>>>>>>>>>> - Rawlin
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
>>>>>>>>>>>> <Jo...@comcast.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> These kinds of conditions should be
>>>> detectable
>>>>>>> with a
>>>>>>>>>>>> sufficiently advanced SQL query. Is it possible to add
>>>>> the
>>>>>>>> constraint if
>>>>>>>>>>>> it passes and emit a warning during TO startup
>>>> otherwise?
>>>>>>>> That would let
>>>>>>>>>>>> you know the condition exists at startup but not
>>>> getting
>>>>> in
>>>>>>>> your way and
>>>>>>>>>>>> keep you out of trouble once you've cleaned up. We
>>>> made
>>>>> a
>>>>>>>> mistake early
>>>>>>>>>>>> on, but this would acknowledge it was bad and encourage
>>>>> it to
>>>>>>>> be fixed at
>>>>>>>>>>>> the speed of operations teams. Also this puts the
>>>>> constraint
>>>>>>>> in the
>>>>>>>>>>>> database rather than the UI which is really where the
>>>>>>>> contention is for
>>>>>>>>>>>> usability.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Jonathan G
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 12/17/18, 11:38 AM, "Rawlin Peters" <
>>>>>>>>>>> rawlin.peters@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> We occasionally discuss this issue but
>>>>> haven't
>>>>>>>> tackled it
>>>>>>>>>>>> yet. I think
>>>>>>>>>>>>> the main issue is just that duplicate
>>>>> origins
>>>>>>>> have been
>>>>>>>>>>>> allowed since
>>>>>>>>>>>>> the beginning, and now everyone's Traffic
>>>>> Ops
>>>>>>>> could be
>>>>>>>>>>>> littered with
>>>>>>>>>>>>> duplicate origins. Also, depending on the
>>>>>>> config
>>>>>>>> of the
>>>>>>>>>>>> duplicate
>>>>>>>>>>>>> delivery services, the origins might not
>>>>> be in
>>>>>>>> conflict at
>>>>>>>>>>>> all (if
>>>>>>>>>>>>> they don't have different topology
>>>>>>> constraints).
>>>>>>>> I would
>>>>>>>>>>>> love for us
>>>>>>>>>>>>> to just add a uniqueness constraint, but
>>>>> there
>>>>>>>> would need
>>>>>>>>>>> to
>>>>>>>>>>>> be a fair
>>>>>>>>>>>>> amount of warning to the community before
>>>>> doing
>>>>>>>> so and
>>>>>>>>>>> might
>>>>>>>>>>>>> invalidate a significant amount of valid
>>>>> use
>>>>>>>> cases.
>>>>>>>>>>>> Operators would
>>>>>>>>>>>>> need time to make DNS CNAME records for
>>>> the
>>>>>>>> duplicate
>>>>>>>>>>>> origins and
>>>>>>>>>>>>> update their DSes to use the different
>>>>> CNAMEs.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think as a good first step to
>>>>> eliminating the
>>>>>>>> use of
>>>>>>>>>>>> duplicate
>>>>>>>>>>>>> origins altogether, we should identify
>>>>> which
>>>>>>>> "topology
>>>>>>>>>>>> constraints"
>>>>>>>>>>>>> actually cause conflicting config when
>>>> used
>>>>>>> with
>>>>>>>> duplicate
>>>>>>>>>>>> origins and
>>>>>>>>>>>>> prevent creating DSes with duplicate
>>>>> origins
>>>>>>> _if
>>>>>>>> it would
>>>>>>>>>>>> cause a
>>>>>>>>>>>>> conflict with an existing DS that uses
>>>> the
>>>>> same
>>>>>>>> origin_.
>>>>>>>>>>>>>
>>>>>>>>>>>>> For instance, I believe an HTTP and
>>>>> DNS-type DS
>>>>>>>> can live
>>>>>>>>>>>> happily
>>>>>>>>>>>>> side-by-side using the same origin
>>>>> (probably
>>>>>>>> need different
>>>>>>>>>>>>> routing_names?), but scenarios like HTTP
>>>>> and
>>>>>>>> HTTP_LIVE, or
>>>>>>>>>>>> DNS and
>>>>>>>>>>>>> HTTP_NO_CACHE sharing the same origin
>>>> will
>>>>>>> cause
>>>>>>>> conflicts
>>>>>>>>>>>> for sure.
>>>>>>>>>>>>> So maybe we can start by making sure the
>>>> DS
>>>>>>>> types "match"
>>>>>>>>>>>> when using
>>>>>>>>>>>>> the same origin:
>>>>>>>>>>>>> HTTP + DNS: possibly good, if they have
>>>>>>>> different routing
>>>>>>>>>>>> names?
>>>>>>>>>>>>> HTTP_LIVE + HTTP_LIVE_NATNL: bad
>>>>>>>>>>>>> HTTP_NO_CACHE + [any other type]: bad
>>>>>>>>>>>>> HTTP_LIVE + HTTP: bad
>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>
>>>>>>>>>>>>> There are most likely other conflict
>>>>> scenarios
>>>>>>>> that don't
>>>>>>>>>>>> involve the
>>>>>>>>>>>>> DS types, but I think this would be a
>>>> good
>>>>>>>> start. In the
>>>>>>>>>>>> future with
>>>>>>>>>>>>> Delivery Service Topologies (aka Flexible
>>>>>>>> Cachegroups aka
>>>>>>>>>>>> Bring Your
>>>>>>>>>>>>> Own Topology), we might be able to
>>>> prohibit
>>>>>>>> assigning a DS
>>>>>>>>>>>> to a
>>>>>>>>>>>>> Topology if the DS's origin is already
>>>>> used by
>>>>>>>> another DS
>>>>>>>>>>> in
>>>>>>>>>>>> a
>>>>>>>>>>>>> different Topology.
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Rawlin
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Dec 17, 2018 at 10:52 AM Fieck,
>>>>> Brennan
>>>>>>>>>>>>> <Br...@comcast.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As some of you may be aware,
>>>>> `parent.config`
>>>>>>>> files
>>>>>>>>>>>> generated by Traffic Ops can vary wildly when an origin
>>>>> is
>>>>>>>> assigned to
>>>>>>>>>>>> multiple Delivery Services. This results in undefined
>>>>>>>> behavior. I'm told
>>>>>>>>>>>> that the conflict only happens when two Delivery
>>>> Services
>>>>>>> with
>>>>>>>> different
>>>>>>>>>>>> "topology requirements" use the same origin, whatever
>>>>> that
>>>>>>>> means (content
>>>>>>>>>>>> routing type?). Regardless, the issue should be
>>>>> addressed.
>>>>>>> The
>>>>>>>> obvious
>>>>>>>>>>>> solution is to put in place a database constraint that
>>>>>>>> prevents an origin
>>>>>>>>>>>> from being assigned to more that one Delivery Service
>>>>> with
>>>>>>> API
>>>>>>>> checks in
>>>>>>>>>>>> place that would provide helpful error messages when an
>>>>>>>> attempt is made
>>>>>>>>>>> to
>>>>>>>>>>>> violate the constraint. However, would that mess with
>>>>> things
>>>>>>>> like
>>>>>>>>>>>> Multi-Site Origin? Or is it just not viable for some
>>>>> other
>>>>>>>> reason? If it
>>>>>>>>>>> is
>>>>>>>>>>>> a good solution, I'm prepared to work on a fix that
>>>>> utilizes
>>>>>>>> it.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Fieck, Brennan" <Br...@comcast.com>.
I just want to come out in strong support of disabling the ability to assign the same
origin FQDN to multiple Delivery Services one last time. The convoluted corner cases
where it's incidentally safe to do so are only needed by huge corporations that would
have the resources and expertise available to create CNAME records. For clarity's sake
it would probably better to use a CNAME even when it isn't strictly necessary. Actually, I can't
even tell you what a use case for that would be, I just assume that they exist for
possibly QAM-related reasons. Allowing that enables undefined behavior in the
system that we can only check for via a very complicated series of checks that are very
closely tied with how our database is currently structured and rely on ATS being the
only supported HTTP caching proxy. I feel like we're making it easy to do the wrong thing,
intentionally leaving our system difficult to test via automation, and creating more work
for ourselves later.
________________________________________
From: Gelinas, Derek <De...@comcast.com>
Sent: Wednesday, January 23, 2019 7:36 PM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
Perhaps we could solve both issues and just allow it to happen on DS config but throw up a message when it is saved that says "hey we're going to do this but it's really not without an element of risk and ats isn't designed to work properly with multiple identical origins. We recommend you use an alternative fqdn blah blah blah."
Something along those lines, anyway.
Derek
> On Jan 23, 2019, at 7:18 PM, Rawlin Peters <ra...@gmail.com> wrote:
>
> Simply prohibiting duplicate origin FQDNs has been vetoed now, so we
> can't really move forward with that unless the vetoers change their
> minds and remove their -1s.
>
> Currently that leaves us with checking at DS create/update whether or
> not it would conflict with a shared origin due to different DS types,
> qstring config, etc. I am -1 on that idea since I think it will be
> difficult to maintain and enumerate all the possible conflicting
> cases, will present confusing error messages to the user to which the
> solution would just be "create a CNAME record" anyways, and that the
> ATS parent.config should really be per-remap rather than "global". I
> know there has been some discussion in the ATS community about making
> parent.config per-remap, but I don't have much information there.
>
> That is to say, I don't think there is a great solution to this
> problem right now that we can all agree upon, but maybe our efforts
> would be better spent in the ATS community by making a per-remap
> parent.config. Then the duplicate origin problem in Traffic Control
> would go away.
>
> Another option could be to have some kind of generic rules engine that
> would allow an organization to create some set of rules that DSes
> would have to abide by. For example, no duplicate origins, an origin
> can't be in a list of known malicious domains, you can't use the DS
> type "HTTP_NO_CACHE", maxDnsAnswers must be < 5, TTLs must be > 30 and
> < 6000, protocol must be HTTPS, intitialDispersion must be < 10, DSCP
> must be 40, etc. It seems like for something truly self-service you'd
> probably need to set up some rules for your users to abide by that
> might not necessarily apply to ALL organizations. Just a thought. I
> don't think the duplicate origins problem alone would warrant
> something like that, but if it also provided a solution for the
> self-service effort, maybe it would be worth it.
>
> - Rawlin
>
> On Wed, Jan 23, 2019 at 4:15 AM Gelinas, Derek
> <De...@comcast.com> wrote:
>>
>> That's the problem. We either need to be smart enough to recognize that in our config and warn the user or just prevent it and suggest to the user that they use a different fqdn.
>>
>> My vote is for preventing it at this point.
>>
>>> On Jan 23, 2019, at 12:50 AM, Gray, Jonathan <Jo...@comcast.com> wrote:
>>>
>>> It's not just differing types, it's anything that affects the parent.config which also includes the, qstring configuration, MSO parents/config, etc. as well.
>>>
>>> Jonathan G
>>>
>>>
>>> On 1/22/19, 2:33 PM, "Dave Neuman" <ne...@apache.org> wrote:
>>>
>>> -1 on the one DS to Origin limitation. I think there are legit use cases
>>> for it.
>>> I would be +1 on the idea if we include type though. Two DSs can share the
>>> same origin as long as they are of the same type (DNS, HTTP, HTTP_LIVE),
>>> etc. You shouldn't be able to have two DSs with the same origin but
>>> different types.
>>>
>>> On Mon, Jan 14, 2019 at 7:53 AM Fieck, Brennan <Br...@comcast.com>
>>> wrote:
>>>
>>>> I don't see this as a complicated use case or a barrier to entry, an
>>>> extremely basic introduction to ATC would only make use of one DS
>>>> (shameless CDN-in-a-Box plug).
>>>> It seems far more complicated for a user to be using many DSes to serve a
>>>> single origin - we can easily make it clear in the docs that the two have a
>>>> 1:1 relationship, with a footnote about CNAME records for users with the
>>>> knowledge, resources, and need for such workarounds.
>>>> ________________________________________
>>>> From: Mark Torluemke <mt...@apache.org>
>>>> Sent: Friday, January 11, 2019 1:21 PM
>>>> To: dev@trafficcontrol.apache.org
>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
>>>> produces indeterminate parent.config
>>>>
>>>>>> 1. Prohibit creating new delivery services that would share an
>>>> existing origin and prohibit updating a delivery service to a shared
>>>> origin
>>>> In case my position has been lost, I'm still -1 on this. :) IMO, we
>>>> shouldn't let the few complicated use cases increase the barrier to entry
>>>> (to using ATC) for the masses.
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Jan 11, 2019 at 11:53 AM Rawlin Peters <ra...@gmail.com>
>>>> wrote:
>>>>
>>>>> Alright, I'm trying to sum up this discussion so far since it seems
>>>>> like everyone went on vacation and didn't really get a chance to wrap
>>>>> this one up:
>>>>> - duplicate origins cause undefined behavior
>>>>> - we need a way to migrate to a future that is free of duplicate
>>>>> origins in Traffic Control
>>>>> - we need a visual and easy way to determine if Traffic Ops currently
>>>>> contains duplicate origins, so that operators are incentivized to fix
>>>>> them rather than let it slide indefinitely
>>>>> - operators should have a fair amount of time to fix their duplicate
>>>>> origins
>>>>>
>>>>> I believe this is what we've mostly agreed upon but haven't clearly voted
>>>>> on:
>>>>>
>>>>> In release N we will:
>>>>> 1. Prohibit creating new delivery services that would share an
>>>>> existing origin and prohibit updating a delivery service to a shared
>>>>> origin
>>>>> 2. Add some kind of visual indicator that duplicate origins are a
>>>>> problem that need to be fixed before release N+1; otherwise, an
>>>>> upgrade to N+1 will be prohibited.
>>>>>
>>>>> In release N+1 we will:
>>>>> 3. Include a DB migration that adds a uniqueness constraint on origin
>>>>> FQDN, removing the API-level checks for that.
>>>>> 4. Prevent an upgrade to N+1 if duplicate origins are found (this
>>>>> might occur as a byproduct of step 3).
>>>>>
>>>>> I am +1 on this plan and believe this would hit on all the summarized
>>>>> points above. Please provide a clear vote on this plan so that we can
>>>>> dive deeper in the details (i.e. what release 'N' is, the best visual
>>>>> indicator for step 2, and a friendly way to handle step 4).
>>>>>
>>>>> Thanks,
>>>>> Rawlin
>>>>>
>>>>> On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <mi...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Not sure TP is the right place for a warning like "clean up this
>>>>>> 'duplicate' origin or your next upgrade will fail". Most users of our
>>>>>> system will be like "not my problem".
>>>>>>
>>>>>> On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <
>>>>> Brennan_Fieck@comcast.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Probably. It would impact load times by a bit, but the page for an
>>>>>>> individual object is not our bottleneck.
>>>>>>> ________________________________________
>>>>>>> From: Robert Butts <ro...@apache.org>
>>>>>>> Sent: Wednesday, December 19, 2018 11:50 AM
>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>>>>> Services
>>>>>>> produces indeterminate parent.config
>>>>>>>
>>>>>>>> - Including a warning on startup and an API constraint preventing
>>>>> adding
>>>>>>> more bad data in the next 3.0.0 Release Candidate
>>>>>>>> - Adding a database constraint immediately into master that won't
>>>> be
>>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
>>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>> I understand Jonathan's objection, but at some point, we have to be
>>>>> able to
>>>>>>> move forward. This is a good compromise: deprecate, then remove. That
>>>>> gives
>>>>>>> people a full major version to fix their data.
>>>>>>>
>>>>>>> I would be ideal if it were more than just a logged warning, though.
>>>>> Can we
>>>>>>> add a big red banner in Traffic Portal, on the Delivery Service page
>>>>> for
>>>>>>> any DS with a duplicate origin, telling users to fix it, and that
>>>> they
>>>>>>> won't be able to upgrade to the next major version until it's fixed?
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <
>>>>> Brennan_Fieck@comcast.com
>>>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> So it seems like nobody has a problem with the "how" - disallowing
>>>>>>>> duplicate origin FQDNs on Delivery Services - but we never reached
>>>> a
>>>>>>>> consensus on "when".
>>>>>>>>
>>>>>>>> I stand by my previous proposal:
>>>>>>>> - Including a warning on startup and an API constraint preventing
>>>>> adding
>>>>>>>> more bad data in the next 3.0.0 Release Candidate
>>>>>>>> - Adding a database constraint immediately into master that won't
>>>> be
>>>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
>>>>>>>> ________________________________________
>>>>>>>> From: Rawlin Peters <ra...@gmail.com>
>>>>>>>> Sent: Tuesday, December 18, 2018 4:59 PM
>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>>>>> Services
>>>>>>>> produces indeterminate parent.config
>>>>>>>>
>>>>>>>> Also, building more around DS types will make it even harder to get
>>>>>>>> away from DS types in the future too, which I know is something
>>>> we've
>>>>>>>> discussed on this mailing list before. It also adds to the overhead
>>>>> of
>>>>>>>> Delivery Service Topologies, since a lot of the DS types won't
>>>>>>>> carryover into that world.
>>>>>>>>
>>>>>>>> - Rawlin
>>>>>>>>
>>>>>>>> On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
>>>>>>>> <Br...@comcast.com> wrote:
>>>>>>>>>
>>>>>>>>> +1.
>>>>>>>>> If there's a simple way to work around duplicate origins being
>>>>>>>> prohibited,
>>>>>>>>> then we should rely on that instead of "enumerating all those
>>>>> possible
>>>>>>>> conflicting
>>>>>>>>> settings, which are not only highly complex and confusing, but
>>>> also
>>>>>>>> further
>>>>>>>>> entrench us in only supporting ATS as a caching proxy (hurting
>>>>> efforts
>>>>>>> to
>>>>>>>>> integrate e.g. Grove, nginx etc.)
>>>>>>>>> ________________________________________
>>>>>>>>> From: Rawlin Peters <ra...@gmail.com>
>>>>>>>>> Sent: Tuesday, December 18, 2018 2:20 PM
>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>
>>>>>>>>> There are a number of different DS settings at play that can
>>>>>>>>> potentially cause conflicts. The question is: do we want to get
>>>>> into
>>>>>>>>> the business of enumerating all those possible conflicting
>>>>> settings or
>>>>>>>>> just simply prohibit duplicate origins altogether? I think we can
>>>>> dig
>>>>>>>>> in and get that "sufficiently advanced sql query" to check for
>>>>>>>>> conflicting origins, but is that something we want to carry along
>>>>> for
>>>>>>>>> the foreseeable future? Aren't CNAMEs relatively cheaper than
>>>>>>>>> developing and maintaining that code and the mental overhead
>>>>> required
>>>>>>>>> in understanding why you're getting an error that says your
>>>>> requested
>>>>>>>>> DS would cause an origin conflict? I think at the point you've
>>>>>>>>> requested a DS that would create a conflict, you've chosen those
>>>>>>>>> settings for a reason and would probably prefer to just
>>>> create/use
>>>>> a
>>>>>>>>> CNAME in your new DS and keep the rest of your settings the same.
>>>>>>>>>
>>>>>>>>> Thinking in terms of errors, I'm imagining:
>>>>>>>>> "cannot create delivery service: origin fqdn 'foo.example.com'
>>>>> already
>>>>>>>> in use"
>>>>>>>>> vs
>>>>>>>>> "cannot create delivery service: origin fqdn 'foo.example.com'
>>>>> already
>>>>>>>>> in use as type DNS_LIVE_NATNL, which is incompatible with your
>>>>> chosen
>>>>>>>>> type of HTTP_NO_CACHE"
>>>>>>>>>
>>>>>>>>> At that point you'd probably say to yourself, "uh, I need
>>>>>>>>> HTTP_NO_CACHE, so what am I supposed to do now?"
>>>>>>>>>
>>>>>>>>> As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
>>>>>>>>> because the resulting code will be simpler, but I think
>>>> eliminating
>>>>>>>>> the mental overhead for operators could be worthwhile too. If we
>>>>> can
>>>>>>>>> agree on an end state of prohibiting duplicate origins
>>>> altogether,
>>>>> we
>>>>>>>>> can start working on a design to smoothly transition us to that
>>>>> point.
>>>>>>>>> Are we willing to live with "just CNAME your origin fqdn" as the
>>>>>>>>> standard solution to duplicates?
>>>>>>>>>
>>>>>>>>> - Rawlin
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
>>>>>>>>> <De...@comcast.com> wrote:
>>>>>>>>>>
>>>>>>>>>> The only situation in which they can share origins is if a) the
>>>>>>>> origins are shared in an MSO configuration but still have different
>>>>>>> defined
>>>>>>>> origin fields in the delivery service, or if they're assigned to
>>>>>>> completely
>>>>>>>> different cachegroups. It's when two delivery services share the
>>>>> same
>>>>>>>> edges that there's an issue, because you end up with parent.config
>>>>>>> issues.
>>>>>>>> Actually you could even get away with it in mids as long as you
>>>>> weren't
>>>>>>>> doing anything like MSO to it.
>>>>>>>>>>
>>>>>>>>>> Could get messy real fast, though. Best to just create a
>>>> second
>>>>>>> FQDN.
>>>>>>>>>>
>>>>>>>>>> Derek
>>>>>>>>>>
>>>>>>>>>> On 12/18/18, 3:23 PM, "Fieck, Brennan" <
>>>>> Brennan_Fieck@comcast.com>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> So no two Delivery Services may share an origin *regardless
>>>>> of
>>>>>>>> cache hierarchy* ? I've been told that DNS Delivery Services can
>>>>> have the
>>>>>>>> same origin as HTTP Delivery Services because they obey the same
>>>>> cache
>>>>>>>> hierarchy. You're saying that would still produce invalid output
>>>>> and/or
>>>>>>> is
>>>>>>>> explicitly disallowed by ATS?
>>>>>>>>>> ________________________________________
>>>>>>>>>> From: Robert Butts <ro...@apache.org>
>>>>>>>>>> Sent: Tuesday, December 18, 2018 1:09 PM
>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>> Delivery
>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>>
>>>>>>>>>>> can you give an example of what parent.config looks like
>>>>> when 2
>>>>>>>> ds's share
>>>>>>>>>> an origin and have different a different topology?
>>>>>>>>>>
>>>>>>>>>> Answering because I encountered this directly, when
>>>> rewriting
>>>>>>>> parent.config.
>>>>>>>>>>
>>>>>>>>>> For example: Suppose you have one Delivery Service:
>>>>>>>>>> XML_ID: foo
>>>>>>>>>> Type: HTPT_LIVE_NATL
>>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass up
>>>>>>>>>> Origin Server Base URL: http://foo.example.net
>>>>>>>>>>
>>>>>>>>>> And another Delivery Service:
>>>>>>>>>> XML_ID: bar
>>>>>>>>>> Type: HTPT_LIVE_NATL
>>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass up
>>>>>>>>>> Origin Server Base URL: http://foo.example.net
>>>>>>>>>>
>>>>>>>>>> ATS only supports unique `dest_domain` entries in
>>>>> parent.config.
>>>>>>>> Therefore,
>>>>>>>>>> the parent.config generated for a server assigned to both
>>>> of
>>>>>>> these
>>>>>>>> Delivery
>>>>>>>>>> Services with either be:
>>>>>>>>>>
>>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=true
>>>>>>>>>>
>>>>>>>>>> Or
>>>>>>>>>>
>>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=false
>>>>>>>> qstring=consider
>>>>>>>>>>
>>>>>>>>>> Right now, it's arbitrary which Perl Traffic Ops inserts,
>>>> and
>>>>>>> Perl
>>>>>>>> provides
>>>>>>>>>> no warning or error of any kind (the pending Go
>>>>> parent.config PR
>>>>>>>> logs an
>>>>>>>>>> error).
>>>>>>>>>>
>>>>>>>>>> Whichever is arbitrarily inserted, the resulting remaps for
>>>>> the
>>>>>>>> other
>>>>>>>>>> delivery service will be wrong. Either `foo` requests will
>>>>> drop
>>>>>>>> the query
>>>>>>>>>> string when they shouldn't, and go to the mid when they
>>>>>>> shouldn't;
>>>>>>>> or `bar`
>>>>>>>>>> requests will use the query string and skip the mid when it
>>>>>>>> shouldn't.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Does that make sense? The only correct solution, is to
>>>>> somehow
>>>>>>>> prevent
>>>>>>>>>> different DSes having the same origin, and tell tenants
>>>> they
>>>>> must
>>>>>>>> use
>>>>>>>>>> CNAMEs if they need.
>>>>>>>>>>
>>>>>>>>>> This isn't a bug in Traffic Control. ATS fundamentally
>>>>> doesn't
>>>>>>>> support
>>>>>>>>>> multiple remap rules with the same parent FQDN with
>>>> different
>>>>>>>>>> configurations. Hence, Traffic Control needs to prohibit
>>>>> that.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
>>>>>>>> mitchell852@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> brennan,
>>>>>>>>>>>
>>>>>>>>>>> can you give an example of what parent.config looks like
>>>>> when 2
>>>>>>>> ds's share
>>>>>>>>>>> an origin and have different a different topology?
>>>>>>>>>>>
>>>>>>>>>>> jeremy
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
>>>>>>>> Brennan_Fieck@comcast.com
>>>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> To be clear, the "Warning" I'm talking about would
>>>>> happen at
>>>>>>>> startup, but
>>>>>>>>>>>> I'd like a UI-only constraint to come with that to
>>>>> disallow
>>>>>>>> using the API
>>>>>>>>>>>> to bind the same origin to multiple Delivery Services
>>>>> with
>>>>>>>> varying
>>>>>>>>>>>> topography requirements. It wouldn't change the
>>>> existing
>>>>>>> data,
>>>>>>>> but
>>>>>>>>>>> prevent
>>>>>>>>>>>> users from creating more bad data.
>>>>>>>>>>>>
>>>>>>>>>>>> "warning" doesn't really sufficiently describe that, my
>>>>> bad.
>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>> From: Fieck, Brennan <Br...@comcast.com>
>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 11:24 AM
>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>>>>> Delivery Services
>>>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>>>
>>>>>>>>>>>> Well the cost of fixing this bug is a constraint on the
>>>>> data.
>>>>>>>> Unless we
>>>>>>>>>>>> make it a UI-only constraint - which I'm personally
>>>>> against -
>>>>>>>> there must
>>>>>>>>>>> be
>>>>>>>>>>>> some point in the future where ATC cannot reasonably be
>>>>>>>> expected to work
>>>>>>>>>>>> with data that violates that constraint. The question
>>>> is
>>>>> when
>>>>>>>> that should
>>>>>>>>>>>> occur, which should likely happen at a minor version
>>>>> release.
>>>>>>>> Minor not
>>>>>>>>>>>> major because it doesn't involve a change in data
>>>>> structures,
>>>>>>>> merely
>>>>>>>>>>>> relationships between them - in my opinion that's a
>>>> minor
>>>>>>>> version change
>>>>>>>>>>>> but that's definitely up for debate. With several
>>>> release
>>>>>>>> candidates for
>>>>>>>>>>>> 3.0.0 that _doesn't_ include this restriction already
>>>> in
>>>>> the
>>>>>>>> wild, I
>>>>>>>>>>>> wouldn't recommend putting it in there. That means to
>>>>> fix the
>>>>>>>> bug as soon
>>>>>>>>>>>> as possible it should go in 3.1.0 which should be the
>>>>> target
>>>>>>>> of "master"
>>>>>>>>>>>> after the 3.0.0 release is cut from it.
>>>>>>>>>>>>
>>>>>>>>>>>> So I'd recommend immediately implementing the
>>>> constraint
>>>>> in
>>>>>>>> master with a
>>>>>>>>>>>> refusal to upgrade with bad data, and backport a
>>>> warning
>>>>>>> about
>>>>>>>> the future
>>>>>>>>>>>> behavior into 3.0.0 or as part of a 3.0.1 provided we
>>>> had
>>>>>>> more
>>>>>>>> changes
>>>>>>>>>>> that
>>>>>>>>>>>> would warrant a micro version bump.
>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 9:34 AM
>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>>>>> Delivery Services
>>>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>>>
>>>>>>>>>>>> -1 Holding an ATC upgrade hostage to data cleanup seems
>>>>> like
>>>>>>> a
>>>>>>>> bad idea.
>>>>>>>>>>>> The issue isn't great, but it's also not new. We
>>>> should
>>>>>>> allow
>>>>>>>> teams to
>>>>>>>>>>> fix
>>>>>>>>>>>> their data at their normal paces if it doesn't create
>>>>>>>> significant
>>>>>>>>>>> overhead
>>>>>>>>>>>> or an inherant blocker for new functionality or
>>>>> correction of
>>>>>>>> other major
>>>>>>>>>>>> problems imho.
>>>>>>>>>>>>
>>>>>>>>>>>> Jonathan G
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 12/18/18, 9:28 AM, "Fieck, Brennan" <
>>>>>>>> Brennan_Fieck@comcast.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Another option is we could detect collisions at
>>>>> startup
>>>>>>>> and simply
>>>>>>>>>>>> refuse to continue with the upgrade until the data is
>>>>> fixed.
>>>>>>>> That would
>>>>>>>>>>>> allow people using the now-unsupported data format to
>>>>>>> continue
>>>>>>>> to use
>>>>>>>>>>> their
>>>>>>>>>>>> old versions of Traffic Ops without wrecking their
>>>>> database,
>>>>>>>> but also
>>>>>>>>>>>> provide an incentive to clean up the data.
>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 5:12 AM
>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to
>>>>> Multipe
>>>>>>>> Delivery
>>>>>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>>>>
>>>>>>>>>>>> I'm generally a fan of constrain your data in your
>>>>>>>> database, but not
>>>>>>>>>>>> necessarily exclusively. I see this as a one-way
>>>>>>>> cleanup/conversion so
>>>>>>>>>>> it
>>>>>>>>>>>> doesn't need to be configurable; otherwise you have to
>>>>> ask
>>>>>>> the
>>>>>>>> question
>>>>>>>>>>>> what happens if someone turns it off. That said,
>>>>> something
>>>>>>> in
>>>>>>>> the UI
>>>>>>>>>>> layer
>>>>>>>>>>>> would be nice to prevent spending significant
>>>> quantities
>>>>> of
>>>>>>>> time
>>>>>>>>>>> building a
>>>>>>>>>>>> complex DS only to have it fail to post for reasons
>>>> that
>>>>>>> could
>>>>>>>> have been
>>>>>>>>>>>> known earlier.
>>>>>>>>>>>>
>>>>>>>>>>>> The way my brain works in this case:
>>>>>>>>>>>> If !unique_constraint_exists_query()
>>>>>>>>>>>> If has_duplicates_query()
>>>>>>>>>>>> show_warning()
>>>>>>>>>>>> else
>>>>>>>>>>>> add_unique_constraint()
>>>>>>>>>>>>
>>>>>>>>>>>> to which the API and UI configuration could also
>>>>> make use
>>>>>>>> of
>>>>>>>>>>>> unique_constraint_exists_query() to drive additional
>>>>> layer
>>>>>>>> constraints if
>>>>>>>>>>>> desired.
>>>>>>>>>>>>
>>>>>>>>>>>> Jonathan G
>>>>>>>>>>>>
>>>>>>>>>>>> On 12/17/18, 1:11 PM, "Rawlin Peters" <
>>>>>>>> rawlin.peters@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> That is an interesting idea...detect at TO
>>>>> startup
>>>>>>>> whether or not
>>>>>>>>>>>> there are duplicate origins and operate in a
>>>>> "prevent
>>>>>>>> duplicate
>>>>>>>>>>>> origins" state if no duplicates are found or
>>>>> "prevent
>>>>>>>> conflicting
>>>>>>>>>>>> DS
>>>>>>>>>>>> topologies" state if duplicates are found? So
>>>>> once
>>>>>>>> operators have
>>>>>>>>>>>> replaced all the duplicate origins with CNAMEs,
>>>>> TO
>>>>>>> will
>>>>>>>>>>> essentially
>>>>>>>>>>>> operate in a "prohibit all duplicate origins"
>>>>> state.
>>>>>>>> That would
>>>>>>>>>>>> probably make for a simpler transition, but I'd
>>>>> want
>>>>>>>> to remove
>>>>>>>>>>> that
>>>>>>>>>>>> logic in a following release that strictly
>>>>> prohibits
>>>>>>>> duplicate
>>>>>>>>>>>> origins
>>>>>>>>>>>> (assuming that the community agrees we should
>>>>>>> prohibit
>>>>>>>> duplicate
>>>>>>>>>>>> origins altogether).
>>>>>>>>>>>>
>>>>>>>>>>>> As for DB constraints vs UI, I was thinking
>>>> those
>>>>>>>> DS-type
>>>>>>>>>>>> constraints
>>>>>>>>>>>> I pointed out would live in the API. It would
>>>>>>>> basically be added
>>>>>>>>>>>> validation in the deliveryservices POST/PUT
>>>>> endpoint
>>>>>>>> that checks
>>>>>>>>>>>> the
>>>>>>>>>>>> DB for existing DSes that conflict with the
>>>>> requested
>>>>>>>> DS.
>>>>>>>>>>>>
>>>>>>>>>>>> - Rawlin
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
>>>>>>>>>>>> <Jo...@comcast.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> These kinds of conditions should be
>>>> detectable
>>>>>>> with a
>>>>>>>>>>>> sufficiently advanced SQL query. Is it possible to add
>>>>> the
>>>>>>>> constraint if
>>>>>>>>>>>> it passes and emit a warning during TO startup
>>>> otherwise?
>>>>>>>> That would let
>>>>>>>>>>>> you know the condition exists at startup but not
>>>> getting
>>>>> in
>>>>>>>> your way and
>>>>>>>>>>>> keep you out of trouble once you've cleaned up. We
>>>> made
>>>>> a
>>>>>>>> mistake early
>>>>>>>>>>>> on, but this would acknowledge it was bad and encourage
>>>>> it to
>>>>>>>> be fixed at
>>>>>>>>>>>> the speed of operations teams. Also this puts the
>>>>> constraint
>>>>>>>> in the
>>>>>>>>>>>> database rather than the UI which is really where the
>>>>>>>> contention is for
>>>>>>>>>>>> usability.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Jonathan G
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 12/17/18, 11:38 AM, "Rawlin Peters" <
>>>>>>>>>>> rawlin.peters@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> We occasionally discuss this issue but
>>>>> haven't
>>>>>>>> tackled it
>>>>>>>>>>>> yet. I think
>>>>>>>>>>>>> the main issue is just that duplicate
>>>>> origins
>>>>>>>> have been
>>>>>>>>>>>> allowed since
>>>>>>>>>>>>> the beginning, and now everyone's Traffic
>>>>> Ops
>>>>>>>> could be
>>>>>>>>>>>> littered with
>>>>>>>>>>>>> duplicate origins. Also, depending on the
>>>>>>> config
>>>>>>>> of the
>>>>>>>>>>>> duplicate
>>>>>>>>>>>>> delivery services, the origins might not
>>>>> be in
>>>>>>>> conflict at
>>>>>>>>>>>> all (if
>>>>>>>>>>>>> they don't have different topology
>>>>>>> constraints).
>>>>>>>> I would
>>>>>>>>>>>> love for us
>>>>>>>>>>>>> to just add a uniqueness constraint, but
>>>>> there
>>>>>>>> would need
>>>>>>>>>>> to
>>>>>>>>>>>> be a fair
>>>>>>>>>>>>> amount of warning to the community before
>>>>> doing
>>>>>>>> so and
>>>>>>>>>>> might
>>>>>>>>>>>>> invalidate a significant amount of valid
>>>>> use
>>>>>>>> cases.
>>>>>>>>>>>> Operators would
>>>>>>>>>>>>> need time to make DNS CNAME records for
>>>> the
>>>>>>>> duplicate
>>>>>>>>>>>> origins and
>>>>>>>>>>>>> update their DSes to use the different
>>>>> CNAMEs.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think as a good first step to
>>>>> eliminating the
>>>>>>>> use of
>>>>>>>>>>>> duplicate
>>>>>>>>>>>>> origins altogether, we should identify
>>>>> which
>>>>>>>> "topology
>>>>>>>>>>>> constraints"
>>>>>>>>>>>>> actually cause conflicting config when
>>>> used
>>>>>>> with
>>>>>>>> duplicate
>>>>>>>>>>>> origins and
>>>>>>>>>>>>> prevent creating DSes with duplicate
>>>>> origins
>>>>>>> _if
>>>>>>>> it would
>>>>>>>>>>>> cause a
>>>>>>>>>>>>> conflict with an existing DS that uses
>>>> the
>>>>> same
>>>>>>>> origin_.
>>>>>>>>>>>>>
>>>>>>>>>>>>> For instance, I believe an HTTP and
>>>>> DNS-type DS
>>>>>>>> can live
>>>>>>>>>>>> happily
>>>>>>>>>>>>> side-by-side using the same origin
>>>>> (probably
>>>>>>>> need different
>>>>>>>>>>>>> routing_names?), but scenarios like HTTP
>>>>> and
>>>>>>>> HTTP_LIVE, or
>>>>>>>>>>>> DNS and
>>>>>>>>>>>>> HTTP_NO_CACHE sharing the same origin
>>>> will
>>>>>>> cause
>>>>>>>> conflicts
>>>>>>>>>>>> for sure.
>>>>>>>>>>>>> So maybe we can start by making sure the
>>>> DS
>>>>>>>> types "match"
>>>>>>>>>>>> when using
>>>>>>>>>>>>> the same origin:
>>>>>>>>>>>>> HTTP + DNS: possibly good, if they have
>>>>>>>> different routing
>>>>>>>>>>>> names?
>>>>>>>>>>>>> HTTP_LIVE + HTTP_LIVE_NATNL: bad
>>>>>>>>>>>>> HTTP_NO_CACHE + [any other type]: bad
>>>>>>>>>>>>> HTTP_LIVE + HTTP: bad
>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>
>>>>>>>>>>>>> There are most likely other conflict
>>>>> scenarios
>>>>>>>> that don't
>>>>>>>>>>>> involve the
>>>>>>>>>>>>> DS types, but I think this would be a
>>>> good
>>>>>>>> start. In the
>>>>>>>>>>>> future with
>>>>>>>>>>>>> Delivery Service Topologies (aka Flexible
>>>>>>>> Cachegroups aka
>>>>>>>>>>>> Bring Your
>>>>>>>>>>>>> Own Topology), we might be able to
>>>> prohibit
>>>>>>>> assigning a DS
>>>>>>>>>>>> to a
>>>>>>>>>>>>> Topology if the DS's origin is already
>>>>> used by
>>>>>>>> another DS
>>>>>>>>>>> in
>>>>>>>>>>>> a
>>>>>>>>>>>>> different Topology.
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Rawlin
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Dec 17, 2018 at 10:52 AM Fieck,
>>>>> Brennan
>>>>>>>>>>>>> <Br...@comcast.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As some of you may be aware,
>>>>> `parent.config`
>>>>>>>> files
>>>>>>>>>>>> generated by Traffic Ops can vary wildly when an origin
>>>>> is
>>>>>>>> assigned to
>>>>>>>>>>>> multiple Delivery Services. This results in undefined
>>>>>>>> behavior. I'm told
>>>>>>>>>>>> that the conflict only happens when two Delivery
>>>> Services
>>>>>>> with
>>>>>>>> different
>>>>>>>>>>>> "topology requirements" use the same origin, whatever
>>>>> that
>>>>>>>> means (content
>>>>>>>>>>>> routing type?). Regardless, the issue should be
>>>>> addressed.
>>>>>>> The
>>>>>>>> obvious
>>>>>>>>>>>> solution is to put in place a database constraint that
>>>>>>>> prevents an origin
>>>>>>>>>>>> from being assigned to more that one Delivery Service
>>>>> with
>>>>>>> API
>>>>>>>> checks in
>>>>>>>>>>>> place that would provide helpful error messages when an
>>>>>>>> attempt is made
>>>>>>>>>>> to
>>>>>>>>>>>> violate the constraint. However, would that mess with
>>>>> things
>>>>>>>> like
>>>>>>>>>>>> Multi-Site Origin? Or is it just not viable for some
>>>>> other
>>>>>>>> reason? If it
>>>>>>>>>>> is
>>>>>>>>>>>> a good solution, I'm prepared to work on a fix that
>>>>> utilizes
>>>>>>>> it.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Gelinas, Derek" <De...@comcast.com>.
Perhaps we could solve both issues and just allow it to happen on DS config but throw up a message when it is saved that says "hey we're going to do this but it's really not without an element of risk and ats isn't designed to work properly with multiple identical origins. We recommend you use an alternative fqdn blah blah blah."
Something along those lines, anyway.
Derek
> On Jan 23, 2019, at 7:18 PM, Rawlin Peters <ra...@gmail.com> wrote:
>
> Simply prohibiting duplicate origin FQDNs has been vetoed now, so we
> can't really move forward with that unless the vetoers change their
> minds and remove their -1s.
>
> Currently that leaves us with checking at DS create/update whether or
> not it would conflict with a shared origin due to different DS types,
> qstring config, etc. I am -1 on that idea since I think it will be
> difficult to maintain and enumerate all the possible conflicting
> cases, will present confusing error messages to the user to which the
> solution would just be "create a CNAME record" anyways, and that the
> ATS parent.config should really be per-remap rather than "global". I
> know there has been some discussion in the ATS community about making
> parent.config per-remap, but I don't have much information there.
>
> That is to say, I don't think there is a great solution to this
> problem right now that we can all agree upon, but maybe our efforts
> would be better spent in the ATS community by making a per-remap
> parent.config. Then the duplicate origin problem in Traffic Control
> would go away.
>
> Another option could be to have some kind of generic rules engine that
> would allow an organization to create some set of rules that DSes
> would have to abide by. For example, no duplicate origins, an origin
> can't be in a list of known malicious domains, you can't use the DS
> type "HTTP_NO_CACHE", maxDnsAnswers must be < 5, TTLs must be > 30 and
> < 6000, protocol must be HTTPS, intitialDispersion must be < 10, DSCP
> must be 40, etc. It seems like for something truly self-service you'd
> probably need to set up some rules for your users to abide by that
> might not necessarily apply to ALL organizations. Just a thought. I
> don't think the duplicate origins problem alone would warrant
> something like that, but if it also provided a solution for the
> self-service effort, maybe it would be worth it.
>
> - Rawlin
>
> On Wed, Jan 23, 2019 at 4:15 AM Gelinas, Derek
> <De...@comcast.com> wrote:
>>
>> That's the problem. We either need to be smart enough to recognize that in our config and warn the user or just prevent it and suggest to the user that they use a different fqdn.
>>
>> My vote is for preventing it at this point.
>>
>>> On Jan 23, 2019, at 12:50 AM, Gray, Jonathan <Jo...@comcast.com> wrote:
>>>
>>> It's not just differing types, it's anything that affects the parent.config which also includes the, qstring configuration, MSO parents/config, etc. as well.
>>>
>>> Jonathan G
>>>
>>>
>>> On 1/22/19, 2:33 PM, "Dave Neuman" <ne...@apache.org> wrote:
>>>
>>> -1 on the one DS to Origin limitation. I think there are legit use cases
>>> for it.
>>> I would be +1 on the idea if we include type though. Two DSs can share the
>>> same origin as long as they are of the same type (DNS, HTTP, HTTP_LIVE),
>>> etc. You shouldn't be able to have two DSs with the same origin but
>>> different types.
>>>
>>> On Mon, Jan 14, 2019 at 7:53 AM Fieck, Brennan <Br...@comcast.com>
>>> wrote:
>>>
>>>> I don't see this as a complicated use case or a barrier to entry, an
>>>> extremely basic introduction to ATC would only make use of one DS
>>>> (shameless CDN-in-a-Box plug).
>>>> It seems far more complicated for a user to be using many DSes to serve a
>>>> single origin - we can easily make it clear in the docs that the two have a
>>>> 1:1 relationship, with a footnote about CNAME records for users with the
>>>> knowledge, resources, and need for such workarounds.
>>>> ________________________________________
>>>> From: Mark Torluemke <mt...@apache.org>
>>>> Sent: Friday, January 11, 2019 1:21 PM
>>>> To: dev@trafficcontrol.apache.org
>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
>>>> produces indeterminate parent.config
>>>>
>>>>>> 1. Prohibit creating new delivery services that would share an
>>>> existing origin and prohibit updating a delivery service to a shared
>>>> origin
>>>> In case my position has been lost, I'm still -1 on this. :) IMO, we
>>>> shouldn't let the few complicated use cases increase the barrier to entry
>>>> (to using ATC) for the masses.
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Jan 11, 2019 at 11:53 AM Rawlin Peters <ra...@gmail.com>
>>>> wrote:
>>>>
>>>>> Alright, I'm trying to sum up this discussion so far since it seems
>>>>> like everyone went on vacation and didn't really get a chance to wrap
>>>>> this one up:
>>>>> - duplicate origins cause undefined behavior
>>>>> - we need a way to migrate to a future that is free of duplicate
>>>>> origins in Traffic Control
>>>>> - we need a visual and easy way to determine if Traffic Ops currently
>>>>> contains duplicate origins, so that operators are incentivized to fix
>>>>> them rather than let it slide indefinitely
>>>>> - operators should have a fair amount of time to fix their duplicate
>>>>> origins
>>>>>
>>>>> I believe this is what we've mostly agreed upon but haven't clearly voted
>>>>> on:
>>>>>
>>>>> In release N we will:
>>>>> 1. Prohibit creating new delivery services that would share an
>>>>> existing origin and prohibit updating a delivery service to a shared
>>>>> origin
>>>>> 2. Add some kind of visual indicator that duplicate origins are a
>>>>> problem that need to be fixed before release N+1; otherwise, an
>>>>> upgrade to N+1 will be prohibited.
>>>>>
>>>>> In release N+1 we will:
>>>>> 3. Include a DB migration that adds a uniqueness constraint on origin
>>>>> FQDN, removing the API-level checks for that.
>>>>> 4. Prevent an upgrade to N+1 if duplicate origins are found (this
>>>>> might occur as a byproduct of step 3).
>>>>>
>>>>> I am +1 on this plan and believe this would hit on all the summarized
>>>>> points above. Please provide a clear vote on this plan so that we can
>>>>> dive deeper in the details (i.e. what release 'N' is, the best visual
>>>>> indicator for step 2, and a friendly way to handle step 4).
>>>>>
>>>>> Thanks,
>>>>> Rawlin
>>>>>
>>>>> On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <mi...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Not sure TP is the right place for a warning like "clean up this
>>>>>> 'duplicate' origin or your next upgrade will fail". Most users of our
>>>>>> system will be like "not my problem".
>>>>>>
>>>>>> On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <
>>>>> Brennan_Fieck@comcast.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Probably. It would impact load times by a bit, but the page for an
>>>>>>> individual object is not our bottleneck.
>>>>>>> ________________________________________
>>>>>>> From: Robert Butts <ro...@apache.org>
>>>>>>> Sent: Wednesday, December 19, 2018 11:50 AM
>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>>>>> Services
>>>>>>> produces indeterminate parent.config
>>>>>>>
>>>>>>>> - Including a warning on startup and an API constraint preventing
>>>>> adding
>>>>>>> more bad data in the next 3.0.0 Release Candidate
>>>>>>>> - Adding a database constraint immediately into master that won't
>>>> be
>>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
>>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>> I understand Jonathan's objection, but at some point, we have to be
>>>>> able to
>>>>>>> move forward. This is a good compromise: deprecate, then remove. That
>>>>> gives
>>>>>>> people a full major version to fix their data.
>>>>>>>
>>>>>>> I would be ideal if it were more than just a logged warning, though.
>>>>> Can we
>>>>>>> add a big red banner in Traffic Portal, on the Delivery Service page
>>>>> for
>>>>>>> any DS with a duplicate origin, telling users to fix it, and that
>>>> they
>>>>>>> won't be able to upgrade to the next major version until it's fixed?
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <
>>>>> Brennan_Fieck@comcast.com
>>>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> So it seems like nobody has a problem with the "how" - disallowing
>>>>>>>> duplicate origin FQDNs on Delivery Services - but we never reached
>>>> a
>>>>>>>> consensus on "when".
>>>>>>>>
>>>>>>>> I stand by my previous proposal:
>>>>>>>> - Including a warning on startup and an API constraint preventing
>>>>> adding
>>>>>>>> more bad data in the next 3.0.0 Release Candidate
>>>>>>>> - Adding a database constraint immediately into master that won't
>>>> be
>>>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
>>>>>>>> ________________________________________
>>>>>>>> From: Rawlin Peters <ra...@gmail.com>
>>>>>>>> Sent: Tuesday, December 18, 2018 4:59 PM
>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>>>>> Services
>>>>>>>> produces indeterminate parent.config
>>>>>>>>
>>>>>>>> Also, building more around DS types will make it even harder to get
>>>>>>>> away from DS types in the future too, which I know is something
>>>> we've
>>>>>>>> discussed on this mailing list before. It also adds to the overhead
>>>>> of
>>>>>>>> Delivery Service Topologies, since a lot of the DS types won't
>>>>>>>> carryover into that world.
>>>>>>>>
>>>>>>>> - Rawlin
>>>>>>>>
>>>>>>>> On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
>>>>>>>> <Br...@comcast.com> wrote:
>>>>>>>>>
>>>>>>>>> +1.
>>>>>>>>> If there's a simple way to work around duplicate origins being
>>>>>>>> prohibited,
>>>>>>>>> then we should rely on that instead of "enumerating all those
>>>>> possible
>>>>>>>> conflicting
>>>>>>>>> settings, which are not only highly complex and confusing, but
>>>> also
>>>>>>>> further
>>>>>>>>> entrench us in only supporting ATS as a caching proxy (hurting
>>>>> efforts
>>>>>>> to
>>>>>>>>> integrate e.g. Grove, nginx etc.)
>>>>>>>>> ________________________________________
>>>>>>>>> From: Rawlin Peters <ra...@gmail.com>
>>>>>>>>> Sent: Tuesday, December 18, 2018 2:20 PM
>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>
>>>>>>>>> There are a number of different DS settings at play that can
>>>>>>>>> potentially cause conflicts. The question is: do we want to get
>>>>> into
>>>>>>>>> the business of enumerating all those possible conflicting
>>>>> settings or
>>>>>>>>> just simply prohibit duplicate origins altogether? I think we can
>>>>> dig
>>>>>>>>> in and get that "sufficiently advanced sql query" to check for
>>>>>>>>> conflicting origins, but is that something we want to carry along
>>>>> for
>>>>>>>>> the foreseeable future? Aren't CNAMEs relatively cheaper than
>>>>>>>>> developing and maintaining that code and the mental overhead
>>>>> required
>>>>>>>>> in understanding why you're getting an error that says your
>>>>> requested
>>>>>>>>> DS would cause an origin conflict? I think at the point you've
>>>>>>>>> requested a DS that would create a conflict, you've chosen those
>>>>>>>>> settings for a reason and would probably prefer to just
>>>> create/use
>>>>> a
>>>>>>>>> CNAME in your new DS and keep the rest of your settings the same.
>>>>>>>>>
>>>>>>>>> Thinking in terms of errors, I'm imagining:
>>>>>>>>> "cannot create delivery service: origin fqdn 'foo.example.com'
>>>>> already
>>>>>>>> in use"
>>>>>>>>> vs
>>>>>>>>> "cannot create delivery service: origin fqdn 'foo.example.com'
>>>>> already
>>>>>>>>> in use as type DNS_LIVE_NATNL, which is incompatible with your
>>>>> chosen
>>>>>>>>> type of HTTP_NO_CACHE"
>>>>>>>>>
>>>>>>>>> At that point you'd probably say to yourself, "uh, I need
>>>>>>>>> HTTP_NO_CACHE, so what am I supposed to do now?"
>>>>>>>>>
>>>>>>>>> As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
>>>>>>>>> because the resulting code will be simpler, but I think
>>>> eliminating
>>>>>>>>> the mental overhead for operators could be worthwhile too. If we
>>>>> can
>>>>>>>>> agree on an end state of prohibiting duplicate origins
>>>> altogether,
>>>>> we
>>>>>>>>> can start working on a design to smoothly transition us to that
>>>>> point.
>>>>>>>>> Are we willing to live with "just CNAME your origin fqdn" as the
>>>>>>>>> standard solution to duplicates?
>>>>>>>>>
>>>>>>>>> - Rawlin
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
>>>>>>>>> <De...@comcast.com> wrote:
>>>>>>>>>>
>>>>>>>>>> The only situation in which they can share origins is if a) the
>>>>>>>> origins are shared in an MSO configuration but still have different
>>>>>>> defined
>>>>>>>> origin fields in the delivery service, or if they're assigned to
>>>>>>> completely
>>>>>>>> different cachegroups. It's when two delivery services share the
>>>>> same
>>>>>>>> edges that there's an issue, because you end up with parent.config
>>>>>>> issues.
>>>>>>>> Actually you could even get away with it in mids as long as you
>>>>> weren't
>>>>>>>> doing anything like MSO to it.
>>>>>>>>>>
>>>>>>>>>> Could get messy real fast, though. Best to just create a
>>>> second
>>>>>>> FQDN.
>>>>>>>>>>
>>>>>>>>>> Derek
>>>>>>>>>>
>>>>>>>>>> On 12/18/18, 3:23 PM, "Fieck, Brennan" <
>>>>> Brennan_Fieck@comcast.com>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> So no two Delivery Services may share an origin *regardless
>>>>> of
>>>>>>>> cache hierarchy* ? I've been told that DNS Delivery Services can
>>>>> have the
>>>>>>>> same origin as HTTP Delivery Services because they obey the same
>>>>> cache
>>>>>>>> hierarchy. You're saying that would still produce invalid output
>>>>> and/or
>>>>>>> is
>>>>>>>> explicitly disallowed by ATS?
>>>>>>>>>> ________________________________________
>>>>>>>>>> From: Robert Butts <ro...@apache.org>
>>>>>>>>>> Sent: Tuesday, December 18, 2018 1:09 PM
>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>> Delivery
>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>>
>>>>>>>>>>> can you give an example of what parent.config looks like
>>>>> when 2
>>>>>>>> ds's share
>>>>>>>>>> an origin and have different a different topology?
>>>>>>>>>>
>>>>>>>>>> Answering because I encountered this directly, when
>>>> rewriting
>>>>>>>> parent.config.
>>>>>>>>>>
>>>>>>>>>> For example: Suppose you have one Delivery Service:
>>>>>>>>>> XML_ID: foo
>>>>>>>>>> Type: HTPT_LIVE_NATL
>>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass up
>>>>>>>>>> Origin Server Base URL: http://foo.example.net
>>>>>>>>>>
>>>>>>>>>> And another Delivery Service:
>>>>>>>>>> XML_ID: bar
>>>>>>>>>> Type: HTPT_LIVE_NATL
>>>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass up
>>>>>>>>>> Origin Server Base URL: http://foo.example.net
>>>>>>>>>>
>>>>>>>>>> ATS only supports unique `dest_domain` entries in
>>>>> parent.config.
>>>>>>>> Therefore,
>>>>>>>>>> the parent.config generated for a server assigned to both
>>>> of
>>>>>>> these
>>>>>>>> Delivery
>>>>>>>>>> Services with either be:
>>>>>>>>>>
>>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=true
>>>>>>>>>>
>>>>>>>>>> Or
>>>>>>>>>>
>>>>>>>>>> dest_domain=foo.example.net port=80 go_direct=false
>>>>>>>> qstring=consider
>>>>>>>>>>
>>>>>>>>>> Right now, it's arbitrary which Perl Traffic Ops inserts,
>>>> and
>>>>>>> Perl
>>>>>>>> provides
>>>>>>>>>> no warning or error of any kind (the pending Go
>>>>> parent.config PR
>>>>>>>> logs an
>>>>>>>>>> error).
>>>>>>>>>>
>>>>>>>>>> Whichever is arbitrarily inserted, the resulting remaps for
>>>>> the
>>>>>>>> other
>>>>>>>>>> delivery service will be wrong. Either `foo` requests will
>>>>> drop
>>>>>>>> the query
>>>>>>>>>> string when they shouldn't, and go to the mid when they
>>>>>>> shouldn't;
>>>>>>>> or `bar`
>>>>>>>>>> requests will use the query string and skip the mid when it
>>>>>>>> shouldn't.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Does that make sense? The only correct solution, is to
>>>>> somehow
>>>>>>>> prevent
>>>>>>>>>> different DSes having the same origin, and tell tenants
>>>> they
>>>>> must
>>>>>>>> use
>>>>>>>>>> CNAMEs if they need.
>>>>>>>>>>
>>>>>>>>>> This isn't a bug in Traffic Control. ATS fundamentally
>>>>> doesn't
>>>>>>>> support
>>>>>>>>>> multiple remap rules with the same parent FQDN with
>>>> different
>>>>>>>>>> configurations. Hence, Traffic Control needs to prohibit
>>>>> that.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
>>>>>>>> mitchell852@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> brennan,
>>>>>>>>>>>
>>>>>>>>>>> can you give an example of what parent.config looks like
>>>>> when 2
>>>>>>>> ds's share
>>>>>>>>>>> an origin and have different a different topology?
>>>>>>>>>>>
>>>>>>>>>>> jeremy
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
>>>>>>>> Brennan_Fieck@comcast.com
>>>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> To be clear, the "Warning" I'm talking about would
>>>>> happen at
>>>>>>>> startup, but
>>>>>>>>>>>> I'd like a UI-only constraint to come with that to
>>>>> disallow
>>>>>>>> using the API
>>>>>>>>>>>> to bind the same origin to multiple Delivery Services
>>>>> with
>>>>>>>> varying
>>>>>>>>>>>> topography requirements. It wouldn't change the
>>>> existing
>>>>>>> data,
>>>>>>>> but
>>>>>>>>>>> prevent
>>>>>>>>>>>> users from creating more bad data.
>>>>>>>>>>>>
>>>>>>>>>>>> "warning" doesn't really sufficiently describe that, my
>>>>> bad.
>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>> From: Fieck, Brennan <Br...@comcast.com>
>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 11:24 AM
>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>>>>> Delivery Services
>>>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>>>
>>>>>>>>>>>> Well the cost of fixing this bug is a constraint on the
>>>>> data.
>>>>>>>> Unless we
>>>>>>>>>>>> make it a UI-only constraint - which I'm personally
>>>>> against -
>>>>>>>> there must
>>>>>>>>>>> be
>>>>>>>>>>>> some point in the future where ATC cannot reasonably be
>>>>>>>> expected to work
>>>>>>>>>>>> with data that violates that constraint. The question
>>>> is
>>>>> when
>>>>>>>> that should
>>>>>>>>>>>> occur, which should likely happen at a minor version
>>>>> release.
>>>>>>>> Minor not
>>>>>>>>>>>> major because it doesn't involve a change in data
>>>>> structures,
>>>>>>>> merely
>>>>>>>>>>>> relationships between them - in my opinion that's a
>>>> minor
>>>>>>>> version change
>>>>>>>>>>>> but that's definitely up for debate. With several
>>>> release
>>>>>>>> candidates for
>>>>>>>>>>>> 3.0.0 that _doesn't_ include this restriction already
>>>> in
>>>>> the
>>>>>>>> wild, I
>>>>>>>>>>>> wouldn't recommend putting it in there. That means to
>>>>> fix the
>>>>>>>> bug as soon
>>>>>>>>>>>> as possible it should go in 3.1.0 which should be the
>>>>> target
>>>>>>>> of "master"
>>>>>>>>>>>> after the 3.0.0 release is cut from it.
>>>>>>>>>>>>
>>>>>>>>>>>> So I'd recommend immediately implementing the
>>>> constraint
>>>>> in
>>>>>>>> master with a
>>>>>>>>>>>> refusal to upgrade with bad data, and backport a
>>>> warning
>>>>>>> about
>>>>>>>> the future
>>>>>>>>>>>> behavior into 3.0.0 or as part of a 3.0.1 provided we
>>>> had
>>>>>>> more
>>>>>>>> changes
>>>>>>>>>>> that
>>>>>>>>>>>> would warrant a micro version bump.
>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 9:34 AM
>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>>>>> Delivery Services
>>>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>>>
>>>>>>>>>>>> -1 Holding an ATC upgrade hostage to data cleanup seems
>>>>> like
>>>>>>> a
>>>>>>>> bad idea.
>>>>>>>>>>>> The issue isn't great, but it's also not new. We
>>>> should
>>>>>>> allow
>>>>>>>> teams to
>>>>>>>>>>> fix
>>>>>>>>>>>> their data at their normal paces if it doesn't create
>>>>>>>> significant
>>>>>>>>>>> overhead
>>>>>>>>>>>> or an inherant blocker for new functionality or
>>>>> correction of
>>>>>>>> other major
>>>>>>>>>>>> problems imho.
>>>>>>>>>>>>
>>>>>>>>>>>> Jonathan G
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 12/18/18, 9:28 AM, "Fieck, Brennan" <
>>>>>>>> Brennan_Fieck@comcast.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Another option is we could detect collisions at
>>>>> startup
>>>>>>>> and simply
>>>>>>>>>>>> refuse to continue with the upgrade until the data is
>>>>> fixed.
>>>>>>>> That would
>>>>>>>>>>>> allow people using the now-unsupported data format to
>>>>>>> continue
>>>>>>>> to use
>>>>>>>>>>> their
>>>>>>>>>>>> old versions of Traffic Ops without wrecking their
>>>>> database,
>>>>>>>> but also
>>>>>>>>>>>> provide an incentive to clean up the data.
>>>>>>>>>>>> ________________________________________
>>>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
>>>>>>>>>>>> Sent: Tuesday, December 18, 2018 5:12 AM
>>>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to
>>>>> Multipe
>>>>>>>> Delivery
>>>>>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>>>>
>>>>>>>>>>>> I'm generally a fan of constrain your data in your
>>>>>>>> database, but not
>>>>>>>>>>>> necessarily exclusively. I see this as a one-way
>>>>>>>> cleanup/conversion so
>>>>>>>>>>> it
>>>>>>>>>>>> doesn't need to be configurable; otherwise you have to
>>>>> ask
>>>>>>> the
>>>>>>>> question
>>>>>>>>>>>> what happens if someone turns it off. That said,
>>>>> something
>>>>>>> in
>>>>>>>> the UI
>>>>>>>>>>> layer
>>>>>>>>>>>> would be nice to prevent spending significant
>>>> quantities
>>>>> of
>>>>>>>> time
>>>>>>>>>>> building a
>>>>>>>>>>>> complex DS only to have it fail to post for reasons
>>>> that
>>>>>>> could
>>>>>>>> have been
>>>>>>>>>>>> known earlier.
>>>>>>>>>>>>
>>>>>>>>>>>> The way my brain works in this case:
>>>>>>>>>>>> If !unique_constraint_exists_query()
>>>>>>>>>>>> If has_duplicates_query()
>>>>>>>>>>>> show_warning()
>>>>>>>>>>>> else
>>>>>>>>>>>> add_unique_constraint()
>>>>>>>>>>>>
>>>>>>>>>>>> to which the API and UI configuration could also
>>>>> make use
>>>>>>>> of
>>>>>>>>>>>> unique_constraint_exists_query() to drive additional
>>>>> layer
>>>>>>>> constraints if
>>>>>>>>>>>> desired.
>>>>>>>>>>>>
>>>>>>>>>>>> Jonathan G
>>>>>>>>>>>>
>>>>>>>>>>>> On 12/17/18, 1:11 PM, "Rawlin Peters" <
>>>>>>>> rawlin.peters@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> That is an interesting idea...detect at TO
>>>>> startup
>>>>>>>> whether or not
>>>>>>>>>>>> there are duplicate origins and operate in a
>>>>> "prevent
>>>>>>>> duplicate
>>>>>>>>>>>> origins" state if no duplicates are found or
>>>>> "prevent
>>>>>>>> conflicting
>>>>>>>>>>>> DS
>>>>>>>>>>>> topologies" state if duplicates are found? So
>>>>> once
>>>>>>>> operators have
>>>>>>>>>>>> replaced all the duplicate origins with CNAMEs,
>>>>> TO
>>>>>>> will
>>>>>>>>>>> essentially
>>>>>>>>>>>> operate in a "prohibit all duplicate origins"
>>>>> state.
>>>>>>>> That would
>>>>>>>>>>>> probably make for a simpler transition, but I'd
>>>>> want
>>>>>>>> to remove
>>>>>>>>>>> that
>>>>>>>>>>>> logic in a following release that strictly
>>>>> prohibits
>>>>>>>> duplicate
>>>>>>>>>>>> origins
>>>>>>>>>>>> (assuming that the community agrees we should
>>>>>>> prohibit
>>>>>>>> duplicate
>>>>>>>>>>>> origins altogether).
>>>>>>>>>>>>
>>>>>>>>>>>> As for DB constraints vs UI, I was thinking
>>>> those
>>>>>>>> DS-type
>>>>>>>>>>>> constraints
>>>>>>>>>>>> I pointed out would live in the API. It would
>>>>>>>> basically be added
>>>>>>>>>>>> validation in the deliveryservices POST/PUT
>>>>> endpoint
>>>>>>>> that checks
>>>>>>>>>>>> the
>>>>>>>>>>>> DB for existing DSes that conflict with the
>>>>> requested
>>>>>>>> DS.
>>>>>>>>>>>>
>>>>>>>>>>>> - Rawlin
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
>>>>>>>>>>>> <Jo...@comcast.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> These kinds of conditions should be
>>>> detectable
>>>>>>> with a
>>>>>>>>>>>> sufficiently advanced SQL query. Is it possible to add
>>>>> the
>>>>>>>> constraint if
>>>>>>>>>>>> it passes and emit a warning during TO startup
>>>> otherwise?
>>>>>>>> That would let
>>>>>>>>>>>> you know the condition exists at startup but not
>>>> getting
>>>>> in
>>>>>>>> your way and
>>>>>>>>>>>> keep you out of trouble once you've cleaned up. We
>>>> made
>>>>> a
>>>>>>>> mistake early
>>>>>>>>>>>> on, but this would acknowledge it was bad and encourage
>>>>> it to
>>>>>>>> be fixed at
>>>>>>>>>>>> the speed of operations teams. Also this puts the
>>>>> constraint
>>>>>>>> in the
>>>>>>>>>>>> database rather than the UI which is really where the
>>>>>>>> contention is for
>>>>>>>>>>>> usability.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Jonathan G
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 12/17/18, 11:38 AM, "Rawlin Peters" <
>>>>>>>>>>> rawlin.peters@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> We occasionally discuss this issue but
>>>>> haven't
>>>>>>>> tackled it
>>>>>>>>>>>> yet. I think
>>>>>>>>>>>>> the main issue is just that duplicate
>>>>> origins
>>>>>>>> have been
>>>>>>>>>>>> allowed since
>>>>>>>>>>>>> the beginning, and now everyone's Traffic
>>>>> Ops
>>>>>>>> could be
>>>>>>>>>>>> littered with
>>>>>>>>>>>>> duplicate origins. Also, depending on the
>>>>>>> config
>>>>>>>> of the
>>>>>>>>>>>> duplicate
>>>>>>>>>>>>> delivery services, the origins might not
>>>>> be in
>>>>>>>> conflict at
>>>>>>>>>>>> all (if
>>>>>>>>>>>>> they don't have different topology
>>>>>>> constraints).
>>>>>>>> I would
>>>>>>>>>>>> love for us
>>>>>>>>>>>>> to just add a uniqueness constraint, but
>>>>> there
>>>>>>>> would need
>>>>>>>>>>> to
>>>>>>>>>>>> be a fair
>>>>>>>>>>>>> amount of warning to the community before
>>>>> doing
>>>>>>>> so and
>>>>>>>>>>> might
>>>>>>>>>>>>> invalidate a significant amount of valid
>>>>> use
>>>>>>>> cases.
>>>>>>>>>>>> Operators would
>>>>>>>>>>>>> need time to make DNS CNAME records for
>>>> the
>>>>>>>> duplicate
>>>>>>>>>>>> origins and
>>>>>>>>>>>>> update their DSes to use the different
>>>>> CNAMEs.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think as a good first step to
>>>>> eliminating the
>>>>>>>> use of
>>>>>>>>>>>> duplicate
>>>>>>>>>>>>> origins altogether, we should identify
>>>>> which
>>>>>>>> "topology
>>>>>>>>>>>> constraints"
>>>>>>>>>>>>> actually cause conflicting config when
>>>> used
>>>>>>> with
>>>>>>>> duplicate
>>>>>>>>>>>> origins and
>>>>>>>>>>>>> prevent creating DSes with duplicate
>>>>> origins
>>>>>>> _if
>>>>>>>> it would
>>>>>>>>>>>> cause a
>>>>>>>>>>>>> conflict with an existing DS that uses
>>>> the
>>>>> same
>>>>>>>> origin_.
>>>>>>>>>>>>>
>>>>>>>>>>>>> For instance, I believe an HTTP and
>>>>> DNS-type DS
>>>>>>>> can live
>>>>>>>>>>>> happily
>>>>>>>>>>>>> side-by-side using the same origin
>>>>> (probably
>>>>>>>> need different
>>>>>>>>>>>>> routing_names?), but scenarios like HTTP
>>>>> and
>>>>>>>> HTTP_LIVE, or
>>>>>>>>>>>> DNS and
>>>>>>>>>>>>> HTTP_NO_CACHE sharing the same origin
>>>> will
>>>>>>> cause
>>>>>>>> conflicts
>>>>>>>>>>>> for sure.
>>>>>>>>>>>>> So maybe we can start by making sure the
>>>> DS
>>>>>>>> types "match"
>>>>>>>>>>>> when using
>>>>>>>>>>>>> the same origin:
>>>>>>>>>>>>> HTTP + DNS: possibly good, if they have
>>>>>>>> different routing
>>>>>>>>>>>> names?
>>>>>>>>>>>>> HTTP_LIVE + HTTP_LIVE_NATNL: bad
>>>>>>>>>>>>> HTTP_NO_CACHE + [any other type]: bad
>>>>>>>>>>>>> HTTP_LIVE + HTTP: bad
>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>
>>>>>>>>>>>>> There are most likely other conflict
>>>>> scenarios
>>>>>>>> that don't
>>>>>>>>>>>> involve the
>>>>>>>>>>>>> DS types, but I think this would be a
>>>> good
>>>>>>>> start. In the
>>>>>>>>>>>> future with
>>>>>>>>>>>>> Delivery Service Topologies (aka Flexible
>>>>>>>> Cachegroups aka
>>>>>>>>>>>> Bring Your
>>>>>>>>>>>>> Own Topology), we might be able to
>>>> prohibit
>>>>>>>> assigning a DS
>>>>>>>>>>>> to a
>>>>>>>>>>>>> Topology if the DS's origin is already
>>>>> used by
>>>>>>>> another DS
>>>>>>>>>>> in
>>>>>>>>>>>> a
>>>>>>>>>>>>> different Topology.
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Rawlin
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Dec 17, 2018 at 10:52 AM Fieck,
>>>>> Brennan
>>>>>>>>>>>>> <Br...@comcast.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As some of you may be aware,
>>>>> `parent.config`
>>>>>>>> files
>>>>>>>>>>>> generated by Traffic Ops can vary wildly when an origin
>>>>> is
>>>>>>>> assigned to
>>>>>>>>>>>> multiple Delivery Services. This results in undefined
>>>>>>>> behavior. I'm told
>>>>>>>>>>>> that the conflict only happens when two Delivery
>>>> Services
>>>>>>> with
>>>>>>>> different
>>>>>>>>>>>> "topology requirements" use the same origin, whatever
>>>>> that
>>>>>>>> means (content
>>>>>>>>>>>> routing type?). Regardless, the issue should be
>>>>> addressed.
>>>>>>> The
>>>>>>>> obvious
>>>>>>>>>>>> solution is to put in place a database constraint that
>>>>>>>> prevents an origin
>>>>>>>>>>>> from being assigned to more that one Delivery Service
>>>>> with
>>>>>>> API
>>>>>>>> checks in
>>>>>>>>>>>> place that would provide helpful error messages when an
>>>>>>>> attempt is made
>>>>>>>>>>> to
>>>>>>>>>>>> violate the constraint. However, would that mess with
>>>>> things
>>>>>>>> like
>>>>>>>>>>>> Multi-Site Origin? Or is it just not viable for some
>>>>> other
>>>>>>>> reason? If it
>>>>>>>>>>> is
>>>>>>>>>>>> a good solution, I'm prepared to work on a fix that
>>>>> utilizes
>>>>>>>> it.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Rawlin Peters <ra...@gmail.com>.
Simply prohibiting duplicate origin FQDNs has been vetoed now, so we
can't really move forward with that unless the vetoers change their
minds and remove their -1s.
Currently that leaves us with checking at DS create/update whether or
not it would conflict with a shared origin due to different DS types,
qstring config, etc. I am -1 on that idea since I think it will be
difficult to maintain and enumerate all the possible conflicting
cases, will present confusing error messages to the user to which the
solution would just be "create a CNAME record" anyways, and that the
ATS parent.config should really be per-remap rather than "global". I
know there has been some discussion in the ATS community about making
parent.config per-remap, but I don't have much information there.
That is to say, I don't think there is a great solution to this
problem right now that we can all agree upon, but maybe our efforts
would be better spent in the ATS community by making a per-remap
parent.config. Then the duplicate origin problem in Traffic Control
would go away.
Another option could be to have some kind of generic rules engine that
would allow an organization to create some set of rules that DSes
would have to abide by. For example, no duplicate origins, an origin
can't be in a list of known malicious domains, you can't use the DS
type "HTTP_NO_CACHE", maxDnsAnswers must be < 5, TTLs must be > 30 and
< 6000, protocol must be HTTPS, intitialDispersion must be < 10, DSCP
must be 40, etc. It seems like for something truly self-service you'd
probably need to set up some rules for your users to abide by that
might not necessarily apply to ALL organizations. Just a thought. I
don't think the duplicate origins problem alone would warrant
something like that, but if it also provided a solution for the
self-service effort, maybe it would be worth it.
- Rawlin
On Wed, Jan 23, 2019 at 4:15 AM Gelinas, Derek
<De...@comcast.com> wrote:
>
> That's the problem. We either need to be smart enough to recognize that in our config and warn the user or just prevent it and suggest to the user that they use a different fqdn.
>
> My vote is for preventing it at this point.
>
> > On Jan 23, 2019, at 12:50 AM, Gray, Jonathan <Jo...@comcast.com> wrote:
> >
> > It's not just differing types, it's anything that affects the parent.config which also includes the, qstring configuration, MSO parents/config, etc. as well.
> >
> > Jonathan G
> >
> >
> > On 1/22/19, 2:33 PM, "Dave Neuman" <ne...@apache.org> wrote:
> >
> > -1 on the one DS to Origin limitation. I think there are legit use cases
> > for it.
> > I would be +1 on the idea if we include type though. Two DSs can share the
> > same origin as long as they are of the same type (DNS, HTTP, HTTP_LIVE),
> > etc. You shouldn't be able to have two DSs with the same origin but
> > different types.
> >
> > On Mon, Jan 14, 2019 at 7:53 AM Fieck, Brennan <Br...@comcast.com>
> > wrote:
> >
> >> I don't see this as a complicated use case or a barrier to entry, an
> >> extremely basic introduction to ATC would only make use of one DS
> >> (shameless CDN-in-a-Box plug).
> >> It seems far more complicated for a user to be using many DSes to serve a
> >> single origin - we can easily make it clear in the docs that the two have a
> >> 1:1 relationship, with a footnote about CNAME records for users with the
> >> knowledge, resources, and need for such workarounds.
> >> ________________________________________
> >> From: Mark Torluemke <mt...@apache.org>
> >> Sent: Friday, January 11, 2019 1:21 PM
> >> To: dev@trafficcontrol.apache.org
> >> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> >> produces indeterminate parent.config
> >>
> >>>> 1. Prohibit creating new delivery services that would share an
> >> existing origin and prohibit updating a delivery service to a shared
> >> origin
> >> In case my position has been lost, I'm still -1 on this. :) IMO, we
> >> shouldn't let the few complicated use cases increase the barrier to entry
> >> (to using ATC) for the masses.
> >>
> >>
> >>
> >>
> >> On Fri, Jan 11, 2019 at 11:53 AM Rawlin Peters <ra...@gmail.com>
> >> wrote:
> >>
> >>> Alright, I'm trying to sum up this discussion so far since it seems
> >>> like everyone went on vacation and didn't really get a chance to wrap
> >>> this one up:
> >>> - duplicate origins cause undefined behavior
> >>> - we need a way to migrate to a future that is free of duplicate
> >>> origins in Traffic Control
> >>> - we need a visual and easy way to determine if Traffic Ops currently
> >>> contains duplicate origins, so that operators are incentivized to fix
> >>> them rather than let it slide indefinitely
> >>> - operators should have a fair amount of time to fix their duplicate
> >>> origins
> >>>
> >>> I believe this is what we've mostly agreed upon but haven't clearly voted
> >>> on:
> >>>
> >>> In release N we will:
> >>> 1. Prohibit creating new delivery services that would share an
> >>> existing origin and prohibit updating a delivery service to a shared
> >>> origin
> >>> 2. Add some kind of visual indicator that duplicate origins are a
> >>> problem that need to be fixed before release N+1; otherwise, an
> >>> upgrade to N+1 will be prohibited.
> >>>
> >>> In release N+1 we will:
> >>> 3. Include a DB migration that adds a uniqueness constraint on origin
> >>> FQDN, removing the API-level checks for that.
> >>> 4. Prevent an upgrade to N+1 if duplicate origins are found (this
> >>> might occur as a byproduct of step 3).
> >>>
> >>> I am +1 on this plan and believe this would hit on all the summarized
> >>> points above. Please provide a clear vote on this plan so that we can
> >>> dive deeper in the details (i.e. what release 'N' is, the best visual
> >>> indicator for step 2, and a friendly way to handle step 4).
> >>>
> >>> Thanks,
> >>> Rawlin
> >>>
> >>> On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <mi...@gmail.com>
> >>> wrote:
> >>>>
> >>>> Not sure TP is the right place for a warning like "clean up this
> >>>> 'duplicate' origin or your next upgrade will fail". Most users of our
> >>>> system will be like "not my problem".
> >>>>
> >>>> On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <
> >>> Brennan_Fieck@comcast.com>
> >>>> wrote:
> >>>>
> >>>>> Probably. It would impact load times by a bit, but the page for an
> >>>>> individual object is not our bottleneck.
> >>>>> ________________________________________
> >>>>> From: Robert Butts <ro...@apache.org>
> >>>>> Sent: Wednesday, December 19, 2018 11:50 AM
> >>>>> To: dev@trafficcontrol.apache.org
> >>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> >>> Services
> >>>>> produces indeterminate parent.config
> >>>>>
> >>>>>> - Including a warning on startup and an API constraint preventing
> >>> adding
> >>>>> more bad data in the next 3.0.0 Release Candidate
> >>>>>> - Adding a database constraint immediately into master that won't
> >> be
> >>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
> >>>>>
> >>>>> +1
> >>>>>
> >>>>> I understand Jonathan's objection, but at some point, we have to be
> >>> able to
> >>>>> move forward. This is a good compromise: deprecate, then remove. That
> >>> gives
> >>>>> people a full major version to fix their data.
> >>>>>
> >>>>> I would be ideal if it were more than just a logged warning, though.
> >>> Can we
> >>>>> add a big red banner in Traffic Portal, on the Delivery Service page
> >>> for
> >>>>> any DS with a duplicate origin, telling users to fix it, and that
> >> they
> >>>>> won't be able to upgrade to the next major version until it's fixed?
> >>>>>
> >>>>>
> >>>>> On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <
> >>> Brennan_Fieck@comcast.com
> >>>>>>
> >>>>> wrote:
> >>>>>
> >>>>>> So it seems like nobody has a problem with the "how" - disallowing
> >>>>>> duplicate origin FQDNs on Delivery Services - but we never reached
> >> a
> >>>>>> consensus on "when".
> >>>>>>
> >>>>>> I stand by my previous proposal:
> >>>>>> - Including a warning on startup and an API constraint preventing
> >>> adding
> >>>>>> more bad data in the next 3.0.0 Release Candidate
> >>>>>> - Adding a database constraint immediately into master that won't
> >> be
> >>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
> >>>>>> ________________________________________
> >>>>>> From: Rawlin Peters <ra...@gmail.com>
> >>>>>> Sent: Tuesday, December 18, 2018 4:59 PM
> >>>>>> To: dev@trafficcontrol.apache.org
> >>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> >>> Services
> >>>>>> produces indeterminate parent.config
> >>>>>>
> >>>>>> Also, building more around DS types will make it even harder to get
> >>>>>> away from DS types in the future too, which I know is something
> >> we've
> >>>>>> discussed on this mailing list before. It also adds to the overhead
> >>> of
> >>>>>> Delivery Service Topologies, since a lot of the DS types won't
> >>>>>> carryover into that world.
> >>>>>>
> >>>>>> - Rawlin
> >>>>>>
> >>>>>> On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
> >>>>>> <Br...@comcast.com> wrote:
> >>>>>>>
> >>>>>>> +1.
> >>>>>>> If there's a simple way to work around duplicate origins being
> >>>>>> prohibited,
> >>>>>>> then we should rely on that instead of "enumerating all those
> >>> possible
> >>>>>> conflicting
> >>>>>>> settings, which are not only highly complex and confusing, but
> >> also
> >>>>>> further
> >>>>>>> entrench us in only supporting ATS as a caching proxy (hurting
> >>> efforts
> >>>>> to
> >>>>>>> integrate e.g. Grove, nginx etc.)
> >>>>>>> ________________________________________
> >>>>>>> From: Rawlin Peters <ra...@gmail.com>
> >>>>>>> Sent: Tuesday, December 18, 2018 2:20 PM
> >>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> >>>>>> Services produces indeterminate parent.config
> >>>>>>>
> >>>>>>> There are a number of different DS settings at play that can
> >>>>>>> potentially cause conflicts. The question is: do we want to get
> >>> into
> >>>>>>> the business of enumerating all those possible conflicting
> >>> settings or
> >>>>>>> just simply prohibit duplicate origins altogether? I think we can
> >>> dig
> >>>>>>> in and get that "sufficiently advanced sql query" to check for
> >>>>>>> conflicting origins, but is that something we want to carry along
> >>> for
> >>>>>>> the foreseeable future? Aren't CNAMEs relatively cheaper than
> >>>>>>> developing and maintaining that code and the mental overhead
> >>> required
> >>>>>>> in understanding why you're getting an error that says your
> >>> requested
> >>>>>>> DS would cause an origin conflict? I think at the point you've
> >>>>>>> requested a DS that would create a conflict, you've chosen those
> >>>>>>> settings for a reason and would probably prefer to just
> >> create/use
> >>> a
> >>>>>>> CNAME in your new DS and keep the rest of your settings the same.
> >>>>>>>
> >>>>>>> Thinking in terms of errors, I'm imagining:
> >>>>>>> "cannot create delivery service: origin fqdn 'foo.example.com'
> >>> already
> >>>>>> in use"
> >>>>>>> vs
> >>>>>>> "cannot create delivery service: origin fqdn 'foo.example.com'
> >>> already
> >>>>>>> in use as type DNS_LIVE_NATNL, which is incompatible with your
> >>> chosen
> >>>>>>> type of HTTP_NO_CACHE"
> >>>>>>>
> >>>>>>> At that point you'd probably say to yourself, "uh, I need
> >>>>>>> HTTP_NO_CACHE, so what am I supposed to do now?"
> >>>>>>>
> >>>>>>> As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
> >>>>>>> because the resulting code will be simpler, but I think
> >> eliminating
> >>>>>>> the mental overhead for operators could be worthwhile too. If we
> >>> can
> >>>>>>> agree on an end state of prohibiting duplicate origins
> >> altogether,
> >>> we
> >>>>>>> can start working on a design to smoothly transition us to that
> >>> point.
> >>>>>>> Are we willing to live with "just CNAME your origin fqdn" as the
> >>>>>>> standard solution to duplicates?
> >>>>>>>
> >>>>>>> - Rawlin
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
> >>>>>>> <De...@comcast.com> wrote:
> >>>>>>>>
> >>>>>>>> The only situation in which they can share origins is if a) the
> >>>>>> origins are shared in an MSO configuration but still have different
> >>>>> defined
> >>>>>> origin fields in the delivery service, or if they're assigned to
> >>>>> completely
> >>>>>> different cachegroups. It's when two delivery services share the
> >>> same
> >>>>>> edges that there's an issue, because you end up with parent.config
> >>>>> issues.
> >>>>>> Actually you could even get away with it in mids as long as you
> >>> weren't
> >>>>>> doing anything like MSO to it.
> >>>>>>>>
> >>>>>>>> Could get messy real fast, though. Best to just create a
> >> second
> >>>>> FQDN.
> >>>>>>>>
> >>>>>>>> Derek
> >>>>>>>>
> >>>>>>>> On 12/18/18, 3:23 PM, "Fieck, Brennan" <
> >>> Brennan_Fieck@comcast.com>
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>> So no two Delivery Services may share an origin *regardless
> >>> of
> >>>>>> cache hierarchy* ? I've been told that DNS Delivery Services can
> >>> have the
> >>>>>> same origin as HTTP Delivery Services because they obey the same
> >>> cache
> >>>>>> hierarchy. You're saying that would still produce invalid output
> >>> and/or
> >>>>> is
> >>>>>> explicitly disallowed by ATS?
> >>>>>>>> ________________________________________
> >>>>>>>> From: Robert Butts <ro...@apache.org>
> >>>>>>>> Sent: Tuesday, December 18, 2018 1:09 PM
> >>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> >>> Delivery
> >>>>>> Services produces indeterminate parent.config
> >>>>>>>>
> >>>>>>>>> can you give an example of what parent.config looks like
> >>> when 2
> >>>>>> ds's share
> >>>>>>>> an origin and have different a different topology?
> >>>>>>>>
> >>>>>>>> Answering because I encountered this directly, when
> >> rewriting
> >>>>>> parent.config.
> >>>>>>>>
> >>>>>>>> For example: Suppose you have one Delivery Service:
> >>>>>>>> XML_ID: foo
> >>>>>>>> Type: HTPT_LIVE_NATL
> >>>>>>>> Query String Handling: 1 - ignore in cache key, and pass up
> >>>>>>>> Origin Server Base URL: http://foo.example.net
> >>>>>>>>
> >>>>>>>> And another Delivery Service:
> >>>>>>>> XML_ID: bar
> >>>>>>>> Type: HTPT_LIVE_NATL
> >>>>>>>> Query String Handling: 1 - ignore in cache key, and pass up
> >>>>>>>> Origin Server Base URL: http://foo.example.net
> >>>>>>>>
> >>>>>>>> ATS only supports unique `dest_domain` entries in
> >>> parent.config.
> >>>>>> Therefore,
> >>>>>>>> the parent.config generated for a server assigned to both
> >> of
> >>>>> these
> >>>>>> Delivery
> >>>>>>>> Services with either be:
> >>>>>>>>
> >>>>>>>> dest_domain=foo.example.net port=80 go_direct=true
> >>>>>>>>
> >>>>>>>> Or
> >>>>>>>>
> >>>>>>>> dest_domain=foo.example.net port=80 go_direct=false
> >>>>>> qstring=consider
> >>>>>>>>
> >>>>>>>> Right now, it's arbitrary which Perl Traffic Ops inserts,
> >> and
> >>>>> Perl
> >>>>>> provides
> >>>>>>>> no warning or error of any kind (the pending Go
> >>> parent.config PR
> >>>>>> logs an
> >>>>>>>> error).
> >>>>>>>>
> >>>>>>>> Whichever is arbitrarily inserted, the resulting remaps for
> >>> the
> >>>>>> other
> >>>>>>>> delivery service will be wrong. Either `foo` requests will
> >>> drop
> >>>>>> the query
> >>>>>>>> string when they shouldn't, and go to the mid when they
> >>>>> shouldn't;
> >>>>>> or `bar`
> >>>>>>>> requests will use the query string and skip the mid when it
> >>>>>> shouldn't.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Does that make sense? The only correct solution, is to
> >>> somehow
> >>>>>> prevent
> >>>>>>>> different DSes having the same origin, and tell tenants
> >> they
> >>> must
> >>>>>> use
> >>>>>>>> CNAMEs if they need.
> >>>>>>>>
> >>>>>>>> This isn't a bug in Traffic Control. ATS fundamentally
> >>> doesn't
> >>>>>> support
> >>>>>>>> multiple remap rules with the same parent FQDN with
> >> different
> >>>>>>>> configurations. Hence, Traffic Control needs to prohibit
> >>> that.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
> >>>>>> mitchell852@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> brennan,
> >>>>>>>>>
> >>>>>>>>> can you give an example of what parent.config looks like
> >>> when 2
> >>>>>> ds's share
> >>>>>>>>> an origin and have different a different topology?
> >>>>>>>>>
> >>>>>>>>> jeremy
> >>>>>>>>>
> >>>>>>>>> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
> >>>>>> Brennan_Fieck@comcast.com
> >>>>>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> To be clear, the "Warning" I'm talking about would
> >>> happen at
> >>>>>> startup, but
> >>>>>>>>>> I'd like a UI-only constraint to come with that to
> >>> disallow
> >>>>>> using the API
> >>>>>>>>>> to bind the same origin to multiple Delivery Services
> >>> with
> >>>>>> varying
> >>>>>>>>>> topography requirements. It wouldn't change the
> >> existing
> >>>>> data,
> >>>>>> but
> >>>>>>>>> prevent
> >>>>>>>>>> users from creating more bad data.
> >>>>>>>>>>
> >>>>>>>>>> "warning" doesn't really sufficiently describe that, my
> >>> bad.
> >>>>>>>>>> ________________________________________
> >>>>>>>>>> From: Fieck, Brennan <Br...@comcast.com>
> >>>>>>>>>> Sent: Tuesday, December 18, 2018 11:24 AM
> >>>>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> >>>>>> Delivery Services
> >>>>>>>>>> produces indeterminate parent.config
> >>>>>>>>>>
> >>>>>>>>>> Well the cost of fixing this bug is a constraint on the
> >>> data.
> >>>>>> Unless we
> >>>>>>>>>> make it a UI-only constraint - which I'm personally
> >>> against -
> >>>>>> there must
> >>>>>>>>> be
> >>>>>>>>>> some point in the future where ATC cannot reasonably be
> >>>>>> expected to work
> >>>>>>>>>> with data that violates that constraint. The question
> >> is
> >>> when
> >>>>>> that should
> >>>>>>>>>> occur, which should likely happen at a minor version
> >>> release.
> >>>>>> Minor not
> >>>>>>>>>> major because it doesn't involve a change in data
> >>> structures,
> >>>>>> merely
> >>>>>>>>>> relationships between them - in my opinion that's a
> >> minor
> >>>>>> version change
> >>>>>>>>>> but that's definitely up for debate. With several
> >> release
> >>>>>> candidates for
> >>>>>>>>>> 3.0.0 that _doesn't_ include this restriction already
> >> in
> >>> the
> >>>>>> wild, I
> >>>>>>>>>> wouldn't recommend putting it in there. That means to
> >>> fix the
> >>>>>> bug as soon
> >>>>>>>>>> as possible it should go in 3.1.0 which should be the
> >>> target
> >>>>>> of "master"
> >>>>>>>>>> after the 3.0.0 release is cut from it.
> >>>>>>>>>>
> >>>>>>>>>> So I'd recommend immediately implementing the
> >> constraint
> >>> in
> >>>>>> master with a
> >>>>>>>>>> refusal to upgrade with bad data, and backport a
> >> warning
> >>>>> about
> >>>>>> the future
> >>>>>>>>>> behavior into 3.0.0 or as part of a 3.0.1 provided we
> >> had
> >>>>> more
> >>>>>> changes
> >>>>>>>>> that
> >>>>>>>>>> would warrant a micro version bump.
> >>>>>>>>>> ________________________________________
> >>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
> >>>>>>>>>> Sent: Tuesday, December 18, 2018 9:34 AM
> >>>>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> >>>>>> Delivery Services
> >>>>>>>>>> produces indeterminate parent.config
> >>>>>>>>>>
> >>>>>>>>>> -1 Holding an ATC upgrade hostage to data cleanup seems
> >>> like
> >>>>> a
> >>>>>> bad idea.
> >>>>>>>>>> The issue isn't great, but it's also not new. We
> >> should
> >>>>> allow
> >>>>>> teams to
> >>>>>>>>> fix
> >>>>>>>>>> their data at their normal paces if it doesn't create
> >>>>>> significant
> >>>>>>>>> overhead
> >>>>>>>>>> or an inherant blocker for new functionality or
> >>> correction of
> >>>>>> other major
> >>>>>>>>>> problems imho.
> >>>>>>>>>>
> >>>>>>>>>> Jonathan G
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On 12/18/18, 9:28 AM, "Fieck, Brennan" <
> >>>>>> Brennan_Fieck@comcast.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Another option is we could detect collisions at
> >>> startup
> >>>>>> and simply
> >>>>>>>>>> refuse to continue with the upgrade until the data is
> >>> fixed.
> >>>>>> That would
> >>>>>>>>>> allow people using the now-unsupported data format to
> >>>>> continue
> >>>>>> to use
> >>>>>>>>> their
> >>>>>>>>>> old versions of Traffic Ops without wrecking their
> >>> database,
> >>>>>> but also
> >>>>>>>>>> provide an incentive to clean up the data.
> >>>>>>>>>> ________________________________________
> >>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
> >>>>>>>>>> Sent: Tuesday, December 18, 2018 5:12 AM
> >>>>>>>>>> To: dev@trafficcontrol.apache.org
> >>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to
> >>> Multipe
> >>>>>> Delivery
> >>>>>>>>>> Services produces indeterminate parent.config
> >>>>>>>>>>
> >>>>>>>>>> I'm generally a fan of constrain your data in your
> >>>>>> database, but not
> >>>>>>>>>> necessarily exclusively. I see this as a one-way
> >>>>>> cleanup/conversion so
> >>>>>>>>> it
> >>>>>>>>>> doesn't need to be configurable; otherwise you have to
> >>> ask
> >>>>> the
> >>>>>> question
> >>>>>>>>>> what happens if someone turns it off. That said,
> >>> something
> >>>>> in
> >>>>>> the UI
> >>>>>>>>> layer
> >>>>>>>>>> would be nice to prevent spending significant
> >> quantities
> >>> of
> >>>>>> time
> >>>>>>>>> building a
> >>>>>>>>>> complex DS only to have it fail to post for reasons
> >> that
> >>>>> could
> >>>>>> have been
> >>>>>>>>>> known earlier.
> >>>>>>>>>>
> >>>>>>>>>> The way my brain works in this case:
> >>>>>>>>>> If !unique_constraint_exists_query()
> >>>>>>>>>> If has_duplicates_query()
> >>>>>>>>>> show_warning()
> >>>>>>>>>> else
> >>>>>>>>>> add_unique_constraint()
> >>>>>>>>>>
> >>>>>>>>>> to which the API and UI configuration could also
> >>> make use
> >>>>>> of
> >>>>>>>>>> unique_constraint_exists_query() to drive additional
> >>> layer
> >>>>>> constraints if
> >>>>>>>>>> desired.
> >>>>>>>>>>
> >>>>>>>>>> Jonathan G
> >>>>>>>>>>
> >>>>>>>>>> On 12/17/18, 1:11 PM, "Rawlin Peters" <
> >>>>>> rawlin.peters@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> That is an interesting idea...detect at TO
> >>> startup
> >>>>>> whether or not
> >>>>>>>>>> there are duplicate origins and operate in a
> >>> "prevent
> >>>>>> duplicate
> >>>>>>>>>> origins" state if no duplicates are found or
> >>> "prevent
> >>>>>> conflicting
> >>>>>>>>>> DS
> >>>>>>>>>> topologies" state if duplicates are found? So
> >>> once
> >>>>>> operators have
> >>>>>>>>>> replaced all the duplicate origins with CNAMEs,
> >>> TO
> >>>>> will
> >>>>>>>>> essentially
> >>>>>>>>>> operate in a "prohibit all duplicate origins"
> >>> state.
> >>>>>> That would
> >>>>>>>>>> probably make for a simpler transition, but I'd
> >>> want
> >>>>>> to remove
> >>>>>>>>> that
> >>>>>>>>>> logic in a following release that strictly
> >>> prohibits
> >>>>>> duplicate
> >>>>>>>>>> origins
> >>>>>>>>>> (assuming that the community agrees we should
> >>>>> prohibit
> >>>>>> duplicate
> >>>>>>>>>> origins altogether).
> >>>>>>>>>>
> >>>>>>>>>> As for DB constraints vs UI, I was thinking
> >> those
> >>>>>> DS-type
> >>>>>>>>>> constraints
> >>>>>>>>>> I pointed out would live in the API. It would
> >>>>>> basically be added
> >>>>>>>>>> validation in the deliveryservices POST/PUT
> >>> endpoint
> >>>>>> that checks
> >>>>>>>>>> the
> >>>>>>>>>> DB for existing DSes that conflict with the
> >>> requested
> >>>>>> DS.
> >>>>>>>>>>
> >>>>>>>>>> - Rawlin
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> >>>>>>>>>> <Jo...@comcast.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> These kinds of conditions should be
> >> detectable
> >>>>> with a
> >>>>>>>>>> sufficiently advanced SQL query. Is it possible to add
> >>> the
> >>>>>> constraint if
> >>>>>>>>>> it passes and emit a warning during TO startup
> >> otherwise?
> >>>>>> That would let
> >>>>>>>>>> you know the condition exists at startup but not
> >> getting
> >>> in
> >>>>>> your way and
> >>>>>>>>>> keep you out of trouble once you've cleaned up. We
> >> made
> >>> a
> >>>>>> mistake early
> >>>>>>>>>> on, but this would acknowledge it was bad and encourage
> >>> it to
> >>>>>> be fixed at
> >>>>>>>>>> the speed of operations teams. Also this puts the
> >>> constraint
> >>>>>> in the
> >>>>>>>>>> database rather than the UI which is really where the
> >>>>>> contention is for
> >>>>>>>>>> usability.
> >>>>>>>>>>>
> >>>>>>>>>>> Jonathan G
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On 12/17/18, 11:38 AM, "Rawlin Peters" <
> >>>>>>>>> rawlin.peters@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> We occasionally discuss this issue but
> >>> haven't
> >>>>>> tackled it
> >>>>>>>>>> yet. I think
> >>>>>>>>>>> the main issue is just that duplicate
> >>> origins
> >>>>>> have been
> >>>>>>>>>> allowed since
> >>>>>>>>>>> the beginning, and now everyone's Traffic
> >>> Ops
> >>>>>> could be
> >>>>>>>>>> littered with
> >>>>>>>>>>> duplicate origins. Also, depending on the
> >>>>> config
> >>>>>> of the
> >>>>>>>>>> duplicate
> >>>>>>>>>>> delivery services, the origins might not
> >>> be in
> >>>>>> conflict at
> >>>>>>>>>> all (if
> >>>>>>>>>>> they don't have different topology
> >>>>> constraints).
> >>>>>> I would
> >>>>>>>>>> love for us
> >>>>>>>>>>> to just add a uniqueness constraint, but
> >>> there
> >>>>>> would need
> >>>>>>>>> to
> >>>>>>>>>> be a fair
> >>>>>>>>>>> amount of warning to the community before
> >>> doing
> >>>>>> so and
> >>>>>>>>> might
> >>>>>>>>>>> invalidate a significant amount of valid
> >>> use
> >>>>>> cases.
> >>>>>>>>>> Operators would
> >>>>>>>>>>> need time to make DNS CNAME records for
> >> the
> >>>>>> duplicate
> >>>>>>>>>> origins and
> >>>>>>>>>>> update their DSes to use the different
> >>> CNAMEs.
> >>>>>>>>>>>
> >>>>>>>>>>> I think as a good first step to
> >>> eliminating the
> >>>>>> use of
> >>>>>>>>>> duplicate
> >>>>>>>>>>> origins altogether, we should identify
> >>> which
> >>>>>> "topology
> >>>>>>>>>> constraints"
> >>>>>>>>>>> actually cause conflicting config when
> >> used
> >>>>> with
> >>>>>> duplicate
> >>>>>>>>>> origins and
> >>>>>>>>>>> prevent creating DSes with duplicate
> >>> origins
> >>>>> _if
> >>>>>> it would
> >>>>>>>>>> cause a
> >>>>>>>>>>> conflict with an existing DS that uses
> >> the
> >>> same
> >>>>>> origin_.
> >>>>>>>>>>>
> >>>>>>>>>>> For instance, I believe an HTTP and
> >>> DNS-type DS
> >>>>>> can live
> >>>>>>>>>> happily
> >>>>>>>>>>> side-by-side using the same origin
> >>> (probably
> >>>>>> need different
> >>>>>>>>>>> routing_names?), but scenarios like HTTP
> >>> and
> >>>>>> HTTP_LIVE, or
> >>>>>>>>>> DNS and
> >>>>>>>>>>> HTTP_NO_CACHE sharing the same origin
> >> will
> >>>>> cause
> >>>>>> conflicts
> >>>>>>>>>> for sure.
> >>>>>>>>>>> So maybe we can start by making sure the
> >> DS
> >>>>>> types "match"
> >>>>>>>>>> when using
> >>>>>>>>>>> the same origin:
> >>>>>>>>>>> HTTP + DNS: possibly good, if they have
> >>>>>> different routing
> >>>>>>>>>> names?
> >>>>>>>>>>> HTTP_LIVE + HTTP_LIVE_NATNL: bad
> >>>>>>>>>>> HTTP_NO_CACHE + [any other type]: bad
> >>>>>>>>>>> HTTP_LIVE + HTTP: bad
> >>>>>>>>>>> etc.
> >>>>>>>>>>>
> >>>>>>>>>>> There are most likely other conflict
> >>> scenarios
> >>>>>> that don't
> >>>>>>>>>> involve the
> >>>>>>>>>>> DS types, but I think this would be a
> >> good
> >>>>>> start. In the
> >>>>>>>>>> future with
> >>>>>>>>>>> Delivery Service Topologies (aka Flexible
> >>>>>> Cachegroups aka
> >>>>>>>>>> Bring Your
> >>>>>>>>>>> Own Topology), we might be able to
> >> prohibit
> >>>>>> assigning a DS
> >>>>>>>>>> to a
> >>>>>>>>>>> Topology if the DS's origin is already
> >>> used by
> >>>>>> another DS
> >>>>>>>>> in
> >>>>>>>>>> a
> >>>>>>>>>>> different Topology.
> >>>>>>>>>>>
> >>>>>>>>>>> - Rawlin
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Dec 17, 2018 at 10:52 AM Fieck,
> >>> Brennan
> >>>>>>>>>>> <Br...@comcast.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> As some of you may be aware,
> >>> `parent.config`
> >>>>>> files
> >>>>>>>>>> generated by Traffic Ops can vary wildly when an origin
> >>> is
> >>>>>> assigned to
> >>>>>>>>>> multiple Delivery Services. This results in undefined
> >>>>>> behavior. I'm told
> >>>>>>>>>> that the conflict only happens when two Delivery
> >> Services
> >>>>> with
> >>>>>> different
> >>>>>>>>>> "topology requirements" use the same origin, whatever
> >>> that
> >>>>>> means (content
> >>>>>>>>>> routing type?). Regardless, the issue should be
> >>> addressed.
> >>>>> The
> >>>>>> obvious
> >>>>>>>>>> solution is to put in place a database constraint that
> >>>>>> prevents an origin
> >>>>>>>>>> from being assigned to more that one Delivery Service
> >>> with
> >>>>> API
> >>>>>> checks in
> >>>>>>>>>> place that would provide helpful error messages when an
> >>>>>> attempt is made
> >>>>>>>>> to
> >>>>>>>>>> violate the constraint. However, would that mess with
> >>> things
> >>>>>> like
> >>>>>>>>>> Multi-Site Origin? Or is it just not viable for some
> >>> other
> >>>>>> reason? If it
> >>>>>>>>> is
> >>>>>>>>>> a good solution, I'm prepared to work on a fix that
> >>> utilizes
> >>>>>> it.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>
> >
> >
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Gelinas, Derek" <De...@comcast.com>.
That's the problem. We either need to be smart enough to recognize that in our config and warn the user or just prevent it and suggest to the user that they use a different fqdn.
My vote is for preventing it at this point.
> On Jan 23, 2019, at 12:50 AM, Gray, Jonathan <Jo...@comcast.com> wrote:
>
> It's not just differing types, it's anything that affects the parent.config which also includes the, qstring configuration, MSO parents/config, etc. as well.
>
> Jonathan G
>
>
> On 1/22/19, 2:33 PM, "Dave Neuman" <ne...@apache.org> wrote:
>
> -1 on the one DS to Origin limitation. I think there are legit use cases
> for it.
> I would be +1 on the idea if we include type though. Two DSs can share the
> same origin as long as they are of the same type (DNS, HTTP, HTTP_LIVE),
> etc. You shouldn't be able to have two DSs with the same origin but
> different types.
>
> On Mon, Jan 14, 2019 at 7:53 AM Fieck, Brennan <Br...@comcast.com>
> wrote:
>
>> I don't see this as a complicated use case or a barrier to entry, an
>> extremely basic introduction to ATC would only make use of one DS
>> (shameless CDN-in-a-Box plug).
>> It seems far more complicated for a user to be using many DSes to serve a
>> single origin - we can easily make it clear in the docs that the two have a
>> 1:1 relationship, with a footnote about CNAME records for users with the
>> knowledge, resources, and need for such workarounds.
>> ________________________________________
>> From: Mark Torluemke <mt...@apache.org>
>> Sent: Friday, January 11, 2019 1:21 PM
>> To: dev@trafficcontrol.apache.org
>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
>> produces indeterminate parent.config
>>
>>>> 1. Prohibit creating new delivery services that would share an
>> existing origin and prohibit updating a delivery service to a shared
>> origin
>> In case my position has been lost, I'm still -1 on this. :) IMO, we
>> shouldn't let the few complicated use cases increase the barrier to entry
>> (to using ATC) for the masses.
>>
>>
>>
>>
>> On Fri, Jan 11, 2019 at 11:53 AM Rawlin Peters <ra...@gmail.com>
>> wrote:
>>
>>> Alright, I'm trying to sum up this discussion so far since it seems
>>> like everyone went on vacation and didn't really get a chance to wrap
>>> this one up:
>>> - duplicate origins cause undefined behavior
>>> - we need a way to migrate to a future that is free of duplicate
>>> origins in Traffic Control
>>> - we need a visual and easy way to determine if Traffic Ops currently
>>> contains duplicate origins, so that operators are incentivized to fix
>>> them rather than let it slide indefinitely
>>> - operators should have a fair amount of time to fix their duplicate
>>> origins
>>>
>>> I believe this is what we've mostly agreed upon but haven't clearly voted
>>> on:
>>>
>>> In release N we will:
>>> 1. Prohibit creating new delivery services that would share an
>>> existing origin and prohibit updating a delivery service to a shared
>>> origin
>>> 2. Add some kind of visual indicator that duplicate origins are a
>>> problem that need to be fixed before release N+1; otherwise, an
>>> upgrade to N+1 will be prohibited.
>>>
>>> In release N+1 we will:
>>> 3. Include a DB migration that adds a uniqueness constraint on origin
>>> FQDN, removing the API-level checks for that.
>>> 4. Prevent an upgrade to N+1 if duplicate origins are found (this
>>> might occur as a byproduct of step 3).
>>>
>>> I am +1 on this plan and believe this would hit on all the summarized
>>> points above. Please provide a clear vote on this plan so that we can
>>> dive deeper in the details (i.e. what release 'N' is, the best visual
>>> indicator for step 2, and a friendly way to handle step 4).
>>>
>>> Thanks,
>>> Rawlin
>>>
>>> On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <mi...@gmail.com>
>>> wrote:
>>>>
>>>> Not sure TP is the right place for a warning like "clean up this
>>>> 'duplicate' origin or your next upgrade will fail". Most users of our
>>>> system will be like "not my problem".
>>>>
>>>> On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <
>>> Brennan_Fieck@comcast.com>
>>>> wrote:
>>>>
>>>>> Probably. It would impact load times by a bit, but the page for an
>>>>> individual object is not our bottleneck.
>>>>> ________________________________________
>>>>> From: Robert Butts <ro...@apache.org>
>>>>> Sent: Wednesday, December 19, 2018 11:50 AM
>>>>> To: dev@trafficcontrol.apache.org
>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>>> Services
>>>>> produces indeterminate parent.config
>>>>>
>>>>>> - Including a warning on startup and an API constraint preventing
>>> adding
>>>>> more bad data in the next 3.0.0 Release Candidate
>>>>>> - Adding a database constraint immediately into master that won't
>> be
>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
>>>>>
>>>>> +1
>>>>>
>>>>> I understand Jonathan's objection, but at some point, we have to be
>>> able to
>>>>> move forward. This is a good compromise: deprecate, then remove. That
>>> gives
>>>>> people a full major version to fix their data.
>>>>>
>>>>> I would be ideal if it were more than just a logged warning, though.
>>> Can we
>>>>> add a big red banner in Traffic Portal, on the Delivery Service page
>>> for
>>>>> any DS with a duplicate origin, telling users to fix it, and that
>> they
>>>>> won't be able to upgrade to the next major version until it's fixed?
>>>>>
>>>>>
>>>>> On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <
>>> Brennan_Fieck@comcast.com
>>>>>>
>>>>> wrote:
>>>>>
>>>>>> So it seems like nobody has a problem with the "how" - disallowing
>>>>>> duplicate origin FQDNs on Delivery Services - but we never reached
>> a
>>>>>> consensus on "when".
>>>>>>
>>>>>> I stand by my previous proposal:
>>>>>> - Including a warning on startup and an API constraint preventing
>>> adding
>>>>>> more bad data in the next 3.0.0 Release Candidate
>>>>>> - Adding a database constraint immediately into master that won't
>> be
>>>>>> cherry-picked into 3.0.0 but should be included in 3.1.0
>>>>>> ________________________________________
>>>>>> From: Rawlin Peters <ra...@gmail.com>
>>>>>> Sent: Tuesday, December 18, 2018 4:59 PM
>>>>>> To: dev@trafficcontrol.apache.org
>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>>> Services
>>>>>> produces indeterminate parent.config
>>>>>>
>>>>>> Also, building more around DS types will make it even harder to get
>>>>>> away from DS types in the future too, which I know is something
>> we've
>>>>>> discussed on this mailing list before. It also adds to the overhead
>>> of
>>>>>> Delivery Service Topologies, since a lot of the DS types won't
>>>>>> carryover into that world.
>>>>>>
>>>>>> - Rawlin
>>>>>>
>>>>>> On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
>>>>>> <Br...@comcast.com> wrote:
>>>>>>>
>>>>>>> +1.
>>>>>>> If there's a simple way to work around duplicate origins being
>>>>>> prohibited,
>>>>>>> then we should rely on that instead of "enumerating all those
>>> possible
>>>>>> conflicting
>>>>>>> settings, which are not only highly complex and confusing, but
>> also
>>>>>> further
>>>>>>> entrench us in only supporting ATS as a caching proxy (hurting
>>> efforts
>>>>> to
>>>>>>> integrate e.g. Grove, nginx etc.)
>>>>>>> ________________________________________
>>>>>>> From: Rawlin Peters <ra...@gmail.com>
>>>>>>> Sent: Tuesday, December 18, 2018 2:20 PM
>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
>>>>>> Services produces indeterminate parent.config
>>>>>>>
>>>>>>> There are a number of different DS settings at play that can
>>>>>>> potentially cause conflicts. The question is: do we want to get
>>> into
>>>>>>> the business of enumerating all those possible conflicting
>>> settings or
>>>>>>> just simply prohibit duplicate origins altogether? I think we can
>>> dig
>>>>>>> in and get that "sufficiently advanced sql query" to check for
>>>>>>> conflicting origins, but is that something we want to carry along
>>> for
>>>>>>> the foreseeable future? Aren't CNAMEs relatively cheaper than
>>>>>>> developing and maintaining that code and the mental overhead
>>> required
>>>>>>> in understanding why you're getting an error that says your
>>> requested
>>>>>>> DS would cause an origin conflict? I think at the point you've
>>>>>>> requested a DS that would create a conflict, you've chosen those
>>>>>>> settings for a reason and would probably prefer to just
>> create/use
>>> a
>>>>>>> CNAME in your new DS and keep the rest of your settings the same.
>>>>>>>
>>>>>>> Thinking in terms of errors, I'm imagining:
>>>>>>> "cannot create delivery service: origin fqdn 'foo.example.com'
>>> already
>>>>>> in use"
>>>>>>> vs
>>>>>>> "cannot create delivery service: origin fqdn 'foo.example.com'
>>> already
>>>>>>> in use as type DNS_LIVE_NATNL, which is incompatible with your
>>> chosen
>>>>>>> type of HTTP_NO_CACHE"
>>>>>>>
>>>>>>> At that point you'd probably say to yourself, "uh, I need
>>>>>>> HTTP_NO_CACHE, so what am I supposed to do now?"
>>>>>>>
>>>>>>> As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
>>>>>>> because the resulting code will be simpler, but I think
>> eliminating
>>>>>>> the mental overhead for operators could be worthwhile too. If we
>>> can
>>>>>>> agree on an end state of prohibiting duplicate origins
>> altogether,
>>> we
>>>>>>> can start working on a design to smoothly transition us to that
>>> point.
>>>>>>> Are we willing to live with "just CNAME your origin fqdn" as the
>>>>>>> standard solution to duplicates?
>>>>>>>
>>>>>>> - Rawlin
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
>>>>>>> <De...@comcast.com> wrote:
>>>>>>>>
>>>>>>>> The only situation in which they can share origins is if a) the
>>>>>> origins are shared in an MSO configuration but still have different
>>>>> defined
>>>>>> origin fields in the delivery service, or if they're assigned to
>>>>> completely
>>>>>> different cachegroups. It's when two delivery services share the
>>> same
>>>>>> edges that there's an issue, because you end up with parent.config
>>>>> issues.
>>>>>> Actually you could even get away with it in mids as long as you
>>> weren't
>>>>>> doing anything like MSO to it.
>>>>>>>>
>>>>>>>> Could get messy real fast, though. Best to just create a
>> second
>>>>> FQDN.
>>>>>>>>
>>>>>>>> Derek
>>>>>>>>
>>>>>>>> On 12/18/18, 3:23 PM, "Fieck, Brennan" <
>>> Brennan_Fieck@comcast.com>
>>>>>> wrote:
>>>>>>>>
>>>>>>>> So no two Delivery Services may share an origin *regardless
>>> of
>>>>>> cache hierarchy* ? I've been told that DNS Delivery Services can
>>> have the
>>>>>> same origin as HTTP Delivery Services because they obey the same
>>> cache
>>>>>> hierarchy. You're saying that would still produce invalid output
>>> and/or
>>>>> is
>>>>>> explicitly disallowed by ATS?
>>>>>>>> ________________________________________
>>>>>>>> From: Robert Butts <ro...@apache.org>
>>>>>>>> Sent: Tuesday, December 18, 2018 1:09 PM
>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>> Delivery
>>>>>> Services produces indeterminate parent.config
>>>>>>>>
>>>>>>>>> can you give an example of what parent.config looks like
>>> when 2
>>>>>> ds's share
>>>>>>>> an origin and have different a different topology?
>>>>>>>>
>>>>>>>> Answering because I encountered this directly, when
>> rewriting
>>>>>> parent.config.
>>>>>>>>
>>>>>>>> For example: Suppose you have one Delivery Service:
>>>>>>>> XML_ID: foo
>>>>>>>> Type: HTPT_LIVE_NATL
>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass up
>>>>>>>> Origin Server Base URL: http://foo.example.net
>>>>>>>>
>>>>>>>> And another Delivery Service:
>>>>>>>> XML_ID: bar
>>>>>>>> Type: HTPT_LIVE_NATL
>>>>>>>> Query String Handling: 1 - ignore in cache key, and pass up
>>>>>>>> Origin Server Base URL: http://foo.example.net
>>>>>>>>
>>>>>>>> ATS only supports unique `dest_domain` entries in
>>> parent.config.
>>>>>> Therefore,
>>>>>>>> the parent.config generated for a server assigned to both
>> of
>>>>> these
>>>>>> Delivery
>>>>>>>> Services with either be:
>>>>>>>>
>>>>>>>> dest_domain=foo.example.net port=80 go_direct=true
>>>>>>>>
>>>>>>>> Or
>>>>>>>>
>>>>>>>> dest_domain=foo.example.net port=80 go_direct=false
>>>>>> qstring=consider
>>>>>>>>
>>>>>>>> Right now, it's arbitrary which Perl Traffic Ops inserts,
>> and
>>>>> Perl
>>>>>> provides
>>>>>>>> no warning or error of any kind (the pending Go
>>> parent.config PR
>>>>>> logs an
>>>>>>>> error).
>>>>>>>>
>>>>>>>> Whichever is arbitrarily inserted, the resulting remaps for
>>> the
>>>>>> other
>>>>>>>> delivery service will be wrong. Either `foo` requests will
>>> drop
>>>>>> the query
>>>>>>>> string when they shouldn't, and go to the mid when they
>>>>> shouldn't;
>>>>>> or `bar`
>>>>>>>> requests will use the query string and skip the mid when it
>>>>>> shouldn't.
>>>>>>>>
>>>>>>>>
>>>>>>>> Does that make sense? The only correct solution, is to
>>> somehow
>>>>>> prevent
>>>>>>>> different DSes having the same origin, and tell tenants
>> they
>>> must
>>>>>> use
>>>>>>>> CNAMEs if they need.
>>>>>>>>
>>>>>>>> This isn't a bug in Traffic Control. ATS fundamentally
>>> doesn't
>>>>>> support
>>>>>>>> multiple remap rules with the same parent FQDN with
>> different
>>>>>>>> configurations. Hence, Traffic Control needs to prohibit
>>> that.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
>>>>>> mitchell852@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> brennan,
>>>>>>>>>
>>>>>>>>> can you give an example of what parent.config looks like
>>> when 2
>>>>>> ds's share
>>>>>>>>> an origin and have different a different topology?
>>>>>>>>>
>>>>>>>>> jeremy
>>>>>>>>>
>>>>>>>>> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
>>>>>> Brennan_Fieck@comcast.com
>>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> To be clear, the "Warning" I'm talking about would
>>> happen at
>>>>>> startup, but
>>>>>>>>>> I'd like a UI-only constraint to come with that to
>>> disallow
>>>>>> using the API
>>>>>>>>>> to bind the same origin to multiple Delivery Services
>>> with
>>>>>> varying
>>>>>>>>>> topography requirements. It wouldn't change the
>> existing
>>>>> data,
>>>>>> but
>>>>>>>>> prevent
>>>>>>>>>> users from creating more bad data.
>>>>>>>>>>
>>>>>>>>>> "warning" doesn't really sufficiently describe that, my
>>> bad.
>>>>>>>>>> ________________________________________
>>>>>>>>>> From: Fieck, Brennan <Br...@comcast.com>
>>>>>>>>>> Sent: Tuesday, December 18, 2018 11:24 AM
>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>>> Delivery Services
>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>
>>>>>>>>>> Well the cost of fixing this bug is a constraint on the
>>> data.
>>>>>> Unless we
>>>>>>>>>> make it a UI-only constraint - which I'm personally
>>> against -
>>>>>> there must
>>>>>>>>> be
>>>>>>>>>> some point in the future where ATC cannot reasonably be
>>>>>> expected to work
>>>>>>>>>> with data that violates that constraint. The question
>> is
>>> when
>>>>>> that should
>>>>>>>>>> occur, which should likely happen at a minor version
>>> release.
>>>>>> Minor not
>>>>>>>>>> major because it doesn't involve a change in data
>>> structures,
>>>>>> merely
>>>>>>>>>> relationships between them - in my opinion that's a
>> minor
>>>>>> version change
>>>>>>>>>> but that's definitely up for debate. With several
>> release
>>>>>> candidates for
>>>>>>>>>> 3.0.0 that _doesn't_ include this restriction already
>> in
>>> the
>>>>>> wild, I
>>>>>>>>>> wouldn't recommend putting it in there. That means to
>>> fix the
>>>>>> bug as soon
>>>>>>>>>> as possible it should go in 3.1.0 which should be the
>>> target
>>>>>> of "master"
>>>>>>>>>> after the 3.0.0 release is cut from it.
>>>>>>>>>>
>>>>>>>>>> So I'd recommend immediately implementing the
>> constraint
>>> in
>>>>>> master with a
>>>>>>>>>> refusal to upgrade with bad data, and backport a
>> warning
>>>>> about
>>>>>> the future
>>>>>>>>>> behavior into 3.0.0 or as part of a 3.0.1 provided we
>> had
>>>>> more
>>>>>> changes
>>>>>>>>> that
>>>>>>>>>> would warrant a micro version bump.
>>>>>>>>>> ________________________________________
>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
>>>>>>>>>> Sent: Tuesday, December 18, 2018 9:34 AM
>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
>>>>>> Delivery Services
>>>>>>>>>> produces indeterminate parent.config
>>>>>>>>>>
>>>>>>>>>> -1 Holding an ATC upgrade hostage to data cleanup seems
>>> like
>>>>> a
>>>>>> bad idea.
>>>>>>>>>> The issue isn't great, but it's also not new. We
>> should
>>>>> allow
>>>>>> teams to
>>>>>>>>> fix
>>>>>>>>>> their data at their normal paces if it doesn't create
>>>>>> significant
>>>>>>>>> overhead
>>>>>>>>>> or an inherant blocker for new functionality or
>>> correction of
>>>>>> other major
>>>>>>>>>> problems imho.
>>>>>>>>>>
>>>>>>>>>> Jonathan G
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 12/18/18, 9:28 AM, "Fieck, Brennan" <
>>>>>> Brennan_Fieck@comcast.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Another option is we could detect collisions at
>>> startup
>>>>>> and simply
>>>>>>>>>> refuse to continue with the upgrade until the data is
>>> fixed.
>>>>>> That would
>>>>>>>>>> allow people using the now-unsupported data format to
>>>>> continue
>>>>>> to use
>>>>>>>>> their
>>>>>>>>>> old versions of Traffic Ops without wrecking their
>>> database,
>>>>>> but also
>>>>>>>>>> provide an incentive to clean up the data.
>>>>>>>>>> ________________________________________
>>>>>>>>>> From: Gray, Jonathan <Jo...@comcast.com>
>>>>>>>>>> Sent: Tuesday, December 18, 2018 5:12 AM
>>>>>>>>>> To: dev@trafficcontrol.apache.org
>>>>>>>>>> Subject: Re: [EXTERNAL] Re: Origins assigned to
>>> Multipe
>>>>>> Delivery
>>>>>>>>>> Services produces indeterminate parent.config
>>>>>>>>>>
>>>>>>>>>> I'm generally a fan of constrain your data in your
>>>>>> database, but not
>>>>>>>>>> necessarily exclusively. I see this as a one-way
>>>>>> cleanup/conversion so
>>>>>>>>> it
>>>>>>>>>> doesn't need to be configurable; otherwise you have to
>>> ask
>>>>> the
>>>>>> question
>>>>>>>>>> what happens if someone turns it off. That said,
>>> something
>>>>> in
>>>>>> the UI
>>>>>>>>> layer
>>>>>>>>>> would be nice to prevent spending significant
>> quantities
>>> of
>>>>>> time
>>>>>>>>> building a
>>>>>>>>>> complex DS only to have it fail to post for reasons
>> that
>>>>> could
>>>>>> have been
>>>>>>>>>> known earlier.
>>>>>>>>>>
>>>>>>>>>> The way my brain works in this case:
>>>>>>>>>> If !unique_constraint_exists_query()
>>>>>>>>>> If has_duplicates_query()
>>>>>>>>>> show_warning()
>>>>>>>>>> else
>>>>>>>>>> add_unique_constraint()
>>>>>>>>>>
>>>>>>>>>> to which the API and UI configuration could also
>>> make use
>>>>>> of
>>>>>>>>>> unique_constraint_exists_query() to drive additional
>>> layer
>>>>>> constraints if
>>>>>>>>>> desired.
>>>>>>>>>>
>>>>>>>>>> Jonathan G
>>>>>>>>>>
>>>>>>>>>> On 12/17/18, 1:11 PM, "Rawlin Peters" <
>>>>>> rawlin.peters@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> That is an interesting idea...detect at TO
>>> startup
>>>>>> whether or not
>>>>>>>>>> there are duplicate origins and operate in a
>>> "prevent
>>>>>> duplicate
>>>>>>>>>> origins" state if no duplicates are found or
>>> "prevent
>>>>>> conflicting
>>>>>>>>>> DS
>>>>>>>>>> topologies" state if duplicates are found? So
>>> once
>>>>>> operators have
>>>>>>>>>> replaced all the duplicate origins with CNAMEs,
>>> TO
>>>>> will
>>>>>>>>> essentially
>>>>>>>>>> operate in a "prohibit all duplicate origins"
>>> state.
>>>>>> That would
>>>>>>>>>> probably make for a simpler transition, but I'd
>>> want
>>>>>> to remove
>>>>>>>>> that
>>>>>>>>>> logic in a following release that strictly
>>> prohibits
>>>>>> duplicate
>>>>>>>>>> origins
>>>>>>>>>> (assuming that the community agrees we should
>>>>> prohibit
>>>>>> duplicate
>>>>>>>>>> origins altogether).
>>>>>>>>>>
>>>>>>>>>> As for DB constraints vs UI, I was thinking
>> those
>>>>>> DS-type
>>>>>>>>>> constraints
>>>>>>>>>> I pointed out would live in the API. It would
>>>>>> basically be added
>>>>>>>>>> validation in the deliveryservices POST/PUT
>>> endpoint
>>>>>> that checks
>>>>>>>>>> the
>>>>>>>>>> DB for existing DSes that conflict with the
>>> requested
>>>>>> DS.
>>>>>>>>>>
>>>>>>>>>> - Rawlin
>>>>>>>>>>
>>>>>>>>>> On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
>>>>>>>>>> <Jo...@comcast.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>> These kinds of conditions should be
>> detectable
>>>>> with a
>>>>>>>>>> sufficiently advanced SQL query. Is it possible to add
>>> the
>>>>>> constraint if
>>>>>>>>>> it passes and emit a warning during TO startup
>> otherwise?
>>>>>> That would let
>>>>>>>>>> you know the condition exists at startup but not
>> getting
>>> in
>>>>>> your way and
>>>>>>>>>> keep you out of trouble once you've cleaned up. We
>> made
>>> a
>>>>>> mistake early
>>>>>>>>>> on, but this would acknowledge it was bad and encourage
>>> it to
>>>>>> be fixed at
>>>>>>>>>> the speed of operations teams. Also this puts the
>>> constraint
>>>>>> in the
>>>>>>>>>> database rather than the UI which is really where the
>>>>>> contention is for
>>>>>>>>>> usability.
>>>>>>>>>>>
>>>>>>>>>>> Jonathan G
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 12/17/18, 11:38 AM, "Rawlin Peters" <
>>>>>>>>> rawlin.peters@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> We occasionally discuss this issue but
>>> haven't
>>>>>> tackled it
>>>>>>>>>> yet. I think
>>>>>>>>>>> the main issue is just that duplicate
>>> origins
>>>>>> have been
>>>>>>>>>> allowed since
>>>>>>>>>>> the beginning, and now everyone's Traffic
>>> Ops
>>>>>> could be
>>>>>>>>>> littered with
>>>>>>>>>>> duplicate origins. Also, depending on the
>>>>> config
>>>>>> of the
>>>>>>>>>> duplicate
>>>>>>>>>>> delivery services, the origins might not
>>> be in
>>>>>> conflict at
>>>>>>>>>> all (if
>>>>>>>>>>> they don't have different topology
>>>>> constraints).
>>>>>> I would
>>>>>>>>>> love for us
>>>>>>>>>>> to just add a uniqueness constraint, but
>>> there
>>>>>> would need
>>>>>>>>> to
>>>>>>>>>> be a fair
>>>>>>>>>>> amount of warning to the community before
>>> doing
>>>>>> so and
>>>>>>>>> might
>>>>>>>>>>> invalidate a significant amount of valid
>>> use
>>>>>> cases.
>>>>>>>>>> Operators would
>>>>>>>>>>> need time to make DNS CNAME records for
>> the
>>>>>> duplicate
>>>>>>>>>> origins and
>>>>>>>>>>> update their DSes to use the different
>>> CNAMEs.
>>>>>>>>>>>
>>>>>>>>>>> I think as a good first step to
>>> eliminating the
>>>>>> use of
>>>>>>>>>> duplicate
>>>>>>>>>>> origins altogether, we should identify
>>> which
>>>>>> "topology
>>>>>>>>>> constraints"
>>>>>>>>>>> actually cause conflicting config when
>> used
>>>>> with
>>>>>> duplicate
>>>>>>>>>> origins and
>>>>>>>>>>> prevent creating DSes with duplicate
>>> origins
>>>>> _if
>>>>>> it would
>>>>>>>>>> cause a
>>>>>>>>>>> conflict with an existing DS that uses
>> the
>>> same
>>>>>> origin_.
>>>>>>>>>>>
>>>>>>>>>>> For instance, I believe an HTTP and
>>> DNS-type DS
>>>>>> can live
>>>>>>>>>> happily
>>>>>>>>>>> side-by-side using the same origin
>>> (probably
>>>>>> need different
>>>>>>>>>>> routing_names?), but scenarios like HTTP
>>> and
>>>>>> HTTP_LIVE, or
>>>>>>>>>> DNS and
>>>>>>>>>>> HTTP_NO_CACHE sharing the same origin
>> will
>>>>> cause
>>>>>> conflicts
>>>>>>>>>> for sure.
>>>>>>>>>>> So maybe we can start by making sure the
>> DS
>>>>>> types "match"
>>>>>>>>>> when using
>>>>>>>>>>> the same origin:
>>>>>>>>>>> HTTP + DNS: possibly good, if they have
>>>>>> different routing
>>>>>>>>>> names?
>>>>>>>>>>> HTTP_LIVE + HTTP_LIVE_NATNL: bad
>>>>>>>>>>> HTTP_NO_CACHE + [any other type]: bad
>>>>>>>>>>> HTTP_LIVE + HTTP: bad
>>>>>>>>>>> etc.
>>>>>>>>>>>
>>>>>>>>>>> There are most likely other conflict
>>> scenarios
>>>>>> that don't
>>>>>>>>>> involve the
>>>>>>>>>>> DS types, but I think this would be a
>> good
>>>>>> start. In the
>>>>>>>>>> future with
>>>>>>>>>>> Delivery Service Topologies (aka Flexible
>>>>>> Cachegroups aka
>>>>>>>>>> Bring Your
>>>>>>>>>>> Own Topology), we might be able to
>> prohibit
>>>>>> assigning a DS
>>>>>>>>>> to a
>>>>>>>>>>> Topology if the DS's origin is already
>>> used by
>>>>>> another DS
>>>>>>>>> in
>>>>>>>>>> a
>>>>>>>>>>> different Topology.
>>>>>>>>>>>
>>>>>>>>>>> - Rawlin
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Dec 17, 2018 at 10:52 AM Fieck,
>>> Brennan
>>>>>>>>>>> <Br...@comcast.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> As some of you may be aware,
>>> `parent.config`
>>>>>> files
>>>>>>>>>> generated by Traffic Ops can vary wildly when an origin
>>> is
>>>>>> assigned to
>>>>>>>>>> multiple Delivery Services. This results in undefined
>>>>>> behavior. I'm told
>>>>>>>>>> that the conflict only happens when two Delivery
>> Services
>>>>> with
>>>>>> different
>>>>>>>>>> "topology requirements" use the same origin, whatever
>>> that
>>>>>> means (content
>>>>>>>>>> routing type?). Regardless, the issue should be
>>> addressed.
>>>>> The
>>>>>> obvious
>>>>>>>>>> solution is to put in place a database constraint that
>>>>>> prevents an origin
>>>>>>>>>> from being assigned to more that one Delivery Service
>>> with
>>>>> API
>>>>>> checks in
>>>>>>>>>> place that would provide helpful error messages when an
>>>>>> attempt is made
>>>>>>>>> to
>>>>>>>>>> violate the constraint. However, would that mess with
>>> things
>>>>>> like
>>>>>>>>>> Multi-Site Origin? Or is it just not viable for some
>>> other
>>>>>> reason? If it
>>>>>>>>> is
>>>>>>>>>> a good solution, I'm prepared to work on a fix that
>>> utilizes
>>>>>> it.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>
>>
>
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Gray, Jonathan" <Jo...@comcast.com>.
It's not just differing types, it's anything that affects the parent.config which also includes the, qstring configuration, MSO parents/config, etc. as well.
Jonathan G
On 1/22/19, 2:33 PM, "Dave Neuman" <ne...@apache.org> wrote:
-1 on the one DS to Origin limitation. I think there are legit use cases
for it.
I would be +1 on the idea if we include type though. Two DSs can share the
same origin as long as they are of the same type (DNS, HTTP, HTTP_LIVE),
etc. You shouldn't be able to have two DSs with the same origin but
different types.
On Mon, Jan 14, 2019 at 7:53 AM Fieck, Brennan <Br...@comcast.com>
wrote:
> I don't see this as a complicated use case or a barrier to entry, an
> extremely basic introduction to ATC would only make use of one DS
> (shameless CDN-in-a-Box plug).
> It seems far more complicated for a user to be using many DSes to serve a
> single origin - we can easily make it clear in the docs that the two have a
> 1:1 relationship, with a footnote about CNAME records for users with the
> knowledge, resources, and need for such workarounds.
> ________________________________________
> From: Mark Torluemke <mt...@apache.org>
> Sent: Friday, January 11, 2019 1:21 PM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> produces indeterminate parent.config
>
> >> 1. Prohibit creating new delivery services that would share an
> existing origin and prohibit updating a delivery service to a shared
> origin
> In case my position has been lost, I'm still -1 on this. :) IMO, we
> shouldn't let the few complicated use cases increase the barrier to entry
> (to using ATC) for the masses.
>
>
>
>
> On Fri, Jan 11, 2019 at 11:53 AM Rawlin Peters <ra...@gmail.com>
> wrote:
>
> > Alright, I'm trying to sum up this discussion so far since it seems
> > like everyone went on vacation and didn't really get a chance to wrap
> > this one up:
> > - duplicate origins cause undefined behavior
> > - we need a way to migrate to a future that is free of duplicate
> > origins in Traffic Control
> > - we need a visual and easy way to determine if Traffic Ops currently
> > contains duplicate origins, so that operators are incentivized to fix
> > them rather than let it slide indefinitely
> > - operators should have a fair amount of time to fix their duplicate
> > origins
> >
> > I believe this is what we've mostly agreed upon but haven't clearly voted
> > on:
> >
> > In release N we will:
> > 1. Prohibit creating new delivery services that would share an
> > existing origin and prohibit updating a delivery service to a shared
> > origin
> > 2. Add some kind of visual indicator that duplicate origins are a
> > problem that need to be fixed before release N+1; otherwise, an
> > upgrade to N+1 will be prohibited.
> >
> > In release N+1 we will:
> > 3. Include a DB migration that adds a uniqueness constraint on origin
> > FQDN, removing the API-level checks for that.
> > 4. Prevent an upgrade to N+1 if duplicate origins are found (this
> > might occur as a byproduct of step 3).
> >
> > I am +1 on this plan and believe this would hit on all the summarized
> > points above. Please provide a clear vote on this plan so that we can
> > dive deeper in the details (i.e. what release 'N' is, the best visual
> > indicator for step 2, and a friendly way to handle step 4).
> >
> > Thanks,
> > Rawlin
> >
> > On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <mi...@gmail.com>
> > wrote:
> > >
> > > Not sure TP is the right place for a warning like "clean up this
> > > 'duplicate' origin or your next upgrade will fail". Most users of our
> > > system will be like "not my problem".
> > >
> > > On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <
> > Brennan_Fieck@comcast.com>
> > > wrote:
> > >
> > > > Probably. It would impact load times by a bit, but the page for an
> > > > individual object is not our bottleneck.
> > > > ________________________________________
> > > > From: Robert Butts <ro...@apache.org>
> > > > Sent: Wednesday, December 19, 2018 11:50 AM
> > > > To: dev@trafficcontrol.apache.org
> > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > Services
> > > > produces indeterminate parent.config
> > > >
> > > > > - Including a warning on startup and an API constraint preventing
> > adding
> > > > more bad data in the next 3.0.0 Release Candidate
> > > > > - Adding a database constraint immediately into master that won't
> be
> > > > cherry-picked into 3.0.0 but should be included in 3.1.0
> > > >
> > > > +1
> > > >
> > > > I understand Jonathan's objection, but at some point, we have to be
> > able to
> > > > move forward. This is a good compromise: deprecate, then remove. That
> > gives
> > > > people a full major version to fix their data.
> > > >
> > > > I would be ideal if it were more than just a logged warning, though.
> > Can we
> > > > add a big red banner in Traffic Portal, on the Delivery Service page
> > for
> > > > any DS with a duplicate origin, telling users to fix it, and that
> they
> > > > won't be able to upgrade to the next major version until it's fixed?
> > > >
> > > >
> > > > On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <
> > Brennan_Fieck@comcast.com
> > > > >
> > > > wrote:
> > > >
> > > > > So it seems like nobody has a problem with the "how" - disallowing
> > > > > duplicate origin FQDNs on Delivery Services - but we never reached
> a
> > > > > consensus on "when".
> > > > >
> > > > > I stand by my previous proposal:
> > > > > - Including a warning on startup and an API constraint preventing
> > adding
> > > > > more bad data in the next 3.0.0 Release Candidate
> > > > > - Adding a database constraint immediately into master that won't
> be
> > > > > cherry-picked into 3.0.0 but should be included in 3.1.0
> > > > > ________________________________________
> > > > > From: Rawlin Peters <ra...@gmail.com>
> > > > > Sent: Tuesday, December 18, 2018 4:59 PM
> > > > > To: dev@trafficcontrol.apache.org
> > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > Services
> > > > > produces indeterminate parent.config
> > > > >
> > > > > Also, building more around DS types will make it even harder to get
> > > > > away from DS types in the future too, which I know is something
> we've
> > > > > discussed on this mailing list before. It also adds to the overhead
> > of
> > > > > Delivery Service Topologies, since a lot of the DS types won't
> > > > > carryover into that world.
> > > > >
> > > > > - Rawlin
> > > > >
> > > > > On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
> > > > > <Br...@comcast.com> wrote:
> > > > > >
> > > > > > +1.
> > > > > > If there's a simple way to work around duplicate origins being
> > > > > prohibited,
> > > > > > then we should rely on that instead of "enumerating all those
> > possible
> > > > > conflicting
> > > > > > settings, which are not only highly complex and confusing, but
> also
> > > > > further
> > > > > > entrench us in only supporting ATS as a caching proxy (hurting
> > efforts
> > > > to
> > > > > > integrate e.g. Grove, nginx etc.)
> > > > > > ________________________________________
> > > > > > From: Rawlin Peters <ra...@gmail.com>
> > > > > > Sent: Tuesday, December 18, 2018 2:20 PM
> > > > > > To: dev@trafficcontrol.apache.org
> > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > > > > Services produces indeterminate parent.config
> > > > > >
> > > > > > There are a number of different DS settings at play that can
> > > > > > potentially cause conflicts. The question is: do we want to get
> > into
> > > > > > the business of enumerating all those possible conflicting
> > settings or
> > > > > > just simply prohibit duplicate origins altogether? I think we can
> > dig
> > > > > > in and get that "sufficiently advanced sql query" to check for
> > > > > > conflicting origins, but is that something we want to carry along
> > for
> > > > > > the foreseeable future? Aren't CNAMEs relatively cheaper than
> > > > > > developing and maintaining that code and the mental overhead
> > required
> > > > > > in understanding why you're getting an error that says your
> > requested
> > > > > > DS would cause an origin conflict? I think at the point you've
> > > > > > requested a DS that would create a conflict, you've chosen those
> > > > > > settings for a reason and would probably prefer to just
> create/use
> > a
> > > > > > CNAME in your new DS and keep the rest of your settings the same.
> > > > > >
> > > > > > Thinking in terms of errors, I'm imagining:
> > > > > > "cannot create delivery service: origin fqdn 'foo.example.com'
> > already
> > > > > in use"
> > > > > > vs
> > > > > > "cannot create delivery service: origin fqdn 'foo.example.com'
> > already
> > > > > > in use as type DNS_LIVE_NATNL, which is incompatible with your
> > chosen
> > > > > > type of HTTP_NO_CACHE"
> > > > > >
> > > > > > At that point you'd probably say to yourself, "uh, I need
> > > > > > HTTP_NO_CACHE, so what am I supposed to do now?"
> > > > > >
> > > > > > As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
> > > > > > because the resulting code will be simpler, but I think
> eliminating
> > > > > > the mental overhead for operators could be worthwhile too. If we
> > can
> > > > > > agree on an end state of prohibiting duplicate origins
> altogether,
> > we
> > > > > > can start working on a design to smoothly transition us to that
> > point.
> > > > > > Are we willing to live with "just CNAME your origin fqdn" as the
> > > > > > standard solution to duplicates?
> > > > > >
> > > > > > - Rawlin
> > > > > >
> > > > > >
> > > > > > On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
> > > > > > <De...@comcast.com> wrote:
> > > > > > >
> > > > > > > The only situation in which they can share origins is if a) the
> > > > > origins are shared in an MSO configuration but still have different
> > > > defined
> > > > > origin fields in the delivery service, or if they're assigned to
> > > > completely
> > > > > different cachegroups. It's when two delivery services share the
> > same
> > > > > edges that there's an issue, because you end up with parent.config
> > > > issues.
> > > > > Actually you could even get away with it in mids as long as you
> > weren't
> > > > > doing anything like MSO to it.
> > > > > > >
> > > > > > > Could get messy real fast, though. Best to just create a
> second
> > > > FQDN.
> > > > > > >
> > > > > > > Derek
> > > > > > >
> > > > > > > On 12/18/18, 3:23 PM, "Fieck, Brennan" <
> > Brennan_Fieck@comcast.com>
> > > > > wrote:
> > > > > > >
> > > > > > > So no two Delivery Services may share an origin *regardless
> > of
> > > > > cache hierarchy* ? I've been told that DNS Delivery Services can
> > have the
> > > > > same origin as HTTP Delivery Services because they obey the same
> > cache
> > > > > hierarchy. You're saying that would still produce invalid output
> > and/or
> > > > is
> > > > > explicitly disallowed by ATS?
> > > > > > > ________________________________________
> > > > > > > From: Robert Butts <ro...@apache.org>
> > > > > > > Sent: Tuesday, December 18, 2018 1:09 PM
> > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > Delivery
> > > > > Services produces indeterminate parent.config
> > > > > > >
> > > > > > > >can you give an example of what parent.config looks like
> > when 2
> > > > > ds's share
> > > > > > > an origin and have different a different topology?
> > > > > > >
> > > > > > > Answering because I encountered this directly, when
> rewriting
> > > > > parent.config.
> > > > > > >
> > > > > > > For example: Suppose you have one Delivery Service:
> > > > > > > XML_ID: foo
> > > > > > > Type: HTPT_LIVE_NATL
> > > > > > > Query String Handling: 1 - ignore in cache key, and pass up
> > > > > > > Origin Server Base URL: http://foo.example.net
> > > > > > >
> > > > > > > And another Delivery Service:
> > > > > > > XML_ID: bar
> > > > > > > Type: HTPT_LIVE_NATL
> > > > > > > Query String Handling: 1 - ignore in cache key, and pass up
> > > > > > > Origin Server Base URL: http://foo.example.net
> > > > > > >
> > > > > > > ATS only supports unique `dest_domain` entries in
> > parent.config.
> > > > > Therefore,
> > > > > > > the parent.config generated for a server assigned to both
> of
> > > > these
> > > > > Delivery
> > > > > > > Services with either be:
> > > > > > >
> > > > > > > dest_domain=foo.example.net port=80 go_direct=true
> > > > > > >
> > > > > > > Or
> > > > > > >
> > > > > > > dest_domain=foo.example.net port=80 go_direct=false
> > > > > qstring=consider
> > > > > > >
> > > > > > > Right now, it's arbitrary which Perl Traffic Ops inserts,
> and
> > > > Perl
> > > > > provides
> > > > > > > no warning or error of any kind (the pending Go
> > parent.config PR
> > > > > logs an
> > > > > > > error).
> > > > > > >
> > > > > > > Whichever is arbitrarily inserted, the resulting remaps for
> > the
> > > > > other
> > > > > > > delivery service will be wrong. Either `foo` requests will
> > drop
> > > > > the query
> > > > > > > string when they shouldn't, and go to the mid when they
> > > > shouldn't;
> > > > > or `bar`
> > > > > > > requests will use the query string and skip the mid when it
> > > > > shouldn't.
> > > > > > >
> > > > > > >
> > > > > > > Does that make sense? The only correct solution, is to
> > somehow
> > > > > prevent
> > > > > > > different DSes having the same origin, and tell tenants
> they
> > must
> > > > > use
> > > > > > > CNAMEs if they need.
> > > > > > >
> > > > > > > This isn't a bug in Traffic Control. ATS fundamentally
> > doesn't
> > > > > support
> > > > > > > multiple remap rules with the same parent FQDN with
> different
> > > > > > > configurations. Hence, Traffic Control needs to prohibit
> > that.
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
> > > > > mitchell852@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > brennan,
> > > > > > > >
> > > > > > > > can you give an example of what parent.config looks like
> > when 2
> > > > > ds's share
> > > > > > > > an origin and have different a different topology?
> > > > > > > >
> > > > > > > > jeremy
> > > > > > > >
> > > > > > > > On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
> > > > > Brennan_Fieck@comcast.com
> > > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > To be clear, the "Warning" I'm talking about would
> > happen at
> > > > > startup, but
> > > > > > > > > I'd like a UI-only constraint to come with that to
> > disallow
> > > > > using the API
> > > > > > > > > to bind the same origin to multiple Delivery Services
> > with
> > > > > varying
> > > > > > > > > topography requirements. It wouldn't change the
> existing
> > > > data,
> > > > > but
> > > > > > > > prevent
> > > > > > > > > users from creating more bad data.
> > > > > > > > >
> > > > > > > > > "warning" doesn't really sufficiently describe that, my
> > bad.
> > > > > > > > > ________________________________________
> > > > > > > > > From: Fieck, Brennan <Br...@comcast.com>
> > > > > > > > > Sent: Tuesday, December 18, 2018 11:24 AM
> > > > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > > > > Delivery Services
> > > > > > > > > produces indeterminate parent.config
> > > > > > > > >
> > > > > > > > > Well the cost of fixing this bug is a constraint on the
> > data.
> > > > > Unless we
> > > > > > > > > make it a UI-only constraint - which I'm personally
> > against -
> > > > > there must
> > > > > > > > be
> > > > > > > > > some point in the future where ATC cannot reasonably be
> > > > > expected to work
> > > > > > > > > with data that violates that constraint. The question
> is
> > when
> > > > > that should
> > > > > > > > > occur, which should likely happen at a minor version
> > release.
> > > > > Minor not
> > > > > > > > > major because it doesn't involve a change in data
> > structures,
> > > > > merely
> > > > > > > > > relationships between them - in my opinion that's a
> minor
> > > > > version change
> > > > > > > > > but that's definitely up for debate. With several
> release
> > > > > candidates for
> > > > > > > > > 3.0.0 that _doesn't_ include this restriction already
> in
> > the
> > > > > wild, I
> > > > > > > > > wouldn't recommend putting it in there. That means to
> > fix the
> > > > > bug as soon
> > > > > > > > > as possible it should go in 3.1.0 which should be the
> > target
> > > > > of "master"
> > > > > > > > > after the 3.0.0 release is cut from it.
> > > > > > > > >
> > > > > > > > > So I'd recommend immediately implementing the
> constraint
> > in
> > > > > master with a
> > > > > > > > > refusal to upgrade with bad data, and backport a
> warning
> > > > about
> > > > > the future
> > > > > > > > > behavior into 3.0.0 or as part of a 3.0.1 provided we
> had
> > > > more
> > > > > changes
> > > > > > > > that
> > > > > > > > > would warrant a micro version bump.
> > > > > > > > > ________________________________________
> > > > > > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > > > > > Sent: Tuesday, December 18, 2018 9:34 AM
> > > > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > > > > Delivery Services
> > > > > > > > > produces indeterminate parent.config
> > > > > > > > >
> > > > > > > > > -1 Holding an ATC upgrade hostage to data cleanup seems
> > like
> > > > a
> > > > > bad idea.
> > > > > > > > > The issue isn't great, but it's also not new. We
> should
> > > > allow
> > > > > teams to
> > > > > > > > fix
> > > > > > > > > their data at their normal paces if it doesn't create
> > > > > significant
> > > > > > > > overhead
> > > > > > > > > or an inherant blocker for new functionality or
> > correction of
> > > > > other major
> > > > > > > > > problems imho.
> > > > > > > > >
> > > > > > > > > Jonathan G
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On 12/18/18, 9:28 AM, "Fieck, Brennan" <
> > > > > Brennan_Fieck@comcast.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Another option is we could detect collisions at
> > startup
> > > > > and simply
> > > > > > > > > refuse to continue with the upgrade until the data is
> > fixed.
> > > > > That would
> > > > > > > > > allow people using the now-unsupported data format to
> > > > continue
> > > > > to use
> > > > > > > > their
> > > > > > > > > old versions of Traffic Ops without wrecking their
> > database,
> > > > > but also
> > > > > > > > > provide an incentive to clean up the data.
> > > > > > > > > ________________________________________
> > > > > > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > > > > > Sent: Tuesday, December 18, 2018 5:12 AM
> > > > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to
> > Multipe
> > > > > Delivery
> > > > > > > > > Services produces indeterminate parent.config
> > > > > > > > >
> > > > > > > > > I'm generally a fan of constrain your data in your
> > > > > database, but not
> > > > > > > > > necessarily exclusively. I see this as a one-way
> > > > > cleanup/conversion so
> > > > > > > > it
> > > > > > > > > doesn't need to be configurable; otherwise you have to
> > ask
> > > > the
> > > > > question
> > > > > > > > > what happens if someone turns it off. That said,
> > something
> > > > in
> > > > > the UI
> > > > > > > > layer
> > > > > > > > > would be nice to prevent spending significant
> quantities
> > of
> > > > > time
> > > > > > > > building a
> > > > > > > > > complex DS only to have it fail to post for reasons
> that
> > > > could
> > > > > have been
> > > > > > > > > known earlier.
> > > > > > > > >
> > > > > > > > > The way my brain works in this case:
> > > > > > > > > If !unique_constraint_exists_query()
> > > > > > > > > If has_duplicates_query()
> > > > > > > > > show_warning()
> > > > > > > > > else
> > > > > > > > > add_unique_constraint()
> > > > > > > > >
> > > > > > > > > to which the API and UI configuration could also
> > make use
> > > > > of
> > > > > > > > > unique_constraint_exists_query() to drive additional
> > layer
> > > > > constraints if
> > > > > > > > > desired.
> > > > > > > > >
> > > > > > > > > Jonathan G
> > > > > > > > >
> > > > > > > > > On 12/17/18, 1:11 PM, "Rawlin Peters" <
> > > > > rawlin.peters@gmail.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > That is an interesting idea...detect at TO
> > startup
> > > > > whether or not
> > > > > > > > > there are duplicate origins and operate in a
> > "prevent
> > > > > duplicate
> > > > > > > > > origins" state if no duplicates are found or
> > "prevent
> > > > > conflicting
> > > > > > > > > DS
> > > > > > > > > topologies" state if duplicates are found? So
> > once
> > > > > operators have
> > > > > > > > > replaced all the duplicate origins with CNAMEs,
> > TO
> > > > will
> > > > > > > > essentially
> > > > > > > > > operate in a "prohibit all duplicate origins"
> > state.
> > > > > That would
> > > > > > > > > probably make for a simpler transition, but I'd
> > want
> > > > > to remove
> > > > > > > > that
> > > > > > > > > logic in a following release that strictly
> > prohibits
> > > > > duplicate
> > > > > > > > > origins
> > > > > > > > > (assuming that the community agrees we should
> > > > prohibit
> > > > > duplicate
> > > > > > > > > origins altogether).
> > > > > > > > >
> > > > > > > > > As for DB constraints vs UI, I was thinking
> those
> > > > > DS-type
> > > > > > > > > constraints
> > > > > > > > > I pointed out would live in the API. It would
> > > > > basically be added
> > > > > > > > > validation in the deliveryservices POST/PUT
> > endpoint
> > > > > that checks
> > > > > > > > > the
> > > > > > > > > DB for existing DSes that conflict with the
> > requested
> > > > > DS.
> > > > > > > > >
> > > > > > > > > - Rawlin
> > > > > > > > >
> > > > > > > > > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > > > > > > > > <Jo...@comcast.com> wrote:
> > > > > > > > > >
> > > > > > > > > > These kinds of conditions should be
> detectable
> > > > with a
> > > > > > > > > sufficiently advanced SQL query. Is it possible to add
> > the
> > > > > constraint if
> > > > > > > > > it passes and emit a warning during TO startup
> otherwise?
> > > > > That would let
> > > > > > > > > you know the condition exists at startup but not
> getting
> > in
> > > > > your way and
> > > > > > > > > keep you out of trouble once you've cleaned up. We
> made
> > a
> > > > > mistake early
> > > > > > > > > on, but this would acknowledge it was bad and encourage
> > it to
> > > > > be fixed at
> > > > > > > > > the speed of operations teams. Also this puts the
> > constraint
> > > > > in the
> > > > > > > > > database rather than the UI which is really where the
> > > > > contention is for
> > > > > > > > > usability.
> > > > > > > > > >
> > > > > > > > > > Jonathan G
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> > > > > > > > rawlin.peters@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > We occasionally discuss this issue but
> > haven't
> > > > > tackled it
> > > > > > > > > yet. I think
> > > > > > > > > > the main issue is just that duplicate
> > origins
> > > > > have been
> > > > > > > > > allowed since
> > > > > > > > > > the beginning, and now everyone's Traffic
> > Ops
> > > > > could be
> > > > > > > > > littered with
> > > > > > > > > > duplicate origins. Also, depending on the
> > > > config
> > > > > of the
> > > > > > > > > duplicate
> > > > > > > > > > delivery services, the origins might not
> > be in
> > > > > conflict at
> > > > > > > > > all (if
> > > > > > > > > > they don't have different topology
> > > > constraints).
> > > > > I would
> > > > > > > > > love for us
> > > > > > > > > > to just add a uniqueness constraint, but
> > there
> > > > > would need
> > > > > > > > to
> > > > > > > > > be a fair
> > > > > > > > > > amount of warning to the community before
> > doing
> > > > > so and
> > > > > > > > might
> > > > > > > > > > invalidate a significant amount of valid
> > use
> > > > > cases.
> > > > > > > > > Operators would
> > > > > > > > > > need time to make DNS CNAME records for
> the
> > > > > duplicate
> > > > > > > > > origins and
> > > > > > > > > > update their DSes to use the different
> > CNAMEs.
> > > > > > > > > >
> > > > > > > > > > I think as a good first step to
> > eliminating the
> > > > > use of
> > > > > > > > > duplicate
> > > > > > > > > > origins altogether, we should identify
> > which
> > > > > "topology
> > > > > > > > > constraints"
> > > > > > > > > > actually cause conflicting config when
> used
> > > > with
> > > > > duplicate
> > > > > > > > > origins and
> > > > > > > > > > prevent creating DSes with duplicate
> > origins
> > > > _if
> > > > > it would
> > > > > > > > > cause a
> > > > > > > > > > conflict with an existing DS that uses
> the
> > same
> > > > > origin_.
> > > > > > > > > >
> > > > > > > > > > For instance, I believe an HTTP and
> > DNS-type DS
> > > > > can live
> > > > > > > > > happily
> > > > > > > > > > side-by-side using the same origin
> > (probably
> > > > > need different
> > > > > > > > > > routing_names?), but scenarios like HTTP
> > and
> > > > > HTTP_LIVE, or
> > > > > > > > > DNS and
> > > > > > > > > > HTTP_NO_CACHE sharing the same origin
> will
> > > > cause
> > > > > conflicts
> > > > > > > > > for sure.
> > > > > > > > > > So maybe we can start by making sure the
> DS
> > > > > types "match"
> > > > > > > > > when using
> > > > > > > > > > the same origin:
> > > > > > > > > > HTTP + DNS: possibly good, if they have
> > > > > different routing
> > > > > > > > > names?
> > > > > > > > > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > > > > > > > > HTTP_NO_CACHE + [any other type]: bad
> > > > > > > > > > HTTP_LIVE + HTTP: bad
> > > > > > > > > > etc.
> > > > > > > > > >
> > > > > > > > > > There are most likely other conflict
> > scenarios
> > > > > that don't
> > > > > > > > > involve the
> > > > > > > > > > DS types, but I think this would be a
> good
> > > > > start. In the
> > > > > > > > > future with
> > > > > > > > > > Delivery Service Topologies (aka Flexible
> > > > > Cachegroups aka
> > > > > > > > > Bring Your
> > > > > > > > > > Own Topology), we might be able to
> prohibit
> > > > > assigning a DS
> > > > > > > > > to a
> > > > > > > > > > Topology if the DS's origin is already
> > used by
> > > > > another DS
> > > > > > > > in
> > > > > > > > > a
> > > > > > > > > > different Topology.
> > > > > > > > > >
> > > > > > > > > > - Rawlin
> > > > > > > > > >
> > > > > > > > > > On Mon, Dec 17, 2018 at 10:52 AM Fieck,
> > Brennan
> > > > > > > > > > <Br...@comcast.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > As some of you may be aware,
> > `parent.config`
> > > > > files
> > > > > > > > > generated by Traffic Ops can vary wildly when an origin
> > is
> > > > > assigned to
> > > > > > > > > multiple Delivery Services. This results in undefined
> > > > > behavior. I'm told
> > > > > > > > > that the conflict only happens when two Delivery
> Services
> > > > with
> > > > > different
> > > > > > > > > "topology requirements" use the same origin, whatever
> > that
> > > > > means (content
> > > > > > > > > routing type?). Regardless, the issue should be
> > addressed.
> > > > The
> > > > > obvious
> > > > > > > > > solution is to put in place a database constraint that
> > > > > prevents an origin
> > > > > > > > > from being assigned to more that one Delivery Service
> > with
> > > > API
> > > > > checks in
> > > > > > > > > place that would provide helpful error messages when an
> > > > > attempt is made
> > > > > > > > to
> > > > > > > > > violate the constraint. However, would that mess with
> > things
> > > > > like
> > > > > > > > > Multi-Site Origin? Or is it just not viable for some
> > other
> > > > > reason? If it
> > > > > > > > is
> > > > > > > > > a good solution, I'm prepared to work on a fix that
> > utilizes
> > > > > it.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > >
> > > >
> >
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Dave Neuman <ne...@apache.org>.
-1 on the one DS to Origin limitation. I think there are legit use cases
for it.
I would be +1 on the idea if we include type though. Two DSs can share the
same origin as long as they are of the same type (DNS, HTTP, HTTP_LIVE),
etc. You shouldn't be able to have two DSs with the same origin but
different types.
On Mon, Jan 14, 2019 at 7:53 AM Fieck, Brennan <Br...@comcast.com>
wrote:
> I don't see this as a complicated use case or a barrier to entry, an
> extremely basic introduction to ATC would only make use of one DS
> (shameless CDN-in-a-Box plug).
> It seems far more complicated for a user to be using many DSes to serve a
> single origin - we can easily make it clear in the docs that the two have a
> 1:1 relationship, with a footnote about CNAME records for users with the
> knowledge, resources, and need for such workarounds.
> ________________________________________
> From: Mark Torluemke <mt...@apache.org>
> Sent: Friday, January 11, 2019 1:21 PM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> produces indeterminate parent.config
>
> >> 1. Prohibit creating new delivery services that would share an
> existing origin and prohibit updating a delivery service to a shared
> origin
> In case my position has been lost, I'm still -1 on this. :) IMO, we
> shouldn't let the few complicated use cases increase the barrier to entry
> (to using ATC) for the masses.
>
>
>
>
> On Fri, Jan 11, 2019 at 11:53 AM Rawlin Peters <ra...@gmail.com>
> wrote:
>
> > Alright, I'm trying to sum up this discussion so far since it seems
> > like everyone went on vacation and didn't really get a chance to wrap
> > this one up:
> > - duplicate origins cause undefined behavior
> > - we need a way to migrate to a future that is free of duplicate
> > origins in Traffic Control
> > - we need a visual and easy way to determine if Traffic Ops currently
> > contains duplicate origins, so that operators are incentivized to fix
> > them rather than let it slide indefinitely
> > - operators should have a fair amount of time to fix their duplicate
> > origins
> >
> > I believe this is what we've mostly agreed upon but haven't clearly voted
> > on:
> >
> > In release N we will:
> > 1. Prohibit creating new delivery services that would share an
> > existing origin and prohibit updating a delivery service to a shared
> > origin
> > 2. Add some kind of visual indicator that duplicate origins are a
> > problem that need to be fixed before release N+1; otherwise, an
> > upgrade to N+1 will be prohibited.
> >
> > In release N+1 we will:
> > 3. Include a DB migration that adds a uniqueness constraint on origin
> > FQDN, removing the API-level checks for that.
> > 4. Prevent an upgrade to N+1 if duplicate origins are found (this
> > might occur as a byproduct of step 3).
> >
> > I am +1 on this plan and believe this would hit on all the summarized
> > points above. Please provide a clear vote on this plan so that we can
> > dive deeper in the details (i.e. what release 'N' is, the best visual
> > indicator for step 2, and a friendly way to handle step 4).
> >
> > Thanks,
> > Rawlin
> >
> > On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <mi...@gmail.com>
> > wrote:
> > >
> > > Not sure TP is the right place for a warning like "clean up this
> > > 'duplicate' origin or your next upgrade will fail". Most users of our
> > > system will be like "not my problem".
> > >
> > > On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <
> > Brennan_Fieck@comcast.com>
> > > wrote:
> > >
> > > > Probably. It would impact load times by a bit, but the page for an
> > > > individual object is not our bottleneck.
> > > > ________________________________________
> > > > From: Robert Butts <ro...@apache.org>
> > > > Sent: Wednesday, December 19, 2018 11:50 AM
> > > > To: dev@trafficcontrol.apache.org
> > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > Services
> > > > produces indeterminate parent.config
> > > >
> > > > > - Including a warning on startup and an API constraint preventing
> > adding
> > > > more bad data in the next 3.0.0 Release Candidate
> > > > > - Adding a database constraint immediately into master that won't
> be
> > > > cherry-picked into 3.0.0 but should be included in 3.1.0
> > > >
> > > > +1
> > > >
> > > > I understand Jonathan's objection, but at some point, we have to be
> > able to
> > > > move forward. This is a good compromise: deprecate, then remove. That
> > gives
> > > > people a full major version to fix their data.
> > > >
> > > > I would be ideal if it were more than just a logged warning, though.
> > Can we
> > > > add a big red banner in Traffic Portal, on the Delivery Service page
> > for
> > > > any DS with a duplicate origin, telling users to fix it, and that
> they
> > > > won't be able to upgrade to the next major version until it's fixed?
> > > >
> > > >
> > > > On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <
> > Brennan_Fieck@comcast.com
> > > > >
> > > > wrote:
> > > >
> > > > > So it seems like nobody has a problem with the "how" - disallowing
> > > > > duplicate origin FQDNs on Delivery Services - but we never reached
> a
> > > > > consensus on "when".
> > > > >
> > > > > I stand by my previous proposal:
> > > > > - Including a warning on startup and an API constraint preventing
> > adding
> > > > > more bad data in the next 3.0.0 Release Candidate
> > > > > - Adding a database constraint immediately into master that won't
> be
> > > > > cherry-picked into 3.0.0 but should be included in 3.1.0
> > > > > ________________________________________
> > > > > From: Rawlin Peters <ra...@gmail.com>
> > > > > Sent: Tuesday, December 18, 2018 4:59 PM
> > > > > To: dev@trafficcontrol.apache.org
> > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > Services
> > > > > produces indeterminate parent.config
> > > > >
> > > > > Also, building more around DS types will make it even harder to get
> > > > > away from DS types in the future too, which I know is something
> we've
> > > > > discussed on this mailing list before. It also adds to the overhead
> > of
> > > > > Delivery Service Topologies, since a lot of the DS types won't
> > > > > carryover into that world.
> > > > >
> > > > > - Rawlin
> > > > >
> > > > > On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
> > > > > <Br...@comcast.com> wrote:
> > > > > >
> > > > > > +1.
> > > > > > If there's a simple way to work around duplicate origins being
> > > > > prohibited,
> > > > > > then we should rely on that instead of "enumerating all those
> > possible
> > > > > conflicting
> > > > > > settings, which are not only highly complex and confusing, but
> also
> > > > > further
> > > > > > entrench us in only supporting ATS as a caching proxy (hurting
> > efforts
> > > > to
> > > > > > integrate e.g. Grove, nginx etc.)
> > > > > > ________________________________________
> > > > > > From: Rawlin Peters <ra...@gmail.com>
> > > > > > Sent: Tuesday, December 18, 2018 2:20 PM
> > > > > > To: dev@trafficcontrol.apache.org
> > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > > > > Services produces indeterminate parent.config
> > > > > >
> > > > > > There are a number of different DS settings at play that can
> > > > > > potentially cause conflicts. The question is: do we want to get
> > into
> > > > > > the business of enumerating all those possible conflicting
> > settings or
> > > > > > just simply prohibit duplicate origins altogether? I think we can
> > dig
> > > > > > in and get that "sufficiently advanced sql query" to check for
> > > > > > conflicting origins, but is that something we want to carry along
> > for
> > > > > > the foreseeable future? Aren't CNAMEs relatively cheaper than
> > > > > > developing and maintaining that code and the mental overhead
> > required
> > > > > > in understanding why you're getting an error that says your
> > requested
> > > > > > DS would cause an origin conflict? I think at the point you've
> > > > > > requested a DS that would create a conflict, you've chosen those
> > > > > > settings for a reason and would probably prefer to just
> create/use
> > a
> > > > > > CNAME in your new DS and keep the rest of your settings the same.
> > > > > >
> > > > > > Thinking in terms of errors, I'm imagining:
> > > > > > "cannot create delivery service: origin fqdn 'foo.example.com'
> > already
> > > > > in use"
> > > > > > vs
> > > > > > "cannot create delivery service: origin fqdn 'foo.example.com'
> > already
> > > > > > in use as type DNS_LIVE_NATNL, which is incompatible with your
> > chosen
> > > > > > type of HTTP_NO_CACHE"
> > > > > >
> > > > > > At that point you'd probably say to yourself, "uh, I need
> > > > > > HTTP_NO_CACHE, so what am I supposed to do now?"
> > > > > >
> > > > > > As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
> > > > > > because the resulting code will be simpler, but I think
> eliminating
> > > > > > the mental overhead for operators could be worthwhile too. If we
> > can
> > > > > > agree on an end state of prohibiting duplicate origins
> altogether,
> > we
> > > > > > can start working on a design to smoothly transition us to that
> > point.
> > > > > > Are we willing to live with "just CNAME your origin fqdn" as the
> > > > > > standard solution to duplicates?
> > > > > >
> > > > > > - Rawlin
> > > > > >
> > > > > >
> > > > > > On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
> > > > > > <De...@comcast.com> wrote:
> > > > > > >
> > > > > > > The only situation in which they can share origins is if a) the
> > > > > origins are shared in an MSO configuration but still have different
> > > > defined
> > > > > origin fields in the delivery service, or if they're assigned to
> > > > completely
> > > > > different cachegroups. It's when two delivery services share the
> > same
> > > > > edges that there's an issue, because you end up with parent.config
> > > > issues.
> > > > > Actually you could even get away with it in mids as long as you
> > weren't
> > > > > doing anything like MSO to it.
> > > > > > >
> > > > > > > Could get messy real fast, though. Best to just create a
> second
> > > > FQDN.
> > > > > > >
> > > > > > > Derek
> > > > > > >
> > > > > > > On 12/18/18, 3:23 PM, "Fieck, Brennan" <
> > Brennan_Fieck@comcast.com>
> > > > > wrote:
> > > > > > >
> > > > > > > So no two Delivery Services may share an origin *regardless
> > of
> > > > > cache hierarchy* ? I've been told that DNS Delivery Services can
> > have the
> > > > > same origin as HTTP Delivery Services because they obey the same
> > cache
> > > > > hierarchy. You're saying that would still produce invalid output
> > and/or
> > > > is
> > > > > explicitly disallowed by ATS?
> > > > > > > ________________________________________
> > > > > > > From: Robert Butts <ro...@apache.org>
> > > > > > > Sent: Tuesday, December 18, 2018 1:09 PM
> > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > Delivery
> > > > > Services produces indeterminate parent.config
> > > > > > >
> > > > > > > >can you give an example of what parent.config looks like
> > when 2
> > > > > ds's share
> > > > > > > an origin and have different a different topology?
> > > > > > >
> > > > > > > Answering because I encountered this directly, when
> rewriting
> > > > > parent.config.
> > > > > > >
> > > > > > > For example: Suppose you have one Delivery Service:
> > > > > > > XML_ID: foo
> > > > > > > Type: HTPT_LIVE_NATL
> > > > > > > Query String Handling: 1 - ignore in cache key, and pass up
> > > > > > > Origin Server Base URL: http://foo.example.net
> > > > > > >
> > > > > > > And another Delivery Service:
> > > > > > > XML_ID: bar
> > > > > > > Type: HTPT_LIVE_NATL
> > > > > > > Query String Handling: 1 - ignore in cache key, and pass up
> > > > > > > Origin Server Base URL: http://foo.example.net
> > > > > > >
> > > > > > > ATS only supports unique `dest_domain` entries in
> > parent.config.
> > > > > Therefore,
> > > > > > > the parent.config generated for a server assigned to both
> of
> > > > these
> > > > > Delivery
> > > > > > > Services with either be:
> > > > > > >
> > > > > > > dest_domain=foo.example.net port=80 go_direct=true
> > > > > > >
> > > > > > > Or
> > > > > > >
> > > > > > > dest_domain=foo.example.net port=80 go_direct=false
> > > > > qstring=consider
> > > > > > >
> > > > > > > Right now, it's arbitrary which Perl Traffic Ops inserts,
> and
> > > > Perl
> > > > > provides
> > > > > > > no warning or error of any kind (the pending Go
> > parent.config PR
> > > > > logs an
> > > > > > > error).
> > > > > > >
> > > > > > > Whichever is arbitrarily inserted, the resulting remaps for
> > the
> > > > > other
> > > > > > > delivery service will be wrong. Either `foo` requests will
> > drop
> > > > > the query
> > > > > > > string when they shouldn't, and go to the mid when they
> > > > shouldn't;
> > > > > or `bar`
> > > > > > > requests will use the query string and skip the mid when it
> > > > > shouldn't.
> > > > > > >
> > > > > > >
> > > > > > > Does that make sense? The only correct solution, is to
> > somehow
> > > > > prevent
> > > > > > > different DSes having the same origin, and tell tenants
> they
> > must
> > > > > use
> > > > > > > CNAMEs if they need.
> > > > > > >
> > > > > > > This isn't a bug in Traffic Control. ATS fundamentally
> > doesn't
> > > > > support
> > > > > > > multiple remap rules with the same parent FQDN with
> different
> > > > > > > configurations. Hence, Traffic Control needs to prohibit
> > that.
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
> > > > > mitchell852@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > brennan,
> > > > > > > >
> > > > > > > > can you give an example of what parent.config looks like
> > when 2
> > > > > ds's share
> > > > > > > > an origin and have different a different topology?
> > > > > > > >
> > > > > > > > jeremy
> > > > > > > >
> > > > > > > > On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
> > > > > Brennan_Fieck@comcast.com
> > > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > To be clear, the "Warning" I'm talking about would
> > happen at
> > > > > startup, but
> > > > > > > > > I'd like a UI-only constraint to come with that to
> > disallow
> > > > > using the API
> > > > > > > > > to bind the same origin to multiple Delivery Services
> > with
> > > > > varying
> > > > > > > > > topography requirements. It wouldn't change the
> existing
> > > > data,
> > > > > but
> > > > > > > > prevent
> > > > > > > > > users from creating more bad data.
> > > > > > > > >
> > > > > > > > > "warning" doesn't really sufficiently describe that, my
> > bad.
> > > > > > > > > ________________________________________
> > > > > > > > > From: Fieck, Brennan <Br...@comcast.com>
> > > > > > > > > Sent: Tuesday, December 18, 2018 11:24 AM
> > > > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > > > > Delivery Services
> > > > > > > > > produces indeterminate parent.config
> > > > > > > > >
> > > > > > > > > Well the cost of fixing this bug is a constraint on the
> > data.
> > > > > Unless we
> > > > > > > > > make it a UI-only constraint - which I'm personally
> > against -
> > > > > there must
> > > > > > > > be
> > > > > > > > > some point in the future where ATC cannot reasonably be
> > > > > expected to work
> > > > > > > > > with data that violates that constraint. The question
> is
> > when
> > > > > that should
> > > > > > > > > occur, which should likely happen at a minor version
> > release.
> > > > > Minor not
> > > > > > > > > major because it doesn't involve a change in data
> > structures,
> > > > > merely
> > > > > > > > > relationships between them - in my opinion that's a
> minor
> > > > > version change
> > > > > > > > > but that's definitely up for debate. With several
> release
> > > > > candidates for
> > > > > > > > > 3.0.0 that _doesn't_ include this restriction already
> in
> > the
> > > > > wild, I
> > > > > > > > > wouldn't recommend putting it in there. That means to
> > fix the
> > > > > bug as soon
> > > > > > > > > as possible it should go in 3.1.0 which should be the
> > target
> > > > > of "master"
> > > > > > > > > after the 3.0.0 release is cut from it.
> > > > > > > > >
> > > > > > > > > So I'd recommend immediately implementing the
> constraint
> > in
> > > > > master with a
> > > > > > > > > refusal to upgrade with bad data, and backport a
> warning
> > > > about
> > > > > the future
> > > > > > > > > behavior into 3.0.0 or as part of a 3.0.1 provided we
> had
> > > > more
> > > > > changes
> > > > > > > > that
> > > > > > > > > would warrant a micro version bump.
> > > > > > > > > ________________________________________
> > > > > > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > > > > > Sent: Tuesday, December 18, 2018 9:34 AM
> > > > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > > > > Delivery Services
> > > > > > > > > produces indeterminate parent.config
> > > > > > > > >
> > > > > > > > > -1 Holding an ATC upgrade hostage to data cleanup seems
> > like
> > > > a
> > > > > bad idea.
> > > > > > > > > The issue isn't great, but it's also not new. We
> should
> > > > allow
> > > > > teams to
> > > > > > > > fix
> > > > > > > > > their data at their normal paces if it doesn't create
> > > > > significant
> > > > > > > > overhead
> > > > > > > > > or an inherant blocker for new functionality or
> > correction of
> > > > > other major
> > > > > > > > > problems imho.
> > > > > > > > >
> > > > > > > > > Jonathan G
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On 12/18/18, 9:28 AM, "Fieck, Brennan" <
> > > > > Brennan_Fieck@comcast.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Another option is we could detect collisions at
> > startup
> > > > > and simply
> > > > > > > > > refuse to continue with the upgrade until the data is
> > fixed.
> > > > > That would
> > > > > > > > > allow people using the now-unsupported data format to
> > > > continue
> > > > > to use
> > > > > > > > their
> > > > > > > > > old versions of Traffic Ops without wrecking their
> > database,
> > > > > but also
> > > > > > > > > provide an incentive to clean up the data.
> > > > > > > > > ________________________________________
> > > > > > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > > > > > Sent: Tuesday, December 18, 2018 5:12 AM
> > > > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to
> > Multipe
> > > > > Delivery
> > > > > > > > > Services produces indeterminate parent.config
> > > > > > > > >
> > > > > > > > > I'm generally a fan of constrain your data in your
> > > > > database, but not
> > > > > > > > > necessarily exclusively. I see this as a one-way
> > > > > cleanup/conversion so
> > > > > > > > it
> > > > > > > > > doesn't need to be configurable; otherwise you have to
> > ask
> > > > the
> > > > > question
> > > > > > > > > what happens if someone turns it off. That said,
> > something
> > > > in
> > > > > the UI
> > > > > > > > layer
> > > > > > > > > would be nice to prevent spending significant
> quantities
> > of
> > > > > time
> > > > > > > > building a
> > > > > > > > > complex DS only to have it fail to post for reasons
> that
> > > > could
> > > > > have been
> > > > > > > > > known earlier.
> > > > > > > > >
> > > > > > > > > The way my brain works in this case:
> > > > > > > > > If !unique_constraint_exists_query()
> > > > > > > > > If has_duplicates_query()
> > > > > > > > > show_warning()
> > > > > > > > > else
> > > > > > > > > add_unique_constraint()
> > > > > > > > >
> > > > > > > > > to which the API and UI configuration could also
> > make use
> > > > > of
> > > > > > > > > unique_constraint_exists_query() to drive additional
> > layer
> > > > > constraints if
> > > > > > > > > desired.
> > > > > > > > >
> > > > > > > > > Jonathan G
> > > > > > > > >
> > > > > > > > > On 12/17/18, 1:11 PM, "Rawlin Peters" <
> > > > > rawlin.peters@gmail.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > That is an interesting idea...detect at TO
> > startup
> > > > > whether or not
> > > > > > > > > there are duplicate origins and operate in a
> > "prevent
> > > > > duplicate
> > > > > > > > > origins" state if no duplicates are found or
> > "prevent
> > > > > conflicting
> > > > > > > > > DS
> > > > > > > > > topologies" state if duplicates are found? So
> > once
> > > > > operators have
> > > > > > > > > replaced all the duplicate origins with CNAMEs,
> > TO
> > > > will
> > > > > > > > essentially
> > > > > > > > > operate in a "prohibit all duplicate origins"
> > state.
> > > > > That would
> > > > > > > > > probably make for a simpler transition, but I'd
> > want
> > > > > to remove
> > > > > > > > that
> > > > > > > > > logic in a following release that strictly
> > prohibits
> > > > > duplicate
> > > > > > > > > origins
> > > > > > > > > (assuming that the community agrees we should
> > > > prohibit
> > > > > duplicate
> > > > > > > > > origins altogether).
> > > > > > > > >
> > > > > > > > > As for DB constraints vs UI, I was thinking
> those
> > > > > DS-type
> > > > > > > > > constraints
> > > > > > > > > I pointed out would live in the API. It would
> > > > > basically be added
> > > > > > > > > validation in the deliveryservices POST/PUT
> > endpoint
> > > > > that checks
> > > > > > > > > the
> > > > > > > > > DB for existing DSes that conflict with the
> > requested
> > > > > DS.
> > > > > > > > >
> > > > > > > > > - Rawlin
> > > > > > > > >
> > > > > > > > > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > > > > > > > > <Jo...@comcast.com> wrote:
> > > > > > > > > >
> > > > > > > > > > These kinds of conditions should be
> detectable
> > > > with a
> > > > > > > > > sufficiently advanced SQL query. Is it possible to add
> > the
> > > > > constraint if
> > > > > > > > > it passes and emit a warning during TO startup
> otherwise?
> > > > > That would let
> > > > > > > > > you know the condition exists at startup but not
> getting
> > in
> > > > > your way and
> > > > > > > > > keep you out of trouble once you've cleaned up. We
> made
> > a
> > > > > mistake early
> > > > > > > > > on, but this would acknowledge it was bad and encourage
> > it to
> > > > > be fixed at
> > > > > > > > > the speed of operations teams. Also this puts the
> > constraint
> > > > > in the
> > > > > > > > > database rather than the UI which is really where the
> > > > > contention is for
> > > > > > > > > usability.
> > > > > > > > > >
> > > > > > > > > > Jonathan G
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> > > > > > > > rawlin.peters@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > We occasionally discuss this issue but
> > haven't
> > > > > tackled it
> > > > > > > > > yet. I think
> > > > > > > > > > the main issue is just that duplicate
> > origins
> > > > > have been
> > > > > > > > > allowed since
> > > > > > > > > > the beginning, and now everyone's Traffic
> > Ops
> > > > > could be
> > > > > > > > > littered with
> > > > > > > > > > duplicate origins. Also, depending on the
> > > > config
> > > > > of the
> > > > > > > > > duplicate
> > > > > > > > > > delivery services, the origins might not
> > be in
> > > > > conflict at
> > > > > > > > > all (if
> > > > > > > > > > they don't have different topology
> > > > constraints).
> > > > > I would
> > > > > > > > > love for us
> > > > > > > > > > to just add a uniqueness constraint, but
> > there
> > > > > would need
> > > > > > > > to
> > > > > > > > > be a fair
> > > > > > > > > > amount of warning to the community before
> > doing
> > > > > so and
> > > > > > > > might
> > > > > > > > > > invalidate a significant amount of valid
> > use
> > > > > cases.
> > > > > > > > > Operators would
> > > > > > > > > > need time to make DNS CNAME records for
> the
> > > > > duplicate
> > > > > > > > > origins and
> > > > > > > > > > update their DSes to use the different
> > CNAMEs.
> > > > > > > > > >
> > > > > > > > > > I think as a good first step to
> > eliminating the
> > > > > use of
> > > > > > > > > duplicate
> > > > > > > > > > origins altogether, we should identify
> > which
> > > > > "topology
> > > > > > > > > constraints"
> > > > > > > > > > actually cause conflicting config when
> used
> > > > with
> > > > > duplicate
> > > > > > > > > origins and
> > > > > > > > > > prevent creating DSes with duplicate
> > origins
> > > > _if
> > > > > it would
> > > > > > > > > cause a
> > > > > > > > > > conflict with an existing DS that uses
> the
> > same
> > > > > origin_.
> > > > > > > > > >
> > > > > > > > > > For instance, I believe an HTTP and
> > DNS-type DS
> > > > > can live
> > > > > > > > > happily
> > > > > > > > > > side-by-side using the same origin
> > (probably
> > > > > need different
> > > > > > > > > > routing_names?), but scenarios like HTTP
> > and
> > > > > HTTP_LIVE, or
> > > > > > > > > DNS and
> > > > > > > > > > HTTP_NO_CACHE sharing the same origin
> will
> > > > cause
> > > > > conflicts
> > > > > > > > > for sure.
> > > > > > > > > > So maybe we can start by making sure the
> DS
> > > > > types "match"
> > > > > > > > > when using
> > > > > > > > > > the same origin:
> > > > > > > > > > HTTP + DNS: possibly good, if they have
> > > > > different routing
> > > > > > > > > names?
> > > > > > > > > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > > > > > > > > HTTP_NO_CACHE + [any other type]: bad
> > > > > > > > > > HTTP_LIVE + HTTP: bad
> > > > > > > > > > etc.
> > > > > > > > > >
> > > > > > > > > > There are most likely other conflict
> > scenarios
> > > > > that don't
> > > > > > > > > involve the
> > > > > > > > > > DS types, but I think this would be a
> good
> > > > > start. In the
> > > > > > > > > future with
> > > > > > > > > > Delivery Service Topologies (aka Flexible
> > > > > Cachegroups aka
> > > > > > > > > Bring Your
> > > > > > > > > > Own Topology), we might be able to
> prohibit
> > > > > assigning a DS
> > > > > > > > > to a
> > > > > > > > > > Topology if the DS's origin is already
> > used by
> > > > > another DS
> > > > > > > > in
> > > > > > > > > a
> > > > > > > > > > different Topology.
> > > > > > > > > >
> > > > > > > > > > - Rawlin
> > > > > > > > > >
> > > > > > > > > > On Mon, Dec 17, 2018 at 10:52 AM Fieck,
> > Brennan
> > > > > > > > > > <Br...@comcast.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > As some of you may be aware,
> > `parent.config`
> > > > > files
> > > > > > > > > generated by Traffic Ops can vary wildly when an origin
> > is
> > > > > assigned to
> > > > > > > > > multiple Delivery Services. This results in undefined
> > > > > behavior. I'm told
> > > > > > > > > that the conflict only happens when two Delivery
> Services
> > > > with
> > > > > different
> > > > > > > > > "topology requirements" use the same origin, whatever
> > that
> > > > > means (content
> > > > > > > > > routing type?). Regardless, the issue should be
> > addressed.
> > > > The
> > > > > obvious
> > > > > > > > > solution is to put in place a database constraint that
> > > > > prevents an origin
> > > > > > > > > from being assigned to more that one Delivery Service
> > with
> > > > API
> > > > > checks in
> > > > > > > > > place that would provide helpful error messages when an
> > > > > attempt is made
> > > > > > > > to
> > > > > > > > > violate the constraint. However, would that mess with
> > things
> > > > > like
> > > > > > > > > Multi-Site Origin? Or is it just not viable for some
> > other
> > > > > reason? If it
> > > > > > > > is
> > > > > > > > > a good solution, I'm prepared to work on a fix that
> > utilizes
> > > > > it.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > >
> > > >
> >
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Fieck, Brennan" <Br...@comcast.com>.
I don't see this as a complicated use case or a barrier to entry, an extremely basic introduction to ATC would only make use of one DS (shameless CDN-in-a-Box plug).
It seems far more complicated for a user to be using many DSes to serve a single origin - we can easily make it clear in the docs that the two have a 1:1 relationship, with a footnote about CNAME records for users with the knowledge, resources, and need for such workarounds.
________________________________________
From: Mark Torluemke <mt...@apache.org>
Sent: Friday, January 11, 2019 1:21 PM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
>> 1. Prohibit creating new delivery services that would share an
existing origin and prohibit updating a delivery service to a shared
origin
In case my position has been lost, I'm still -1 on this. :) IMO, we
shouldn't let the few complicated use cases increase the barrier to entry
(to using ATC) for the masses.
On Fri, Jan 11, 2019 at 11:53 AM Rawlin Peters <ra...@gmail.com>
wrote:
> Alright, I'm trying to sum up this discussion so far since it seems
> like everyone went on vacation and didn't really get a chance to wrap
> this one up:
> - duplicate origins cause undefined behavior
> - we need a way to migrate to a future that is free of duplicate
> origins in Traffic Control
> - we need a visual and easy way to determine if Traffic Ops currently
> contains duplicate origins, so that operators are incentivized to fix
> them rather than let it slide indefinitely
> - operators should have a fair amount of time to fix their duplicate
> origins
>
> I believe this is what we've mostly agreed upon but haven't clearly voted
> on:
>
> In release N we will:
> 1. Prohibit creating new delivery services that would share an
> existing origin and prohibit updating a delivery service to a shared
> origin
> 2. Add some kind of visual indicator that duplicate origins are a
> problem that need to be fixed before release N+1; otherwise, an
> upgrade to N+1 will be prohibited.
>
> In release N+1 we will:
> 3. Include a DB migration that adds a uniqueness constraint on origin
> FQDN, removing the API-level checks for that.
> 4. Prevent an upgrade to N+1 if duplicate origins are found (this
> might occur as a byproduct of step 3).
>
> I am +1 on this plan and believe this would hit on all the summarized
> points above. Please provide a clear vote on this plan so that we can
> dive deeper in the details (i.e. what release 'N' is, the best visual
> indicator for step 2, and a friendly way to handle step 4).
>
> Thanks,
> Rawlin
>
> On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <mi...@gmail.com>
> wrote:
> >
> > Not sure TP is the right place for a warning like "clean up this
> > 'duplicate' origin or your next upgrade will fail". Most users of our
> > system will be like "not my problem".
> >
> > On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <
> Brennan_Fieck@comcast.com>
> > wrote:
> >
> > > Probably. It would impact load times by a bit, but the page for an
> > > individual object is not our bottleneck.
> > > ________________________________________
> > > From: Robert Butts <ro...@apache.org>
> > > Sent: Wednesday, December 19, 2018 11:50 AM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> Services
> > > produces indeterminate parent.config
> > >
> > > > - Including a warning on startup and an API constraint preventing
> adding
> > > more bad data in the next 3.0.0 Release Candidate
> > > > - Adding a database constraint immediately into master that won't be
> > > cherry-picked into 3.0.0 but should be included in 3.1.0
> > >
> > > +1
> > >
> > > I understand Jonathan's objection, but at some point, we have to be
> able to
> > > move forward. This is a good compromise: deprecate, then remove. That
> gives
> > > people a full major version to fix their data.
> > >
> > > I would be ideal if it were more than just a logged warning, though.
> Can we
> > > add a big red banner in Traffic Portal, on the Delivery Service page
> for
> > > any DS with a duplicate origin, telling users to fix it, and that they
> > > won't be able to upgrade to the next major version until it's fixed?
> > >
> > >
> > > On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <
> Brennan_Fieck@comcast.com
> > > >
> > > wrote:
> > >
> > > > So it seems like nobody has a problem with the "how" - disallowing
> > > > duplicate origin FQDNs on Delivery Services - but we never reached a
> > > > consensus on "when".
> > > >
> > > > I stand by my previous proposal:
> > > > - Including a warning on startup and an API constraint preventing
> adding
> > > > more bad data in the next 3.0.0 Release Candidate
> > > > - Adding a database constraint immediately into master that won't be
> > > > cherry-picked into 3.0.0 but should be included in 3.1.0
> > > > ________________________________________
> > > > From: Rawlin Peters <ra...@gmail.com>
> > > > Sent: Tuesday, December 18, 2018 4:59 PM
> > > > To: dev@trafficcontrol.apache.org
> > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> Services
> > > > produces indeterminate parent.config
> > > >
> > > > Also, building more around DS types will make it even harder to get
> > > > away from DS types in the future too, which I know is something we've
> > > > discussed on this mailing list before. It also adds to the overhead
> of
> > > > Delivery Service Topologies, since a lot of the DS types won't
> > > > carryover into that world.
> > > >
> > > > - Rawlin
> > > >
> > > > On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
> > > > <Br...@comcast.com> wrote:
> > > > >
> > > > > +1.
> > > > > If there's a simple way to work around duplicate origins being
> > > > prohibited,
> > > > > then we should rely on that instead of "enumerating all those
> possible
> > > > conflicting
> > > > > settings, which are not only highly complex and confusing, but also
> > > > further
> > > > > entrench us in only supporting ATS as a caching proxy (hurting
> efforts
> > > to
> > > > > integrate e.g. Grove, nginx etc.)
> > > > > ________________________________________
> > > > > From: Rawlin Peters <ra...@gmail.com>
> > > > > Sent: Tuesday, December 18, 2018 2:20 PM
> > > > > To: dev@trafficcontrol.apache.org
> > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > > > Services produces indeterminate parent.config
> > > > >
> > > > > There are a number of different DS settings at play that can
> > > > > potentially cause conflicts. The question is: do we want to get
> into
> > > > > the business of enumerating all those possible conflicting
> settings or
> > > > > just simply prohibit duplicate origins altogether? I think we can
> dig
> > > > > in and get that "sufficiently advanced sql query" to check for
> > > > > conflicting origins, but is that something we want to carry along
> for
> > > > > the foreseeable future? Aren't CNAMEs relatively cheaper than
> > > > > developing and maintaining that code and the mental overhead
> required
> > > > > in understanding why you're getting an error that says your
> requested
> > > > > DS would cause an origin conflict? I think at the point you've
> > > > > requested a DS that would create a conflict, you've chosen those
> > > > > settings for a reason and would probably prefer to just create/use
> a
> > > > > CNAME in your new DS and keep the rest of your settings the same.
> > > > >
> > > > > Thinking in terms of errors, I'm imagining:
> > > > > "cannot create delivery service: origin fqdn 'foo.example.com'
> already
> > > > in use"
> > > > > vs
> > > > > "cannot create delivery service: origin fqdn 'foo.example.com'
> already
> > > > > in use as type DNS_LIVE_NATNL, which is incompatible with your
> chosen
> > > > > type of HTTP_NO_CACHE"
> > > > >
> > > > > At that point you'd probably say to yourself, "uh, I need
> > > > > HTTP_NO_CACHE, so what am I supposed to do now?"
> > > > >
> > > > > As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
> > > > > because the resulting code will be simpler, but I think eliminating
> > > > > the mental overhead for operators could be worthwhile too. If we
> can
> > > > > agree on an end state of prohibiting duplicate origins altogether,
> we
> > > > > can start working on a design to smoothly transition us to that
> point.
> > > > > Are we willing to live with "just CNAME your origin fqdn" as the
> > > > > standard solution to duplicates?
> > > > >
> > > > > - Rawlin
> > > > >
> > > > >
> > > > > On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
> > > > > <De...@comcast.com> wrote:
> > > > > >
> > > > > > The only situation in which they can share origins is if a) the
> > > > origins are shared in an MSO configuration but still have different
> > > defined
> > > > origin fields in the delivery service, or if they're assigned to
> > > completely
> > > > different cachegroups. It's when two delivery services share the
> same
> > > > edges that there's an issue, because you end up with parent.config
> > > issues.
> > > > Actually you could even get away with it in mids as long as you
> weren't
> > > > doing anything like MSO to it.
> > > > > >
> > > > > > Could get messy real fast, though. Best to just create a second
> > > FQDN.
> > > > > >
> > > > > > Derek
> > > > > >
> > > > > > On 12/18/18, 3:23 PM, "Fieck, Brennan" <
> Brennan_Fieck@comcast.com>
> > > > wrote:
> > > > > >
> > > > > > So no two Delivery Services may share an origin *regardless
> of
> > > > cache hierarchy* ? I've been told that DNS Delivery Services can
> have the
> > > > same origin as HTTP Delivery Services because they obey the same
> cache
> > > > hierarchy. You're saying that would still produce invalid output
> and/or
> > > is
> > > > explicitly disallowed by ATS?
> > > > > > ________________________________________
> > > > > > From: Robert Butts <ro...@apache.org>
> > > > > > Sent: Tuesday, December 18, 2018 1:09 PM
> > > > > > To: dev@trafficcontrol.apache.org
> > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> Delivery
> > > > Services produces indeterminate parent.config
> > > > > >
> > > > > > >can you give an example of what parent.config looks like
> when 2
> > > > ds's share
> > > > > > an origin and have different a different topology?
> > > > > >
> > > > > > Answering because I encountered this directly, when rewriting
> > > > parent.config.
> > > > > >
> > > > > > For example: Suppose you have one Delivery Service:
> > > > > > XML_ID: foo
> > > > > > Type: HTPT_LIVE_NATL
> > > > > > Query String Handling: 1 - ignore in cache key, and pass up
> > > > > > Origin Server Base URL: http://foo.example.net
> > > > > >
> > > > > > And another Delivery Service:
> > > > > > XML_ID: bar
> > > > > > Type: HTPT_LIVE_NATL
> > > > > > Query String Handling: 1 - ignore in cache key, and pass up
> > > > > > Origin Server Base URL: http://foo.example.net
> > > > > >
> > > > > > ATS only supports unique `dest_domain` entries in
> parent.config.
> > > > Therefore,
> > > > > > the parent.config generated for a server assigned to both of
> > > these
> > > > Delivery
> > > > > > Services with either be:
> > > > > >
> > > > > > dest_domain=foo.example.net port=80 go_direct=true
> > > > > >
> > > > > > Or
> > > > > >
> > > > > > dest_domain=foo.example.net port=80 go_direct=false
> > > > qstring=consider
> > > > > >
> > > > > > Right now, it's arbitrary which Perl Traffic Ops inserts, and
> > > Perl
> > > > provides
> > > > > > no warning or error of any kind (the pending Go
> parent.config PR
> > > > logs an
> > > > > > error).
> > > > > >
> > > > > > Whichever is arbitrarily inserted, the resulting remaps for
> the
> > > > other
> > > > > > delivery service will be wrong. Either `foo` requests will
> drop
> > > > the query
> > > > > > string when they shouldn't, and go to the mid when they
> > > shouldn't;
> > > > or `bar`
> > > > > > requests will use the query string and skip the mid when it
> > > > shouldn't.
> > > > > >
> > > > > >
> > > > > > Does that make sense? The only correct solution, is to
> somehow
> > > > prevent
> > > > > > different DSes having the same origin, and tell tenants they
> must
> > > > use
> > > > > > CNAMEs if they need.
> > > > > >
> > > > > > This isn't a bug in Traffic Control. ATS fundamentally
> doesn't
> > > > support
> > > > > > multiple remap rules with the same parent FQDN with different
> > > > > > configurations. Hence, Traffic Control needs to prohibit
> that.
> > > > > >
> > > > > >
> > > > > > On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
> > > > mitchell852@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > brennan,
> > > > > > >
> > > > > > > can you give an example of what parent.config looks like
> when 2
> > > > ds's share
> > > > > > > an origin and have different a different topology?
> > > > > > >
> > > > > > > jeremy
> > > > > > >
> > > > > > > On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
> > > > Brennan_Fieck@comcast.com
> > > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > To be clear, the "Warning" I'm talking about would
> happen at
> > > > startup, but
> > > > > > > > I'd like a UI-only constraint to come with that to
> disallow
> > > > using the API
> > > > > > > > to bind the same origin to multiple Delivery Services
> with
> > > > varying
> > > > > > > > topography requirements. It wouldn't change the existing
> > > data,
> > > > but
> > > > > > > prevent
> > > > > > > > users from creating more bad data.
> > > > > > > >
> > > > > > > > "warning" doesn't really sufficiently describe that, my
> bad.
> > > > > > > > ________________________________________
> > > > > > > > From: Fieck, Brennan <Br...@comcast.com>
> > > > > > > > Sent: Tuesday, December 18, 2018 11:24 AM
> > > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > > > Delivery Services
> > > > > > > > produces indeterminate parent.config
> > > > > > > >
> > > > > > > > Well the cost of fixing this bug is a constraint on the
> data.
> > > > Unless we
> > > > > > > > make it a UI-only constraint - which I'm personally
> against -
> > > > there must
> > > > > > > be
> > > > > > > > some point in the future where ATC cannot reasonably be
> > > > expected to work
> > > > > > > > with data that violates that constraint. The question is
> when
> > > > that should
> > > > > > > > occur, which should likely happen at a minor version
> release.
> > > > Minor not
> > > > > > > > major because it doesn't involve a change in data
> structures,
> > > > merely
> > > > > > > > relationships between them - in my opinion that's a minor
> > > > version change
> > > > > > > > but that's definitely up for debate. With several release
> > > > candidates for
> > > > > > > > 3.0.0 that _doesn't_ include this restriction already in
> the
> > > > wild, I
> > > > > > > > wouldn't recommend putting it in there. That means to
> fix the
> > > > bug as soon
> > > > > > > > as possible it should go in 3.1.0 which should be the
> target
> > > > of "master"
> > > > > > > > after the 3.0.0 release is cut from it.
> > > > > > > >
> > > > > > > > So I'd recommend immediately implementing the constraint
> in
> > > > master with a
> > > > > > > > refusal to upgrade with bad data, and backport a warning
> > > about
> > > > the future
> > > > > > > > behavior into 3.0.0 or as part of a 3.0.1 provided we had
> > > more
> > > > changes
> > > > > > > that
> > > > > > > > would warrant a micro version bump.
> > > > > > > > ________________________________________
> > > > > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > > > > Sent: Tuesday, December 18, 2018 9:34 AM
> > > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > > > Delivery Services
> > > > > > > > produces indeterminate parent.config
> > > > > > > >
> > > > > > > > -1 Holding an ATC upgrade hostage to data cleanup seems
> like
> > > a
> > > > bad idea.
> > > > > > > > The issue isn't great, but it's also not new. We should
> > > allow
> > > > teams to
> > > > > > > fix
> > > > > > > > their data at their normal paces if it doesn't create
> > > > significant
> > > > > > > overhead
> > > > > > > > or an inherant blocker for new functionality or
> correction of
> > > > other major
> > > > > > > > problems imho.
> > > > > > > >
> > > > > > > > Jonathan G
> > > > > > > >
> > > > > > > >
> > > > > > > > On 12/18/18, 9:28 AM, "Fieck, Brennan" <
> > > > Brennan_Fieck@comcast.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > Another option is we could detect collisions at
> startup
> > > > and simply
> > > > > > > > refuse to continue with the upgrade until the data is
> fixed.
> > > > That would
> > > > > > > > allow people using the now-unsupported data format to
> > > continue
> > > > to use
> > > > > > > their
> > > > > > > > old versions of Traffic Ops without wrecking their
> database,
> > > > but also
> > > > > > > > provide an incentive to clean up the data.
> > > > > > > > ________________________________________
> > > > > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > > > > Sent: Tuesday, December 18, 2018 5:12 AM
> > > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to
> Multipe
> > > > Delivery
> > > > > > > > Services produces indeterminate parent.config
> > > > > > > >
> > > > > > > > I'm generally a fan of constrain your data in your
> > > > database, but not
> > > > > > > > necessarily exclusively. I see this as a one-way
> > > > cleanup/conversion so
> > > > > > > it
> > > > > > > > doesn't need to be configurable; otherwise you have to
> ask
> > > the
> > > > question
> > > > > > > > what happens if someone turns it off. That said,
> something
> > > in
> > > > the UI
> > > > > > > layer
> > > > > > > > would be nice to prevent spending significant quantities
> of
> > > > time
> > > > > > > building a
> > > > > > > > complex DS only to have it fail to post for reasons that
> > > could
> > > > have been
> > > > > > > > known earlier.
> > > > > > > >
> > > > > > > > The way my brain works in this case:
> > > > > > > > If !unique_constraint_exists_query()
> > > > > > > > If has_duplicates_query()
> > > > > > > > show_warning()
> > > > > > > > else
> > > > > > > > add_unique_constraint()
> > > > > > > >
> > > > > > > > to which the API and UI configuration could also
> make use
> > > > of
> > > > > > > > unique_constraint_exists_query() to drive additional
> layer
> > > > constraints if
> > > > > > > > desired.
> > > > > > > >
> > > > > > > > Jonathan G
> > > > > > > >
> > > > > > > > On 12/17/18, 1:11 PM, "Rawlin Peters" <
> > > > rawlin.peters@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > That is an interesting idea...detect at TO
> startup
> > > > whether or not
> > > > > > > > there are duplicate origins and operate in a
> "prevent
> > > > duplicate
> > > > > > > > origins" state if no duplicates are found or
> "prevent
> > > > conflicting
> > > > > > > > DS
> > > > > > > > topologies" state if duplicates are found? So
> once
> > > > operators have
> > > > > > > > replaced all the duplicate origins with CNAMEs,
> TO
> > > will
> > > > > > > essentially
> > > > > > > > operate in a "prohibit all duplicate origins"
> state.
> > > > That would
> > > > > > > > probably make for a simpler transition, but I'd
> want
> > > > to remove
> > > > > > > that
> > > > > > > > logic in a following release that strictly
> prohibits
> > > > duplicate
> > > > > > > > origins
> > > > > > > > (assuming that the community agrees we should
> > > prohibit
> > > > duplicate
> > > > > > > > origins altogether).
> > > > > > > >
> > > > > > > > As for DB constraints vs UI, I was thinking those
> > > > DS-type
> > > > > > > > constraints
> > > > > > > > I pointed out would live in the API. It would
> > > > basically be added
> > > > > > > > validation in the deliveryservices POST/PUT
> endpoint
> > > > that checks
> > > > > > > > the
> > > > > > > > DB for existing DSes that conflict with the
> requested
> > > > DS.
> > > > > > > >
> > > > > > > > - Rawlin
> > > > > > > >
> > > > > > > > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > > > > > > > <Jo...@comcast.com> wrote:
> > > > > > > > >
> > > > > > > > > These kinds of conditions should be detectable
> > > with a
> > > > > > > > sufficiently advanced SQL query. Is it possible to add
> the
> > > > constraint if
> > > > > > > > it passes and emit a warning during TO startup otherwise?
> > > > That would let
> > > > > > > > you know the condition exists at startup but not getting
> in
> > > > your way and
> > > > > > > > keep you out of trouble once you've cleaned up. We made
> a
> > > > mistake early
> > > > > > > > on, but this would acknowledge it was bad and encourage
> it to
> > > > be fixed at
> > > > > > > > the speed of operations teams. Also this puts the
> constraint
> > > > in the
> > > > > > > > database rather than the UI which is really where the
> > > > contention is for
> > > > > > > > usability.
> > > > > > > > >
> > > > > > > > > Jonathan G
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> > > > > > > rawlin.peters@gmail.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > We occasionally discuss this issue but
> haven't
> > > > tackled it
> > > > > > > > yet. I think
> > > > > > > > > the main issue is just that duplicate
> origins
> > > > have been
> > > > > > > > allowed since
> > > > > > > > > the beginning, and now everyone's Traffic
> Ops
> > > > could be
> > > > > > > > littered with
> > > > > > > > > duplicate origins. Also, depending on the
> > > config
> > > > of the
> > > > > > > > duplicate
> > > > > > > > > delivery services, the origins might not
> be in
> > > > conflict at
> > > > > > > > all (if
> > > > > > > > > they don't have different topology
> > > constraints).
> > > > I would
> > > > > > > > love for us
> > > > > > > > > to just add a uniqueness constraint, but
> there
> > > > would need
> > > > > > > to
> > > > > > > > be a fair
> > > > > > > > > amount of warning to the community before
> doing
> > > > so and
> > > > > > > might
> > > > > > > > > invalidate a significant amount of valid
> use
> > > > cases.
> > > > > > > > Operators would
> > > > > > > > > need time to make DNS CNAME records for the
> > > > duplicate
> > > > > > > > origins and
> > > > > > > > > update their DSes to use the different
> CNAMEs.
> > > > > > > > >
> > > > > > > > > I think as a good first step to
> eliminating the
> > > > use of
> > > > > > > > duplicate
> > > > > > > > > origins altogether, we should identify
> which
> > > > "topology
> > > > > > > > constraints"
> > > > > > > > > actually cause conflicting config when used
> > > with
> > > > duplicate
> > > > > > > > origins and
> > > > > > > > > prevent creating DSes with duplicate
> origins
> > > _if
> > > > it would
> > > > > > > > cause a
> > > > > > > > > conflict with an existing DS that uses the
> same
> > > > origin_.
> > > > > > > > >
> > > > > > > > > For instance, I believe an HTTP and
> DNS-type DS
> > > > can live
> > > > > > > > happily
> > > > > > > > > side-by-side using the same origin
> (probably
> > > > need different
> > > > > > > > > routing_names?), but scenarios like HTTP
> and
> > > > HTTP_LIVE, or
> > > > > > > > DNS and
> > > > > > > > > HTTP_NO_CACHE sharing the same origin will
> > > cause
> > > > conflicts
> > > > > > > > for sure.
> > > > > > > > > So maybe we can start by making sure the DS
> > > > types "match"
> > > > > > > > when using
> > > > > > > > > the same origin:
> > > > > > > > > HTTP + DNS: possibly good, if they have
> > > > different routing
> > > > > > > > names?
> > > > > > > > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > > > > > > > HTTP_NO_CACHE + [any other type]: bad
> > > > > > > > > HTTP_LIVE + HTTP: bad
> > > > > > > > > etc.
> > > > > > > > >
> > > > > > > > > There are most likely other conflict
> scenarios
> > > > that don't
> > > > > > > > involve the
> > > > > > > > > DS types, but I think this would be a good
> > > > start. In the
> > > > > > > > future with
> > > > > > > > > Delivery Service Topologies (aka Flexible
> > > > Cachegroups aka
> > > > > > > > Bring Your
> > > > > > > > > Own Topology), we might be able to prohibit
> > > > assigning a DS
> > > > > > > > to a
> > > > > > > > > Topology if the DS's origin is already
> used by
> > > > another DS
> > > > > > > in
> > > > > > > > a
> > > > > > > > > different Topology.
> > > > > > > > >
> > > > > > > > > - Rawlin
> > > > > > > > >
> > > > > > > > > On Mon, Dec 17, 2018 at 10:52 AM Fieck,
> Brennan
> > > > > > > > > <Br...@comcast.com> wrote:
> > > > > > > > > >
> > > > > > > > > > As some of you may be aware,
> `parent.config`
> > > > files
> > > > > > > > generated by Traffic Ops can vary wildly when an origin
> is
> > > > assigned to
> > > > > > > > multiple Delivery Services. This results in undefined
> > > > behavior. I'm told
> > > > > > > > that the conflict only happens when two Delivery Services
> > > with
> > > > different
> > > > > > > > "topology requirements" use the same origin, whatever
> that
> > > > means (content
> > > > > > > > routing type?). Regardless, the issue should be
> addressed.
> > > The
> > > > obvious
> > > > > > > > solution is to put in place a database constraint that
> > > > prevents an origin
> > > > > > > > from being assigned to more that one Delivery Service
> with
> > > API
> > > > checks in
> > > > > > > > place that would provide helpful error messages when an
> > > > attempt is made
> > > > > > > to
> > > > > > > > violate the constraint. However, would that mess with
> things
> > > > like
> > > > > > > > Multi-Site Origin? Or is it just not viable for some
> other
> > > > reason? If it
> > > > > > > is
> > > > > > > > a good solution, I'm prepared to work on a fix that
> utilizes
> > > > it.
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > >
> > >
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Mark Torluemke <mt...@apache.org>.
>> 1. Prohibit creating new delivery services that would share an
existing origin and prohibit updating a delivery service to a shared
origin
In case my position has been lost, I'm still -1 on this. :) IMO, we
shouldn't let the few complicated use cases increase the barrier to entry
(to using ATC) for the masses.
On Fri, Jan 11, 2019 at 11:53 AM Rawlin Peters <ra...@gmail.com>
wrote:
> Alright, I'm trying to sum up this discussion so far since it seems
> like everyone went on vacation and didn't really get a chance to wrap
> this one up:
> - duplicate origins cause undefined behavior
> - we need a way to migrate to a future that is free of duplicate
> origins in Traffic Control
> - we need a visual and easy way to determine if Traffic Ops currently
> contains duplicate origins, so that operators are incentivized to fix
> them rather than let it slide indefinitely
> - operators should have a fair amount of time to fix their duplicate
> origins
>
> I believe this is what we've mostly agreed upon but haven't clearly voted
> on:
>
> In release N we will:
> 1. Prohibit creating new delivery services that would share an
> existing origin and prohibit updating a delivery service to a shared
> origin
> 2. Add some kind of visual indicator that duplicate origins are a
> problem that need to be fixed before release N+1; otherwise, an
> upgrade to N+1 will be prohibited.
>
> In release N+1 we will:
> 3. Include a DB migration that adds a uniqueness constraint on origin
> FQDN, removing the API-level checks for that.
> 4. Prevent an upgrade to N+1 if duplicate origins are found (this
> might occur as a byproduct of step 3).
>
> I am +1 on this plan and believe this would hit on all the summarized
> points above. Please provide a clear vote on this plan so that we can
> dive deeper in the details (i.e. what release 'N' is, the best visual
> indicator for step 2, and a friendly way to handle step 4).
>
> Thanks,
> Rawlin
>
> On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <mi...@gmail.com>
> wrote:
> >
> > Not sure TP is the right place for a warning like "clean up this
> > 'duplicate' origin or your next upgrade will fail". Most users of our
> > system will be like "not my problem".
> >
> > On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <
> Brennan_Fieck@comcast.com>
> > wrote:
> >
> > > Probably. It would impact load times by a bit, but the page for an
> > > individual object is not our bottleneck.
> > > ________________________________________
> > > From: Robert Butts <ro...@apache.org>
> > > Sent: Wednesday, December 19, 2018 11:50 AM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> Services
> > > produces indeterminate parent.config
> > >
> > > > - Including a warning on startup and an API constraint preventing
> adding
> > > more bad data in the next 3.0.0 Release Candidate
> > > > - Adding a database constraint immediately into master that won't be
> > > cherry-picked into 3.0.0 but should be included in 3.1.0
> > >
> > > +1
> > >
> > > I understand Jonathan's objection, but at some point, we have to be
> able to
> > > move forward. This is a good compromise: deprecate, then remove. That
> gives
> > > people a full major version to fix their data.
> > >
> > > I would be ideal if it were more than just a logged warning, though.
> Can we
> > > add a big red banner in Traffic Portal, on the Delivery Service page
> for
> > > any DS with a duplicate origin, telling users to fix it, and that they
> > > won't be able to upgrade to the next major version until it's fixed?
> > >
> > >
> > > On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <
> Brennan_Fieck@comcast.com
> > > >
> > > wrote:
> > >
> > > > So it seems like nobody has a problem with the "how" - disallowing
> > > > duplicate origin FQDNs on Delivery Services - but we never reached a
> > > > consensus on "when".
> > > >
> > > > I stand by my previous proposal:
> > > > - Including a warning on startup and an API constraint preventing
> adding
> > > > more bad data in the next 3.0.0 Release Candidate
> > > > - Adding a database constraint immediately into master that won't be
> > > > cherry-picked into 3.0.0 but should be included in 3.1.0
> > > > ________________________________________
> > > > From: Rawlin Peters <ra...@gmail.com>
> > > > Sent: Tuesday, December 18, 2018 4:59 PM
> > > > To: dev@trafficcontrol.apache.org
> > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> Services
> > > > produces indeterminate parent.config
> > > >
> > > > Also, building more around DS types will make it even harder to get
> > > > away from DS types in the future too, which I know is something we've
> > > > discussed on this mailing list before. It also adds to the overhead
> of
> > > > Delivery Service Topologies, since a lot of the DS types won't
> > > > carryover into that world.
> > > >
> > > > - Rawlin
> > > >
> > > > On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
> > > > <Br...@comcast.com> wrote:
> > > > >
> > > > > +1.
> > > > > If there's a simple way to work around duplicate origins being
> > > > prohibited,
> > > > > then we should rely on that instead of "enumerating all those
> possible
> > > > conflicting
> > > > > settings, which are not only highly complex and confusing, but also
> > > > further
> > > > > entrench us in only supporting ATS as a caching proxy (hurting
> efforts
> > > to
> > > > > integrate e.g. Grove, nginx etc.)
> > > > > ________________________________________
> > > > > From: Rawlin Peters <ra...@gmail.com>
> > > > > Sent: Tuesday, December 18, 2018 2:20 PM
> > > > > To: dev@trafficcontrol.apache.org
> > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > > > Services produces indeterminate parent.config
> > > > >
> > > > > There are a number of different DS settings at play that can
> > > > > potentially cause conflicts. The question is: do we want to get
> into
> > > > > the business of enumerating all those possible conflicting
> settings or
> > > > > just simply prohibit duplicate origins altogether? I think we can
> dig
> > > > > in and get that "sufficiently advanced sql query" to check for
> > > > > conflicting origins, but is that something we want to carry along
> for
> > > > > the foreseeable future? Aren't CNAMEs relatively cheaper than
> > > > > developing and maintaining that code and the mental overhead
> required
> > > > > in understanding why you're getting an error that says your
> requested
> > > > > DS would cause an origin conflict? I think at the point you've
> > > > > requested a DS that would create a conflict, you've chosen those
> > > > > settings for a reason and would probably prefer to just create/use
> a
> > > > > CNAME in your new DS and keep the rest of your settings the same.
> > > > >
> > > > > Thinking in terms of errors, I'm imagining:
> > > > > "cannot create delivery service: origin fqdn 'foo.example.com'
> already
> > > > in use"
> > > > > vs
> > > > > "cannot create delivery service: origin fqdn 'foo.example.com'
> already
> > > > > in use as type DNS_LIVE_NATNL, which is incompatible with your
> chosen
> > > > > type of HTTP_NO_CACHE"
> > > > >
> > > > > At that point you'd probably say to yourself, "uh, I need
> > > > > HTTP_NO_CACHE, so what am I supposed to do now?"
> > > > >
> > > > > As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
> > > > > because the resulting code will be simpler, but I think eliminating
> > > > > the mental overhead for operators could be worthwhile too. If we
> can
> > > > > agree on an end state of prohibiting duplicate origins altogether,
> we
> > > > > can start working on a design to smoothly transition us to that
> point.
> > > > > Are we willing to live with "just CNAME your origin fqdn" as the
> > > > > standard solution to duplicates?
> > > > >
> > > > > - Rawlin
> > > > >
> > > > >
> > > > > On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
> > > > > <De...@comcast.com> wrote:
> > > > > >
> > > > > > The only situation in which they can share origins is if a) the
> > > > origins are shared in an MSO configuration but still have different
> > > defined
> > > > origin fields in the delivery service, or if they're assigned to
> > > completely
> > > > different cachegroups. It's when two delivery services share the
> same
> > > > edges that there's an issue, because you end up with parent.config
> > > issues.
> > > > Actually you could even get away with it in mids as long as you
> weren't
> > > > doing anything like MSO to it.
> > > > > >
> > > > > > Could get messy real fast, though. Best to just create a second
> > > FQDN.
> > > > > >
> > > > > > Derek
> > > > > >
> > > > > > On 12/18/18, 3:23 PM, "Fieck, Brennan" <
> Brennan_Fieck@comcast.com>
> > > > wrote:
> > > > > >
> > > > > > So no two Delivery Services may share an origin *regardless
> of
> > > > cache hierarchy* ? I've been told that DNS Delivery Services can
> have the
> > > > same origin as HTTP Delivery Services because they obey the same
> cache
> > > > hierarchy. You're saying that would still produce invalid output
> and/or
> > > is
> > > > explicitly disallowed by ATS?
> > > > > > ________________________________________
> > > > > > From: Robert Butts <ro...@apache.org>
> > > > > > Sent: Tuesday, December 18, 2018 1:09 PM
> > > > > > To: dev@trafficcontrol.apache.org
> > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> Delivery
> > > > Services produces indeterminate parent.config
> > > > > >
> > > > > > >can you give an example of what parent.config looks like
> when 2
> > > > ds's share
> > > > > > an origin and have different a different topology?
> > > > > >
> > > > > > Answering because I encountered this directly, when rewriting
> > > > parent.config.
> > > > > >
> > > > > > For example: Suppose you have one Delivery Service:
> > > > > > XML_ID: foo
> > > > > > Type: HTPT_LIVE_NATL
> > > > > > Query String Handling: 1 - ignore in cache key, and pass up
> > > > > > Origin Server Base URL: http://foo.example.net
> > > > > >
> > > > > > And another Delivery Service:
> > > > > > XML_ID: bar
> > > > > > Type: HTPT_LIVE_NATL
> > > > > > Query String Handling: 1 - ignore in cache key, and pass up
> > > > > > Origin Server Base URL: http://foo.example.net
> > > > > >
> > > > > > ATS only supports unique `dest_domain` entries in
> parent.config.
> > > > Therefore,
> > > > > > the parent.config generated for a server assigned to both of
> > > these
> > > > Delivery
> > > > > > Services with either be:
> > > > > >
> > > > > > dest_domain=foo.example.net port=80 go_direct=true
> > > > > >
> > > > > > Or
> > > > > >
> > > > > > dest_domain=foo.example.net port=80 go_direct=false
> > > > qstring=consider
> > > > > >
> > > > > > Right now, it's arbitrary which Perl Traffic Ops inserts, and
> > > Perl
> > > > provides
> > > > > > no warning or error of any kind (the pending Go
> parent.config PR
> > > > logs an
> > > > > > error).
> > > > > >
> > > > > > Whichever is arbitrarily inserted, the resulting remaps for
> the
> > > > other
> > > > > > delivery service will be wrong. Either `foo` requests will
> drop
> > > > the query
> > > > > > string when they shouldn't, and go to the mid when they
> > > shouldn't;
> > > > or `bar`
> > > > > > requests will use the query string and skip the mid when it
> > > > shouldn't.
> > > > > >
> > > > > >
> > > > > > Does that make sense? The only correct solution, is to
> somehow
> > > > prevent
> > > > > > different DSes having the same origin, and tell tenants they
> must
> > > > use
> > > > > > CNAMEs if they need.
> > > > > >
> > > > > > This isn't a bug in Traffic Control. ATS fundamentally
> doesn't
> > > > support
> > > > > > multiple remap rules with the same parent FQDN with different
> > > > > > configurations. Hence, Traffic Control needs to prohibit
> that.
> > > > > >
> > > > > >
> > > > > > On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
> > > > mitchell852@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > brennan,
> > > > > > >
> > > > > > > can you give an example of what parent.config looks like
> when 2
> > > > ds's share
> > > > > > > an origin and have different a different topology?
> > > > > > >
> > > > > > > jeremy
> > > > > > >
> > > > > > > On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
> > > > Brennan_Fieck@comcast.com
> > > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > To be clear, the "Warning" I'm talking about would
> happen at
> > > > startup, but
> > > > > > > > I'd like a UI-only constraint to come with that to
> disallow
> > > > using the API
> > > > > > > > to bind the same origin to multiple Delivery Services
> with
> > > > varying
> > > > > > > > topography requirements. It wouldn't change the existing
> > > data,
> > > > but
> > > > > > > prevent
> > > > > > > > users from creating more bad data.
> > > > > > > >
> > > > > > > > "warning" doesn't really sufficiently describe that, my
> bad.
> > > > > > > > ________________________________________
> > > > > > > > From: Fieck, Brennan <Br...@comcast.com>
> > > > > > > > Sent: Tuesday, December 18, 2018 11:24 AM
> > > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > > > Delivery Services
> > > > > > > > produces indeterminate parent.config
> > > > > > > >
> > > > > > > > Well the cost of fixing this bug is a constraint on the
> data.
> > > > Unless we
> > > > > > > > make it a UI-only constraint - which I'm personally
> against -
> > > > there must
> > > > > > > be
> > > > > > > > some point in the future where ATC cannot reasonably be
> > > > expected to work
> > > > > > > > with data that violates that constraint. The question is
> when
> > > > that should
> > > > > > > > occur, which should likely happen at a minor version
> release.
> > > > Minor not
> > > > > > > > major because it doesn't involve a change in data
> structures,
> > > > merely
> > > > > > > > relationships between them - in my opinion that's a minor
> > > > version change
> > > > > > > > but that's definitely up for debate. With several release
> > > > candidates for
> > > > > > > > 3.0.0 that _doesn't_ include this restriction already in
> the
> > > > wild, I
> > > > > > > > wouldn't recommend putting it in there. That means to
> fix the
> > > > bug as soon
> > > > > > > > as possible it should go in 3.1.0 which should be the
> target
> > > > of "master"
> > > > > > > > after the 3.0.0 release is cut from it.
> > > > > > > >
> > > > > > > > So I'd recommend immediately implementing the constraint
> in
> > > > master with a
> > > > > > > > refusal to upgrade with bad data, and backport a warning
> > > about
> > > > the future
> > > > > > > > behavior into 3.0.0 or as part of a 3.0.1 provided we had
> > > more
> > > > changes
> > > > > > > that
> > > > > > > > would warrant a micro version bump.
> > > > > > > > ________________________________________
> > > > > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > > > > Sent: Tuesday, December 18, 2018 9:34 AM
> > > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > > > Delivery Services
> > > > > > > > produces indeterminate parent.config
> > > > > > > >
> > > > > > > > -1 Holding an ATC upgrade hostage to data cleanup seems
> like
> > > a
> > > > bad idea.
> > > > > > > > The issue isn't great, but it's also not new. We should
> > > allow
> > > > teams to
> > > > > > > fix
> > > > > > > > their data at their normal paces if it doesn't create
> > > > significant
> > > > > > > overhead
> > > > > > > > or an inherant blocker for new functionality or
> correction of
> > > > other major
> > > > > > > > problems imho.
> > > > > > > >
> > > > > > > > Jonathan G
> > > > > > > >
> > > > > > > >
> > > > > > > > On 12/18/18, 9:28 AM, "Fieck, Brennan" <
> > > > Brennan_Fieck@comcast.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > Another option is we could detect collisions at
> startup
> > > > and simply
> > > > > > > > refuse to continue with the upgrade until the data is
> fixed.
> > > > That would
> > > > > > > > allow people using the now-unsupported data format to
> > > continue
> > > > to use
> > > > > > > their
> > > > > > > > old versions of Traffic Ops without wrecking their
> database,
> > > > but also
> > > > > > > > provide an incentive to clean up the data.
> > > > > > > > ________________________________________
> > > > > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > > > > Sent: Tuesday, December 18, 2018 5:12 AM
> > > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to
> Multipe
> > > > Delivery
> > > > > > > > Services produces indeterminate parent.config
> > > > > > > >
> > > > > > > > I'm generally a fan of constrain your data in your
> > > > database, but not
> > > > > > > > necessarily exclusively. I see this as a one-way
> > > > cleanup/conversion so
> > > > > > > it
> > > > > > > > doesn't need to be configurable; otherwise you have to
> ask
> > > the
> > > > question
> > > > > > > > what happens if someone turns it off. That said,
> something
> > > in
> > > > the UI
> > > > > > > layer
> > > > > > > > would be nice to prevent spending significant quantities
> of
> > > > time
> > > > > > > building a
> > > > > > > > complex DS only to have it fail to post for reasons that
> > > could
> > > > have been
> > > > > > > > known earlier.
> > > > > > > >
> > > > > > > > The way my brain works in this case:
> > > > > > > > If !unique_constraint_exists_query()
> > > > > > > > If has_duplicates_query()
> > > > > > > > show_warning()
> > > > > > > > else
> > > > > > > > add_unique_constraint()
> > > > > > > >
> > > > > > > > to which the API and UI configuration could also
> make use
> > > > of
> > > > > > > > unique_constraint_exists_query() to drive additional
> layer
> > > > constraints if
> > > > > > > > desired.
> > > > > > > >
> > > > > > > > Jonathan G
> > > > > > > >
> > > > > > > > On 12/17/18, 1:11 PM, "Rawlin Peters" <
> > > > rawlin.peters@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > That is an interesting idea...detect at TO
> startup
> > > > whether or not
> > > > > > > > there are duplicate origins and operate in a
> "prevent
> > > > duplicate
> > > > > > > > origins" state if no duplicates are found or
> "prevent
> > > > conflicting
> > > > > > > > DS
> > > > > > > > topologies" state if duplicates are found? So
> once
> > > > operators have
> > > > > > > > replaced all the duplicate origins with CNAMEs,
> TO
> > > will
> > > > > > > essentially
> > > > > > > > operate in a "prohibit all duplicate origins"
> state.
> > > > That would
> > > > > > > > probably make for a simpler transition, but I'd
> want
> > > > to remove
> > > > > > > that
> > > > > > > > logic in a following release that strictly
> prohibits
> > > > duplicate
> > > > > > > > origins
> > > > > > > > (assuming that the community agrees we should
> > > prohibit
> > > > duplicate
> > > > > > > > origins altogether).
> > > > > > > >
> > > > > > > > As for DB constraints vs UI, I was thinking those
> > > > DS-type
> > > > > > > > constraints
> > > > > > > > I pointed out would live in the API. It would
> > > > basically be added
> > > > > > > > validation in the deliveryservices POST/PUT
> endpoint
> > > > that checks
> > > > > > > > the
> > > > > > > > DB for existing DSes that conflict with the
> requested
> > > > DS.
> > > > > > > >
> > > > > > > > - Rawlin
> > > > > > > >
> > > > > > > > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > > > > > > > <Jo...@comcast.com> wrote:
> > > > > > > > >
> > > > > > > > > These kinds of conditions should be detectable
> > > with a
> > > > > > > > sufficiently advanced SQL query. Is it possible to add
> the
> > > > constraint if
> > > > > > > > it passes and emit a warning during TO startup otherwise?
> > > > That would let
> > > > > > > > you know the condition exists at startup but not getting
> in
> > > > your way and
> > > > > > > > keep you out of trouble once you've cleaned up. We made
> a
> > > > mistake early
> > > > > > > > on, but this would acknowledge it was bad and encourage
> it to
> > > > be fixed at
> > > > > > > > the speed of operations teams. Also this puts the
> constraint
> > > > in the
> > > > > > > > database rather than the UI which is really where the
> > > > contention is for
> > > > > > > > usability.
> > > > > > > > >
> > > > > > > > > Jonathan G
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> > > > > > > rawlin.peters@gmail.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > We occasionally discuss this issue but
> haven't
> > > > tackled it
> > > > > > > > yet. I think
> > > > > > > > > the main issue is just that duplicate
> origins
> > > > have been
> > > > > > > > allowed since
> > > > > > > > > the beginning, and now everyone's Traffic
> Ops
> > > > could be
> > > > > > > > littered with
> > > > > > > > > duplicate origins. Also, depending on the
> > > config
> > > > of the
> > > > > > > > duplicate
> > > > > > > > > delivery services, the origins might not
> be in
> > > > conflict at
> > > > > > > > all (if
> > > > > > > > > they don't have different topology
> > > constraints).
> > > > I would
> > > > > > > > love for us
> > > > > > > > > to just add a uniqueness constraint, but
> there
> > > > would need
> > > > > > > to
> > > > > > > > be a fair
> > > > > > > > > amount of warning to the community before
> doing
> > > > so and
> > > > > > > might
> > > > > > > > > invalidate a significant amount of valid
> use
> > > > cases.
> > > > > > > > Operators would
> > > > > > > > > need time to make DNS CNAME records for the
> > > > duplicate
> > > > > > > > origins and
> > > > > > > > > update their DSes to use the different
> CNAMEs.
> > > > > > > > >
> > > > > > > > > I think as a good first step to
> eliminating the
> > > > use of
> > > > > > > > duplicate
> > > > > > > > > origins altogether, we should identify
> which
> > > > "topology
> > > > > > > > constraints"
> > > > > > > > > actually cause conflicting config when used
> > > with
> > > > duplicate
> > > > > > > > origins and
> > > > > > > > > prevent creating DSes with duplicate
> origins
> > > _if
> > > > it would
> > > > > > > > cause a
> > > > > > > > > conflict with an existing DS that uses the
> same
> > > > origin_.
> > > > > > > > >
> > > > > > > > > For instance, I believe an HTTP and
> DNS-type DS
> > > > can live
> > > > > > > > happily
> > > > > > > > > side-by-side using the same origin
> (probably
> > > > need different
> > > > > > > > > routing_names?), but scenarios like HTTP
> and
> > > > HTTP_LIVE, or
> > > > > > > > DNS and
> > > > > > > > > HTTP_NO_CACHE sharing the same origin will
> > > cause
> > > > conflicts
> > > > > > > > for sure.
> > > > > > > > > So maybe we can start by making sure the DS
> > > > types "match"
> > > > > > > > when using
> > > > > > > > > the same origin:
> > > > > > > > > HTTP + DNS: possibly good, if they have
> > > > different routing
> > > > > > > > names?
> > > > > > > > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > > > > > > > HTTP_NO_CACHE + [any other type]: bad
> > > > > > > > > HTTP_LIVE + HTTP: bad
> > > > > > > > > etc.
> > > > > > > > >
> > > > > > > > > There are most likely other conflict
> scenarios
> > > > that don't
> > > > > > > > involve the
> > > > > > > > > DS types, but I think this would be a good
> > > > start. In the
> > > > > > > > future with
> > > > > > > > > Delivery Service Topologies (aka Flexible
> > > > Cachegroups aka
> > > > > > > > Bring Your
> > > > > > > > > Own Topology), we might be able to prohibit
> > > > assigning a DS
> > > > > > > > to a
> > > > > > > > > Topology if the DS's origin is already
> used by
> > > > another DS
> > > > > > > in
> > > > > > > > a
> > > > > > > > > different Topology.
> > > > > > > > >
> > > > > > > > > - Rawlin
> > > > > > > > >
> > > > > > > > > On Mon, Dec 17, 2018 at 10:52 AM Fieck,
> Brennan
> > > > > > > > > <Br...@comcast.com> wrote:
> > > > > > > > > >
> > > > > > > > > > As some of you may be aware,
> `parent.config`
> > > > files
> > > > > > > > generated by Traffic Ops can vary wildly when an origin
> is
> > > > assigned to
> > > > > > > > multiple Delivery Services. This results in undefined
> > > > behavior. I'm told
> > > > > > > > that the conflict only happens when two Delivery Services
> > > with
> > > > different
> > > > > > > > "topology requirements" use the same origin, whatever
> that
> > > > means (content
> > > > > > > > routing type?). Regardless, the issue should be
> addressed.
> > > The
> > > > obvious
> > > > > > > > solution is to put in place a database constraint that
> > > > prevents an origin
> > > > > > > > from being assigned to more that one Delivery Service
> with
> > > API
> > > > checks in
> > > > > > > > place that would provide helpful error messages when an
> > > > attempt is made
> > > > > > > to
> > > > > > > > violate the constraint. However, would that mess with
> things
> > > > like
> > > > > > > > Multi-Site Origin? Or is it just not viable for some
> other
> > > > reason? If it
> > > > > > > is
> > > > > > > > a good solution, I'm prepared to work on a fix that
> utilizes
> > > > it.
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > >
> > >
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Rawlin Peters <ra...@gmail.com>.
Alright, I'm trying to sum up this discussion so far since it seems
like everyone went on vacation and didn't really get a chance to wrap
this one up:
- duplicate origins cause undefined behavior
- we need a way to migrate to a future that is free of duplicate
origins in Traffic Control
- we need a visual and easy way to determine if Traffic Ops currently
contains duplicate origins, so that operators are incentivized to fix
them rather than let it slide indefinitely
- operators should have a fair amount of time to fix their duplicate origins
I believe this is what we've mostly agreed upon but haven't clearly voted on:
In release N we will:
1. Prohibit creating new delivery services that would share an
existing origin and prohibit updating a delivery service to a shared
origin
2. Add some kind of visual indicator that duplicate origins are a
problem that need to be fixed before release N+1; otherwise, an
upgrade to N+1 will be prohibited.
In release N+1 we will:
3. Include a DB migration that adds a uniqueness constraint on origin
FQDN, removing the API-level checks for that.
4. Prevent an upgrade to N+1 if duplicate origins are found (this
might occur as a byproduct of step 3).
I am +1 on this plan and believe this would hit on all the summarized
points above. Please provide a clear vote on this plan so that we can
dive deeper in the details (i.e. what release 'N' is, the best visual
indicator for step 2, and a friendly way to handle step 4).
Thanks,
Rawlin
On Wed, Dec 19, 2018 at 3:39 PM Jeremy Mitchell <mi...@gmail.com> wrote:
>
> Not sure TP is the right place for a warning like "clean up this
> 'duplicate' origin or your next upgrade will fail". Most users of our
> system will be like "not my problem".
>
> On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <Br...@comcast.com>
> wrote:
>
> > Probably. It would impact load times by a bit, but the page for an
> > individual object is not our bottleneck.
> > ________________________________________
> > From: Robert Butts <ro...@apache.org>
> > Sent: Wednesday, December 19, 2018 11:50 AM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > produces indeterminate parent.config
> >
> > > - Including a warning on startup and an API constraint preventing adding
> > more bad data in the next 3.0.0 Release Candidate
> > > - Adding a database constraint immediately into master that won't be
> > cherry-picked into 3.0.0 but should be included in 3.1.0
> >
> > +1
> >
> > I understand Jonathan's objection, but at some point, we have to be able to
> > move forward. This is a good compromise: deprecate, then remove. That gives
> > people a full major version to fix their data.
> >
> > I would be ideal if it were more than just a logged warning, though. Can we
> > add a big red banner in Traffic Portal, on the Delivery Service page for
> > any DS with a duplicate origin, telling users to fix it, and that they
> > won't be able to upgrade to the next major version until it's fixed?
> >
> >
> > On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <Brennan_Fieck@comcast.com
> > >
> > wrote:
> >
> > > So it seems like nobody has a problem with the "how" - disallowing
> > > duplicate origin FQDNs on Delivery Services - but we never reached a
> > > consensus on "when".
> > >
> > > I stand by my previous proposal:
> > > - Including a warning on startup and an API constraint preventing adding
> > > more bad data in the next 3.0.0 Release Candidate
> > > - Adding a database constraint immediately into master that won't be
> > > cherry-picked into 3.0.0 but should be included in 3.1.0
> > > ________________________________________
> > > From: Rawlin Peters <ra...@gmail.com>
> > > Sent: Tuesday, December 18, 2018 4:59 PM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > > produces indeterminate parent.config
> > >
> > > Also, building more around DS types will make it even harder to get
> > > away from DS types in the future too, which I know is something we've
> > > discussed on this mailing list before. It also adds to the overhead of
> > > Delivery Service Topologies, since a lot of the DS types won't
> > > carryover into that world.
> > >
> > > - Rawlin
> > >
> > > On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
> > > <Br...@comcast.com> wrote:
> > > >
> > > > +1.
> > > > If there's a simple way to work around duplicate origins being
> > > prohibited,
> > > > then we should rely on that instead of "enumerating all those possible
> > > conflicting
> > > > settings, which are not only highly complex and confusing, but also
> > > further
> > > > entrench us in only supporting ATS as a caching proxy (hurting efforts
> > to
> > > > integrate e.g. Grove, nginx etc.)
> > > > ________________________________________
> > > > From: Rawlin Peters <ra...@gmail.com>
> > > > Sent: Tuesday, December 18, 2018 2:20 PM
> > > > To: dev@trafficcontrol.apache.org
> > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > > Services produces indeterminate parent.config
> > > >
> > > > There are a number of different DS settings at play that can
> > > > potentially cause conflicts. The question is: do we want to get into
> > > > the business of enumerating all those possible conflicting settings or
> > > > just simply prohibit duplicate origins altogether? I think we can dig
> > > > in and get that "sufficiently advanced sql query" to check for
> > > > conflicting origins, but is that something we want to carry along for
> > > > the foreseeable future? Aren't CNAMEs relatively cheaper than
> > > > developing and maintaining that code and the mental overhead required
> > > > in understanding why you're getting an error that says your requested
> > > > DS would cause an origin conflict? I think at the point you've
> > > > requested a DS that would create a conflict, you've chosen those
> > > > settings for a reason and would probably prefer to just create/use a
> > > > CNAME in your new DS and keep the rest of your settings the same.
> > > >
> > > > Thinking in terms of errors, I'm imagining:
> > > > "cannot create delivery service: origin fqdn 'foo.example.com' already
> > > in use"
> > > > vs
> > > > "cannot create delivery service: origin fqdn 'foo.example.com' already
> > > > in use as type DNS_LIVE_NATNL, which is incompatible with your chosen
> > > > type of HTTP_NO_CACHE"
> > > >
> > > > At that point you'd probably say to yourself, "uh, I need
> > > > HTTP_NO_CACHE, so what am I supposed to do now?"
> > > >
> > > > As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
> > > > because the resulting code will be simpler, but I think eliminating
> > > > the mental overhead for operators could be worthwhile too. If we can
> > > > agree on an end state of prohibiting duplicate origins altogether, we
> > > > can start working on a design to smoothly transition us to that point.
> > > > Are we willing to live with "just CNAME your origin fqdn" as the
> > > > standard solution to duplicates?
> > > >
> > > > - Rawlin
> > > >
> > > >
> > > > On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
> > > > <De...@comcast.com> wrote:
> > > > >
> > > > > The only situation in which they can share origins is if a) the
> > > origins are shared in an MSO configuration but still have different
> > defined
> > > origin fields in the delivery service, or if they're assigned to
> > completely
> > > different cachegroups. It's when two delivery services share the same
> > > edges that there's an issue, because you end up with parent.config
> > issues.
> > > Actually you could even get away with it in mids as long as you weren't
> > > doing anything like MSO to it.
> > > > >
> > > > > Could get messy real fast, though. Best to just create a second
> > FQDN.
> > > > >
> > > > > Derek
> > > > >
> > > > > On 12/18/18, 3:23 PM, "Fieck, Brennan" <Br...@comcast.com>
> > > wrote:
> > > > >
> > > > > So no two Delivery Services may share an origin *regardless of
> > > cache hierarchy* ? I've been told that DNS Delivery Services can have the
> > > same origin as HTTP Delivery Services because they obey the same cache
> > > hierarchy. You're saying that would still produce invalid output and/or
> > is
> > > explicitly disallowed by ATS?
> > > > > ________________________________________
> > > > > From: Robert Butts <ro...@apache.org>
> > > > > Sent: Tuesday, December 18, 2018 1:09 PM
> > > > > To: dev@trafficcontrol.apache.org
> > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > > Services produces indeterminate parent.config
> > > > >
> > > > > >can you give an example of what parent.config looks like when 2
> > > ds's share
> > > > > an origin and have different a different topology?
> > > > >
> > > > > Answering because I encountered this directly, when rewriting
> > > parent.config.
> > > > >
> > > > > For example: Suppose you have one Delivery Service:
> > > > > XML_ID: foo
> > > > > Type: HTPT_LIVE_NATL
> > > > > Query String Handling: 1 - ignore in cache key, and pass up
> > > > > Origin Server Base URL: http://foo.example.net
> > > > >
> > > > > And another Delivery Service:
> > > > > XML_ID: bar
> > > > > Type: HTPT_LIVE_NATL
> > > > > Query String Handling: 1 - ignore in cache key, and pass up
> > > > > Origin Server Base URL: http://foo.example.net
> > > > >
> > > > > ATS only supports unique `dest_domain` entries in parent.config.
> > > Therefore,
> > > > > the parent.config generated for a server assigned to both of
> > these
> > > Delivery
> > > > > Services with either be:
> > > > >
> > > > > dest_domain=foo.example.net port=80 go_direct=true
> > > > >
> > > > > Or
> > > > >
> > > > > dest_domain=foo.example.net port=80 go_direct=false
> > > qstring=consider
> > > > >
> > > > > Right now, it's arbitrary which Perl Traffic Ops inserts, and
> > Perl
> > > provides
> > > > > no warning or error of any kind (the pending Go parent.config PR
> > > logs an
> > > > > error).
> > > > >
> > > > > Whichever is arbitrarily inserted, the resulting remaps for the
> > > other
> > > > > delivery service will be wrong. Either `foo` requests will drop
> > > the query
> > > > > string when they shouldn't, and go to the mid when they
> > shouldn't;
> > > or `bar`
> > > > > requests will use the query string and skip the mid when it
> > > shouldn't.
> > > > >
> > > > >
> > > > > Does that make sense? The only correct solution, is to somehow
> > > prevent
> > > > > different DSes having the same origin, and tell tenants they must
> > > use
> > > > > CNAMEs if they need.
> > > > >
> > > > > This isn't a bug in Traffic Control. ATS fundamentally doesn't
> > > support
> > > > > multiple remap rules with the same parent FQDN with different
> > > > > configurations. Hence, Traffic Control needs to prohibit that.
> > > > >
> > > > >
> > > > > On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
> > > mitchell852@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > brennan,
> > > > > >
> > > > > > can you give an example of what parent.config looks like when 2
> > > ds's share
> > > > > > an origin and have different a different topology?
> > > > > >
> > > > > > jeremy
> > > > > >
> > > > > > On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
> > > Brennan_Fieck@comcast.com
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > To be clear, the "Warning" I'm talking about would happen at
> > > startup, but
> > > > > > > I'd like a UI-only constraint to come with that to disallow
> > > using the API
> > > > > > > to bind the same origin to multiple Delivery Services with
> > > varying
> > > > > > > topography requirements. It wouldn't change the existing
> > data,
> > > but
> > > > > > prevent
> > > > > > > users from creating more bad data.
> > > > > > >
> > > > > > > "warning" doesn't really sufficiently describe that, my bad.
> > > > > > > ________________________________________
> > > > > > > From: Fieck, Brennan <Br...@comcast.com>
> > > > > > > Sent: Tuesday, December 18, 2018 11:24 AM
> > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > > Delivery Services
> > > > > > > produces indeterminate parent.config
> > > > > > >
> > > > > > > Well the cost of fixing this bug is a constraint on the data.
> > > Unless we
> > > > > > > make it a UI-only constraint - which I'm personally against -
> > > there must
> > > > > > be
> > > > > > > some point in the future where ATC cannot reasonably be
> > > expected to work
> > > > > > > with data that violates that constraint. The question is when
> > > that should
> > > > > > > occur, which should likely happen at a minor version release.
> > > Minor not
> > > > > > > major because it doesn't involve a change in data structures,
> > > merely
> > > > > > > relationships between them - in my opinion that's a minor
> > > version change
> > > > > > > but that's definitely up for debate. With several release
> > > candidates for
> > > > > > > 3.0.0 that _doesn't_ include this restriction already in the
> > > wild, I
> > > > > > > wouldn't recommend putting it in there. That means to fix the
> > > bug as soon
> > > > > > > as possible it should go in 3.1.0 which should be the target
> > > of "master"
> > > > > > > after the 3.0.0 release is cut from it.
> > > > > > >
> > > > > > > So I'd recommend immediately implementing the constraint in
> > > master with a
> > > > > > > refusal to upgrade with bad data, and backport a warning
> > about
> > > the future
> > > > > > > behavior into 3.0.0 or as part of a 3.0.1 provided we had
> > more
> > > changes
> > > > > > that
> > > > > > > would warrant a micro version bump.
> > > > > > > ________________________________________
> > > > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > > > Sent: Tuesday, December 18, 2018 9:34 AM
> > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > > Delivery Services
> > > > > > > produces indeterminate parent.config
> > > > > > >
> > > > > > > -1 Holding an ATC upgrade hostage to data cleanup seems like
> > a
> > > bad idea.
> > > > > > > The issue isn't great, but it's also not new. We should
> > allow
> > > teams to
> > > > > > fix
> > > > > > > their data at their normal paces if it doesn't create
> > > significant
> > > > > > overhead
> > > > > > > or an inherant blocker for new functionality or correction of
> > > other major
> > > > > > > problems imho.
> > > > > > >
> > > > > > > Jonathan G
> > > > > > >
> > > > > > >
> > > > > > > On 12/18/18, 9:28 AM, "Fieck, Brennan" <
> > > Brennan_Fieck@comcast.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > Another option is we could detect collisions at startup
> > > and simply
> > > > > > > refuse to continue with the upgrade until the data is fixed.
> > > That would
> > > > > > > allow people using the now-unsupported data format to
> > continue
> > > to use
> > > > > > their
> > > > > > > old versions of Traffic Ops without wrecking their database,
> > > but also
> > > > > > > provide an incentive to clean up the data.
> > > > > > > ________________________________________
> > > > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > > > Sent: Tuesday, December 18, 2018 5:12 AM
> > > > > > > To: dev@trafficcontrol.apache.org
> > > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > > Delivery
> > > > > > > Services produces indeterminate parent.config
> > > > > > >
> > > > > > > I'm generally a fan of constrain your data in your
> > > database, but not
> > > > > > > necessarily exclusively. I see this as a one-way
> > > cleanup/conversion so
> > > > > > it
> > > > > > > doesn't need to be configurable; otherwise you have to ask
> > the
> > > question
> > > > > > > what happens if someone turns it off. That said, something
> > in
> > > the UI
> > > > > > layer
> > > > > > > would be nice to prevent spending significant quantities of
> > > time
> > > > > > building a
> > > > > > > complex DS only to have it fail to post for reasons that
> > could
> > > have been
> > > > > > > known earlier.
> > > > > > >
> > > > > > > The way my brain works in this case:
> > > > > > > If !unique_constraint_exists_query()
> > > > > > > If has_duplicates_query()
> > > > > > > show_warning()
> > > > > > > else
> > > > > > > add_unique_constraint()
> > > > > > >
> > > > > > > to which the API and UI configuration could also make use
> > > of
> > > > > > > unique_constraint_exists_query() to drive additional layer
> > > constraints if
> > > > > > > desired.
> > > > > > >
> > > > > > > Jonathan G
> > > > > > >
> > > > > > > On 12/17/18, 1:11 PM, "Rawlin Peters" <
> > > rawlin.peters@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > That is an interesting idea...detect at TO startup
> > > whether or not
> > > > > > > there are duplicate origins and operate in a "prevent
> > > duplicate
> > > > > > > origins" state if no duplicates are found or "prevent
> > > conflicting
> > > > > > > DS
> > > > > > > topologies" state if duplicates are found? So once
> > > operators have
> > > > > > > replaced all the duplicate origins with CNAMEs, TO
> > will
> > > > > > essentially
> > > > > > > operate in a "prohibit all duplicate origins" state.
> > > That would
> > > > > > > probably make for a simpler transition, but I'd want
> > > to remove
> > > > > > that
> > > > > > > logic in a following release that strictly prohibits
> > > duplicate
> > > > > > > origins
> > > > > > > (assuming that the community agrees we should
> > prohibit
> > > duplicate
> > > > > > > origins altogether).
> > > > > > >
> > > > > > > As for DB constraints vs UI, I was thinking those
> > > DS-type
> > > > > > > constraints
> > > > > > > I pointed out would live in the API. It would
> > > basically be added
> > > > > > > validation in the deliveryservices POST/PUT endpoint
> > > that checks
> > > > > > > the
> > > > > > > DB for existing DSes that conflict with the requested
> > > DS.
> > > > > > >
> > > > > > > - Rawlin
> > > > > > >
> > > > > > > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > > > > > > <Jo...@comcast.com> wrote:
> > > > > > > >
> > > > > > > > These kinds of conditions should be detectable
> > with a
> > > > > > > sufficiently advanced SQL query. Is it possible to add the
> > > constraint if
> > > > > > > it passes and emit a warning during TO startup otherwise?
> > > That would let
> > > > > > > you know the condition exists at startup but not getting in
> > > your way and
> > > > > > > keep you out of trouble once you've cleaned up. We made a
> > > mistake early
> > > > > > > on, but this would acknowledge it was bad and encourage it to
> > > be fixed at
> > > > > > > the speed of operations teams. Also this puts the constraint
> > > in the
> > > > > > > database rather than the UI which is really where the
> > > contention is for
> > > > > > > usability.
> > > > > > > >
> > > > > > > > Jonathan G
> > > > > > > >
> > > > > > > >
> > > > > > > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> > > > > > rawlin.peters@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > We occasionally discuss this issue but haven't
> > > tackled it
> > > > > > > yet. I think
> > > > > > > > the main issue is just that duplicate origins
> > > have been
> > > > > > > allowed since
> > > > > > > > the beginning, and now everyone's Traffic Ops
> > > could be
> > > > > > > littered with
> > > > > > > > duplicate origins. Also, depending on the
> > config
> > > of the
> > > > > > > duplicate
> > > > > > > > delivery services, the origins might not be in
> > > conflict at
> > > > > > > all (if
> > > > > > > > they don't have different topology
> > constraints).
> > > I would
> > > > > > > love for us
> > > > > > > > to just add a uniqueness constraint, but there
> > > would need
> > > > > > to
> > > > > > > be a fair
> > > > > > > > amount of warning to the community before doing
> > > so and
> > > > > > might
> > > > > > > > invalidate a significant amount of valid use
> > > cases.
> > > > > > > Operators would
> > > > > > > > need time to make DNS CNAME records for the
> > > duplicate
> > > > > > > origins and
> > > > > > > > update their DSes to use the different CNAMEs.
> > > > > > > >
> > > > > > > > I think as a good first step to eliminating the
> > > use of
> > > > > > > duplicate
> > > > > > > > origins altogether, we should identify which
> > > "topology
> > > > > > > constraints"
> > > > > > > > actually cause conflicting config when used
> > with
> > > duplicate
> > > > > > > origins and
> > > > > > > > prevent creating DSes with duplicate origins
> > _if
> > > it would
> > > > > > > cause a
> > > > > > > > conflict with an existing DS that uses the same
> > > origin_.
> > > > > > > >
> > > > > > > > For instance, I believe an HTTP and DNS-type DS
> > > can live
> > > > > > > happily
> > > > > > > > side-by-side using the same origin (probably
> > > need different
> > > > > > > > routing_names?), but scenarios like HTTP and
> > > HTTP_LIVE, or
> > > > > > > DNS and
> > > > > > > > HTTP_NO_CACHE sharing the same origin will
> > cause
> > > conflicts
> > > > > > > for sure.
> > > > > > > > So maybe we can start by making sure the DS
> > > types "match"
> > > > > > > when using
> > > > > > > > the same origin:
> > > > > > > > HTTP + DNS: possibly good, if they have
> > > different routing
> > > > > > > names?
> > > > > > > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > > > > > > HTTP_NO_CACHE + [any other type]: bad
> > > > > > > > HTTP_LIVE + HTTP: bad
> > > > > > > > etc.
> > > > > > > >
> > > > > > > > There are most likely other conflict scenarios
> > > that don't
> > > > > > > involve the
> > > > > > > > DS types, but I think this would be a good
> > > start. In the
> > > > > > > future with
> > > > > > > > Delivery Service Topologies (aka Flexible
> > > Cachegroups aka
> > > > > > > Bring Your
> > > > > > > > Own Topology), we might be able to prohibit
> > > assigning a DS
> > > > > > > to a
> > > > > > > > Topology if the DS's origin is already used by
> > > another DS
> > > > > > in
> > > > > > > a
> > > > > > > > different Topology.
> > > > > > > >
> > > > > > > > - Rawlin
> > > > > > > >
> > > > > > > > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> > > > > > > > <Br...@comcast.com> wrote:
> > > > > > > > >
> > > > > > > > > As some of you may be aware, `parent.config`
> > > files
> > > > > > > generated by Traffic Ops can vary wildly when an origin is
> > > assigned to
> > > > > > > multiple Delivery Services. This results in undefined
> > > behavior. I'm told
> > > > > > > that the conflict only happens when two Delivery Services
> > with
> > > different
> > > > > > > "topology requirements" use the same origin, whatever that
> > > means (content
> > > > > > > routing type?). Regardless, the issue should be addressed.
> > The
> > > obvious
> > > > > > > solution is to put in place a database constraint that
> > > prevents an origin
> > > > > > > from being assigned to more that one Delivery Service with
> > API
> > > checks in
> > > > > > > place that would provide helpful error messages when an
> > > attempt is made
> > > > > > to
> > > > > > > violate the constraint. However, would that mess with things
> > > like
> > > > > > > Multi-Site Origin? Or is it just not viable for some other
> > > reason? If it
> > > > > > is
> > > > > > > a good solution, I'm prepared to work on a fix that utilizes
> > > it.
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > >
> >
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Jeremy Mitchell <mi...@gmail.com>.
Not sure TP is the right place for a warning like "clean up this
'duplicate' origin or your next upgrade will fail". Most users of our
system will be like "not my problem".
On Wed, Dec 19, 2018 at 11:58 AM Fieck, Brennan <Br...@comcast.com>
wrote:
> Probably. It would impact load times by a bit, but the page for an
> individual object is not our bottleneck.
> ________________________________________
> From: Robert Butts <ro...@apache.org>
> Sent: Wednesday, December 19, 2018 11:50 AM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> produces indeterminate parent.config
>
> > - Including a warning on startup and an API constraint preventing adding
> more bad data in the next 3.0.0 Release Candidate
> > - Adding a database constraint immediately into master that won't be
> cherry-picked into 3.0.0 but should be included in 3.1.0
>
> +1
>
> I understand Jonathan's objection, but at some point, we have to be able to
> move forward. This is a good compromise: deprecate, then remove. That gives
> people a full major version to fix their data.
>
> I would be ideal if it were more than just a logged warning, though. Can we
> add a big red banner in Traffic Portal, on the Delivery Service page for
> any DS with a duplicate origin, telling users to fix it, and that they
> won't be able to upgrade to the next major version until it's fixed?
>
>
> On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <Brennan_Fieck@comcast.com
> >
> wrote:
>
> > So it seems like nobody has a problem with the "how" - disallowing
> > duplicate origin FQDNs on Delivery Services - but we never reached a
> > consensus on "when".
> >
> > I stand by my previous proposal:
> > - Including a warning on startup and an API constraint preventing adding
> > more bad data in the next 3.0.0 Release Candidate
> > - Adding a database constraint immediately into master that won't be
> > cherry-picked into 3.0.0 but should be included in 3.1.0
> > ________________________________________
> > From: Rawlin Peters <ra...@gmail.com>
> > Sent: Tuesday, December 18, 2018 4:59 PM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > produces indeterminate parent.config
> >
> > Also, building more around DS types will make it even harder to get
> > away from DS types in the future too, which I know is something we've
> > discussed on this mailing list before. It also adds to the overhead of
> > Delivery Service Topologies, since a lot of the DS types won't
> > carryover into that world.
> >
> > - Rawlin
> >
> > On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
> > <Br...@comcast.com> wrote:
> > >
> > > +1.
> > > If there's a simple way to work around duplicate origins being
> > prohibited,
> > > then we should rely on that instead of "enumerating all those possible
> > conflicting
> > > settings, which are not only highly complex and confusing, but also
> > further
> > > entrench us in only supporting ATS as a caching proxy (hurting efforts
> to
> > > integrate e.g. Grove, nginx etc.)
> > > ________________________________________
> > > From: Rawlin Peters <ra...@gmail.com>
> > > Sent: Tuesday, December 18, 2018 2:20 PM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > Services produces indeterminate parent.config
> > >
> > > There are a number of different DS settings at play that can
> > > potentially cause conflicts. The question is: do we want to get into
> > > the business of enumerating all those possible conflicting settings or
> > > just simply prohibit duplicate origins altogether? I think we can dig
> > > in and get that "sufficiently advanced sql query" to check for
> > > conflicting origins, but is that something we want to carry along for
> > > the foreseeable future? Aren't CNAMEs relatively cheaper than
> > > developing and maintaining that code and the mental overhead required
> > > in understanding why you're getting an error that says your requested
> > > DS would cause an origin conflict? I think at the point you've
> > > requested a DS that would create a conflict, you've chosen those
> > > settings for a reason and would probably prefer to just create/use a
> > > CNAME in your new DS and keep the rest of your settings the same.
> > >
> > > Thinking in terms of errors, I'm imagining:
> > > "cannot create delivery service: origin fqdn 'foo.example.com' already
> > in use"
> > > vs
> > > "cannot create delivery service: origin fqdn 'foo.example.com' already
> > > in use as type DNS_LIVE_NATNL, which is incompatible with your chosen
> > > type of HTTP_NO_CACHE"
> > >
> > > At that point you'd probably say to yourself, "uh, I need
> > > HTTP_NO_CACHE, so what am I supposed to do now?"
> > >
> > > As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
> > > because the resulting code will be simpler, but I think eliminating
> > > the mental overhead for operators could be worthwhile too. If we can
> > > agree on an end state of prohibiting duplicate origins altogether, we
> > > can start working on a design to smoothly transition us to that point.
> > > Are we willing to live with "just CNAME your origin fqdn" as the
> > > standard solution to duplicates?
> > >
> > > - Rawlin
> > >
> > >
> > > On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
> > > <De...@comcast.com> wrote:
> > > >
> > > > The only situation in which they can share origins is if a) the
> > origins are shared in an MSO configuration but still have different
> defined
> > origin fields in the delivery service, or if they're assigned to
> completely
> > different cachegroups. It's when two delivery services share the same
> > edges that there's an issue, because you end up with parent.config
> issues.
> > Actually you could even get away with it in mids as long as you weren't
> > doing anything like MSO to it.
> > > >
> > > > Could get messy real fast, though. Best to just create a second
> FQDN.
> > > >
> > > > Derek
> > > >
> > > > On 12/18/18, 3:23 PM, "Fieck, Brennan" <Br...@comcast.com>
> > wrote:
> > > >
> > > > So no two Delivery Services may share an origin *regardless of
> > cache hierarchy* ? I've been told that DNS Delivery Services can have the
> > same origin as HTTP Delivery Services because they obey the same cache
> > hierarchy. You're saying that would still produce invalid output and/or
> is
> > explicitly disallowed by ATS?
> > > > ________________________________________
> > > > From: Robert Butts <ro...@apache.org>
> > > > Sent: Tuesday, December 18, 2018 1:09 PM
> > > > To: dev@trafficcontrol.apache.org
> > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > Services produces indeterminate parent.config
> > > >
> > > > >can you give an example of what parent.config looks like when 2
> > ds's share
> > > > an origin and have different a different topology?
> > > >
> > > > Answering because I encountered this directly, when rewriting
> > parent.config.
> > > >
> > > > For example: Suppose you have one Delivery Service:
> > > > XML_ID: foo
> > > > Type: HTPT_LIVE_NATL
> > > > Query String Handling: 1 - ignore in cache key, and pass up
> > > > Origin Server Base URL: http://foo.example.net
> > > >
> > > > And another Delivery Service:
> > > > XML_ID: bar
> > > > Type: HTPT_LIVE_NATL
> > > > Query String Handling: 1 - ignore in cache key, and pass up
> > > > Origin Server Base URL: http://foo.example.net
> > > >
> > > > ATS only supports unique `dest_domain` entries in parent.config.
> > Therefore,
> > > > the parent.config generated for a server assigned to both of
> these
> > Delivery
> > > > Services with either be:
> > > >
> > > > dest_domain=foo.example.net port=80 go_direct=true
> > > >
> > > > Or
> > > >
> > > > dest_domain=foo.example.net port=80 go_direct=false
> > qstring=consider
> > > >
> > > > Right now, it's arbitrary which Perl Traffic Ops inserts, and
> Perl
> > provides
> > > > no warning or error of any kind (the pending Go parent.config PR
> > logs an
> > > > error).
> > > >
> > > > Whichever is arbitrarily inserted, the resulting remaps for the
> > other
> > > > delivery service will be wrong. Either `foo` requests will drop
> > the query
> > > > string when they shouldn't, and go to the mid when they
> shouldn't;
> > or `bar`
> > > > requests will use the query string and skip the mid when it
> > shouldn't.
> > > >
> > > >
> > > > Does that make sense? The only correct solution, is to somehow
> > prevent
> > > > different DSes having the same origin, and tell tenants they must
> > use
> > > > CNAMEs if they need.
> > > >
> > > > This isn't a bug in Traffic Control. ATS fundamentally doesn't
> > support
> > > > multiple remap rules with the same parent FQDN with different
> > > > configurations. Hence, Traffic Control needs to prohibit that.
> > > >
> > > >
> > > > On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
> > mitchell852@gmail.com>
> > > > wrote:
> > > >
> > > > > brennan,
> > > > >
> > > > > can you give an example of what parent.config looks like when 2
> > ds's share
> > > > > an origin and have different a different topology?
> > > > >
> > > > > jeremy
> > > > >
> > > > > On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
> > Brennan_Fieck@comcast.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > To be clear, the "Warning" I'm talking about would happen at
> > startup, but
> > > > > > I'd like a UI-only constraint to come with that to disallow
> > using the API
> > > > > > to bind the same origin to multiple Delivery Services with
> > varying
> > > > > > topography requirements. It wouldn't change the existing
> data,
> > but
> > > > > prevent
> > > > > > users from creating more bad data.
> > > > > >
> > > > > > "warning" doesn't really sufficiently describe that, my bad.
> > > > > > ________________________________________
> > > > > > From: Fieck, Brennan <Br...@comcast.com>
> > > > > > Sent: Tuesday, December 18, 2018 11:24 AM
> > > > > > To: dev@trafficcontrol.apache.org
> > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > Delivery Services
> > > > > > produces indeterminate parent.config
> > > > > >
> > > > > > Well the cost of fixing this bug is a constraint on the data.
> > Unless we
> > > > > > make it a UI-only constraint - which I'm personally against -
> > there must
> > > > > be
> > > > > > some point in the future where ATC cannot reasonably be
> > expected to work
> > > > > > with data that violates that constraint. The question is when
> > that should
> > > > > > occur, which should likely happen at a minor version release.
> > Minor not
> > > > > > major because it doesn't involve a change in data structures,
> > merely
> > > > > > relationships between them - in my opinion that's a minor
> > version change
> > > > > > but that's definitely up for debate. With several release
> > candidates for
> > > > > > 3.0.0 that _doesn't_ include this restriction already in the
> > wild, I
> > > > > > wouldn't recommend putting it in there. That means to fix the
> > bug as soon
> > > > > > as possible it should go in 3.1.0 which should be the target
> > of "master"
> > > > > > after the 3.0.0 release is cut from it.
> > > > > >
> > > > > > So I'd recommend immediately implementing the constraint in
> > master with a
> > > > > > refusal to upgrade with bad data, and backport a warning
> about
> > the future
> > > > > > behavior into 3.0.0 or as part of a 3.0.1 provided we had
> more
> > changes
> > > > > that
> > > > > > would warrant a micro version bump.
> > > > > > ________________________________________
> > > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > > Sent: Tuesday, December 18, 2018 9:34 AM
> > > > > > To: dev@trafficcontrol.apache.org
> > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > Delivery Services
> > > > > > produces indeterminate parent.config
> > > > > >
> > > > > > -1 Holding an ATC upgrade hostage to data cleanup seems like
> a
> > bad idea.
> > > > > > The issue isn't great, but it's also not new. We should
> allow
> > teams to
> > > > > fix
> > > > > > their data at their normal paces if it doesn't create
> > significant
> > > > > overhead
> > > > > > or an inherant blocker for new functionality or correction of
> > other major
> > > > > > problems imho.
> > > > > >
> > > > > > Jonathan G
> > > > > >
> > > > > >
> > > > > > On 12/18/18, 9:28 AM, "Fieck, Brennan" <
> > Brennan_Fieck@comcast.com>
> > > > > wrote:
> > > > > >
> > > > > > Another option is we could detect collisions at startup
> > and simply
> > > > > > refuse to continue with the upgrade until the data is fixed.
> > That would
> > > > > > allow people using the now-unsupported data format to
> continue
> > to use
> > > > > their
> > > > > > old versions of Traffic Ops without wrecking their database,
> > but also
> > > > > > provide an incentive to clean up the data.
> > > > > > ________________________________________
> > > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > > Sent: Tuesday, December 18, 2018 5:12 AM
> > > > > > To: dev@trafficcontrol.apache.org
> > > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> > Delivery
> > > > > > Services produces indeterminate parent.config
> > > > > >
> > > > > > I'm generally a fan of constrain your data in your
> > database, but not
> > > > > > necessarily exclusively. I see this as a one-way
> > cleanup/conversion so
> > > > > it
> > > > > > doesn't need to be configurable; otherwise you have to ask
> the
> > question
> > > > > > what happens if someone turns it off. That said, something
> in
> > the UI
> > > > > layer
> > > > > > would be nice to prevent spending significant quantities of
> > time
> > > > > building a
> > > > > > complex DS only to have it fail to post for reasons that
> could
> > have been
> > > > > > known earlier.
> > > > > >
> > > > > > The way my brain works in this case:
> > > > > > If !unique_constraint_exists_query()
> > > > > > If has_duplicates_query()
> > > > > > show_warning()
> > > > > > else
> > > > > > add_unique_constraint()
> > > > > >
> > > > > > to which the API and UI configuration could also make use
> > of
> > > > > > unique_constraint_exists_query() to drive additional layer
> > constraints if
> > > > > > desired.
> > > > > >
> > > > > > Jonathan G
> > > > > >
> > > > > > On 12/17/18, 1:11 PM, "Rawlin Peters" <
> > rawlin.peters@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > That is an interesting idea...detect at TO startup
> > whether or not
> > > > > > there are duplicate origins and operate in a "prevent
> > duplicate
> > > > > > origins" state if no duplicates are found or "prevent
> > conflicting
> > > > > > DS
> > > > > > topologies" state if duplicates are found? So once
> > operators have
> > > > > > replaced all the duplicate origins with CNAMEs, TO
> will
> > > > > essentially
> > > > > > operate in a "prohibit all duplicate origins" state.
> > That would
> > > > > > probably make for a simpler transition, but I'd want
> > to remove
> > > > > that
> > > > > > logic in a following release that strictly prohibits
> > duplicate
> > > > > > origins
> > > > > > (assuming that the community agrees we should
> prohibit
> > duplicate
> > > > > > origins altogether).
> > > > > >
> > > > > > As for DB constraints vs UI, I was thinking those
> > DS-type
> > > > > > constraints
> > > > > > I pointed out would live in the API. It would
> > basically be added
> > > > > > validation in the deliveryservices POST/PUT endpoint
> > that checks
> > > > > > the
> > > > > > DB for existing DSes that conflict with the requested
> > DS.
> > > > > >
> > > > > > - Rawlin
> > > > > >
> > > > > > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > > > > > <Jo...@comcast.com> wrote:
> > > > > > >
> > > > > > > These kinds of conditions should be detectable
> with a
> > > > > > sufficiently advanced SQL query. Is it possible to add the
> > constraint if
> > > > > > it passes and emit a warning during TO startup otherwise?
> > That would let
> > > > > > you know the condition exists at startup but not getting in
> > your way and
> > > > > > keep you out of trouble once you've cleaned up. We made a
> > mistake early
> > > > > > on, but this would acknowledge it was bad and encourage it to
> > be fixed at
> > > > > > the speed of operations teams. Also this puts the constraint
> > in the
> > > > > > database rather than the UI which is really where the
> > contention is for
> > > > > > usability.
> > > > > > >
> > > > > > > Jonathan G
> > > > > > >
> > > > > > >
> > > > > > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> > > > > rawlin.peters@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > We occasionally discuss this issue but haven't
> > tackled it
> > > > > > yet. I think
> > > > > > > the main issue is just that duplicate origins
> > have been
> > > > > > allowed since
> > > > > > > the beginning, and now everyone's Traffic Ops
> > could be
> > > > > > littered with
> > > > > > > duplicate origins. Also, depending on the
> config
> > of the
> > > > > > duplicate
> > > > > > > delivery services, the origins might not be in
> > conflict at
> > > > > > all (if
> > > > > > > they don't have different topology
> constraints).
> > I would
> > > > > > love for us
> > > > > > > to just add a uniqueness constraint, but there
> > would need
> > > > > to
> > > > > > be a fair
> > > > > > > amount of warning to the community before doing
> > so and
> > > > > might
> > > > > > > invalidate a significant amount of valid use
> > cases.
> > > > > > Operators would
> > > > > > > need time to make DNS CNAME records for the
> > duplicate
> > > > > > origins and
> > > > > > > update their DSes to use the different CNAMEs.
> > > > > > >
> > > > > > > I think as a good first step to eliminating the
> > use of
> > > > > > duplicate
> > > > > > > origins altogether, we should identify which
> > "topology
> > > > > > constraints"
> > > > > > > actually cause conflicting config when used
> with
> > duplicate
> > > > > > origins and
> > > > > > > prevent creating DSes with duplicate origins
> _if
> > it would
> > > > > > cause a
> > > > > > > conflict with an existing DS that uses the same
> > origin_.
> > > > > > >
> > > > > > > For instance, I believe an HTTP and DNS-type DS
> > can live
> > > > > > happily
> > > > > > > side-by-side using the same origin (probably
> > need different
> > > > > > > routing_names?), but scenarios like HTTP and
> > HTTP_LIVE, or
> > > > > > DNS and
> > > > > > > HTTP_NO_CACHE sharing the same origin will
> cause
> > conflicts
> > > > > > for sure.
> > > > > > > So maybe we can start by making sure the DS
> > types "match"
> > > > > > when using
> > > > > > > the same origin:
> > > > > > > HTTP + DNS: possibly good, if they have
> > different routing
> > > > > > names?
> > > > > > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > > > > > HTTP_NO_CACHE + [any other type]: bad
> > > > > > > HTTP_LIVE + HTTP: bad
> > > > > > > etc.
> > > > > > >
> > > > > > > There are most likely other conflict scenarios
> > that don't
> > > > > > involve the
> > > > > > > DS types, but I think this would be a good
> > start. In the
> > > > > > future with
> > > > > > > Delivery Service Topologies (aka Flexible
> > Cachegroups aka
> > > > > > Bring Your
> > > > > > > Own Topology), we might be able to prohibit
> > assigning a DS
> > > > > > to a
> > > > > > > Topology if the DS's origin is already used by
> > another DS
> > > > > in
> > > > > > a
> > > > > > > different Topology.
> > > > > > >
> > > > > > > - Rawlin
> > > > > > >
> > > > > > > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> > > > > > > <Br...@comcast.com> wrote:
> > > > > > > >
> > > > > > > > As some of you may be aware, `parent.config`
> > files
> > > > > > generated by Traffic Ops can vary wildly when an origin is
> > assigned to
> > > > > > multiple Delivery Services. This results in undefined
> > behavior. I'm told
> > > > > > that the conflict only happens when two Delivery Services
> with
> > different
> > > > > > "topology requirements" use the same origin, whatever that
> > means (content
> > > > > > routing type?). Regardless, the issue should be addressed.
> The
> > obvious
> > > > > > solution is to put in place a database constraint that
> > prevents an origin
> > > > > > from being assigned to more that one Delivery Service with
> API
> > checks in
> > > > > > place that would provide helpful error messages when an
> > attempt is made
> > > > > to
> > > > > > violate the constraint. However, would that mess with things
> > like
> > > > > > Multi-Site Origin? Or is it just not viable for some other
> > reason? If it
> > > > > is
> > > > > > a good solution, I'm prepared to work on a fix that utilizes
> > it.
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> >
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Fieck, Brennan" <Br...@comcast.com>.
Probably. It would impact load times by a bit, but the page for an individual object is not our bottleneck.
________________________________________
From: Robert Butts <ro...@apache.org>
Sent: Wednesday, December 19, 2018 11:50 AM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
> - Including a warning on startup and an API constraint preventing adding
more bad data in the next 3.0.0 Release Candidate
> - Adding a database constraint immediately into master that won't be
cherry-picked into 3.0.0 but should be included in 3.1.0
+1
I understand Jonathan's objection, but at some point, we have to be able to
move forward. This is a good compromise: deprecate, then remove. That gives
people a full major version to fix their data.
I would be ideal if it were more than just a logged warning, though. Can we
add a big red banner in Traffic Portal, on the Delivery Service page for
any DS with a duplicate origin, telling users to fix it, and that they
won't be able to upgrade to the next major version until it's fixed?
On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <Br...@comcast.com>
wrote:
> So it seems like nobody has a problem with the "how" - disallowing
> duplicate origin FQDNs on Delivery Services - but we never reached a
> consensus on "when".
>
> I stand by my previous proposal:
> - Including a warning on startup and an API constraint preventing adding
> more bad data in the next 3.0.0 Release Candidate
> - Adding a database constraint immediately into master that won't be
> cherry-picked into 3.0.0 but should be included in 3.1.0
> ________________________________________
> From: Rawlin Peters <ra...@gmail.com>
> Sent: Tuesday, December 18, 2018 4:59 PM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> produces indeterminate parent.config
>
> Also, building more around DS types will make it even harder to get
> away from DS types in the future too, which I know is something we've
> discussed on this mailing list before. It also adds to the overhead of
> Delivery Service Topologies, since a lot of the DS types won't
> carryover into that world.
>
> - Rawlin
>
> On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
> <Br...@comcast.com> wrote:
> >
> > +1.
> > If there's a simple way to work around duplicate origins being
> prohibited,
> > then we should rely on that instead of "enumerating all those possible
> conflicting
> > settings, which are not only highly complex and confusing, but also
> further
> > entrench us in only supporting ATS as a caching proxy (hurting efforts to
> > integrate e.g. Grove, nginx etc.)
> > ________________________________________
> > From: Rawlin Peters <ra...@gmail.com>
> > Sent: Tuesday, December 18, 2018 2:20 PM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> Services produces indeterminate parent.config
> >
> > There are a number of different DS settings at play that can
> > potentially cause conflicts. The question is: do we want to get into
> > the business of enumerating all those possible conflicting settings or
> > just simply prohibit duplicate origins altogether? I think we can dig
> > in and get that "sufficiently advanced sql query" to check for
> > conflicting origins, but is that something we want to carry along for
> > the foreseeable future? Aren't CNAMEs relatively cheaper than
> > developing and maintaining that code and the mental overhead required
> > in understanding why you're getting an error that says your requested
> > DS would cause an origin conflict? I think at the point you've
> > requested a DS that would create a conflict, you've chosen those
> > settings for a reason and would probably prefer to just create/use a
> > CNAME in your new DS and keep the rest of your settings the same.
> >
> > Thinking in terms of errors, I'm imagining:
> > "cannot create delivery service: origin fqdn 'foo.example.com' already
> in use"
> > vs
> > "cannot create delivery service: origin fqdn 'foo.example.com' already
> > in use as type DNS_LIVE_NATNL, which is incompatible with your chosen
> > type of HTTP_NO_CACHE"
> >
> > At that point you'd probably say to yourself, "uh, I need
> > HTTP_NO_CACHE, so what am I supposed to do now?"
> >
> > As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
> > because the resulting code will be simpler, but I think eliminating
> > the mental overhead for operators could be worthwhile too. If we can
> > agree on an end state of prohibiting duplicate origins altogether, we
> > can start working on a design to smoothly transition us to that point.
> > Are we willing to live with "just CNAME your origin fqdn" as the
> > standard solution to duplicates?
> >
> > - Rawlin
> >
> >
> > On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
> > <De...@comcast.com> wrote:
> > >
> > > The only situation in which they can share origins is if a) the
> origins are shared in an MSO configuration but still have different defined
> origin fields in the delivery service, or if they're assigned to completely
> different cachegroups. It's when two delivery services share the same
> edges that there's an issue, because you end up with parent.config issues.
> Actually you could even get away with it in mids as long as you weren't
> doing anything like MSO to it.
> > >
> > > Could get messy real fast, though. Best to just create a second FQDN.
> > >
> > > Derek
> > >
> > > On 12/18/18, 3:23 PM, "Fieck, Brennan" <Br...@comcast.com>
> wrote:
> > >
> > > So no two Delivery Services may share an origin *regardless of
> cache hierarchy* ? I've been told that DNS Delivery Services can have the
> same origin as HTTP Delivery Services because they obey the same cache
> hierarchy. You're saying that would still produce invalid output and/or is
> explicitly disallowed by ATS?
> > > ________________________________________
> > > From: Robert Butts <ro...@apache.org>
> > > Sent: Tuesday, December 18, 2018 1:09 PM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> Services produces indeterminate parent.config
> > >
> > > >can you give an example of what parent.config looks like when 2
> ds's share
> > > an origin and have different a different topology?
> > >
> > > Answering because I encountered this directly, when rewriting
> parent.config.
> > >
> > > For example: Suppose you have one Delivery Service:
> > > XML_ID: foo
> > > Type: HTPT_LIVE_NATL
> > > Query String Handling: 1 - ignore in cache key, and pass up
> > > Origin Server Base URL: http://foo.example.net
> > >
> > > And another Delivery Service:
> > > XML_ID: bar
> > > Type: HTPT_LIVE_NATL
> > > Query String Handling: 1 - ignore in cache key, and pass up
> > > Origin Server Base URL: http://foo.example.net
> > >
> > > ATS only supports unique `dest_domain` entries in parent.config.
> Therefore,
> > > the parent.config generated for a server assigned to both of these
> Delivery
> > > Services with either be:
> > >
> > > dest_domain=foo.example.net port=80 go_direct=true
> > >
> > > Or
> > >
> > > dest_domain=foo.example.net port=80 go_direct=false
> qstring=consider
> > >
> > > Right now, it's arbitrary which Perl Traffic Ops inserts, and Perl
> provides
> > > no warning or error of any kind (the pending Go parent.config PR
> logs an
> > > error).
> > >
> > > Whichever is arbitrarily inserted, the resulting remaps for the
> other
> > > delivery service will be wrong. Either `foo` requests will drop
> the query
> > > string when they shouldn't, and go to the mid when they shouldn't;
> or `bar`
> > > requests will use the query string and skip the mid when it
> shouldn't.
> > >
> > >
> > > Does that make sense? The only correct solution, is to somehow
> prevent
> > > different DSes having the same origin, and tell tenants they must
> use
> > > CNAMEs if they need.
> > >
> > > This isn't a bug in Traffic Control. ATS fundamentally doesn't
> support
> > > multiple remap rules with the same parent FQDN with different
> > > configurations. Hence, Traffic Control needs to prohibit that.
> > >
> > >
> > > On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
> mitchell852@gmail.com>
> > > wrote:
> > >
> > > > brennan,
> > > >
> > > > can you give an example of what parent.config looks like when 2
> ds's share
> > > > an origin and have different a different topology?
> > > >
> > > > jeremy
> > > >
> > > > On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
> Brennan_Fieck@comcast.com
> > > > >
> > > > wrote:
> > > >
> > > > > To be clear, the "Warning" I'm talking about would happen at
> startup, but
> > > > > I'd like a UI-only constraint to come with that to disallow
> using the API
> > > > > to bind the same origin to multiple Delivery Services with
> varying
> > > > > topography requirements. It wouldn't change the existing data,
> but
> > > > prevent
> > > > > users from creating more bad data.
> > > > >
> > > > > "warning" doesn't really sufficiently describe that, my bad.
> > > > > ________________________________________
> > > > > From: Fieck, Brennan <Br...@comcast.com>
> > > > > Sent: Tuesday, December 18, 2018 11:24 AM
> > > > > To: dev@trafficcontrol.apache.org
> > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> Delivery Services
> > > > > produces indeterminate parent.config
> > > > >
> > > > > Well the cost of fixing this bug is a constraint on the data.
> Unless we
> > > > > make it a UI-only constraint - which I'm personally against -
> there must
> > > > be
> > > > > some point in the future where ATC cannot reasonably be
> expected to work
> > > > > with data that violates that constraint. The question is when
> that should
> > > > > occur, which should likely happen at a minor version release.
> Minor not
> > > > > major because it doesn't involve a change in data structures,
> merely
> > > > > relationships between them - in my opinion that's a minor
> version change
> > > > > but that's definitely up for debate. With several release
> candidates for
> > > > > 3.0.0 that _doesn't_ include this restriction already in the
> wild, I
> > > > > wouldn't recommend putting it in there. That means to fix the
> bug as soon
> > > > > as possible it should go in 3.1.0 which should be the target
> of "master"
> > > > > after the 3.0.0 release is cut from it.
> > > > >
> > > > > So I'd recommend immediately implementing the constraint in
> master with a
> > > > > refusal to upgrade with bad data, and backport a warning about
> the future
> > > > > behavior into 3.0.0 or as part of a 3.0.1 provided we had more
> changes
> > > > that
> > > > > would warrant a micro version bump.
> > > > > ________________________________________
> > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > Sent: Tuesday, December 18, 2018 9:34 AM
> > > > > To: dev@trafficcontrol.apache.org
> > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> Delivery Services
> > > > > produces indeterminate parent.config
> > > > >
> > > > > -1 Holding an ATC upgrade hostage to data cleanup seems like a
> bad idea.
> > > > > The issue isn't great, but it's also not new. We should allow
> teams to
> > > > fix
> > > > > their data at their normal paces if it doesn't create
> significant
> > > > overhead
> > > > > or an inherant blocker for new functionality or correction of
> other major
> > > > > problems imho.
> > > > >
> > > > > Jonathan G
> > > > >
> > > > >
> > > > > On 12/18/18, 9:28 AM, "Fieck, Brennan" <
> Brennan_Fieck@comcast.com>
> > > > wrote:
> > > > >
> > > > > Another option is we could detect collisions at startup
> and simply
> > > > > refuse to continue with the upgrade until the data is fixed.
> That would
> > > > > allow people using the now-unsupported data format to continue
> to use
> > > > their
> > > > > old versions of Traffic Ops without wrecking their database,
> but also
> > > > > provide an incentive to clean up the data.
> > > > > ________________________________________
> > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > Sent: Tuesday, December 18, 2018 5:12 AM
> > > > > To: dev@trafficcontrol.apache.org
> > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> Delivery
> > > > > Services produces indeterminate parent.config
> > > > >
> > > > > I'm generally a fan of constrain your data in your
> database, but not
> > > > > necessarily exclusively. I see this as a one-way
> cleanup/conversion so
> > > > it
> > > > > doesn't need to be configurable; otherwise you have to ask the
> question
> > > > > what happens if someone turns it off. That said, something in
> the UI
> > > > layer
> > > > > would be nice to prevent spending significant quantities of
> time
> > > > building a
> > > > > complex DS only to have it fail to post for reasons that could
> have been
> > > > > known earlier.
> > > > >
> > > > > The way my brain works in this case:
> > > > > If !unique_constraint_exists_query()
> > > > > If has_duplicates_query()
> > > > > show_warning()
> > > > > else
> > > > > add_unique_constraint()
> > > > >
> > > > > to which the API and UI configuration could also make use
> of
> > > > > unique_constraint_exists_query() to drive additional layer
> constraints if
> > > > > desired.
> > > > >
> > > > > Jonathan G
> > > > >
> > > > > On 12/17/18, 1:11 PM, "Rawlin Peters" <
> rawlin.peters@gmail.com>
> > > > wrote:
> > > > >
> > > > > That is an interesting idea...detect at TO startup
> whether or not
> > > > > there are duplicate origins and operate in a "prevent
> duplicate
> > > > > origins" state if no duplicates are found or "prevent
> conflicting
> > > > > DS
> > > > > topologies" state if duplicates are found? So once
> operators have
> > > > > replaced all the duplicate origins with CNAMEs, TO will
> > > > essentially
> > > > > operate in a "prohibit all duplicate origins" state.
> That would
> > > > > probably make for a simpler transition, but I'd want
> to remove
> > > > that
> > > > > logic in a following release that strictly prohibits
> duplicate
> > > > > origins
> > > > > (assuming that the community agrees we should prohibit
> duplicate
> > > > > origins altogether).
> > > > >
> > > > > As for DB constraints vs UI, I was thinking those
> DS-type
> > > > > constraints
> > > > > I pointed out would live in the API. It would
> basically be added
> > > > > validation in the deliveryservices POST/PUT endpoint
> that checks
> > > > > the
> > > > > DB for existing DSes that conflict with the requested
> DS.
> > > > >
> > > > > - Rawlin
> > > > >
> > > > > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > > > > <Jo...@comcast.com> wrote:
> > > > > >
> > > > > > These kinds of conditions should be detectable with a
> > > > > sufficiently advanced SQL query. Is it possible to add the
> constraint if
> > > > > it passes and emit a warning during TO startup otherwise?
> That would let
> > > > > you know the condition exists at startup but not getting in
> your way and
> > > > > keep you out of trouble once you've cleaned up. We made a
> mistake early
> > > > > on, but this would acknowledge it was bad and encourage it to
> be fixed at
> > > > > the speed of operations teams. Also this puts the constraint
> in the
> > > > > database rather than the UI which is really where the
> contention is for
> > > > > usability.
> > > > > >
> > > > > > Jonathan G
> > > > > >
> > > > > >
> > > > > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> > > > rawlin.peters@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > We occasionally discuss this issue but haven't
> tackled it
> > > > > yet. I think
> > > > > > the main issue is just that duplicate origins
> have been
> > > > > allowed since
> > > > > > the beginning, and now everyone's Traffic Ops
> could be
> > > > > littered with
> > > > > > duplicate origins. Also, depending on the config
> of the
> > > > > duplicate
> > > > > > delivery services, the origins might not be in
> conflict at
> > > > > all (if
> > > > > > they don't have different topology constraints).
> I would
> > > > > love for us
> > > > > > to just add a uniqueness constraint, but there
> would need
> > > > to
> > > > > be a fair
> > > > > > amount of warning to the community before doing
> so and
> > > > might
> > > > > > invalidate a significant amount of valid use
> cases.
> > > > > Operators would
> > > > > > need time to make DNS CNAME records for the
> duplicate
> > > > > origins and
> > > > > > update their DSes to use the different CNAMEs.
> > > > > >
> > > > > > I think as a good first step to eliminating the
> use of
> > > > > duplicate
> > > > > > origins altogether, we should identify which
> "topology
> > > > > constraints"
> > > > > > actually cause conflicting config when used with
> duplicate
> > > > > origins and
> > > > > > prevent creating DSes with duplicate origins _if
> it would
> > > > > cause a
> > > > > > conflict with an existing DS that uses the same
> origin_.
> > > > > >
> > > > > > For instance, I believe an HTTP and DNS-type DS
> can live
> > > > > happily
> > > > > > side-by-side using the same origin (probably
> need different
> > > > > > routing_names?), but scenarios like HTTP and
> HTTP_LIVE, or
> > > > > DNS and
> > > > > > HTTP_NO_CACHE sharing the same origin will cause
> conflicts
> > > > > for sure.
> > > > > > So maybe we can start by making sure the DS
> types "match"
> > > > > when using
> > > > > > the same origin:
> > > > > > HTTP + DNS: possibly good, if they have
> different routing
> > > > > names?
> > > > > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > > > > HTTP_NO_CACHE + [any other type]: bad
> > > > > > HTTP_LIVE + HTTP: bad
> > > > > > etc.
> > > > > >
> > > > > > There are most likely other conflict scenarios
> that don't
> > > > > involve the
> > > > > > DS types, but I think this would be a good
> start. In the
> > > > > future with
> > > > > > Delivery Service Topologies (aka Flexible
> Cachegroups aka
> > > > > Bring Your
> > > > > > Own Topology), we might be able to prohibit
> assigning a DS
> > > > > to a
> > > > > > Topology if the DS's origin is already used by
> another DS
> > > > in
> > > > > a
> > > > > > different Topology.
> > > > > >
> > > > > > - Rawlin
> > > > > >
> > > > > > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> > > > > > <Br...@comcast.com> wrote:
> > > > > > >
> > > > > > > As some of you may be aware, `parent.config`
> files
> > > > > generated by Traffic Ops can vary wildly when an origin is
> assigned to
> > > > > multiple Delivery Services. This results in undefined
> behavior. I'm told
> > > > > that the conflict only happens when two Delivery Services with
> different
> > > > > "topology requirements" use the same origin, whatever that
> means (content
> > > > > routing type?). Regardless, the issue should be addressed. The
> obvious
> > > > > solution is to put in place a database constraint that
> prevents an origin
> > > > > from being assigned to more that one Delivery Service with API
> checks in
> > > > > place that would provide helpful error messages when an
> attempt is made
> > > > to
> > > > > violate the constraint. However, would that mess with things
> like
> > > > > Multi-Site Origin? Or is it just not viable for some other
> reason? If it
> > > > is
> > > > > a good solution, I'm prepared to work on a fix that utilizes
> it.
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Robert Butts <ro...@apache.org>.
> - Including a warning on startup and an API constraint preventing adding
more bad data in the next 3.0.0 Release Candidate
> - Adding a database constraint immediately into master that won't be
cherry-picked into 3.0.0 but should be included in 3.1.0
+1
I understand Jonathan's objection, but at some point, we have to be able to
move forward. This is a good compromise: deprecate, then remove. That gives
people a full major version to fix their data.
I would be ideal if it were more than just a logged warning, though. Can we
add a big red banner in Traffic Portal, on the Delivery Service page for
any DS with a duplicate origin, telling users to fix it, and that they
won't be able to upgrade to the next major version until it's fixed?
On Wed, Dec 19, 2018 at 10:57 AM Fieck, Brennan <Br...@comcast.com>
wrote:
> So it seems like nobody has a problem with the "how" - disallowing
> duplicate origin FQDNs on Delivery Services - but we never reached a
> consensus on "when".
>
> I stand by my previous proposal:
> - Including a warning on startup and an API constraint preventing adding
> more bad data in the next 3.0.0 Release Candidate
> - Adding a database constraint immediately into master that won't be
> cherry-picked into 3.0.0 but should be included in 3.1.0
> ________________________________________
> From: Rawlin Peters <ra...@gmail.com>
> Sent: Tuesday, December 18, 2018 4:59 PM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> produces indeterminate parent.config
>
> Also, building more around DS types will make it even harder to get
> away from DS types in the future too, which I know is something we've
> discussed on this mailing list before. It also adds to the overhead of
> Delivery Service Topologies, since a lot of the DS types won't
> carryover into that world.
>
> - Rawlin
>
> On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
> <Br...@comcast.com> wrote:
> >
> > +1.
> > If there's a simple way to work around duplicate origins being
> prohibited,
> > then we should rely on that instead of "enumerating all those possible
> conflicting
> > settings, which are not only highly complex and confusing, but also
> further
> > entrench us in only supporting ATS as a caching proxy (hurting efforts to
> > integrate e.g. Grove, nginx etc.)
> > ________________________________________
> > From: Rawlin Peters <ra...@gmail.com>
> > Sent: Tuesday, December 18, 2018 2:20 PM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> Services produces indeterminate parent.config
> >
> > There are a number of different DS settings at play that can
> > potentially cause conflicts. The question is: do we want to get into
> > the business of enumerating all those possible conflicting settings or
> > just simply prohibit duplicate origins altogether? I think we can dig
> > in and get that "sufficiently advanced sql query" to check for
> > conflicting origins, but is that something we want to carry along for
> > the foreseeable future? Aren't CNAMEs relatively cheaper than
> > developing and maintaining that code and the mental overhead required
> > in understanding why you're getting an error that says your requested
> > DS would cause an origin conflict? I think at the point you've
> > requested a DS that would create a conflict, you've chosen those
> > settings for a reason and would probably prefer to just create/use a
> > CNAME in your new DS and keep the rest of your settings the same.
> >
> > Thinking in terms of errors, I'm imagining:
> > "cannot create delivery service: origin fqdn 'foo.example.com' already
> in use"
> > vs
> > "cannot create delivery service: origin fqdn 'foo.example.com' already
> > in use as type DNS_LIVE_NATNL, which is incompatible with your chosen
> > type of HTTP_NO_CACHE"
> >
> > At that point you'd probably say to yourself, "uh, I need
> > HTTP_NO_CACHE, so what am I supposed to do now?"
> >
> > As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
> > because the resulting code will be simpler, but I think eliminating
> > the mental overhead for operators could be worthwhile too. If we can
> > agree on an end state of prohibiting duplicate origins altogether, we
> > can start working on a design to smoothly transition us to that point.
> > Are we willing to live with "just CNAME your origin fqdn" as the
> > standard solution to duplicates?
> >
> > - Rawlin
> >
> >
> > On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
> > <De...@comcast.com> wrote:
> > >
> > > The only situation in which they can share origins is if a) the
> origins are shared in an MSO configuration but still have different defined
> origin fields in the delivery service, or if they're assigned to completely
> different cachegroups. It's when two delivery services share the same
> edges that there's an issue, because you end up with parent.config issues.
> Actually you could even get away with it in mids as long as you weren't
> doing anything like MSO to it.
> > >
> > > Could get messy real fast, though. Best to just create a second FQDN.
> > >
> > > Derek
> > >
> > > On 12/18/18, 3:23 PM, "Fieck, Brennan" <Br...@comcast.com>
> wrote:
> > >
> > > So no two Delivery Services may share an origin *regardless of
> cache hierarchy* ? I've been told that DNS Delivery Services can have the
> same origin as HTTP Delivery Services because they obey the same cache
> hierarchy. You're saying that would still produce invalid output and/or is
> explicitly disallowed by ATS?
> > > ________________________________________
> > > From: Robert Butts <ro...@apache.org>
> > > Sent: Tuesday, December 18, 2018 1:09 PM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> Services produces indeterminate parent.config
> > >
> > > >can you give an example of what parent.config looks like when 2
> ds's share
> > > an origin and have different a different topology?
> > >
> > > Answering because I encountered this directly, when rewriting
> parent.config.
> > >
> > > For example: Suppose you have one Delivery Service:
> > > XML_ID: foo
> > > Type: HTPT_LIVE_NATL
> > > Query String Handling: 1 - ignore in cache key, and pass up
> > > Origin Server Base URL: http://foo.example.net
> > >
> > > And another Delivery Service:
> > > XML_ID: bar
> > > Type: HTPT_LIVE_NATL
> > > Query String Handling: 1 - ignore in cache key, and pass up
> > > Origin Server Base URL: http://foo.example.net
> > >
> > > ATS only supports unique `dest_domain` entries in parent.config.
> Therefore,
> > > the parent.config generated for a server assigned to both of these
> Delivery
> > > Services with either be:
> > >
> > > dest_domain=foo.example.net port=80 go_direct=true
> > >
> > > Or
> > >
> > > dest_domain=foo.example.net port=80 go_direct=false
> qstring=consider
> > >
> > > Right now, it's arbitrary which Perl Traffic Ops inserts, and Perl
> provides
> > > no warning or error of any kind (the pending Go parent.config PR
> logs an
> > > error).
> > >
> > > Whichever is arbitrarily inserted, the resulting remaps for the
> other
> > > delivery service will be wrong. Either `foo` requests will drop
> the query
> > > string when they shouldn't, and go to the mid when they shouldn't;
> or `bar`
> > > requests will use the query string and skip the mid when it
> shouldn't.
> > >
> > >
> > > Does that make sense? The only correct solution, is to somehow
> prevent
> > > different DSes having the same origin, and tell tenants they must
> use
> > > CNAMEs if they need.
> > >
> > > This isn't a bug in Traffic Control. ATS fundamentally doesn't
> support
> > > multiple remap rules with the same parent FQDN with different
> > > configurations. Hence, Traffic Control needs to prohibit that.
> > >
> > >
> > > On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <
> mitchell852@gmail.com>
> > > wrote:
> > >
> > > > brennan,
> > > >
> > > > can you give an example of what parent.config looks like when 2
> ds's share
> > > > an origin and have different a different topology?
> > > >
> > > > jeremy
> > > >
> > > > On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <
> Brennan_Fieck@comcast.com
> > > > >
> > > > wrote:
> > > >
> > > > > To be clear, the "Warning" I'm talking about would happen at
> startup, but
> > > > > I'd like a UI-only constraint to come with that to disallow
> using the API
> > > > > to bind the same origin to multiple Delivery Services with
> varying
> > > > > topography requirements. It wouldn't change the existing data,
> but
> > > > prevent
> > > > > users from creating more bad data.
> > > > >
> > > > > "warning" doesn't really sufficiently describe that, my bad.
> > > > > ________________________________________
> > > > > From: Fieck, Brennan <Br...@comcast.com>
> > > > > Sent: Tuesday, December 18, 2018 11:24 AM
> > > > > To: dev@trafficcontrol.apache.org
> > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> Delivery Services
> > > > > produces indeterminate parent.config
> > > > >
> > > > > Well the cost of fixing this bug is a constraint on the data.
> Unless we
> > > > > make it a UI-only constraint - which I'm personally against -
> there must
> > > > be
> > > > > some point in the future where ATC cannot reasonably be
> expected to work
> > > > > with data that violates that constraint. The question is when
> that should
> > > > > occur, which should likely happen at a minor version release.
> Minor not
> > > > > major because it doesn't involve a change in data structures,
> merely
> > > > > relationships between them - in my opinion that's a minor
> version change
> > > > > but that's definitely up for debate. With several release
> candidates for
> > > > > 3.0.0 that _doesn't_ include this restriction already in the
> wild, I
> > > > > wouldn't recommend putting it in there. That means to fix the
> bug as soon
> > > > > as possible it should go in 3.1.0 which should be the target
> of "master"
> > > > > after the 3.0.0 release is cut from it.
> > > > >
> > > > > So I'd recommend immediately implementing the constraint in
> master with a
> > > > > refusal to upgrade with bad data, and backport a warning about
> the future
> > > > > behavior into 3.0.0 or as part of a 3.0.1 provided we had more
> changes
> > > > that
> > > > > would warrant a micro version bump.
> > > > > ________________________________________
> > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > Sent: Tuesday, December 18, 2018 9:34 AM
> > > > > To: dev@trafficcontrol.apache.org
> > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> Delivery Services
> > > > > produces indeterminate parent.config
> > > > >
> > > > > -1 Holding an ATC upgrade hostage to data cleanup seems like a
> bad idea.
> > > > > The issue isn't great, but it's also not new. We should allow
> teams to
> > > > fix
> > > > > their data at their normal paces if it doesn't create
> significant
> > > > overhead
> > > > > or an inherant blocker for new functionality or correction of
> other major
> > > > > problems imho.
> > > > >
> > > > > Jonathan G
> > > > >
> > > > >
> > > > > On 12/18/18, 9:28 AM, "Fieck, Brennan" <
> Brennan_Fieck@comcast.com>
> > > > wrote:
> > > > >
> > > > > Another option is we could detect collisions at startup
> and simply
> > > > > refuse to continue with the upgrade until the data is fixed.
> That would
> > > > > allow people using the now-unsupported data format to continue
> to use
> > > > their
> > > > > old versions of Traffic Ops without wrecking their database,
> but also
> > > > > provide an incentive to clean up the data.
> > > > > ________________________________________
> > > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > > Sent: Tuesday, December 18, 2018 5:12 AM
> > > > > To: dev@trafficcontrol.apache.org
> > > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe
> Delivery
> > > > > Services produces indeterminate parent.config
> > > > >
> > > > > I'm generally a fan of constrain your data in your
> database, but not
> > > > > necessarily exclusively. I see this as a one-way
> cleanup/conversion so
> > > > it
> > > > > doesn't need to be configurable; otherwise you have to ask the
> question
> > > > > what happens if someone turns it off. That said, something in
> the UI
> > > > layer
> > > > > would be nice to prevent spending significant quantities of
> time
> > > > building a
> > > > > complex DS only to have it fail to post for reasons that could
> have been
> > > > > known earlier.
> > > > >
> > > > > The way my brain works in this case:
> > > > > If !unique_constraint_exists_query()
> > > > > If has_duplicates_query()
> > > > > show_warning()
> > > > > else
> > > > > add_unique_constraint()
> > > > >
> > > > > to which the API and UI configuration could also make use
> of
> > > > > unique_constraint_exists_query() to drive additional layer
> constraints if
> > > > > desired.
> > > > >
> > > > > Jonathan G
> > > > >
> > > > > On 12/17/18, 1:11 PM, "Rawlin Peters" <
> rawlin.peters@gmail.com>
> > > > wrote:
> > > > >
> > > > > That is an interesting idea...detect at TO startup
> whether or not
> > > > > there are duplicate origins and operate in a "prevent
> duplicate
> > > > > origins" state if no duplicates are found or "prevent
> conflicting
> > > > > DS
> > > > > topologies" state if duplicates are found? So once
> operators have
> > > > > replaced all the duplicate origins with CNAMEs, TO will
> > > > essentially
> > > > > operate in a "prohibit all duplicate origins" state.
> That would
> > > > > probably make for a simpler transition, but I'd want
> to remove
> > > > that
> > > > > logic in a following release that strictly prohibits
> duplicate
> > > > > origins
> > > > > (assuming that the community agrees we should prohibit
> duplicate
> > > > > origins altogether).
> > > > >
> > > > > As for DB constraints vs UI, I was thinking those
> DS-type
> > > > > constraints
> > > > > I pointed out would live in the API. It would
> basically be added
> > > > > validation in the deliveryservices POST/PUT endpoint
> that checks
> > > > > the
> > > > > DB for existing DSes that conflict with the requested
> DS.
> > > > >
> > > > > - Rawlin
> > > > >
> > > > > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > > > > <Jo...@comcast.com> wrote:
> > > > > >
> > > > > > These kinds of conditions should be detectable with a
> > > > > sufficiently advanced SQL query. Is it possible to add the
> constraint if
> > > > > it passes and emit a warning during TO startup otherwise?
> That would let
> > > > > you know the condition exists at startup but not getting in
> your way and
> > > > > keep you out of trouble once you've cleaned up. We made a
> mistake early
> > > > > on, but this would acknowledge it was bad and encourage it to
> be fixed at
> > > > > the speed of operations teams. Also this puts the constraint
> in the
> > > > > database rather than the UI which is really where the
> contention is for
> > > > > usability.
> > > > > >
> > > > > > Jonathan G
> > > > > >
> > > > > >
> > > > > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> > > > rawlin.peters@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > We occasionally discuss this issue but haven't
> tackled it
> > > > > yet. I think
> > > > > > the main issue is just that duplicate origins
> have been
> > > > > allowed since
> > > > > > the beginning, and now everyone's Traffic Ops
> could be
> > > > > littered with
> > > > > > duplicate origins. Also, depending on the config
> of the
> > > > > duplicate
> > > > > > delivery services, the origins might not be in
> conflict at
> > > > > all (if
> > > > > > they don't have different topology constraints).
> I would
> > > > > love for us
> > > > > > to just add a uniqueness constraint, but there
> would need
> > > > to
> > > > > be a fair
> > > > > > amount of warning to the community before doing
> so and
> > > > might
> > > > > > invalidate a significant amount of valid use
> cases.
> > > > > Operators would
> > > > > > need time to make DNS CNAME records for the
> duplicate
> > > > > origins and
> > > > > > update their DSes to use the different CNAMEs.
> > > > > >
> > > > > > I think as a good first step to eliminating the
> use of
> > > > > duplicate
> > > > > > origins altogether, we should identify which
> "topology
> > > > > constraints"
> > > > > > actually cause conflicting config when used with
> duplicate
> > > > > origins and
> > > > > > prevent creating DSes with duplicate origins _if
> it would
> > > > > cause a
> > > > > > conflict with an existing DS that uses the same
> origin_.
> > > > > >
> > > > > > For instance, I believe an HTTP and DNS-type DS
> can live
> > > > > happily
> > > > > > side-by-side using the same origin (probably
> need different
> > > > > > routing_names?), but scenarios like HTTP and
> HTTP_LIVE, or
> > > > > DNS and
> > > > > > HTTP_NO_CACHE sharing the same origin will cause
> conflicts
> > > > > for sure.
> > > > > > So maybe we can start by making sure the DS
> types "match"
> > > > > when using
> > > > > > the same origin:
> > > > > > HTTP + DNS: possibly good, if they have
> different routing
> > > > > names?
> > > > > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > > > > HTTP_NO_CACHE + [any other type]: bad
> > > > > > HTTP_LIVE + HTTP: bad
> > > > > > etc.
> > > > > >
> > > > > > There are most likely other conflict scenarios
> that don't
> > > > > involve the
> > > > > > DS types, but I think this would be a good
> start. In the
> > > > > future with
> > > > > > Delivery Service Topologies (aka Flexible
> Cachegroups aka
> > > > > Bring Your
> > > > > > Own Topology), we might be able to prohibit
> assigning a DS
> > > > > to a
> > > > > > Topology if the DS's origin is already used by
> another DS
> > > > in
> > > > > a
> > > > > > different Topology.
> > > > > >
> > > > > > - Rawlin
> > > > > >
> > > > > > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> > > > > > <Br...@comcast.com> wrote:
> > > > > > >
> > > > > > > As some of you may be aware, `parent.config`
> files
> > > > > generated by Traffic Ops can vary wildly when an origin is
> assigned to
> > > > > multiple Delivery Services. This results in undefined
> behavior. I'm told
> > > > > that the conflict only happens when two Delivery Services with
> different
> > > > > "topology requirements" use the same origin, whatever that
> means (content
> > > > > routing type?). Regardless, the issue should be addressed. The
> obvious
> > > > > solution is to put in place a database constraint that
> prevents an origin
> > > > > from being assigned to more that one Delivery Service with API
> checks in
> > > > > place that would provide helpful error messages when an
> attempt is made
> > > > to
> > > > > violate the constraint. However, would that mess with things
> like
> > > > > Multi-Site Origin? Or is it just not viable for some other
> reason? If it
> > > > is
> > > > > a good solution, I'm prepared to work on a fix that utilizes
> it.
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Fieck, Brennan" <Br...@comcast.com>.
So it seems like nobody has a problem with the "how" - disallowing duplicate origin FQDNs on Delivery Services - but we never reached a consensus on "when".
I stand by my previous proposal:
- Including a warning on startup and an API constraint preventing adding more bad data in the next 3.0.0 Release Candidate
- Adding a database constraint immediately into master that won't be cherry-picked into 3.0.0 but should be included in 3.1.0
________________________________________
From: Rawlin Peters <ra...@gmail.com>
Sent: Tuesday, December 18, 2018 4:59 PM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
Also, building more around DS types will make it even harder to get
away from DS types in the future too, which I know is something we've
discussed on this mailing list before. It also adds to the overhead of
Delivery Service Topologies, since a lot of the DS types won't
carryover into that world.
- Rawlin
On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
<Br...@comcast.com> wrote:
>
> +1.
> If there's a simple way to work around duplicate origins being prohibited,
> then we should rely on that instead of "enumerating all those possible conflicting
> settings, which are not only highly complex and confusing, but also further
> entrench us in only supporting ATS as a caching proxy (hurting efforts to
> integrate e.g. Grove, nginx etc.)
> ________________________________________
> From: Rawlin Peters <ra...@gmail.com>
> Sent: Tuesday, December 18, 2018 2:20 PM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
>
> There are a number of different DS settings at play that can
> potentially cause conflicts. The question is: do we want to get into
> the business of enumerating all those possible conflicting settings or
> just simply prohibit duplicate origins altogether? I think we can dig
> in and get that "sufficiently advanced sql query" to check for
> conflicting origins, but is that something we want to carry along for
> the foreseeable future? Aren't CNAMEs relatively cheaper than
> developing and maintaining that code and the mental overhead required
> in understanding why you're getting an error that says your requested
> DS would cause an origin conflict? I think at the point you've
> requested a DS that would create a conflict, you've chosen those
> settings for a reason and would probably prefer to just create/use a
> CNAME in your new DS and keep the rest of your settings the same.
>
> Thinking in terms of errors, I'm imagining:
> "cannot create delivery service: origin fqdn 'foo.example.com' already in use"
> vs
> "cannot create delivery service: origin fqdn 'foo.example.com' already
> in use as type DNS_LIVE_NATNL, which is incompatible with your chosen
> type of HTTP_NO_CACHE"
>
> At that point you'd probably say to yourself, "uh, I need
> HTTP_NO_CACHE, so what am I supposed to do now?"
>
> As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
> because the resulting code will be simpler, but I think eliminating
> the mental overhead for operators could be worthwhile too. If we can
> agree on an end state of prohibiting duplicate origins altogether, we
> can start working on a design to smoothly transition us to that point.
> Are we willing to live with "just CNAME your origin fqdn" as the
> standard solution to duplicates?
>
> - Rawlin
>
>
> On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
> <De...@comcast.com> wrote:
> >
> > The only situation in which they can share origins is if a) the origins are shared in an MSO configuration but still have different defined origin fields in the delivery service, or if they're assigned to completely different cachegroups. It's when two delivery services share the same edges that there's an issue, because you end up with parent.config issues. Actually you could even get away with it in mids as long as you weren't doing anything like MSO to it.
> >
> > Could get messy real fast, though. Best to just create a second FQDN.
> >
> > Derek
> >
> > On 12/18/18, 3:23 PM, "Fieck, Brennan" <Br...@comcast.com> wrote:
> >
> > So no two Delivery Services may share an origin *regardless of cache hierarchy* ? I've been told that DNS Delivery Services can have the same origin as HTTP Delivery Services because they obey the same cache hierarchy. You're saying that would still produce invalid output and/or is explicitly disallowed by ATS?
> > ________________________________________
> > From: Robert Butts <ro...@apache.org>
> > Sent: Tuesday, December 18, 2018 1:09 PM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
> >
> > >can you give an example of what parent.config looks like when 2 ds's share
> > an origin and have different a different topology?
> >
> > Answering because I encountered this directly, when rewriting parent.config.
> >
> > For example: Suppose you have one Delivery Service:
> > XML_ID: foo
> > Type: HTPT_LIVE_NATL
> > Query String Handling: 1 - ignore in cache key, and pass up
> > Origin Server Base URL: http://foo.example.net
> >
> > And another Delivery Service:
> > XML_ID: bar
> > Type: HTPT_LIVE_NATL
> > Query String Handling: 1 - ignore in cache key, and pass up
> > Origin Server Base URL: http://foo.example.net
> >
> > ATS only supports unique `dest_domain` entries in parent.config. Therefore,
> > the parent.config generated for a server assigned to both of these Delivery
> > Services with either be:
> >
> > dest_domain=foo.example.net port=80 go_direct=true
> >
> > Or
> >
> > dest_domain=foo.example.net port=80 go_direct=false qstring=consider
> >
> > Right now, it's arbitrary which Perl Traffic Ops inserts, and Perl provides
> > no warning or error of any kind (the pending Go parent.config PR logs an
> > error).
> >
> > Whichever is arbitrarily inserted, the resulting remaps for the other
> > delivery service will be wrong. Either `foo` requests will drop the query
> > string when they shouldn't, and go to the mid when they shouldn't; or `bar`
> > requests will use the query string and skip the mid when it shouldn't.
> >
> >
> > Does that make sense? The only correct solution, is to somehow prevent
> > different DSes having the same origin, and tell tenants they must use
> > CNAMEs if they need.
> >
> > This isn't a bug in Traffic Control. ATS fundamentally doesn't support
> > multiple remap rules with the same parent FQDN with different
> > configurations. Hence, Traffic Control needs to prohibit that.
> >
> >
> > On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <mi...@gmail.com>
> > wrote:
> >
> > > brennan,
> > >
> > > can you give an example of what parent.config looks like when 2 ds's share
> > > an origin and have different a different topology?
> > >
> > > jeremy
> > >
> > > On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <Brennan_Fieck@comcast.com
> > > >
> > > wrote:
> > >
> > > > To be clear, the "Warning" I'm talking about would happen at startup, but
> > > > I'd like a UI-only constraint to come with that to disallow using the API
> > > > to bind the same origin to multiple Delivery Services with varying
> > > > topography requirements. It wouldn't change the existing data, but
> > > prevent
> > > > users from creating more bad data.
> > > >
> > > > "warning" doesn't really sufficiently describe that, my bad.
> > > > ________________________________________
> > > > From: Fieck, Brennan <Br...@comcast.com>
> > > > Sent: Tuesday, December 18, 2018 11:24 AM
> > > > To: dev@trafficcontrol.apache.org
> > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > > > produces indeterminate parent.config
> > > >
> > > > Well the cost of fixing this bug is a constraint on the data. Unless we
> > > > make it a UI-only constraint - which I'm personally against - there must
> > > be
> > > > some point in the future where ATC cannot reasonably be expected to work
> > > > with data that violates that constraint. The question is when that should
> > > > occur, which should likely happen at a minor version release. Minor not
> > > > major because it doesn't involve a change in data structures, merely
> > > > relationships between them - in my opinion that's a minor version change
> > > > but that's definitely up for debate. With several release candidates for
> > > > 3.0.0 that _doesn't_ include this restriction already in the wild, I
> > > > wouldn't recommend putting it in there. That means to fix the bug as soon
> > > > as possible it should go in 3.1.0 which should be the target of "master"
> > > > after the 3.0.0 release is cut from it.
> > > >
> > > > So I'd recommend immediately implementing the constraint in master with a
> > > > refusal to upgrade with bad data, and backport a warning about the future
> > > > behavior into 3.0.0 or as part of a 3.0.1 provided we had more changes
> > > that
> > > > would warrant a micro version bump.
> > > > ________________________________________
> > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > Sent: Tuesday, December 18, 2018 9:34 AM
> > > > To: dev@trafficcontrol.apache.org
> > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > > > produces indeterminate parent.config
> > > >
> > > > -1 Holding an ATC upgrade hostage to data cleanup seems like a bad idea.
> > > > The issue isn't great, but it's also not new. We should allow teams to
> > > fix
> > > > their data at their normal paces if it doesn't create significant
> > > overhead
> > > > or an inherant blocker for new functionality or correction of other major
> > > > problems imho.
> > > >
> > > > Jonathan G
> > > >
> > > >
> > > > On 12/18/18, 9:28 AM, "Fieck, Brennan" <Br...@comcast.com>
> > > wrote:
> > > >
> > > > Another option is we could detect collisions at startup and simply
> > > > refuse to continue with the upgrade until the data is fixed. That would
> > > > allow people using the now-unsupported data format to continue to use
> > > their
> > > > old versions of Traffic Ops without wrecking their database, but also
> > > > provide an incentive to clean up the data.
> > > > ________________________________________
> > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > Sent: Tuesday, December 18, 2018 5:12 AM
> > > > To: dev@trafficcontrol.apache.org
> > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > > > Services produces indeterminate parent.config
> > > >
> > > > I'm generally a fan of constrain your data in your database, but not
> > > > necessarily exclusively. I see this as a one-way cleanup/conversion so
> > > it
> > > > doesn't need to be configurable; otherwise you have to ask the question
> > > > what happens if someone turns it off. That said, something in the UI
> > > layer
> > > > would be nice to prevent spending significant quantities of time
> > > building a
> > > > complex DS only to have it fail to post for reasons that could have been
> > > > known earlier.
> > > >
> > > > The way my brain works in this case:
> > > > If !unique_constraint_exists_query()
> > > > If has_duplicates_query()
> > > > show_warning()
> > > > else
> > > > add_unique_constraint()
> > > >
> > > > to which the API and UI configuration could also make use of
> > > > unique_constraint_exists_query() to drive additional layer constraints if
> > > > desired.
> > > >
> > > > Jonathan G
> > > >
> > > > On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com>
> > > wrote:
> > > >
> > > > That is an interesting idea...detect at TO startup whether or not
> > > > there are duplicate origins and operate in a "prevent duplicate
> > > > origins" state if no duplicates are found or "prevent conflicting
> > > > DS
> > > > topologies" state if duplicates are found? So once operators have
> > > > replaced all the duplicate origins with CNAMEs, TO will
> > > essentially
> > > > operate in a "prohibit all duplicate origins" state. That would
> > > > probably make for a simpler transition, but I'd want to remove
> > > that
> > > > logic in a following release that strictly prohibits duplicate
> > > > origins
> > > > (assuming that the community agrees we should prohibit duplicate
> > > > origins altogether).
> > > >
> > > > As for DB constraints vs UI, I was thinking those DS-type
> > > > constraints
> > > > I pointed out would live in the API. It would basically be added
> > > > validation in the deliveryservices POST/PUT endpoint that checks
> > > > the
> > > > DB for existing DSes that conflict with the requested DS.
> > > >
> > > > - Rawlin
> > > >
> > > > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > > > <Jo...@comcast.com> wrote:
> > > > >
> > > > > These kinds of conditions should be detectable with a
> > > > sufficiently advanced SQL query. Is it possible to add the constraint if
> > > > it passes and emit a warning during TO startup otherwise? That would let
> > > > you know the condition exists at startup but not getting in your way and
> > > > keep you out of trouble once you've cleaned up. We made a mistake early
> > > > on, but this would acknowledge it was bad and encourage it to be fixed at
> > > > the speed of operations teams. Also this puts the constraint in the
> > > > database rather than the UI which is really where the contention is for
> > > > usability.
> > > > >
> > > > > Jonathan G
> > > > >
> > > > >
> > > > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> > > rawlin.peters@gmail.com>
> > > > wrote:
> > > > >
> > > > > We occasionally discuss this issue but haven't tackled it
> > > > yet. I think
> > > > > the main issue is just that duplicate origins have been
> > > > allowed since
> > > > > the beginning, and now everyone's Traffic Ops could be
> > > > littered with
> > > > > duplicate origins. Also, depending on the config of the
> > > > duplicate
> > > > > delivery services, the origins might not be in conflict at
> > > > all (if
> > > > > they don't have different topology constraints). I would
> > > > love for us
> > > > > to just add a uniqueness constraint, but there would need
> > > to
> > > > be a fair
> > > > > amount of warning to the community before doing so and
> > > might
> > > > > invalidate a significant amount of valid use cases.
> > > > Operators would
> > > > > need time to make DNS CNAME records for the duplicate
> > > > origins and
> > > > > update their DSes to use the different CNAMEs.
> > > > >
> > > > > I think as a good first step to eliminating the use of
> > > > duplicate
> > > > > origins altogether, we should identify which "topology
> > > > constraints"
> > > > > actually cause conflicting config when used with duplicate
> > > > origins and
> > > > > prevent creating DSes with duplicate origins _if it would
> > > > cause a
> > > > > conflict with an existing DS that uses the same origin_.
> > > > >
> > > > > For instance, I believe an HTTP and DNS-type DS can live
> > > > happily
> > > > > side-by-side using the same origin (probably need different
> > > > > routing_names?), but scenarios like HTTP and HTTP_LIVE, or
> > > > DNS and
> > > > > HTTP_NO_CACHE sharing the same origin will cause conflicts
> > > > for sure.
> > > > > So maybe we can start by making sure the DS types "match"
> > > > when using
> > > > > the same origin:
> > > > > HTTP + DNS: possibly good, if they have different routing
> > > > names?
> > > > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > > > HTTP_NO_CACHE + [any other type]: bad
> > > > > HTTP_LIVE + HTTP: bad
> > > > > etc.
> > > > >
> > > > > There are most likely other conflict scenarios that don't
> > > > involve the
> > > > > DS types, but I think this would be a good start. In the
> > > > future with
> > > > > Delivery Service Topologies (aka Flexible Cachegroups aka
> > > > Bring Your
> > > > > Own Topology), we might be able to prohibit assigning a DS
> > > > to a
> > > > > Topology if the DS's origin is already used by another DS
> > > in
> > > > a
> > > > > different Topology.
> > > > >
> > > > > - Rawlin
> > > > >
> > > > > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> > > > > <Br...@comcast.com> wrote:
> > > > > >
> > > > > > As some of you may be aware, `parent.config` files
> > > > generated by Traffic Ops can vary wildly when an origin is assigned to
> > > > multiple Delivery Services. This results in undefined behavior. I'm told
> > > > that the conflict only happens when two Delivery Services with different
> > > > "topology requirements" use the same origin, whatever that means (content
> > > > routing type?). Regardless, the issue should be addressed. The obvious
> > > > solution is to put in place a database constraint that prevents an origin
> > > > from being assigned to more that one Delivery Service with API checks in
> > > > place that would provide helpful error messages when an attempt is made
> > > to
> > > > violate the constraint. However, would that mess with things like
> > > > Multi-Site Origin? Or is it just not viable for some other reason? If it
> > > is
> > > > a good solution, I'm prepared to work on a fix that utilizes it.
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> >
> >
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Rawlin Peters <ra...@gmail.com>.
Also, building more around DS types will make it even harder to get
away from DS types in the future too, which I know is something we've
discussed on this mailing list before. It also adds to the overhead of
Delivery Service Topologies, since a lot of the DS types won't
carryover into that world.
- Rawlin
On Tue, Dec 18, 2018 at 2:42 PM Fieck, Brennan
<Br...@comcast.com> wrote:
>
> +1.
> If there's a simple way to work around duplicate origins being prohibited,
> then we should rely on that instead of "enumerating all those possible conflicting
> settings, which are not only highly complex and confusing, but also further
> entrench us in only supporting ATS as a caching proxy (hurting efforts to
> integrate e.g. Grove, nginx etc.)
> ________________________________________
> From: Rawlin Peters <ra...@gmail.com>
> Sent: Tuesday, December 18, 2018 2:20 PM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
>
> There are a number of different DS settings at play that can
> potentially cause conflicts. The question is: do we want to get into
> the business of enumerating all those possible conflicting settings or
> just simply prohibit duplicate origins altogether? I think we can dig
> in and get that "sufficiently advanced sql query" to check for
> conflicting origins, but is that something we want to carry along for
> the foreseeable future? Aren't CNAMEs relatively cheaper than
> developing and maintaining that code and the mental overhead required
> in understanding why you're getting an error that says your requested
> DS would cause an origin conflict? I think at the point you've
> requested a DS that would create a conflict, you've chosen those
> settings for a reason and would probably prefer to just create/use a
> CNAME in your new DS and keep the rest of your settings the same.
>
> Thinking in terms of errors, I'm imagining:
> "cannot create delivery service: origin fqdn 'foo.example.com' already in use"
> vs
> "cannot create delivery service: origin fqdn 'foo.example.com' already
> in use as type DNS_LIVE_NATNL, which is incompatible with your chosen
> type of HTTP_NO_CACHE"
>
> At that point you'd probably say to yourself, "uh, I need
> HTTP_NO_CACHE, so what am I supposed to do now?"
>
> As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
> because the resulting code will be simpler, but I think eliminating
> the mental overhead for operators could be worthwhile too. If we can
> agree on an end state of prohibiting duplicate origins altogether, we
> can start working on a design to smoothly transition us to that point.
> Are we willing to live with "just CNAME your origin fqdn" as the
> standard solution to duplicates?
>
> - Rawlin
>
>
> On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
> <De...@comcast.com> wrote:
> >
> > The only situation in which they can share origins is if a) the origins are shared in an MSO configuration but still have different defined origin fields in the delivery service, or if they're assigned to completely different cachegroups. It's when two delivery services share the same edges that there's an issue, because you end up with parent.config issues. Actually you could even get away with it in mids as long as you weren't doing anything like MSO to it.
> >
> > Could get messy real fast, though. Best to just create a second FQDN.
> >
> > Derek
> >
> > On 12/18/18, 3:23 PM, "Fieck, Brennan" <Br...@comcast.com> wrote:
> >
> > So no two Delivery Services may share an origin *regardless of cache hierarchy* ? I've been told that DNS Delivery Services can have the same origin as HTTP Delivery Services because they obey the same cache hierarchy. You're saying that would still produce invalid output and/or is explicitly disallowed by ATS?
> > ________________________________________
> > From: Robert Butts <ro...@apache.org>
> > Sent: Tuesday, December 18, 2018 1:09 PM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
> >
> > >can you give an example of what parent.config looks like when 2 ds's share
> > an origin and have different a different topology?
> >
> > Answering because I encountered this directly, when rewriting parent.config.
> >
> > For example: Suppose you have one Delivery Service:
> > XML_ID: foo
> > Type: HTPT_LIVE_NATL
> > Query String Handling: 1 - ignore in cache key, and pass up
> > Origin Server Base URL: http://foo.example.net
> >
> > And another Delivery Service:
> > XML_ID: bar
> > Type: HTPT_LIVE_NATL
> > Query String Handling: 1 - ignore in cache key, and pass up
> > Origin Server Base URL: http://foo.example.net
> >
> > ATS only supports unique `dest_domain` entries in parent.config. Therefore,
> > the parent.config generated for a server assigned to both of these Delivery
> > Services with either be:
> >
> > dest_domain=foo.example.net port=80 go_direct=true
> >
> > Or
> >
> > dest_domain=foo.example.net port=80 go_direct=false qstring=consider
> >
> > Right now, it's arbitrary which Perl Traffic Ops inserts, and Perl provides
> > no warning or error of any kind (the pending Go parent.config PR logs an
> > error).
> >
> > Whichever is arbitrarily inserted, the resulting remaps for the other
> > delivery service will be wrong. Either `foo` requests will drop the query
> > string when they shouldn't, and go to the mid when they shouldn't; or `bar`
> > requests will use the query string and skip the mid when it shouldn't.
> >
> >
> > Does that make sense? The only correct solution, is to somehow prevent
> > different DSes having the same origin, and tell tenants they must use
> > CNAMEs if they need.
> >
> > This isn't a bug in Traffic Control. ATS fundamentally doesn't support
> > multiple remap rules with the same parent FQDN with different
> > configurations. Hence, Traffic Control needs to prohibit that.
> >
> >
> > On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <mi...@gmail.com>
> > wrote:
> >
> > > brennan,
> > >
> > > can you give an example of what parent.config looks like when 2 ds's share
> > > an origin and have different a different topology?
> > >
> > > jeremy
> > >
> > > On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <Brennan_Fieck@comcast.com
> > > >
> > > wrote:
> > >
> > > > To be clear, the "Warning" I'm talking about would happen at startup, but
> > > > I'd like a UI-only constraint to come with that to disallow using the API
> > > > to bind the same origin to multiple Delivery Services with varying
> > > > topography requirements. It wouldn't change the existing data, but
> > > prevent
> > > > users from creating more bad data.
> > > >
> > > > "warning" doesn't really sufficiently describe that, my bad.
> > > > ________________________________________
> > > > From: Fieck, Brennan <Br...@comcast.com>
> > > > Sent: Tuesday, December 18, 2018 11:24 AM
> > > > To: dev@trafficcontrol.apache.org
> > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > > > produces indeterminate parent.config
> > > >
> > > > Well the cost of fixing this bug is a constraint on the data. Unless we
> > > > make it a UI-only constraint - which I'm personally against - there must
> > > be
> > > > some point in the future where ATC cannot reasonably be expected to work
> > > > with data that violates that constraint. The question is when that should
> > > > occur, which should likely happen at a minor version release. Minor not
> > > > major because it doesn't involve a change in data structures, merely
> > > > relationships between them - in my opinion that's a minor version change
> > > > but that's definitely up for debate. With several release candidates for
> > > > 3.0.0 that _doesn't_ include this restriction already in the wild, I
> > > > wouldn't recommend putting it in there. That means to fix the bug as soon
> > > > as possible it should go in 3.1.0 which should be the target of "master"
> > > > after the 3.0.0 release is cut from it.
> > > >
> > > > So I'd recommend immediately implementing the constraint in master with a
> > > > refusal to upgrade with bad data, and backport a warning about the future
> > > > behavior into 3.0.0 or as part of a 3.0.1 provided we had more changes
> > > that
> > > > would warrant a micro version bump.
> > > > ________________________________________
> > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > Sent: Tuesday, December 18, 2018 9:34 AM
> > > > To: dev@trafficcontrol.apache.org
> > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > > > produces indeterminate parent.config
> > > >
> > > > -1 Holding an ATC upgrade hostage to data cleanup seems like a bad idea.
> > > > The issue isn't great, but it's also not new. We should allow teams to
> > > fix
> > > > their data at their normal paces if it doesn't create significant
> > > overhead
> > > > or an inherant blocker for new functionality or correction of other major
> > > > problems imho.
> > > >
> > > > Jonathan G
> > > >
> > > >
> > > > On 12/18/18, 9:28 AM, "Fieck, Brennan" <Br...@comcast.com>
> > > wrote:
> > > >
> > > > Another option is we could detect collisions at startup and simply
> > > > refuse to continue with the upgrade until the data is fixed. That would
> > > > allow people using the now-unsupported data format to continue to use
> > > their
> > > > old versions of Traffic Ops without wrecking their database, but also
> > > > provide an incentive to clean up the data.
> > > > ________________________________________
> > > > From: Gray, Jonathan <Jo...@comcast.com>
> > > > Sent: Tuesday, December 18, 2018 5:12 AM
> > > > To: dev@trafficcontrol.apache.org
> > > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > > > Services produces indeterminate parent.config
> > > >
> > > > I'm generally a fan of constrain your data in your database, but not
> > > > necessarily exclusively. I see this as a one-way cleanup/conversion so
> > > it
> > > > doesn't need to be configurable; otherwise you have to ask the question
> > > > what happens if someone turns it off. That said, something in the UI
> > > layer
> > > > would be nice to prevent spending significant quantities of time
> > > building a
> > > > complex DS only to have it fail to post for reasons that could have been
> > > > known earlier.
> > > >
> > > > The way my brain works in this case:
> > > > If !unique_constraint_exists_query()
> > > > If has_duplicates_query()
> > > > show_warning()
> > > > else
> > > > add_unique_constraint()
> > > >
> > > > to which the API and UI configuration could also make use of
> > > > unique_constraint_exists_query() to drive additional layer constraints if
> > > > desired.
> > > >
> > > > Jonathan G
> > > >
> > > > On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com>
> > > wrote:
> > > >
> > > > That is an interesting idea...detect at TO startup whether or not
> > > > there are duplicate origins and operate in a "prevent duplicate
> > > > origins" state if no duplicates are found or "prevent conflicting
> > > > DS
> > > > topologies" state if duplicates are found? So once operators have
> > > > replaced all the duplicate origins with CNAMEs, TO will
> > > essentially
> > > > operate in a "prohibit all duplicate origins" state. That would
> > > > probably make for a simpler transition, but I'd want to remove
> > > that
> > > > logic in a following release that strictly prohibits duplicate
> > > > origins
> > > > (assuming that the community agrees we should prohibit duplicate
> > > > origins altogether).
> > > >
> > > > As for DB constraints vs UI, I was thinking those DS-type
> > > > constraints
> > > > I pointed out would live in the API. It would basically be added
> > > > validation in the deliveryservices POST/PUT endpoint that checks
> > > > the
> > > > DB for existing DSes that conflict with the requested DS.
> > > >
> > > > - Rawlin
> > > >
> > > > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > > > <Jo...@comcast.com> wrote:
> > > > >
> > > > > These kinds of conditions should be detectable with a
> > > > sufficiently advanced SQL query. Is it possible to add the constraint if
> > > > it passes and emit a warning during TO startup otherwise? That would let
> > > > you know the condition exists at startup but not getting in your way and
> > > > keep you out of trouble once you've cleaned up. We made a mistake early
> > > > on, but this would acknowledge it was bad and encourage it to be fixed at
> > > > the speed of operations teams. Also this puts the constraint in the
> > > > database rather than the UI which is really where the contention is for
> > > > usability.
> > > > >
> > > > > Jonathan G
> > > > >
> > > > >
> > > > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> > > rawlin.peters@gmail.com>
> > > > wrote:
> > > > >
> > > > > We occasionally discuss this issue but haven't tackled it
> > > > yet. I think
> > > > > the main issue is just that duplicate origins have been
> > > > allowed since
> > > > > the beginning, and now everyone's Traffic Ops could be
> > > > littered with
> > > > > duplicate origins. Also, depending on the config of the
> > > > duplicate
> > > > > delivery services, the origins might not be in conflict at
> > > > all (if
> > > > > they don't have different topology constraints). I would
> > > > love for us
> > > > > to just add a uniqueness constraint, but there would need
> > > to
> > > > be a fair
> > > > > amount of warning to the community before doing so and
> > > might
> > > > > invalidate a significant amount of valid use cases.
> > > > Operators would
> > > > > need time to make DNS CNAME records for the duplicate
> > > > origins and
> > > > > update their DSes to use the different CNAMEs.
> > > > >
> > > > > I think as a good first step to eliminating the use of
> > > > duplicate
> > > > > origins altogether, we should identify which "topology
> > > > constraints"
> > > > > actually cause conflicting config when used with duplicate
> > > > origins and
> > > > > prevent creating DSes with duplicate origins _if it would
> > > > cause a
> > > > > conflict with an existing DS that uses the same origin_.
> > > > >
> > > > > For instance, I believe an HTTP and DNS-type DS can live
> > > > happily
> > > > > side-by-side using the same origin (probably need different
> > > > > routing_names?), but scenarios like HTTP and HTTP_LIVE, or
> > > > DNS and
> > > > > HTTP_NO_CACHE sharing the same origin will cause conflicts
> > > > for sure.
> > > > > So maybe we can start by making sure the DS types "match"
> > > > when using
> > > > > the same origin:
> > > > > HTTP + DNS: possibly good, if they have different routing
> > > > names?
> > > > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > > > HTTP_NO_CACHE + [any other type]: bad
> > > > > HTTP_LIVE + HTTP: bad
> > > > > etc.
> > > > >
> > > > > There are most likely other conflict scenarios that don't
> > > > involve the
> > > > > DS types, but I think this would be a good start. In the
> > > > future with
> > > > > Delivery Service Topologies (aka Flexible Cachegroups aka
> > > > Bring Your
> > > > > Own Topology), we might be able to prohibit assigning a DS
> > > > to a
> > > > > Topology if the DS's origin is already used by another DS
> > > in
> > > > a
> > > > > different Topology.
> > > > >
> > > > > - Rawlin
> > > > >
> > > > > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> > > > > <Br...@comcast.com> wrote:
> > > > > >
> > > > > > As some of you may be aware, `parent.config` files
> > > > generated by Traffic Ops can vary wildly when an origin is assigned to
> > > > multiple Delivery Services. This results in undefined behavior. I'm told
> > > > that the conflict only happens when two Delivery Services with different
> > > > "topology requirements" use the same origin, whatever that means (content
> > > > routing type?). Regardless, the issue should be addressed. The obvious
> > > > solution is to put in place a database constraint that prevents an origin
> > > > from being assigned to more that one Delivery Service with API checks in
> > > > place that would provide helpful error messages when an attempt is made
> > > to
> > > > violate the constraint. However, would that mess with things like
> > > > Multi-Site Origin? Or is it just not viable for some other reason? If it
> > > is
> > > > a good solution, I'm prepared to work on a fix that utilizes it.
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> >
> >
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Fieck, Brennan" <Br...@comcast.com>.
+1.
If there's a simple way to work around duplicate origins being prohibited,
then we should rely on that instead of "enumerating all those possible conflicting
settings, which are not only highly complex and confusing, but also further
entrench us in only supporting ATS as a caching proxy (hurting efforts to
integrate e.g. Grove, nginx etc.)
________________________________________
From: Rawlin Peters <ra...@gmail.com>
Sent: Tuesday, December 18, 2018 2:20 PM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
There are a number of different DS settings at play that can
potentially cause conflicts. The question is: do we want to get into
the business of enumerating all those possible conflicting settings or
just simply prohibit duplicate origins altogether? I think we can dig
in and get that "sufficiently advanced sql query" to check for
conflicting origins, but is that something we want to carry along for
the foreseeable future? Aren't CNAMEs relatively cheaper than
developing and maintaining that code and the mental overhead required
in understanding why you're getting an error that says your requested
DS would cause an origin conflict? I think at the point you've
requested a DS that would create a conflict, you've chosen those
settings for a reason and would probably prefer to just create/use a
CNAME in your new DS and keep the rest of your settings the same.
Thinking in terms of errors, I'm imagining:
"cannot create delivery service: origin fqdn 'foo.example.com' already in use"
vs
"cannot create delivery service: origin fqdn 'foo.example.com' already
in use as type DNS_LIVE_NATNL, which is incompatible with your chosen
type of HTTP_NO_CACHE"
At that point you'd probably say to yourself, "uh, I need
HTTP_NO_CACHE, so what am I supposed to do now?"
As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
because the resulting code will be simpler, but I think eliminating
the mental overhead for operators could be worthwhile too. If we can
agree on an end state of prohibiting duplicate origins altogether, we
can start working on a design to smoothly transition us to that point.
Are we willing to live with "just CNAME your origin fqdn" as the
standard solution to duplicates?
- Rawlin
On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
<De...@comcast.com> wrote:
>
> The only situation in which they can share origins is if a) the origins are shared in an MSO configuration but still have different defined origin fields in the delivery service, or if they're assigned to completely different cachegroups. It's when two delivery services share the same edges that there's an issue, because you end up with parent.config issues. Actually you could even get away with it in mids as long as you weren't doing anything like MSO to it.
>
> Could get messy real fast, though. Best to just create a second FQDN.
>
> Derek
>
> On 12/18/18, 3:23 PM, "Fieck, Brennan" <Br...@comcast.com> wrote:
>
> So no two Delivery Services may share an origin *regardless of cache hierarchy* ? I've been told that DNS Delivery Services can have the same origin as HTTP Delivery Services because they obey the same cache hierarchy. You're saying that would still produce invalid output and/or is explicitly disallowed by ATS?
> ________________________________________
> From: Robert Butts <ro...@apache.org>
> Sent: Tuesday, December 18, 2018 1:09 PM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
>
> >can you give an example of what parent.config looks like when 2 ds's share
> an origin and have different a different topology?
>
> Answering because I encountered this directly, when rewriting parent.config.
>
> For example: Suppose you have one Delivery Service:
> XML_ID: foo
> Type: HTPT_LIVE_NATL
> Query String Handling: 1 - ignore in cache key, and pass up
> Origin Server Base URL: http://foo.example.net
>
> And another Delivery Service:
> XML_ID: bar
> Type: HTPT_LIVE_NATL
> Query String Handling: 1 - ignore in cache key, and pass up
> Origin Server Base URL: http://foo.example.net
>
> ATS only supports unique `dest_domain` entries in parent.config. Therefore,
> the parent.config generated for a server assigned to both of these Delivery
> Services with either be:
>
> dest_domain=foo.example.net port=80 go_direct=true
>
> Or
>
> dest_domain=foo.example.net port=80 go_direct=false qstring=consider
>
> Right now, it's arbitrary which Perl Traffic Ops inserts, and Perl provides
> no warning or error of any kind (the pending Go parent.config PR logs an
> error).
>
> Whichever is arbitrarily inserted, the resulting remaps for the other
> delivery service will be wrong. Either `foo` requests will drop the query
> string when they shouldn't, and go to the mid when they shouldn't; or `bar`
> requests will use the query string and skip the mid when it shouldn't.
>
>
> Does that make sense? The only correct solution, is to somehow prevent
> different DSes having the same origin, and tell tenants they must use
> CNAMEs if they need.
>
> This isn't a bug in Traffic Control. ATS fundamentally doesn't support
> multiple remap rules with the same parent FQDN with different
> configurations. Hence, Traffic Control needs to prohibit that.
>
>
> On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
> > brennan,
> >
> > can you give an example of what parent.config looks like when 2 ds's share
> > an origin and have different a different topology?
> >
> > jeremy
> >
> > On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <Brennan_Fieck@comcast.com
> > >
> > wrote:
> >
> > > To be clear, the "Warning" I'm talking about would happen at startup, but
> > > I'd like a UI-only constraint to come with that to disallow using the API
> > > to bind the same origin to multiple Delivery Services with varying
> > > topography requirements. It wouldn't change the existing data, but
> > prevent
> > > users from creating more bad data.
> > >
> > > "warning" doesn't really sufficiently describe that, my bad.
> > > ________________________________________
> > > From: Fieck, Brennan <Br...@comcast.com>
> > > Sent: Tuesday, December 18, 2018 11:24 AM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > > produces indeterminate parent.config
> > >
> > > Well the cost of fixing this bug is a constraint on the data. Unless we
> > > make it a UI-only constraint - which I'm personally against - there must
> > be
> > > some point in the future where ATC cannot reasonably be expected to work
> > > with data that violates that constraint. The question is when that should
> > > occur, which should likely happen at a minor version release. Minor not
> > > major because it doesn't involve a change in data structures, merely
> > > relationships between them - in my opinion that's a minor version change
> > > but that's definitely up for debate. With several release candidates for
> > > 3.0.0 that _doesn't_ include this restriction already in the wild, I
> > > wouldn't recommend putting it in there. That means to fix the bug as soon
> > > as possible it should go in 3.1.0 which should be the target of "master"
> > > after the 3.0.0 release is cut from it.
> > >
> > > So I'd recommend immediately implementing the constraint in master with a
> > > refusal to upgrade with bad data, and backport a warning about the future
> > > behavior into 3.0.0 or as part of a 3.0.1 provided we had more changes
> > that
> > > would warrant a micro version bump.
> > > ________________________________________
> > > From: Gray, Jonathan <Jo...@comcast.com>
> > > Sent: Tuesday, December 18, 2018 9:34 AM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > > produces indeterminate parent.config
> > >
> > > -1 Holding an ATC upgrade hostage to data cleanup seems like a bad idea.
> > > The issue isn't great, but it's also not new. We should allow teams to
> > fix
> > > their data at their normal paces if it doesn't create significant
> > overhead
> > > or an inherant blocker for new functionality or correction of other major
> > > problems imho.
> > >
> > > Jonathan G
> > >
> > >
> > > On 12/18/18, 9:28 AM, "Fieck, Brennan" <Br...@comcast.com>
> > wrote:
> > >
> > > Another option is we could detect collisions at startup and simply
> > > refuse to continue with the upgrade until the data is fixed. That would
> > > allow people using the now-unsupported data format to continue to use
> > their
> > > old versions of Traffic Ops without wrecking their database, but also
> > > provide an incentive to clean up the data.
> > > ________________________________________
> > > From: Gray, Jonathan <Jo...@comcast.com>
> > > Sent: Tuesday, December 18, 2018 5:12 AM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > > Services produces indeterminate parent.config
> > >
> > > I'm generally a fan of constrain your data in your database, but not
> > > necessarily exclusively. I see this as a one-way cleanup/conversion so
> > it
> > > doesn't need to be configurable; otherwise you have to ask the question
> > > what happens if someone turns it off. That said, something in the UI
> > layer
> > > would be nice to prevent spending significant quantities of time
> > building a
> > > complex DS only to have it fail to post for reasons that could have been
> > > known earlier.
> > >
> > > The way my brain works in this case:
> > > If !unique_constraint_exists_query()
> > > If has_duplicates_query()
> > > show_warning()
> > > else
> > > add_unique_constraint()
> > >
> > > to which the API and UI configuration could also make use of
> > > unique_constraint_exists_query() to drive additional layer constraints if
> > > desired.
> > >
> > > Jonathan G
> > >
> > > On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com>
> > wrote:
> > >
> > > That is an interesting idea...detect at TO startup whether or not
> > > there are duplicate origins and operate in a "prevent duplicate
> > > origins" state if no duplicates are found or "prevent conflicting
> > > DS
> > > topologies" state if duplicates are found? So once operators have
> > > replaced all the duplicate origins with CNAMEs, TO will
> > essentially
> > > operate in a "prohibit all duplicate origins" state. That would
> > > probably make for a simpler transition, but I'd want to remove
> > that
> > > logic in a following release that strictly prohibits duplicate
> > > origins
> > > (assuming that the community agrees we should prohibit duplicate
> > > origins altogether).
> > >
> > > As for DB constraints vs UI, I was thinking those DS-type
> > > constraints
> > > I pointed out would live in the API. It would basically be added
> > > validation in the deliveryservices POST/PUT endpoint that checks
> > > the
> > > DB for existing DSes that conflict with the requested DS.
> > >
> > > - Rawlin
> > >
> > > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > > <Jo...@comcast.com> wrote:
> > > >
> > > > These kinds of conditions should be detectable with a
> > > sufficiently advanced SQL query. Is it possible to add the constraint if
> > > it passes and emit a warning during TO startup otherwise? That would let
> > > you know the condition exists at startup but not getting in your way and
> > > keep you out of trouble once you've cleaned up. We made a mistake early
> > > on, but this would acknowledge it was bad and encourage it to be fixed at
> > > the speed of operations teams. Also this puts the constraint in the
> > > database rather than the UI which is really where the contention is for
> > > usability.
> > > >
> > > > Jonathan G
> > > >
> > > >
> > > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> > rawlin.peters@gmail.com>
> > > wrote:
> > > >
> > > > We occasionally discuss this issue but haven't tackled it
> > > yet. I think
> > > > the main issue is just that duplicate origins have been
> > > allowed since
> > > > the beginning, and now everyone's Traffic Ops could be
> > > littered with
> > > > duplicate origins. Also, depending on the config of the
> > > duplicate
> > > > delivery services, the origins might not be in conflict at
> > > all (if
> > > > they don't have different topology constraints). I would
> > > love for us
> > > > to just add a uniqueness constraint, but there would need
> > to
> > > be a fair
> > > > amount of warning to the community before doing so and
> > might
> > > > invalidate a significant amount of valid use cases.
> > > Operators would
> > > > need time to make DNS CNAME records for the duplicate
> > > origins and
> > > > update their DSes to use the different CNAMEs.
> > > >
> > > > I think as a good first step to eliminating the use of
> > > duplicate
> > > > origins altogether, we should identify which "topology
> > > constraints"
> > > > actually cause conflicting config when used with duplicate
> > > origins and
> > > > prevent creating DSes with duplicate origins _if it would
> > > cause a
> > > > conflict with an existing DS that uses the same origin_.
> > > >
> > > > For instance, I believe an HTTP and DNS-type DS can live
> > > happily
> > > > side-by-side using the same origin (probably need different
> > > > routing_names?), but scenarios like HTTP and HTTP_LIVE, or
> > > DNS and
> > > > HTTP_NO_CACHE sharing the same origin will cause conflicts
> > > for sure.
> > > > So maybe we can start by making sure the DS types "match"
> > > when using
> > > > the same origin:
> > > > HTTP + DNS: possibly good, if they have different routing
> > > names?
> > > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > > HTTP_NO_CACHE + [any other type]: bad
> > > > HTTP_LIVE + HTTP: bad
> > > > etc.
> > > >
> > > > There are most likely other conflict scenarios that don't
> > > involve the
> > > > DS types, but I think this would be a good start. In the
> > > future with
> > > > Delivery Service Topologies (aka Flexible Cachegroups aka
> > > Bring Your
> > > > Own Topology), we might be able to prohibit assigning a DS
> > > to a
> > > > Topology if the DS's origin is already used by another DS
> > in
> > > a
> > > > different Topology.
> > > >
> > > > - Rawlin
> > > >
> > > > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> > > > <Br...@comcast.com> wrote:
> > > > >
> > > > > As some of you may be aware, `parent.config` files
> > > generated by Traffic Ops can vary wildly when an origin is assigned to
> > > multiple Delivery Services. This results in undefined behavior. I'm told
> > > that the conflict only happens when two Delivery Services with different
> > > "topology requirements" use the same origin, whatever that means (content
> > > routing type?). Regardless, the issue should be addressed. The obvious
> > > solution is to put in place a database constraint that prevents an origin
> > > from being assigned to more that one Delivery Service with API checks in
> > > place that would provide helpful error messages when an attempt is made
> > to
> > > violate the constraint. However, would that mess with things like
> > > Multi-Site Origin? Or is it just not viable for some other reason? If it
> > is
> > > a good solution, I'm prepared to work on a fix that utilizes it.
> > > >
> > > >
> > >
> > >
> > >
> > >
> > >
> >
>
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Rawlin Peters <ra...@gmail.com>.
There are a number of different DS settings at play that can
potentially cause conflicts. The question is: do we want to get into
the business of enumerating all those possible conflicting settings or
just simply prohibit duplicate origins altogether? I think we can dig
in and get that "sufficiently advanced sql query" to check for
conflicting origins, but is that something we want to carry along for
the foreseeable future? Aren't CNAMEs relatively cheaper than
developing and maintaining that code and the mental overhead required
in understanding why you're getting an error that says your requested
DS would cause an origin conflict? I think at the point you've
requested a DS that would create a conflict, you've chosen those
settings for a reason and would probably prefer to just create/use a
CNAME in your new DS and keep the rest of your settings the same.
Thinking in terms of errors, I'm imagining:
"cannot create delivery service: origin fqdn 'foo.example.com' already in use"
vs
"cannot create delivery service: origin fqdn 'foo.example.com' already
in use as type DNS_LIVE_NATNL, which is incompatible with your chosen
type of HTTP_NO_CACHE"
At that point you'd probably say to yourself, "uh, I need
HTTP_NO_CACHE, so what am I supposed to do now?"
As a lazy developer I'm +1 on prohibiting duplicate origin fqdns
because the resulting code will be simpler, but I think eliminating
the mental overhead for operators could be worthwhile too. If we can
agree on an end state of prohibiting duplicate origins altogether, we
can start working on a design to smoothly transition us to that point.
Are we willing to live with "just CNAME your origin fqdn" as the
standard solution to duplicates?
- Rawlin
On Tue, Dec 18, 2018 at 1:27 PM Gelinas, Derek
<De...@comcast.com> wrote:
>
> The only situation in which they can share origins is if a) the origins are shared in an MSO configuration but still have different defined origin fields in the delivery service, or if they're assigned to completely different cachegroups. It's when two delivery services share the same edges that there's an issue, because you end up with parent.config issues. Actually you could even get away with it in mids as long as you weren't doing anything like MSO to it.
>
> Could get messy real fast, though. Best to just create a second FQDN.
>
> Derek
>
> On 12/18/18, 3:23 PM, "Fieck, Brennan" <Br...@comcast.com> wrote:
>
> So no two Delivery Services may share an origin *regardless of cache hierarchy* ? I've been told that DNS Delivery Services can have the same origin as HTTP Delivery Services because they obey the same cache hierarchy. You're saying that would still produce invalid output and/or is explicitly disallowed by ATS?
> ________________________________________
> From: Robert Butts <ro...@apache.org>
> Sent: Tuesday, December 18, 2018 1:09 PM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
>
> >can you give an example of what parent.config looks like when 2 ds's share
> an origin and have different a different topology?
>
> Answering because I encountered this directly, when rewriting parent.config.
>
> For example: Suppose you have one Delivery Service:
> XML_ID: foo
> Type: HTPT_LIVE_NATL
> Query String Handling: 1 - ignore in cache key, and pass up
> Origin Server Base URL: http://foo.example.net
>
> And another Delivery Service:
> XML_ID: bar
> Type: HTPT_LIVE_NATL
> Query String Handling: 1 - ignore in cache key, and pass up
> Origin Server Base URL: http://foo.example.net
>
> ATS only supports unique `dest_domain` entries in parent.config. Therefore,
> the parent.config generated for a server assigned to both of these Delivery
> Services with either be:
>
> dest_domain=foo.example.net port=80 go_direct=true
>
> Or
>
> dest_domain=foo.example.net port=80 go_direct=false qstring=consider
>
> Right now, it's arbitrary which Perl Traffic Ops inserts, and Perl provides
> no warning or error of any kind (the pending Go parent.config PR logs an
> error).
>
> Whichever is arbitrarily inserted, the resulting remaps for the other
> delivery service will be wrong. Either `foo` requests will drop the query
> string when they shouldn't, and go to the mid when they shouldn't; or `bar`
> requests will use the query string and skip the mid when it shouldn't.
>
>
> Does that make sense? The only correct solution, is to somehow prevent
> different DSes having the same origin, and tell tenants they must use
> CNAMEs if they need.
>
> This isn't a bug in Traffic Control. ATS fundamentally doesn't support
> multiple remap rules with the same parent FQDN with different
> configurations. Hence, Traffic Control needs to prohibit that.
>
>
> On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
> > brennan,
> >
> > can you give an example of what parent.config looks like when 2 ds's share
> > an origin and have different a different topology?
> >
> > jeremy
> >
> > On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <Brennan_Fieck@comcast.com
> > >
> > wrote:
> >
> > > To be clear, the "Warning" I'm talking about would happen at startup, but
> > > I'd like a UI-only constraint to come with that to disallow using the API
> > > to bind the same origin to multiple Delivery Services with varying
> > > topography requirements. It wouldn't change the existing data, but
> > prevent
> > > users from creating more bad data.
> > >
> > > "warning" doesn't really sufficiently describe that, my bad.
> > > ________________________________________
> > > From: Fieck, Brennan <Br...@comcast.com>
> > > Sent: Tuesday, December 18, 2018 11:24 AM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > > produces indeterminate parent.config
> > >
> > > Well the cost of fixing this bug is a constraint on the data. Unless we
> > > make it a UI-only constraint - which I'm personally against - there must
> > be
> > > some point in the future where ATC cannot reasonably be expected to work
> > > with data that violates that constraint. The question is when that should
> > > occur, which should likely happen at a minor version release. Minor not
> > > major because it doesn't involve a change in data structures, merely
> > > relationships between them - in my opinion that's a minor version change
> > > but that's definitely up for debate. With several release candidates for
> > > 3.0.0 that _doesn't_ include this restriction already in the wild, I
> > > wouldn't recommend putting it in there. That means to fix the bug as soon
> > > as possible it should go in 3.1.0 which should be the target of "master"
> > > after the 3.0.0 release is cut from it.
> > >
> > > So I'd recommend immediately implementing the constraint in master with a
> > > refusal to upgrade with bad data, and backport a warning about the future
> > > behavior into 3.0.0 or as part of a 3.0.1 provided we had more changes
> > that
> > > would warrant a micro version bump.
> > > ________________________________________
> > > From: Gray, Jonathan <Jo...@comcast.com>
> > > Sent: Tuesday, December 18, 2018 9:34 AM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > > produces indeterminate parent.config
> > >
> > > -1 Holding an ATC upgrade hostage to data cleanup seems like a bad idea.
> > > The issue isn't great, but it's also not new. We should allow teams to
> > fix
> > > their data at their normal paces if it doesn't create significant
> > overhead
> > > or an inherant blocker for new functionality or correction of other major
> > > problems imho.
> > >
> > > Jonathan G
> > >
> > >
> > > On 12/18/18, 9:28 AM, "Fieck, Brennan" <Br...@comcast.com>
> > wrote:
> > >
> > > Another option is we could detect collisions at startup and simply
> > > refuse to continue with the upgrade until the data is fixed. That would
> > > allow people using the now-unsupported data format to continue to use
> > their
> > > old versions of Traffic Ops without wrecking their database, but also
> > > provide an incentive to clean up the data.
> > > ________________________________________
> > > From: Gray, Jonathan <Jo...@comcast.com>
> > > Sent: Tuesday, December 18, 2018 5:12 AM
> > > To: dev@trafficcontrol.apache.org
> > > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > > Services produces indeterminate parent.config
> > >
> > > I'm generally a fan of constrain your data in your database, but not
> > > necessarily exclusively. I see this as a one-way cleanup/conversion so
> > it
> > > doesn't need to be configurable; otherwise you have to ask the question
> > > what happens if someone turns it off. That said, something in the UI
> > layer
> > > would be nice to prevent spending significant quantities of time
> > building a
> > > complex DS only to have it fail to post for reasons that could have been
> > > known earlier.
> > >
> > > The way my brain works in this case:
> > > If !unique_constraint_exists_query()
> > > If has_duplicates_query()
> > > show_warning()
> > > else
> > > add_unique_constraint()
> > >
> > > to which the API and UI configuration could also make use of
> > > unique_constraint_exists_query() to drive additional layer constraints if
> > > desired.
> > >
> > > Jonathan G
> > >
> > > On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com>
> > wrote:
> > >
> > > That is an interesting idea...detect at TO startup whether or not
> > > there are duplicate origins and operate in a "prevent duplicate
> > > origins" state if no duplicates are found or "prevent conflicting
> > > DS
> > > topologies" state if duplicates are found? So once operators have
> > > replaced all the duplicate origins with CNAMEs, TO will
> > essentially
> > > operate in a "prohibit all duplicate origins" state. That would
> > > probably make for a simpler transition, but I'd want to remove
> > that
> > > logic in a following release that strictly prohibits duplicate
> > > origins
> > > (assuming that the community agrees we should prohibit duplicate
> > > origins altogether).
> > >
> > > As for DB constraints vs UI, I was thinking those DS-type
> > > constraints
> > > I pointed out would live in the API. It would basically be added
> > > validation in the deliveryservices POST/PUT endpoint that checks
> > > the
> > > DB for existing DSes that conflict with the requested DS.
> > >
> > > - Rawlin
> > >
> > > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > > <Jo...@comcast.com> wrote:
> > > >
> > > > These kinds of conditions should be detectable with a
> > > sufficiently advanced SQL query. Is it possible to add the constraint if
> > > it passes and emit a warning during TO startup otherwise? That would let
> > > you know the condition exists at startup but not getting in your way and
> > > keep you out of trouble once you've cleaned up. We made a mistake early
> > > on, but this would acknowledge it was bad and encourage it to be fixed at
> > > the speed of operations teams. Also this puts the constraint in the
> > > database rather than the UI which is really where the contention is for
> > > usability.
> > > >
> > > > Jonathan G
> > > >
> > > >
> > > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> > rawlin.peters@gmail.com>
> > > wrote:
> > > >
> > > > We occasionally discuss this issue but haven't tackled it
> > > yet. I think
> > > > the main issue is just that duplicate origins have been
> > > allowed since
> > > > the beginning, and now everyone's Traffic Ops could be
> > > littered with
> > > > duplicate origins. Also, depending on the config of the
> > > duplicate
> > > > delivery services, the origins might not be in conflict at
> > > all (if
> > > > they don't have different topology constraints). I would
> > > love for us
> > > > to just add a uniqueness constraint, but there would need
> > to
> > > be a fair
> > > > amount of warning to the community before doing so and
> > might
> > > > invalidate a significant amount of valid use cases.
> > > Operators would
> > > > need time to make DNS CNAME records for the duplicate
> > > origins and
> > > > update their DSes to use the different CNAMEs.
> > > >
> > > > I think as a good first step to eliminating the use of
> > > duplicate
> > > > origins altogether, we should identify which "topology
> > > constraints"
> > > > actually cause conflicting config when used with duplicate
> > > origins and
> > > > prevent creating DSes with duplicate origins _if it would
> > > cause a
> > > > conflict with an existing DS that uses the same origin_.
> > > >
> > > > For instance, I believe an HTTP and DNS-type DS can live
> > > happily
> > > > side-by-side using the same origin (probably need different
> > > > routing_names?), but scenarios like HTTP and HTTP_LIVE, or
> > > DNS and
> > > > HTTP_NO_CACHE sharing the same origin will cause conflicts
> > > for sure.
> > > > So maybe we can start by making sure the DS types "match"
> > > when using
> > > > the same origin:
> > > > HTTP + DNS: possibly good, if they have different routing
> > > names?
> > > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > > HTTP_NO_CACHE + [any other type]: bad
> > > > HTTP_LIVE + HTTP: bad
> > > > etc.
> > > >
> > > > There are most likely other conflict scenarios that don't
> > > involve the
> > > > DS types, but I think this would be a good start. In the
> > > future with
> > > > Delivery Service Topologies (aka Flexible Cachegroups aka
> > > Bring Your
> > > > Own Topology), we might be able to prohibit assigning a DS
> > > to a
> > > > Topology if the DS's origin is already used by another DS
> > in
> > > a
> > > > different Topology.
> > > >
> > > > - Rawlin
> > > >
> > > > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> > > > <Br...@comcast.com> wrote:
> > > > >
> > > > > As some of you may be aware, `parent.config` files
> > > generated by Traffic Ops can vary wildly when an origin is assigned to
> > > multiple Delivery Services. This results in undefined behavior. I'm told
> > > that the conflict only happens when two Delivery Services with different
> > > "topology requirements" use the same origin, whatever that means (content
> > > routing type?). Regardless, the issue should be addressed. The obvious
> > > solution is to put in place a database constraint that prevents an origin
> > > from being assigned to more that one Delivery Service with API checks in
> > > place that would provide helpful error messages when an attempt is made
> > to
> > > violate the constraint. However, would that mess with things like
> > > Multi-Site Origin? Or is it just not viable for some other reason? If it
> > is
> > > a good solution, I'm prepared to work on a fix that utilizes it.
> > > >
> > > >
> > >
> > >
> > >
> > >
> > >
> >
>
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Gelinas, Derek" <De...@comcast.com>.
The only situation in which they can share origins is if a) the origins are shared in an MSO configuration but still have different defined origin fields in the delivery service, or if they're assigned to completely different cachegroups. It's when two delivery services share the same edges that there's an issue, because you end up with parent.config issues. Actually you could even get away with it in mids as long as you weren't doing anything like MSO to it.
Could get messy real fast, though. Best to just create a second FQDN.
Derek
On 12/18/18, 3:23 PM, "Fieck, Brennan" <Br...@comcast.com> wrote:
So no two Delivery Services may share an origin *regardless of cache hierarchy* ? I've been told that DNS Delivery Services can have the same origin as HTTP Delivery Services because they obey the same cache hierarchy. You're saying that would still produce invalid output and/or is explicitly disallowed by ATS?
________________________________________
From: Robert Butts <ro...@apache.org>
Sent: Tuesday, December 18, 2018 1:09 PM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
>can you give an example of what parent.config looks like when 2 ds's share
an origin and have different a different topology?
Answering because I encountered this directly, when rewriting parent.config.
For example: Suppose you have one Delivery Service:
XML_ID: foo
Type: HTPT_LIVE_NATL
Query String Handling: 1 - ignore in cache key, and pass up
Origin Server Base URL: http://foo.example.net
And another Delivery Service:
XML_ID: bar
Type: HTPT_LIVE_NATL
Query String Handling: 1 - ignore in cache key, and pass up
Origin Server Base URL: http://foo.example.net
ATS only supports unique `dest_domain` entries in parent.config. Therefore,
the parent.config generated for a server assigned to both of these Delivery
Services with either be:
dest_domain=foo.example.net port=80 go_direct=true
Or
dest_domain=foo.example.net port=80 go_direct=false qstring=consider
Right now, it's arbitrary which Perl Traffic Ops inserts, and Perl provides
no warning or error of any kind (the pending Go parent.config PR logs an
error).
Whichever is arbitrarily inserted, the resulting remaps for the other
delivery service will be wrong. Either `foo` requests will drop the query
string when they shouldn't, and go to the mid when they shouldn't; or `bar`
requests will use the query string and skip the mid when it shouldn't.
Does that make sense? The only correct solution, is to somehow prevent
different DSes having the same origin, and tell tenants they must use
CNAMEs if they need.
This isn't a bug in Traffic Control. ATS fundamentally doesn't support
multiple remap rules with the same parent FQDN with different
configurations. Hence, Traffic Control needs to prohibit that.
On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <mi...@gmail.com>
wrote:
> brennan,
>
> can you give an example of what parent.config looks like when 2 ds's share
> an origin and have different a different topology?
>
> jeremy
>
> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <Brennan_Fieck@comcast.com
> >
> wrote:
>
> > To be clear, the "Warning" I'm talking about would happen at startup, but
> > I'd like a UI-only constraint to come with that to disallow using the API
> > to bind the same origin to multiple Delivery Services with varying
> > topography requirements. It wouldn't change the existing data, but
> prevent
> > users from creating more bad data.
> >
> > "warning" doesn't really sufficiently describe that, my bad.
> > ________________________________________
> > From: Fieck, Brennan <Br...@comcast.com>
> > Sent: Tuesday, December 18, 2018 11:24 AM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > produces indeterminate parent.config
> >
> > Well the cost of fixing this bug is a constraint on the data. Unless we
> > make it a UI-only constraint - which I'm personally against - there must
> be
> > some point in the future where ATC cannot reasonably be expected to work
> > with data that violates that constraint. The question is when that should
> > occur, which should likely happen at a minor version release. Minor not
> > major because it doesn't involve a change in data structures, merely
> > relationships between them - in my opinion that's a minor version change
> > but that's definitely up for debate. With several release candidates for
> > 3.0.0 that _doesn't_ include this restriction already in the wild, I
> > wouldn't recommend putting it in there. That means to fix the bug as soon
> > as possible it should go in 3.1.0 which should be the target of "master"
> > after the 3.0.0 release is cut from it.
> >
> > So I'd recommend immediately implementing the constraint in master with a
> > refusal to upgrade with bad data, and backport a warning about the future
> > behavior into 3.0.0 or as part of a 3.0.1 provided we had more changes
> that
> > would warrant a micro version bump.
> > ________________________________________
> > From: Gray, Jonathan <Jo...@comcast.com>
> > Sent: Tuesday, December 18, 2018 9:34 AM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > produces indeterminate parent.config
> >
> > -1 Holding an ATC upgrade hostage to data cleanup seems like a bad idea.
> > The issue isn't great, but it's also not new. We should allow teams to
> fix
> > their data at their normal paces if it doesn't create significant
> overhead
> > or an inherant blocker for new functionality or correction of other major
> > problems imho.
> >
> > Jonathan G
> >
> >
> > On 12/18/18, 9:28 AM, "Fieck, Brennan" <Br...@comcast.com>
> wrote:
> >
> > Another option is we could detect collisions at startup and simply
> > refuse to continue with the upgrade until the data is fixed. That would
> > allow people using the now-unsupported data format to continue to use
> their
> > old versions of Traffic Ops without wrecking their database, but also
> > provide an incentive to clean up the data.
> > ________________________________________
> > From: Gray, Jonathan <Jo...@comcast.com>
> > Sent: Tuesday, December 18, 2018 5:12 AM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > Services produces indeterminate parent.config
> >
> > I'm generally a fan of constrain your data in your database, but not
> > necessarily exclusively. I see this as a one-way cleanup/conversion so
> it
> > doesn't need to be configurable; otherwise you have to ask the question
> > what happens if someone turns it off. That said, something in the UI
> layer
> > would be nice to prevent spending significant quantities of time
> building a
> > complex DS only to have it fail to post for reasons that could have been
> > known earlier.
> >
> > The way my brain works in this case:
> > If !unique_constraint_exists_query()
> > If has_duplicates_query()
> > show_warning()
> > else
> > add_unique_constraint()
> >
> > to which the API and UI configuration could also make use of
> > unique_constraint_exists_query() to drive additional layer constraints if
> > desired.
> >
> > Jonathan G
> >
> > On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com>
> wrote:
> >
> > That is an interesting idea...detect at TO startup whether or not
> > there are duplicate origins and operate in a "prevent duplicate
> > origins" state if no duplicates are found or "prevent conflicting
> > DS
> > topologies" state if duplicates are found? So once operators have
> > replaced all the duplicate origins with CNAMEs, TO will
> essentially
> > operate in a "prohibit all duplicate origins" state. That would
> > probably make for a simpler transition, but I'd want to remove
> that
> > logic in a following release that strictly prohibits duplicate
> > origins
> > (assuming that the community agrees we should prohibit duplicate
> > origins altogether).
> >
> > As for DB constraints vs UI, I was thinking those DS-type
> > constraints
> > I pointed out would live in the API. It would basically be added
> > validation in the deliveryservices POST/PUT endpoint that checks
> > the
> > DB for existing DSes that conflict with the requested DS.
> >
> > - Rawlin
> >
> > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > <Jo...@comcast.com> wrote:
> > >
> > > These kinds of conditions should be detectable with a
> > sufficiently advanced SQL query. Is it possible to add the constraint if
> > it passes and emit a warning during TO startup otherwise? That would let
> > you know the condition exists at startup but not getting in your way and
> > keep you out of trouble once you've cleaned up. We made a mistake early
> > on, but this would acknowledge it was bad and encourage it to be fixed at
> > the speed of operations teams. Also this puts the constraint in the
> > database rather than the UI which is really where the contention is for
> > usability.
> > >
> > > Jonathan G
> > >
> > >
> > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> rawlin.peters@gmail.com>
> > wrote:
> > >
> > > We occasionally discuss this issue but haven't tackled it
> > yet. I think
> > > the main issue is just that duplicate origins have been
> > allowed since
> > > the beginning, and now everyone's Traffic Ops could be
> > littered with
> > > duplicate origins. Also, depending on the config of the
> > duplicate
> > > delivery services, the origins might not be in conflict at
> > all (if
> > > they don't have different topology constraints). I would
> > love for us
> > > to just add a uniqueness constraint, but there would need
> to
> > be a fair
> > > amount of warning to the community before doing so and
> might
> > > invalidate a significant amount of valid use cases.
> > Operators would
> > > need time to make DNS CNAME records for the duplicate
> > origins and
> > > update their DSes to use the different CNAMEs.
> > >
> > > I think as a good first step to eliminating the use of
> > duplicate
> > > origins altogether, we should identify which "topology
> > constraints"
> > > actually cause conflicting config when used with duplicate
> > origins and
> > > prevent creating DSes with duplicate origins _if it would
> > cause a
> > > conflict with an existing DS that uses the same origin_.
> > >
> > > For instance, I believe an HTTP and DNS-type DS can live
> > happily
> > > side-by-side using the same origin (probably need different
> > > routing_names?), but scenarios like HTTP and HTTP_LIVE, or
> > DNS and
> > > HTTP_NO_CACHE sharing the same origin will cause conflicts
> > for sure.
> > > So maybe we can start by making sure the DS types "match"
> > when using
> > > the same origin:
> > > HTTP + DNS: possibly good, if they have different routing
> > names?
> > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > HTTP_NO_CACHE + [any other type]: bad
> > > HTTP_LIVE + HTTP: bad
> > > etc.
> > >
> > > There are most likely other conflict scenarios that don't
> > involve the
> > > DS types, but I think this would be a good start. In the
> > future with
> > > Delivery Service Topologies (aka Flexible Cachegroups aka
> > Bring Your
> > > Own Topology), we might be able to prohibit assigning a DS
> > to a
> > > Topology if the DS's origin is already used by another DS
> in
> > a
> > > different Topology.
> > >
> > > - Rawlin
> > >
> > > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> > > <Br...@comcast.com> wrote:
> > > >
> > > > As some of you may be aware, `parent.config` files
> > generated by Traffic Ops can vary wildly when an origin is assigned to
> > multiple Delivery Services. This results in undefined behavior. I'm told
> > that the conflict only happens when two Delivery Services with different
> > "topology requirements" use the same origin, whatever that means (content
> > routing type?). Regardless, the issue should be addressed. The obvious
> > solution is to put in place a database constraint that prevents an origin
> > from being assigned to more that one Delivery Service with API checks in
> > place that would provide helpful error messages when an attempt is made
> to
> > violate the constraint. However, would that mess with things like
> > Multi-Site Origin? Or is it just not viable for some other reason? If it
> is
> > a good solution, I'm prepared to work on a fix that utilizes it.
> > >
> > >
> >
> >
> >
> >
> >
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Fieck, Brennan" <Br...@comcast.com>.
So no two Delivery Services may share an origin *regardless of cache hierarchy* ? I've been told that DNS Delivery Services can have the same origin as HTTP Delivery Services because they obey the same cache hierarchy. You're saying that would still produce invalid output and/or is explicitly disallowed by ATS?
________________________________________
From: Robert Butts <ro...@apache.org>
Sent: Tuesday, December 18, 2018 1:09 PM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
>can you give an example of what parent.config looks like when 2 ds's share
an origin and have different a different topology?
Answering because I encountered this directly, when rewriting parent.config.
For example: Suppose you have one Delivery Service:
XML_ID: foo
Type: HTPT_LIVE_NATL
Query String Handling: 1 - ignore in cache key, and pass up
Origin Server Base URL: http://foo.example.net
And another Delivery Service:
XML_ID: bar
Type: HTPT_LIVE_NATL
Query String Handling: 1 - ignore in cache key, and pass up
Origin Server Base URL: http://foo.example.net
ATS only supports unique `dest_domain` entries in parent.config. Therefore,
the parent.config generated for a server assigned to both of these Delivery
Services with either be:
dest_domain=foo.example.net port=80 go_direct=true
Or
dest_domain=foo.example.net port=80 go_direct=false qstring=consider
Right now, it's arbitrary which Perl Traffic Ops inserts, and Perl provides
no warning or error of any kind (the pending Go parent.config PR logs an
error).
Whichever is arbitrarily inserted, the resulting remaps for the other
delivery service will be wrong. Either `foo` requests will drop the query
string when they shouldn't, and go to the mid when they shouldn't; or `bar`
requests will use the query string and skip the mid when it shouldn't.
Does that make sense? The only correct solution, is to somehow prevent
different DSes having the same origin, and tell tenants they must use
CNAMEs if they need.
This isn't a bug in Traffic Control. ATS fundamentally doesn't support
multiple remap rules with the same parent FQDN with different
configurations. Hence, Traffic Control needs to prohibit that.
On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <mi...@gmail.com>
wrote:
> brennan,
>
> can you give an example of what parent.config looks like when 2 ds's share
> an origin and have different a different topology?
>
> jeremy
>
> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <Brennan_Fieck@comcast.com
> >
> wrote:
>
> > To be clear, the "Warning" I'm talking about would happen at startup, but
> > I'd like a UI-only constraint to come with that to disallow using the API
> > to bind the same origin to multiple Delivery Services with varying
> > topography requirements. It wouldn't change the existing data, but
> prevent
> > users from creating more bad data.
> >
> > "warning" doesn't really sufficiently describe that, my bad.
> > ________________________________________
> > From: Fieck, Brennan <Br...@comcast.com>
> > Sent: Tuesday, December 18, 2018 11:24 AM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > produces indeterminate parent.config
> >
> > Well the cost of fixing this bug is a constraint on the data. Unless we
> > make it a UI-only constraint - which I'm personally against - there must
> be
> > some point in the future where ATC cannot reasonably be expected to work
> > with data that violates that constraint. The question is when that should
> > occur, which should likely happen at a minor version release. Minor not
> > major because it doesn't involve a change in data structures, merely
> > relationships between them - in my opinion that's a minor version change
> > but that's definitely up for debate. With several release candidates for
> > 3.0.0 that _doesn't_ include this restriction already in the wild, I
> > wouldn't recommend putting it in there. That means to fix the bug as soon
> > as possible it should go in 3.1.0 which should be the target of "master"
> > after the 3.0.0 release is cut from it.
> >
> > So I'd recommend immediately implementing the constraint in master with a
> > refusal to upgrade with bad data, and backport a warning about the future
> > behavior into 3.0.0 or as part of a 3.0.1 provided we had more changes
> that
> > would warrant a micro version bump.
> > ________________________________________
> > From: Gray, Jonathan <Jo...@comcast.com>
> > Sent: Tuesday, December 18, 2018 9:34 AM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > produces indeterminate parent.config
> >
> > -1 Holding an ATC upgrade hostage to data cleanup seems like a bad idea.
> > The issue isn't great, but it's also not new. We should allow teams to
> fix
> > their data at their normal paces if it doesn't create significant
> overhead
> > or an inherant blocker for new functionality or correction of other major
> > problems imho.
> >
> > Jonathan G
> >
> >
> > On 12/18/18, 9:28 AM, "Fieck, Brennan" <Br...@comcast.com>
> wrote:
> >
> > Another option is we could detect collisions at startup and simply
> > refuse to continue with the upgrade until the data is fixed. That would
> > allow people using the now-unsupported data format to continue to use
> their
> > old versions of Traffic Ops without wrecking their database, but also
> > provide an incentive to clean up the data.
> > ________________________________________
> > From: Gray, Jonathan <Jo...@comcast.com>
> > Sent: Tuesday, December 18, 2018 5:12 AM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > Services produces indeterminate parent.config
> >
> > I'm generally a fan of constrain your data in your database, but not
> > necessarily exclusively. I see this as a one-way cleanup/conversion so
> it
> > doesn't need to be configurable; otherwise you have to ask the question
> > what happens if someone turns it off. That said, something in the UI
> layer
> > would be nice to prevent spending significant quantities of time
> building a
> > complex DS only to have it fail to post for reasons that could have been
> > known earlier.
> >
> > The way my brain works in this case:
> > If !unique_constraint_exists_query()
> > If has_duplicates_query()
> > show_warning()
> > else
> > add_unique_constraint()
> >
> > to which the API and UI configuration could also make use of
> > unique_constraint_exists_query() to drive additional layer constraints if
> > desired.
> >
> > Jonathan G
> >
> > On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com>
> wrote:
> >
> > That is an interesting idea...detect at TO startup whether or not
> > there are duplicate origins and operate in a "prevent duplicate
> > origins" state if no duplicates are found or "prevent conflicting
> > DS
> > topologies" state if duplicates are found? So once operators have
> > replaced all the duplicate origins with CNAMEs, TO will
> essentially
> > operate in a "prohibit all duplicate origins" state. That would
> > probably make for a simpler transition, but I'd want to remove
> that
> > logic in a following release that strictly prohibits duplicate
> > origins
> > (assuming that the community agrees we should prohibit duplicate
> > origins altogether).
> >
> > As for DB constraints vs UI, I was thinking those DS-type
> > constraints
> > I pointed out would live in the API. It would basically be added
> > validation in the deliveryservices POST/PUT endpoint that checks
> > the
> > DB for existing DSes that conflict with the requested DS.
> >
> > - Rawlin
> >
> > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > <Jo...@comcast.com> wrote:
> > >
> > > These kinds of conditions should be detectable with a
> > sufficiently advanced SQL query. Is it possible to add the constraint if
> > it passes and emit a warning during TO startup otherwise? That would let
> > you know the condition exists at startup but not getting in your way and
> > keep you out of trouble once you've cleaned up. We made a mistake early
> > on, but this would acknowledge it was bad and encourage it to be fixed at
> > the speed of operations teams. Also this puts the constraint in the
> > database rather than the UI which is really where the contention is for
> > usability.
> > >
> > > Jonathan G
> > >
> > >
> > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> rawlin.peters@gmail.com>
> > wrote:
> > >
> > > We occasionally discuss this issue but haven't tackled it
> > yet. I think
> > > the main issue is just that duplicate origins have been
> > allowed since
> > > the beginning, and now everyone's Traffic Ops could be
> > littered with
> > > duplicate origins. Also, depending on the config of the
> > duplicate
> > > delivery services, the origins might not be in conflict at
> > all (if
> > > they don't have different topology constraints). I would
> > love for us
> > > to just add a uniqueness constraint, but there would need
> to
> > be a fair
> > > amount of warning to the community before doing so and
> might
> > > invalidate a significant amount of valid use cases.
> > Operators would
> > > need time to make DNS CNAME records for the duplicate
> > origins and
> > > update their DSes to use the different CNAMEs.
> > >
> > > I think as a good first step to eliminating the use of
> > duplicate
> > > origins altogether, we should identify which "topology
> > constraints"
> > > actually cause conflicting config when used with duplicate
> > origins and
> > > prevent creating DSes with duplicate origins _if it would
> > cause a
> > > conflict with an existing DS that uses the same origin_.
> > >
> > > For instance, I believe an HTTP and DNS-type DS can live
> > happily
> > > side-by-side using the same origin (probably need different
> > > routing_names?), but scenarios like HTTP and HTTP_LIVE, or
> > DNS and
> > > HTTP_NO_CACHE sharing the same origin will cause conflicts
> > for sure.
> > > So maybe we can start by making sure the DS types "match"
> > when using
> > > the same origin:
> > > HTTP + DNS: possibly good, if they have different routing
> > names?
> > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > HTTP_NO_CACHE + [any other type]: bad
> > > HTTP_LIVE + HTTP: bad
> > > etc.
> > >
> > > There are most likely other conflict scenarios that don't
> > involve the
> > > DS types, but I think this would be a good start. In the
> > future with
> > > Delivery Service Topologies (aka Flexible Cachegroups aka
> > Bring Your
> > > Own Topology), we might be able to prohibit assigning a DS
> > to a
> > > Topology if the DS's origin is already used by another DS
> in
> > a
> > > different Topology.
> > >
> > > - Rawlin
> > >
> > > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> > > <Br...@comcast.com> wrote:
> > > >
> > > > As some of you may be aware, `parent.config` files
> > generated by Traffic Ops can vary wildly when an origin is assigned to
> > multiple Delivery Services. This results in undefined behavior. I'm told
> > that the conflict only happens when two Delivery Services with different
> > "topology requirements" use the same origin, whatever that means (content
> > routing type?). Regardless, the issue should be addressed. The obvious
> > solution is to put in place a database constraint that prevents an origin
> > from being assigned to more that one Delivery Service with API checks in
> > place that would provide helpful error messages when an attempt is made
> to
> > violate the constraint. However, would that mess with things like
> > Multi-Site Origin? Or is it just not viable for some other reason? If it
> is
> > a good solution, I'm prepared to work on a fix that utilizes it.
> > >
> > >
> >
> >
> >
> >
> >
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Robert Butts <ro...@apache.org>.
>can you give an example of what parent.config looks like when 2 ds's share
an origin and have different a different topology?
Answering because I encountered this directly, when rewriting parent.config.
For example: Suppose you have one Delivery Service:
XML_ID: foo
Type: HTPT_LIVE_NATL
Query String Handling: 1 - ignore in cache key, and pass up
Origin Server Base URL: http://foo.example.net
And another Delivery Service:
XML_ID: bar
Type: HTPT_LIVE_NATL
Query String Handling: 1 - ignore in cache key, and pass up
Origin Server Base URL: http://foo.example.net
ATS only supports unique `dest_domain` entries in parent.config. Therefore,
the parent.config generated for a server assigned to both of these Delivery
Services with either be:
dest_domain=foo.example.net port=80 go_direct=true
Or
dest_domain=foo.example.net port=80 go_direct=false qstring=consider
Right now, it's arbitrary which Perl Traffic Ops inserts, and Perl provides
no warning or error of any kind (the pending Go parent.config PR logs an
error).
Whichever is arbitrarily inserted, the resulting remaps for the other
delivery service will be wrong. Either `foo` requests will drop the query
string when they shouldn't, and go to the mid when they shouldn't; or `bar`
requests will use the query string and skip the mid when it shouldn't.
Does that make sense? The only correct solution, is to somehow prevent
different DSes having the same origin, and tell tenants they must use
CNAMEs if they need.
This isn't a bug in Traffic Control. ATS fundamentally doesn't support
multiple remap rules with the same parent FQDN with different
configurations. Hence, Traffic Control needs to prohibit that.
On Tue, Dec 18, 2018 at 12:24 PM Jeremy Mitchell <mi...@gmail.com>
wrote:
> brennan,
>
> can you give an example of what parent.config looks like when 2 ds's share
> an origin and have different a different topology?
>
> jeremy
>
> On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <Brennan_Fieck@comcast.com
> >
> wrote:
>
> > To be clear, the "Warning" I'm talking about would happen at startup, but
> > I'd like a UI-only constraint to come with that to disallow using the API
> > to bind the same origin to multiple Delivery Services with varying
> > topography requirements. It wouldn't change the existing data, but
> prevent
> > users from creating more bad data.
> >
> > "warning" doesn't really sufficiently describe that, my bad.
> > ________________________________________
> > From: Fieck, Brennan <Br...@comcast.com>
> > Sent: Tuesday, December 18, 2018 11:24 AM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > produces indeterminate parent.config
> >
> > Well the cost of fixing this bug is a constraint on the data. Unless we
> > make it a UI-only constraint - which I'm personally against - there must
> be
> > some point in the future where ATC cannot reasonably be expected to work
> > with data that violates that constraint. The question is when that should
> > occur, which should likely happen at a minor version release. Minor not
> > major because it doesn't involve a change in data structures, merely
> > relationships between them - in my opinion that's a minor version change
> > but that's definitely up for debate. With several release candidates for
> > 3.0.0 that _doesn't_ include this restriction already in the wild, I
> > wouldn't recommend putting it in there. That means to fix the bug as soon
> > as possible it should go in 3.1.0 which should be the target of "master"
> > after the 3.0.0 release is cut from it.
> >
> > So I'd recommend immediately implementing the constraint in master with a
> > refusal to upgrade with bad data, and backport a warning about the future
> > behavior into 3.0.0 or as part of a 3.0.1 provided we had more changes
> that
> > would warrant a micro version bump.
> > ________________________________________
> > From: Gray, Jonathan <Jo...@comcast.com>
> > Sent: Tuesday, December 18, 2018 9:34 AM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> > produces indeterminate parent.config
> >
> > -1 Holding an ATC upgrade hostage to data cleanup seems like a bad idea.
> > The issue isn't great, but it's also not new. We should allow teams to
> fix
> > their data at their normal paces if it doesn't create significant
> overhead
> > or an inherant blocker for new functionality or correction of other major
> > problems imho.
> >
> > Jonathan G
> >
> >
> > On 12/18/18, 9:28 AM, "Fieck, Brennan" <Br...@comcast.com>
> wrote:
> >
> > Another option is we could detect collisions at startup and simply
> > refuse to continue with the upgrade until the data is fixed. That would
> > allow people using the now-unsupported data format to continue to use
> their
> > old versions of Traffic Ops without wrecking their database, but also
> > provide an incentive to clean up the data.
> > ________________________________________
> > From: Gray, Jonathan <Jo...@comcast.com>
> > Sent: Tuesday, December 18, 2018 5:12 AM
> > To: dev@trafficcontrol.apache.org
> > Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> > Services produces indeterminate parent.config
> >
> > I'm generally a fan of constrain your data in your database, but not
> > necessarily exclusively. I see this as a one-way cleanup/conversion so
> it
> > doesn't need to be configurable; otherwise you have to ask the question
> > what happens if someone turns it off. That said, something in the UI
> layer
> > would be nice to prevent spending significant quantities of time
> building a
> > complex DS only to have it fail to post for reasons that could have been
> > known earlier.
> >
> > The way my brain works in this case:
> > If !unique_constraint_exists_query()
> > If has_duplicates_query()
> > show_warning()
> > else
> > add_unique_constraint()
> >
> > to which the API and UI configuration could also make use of
> > unique_constraint_exists_query() to drive additional layer constraints if
> > desired.
> >
> > Jonathan G
> >
> > On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com>
> wrote:
> >
> > That is an interesting idea...detect at TO startup whether or not
> > there are duplicate origins and operate in a "prevent duplicate
> > origins" state if no duplicates are found or "prevent conflicting
> > DS
> > topologies" state if duplicates are found? So once operators have
> > replaced all the duplicate origins with CNAMEs, TO will
> essentially
> > operate in a "prohibit all duplicate origins" state. That would
> > probably make for a simpler transition, but I'd want to remove
> that
> > logic in a following release that strictly prohibits duplicate
> > origins
> > (assuming that the community agrees we should prohibit duplicate
> > origins altogether).
> >
> > As for DB constraints vs UI, I was thinking those DS-type
> > constraints
> > I pointed out would live in the API. It would basically be added
> > validation in the deliveryservices POST/PUT endpoint that checks
> > the
> > DB for existing DSes that conflict with the requested DS.
> >
> > - Rawlin
> >
> > On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> > <Jo...@comcast.com> wrote:
> > >
> > > These kinds of conditions should be detectable with a
> > sufficiently advanced SQL query. Is it possible to add the constraint if
> > it passes and emit a warning during TO startup otherwise? That would let
> > you know the condition exists at startup but not getting in your way and
> > keep you out of trouble once you've cleaned up. We made a mistake early
> > on, but this would acknowledge it was bad and encourage it to be fixed at
> > the speed of operations teams. Also this puts the constraint in the
> > database rather than the UI which is really where the contention is for
> > usability.
> > >
> > > Jonathan G
> > >
> > >
> > > On 12/17/18, 11:38 AM, "Rawlin Peters" <
> rawlin.peters@gmail.com>
> > wrote:
> > >
> > > We occasionally discuss this issue but haven't tackled it
> > yet. I think
> > > the main issue is just that duplicate origins have been
> > allowed since
> > > the beginning, and now everyone's Traffic Ops could be
> > littered with
> > > duplicate origins. Also, depending on the config of the
> > duplicate
> > > delivery services, the origins might not be in conflict at
> > all (if
> > > they don't have different topology constraints). I would
> > love for us
> > > to just add a uniqueness constraint, but there would need
> to
> > be a fair
> > > amount of warning to the community before doing so and
> might
> > > invalidate a significant amount of valid use cases.
> > Operators would
> > > need time to make DNS CNAME records for the duplicate
> > origins and
> > > update their DSes to use the different CNAMEs.
> > >
> > > I think as a good first step to eliminating the use of
> > duplicate
> > > origins altogether, we should identify which "topology
> > constraints"
> > > actually cause conflicting config when used with duplicate
> > origins and
> > > prevent creating DSes with duplicate origins _if it would
> > cause a
> > > conflict with an existing DS that uses the same origin_.
> > >
> > > For instance, I believe an HTTP and DNS-type DS can live
> > happily
> > > side-by-side using the same origin (probably need different
> > > routing_names?), but scenarios like HTTP and HTTP_LIVE, or
> > DNS and
> > > HTTP_NO_CACHE sharing the same origin will cause conflicts
> > for sure.
> > > So maybe we can start by making sure the DS types "match"
> > when using
> > > the same origin:
> > > HTTP + DNS: possibly good, if they have different routing
> > names?
> > > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > > HTTP_NO_CACHE + [any other type]: bad
> > > HTTP_LIVE + HTTP: bad
> > > etc.
> > >
> > > There are most likely other conflict scenarios that don't
> > involve the
> > > DS types, but I think this would be a good start. In the
> > future with
> > > Delivery Service Topologies (aka Flexible Cachegroups aka
> > Bring Your
> > > Own Topology), we might be able to prohibit assigning a DS
> > to a
> > > Topology if the DS's origin is already used by another DS
> in
> > a
> > > different Topology.
> > >
> > > - Rawlin
> > >
> > > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> > > <Br...@comcast.com> wrote:
> > > >
> > > > As some of you may be aware, `parent.config` files
> > generated by Traffic Ops can vary wildly when an origin is assigned to
> > multiple Delivery Services. This results in undefined behavior. I'm told
> > that the conflict only happens when two Delivery Services with different
> > "topology requirements" use the same origin, whatever that means (content
> > routing type?). Regardless, the issue should be addressed. The obvious
> > solution is to put in place a database constraint that prevents an origin
> > from being assigned to more that one Delivery Service with API checks in
> > place that would provide helpful error messages when an attempt is made
> to
> > violate the constraint. However, would that mess with things like
> > Multi-Site Origin? Or is it just not viable for some other reason? If it
> is
> > a good solution, I'm prepared to work on a fix that utilizes it.
> > >
> > >
> >
> >
> >
> >
> >
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Jeremy Mitchell <mi...@gmail.com>.
brennan,
can you give an example of what parent.config looks like when 2 ds's share
an origin and have different a different topology?
jeremy
On Tue, Dec 18, 2018 at 11:39 AM Fieck, Brennan <Br...@comcast.com>
wrote:
> To be clear, the "Warning" I'm talking about would happen at startup, but
> I'd like a UI-only constraint to come with that to disallow using the API
> to bind the same origin to multiple Delivery Services with varying
> topography requirements. It wouldn't change the existing data, but prevent
> users from creating more bad data.
>
> "warning" doesn't really sufficiently describe that, my bad.
> ________________________________________
> From: Fieck, Brennan <Br...@comcast.com>
> Sent: Tuesday, December 18, 2018 11:24 AM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> produces indeterminate parent.config
>
> Well the cost of fixing this bug is a constraint on the data. Unless we
> make it a UI-only constraint - which I'm personally against - there must be
> some point in the future where ATC cannot reasonably be expected to work
> with data that violates that constraint. The question is when that should
> occur, which should likely happen at a minor version release. Minor not
> major because it doesn't involve a change in data structures, merely
> relationships between them - in my opinion that's a minor version change
> but that's definitely up for debate. With several release candidates for
> 3.0.0 that _doesn't_ include this restriction already in the wild, I
> wouldn't recommend putting it in there. That means to fix the bug as soon
> as possible it should go in 3.1.0 which should be the target of "master"
> after the 3.0.0 release is cut from it.
>
> So I'd recommend immediately implementing the constraint in master with a
> refusal to upgrade with bad data, and backport a warning about the future
> behavior into 3.0.0 or as part of a 3.0.1 provided we had more changes that
> would warrant a micro version bump.
> ________________________________________
> From: Gray, Jonathan <Jo...@comcast.com>
> Sent: Tuesday, December 18, 2018 9:34 AM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
> produces indeterminate parent.config
>
> -1 Holding an ATC upgrade hostage to data cleanup seems like a bad idea.
> The issue isn't great, but it's also not new. We should allow teams to fix
> their data at their normal paces if it doesn't create significant overhead
> or an inherant blocker for new functionality or correction of other major
> problems imho.
>
> Jonathan G
>
>
> On 12/18/18, 9:28 AM, "Fieck, Brennan" <Br...@comcast.com> wrote:
>
> Another option is we could detect collisions at startup and simply
> refuse to continue with the upgrade until the data is fixed. That would
> allow people using the now-unsupported data format to continue to use their
> old versions of Traffic Ops without wrecking their database, but also
> provide an incentive to clean up the data.
> ________________________________________
> From: Gray, Jonathan <Jo...@comcast.com>
> Sent: Tuesday, December 18, 2018 5:12 AM
> To: dev@trafficcontrol.apache.org
> Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery
> Services produces indeterminate parent.config
>
> I'm generally a fan of constrain your data in your database, but not
> necessarily exclusively. I see this as a one-way cleanup/conversion so it
> doesn't need to be configurable; otherwise you have to ask the question
> what happens if someone turns it off. That said, something in the UI layer
> would be nice to prevent spending significant quantities of time building a
> complex DS only to have it fail to post for reasons that could have been
> known earlier.
>
> The way my brain works in this case:
> If !unique_constraint_exists_query()
> If has_duplicates_query()
> show_warning()
> else
> add_unique_constraint()
>
> to which the API and UI configuration could also make use of
> unique_constraint_exists_query() to drive additional layer constraints if
> desired.
>
> Jonathan G
>
> On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com> wrote:
>
> That is an interesting idea...detect at TO startup whether or not
> there are duplicate origins and operate in a "prevent duplicate
> origins" state if no duplicates are found or "prevent conflicting
> DS
> topologies" state if duplicates are found? So once operators have
> replaced all the duplicate origins with CNAMEs, TO will essentially
> operate in a "prohibit all duplicate origins" state. That would
> probably make for a simpler transition, but I'd want to remove that
> logic in a following release that strictly prohibits duplicate
> origins
> (assuming that the community agrees we should prohibit duplicate
> origins altogether).
>
> As for DB constraints vs UI, I was thinking those DS-type
> constraints
> I pointed out would live in the API. It would basically be added
> validation in the deliveryservices POST/PUT endpoint that checks
> the
> DB for existing DSes that conflict with the requested DS.
>
> - Rawlin
>
> On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> <Jo...@comcast.com> wrote:
> >
> > These kinds of conditions should be detectable with a
> sufficiently advanced SQL query. Is it possible to add the constraint if
> it passes and emit a warning during TO startup otherwise? That would let
> you know the condition exists at startup but not getting in your way and
> keep you out of trouble once you've cleaned up. We made a mistake early
> on, but this would acknowledge it was bad and encourage it to be fixed at
> the speed of operations teams. Also this puts the constraint in the
> database rather than the UI which is really where the contention is for
> usability.
> >
> > Jonathan G
> >
> >
> > On 12/17/18, 11:38 AM, "Rawlin Peters" <ra...@gmail.com>
> wrote:
> >
> > We occasionally discuss this issue but haven't tackled it
> yet. I think
> > the main issue is just that duplicate origins have been
> allowed since
> > the beginning, and now everyone's Traffic Ops could be
> littered with
> > duplicate origins. Also, depending on the config of the
> duplicate
> > delivery services, the origins might not be in conflict at
> all (if
> > they don't have different topology constraints). I would
> love for us
> > to just add a uniqueness constraint, but there would need to
> be a fair
> > amount of warning to the community before doing so and might
> > invalidate a significant amount of valid use cases.
> Operators would
> > need time to make DNS CNAME records for the duplicate
> origins and
> > update their DSes to use the different CNAMEs.
> >
> > I think as a good first step to eliminating the use of
> duplicate
> > origins altogether, we should identify which "topology
> constraints"
> > actually cause conflicting config when used with duplicate
> origins and
> > prevent creating DSes with duplicate origins _if it would
> cause a
> > conflict with an existing DS that uses the same origin_.
> >
> > For instance, I believe an HTTP and DNS-type DS can live
> happily
> > side-by-side using the same origin (probably need different
> > routing_names?), but scenarios like HTTP and HTTP_LIVE, or
> DNS and
> > HTTP_NO_CACHE sharing the same origin will cause conflicts
> for sure.
> > So maybe we can start by making sure the DS types "match"
> when using
> > the same origin:
> > HTTP + DNS: possibly good, if they have different routing
> names?
> > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > HTTP_NO_CACHE + [any other type]: bad
> > HTTP_LIVE + HTTP: bad
> > etc.
> >
> > There are most likely other conflict scenarios that don't
> involve the
> > DS types, but I think this would be a good start. In the
> future with
> > Delivery Service Topologies (aka Flexible Cachegroups aka
> Bring Your
> > Own Topology), we might be able to prohibit assigning a DS
> to a
> > Topology if the DS's origin is already used by another DS in
> a
> > different Topology.
> >
> > - Rawlin
> >
> > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> > <Br...@comcast.com> wrote:
> > >
> > > As some of you may be aware, `parent.config` files
> generated by Traffic Ops can vary wildly when an origin is assigned to
> multiple Delivery Services. This results in undefined behavior. I'm told
> that the conflict only happens when two Delivery Services with different
> "topology requirements" use the same origin, whatever that means (content
> routing type?). Regardless, the issue should be addressed. The obvious
> solution is to put in place a database constraint that prevents an origin
> from being assigned to more that one Delivery Service with API checks in
> place that would provide helpful error messages when an attempt is made to
> violate the constraint. However, would that mess with things like
> Multi-Site Origin? Or is it just not viable for some other reason? If it is
> a good solution, I'm prepared to work on a fix that utilizes it.
> >
> >
>
>
>
>
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Fieck, Brennan" <Br...@comcast.com>.
To be clear, the "Warning" I'm talking about would happen at startup, but I'd like a UI-only constraint to come with that to disallow using the API to bind the same origin to multiple Delivery Services with varying topography requirements. It wouldn't change the existing data, but prevent users from creating more bad data.
"warning" doesn't really sufficiently describe that, my bad.
________________________________________
From: Fieck, Brennan <Br...@comcast.com>
Sent: Tuesday, December 18, 2018 11:24 AM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
Well the cost of fixing this bug is a constraint on the data. Unless we make it a UI-only constraint - which I'm personally against - there must be some point in the future where ATC cannot reasonably be expected to work with data that violates that constraint. The question is when that should occur, which should likely happen at a minor version release. Minor not major because it doesn't involve a change in data structures, merely relationships between them - in my opinion that's a minor version change but that's definitely up for debate. With several release candidates for 3.0.0 that _doesn't_ include this restriction already in the wild, I wouldn't recommend putting it in there. That means to fix the bug as soon as possible it should go in 3.1.0 which should be the target of "master" after the 3.0.0 release is cut from it.
So I'd recommend immediately implementing the constraint in master with a refusal to upgrade with bad data, and backport a warning about the future behavior into 3.0.0 or as part of a 3.0.1 provided we had more changes that would warrant a micro version bump.
________________________________________
From: Gray, Jonathan <Jo...@comcast.com>
Sent: Tuesday, December 18, 2018 9:34 AM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
-1 Holding an ATC upgrade hostage to data cleanup seems like a bad idea. The issue isn't great, but it's also not new. We should allow teams to fix their data at their normal paces if it doesn't create significant overhead or an inherant blocker for new functionality or correction of other major problems imho.
Jonathan G
On 12/18/18, 9:28 AM, "Fieck, Brennan" <Br...@comcast.com> wrote:
Another option is we could detect collisions at startup and simply refuse to continue with the upgrade until the data is fixed. That would allow people using the now-unsupported data format to continue to use their old versions of Traffic Ops without wrecking their database, but also provide an incentive to clean up the data.
________________________________________
From: Gray, Jonathan <Jo...@comcast.com>
Sent: Tuesday, December 18, 2018 5:12 AM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
I'm generally a fan of constrain your data in your database, but not necessarily exclusively. I see this as a one-way cleanup/conversion so it doesn't need to be configurable; otherwise you have to ask the question what happens if someone turns it off. That said, something in the UI layer would be nice to prevent spending significant quantities of time building a complex DS only to have it fail to post for reasons that could have been known earlier.
The way my brain works in this case:
If !unique_constraint_exists_query()
If has_duplicates_query()
show_warning()
else
add_unique_constraint()
to which the API and UI configuration could also make use of unique_constraint_exists_query() to drive additional layer constraints if desired.
Jonathan G
On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com> wrote:
That is an interesting idea...detect at TO startup whether or not
there are duplicate origins and operate in a "prevent duplicate
origins" state if no duplicates are found or "prevent conflicting DS
topologies" state if duplicates are found? So once operators have
replaced all the duplicate origins with CNAMEs, TO will essentially
operate in a "prohibit all duplicate origins" state. That would
probably make for a simpler transition, but I'd want to remove that
logic in a following release that strictly prohibits duplicate origins
(assuming that the community agrees we should prohibit duplicate
origins altogether).
As for DB constraints vs UI, I was thinking those DS-type constraints
I pointed out would live in the API. It would basically be added
validation in the deliveryservices POST/PUT endpoint that checks the
DB for existing DSes that conflict with the requested DS.
- Rawlin
On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
<Jo...@comcast.com> wrote:
>
> These kinds of conditions should be detectable with a sufficiently advanced SQL query. Is it possible to add the constraint if it passes and emit a warning during TO startup otherwise? That would let you know the condition exists at startup but not getting in your way and keep you out of trouble once you've cleaned up. We made a mistake early on, but this would acknowledge it was bad and encourage it to be fixed at the speed of operations teams. Also this puts the constraint in the database rather than the UI which is really where the contention is for usability.
>
> Jonathan G
>
>
> On 12/17/18, 11:38 AM, "Rawlin Peters" <ra...@gmail.com> wrote:
>
> We occasionally discuss this issue but haven't tackled it yet. I think
> the main issue is just that duplicate origins have been allowed since
> the beginning, and now everyone's Traffic Ops could be littered with
> duplicate origins. Also, depending on the config of the duplicate
> delivery services, the origins might not be in conflict at all (if
> they don't have different topology constraints). I would love for us
> to just add a uniqueness constraint, but there would need to be a fair
> amount of warning to the community before doing so and might
> invalidate a significant amount of valid use cases. Operators would
> need time to make DNS CNAME records for the duplicate origins and
> update their DSes to use the different CNAMEs.
>
> I think as a good first step to eliminating the use of duplicate
> origins altogether, we should identify which "topology constraints"
> actually cause conflicting config when used with duplicate origins and
> prevent creating DSes with duplicate origins _if it would cause a
> conflict with an existing DS that uses the same origin_.
>
> For instance, I believe an HTTP and DNS-type DS can live happily
> side-by-side using the same origin (probably need different
> routing_names?), but scenarios like HTTP and HTTP_LIVE, or DNS and
> HTTP_NO_CACHE sharing the same origin will cause conflicts for sure.
> So maybe we can start by making sure the DS types "match" when using
> the same origin:
> HTTP + DNS: possibly good, if they have different routing names?
> HTTP_LIVE + HTTP_LIVE_NATNL: bad
> HTTP_NO_CACHE + [any other type]: bad
> HTTP_LIVE + HTTP: bad
> etc.
>
> There are most likely other conflict scenarios that don't involve the
> DS types, but I think this would be a good start. In the future with
> Delivery Service Topologies (aka Flexible Cachegroups aka Bring Your
> Own Topology), we might be able to prohibit assigning a DS to a
> Topology if the DS's origin is already used by another DS in a
> different Topology.
>
> - Rawlin
>
> On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> <Br...@comcast.com> wrote:
> >
> > As some of you may be aware, `parent.config` files generated by Traffic Ops can vary wildly when an origin is assigned to multiple Delivery Services. This results in undefined behavior. I'm told that the conflict only happens when two Delivery Services with different "topology requirements" use the same origin, whatever that means (content routing type?). Regardless, the issue should be addressed. The obvious solution is to put in place a database constraint that prevents an origin from being assigned to more that one Delivery Service with API checks in place that would provide helpful error messages when an attempt is made to violate the constraint. However, would that mess with things like Multi-Site Origin? Or is it just not viable for some other reason? If it is a good solution, I'm prepared to work on a fix that utilizes it.
>
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Fieck, Brennan" <Br...@comcast.com>.
Well the cost of fixing this bug is a constraint on the data. Unless we make it a UI-only constraint - which I'm personally against - there must be some point in the future where ATC cannot reasonably be expected to work with data that violates that constraint. The question is when that should occur, which should likely happen at a minor version release. Minor not major because it doesn't involve a change in data structures, merely relationships between them - in my opinion that's a minor version change but that's definitely up for debate. With several release candidates for 3.0.0 that _doesn't_ include this restriction already in the wild, I wouldn't recommend putting it in there. That means to fix the bug as soon as possible it should go in 3.1.0 which should be the target of "master" after the 3.0.0 release is cut from it.
So I'd recommend immediately implementing the constraint in master with a refusal to upgrade with bad data, and backport a warning about the future behavior into 3.0.0 or as part of a 3.0.1 provided we had more changes that would warrant a micro version bump.
________________________________________
From: Gray, Jonathan <Jo...@comcast.com>
Sent: Tuesday, December 18, 2018 9:34 AM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
-1 Holding an ATC upgrade hostage to data cleanup seems like a bad idea. The issue isn't great, but it's also not new. We should allow teams to fix their data at their normal paces if it doesn't create significant overhead or an inherant blocker for new functionality or correction of other major problems imho.
Jonathan G
On 12/18/18, 9:28 AM, "Fieck, Brennan" <Br...@comcast.com> wrote:
Another option is we could detect collisions at startup and simply refuse to continue with the upgrade until the data is fixed. That would allow people using the now-unsupported data format to continue to use their old versions of Traffic Ops without wrecking their database, but also provide an incentive to clean up the data.
________________________________________
From: Gray, Jonathan <Jo...@comcast.com>
Sent: Tuesday, December 18, 2018 5:12 AM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
I'm generally a fan of constrain your data in your database, but not necessarily exclusively. I see this as a one-way cleanup/conversion so it doesn't need to be configurable; otherwise you have to ask the question what happens if someone turns it off. That said, something in the UI layer would be nice to prevent spending significant quantities of time building a complex DS only to have it fail to post for reasons that could have been known earlier.
The way my brain works in this case:
If !unique_constraint_exists_query()
If has_duplicates_query()
show_warning()
else
add_unique_constraint()
to which the API and UI configuration could also make use of unique_constraint_exists_query() to drive additional layer constraints if desired.
Jonathan G
On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com> wrote:
That is an interesting idea...detect at TO startup whether or not
there are duplicate origins and operate in a "prevent duplicate
origins" state if no duplicates are found or "prevent conflicting DS
topologies" state if duplicates are found? So once operators have
replaced all the duplicate origins with CNAMEs, TO will essentially
operate in a "prohibit all duplicate origins" state. That would
probably make for a simpler transition, but I'd want to remove that
logic in a following release that strictly prohibits duplicate origins
(assuming that the community agrees we should prohibit duplicate
origins altogether).
As for DB constraints vs UI, I was thinking those DS-type constraints
I pointed out would live in the API. It would basically be added
validation in the deliveryservices POST/PUT endpoint that checks the
DB for existing DSes that conflict with the requested DS.
- Rawlin
On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
<Jo...@comcast.com> wrote:
>
> These kinds of conditions should be detectable with a sufficiently advanced SQL query. Is it possible to add the constraint if it passes and emit a warning during TO startup otherwise? That would let you know the condition exists at startup but not getting in your way and keep you out of trouble once you've cleaned up. We made a mistake early on, but this would acknowledge it was bad and encourage it to be fixed at the speed of operations teams. Also this puts the constraint in the database rather than the UI which is really where the contention is for usability.
>
> Jonathan G
>
>
> On 12/17/18, 11:38 AM, "Rawlin Peters" <ra...@gmail.com> wrote:
>
> We occasionally discuss this issue but haven't tackled it yet. I think
> the main issue is just that duplicate origins have been allowed since
> the beginning, and now everyone's Traffic Ops could be littered with
> duplicate origins. Also, depending on the config of the duplicate
> delivery services, the origins might not be in conflict at all (if
> they don't have different topology constraints). I would love for us
> to just add a uniqueness constraint, but there would need to be a fair
> amount of warning to the community before doing so and might
> invalidate a significant amount of valid use cases. Operators would
> need time to make DNS CNAME records for the duplicate origins and
> update their DSes to use the different CNAMEs.
>
> I think as a good first step to eliminating the use of duplicate
> origins altogether, we should identify which "topology constraints"
> actually cause conflicting config when used with duplicate origins and
> prevent creating DSes with duplicate origins _if it would cause a
> conflict with an existing DS that uses the same origin_.
>
> For instance, I believe an HTTP and DNS-type DS can live happily
> side-by-side using the same origin (probably need different
> routing_names?), but scenarios like HTTP and HTTP_LIVE, or DNS and
> HTTP_NO_CACHE sharing the same origin will cause conflicts for sure.
> So maybe we can start by making sure the DS types "match" when using
> the same origin:
> HTTP + DNS: possibly good, if they have different routing names?
> HTTP_LIVE + HTTP_LIVE_NATNL: bad
> HTTP_NO_CACHE + [any other type]: bad
> HTTP_LIVE + HTTP: bad
> etc.
>
> There are most likely other conflict scenarios that don't involve the
> DS types, but I think this would be a good start. In the future with
> Delivery Service Topologies (aka Flexible Cachegroups aka Bring Your
> Own Topology), we might be able to prohibit assigning a DS to a
> Topology if the DS's origin is already used by another DS in a
> different Topology.
>
> - Rawlin
>
> On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> <Br...@comcast.com> wrote:
> >
> > As some of you may be aware, `parent.config` files generated by Traffic Ops can vary wildly when an origin is assigned to multiple Delivery Services. This results in undefined behavior. I'm told that the conflict only happens when two Delivery Services with different "topology requirements" use the same origin, whatever that means (content routing type?). Regardless, the issue should be addressed. The obvious solution is to put in place a database constraint that prevents an origin from being assigned to more that one Delivery Service with API checks in place that would provide helpful error messages when an attempt is made to violate the constraint. However, would that mess with things like Multi-Site Origin? Or is it just not viable for some other reason? If it is a good solution, I'm prepared to work on a fix that utilizes it.
>
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Gray, Jonathan" <Jo...@comcast.com>.
-1 Holding an ATC upgrade hostage to data cleanup seems like a bad idea. The issue isn't great, but it's also not new. We should allow teams to fix their data at their normal paces if it doesn't create significant overhead or an inherant blocker for new functionality or correction of other major problems imho.
Jonathan G
On 12/18/18, 9:28 AM, "Fieck, Brennan" <Br...@comcast.com> wrote:
Another option is we could detect collisions at startup and simply refuse to continue with the upgrade until the data is fixed. That would allow people using the now-unsupported data format to continue to use their old versions of Traffic Ops without wrecking their database, but also provide an incentive to clean up the data.
________________________________________
From: Gray, Jonathan <Jo...@comcast.com>
Sent: Tuesday, December 18, 2018 5:12 AM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
I'm generally a fan of constrain your data in your database, but not necessarily exclusively. I see this as a one-way cleanup/conversion so it doesn't need to be configurable; otherwise you have to ask the question what happens if someone turns it off. That said, something in the UI layer would be nice to prevent spending significant quantities of time building a complex DS only to have it fail to post for reasons that could have been known earlier.
The way my brain works in this case:
If !unique_constraint_exists_query()
If has_duplicates_query()
show_warning()
else
add_unique_constraint()
to which the API and UI configuration could also make use of unique_constraint_exists_query() to drive additional layer constraints if desired.
Jonathan G
On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com> wrote:
That is an interesting idea...detect at TO startup whether or not
there are duplicate origins and operate in a "prevent duplicate
origins" state if no duplicates are found or "prevent conflicting DS
topologies" state if duplicates are found? So once operators have
replaced all the duplicate origins with CNAMEs, TO will essentially
operate in a "prohibit all duplicate origins" state. That would
probably make for a simpler transition, but I'd want to remove that
logic in a following release that strictly prohibits duplicate origins
(assuming that the community agrees we should prohibit duplicate
origins altogether).
As for DB constraints vs UI, I was thinking those DS-type constraints
I pointed out would live in the API. It would basically be added
validation in the deliveryservices POST/PUT endpoint that checks the
DB for existing DSes that conflict with the requested DS.
- Rawlin
On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
<Jo...@comcast.com> wrote:
>
> These kinds of conditions should be detectable with a sufficiently advanced SQL query. Is it possible to add the constraint if it passes and emit a warning during TO startup otherwise? That would let you know the condition exists at startup but not getting in your way and keep you out of trouble once you've cleaned up. We made a mistake early on, but this would acknowledge it was bad and encourage it to be fixed at the speed of operations teams. Also this puts the constraint in the database rather than the UI which is really where the contention is for usability.
>
> Jonathan G
>
>
> On 12/17/18, 11:38 AM, "Rawlin Peters" <ra...@gmail.com> wrote:
>
> We occasionally discuss this issue but haven't tackled it yet. I think
> the main issue is just that duplicate origins have been allowed since
> the beginning, and now everyone's Traffic Ops could be littered with
> duplicate origins. Also, depending on the config of the duplicate
> delivery services, the origins might not be in conflict at all (if
> they don't have different topology constraints). I would love for us
> to just add a uniqueness constraint, but there would need to be a fair
> amount of warning to the community before doing so and might
> invalidate a significant amount of valid use cases. Operators would
> need time to make DNS CNAME records for the duplicate origins and
> update their DSes to use the different CNAMEs.
>
> I think as a good first step to eliminating the use of duplicate
> origins altogether, we should identify which "topology constraints"
> actually cause conflicting config when used with duplicate origins and
> prevent creating DSes with duplicate origins _if it would cause a
> conflict with an existing DS that uses the same origin_.
>
> For instance, I believe an HTTP and DNS-type DS can live happily
> side-by-side using the same origin (probably need different
> routing_names?), but scenarios like HTTP and HTTP_LIVE, or DNS and
> HTTP_NO_CACHE sharing the same origin will cause conflicts for sure.
> So maybe we can start by making sure the DS types "match" when using
> the same origin:
> HTTP + DNS: possibly good, if they have different routing names?
> HTTP_LIVE + HTTP_LIVE_NATNL: bad
> HTTP_NO_CACHE + [any other type]: bad
> HTTP_LIVE + HTTP: bad
> etc.
>
> There are most likely other conflict scenarios that don't involve the
> DS types, but I think this would be a good start. In the future with
> Delivery Service Topologies (aka Flexible Cachegroups aka Bring Your
> Own Topology), we might be able to prohibit assigning a DS to a
> Topology if the DS's origin is already used by another DS in a
> different Topology.
>
> - Rawlin
>
> On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> <Br...@comcast.com> wrote:
> >
> > As some of you may be aware, `parent.config` files generated by Traffic Ops can vary wildly when an origin is assigned to multiple Delivery Services. This results in undefined behavior. I'm told that the conflict only happens when two Delivery Services with different "topology requirements" use the same origin, whatever that means (content routing type?). Regardless, the issue should be addressed. The obvious solution is to put in place a database constraint that prevents an origin from being assigned to more that one Delivery Service with API checks in place that would provide helpful error messages when an attempt is made to violate the constraint. However, would that mess with things like Multi-Site Origin? Or is it just not viable for some other reason? If it is a good solution, I'm prepared to work on a fix that utilizes it.
>
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Fieck, Brennan" <Br...@comcast.com>.
Another option is we could detect collisions at startup and simply refuse to continue with the upgrade until the data is fixed. That would allow people using the now-unsupported data format to continue to use their old versions of Traffic Ops without wrecking their database, but also provide an incentive to clean up the data.
________________________________________
From: Gray, Jonathan <Jo...@comcast.com>
Sent: Tuesday, December 18, 2018 5:12 AM
To: dev@trafficcontrol.apache.org
Subject: Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services produces indeterminate parent.config
I'm generally a fan of constrain your data in your database, but not necessarily exclusively. I see this as a one-way cleanup/conversion so it doesn't need to be configurable; otherwise you have to ask the question what happens if someone turns it off. That said, something in the UI layer would be nice to prevent spending significant quantities of time building a complex DS only to have it fail to post for reasons that could have been known earlier.
The way my brain works in this case:
If !unique_constraint_exists_query()
If has_duplicates_query()
show_warning()
else
add_unique_constraint()
to which the API and UI configuration could also make use of unique_constraint_exists_query() to drive additional layer constraints if desired.
Jonathan G
On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com> wrote:
That is an interesting idea...detect at TO startup whether or not
there are duplicate origins and operate in a "prevent duplicate
origins" state if no duplicates are found or "prevent conflicting DS
topologies" state if duplicates are found? So once operators have
replaced all the duplicate origins with CNAMEs, TO will essentially
operate in a "prohibit all duplicate origins" state. That would
probably make for a simpler transition, but I'd want to remove that
logic in a following release that strictly prohibits duplicate origins
(assuming that the community agrees we should prohibit duplicate
origins altogether).
As for DB constraints vs UI, I was thinking those DS-type constraints
I pointed out would live in the API. It would basically be added
validation in the deliveryservices POST/PUT endpoint that checks the
DB for existing DSes that conflict with the requested DS.
- Rawlin
On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
<Jo...@comcast.com> wrote:
>
> These kinds of conditions should be detectable with a sufficiently advanced SQL query. Is it possible to add the constraint if it passes and emit a warning during TO startup otherwise? That would let you know the condition exists at startup but not getting in your way and keep you out of trouble once you've cleaned up. We made a mistake early on, but this would acknowledge it was bad and encourage it to be fixed at the speed of operations teams. Also this puts the constraint in the database rather than the UI which is really where the contention is for usability.
>
> Jonathan G
>
>
> On 12/17/18, 11:38 AM, "Rawlin Peters" <ra...@gmail.com> wrote:
>
> We occasionally discuss this issue but haven't tackled it yet. I think
> the main issue is just that duplicate origins have been allowed since
> the beginning, and now everyone's Traffic Ops could be littered with
> duplicate origins. Also, depending on the config of the duplicate
> delivery services, the origins might not be in conflict at all (if
> they don't have different topology constraints). I would love for us
> to just add a uniqueness constraint, but there would need to be a fair
> amount of warning to the community before doing so and might
> invalidate a significant amount of valid use cases. Operators would
> need time to make DNS CNAME records for the duplicate origins and
> update their DSes to use the different CNAMEs.
>
> I think as a good first step to eliminating the use of duplicate
> origins altogether, we should identify which "topology constraints"
> actually cause conflicting config when used with duplicate origins and
> prevent creating DSes with duplicate origins _if it would cause a
> conflict with an existing DS that uses the same origin_.
>
> For instance, I believe an HTTP and DNS-type DS can live happily
> side-by-side using the same origin (probably need different
> routing_names?), but scenarios like HTTP and HTTP_LIVE, or DNS and
> HTTP_NO_CACHE sharing the same origin will cause conflicts for sure.
> So maybe we can start by making sure the DS types "match" when using
> the same origin:
> HTTP + DNS: possibly good, if they have different routing names?
> HTTP_LIVE + HTTP_LIVE_NATNL: bad
> HTTP_NO_CACHE + [any other type]: bad
> HTTP_LIVE + HTTP: bad
> etc.
>
> There are most likely other conflict scenarios that don't involve the
> DS types, but I think this would be a good start. In the future with
> Delivery Service Topologies (aka Flexible Cachegroups aka Bring Your
> Own Topology), we might be able to prohibit assigning a DS to a
> Topology if the DS's origin is already used by another DS in a
> different Topology.
>
> - Rawlin
>
> On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> <Br...@comcast.com> wrote:
> >
> > As some of you may be aware, `parent.config` files generated by Traffic Ops can vary wildly when an origin is assigned to multiple Delivery Services. This results in undefined behavior. I'm told that the conflict only happens when two Delivery Services with different "topology requirements" use the same origin, whatever that means (content routing type?). Regardless, the issue should be addressed. The obvious solution is to put in place a database constraint that prevents an origin from being assigned to more that one Delivery Service with API checks in place that would provide helpful error messages when an attempt is made to violate the constraint. However, would that mess with things like Multi-Site Origin? Or is it just not viable for some other reason? If it is a good solution, I'm prepared to work on a fix that utilizes it.
>
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Gray, Jonathan" <Jo...@comcast.com>.
I'm generally a fan of constrain your data in your database, but not necessarily exclusively. I see this as a one-way cleanup/conversion so it doesn't need to be configurable; otherwise you have to ask the question what happens if someone turns it off. That said, something in the UI layer would be nice to prevent spending significant quantities of time building a complex DS only to have it fail to post for reasons that could have been known earlier.
The way my brain works in this case:
If !unique_constraint_exists_query()
If has_duplicates_query()
show_warning()
else
add_unique_constraint()
to which the API and UI configuration could also make use of unique_constraint_exists_query() to drive additional layer constraints if desired.
Jonathan G
On 12/17/18, 1:11 PM, "Rawlin Peters" <ra...@gmail.com> wrote:
That is an interesting idea...detect at TO startup whether or not
there are duplicate origins and operate in a "prevent duplicate
origins" state if no duplicates are found or "prevent conflicting DS
topologies" state if duplicates are found? So once operators have
replaced all the duplicate origins with CNAMEs, TO will essentially
operate in a "prohibit all duplicate origins" state. That would
probably make for a simpler transition, but I'd want to remove that
logic in a following release that strictly prohibits duplicate origins
(assuming that the community agrees we should prohibit duplicate
origins altogether).
As for DB constraints vs UI, I was thinking those DS-type constraints
I pointed out would live in the API. It would basically be added
validation in the deliveryservices POST/PUT endpoint that checks the
DB for existing DSes that conflict with the requested DS.
- Rawlin
On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
<Jo...@comcast.com> wrote:
>
> These kinds of conditions should be detectable with a sufficiently advanced SQL query. Is it possible to add the constraint if it passes and emit a warning during TO startup otherwise? That would let you know the condition exists at startup but not getting in your way and keep you out of trouble once you've cleaned up. We made a mistake early on, but this would acknowledge it was bad and encourage it to be fixed at the speed of operations teams. Also this puts the constraint in the database rather than the UI which is really where the contention is for usability.
>
> Jonathan G
>
>
> On 12/17/18, 11:38 AM, "Rawlin Peters" <ra...@gmail.com> wrote:
>
> We occasionally discuss this issue but haven't tackled it yet. I think
> the main issue is just that duplicate origins have been allowed since
> the beginning, and now everyone's Traffic Ops could be littered with
> duplicate origins. Also, depending on the config of the duplicate
> delivery services, the origins might not be in conflict at all (if
> they don't have different topology constraints). I would love for us
> to just add a uniqueness constraint, but there would need to be a fair
> amount of warning to the community before doing so and might
> invalidate a significant amount of valid use cases. Operators would
> need time to make DNS CNAME records for the duplicate origins and
> update their DSes to use the different CNAMEs.
>
> I think as a good first step to eliminating the use of duplicate
> origins altogether, we should identify which "topology constraints"
> actually cause conflicting config when used with duplicate origins and
> prevent creating DSes with duplicate origins _if it would cause a
> conflict with an existing DS that uses the same origin_.
>
> For instance, I believe an HTTP and DNS-type DS can live happily
> side-by-side using the same origin (probably need different
> routing_names?), but scenarios like HTTP and HTTP_LIVE, or DNS and
> HTTP_NO_CACHE sharing the same origin will cause conflicts for sure.
> So maybe we can start by making sure the DS types "match" when using
> the same origin:
> HTTP + DNS: possibly good, if they have different routing names?
> HTTP_LIVE + HTTP_LIVE_NATNL: bad
> HTTP_NO_CACHE + [any other type]: bad
> HTTP_LIVE + HTTP: bad
> etc.
>
> There are most likely other conflict scenarios that don't involve the
> DS types, but I think this would be a good start. In the future with
> Delivery Service Topologies (aka Flexible Cachegroups aka Bring Your
> Own Topology), we might be able to prohibit assigning a DS to a
> Topology if the DS's origin is already used by another DS in a
> different Topology.
>
> - Rawlin
>
> On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> <Br...@comcast.com> wrote:
> >
> > As some of you may be aware, `parent.config` files generated by Traffic Ops can vary wildly when an origin is assigned to multiple Delivery Services. This results in undefined behavior. I'm told that the conflict only happens when two Delivery Services with different "topology requirements" use the same origin, whatever that means (content routing type?). Regardless, the issue should be addressed. The obvious solution is to put in place a database constraint that prevents an origin from being assigned to more that one Delivery Service with API checks in place that would provide helpful error messages when an attempt is made to violate the constraint. However, would that mess with things like Multi-Site Origin? Or is it just not viable for some other reason? If it is a good solution, I'm prepared to work on a fix that utilizes it.
>
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Mark Torluemke <mt...@apache.org>.
I'm not too concerned with existing deployments -- if an existing TO has
this config, it's exposed to this (what I would call) bug and their system
will have the negative impacts this causes.
+1 on the 'sufficiently advanced SQL query'. I could be +1 on using that to
drive config.
-1 on adding a global option to 'ignore all duplicate origins'
-1 on adding a unique constraint for origins as a blanket rule to delivery
services
Thanks,
Mark
On Mon, Dec 17, 2018 at 1:11 PM Rawlin Peters <ra...@gmail.com>
wrote:
> That is an interesting idea...detect at TO startup whether or not
> there are duplicate origins and operate in a "prevent duplicate
> origins" state if no duplicates are found or "prevent conflicting DS
> topologies" state if duplicates are found? So once operators have
> replaced all the duplicate origins with CNAMEs, TO will essentially
> operate in a "prohibit all duplicate origins" state. That would
> probably make for a simpler transition, but I'd want to remove that
> logic in a following release that strictly prohibits duplicate origins
> (assuming that the community agrees we should prohibit duplicate
> origins altogether).
>
> As for DB constraints vs UI, I was thinking those DS-type constraints
> I pointed out would live in the API. It would basically be added
> validation in the deliveryservices POST/PUT endpoint that checks the
> DB for existing DSes that conflict with the requested DS.
>
> - Rawlin
>
> On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
> <Jo...@comcast.com> wrote:
> >
> > These kinds of conditions should be detectable with a sufficiently
> advanced SQL query. Is it possible to add the constraint if it passes and
> emit a warning during TO startup otherwise? That would let you know the
> condition exists at startup but not getting in your way and keep you out of
> trouble once you've cleaned up. We made a mistake early on, but this would
> acknowledge it was bad and encourage it to be fixed at the speed of
> operations teams. Also this puts the constraint in the database rather
> than the UI which is really where the contention is for usability.
> >
> > Jonathan G
> >
> >
> > On 12/17/18, 11:38 AM, "Rawlin Peters" <ra...@gmail.com> wrote:
> >
> > We occasionally discuss this issue but haven't tackled it yet. I
> think
> > the main issue is just that duplicate origins have been allowed since
> > the beginning, and now everyone's Traffic Ops could be littered with
> > duplicate origins. Also, depending on the config of the duplicate
> > delivery services, the origins might not be in conflict at all (if
> > they don't have different topology constraints). I would love for us
> > to just add a uniqueness constraint, but there would need to be a
> fair
> > amount of warning to the community before doing so and might
> > invalidate a significant amount of valid use cases. Operators would
> > need time to make DNS CNAME records for the duplicate origins and
> > update their DSes to use the different CNAMEs.
> >
> > I think as a good first step to eliminating the use of duplicate
> > origins altogether, we should identify which "topology constraints"
> > actually cause conflicting config when used with duplicate origins
> and
> > prevent creating DSes with duplicate origins _if it would cause a
> > conflict with an existing DS that uses the same origin_.
> >
> > For instance, I believe an HTTP and DNS-type DS can live happily
> > side-by-side using the same origin (probably need different
> > routing_names?), but scenarios like HTTP and HTTP_LIVE, or DNS and
> > HTTP_NO_CACHE sharing the same origin will cause conflicts for sure.
> > So maybe we can start by making sure the DS types "match" when using
> > the same origin:
> > HTTP + DNS: possibly good, if they have different routing names?
> > HTTP_LIVE + HTTP_LIVE_NATNL: bad
> > HTTP_NO_CACHE + [any other type]: bad
> > HTTP_LIVE + HTTP: bad
> > etc.
> >
> > There are most likely other conflict scenarios that don't involve the
> > DS types, but I think this would be a good start. In the future with
> > Delivery Service Topologies (aka Flexible Cachegroups aka Bring Your
> > Own Topology), we might be able to prohibit assigning a DS to a
> > Topology if the DS's origin is already used by another DS in a
> > different Topology.
> >
> > - Rawlin
> >
> > On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> > <Br...@comcast.com> wrote:
> > >
> > > As some of you may be aware, `parent.config` files generated by
> Traffic Ops can vary wildly when an origin is assigned to multiple Delivery
> Services. This results in undefined behavior. I'm told that the conflict
> only happens when two Delivery Services with different "topology
> requirements" use the same origin, whatever that means (content routing
> type?). Regardless, the issue should be addressed. The obvious solution is
> to put in place a database constraint that prevents an origin from being
> assigned to more that one Delivery Service with API checks in place that
> would provide helpful error messages when an attempt is made to violate the
> constraint. However, would that mess with things like Multi-Site Origin? Or
> is it just not viable for some other reason? If it is a good solution, I'm
> prepared to work on a fix that utilizes it.
> >
> >
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by Rawlin Peters <ra...@gmail.com>.
That is an interesting idea...detect at TO startup whether or not
there are duplicate origins and operate in a "prevent duplicate
origins" state if no duplicates are found or "prevent conflicting DS
topologies" state if duplicates are found? So once operators have
replaced all the duplicate origins with CNAMEs, TO will essentially
operate in a "prohibit all duplicate origins" state. That would
probably make for a simpler transition, but I'd want to remove that
logic in a following release that strictly prohibits duplicate origins
(assuming that the community agrees we should prohibit duplicate
origins altogether).
As for DB constraints vs UI, I was thinking those DS-type constraints
I pointed out would live in the API. It would basically be added
validation in the deliveryservices POST/PUT endpoint that checks the
DB for existing DSes that conflict with the requested DS.
- Rawlin
On Mon, Dec 17, 2018 at 12:35 PM Gray, Jonathan
<Jo...@comcast.com> wrote:
>
> These kinds of conditions should be detectable with a sufficiently advanced SQL query. Is it possible to add the constraint if it passes and emit a warning during TO startup otherwise? That would let you know the condition exists at startup but not getting in your way and keep you out of trouble once you've cleaned up. We made a mistake early on, but this would acknowledge it was bad and encourage it to be fixed at the speed of operations teams. Also this puts the constraint in the database rather than the UI which is really where the contention is for usability.
>
> Jonathan G
>
>
> On 12/17/18, 11:38 AM, "Rawlin Peters" <ra...@gmail.com> wrote:
>
> We occasionally discuss this issue but haven't tackled it yet. I think
> the main issue is just that duplicate origins have been allowed since
> the beginning, and now everyone's Traffic Ops could be littered with
> duplicate origins. Also, depending on the config of the duplicate
> delivery services, the origins might not be in conflict at all (if
> they don't have different topology constraints). I would love for us
> to just add a uniqueness constraint, but there would need to be a fair
> amount of warning to the community before doing so and might
> invalidate a significant amount of valid use cases. Operators would
> need time to make DNS CNAME records for the duplicate origins and
> update their DSes to use the different CNAMEs.
>
> I think as a good first step to eliminating the use of duplicate
> origins altogether, we should identify which "topology constraints"
> actually cause conflicting config when used with duplicate origins and
> prevent creating DSes with duplicate origins _if it would cause a
> conflict with an existing DS that uses the same origin_.
>
> For instance, I believe an HTTP and DNS-type DS can live happily
> side-by-side using the same origin (probably need different
> routing_names?), but scenarios like HTTP and HTTP_LIVE, or DNS and
> HTTP_NO_CACHE sharing the same origin will cause conflicts for sure.
> So maybe we can start by making sure the DS types "match" when using
> the same origin:
> HTTP + DNS: possibly good, if they have different routing names?
> HTTP_LIVE + HTTP_LIVE_NATNL: bad
> HTTP_NO_CACHE + [any other type]: bad
> HTTP_LIVE + HTTP: bad
> etc.
>
> There are most likely other conflict scenarios that don't involve the
> DS types, but I think this would be a good start. In the future with
> Delivery Service Topologies (aka Flexible Cachegroups aka Bring Your
> Own Topology), we might be able to prohibit assigning a DS to a
> Topology if the DS's origin is already used by another DS in a
> different Topology.
>
> - Rawlin
>
> On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
> <Br...@comcast.com> wrote:
> >
> > As some of you may be aware, `parent.config` files generated by Traffic Ops can vary wildly when an origin is assigned to multiple Delivery Services. This results in undefined behavior. I'm told that the conflict only happens when two Delivery Services with different "topology requirements" use the same origin, whatever that means (content routing type?). Regardless, the issue should be addressed. The obvious solution is to put in place a database constraint that prevents an origin from being assigned to more that one Delivery Service with API checks in place that would provide helpful error messages when an attempt is made to violate the constraint. However, would that mess with things like Multi-Site Origin? Or is it just not viable for some other reason? If it is a good solution, I'm prepared to work on a fix that utilizes it.
>
>
Re: [EXTERNAL] Re: Origins assigned to Multipe Delivery Services
produces indeterminate parent.config
Posted by "Gray, Jonathan" <Jo...@comcast.com>.
These kinds of conditions should be detectable with a sufficiently advanced SQL query. Is it possible to add the constraint if it passes and emit a warning during TO startup otherwise? That would let you know the condition exists at startup but not getting in your way and keep you out of trouble once you've cleaned up. We made a mistake early on, but this would acknowledge it was bad and encourage it to be fixed at the speed of operations teams. Also this puts the constraint in the database rather than the UI which is really where the contention is for usability.
Jonathan G
On 12/17/18, 11:38 AM, "Rawlin Peters" <ra...@gmail.com> wrote:
We occasionally discuss this issue but haven't tackled it yet. I think
the main issue is just that duplicate origins have been allowed since
the beginning, and now everyone's Traffic Ops could be littered with
duplicate origins. Also, depending on the config of the duplicate
delivery services, the origins might not be in conflict at all (if
they don't have different topology constraints). I would love for us
to just add a uniqueness constraint, but there would need to be a fair
amount of warning to the community before doing so and might
invalidate a significant amount of valid use cases. Operators would
need time to make DNS CNAME records for the duplicate origins and
update their DSes to use the different CNAMEs.
I think as a good first step to eliminating the use of duplicate
origins altogether, we should identify which "topology constraints"
actually cause conflicting config when used with duplicate origins and
prevent creating DSes with duplicate origins _if it would cause a
conflict with an existing DS that uses the same origin_.
For instance, I believe an HTTP and DNS-type DS can live happily
side-by-side using the same origin (probably need different
routing_names?), but scenarios like HTTP and HTTP_LIVE, or DNS and
HTTP_NO_CACHE sharing the same origin will cause conflicts for sure.
So maybe we can start by making sure the DS types "match" when using
the same origin:
HTTP + DNS: possibly good, if they have different routing names?
HTTP_LIVE + HTTP_LIVE_NATNL: bad
HTTP_NO_CACHE + [any other type]: bad
HTTP_LIVE + HTTP: bad
etc.
There are most likely other conflict scenarios that don't involve the
DS types, but I think this would be a good start. In the future with
Delivery Service Topologies (aka Flexible Cachegroups aka Bring Your
Own Topology), we might be able to prohibit assigning a DS to a
Topology if the DS's origin is already used by another DS in a
different Topology.
- Rawlin
On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
<Br...@comcast.com> wrote:
>
> As some of you may be aware, `parent.config` files generated by Traffic Ops can vary wildly when an origin is assigned to multiple Delivery Services. This results in undefined behavior. I'm told that the conflict only happens when two Delivery Services with different "topology requirements" use the same origin, whatever that means (content routing type?). Regardless, the issue should be addressed. The obvious solution is to put in place a database constraint that prevents an origin from being assigned to more that one Delivery Service with API checks in place that would provide helpful error messages when an attempt is made to violate the constraint. However, would that mess with things like Multi-Site Origin? Or is it just not viable for some other reason? If it is a good solution, I'm prepared to work on a fix that utilizes it.
Re: Origins assigned to Multipe Delivery Services produces
indeterminate parent.config
Posted by Rawlin Peters <ra...@gmail.com>.
We occasionally discuss this issue but haven't tackled it yet. I think
the main issue is just that duplicate origins have been allowed since
the beginning, and now everyone's Traffic Ops could be littered with
duplicate origins. Also, depending on the config of the duplicate
delivery services, the origins might not be in conflict at all (if
they don't have different topology constraints). I would love for us
to just add a uniqueness constraint, but there would need to be a fair
amount of warning to the community before doing so and might
invalidate a significant amount of valid use cases. Operators would
need time to make DNS CNAME records for the duplicate origins and
update their DSes to use the different CNAMEs.
I think as a good first step to eliminating the use of duplicate
origins altogether, we should identify which "topology constraints"
actually cause conflicting config when used with duplicate origins and
prevent creating DSes with duplicate origins _if it would cause a
conflict with an existing DS that uses the same origin_.
For instance, I believe an HTTP and DNS-type DS can live happily
side-by-side using the same origin (probably need different
routing_names?), but scenarios like HTTP and HTTP_LIVE, or DNS and
HTTP_NO_CACHE sharing the same origin will cause conflicts for sure.
So maybe we can start by making sure the DS types "match" when using
the same origin:
HTTP + DNS: possibly good, if they have different routing names?
HTTP_LIVE + HTTP_LIVE_NATNL: bad
HTTP_NO_CACHE + [any other type]: bad
HTTP_LIVE + HTTP: bad
etc.
There are most likely other conflict scenarios that don't involve the
DS types, but I think this would be a good start. In the future with
Delivery Service Topologies (aka Flexible Cachegroups aka Bring Your
Own Topology), we might be able to prohibit assigning a DS to a
Topology if the DS's origin is already used by another DS in a
different Topology.
- Rawlin
On Mon, Dec 17, 2018 at 10:52 AM Fieck, Brennan
<Br...@comcast.com> wrote:
>
> As some of you may be aware, `parent.config` files generated by Traffic Ops can vary wildly when an origin is assigned to multiple Delivery Services. This results in undefined behavior. I'm told that the conflict only happens when two Delivery Services with different "topology requirements" use the same origin, whatever that means (content routing type?). Regardless, the issue should be addressed. The obvious solution is to put in place a database constraint that prevents an origin from being assigned to more that one Delivery Service with API checks in place that would provide helpful error messages when an attempt is made to violate the constraint. However, would that mess with things like Multi-Site Origin? Or is it just not viable for some other reason? If it is a good solution, I'm prepared to work on a fix that utilizes it.