You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficcontrol.apache.org by Rawlin Peters <ra...@gmail.com> on 2018/02/23 22:40:25 UTC
Steering Target Geo-Ordering
Hey folks,
At Comcast we have a need to augment CLIENT_STEERING (also regular
STEERING while we're at it) to allow targets to be ordered/sorted
based upon the client's proximity to the origin of the target delivery
services. I'd like to get your feedback on my proposed design and
address any of your concerns.
For HTTP_LIVE targets for instance, we'd like edge caches to ideally
retrieve/serve data from an Origin that is close to the client and
fall back to Origins that are farther and farther away. This allows
for increased redundancy while ordering optimal Origins (Delivery
Services) for a client to choose from.
For example, I have 3 Origins in different locations: Seattle, Denver,
and Boston. I would create an HTTP_LIVE delivery service for each
origin and a CLIENT_STEERING delivery service with those delivery
services as targets. I would then like to have those targets ordered
based upon proximity to the client. So a client in Seattle would get
the list [Seattle, Denver, Boston], while a client in Boston would get
the list [Boston, Denver, Seattle].
To make things more complicated, I want to add a redundant origin in
each location and split traffic between them (like STEERING_WEIGHT
today) while taking into account the geo-ordering. I also want to be
able to force an ordering (like STEERING_ORDER today) among co-located
targets.
In order to accomplish this I propose to:
1. add two new steering types: GEO_ORDER and GEO_WEIGHT (by adding a
target of type GEO_*, a steering DS would then enable geo-ordering)
2. associate a Delivery Service Origin with a latitude/longitude,
thereby associating a lat/long to a GEO_* target
Item 1 is pretty straightforward and will also play nicely with the
current steering types (STEERING_ORDER and STEERING_WEIGHT). I've
completed a POC within Traffic Router that basically provides the
following ordering when mixing all 4 types in a single steering
delivery service:
- Negative STEERING_ORDER targets
- GEO_WEIGHT and GEO_ORDER targets, grouped by proximity to the
client, ordered by geo-order and the consistent-hashing from the
weightings
- STEERING_WEIGHT targets (consistent-hashed)
- Positive STEERING_ORDER targets
Item 2 is not as straightforward because the simple thing would be to
just add an Origin Lat/Long field to the Delivery Service and call it
a day. However I don't think we should do that, and I'll dive more
into that in a separate thread (coming soon).
Does anyone have questions/concerns about adding these new GEO_ORDER
and GEO_STEERING target types and geo-sorting them based upon
proximity to the client? Are you okay with the proposed ordering when
all the steering types are mixed together?
- Rawlin
Re: Steering Target Geo-Ordering
Posted by Jeremy Mitchell <mi...@gmail.com>.
Makes perfect sense. thanks for the clarification.
On Tue, Feb 27, 2018 at 11:32 AM, Rawlin Peters <ra...@gmail.com>
wrote:
> Hey Jeremy,
>
> With a redundant origin in each location, there would be a total of 6
> HTTP_LIVE DSes. Then you'd create a steering DS with each of those 6
> HTTP_LIVE DSes as targets. The type of each target would either be
> GEO_ORDER or GEO_WEIGHT, depending on if we want a forced ordering or
> something like a 50/50 split, respectively.
>
> Say I have the following targets in my Steering DS:
>
> Seattle1 (S1):
> GEO_ORDER: 1
>
> Seattle2 (S2):
> GEO_ORDER: 2
>
> Denver1 (D1):
> GEO_WEIGHT: 50
>
> Denver2 (D2):
> GEO_WEIGHT: 50
>
> Boston1 (B1):
> GEO_ORDER: 1
>
> Boston2 (B2):
> GEO_ORDER: 2
>
> For a client in Seattle requesting /foo, they would get the list [S1,
> S2, D2, D1, B1, B2], where D1 and D2 are consistent hashed based upon
> the requested URL and given weight. So if they request /bar maybe D1
> and D2 are switched so they get [S1, S2, D1, D2, B1, B2].
>
> - Rawlin
>
> On Mon, Feb 26, 2018 at 8:18 PM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
> > So, back to the Seattle, Denver, Boston example...
> >
> > To accomplish the geo ordering and redundant origin use case, you'd
> create
> > 3 HTTP_LIVE delivery services? One with an origin in seattle. One with an
> > origin in denver. One with an origin in boston. (or would you create 6
> ds's
> > so you have 2 in each location?)
> >
> > Then you'd create a steering delivery service with 3 targets (the
> HTTP_LIVE
> > ds's you just created.) Each target would have type = GEO_ORDER and
> value =
> > ?
> >
> > I'm getting confused with that above scenario. How many targets would you
> > have? What would the type of each target be and what would the value of
> > each target be?
> >
> > Jeremy
> >
> >
> > On Fri, Feb 23, 2018 at 3:40 PM, Rawlin Peters <ra...@gmail.com>
> > wrote:
> >
> >> Hey folks,
> >>
> >> At Comcast we have a need to augment CLIENT_STEERING (also regular
> >> STEERING while we're at it) to allow targets to be ordered/sorted
> >> based upon the client's proximity to the origin of the target delivery
> >> services. I'd like to get your feedback on my proposed design and
> >> address any of your concerns.
> >>
> >> For HTTP_LIVE targets for instance, we'd like edge caches to ideally
> >> retrieve/serve data from an Origin that is close to the client and
> >> fall back to Origins that are farther and farther away. This allows
> >> for increased redundancy while ordering optimal Origins (Delivery
> >> Services) for a client to choose from.
> >>
> >> For example, I have 3 Origins in different locations: Seattle, Denver,
> >> and Boston. I would create an HTTP_LIVE delivery service for each
> >> origin and a CLIENT_STEERING delivery service with those delivery
> >> services as targets. I would then like to have those targets ordered
> >> based upon proximity to the client. So a client in Seattle would get
> >> the list [Seattle, Denver, Boston], while a client in Boston would get
> >> the list [Boston, Denver, Seattle].
> >>
> >> To make things more complicated, I want to add a redundant origin in
> >> each location and split traffic between them (like STEERING_WEIGHT
> >> today) while taking into account the geo-ordering. I also want to be
> >> able to force an ordering (like STEERING_ORDER today) among co-located
> >> targets.
> >>
> >> In order to accomplish this I propose to:
> >> 1. add two new steering types: GEO_ORDER and GEO_WEIGHT (by adding a
> >> target of type GEO_*, a steering DS would then enable geo-ordering)
> >> 2. associate a Delivery Service Origin with a latitude/longitude,
> >> thereby associating a lat/long to a GEO_* target
> >>
> >> Item 1 is pretty straightforward and will also play nicely with the
> >> current steering types (STEERING_ORDER and STEERING_WEIGHT). I've
> >> completed a POC within Traffic Router that basically provides the
> >> following ordering when mixing all 4 types in a single steering
> >> delivery service:
> >> - Negative STEERING_ORDER targets
> >> - GEO_WEIGHT and GEO_ORDER targets, grouped by proximity to the
> >> client, ordered by geo-order and the consistent-hashing from the
> >> weightings
> >> - STEERING_WEIGHT targets (consistent-hashed)
> >> - Positive STEERING_ORDER targets
> >>
> >> Item 2 is not as straightforward because the simple thing would be to
> >> just add an Origin Lat/Long field to the Delivery Service and call it
> >> a day. However I don't think we should do that, and I'll dive more
> >> into that in a separate thread (coming soon).
> >>
> >> Does anyone have questions/concerns about adding these new GEO_ORDER
> >> and GEO_STEERING target types and geo-sorting them based upon
> >> proximity to the client? Are you okay with the proposed ordering when
> >> all the steering types are mixed together?
> >>
> >> - Rawlin
> >>
>
Re: Steering Target Geo-Ordering
Posted by Rawlin Peters <ra...@gmail.com>.
Hey Jeremy,
With a redundant origin in each location, there would be a total of 6
HTTP_LIVE DSes. Then you'd create a steering DS with each of those 6
HTTP_LIVE DSes as targets. The type of each target would either be
GEO_ORDER or GEO_WEIGHT, depending on if we want a forced ordering or
something like a 50/50 split, respectively.
Say I have the following targets in my Steering DS:
Seattle1 (S1):
GEO_ORDER: 1
Seattle2 (S2):
GEO_ORDER: 2
Denver1 (D1):
GEO_WEIGHT: 50
Denver2 (D2):
GEO_WEIGHT: 50
Boston1 (B1):
GEO_ORDER: 1
Boston2 (B2):
GEO_ORDER: 2
For a client in Seattle requesting /foo, they would get the list [S1,
S2, D2, D1, B1, B2], where D1 and D2 are consistent hashed based upon
the requested URL and given weight. So if they request /bar maybe D1
and D2 are switched so they get [S1, S2, D1, D2, B1, B2].
- Rawlin
On Mon, Feb 26, 2018 at 8:18 PM, Jeremy Mitchell <mi...@gmail.com> wrote:
> So, back to the Seattle, Denver, Boston example...
>
> To accomplish the geo ordering and redundant origin use case, you'd create
> 3 HTTP_LIVE delivery services? One with an origin in seattle. One with an
> origin in denver. One with an origin in boston. (or would you create 6 ds's
> so you have 2 in each location?)
>
> Then you'd create a steering delivery service with 3 targets (the HTTP_LIVE
> ds's you just created.) Each target would have type = GEO_ORDER and value =
> ?
>
> I'm getting confused with that above scenario. How many targets would you
> have? What would the type of each target be and what would the value of
> each target be?
>
> Jeremy
>
>
> On Fri, Feb 23, 2018 at 3:40 PM, Rawlin Peters <ra...@gmail.com>
> wrote:
>
>> Hey folks,
>>
>> At Comcast we have a need to augment CLIENT_STEERING (also regular
>> STEERING while we're at it) to allow targets to be ordered/sorted
>> based upon the client's proximity to the origin of the target delivery
>> services. I'd like to get your feedback on my proposed design and
>> address any of your concerns.
>>
>> For HTTP_LIVE targets for instance, we'd like edge caches to ideally
>> retrieve/serve data from an Origin that is close to the client and
>> fall back to Origins that are farther and farther away. This allows
>> for increased redundancy while ordering optimal Origins (Delivery
>> Services) for a client to choose from.
>>
>> For example, I have 3 Origins in different locations: Seattle, Denver,
>> and Boston. I would create an HTTP_LIVE delivery service for each
>> origin and a CLIENT_STEERING delivery service with those delivery
>> services as targets. I would then like to have those targets ordered
>> based upon proximity to the client. So a client in Seattle would get
>> the list [Seattle, Denver, Boston], while a client in Boston would get
>> the list [Boston, Denver, Seattle].
>>
>> To make things more complicated, I want to add a redundant origin in
>> each location and split traffic between them (like STEERING_WEIGHT
>> today) while taking into account the geo-ordering. I also want to be
>> able to force an ordering (like STEERING_ORDER today) among co-located
>> targets.
>>
>> In order to accomplish this I propose to:
>> 1. add two new steering types: GEO_ORDER and GEO_WEIGHT (by adding a
>> target of type GEO_*, a steering DS would then enable geo-ordering)
>> 2. associate a Delivery Service Origin with a latitude/longitude,
>> thereby associating a lat/long to a GEO_* target
>>
>> Item 1 is pretty straightforward and will also play nicely with the
>> current steering types (STEERING_ORDER and STEERING_WEIGHT). I've
>> completed a POC within Traffic Router that basically provides the
>> following ordering when mixing all 4 types in a single steering
>> delivery service:
>> - Negative STEERING_ORDER targets
>> - GEO_WEIGHT and GEO_ORDER targets, grouped by proximity to the
>> client, ordered by geo-order and the consistent-hashing from the
>> weightings
>> - STEERING_WEIGHT targets (consistent-hashed)
>> - Positive STEERING_ORDER targets
>>
>> Item 2 is not as straightforward because the simple thing would be to
>> just add an Origin Lat/Long field to the Delivery Service and call it
>> a day. However I don't think we should do that, and I'll dive more
>> into that in a separate thread (coming soon).
>>
>> Does anyone have questions/concerns about adding these new GEO_ORDER
>> and GEO_STEERING target types and geo-sorting them based upon
>> proximity to the client? Are you okay with the proposed ordering when
>> all the steering types are mixed together?
>>
>> - Rawlin
>>
Re: Steering Target Geo-Ordering
Posted by Jeremy Mitchell <mi...@gmail.com>.
So, back to the Seattle, Denver, Boston example...
To accomplish the geo ordering and redundant origin use case, you'd create
3 HTTP_LIVE delivery services? One with an origin in seattle. One with an
origin in denver. One with an origin in boston. (or would you create 6 ds's
so you have 2 in each location?)
Then you'd create a steering delivery service with 3 targets (the HTTP_LIVE
ds's you just created.) Each target would have type = GEO_ORDER and value =
?
I'm getting confused with that above scenario. How many targets would you
have? What would the type of each target be and what would the value of
each target be?
Jeremy
On Fri, Feb 23, 2018 at 3:40 PM, Rawlin Peters <ra...@gmail.com>
wrote:
> Hey folks,
>
> At Comcast we have a need to augment CLIENT_STEERING (also regular
> STEERING while we're at it) to allow targets to be ordered/sorted
> based upon the client's proximity to the origin of the target delivery
> services. I'd like to get your feedback on my proposed design and
> address any of your concerns.
>
> For HTTP_LIVE targets for instance, we'd like edge caches to ideally
> retrieve/serve data from an Origin that is close to the client and
> fall back to Origins that are farther and farther away. This allows
> for increased redundancy while ordering optimal Origins (Delivery
> Services) for a client to choose from.
>
> For example, I have 3 Origins in different locations: Seattle, Denver,
> and Boston. I would create an HTTP_LIVE delivery service for each
> origin and a CLIENT_STEERING delivery service with those delivery
> services as targets. I would then like to have those targets ordered
> based upon proximity to the client. So a client in Seattle would get
> the list [Seattle, Denver, Boston], while a client in Boston would get
> the list [Boston, Denver, Seattle].
>
> To make things more complicated, I want to add a redundant origin in
> each location and split traffic between them (like STEERING_WEIGHT
> today) while taking into account the geo-ordering. I also want to be
> able to force an ordering (like STEERING_ORDER today) among co-located
> targets.
>
> In order to accomplish this I propose to:
> 1. add two new steering types: GEO_ORDER and GEO_WEIGHT (by adding a
> target of type GEO_*, a steering DS would then enable geo-ordering)
> 2. associate a Delivery Service Origin with a latitude/longitude,
> thereby associating a lat/long to a GEO_* target
>
> Item 1 is pretty straightforward and will also play nicely with the
> current steering types (STEERING_ORDER and STEERING_WEIGHT). I've
> completed a POC within Traffic Router that basically provides the
> following ordering when mixing all 4 types in a single steering
> delivery service:
> - Negative STEERING_ORDER targets
> - GEO_WEIGHT and GEO_ORDER targets, grouped by proximity to the
> client, ordered by geo-order and the consistent-hashing from the
> weightings
> - STEERING_WEIGHT targets (consistent-hashed)
> - Positive STEERING_ORDER targets
>
> Item 2 is not as straightforward because the simple thing would be to
> just add an Origin Lat/Long field to the Delivery Service and call it
> a day. However I don't think we should do that, and I'll dive more
> into that in a separate thread (coming soon).
>
> Does anyone have questions/concerns about adding these new GEO_ORDER
> and GEO_STEERING target types and geo-sorting them based upon
> proximity to the client? Are you okay with the proposed ordering when
> all the steering types are mixed together?
>
> - Rawlin
>
Re: Steering Target Geo-Ordering
Posted by Rawlin Peters <ra...@gmail.com>.
So I've done some more thinking about client-steering geo-ordering,
and I think the geo-sorting part should be summarized as follows:
Sort the geo-steering targets ascending by the total distance from
[client -> edge -> origin]. For targets where the total distance is
equal, prefer the target with the shortest [client -> edge] distance.
This is a more difficult task than just sorting by distance between
the client and origin, but it takes into account the fact that the
most optimal cachegroups might not be available.
- Rawlin
On Wed, Feb 28, 2018 at 11:02 AM, Rawlin Peters <ra...@gmail.com> wrote:
> No problem, Eric. I appreciate a good brainstorming. Really the
> Seattle-Denver-Boston example I gave was just to illustrate the idea
> of the feature; I'm not sure we'd actually go with a Geo Steering DS
> architecture that dispersed.
>
> Steering DSes aren't actually assignable to specific cachegroups,
> because the assignments of the target DSes are what's actually used in
> order to find a cache that supports the target DS. As long as the
> target DSes are assigned to all cache groups, the client will be
> routed to an optimal edge cache. If the targets are geo-ordered, then
> the client's first option is to request a target DS with an Origin
> that is also optimal to that edge cache.
>
> I think I might've confused the way I worded the problem initially.
> The target DSes should always be ordered by distance between the
> routed-to cachegroup and the Origin. That way a client in Seattle that
> was routed to an edge in Boston (if the Seattle and Denver CGs were
> down) would first request an Origin in Boston not Seattle. That way
> traffic wouldn't be be crossing the country twice.
>
> Does that make more sense?
>
> - Rawlin
>
> On Wed, Feb 28, 2018 at 7:39 AM, Eric Friedrich (efriedri)
> <ef...@cisco.com> wrote:
>> Thanks Rawlin-
>> Thanks for bearing with me. I’m thinking through this and wanted to brainstorm a little.
>>
>> What’s the benefit of going to the Boston edge cache to get the Seattle or Denver DeliveryServices?
>> - If you already have redundant origins in Boston, you’re protected against failure of a single origin.
>> - If both Boston origins fail, the client would be routed to Denver and hit the local Denver origins.
>> - If both Boston origins are down, then you wouldn’t be able to get Boston content in Denver anyway
>> - But clients in Boston would now need to hit the Denver edges, perhaps overloading that CG. If this is a failure case you want to address, maybe we could do something like the following:
>>
>> Would it be possible to create multiple levels of steering delivery services?
>> - Top Level: Client Steering DS (tr.nbc.cdn.customer.com<http://tr.nbc.cdn.com>) - assigned to all CG. Does GeoOrdering based on client proximity to assigned cache groups of the target DS'
>> - Target 1: Boston Client Steering DS (tr.bos-nbc.cdn.customer.com<http://tr.boston-nbc.cdn.customer.com>) - assigned only to Boston CG. Ranked steering policy
>> - Target 1.1: Boston Live DS (tr.bos-bos-nbc.cdn.customer.com<http://tr.bos-bos-nbc.cdn.customer.com>), rank=1
>> - Target 1.2: Denver Live DS (tr.den-bos-nbc.cdn.customer.com<http://tr.den-bos-nbc.cdn.customer.com>), rank=2
>> - Target 1.3: Seattle Live DS (tr.sea-bos-nbc.cdn.customer.com<http://tr.sea-bos-nbc.cdn.customer.com>), rank=3
>>
>> - Target 2: Denver Client Steering DS (tr.den-nbc.cdn.customer.com<http://tr.denver-nbc.cdn.customer.com>) - assigned only to Denver CG
>> - Target 3: Seattle Client Steering DS (tr.seat-nbc.cdn.customer.com<http://tr.seattle-nbc.cdn.customer.com>) - assigned only to Seattle CG
>>
>> The goal would be this order of DS’s returned to the client
>> Location: [tr.bos-bos-nbc, tr.den-bos-nbc, tr.sea-bos-nbc, tr.den-den-nbc, tr.sea-den-nbc, r.bos-den-nbc,…. ]
>>
>> Is this what are you ultimately looking for?
>>
>> This lets us choose CG and DS based on the location of the client relative to the edge cache, rather than proximity of client to origin. Since the client is talking to the edge cache and not the origin, this seems like a better metric. Being able to compose Steering DS would also give us more flexibility for other use cases in the future as well
>>
>>
>>
>>
>> On Feb 27, 2018, at 4:40 PM, Rawlin Peters <ra...@gmail.com>> wrote:
>>
>> Hey Eric,
>>
>> In this example I'd think that all the target DSes would need to be
>> assigned to all 3 Cache Groups. That way the client could possibly use
>> any of the target DSes from the cachegroup they're routed to. But I
>> suppose that could change on a case-by-case basis where maybe the
>> target DSes are only assigned to cache groups that are close to their
>> origins. In that case I'd think a client in Seattle would possibly be
>> routed to an edge cache in Boston, which would optimize the connection
>> between the Boston edge cache and the Boston origin. But it might be
>> better to have the client connect to an edge cache in Seattle which
>> then retrieves the content from a Boston origin (although these are
>> both worst-case scenarios).
>>
>> As far as the content on the origins themselves go, they'd have to be
>> interchangeable from the client's perspective, but I'm not sure if it
>> would have to be identical or not. I imagine that would really depend
>> on what the steering DS is providing the client.
>>
>> - Rawlin
>>
>> On Tue, Feb 27, 2018 at 10:37 AM, Eric Friedrich (efriedri)
>> <ef...@cisco.com>> wrote:
>> In this example, what would be the assignments of delivery services to edge Cache Groups? Are all 3DS’ assigned to all 3 Cache Groups?
>>
>> I’ll also assume that the content on the origins, while interchangeable from a clients perspective, is not identical? (i.e. might contain regionalized content?)
>>
>> —Eric
>>
>>
>>
>>
>> On Feb 23, 2018, at 5:40 PM, Rawlin Peters <ra...@gmail.com>> wrote:
>>
>> Hey folks,
>>
>> At Comcast we have a need to augment CLIENT_STEERING (also regular
>> STEERING while we're at it) to allow targets to be ordered/sorted
>> based upon the client's proximity to the origin of the target delivery
>> services. I'd like to get your feedback on my proposed design and
>> address any of your concerns.
>>
>> For HTTP_LIVE targets for instance, we'd like edge caches to ideally
>> retrieve/serve data from an Origin that is close to the client and
>> fall back to Origins that are farther and farther away. This allows
>> for increased redundancy while ordering optimal Origins (Delivery
>> Services) for a client to choose from.
>>
>> For example, I have 3 Origins in different locations: Seattle, Denver,
>> and Boston. I would create an HTTP_LIVE delivery service for each
>> origin and a CLIENT_STEERING delivery service with those delivery
>> services as targets. I would then like to have those targets ordered
>> based upon proximity to the client. So a client in Seattle would get
>> the list [Seattle, Denver, Boston], while a client in Boston would get
>> the list [Boston, Denver, Seattle].
>>
>> To make things more complicated, I want to add a redundant origin in
>> each location and split traffic between them (like STEERING_WEIGHT
>> today) while taking into account the geo-ordering. I also want to be
>> able to force an ordering (like STEERING_ORDER today) among co-located
>> targets.
>>
>> In order to accomplish this I propose to:
>> 1. add two new steering types: GEO_ORDER and GEO_WEIGHT (by adding a
>> target of type GEO_*, a steering DS would then enable geo-ordering)
>> 2. associate a Delivery Service Origin with a latitude/longitude,
>> thereby associating a lat/long to a GEO_* target
>>
>> Item 1 is pretty straightforward and will also play nicely with the
>> current steering types (STEERING_ORDER and STEERING_WEIGHT). I've
>> completed a POC within Traffic Router that basically provides the
>> following ordering when mixing all 4 types in a single steering
>> delivery service:
>> - Negative STEERING_ORDER targets
>> - GEO_WEIGHT and GEO_ORDER targets, grouped by proximity to the
>> client, ordered by geo-order and the consistent-hashing from the
>> weightings
>> - STEERING_WEIGHT targets (consistent-hashed)
>> - Positive STEERING_ORDER targets
>>
>> Item 2 is not as straightforward because the simple thing would be to
>> just add an Origin Lat/Long field to the Delivery Service and call it
>> a day. However I don't think we should do that, and I'll dive more
>> into that in a separate thread (coming soon).
>>
>> Does anyone have questions/concerns about adding these new GEO_ORDER
>> and GEO_STEERING target types and geo-sorting them based upon
>> proximity to the client? Are you okay with the proposed ordering when
>> all the steering types are mixed together?
>>
>> - Rawlin
>>
>>
Re: Steering Target Geo-Ordering
Posted by Rawlin Peters <ra...@gmail.com>.
No problem, Eric. I appreciate a good brainstorming. Really the
Seattle-Denver-Boston example I gave was just to illustrate the idea
of the feature; I'm not sure we'd actually go with a Geo Steering DS
architecture that dispersed.
Steering DSes aren't actually assignable to specific cachegroups,
because the assignments of the target DSes are what's actually used in
order to find a cache that supports the target DS. As long as the
target DSes are assigned to all cache groups, the client will be
routed to an optimal edge cache. If the targets are geo-ordered, then
the client's first option is to request a target DS with an Origin
that is also optimal to that edge cache.
I think I might've confused the way I worded the problem initially.
The target DSes should always be ordered by distance between the
routed-to cachegroup and the Origin. That way a client in Seattle that
was routed to an edge in Boston (if the Seattle and Denver CGs were
down) would first request an Origin in Boston not Seattle. That way
traffic wouldn't be be crossing the country twice.
Does that make more sense?
- Rawlin
On Wed, Feb 28, 2018 at 7:39 AM, Eric Friedrich (efriedri)
<ef...@cisco.com> wrote:
> Thanks Rawlin-
> Thanks for bearing with me. I’m thinking through this and wanted to brainstorm a little.
>
> What’s the benefit of going to the Boston edge cache to get the Seattle or Denver DeliveryServices?
> - If you already have redundant origins in Boston, you’re protected against failure of a single origin.
> - If both Boston origins fail, the client would be routed to Denver and hit the local Denver origins.
> - If both Boston origins are down, then you wouldn’t be able to get Boston content in Denver anyway
> - But clients in Boston would now need to hit the Denver edges, perhaps overloading that CG. If this is a failure case you want to address, maybe we could do something like the following:
>
> Would it be possible to create multiple levels of steering delivery services?
> - Top Level: Client Steering DS (tr.nbc.cdn.customer.com<http://tr.nbc.cdn.com>) - assigned to all CG. Does GeoOrdering based on client proximity to assigned cache groups of the target DS'
> - Target 1: Boston Client Steering DS (tr.bos-nbc.cdn.customer.com<http://tr.boston-nbc.cdn.customer.com>) - assigned only to Boston CG. Ranked steering policy
> - Target 1.1: Boston Live DS (tr.bos-bos-nbc.cdn.customer.com<http://tr.bos-bos-nbc.cdn.customer.com>), rank=1
> - Target 1.2: Denver Live DS (tr.den-bos-nbc.cdn.customer.com<http://tr.den-bos-nbc.cdn.customer.com>), rank=2
> - Target 1.3: Seattle Live DS (tr.sea-bos-nbc.cdn.customer.com<http://tr.sea-bos-nbc.cdn.customer.com>), rank=3
>
> - Target 2: Denver Client Steering DS (tr.den-nbc.cdn.customer.com<http://tr.denver-nbc.cdn.customer.com>) - assigned only to Denver CG
> - Target 3: Seattle Client Steering DS (tr.seat-nbc.cdn.customer.com<http://tr.seattle-nbc.cdn.customer.com>) - assigned only to Seattle CG
>
> The goal would be this order of DS’s returned to the client
> Location: [tr.bos-bos-nbc, tr.den-bos-nbc, tr.sea-bos-nbc, tr.den-den-nbc, tr.sea-den-nbc, r.bos-den-nbc,…. ]
>
> Is this what are you ultimately looking for?
>
> This lets us choose CG and DS based on the location of the client relative to the edge cache, rather than proximity of client to origin. Since the client is talking to the edge cache and not the origin, this seems like a better metric. Being able to compose Steering DS would also give us more flexibility for other use cases in the future as well
>
>
>
>
> On Feb 27, 2018, at 4:40 PM, Rawlin Peters <ra...@gmail.com>> wrote:
>
> Hey Eric,
>
> In this example I'd think that all the target DSes would need to be
> assigned to all 3 Cache Groups. That way the client could possibly use
> any of the target DSes from the cachegroup they're routed to. But I
> suppose that could change on a case-by-case basis where maybe the
> target DSes are only assigned to cache groups that are close to their
> origins. In that case I'd think a client in Seattle would possibly be
> routed to an edge cache in Boston, which would optimize the connection
> between the Boston edge cache and the Boston origin. But it might be
> better to have the client connect to an edge cache in Seattle which
> then retrieves the content from a Boston origin (although these are
> both worst-case scenarios).
>
> As far as the content on the origins themselves go, they'd have to be
> interchangeable from the client's perspective, but I'm not sure if it
> would have to be identical or not. I imagine that would really depend
> on what the steering DS is providing the client.
>
> - Rawlin
>
> On Tue, Feb 27, 2018 at 10:37 AM, Eric Friedrich (efriedri)
> <ef...@cisco.com>> wrote:
> In this example, what would be the assignments of delivery services to edge Cache Groups? Are all 3DS’ assigned to all 3 Cache Groups?
>
> I’ll also assume that the content on the origins, while interchangeable from a clients perspective, is not identical? (i.e. might contain regionalized content?)
>
> —Eric
>
>
>
>
> On Feb 23, 2018, at 5:40 PM, Rawlin Peters <ra...@gmail.com>> wrote:
>
> Hey folks,
>
> At Comcast we have a need to augment CLIENT_STEERING (also regular
> STEERING while we're at it) to allow targets to be ordered/sorted
> based upon the client's proximity to the origin of the target delivery
> services. I'd like to get your feedback on my proposed design and
> address any of your concerns.
>
> For HTTP_LIVE targets for instance, we'd like edge caches to ideally
> retrieve/serve data from an Origin that is close to the client and
> fall back to Origins that are farther and farther away. This allows
> for increased redundancy while ordering optimal Origins (Delivery
> Services) for a client to choose from.
>
> For example, I have 3 Origins in different locations: Seattle, Denver,
> and Boston. I would create an HTTP_LIVE delivery service for each
> origin and a CLIENT_STEERING delivery service with those delivery
> services as targets. I would then like to have those targets ordered
> based upon proximity to the client. So a client in Seattle would get
> the list [Seattle, Denver, Boston], while a client in Boston would get
> the list [Boston, Denver, Seattle].
>
> To make things more complicated, I want to add a redundant origin in
> each location and split traffic between them (like STEERING_WEIGHT
> today) while taking into account the geo-ordering. I also want to be
> able to force an ordering (like STEERING_ORDER today) among co-located
> targets.
>
> In order to accomplish this I propose to:
> 1. add two new steering types: GEO_ORDER and GEO_WEIGHT (by adding a
> target of type GEO_*, a steering DS would then enable geo-ordering)
> 2. associate a Delivery Service Origin with a latitude/longitude,
> thereby associating a lat/long to a GEO_* target
>
> Item 1 is pretty straightforward and will also play nicely with the
> current steering types (STEERING_ORDER and STEERING_WEIGHT). I've
> completed a POC within Traffic Router that basically provides the
> following ordering when mixing all 4 types in a single steering
> delivery service:
> - Negative STEERING_ORDER targets
> - GEO_WEIGHT and GEO_ORDER targets, grouped by proximity to the
> client, ordered by geo-order and the consistent-hashing from the
> weightings
> - STEERING_WEIGHT targets (consistent-hashed)
> - Positive STEERING_ORDER targets
>
> Item 2 is not as straightforward because the simple thing would be to
> just add an Origin Lat/Long field to the Delivery Service and call it
> a day. However I don't think we should do that, and I'll dive more
> into that in a separate thread (coming soon).
>
> Does anyone have questions/concerns about adding these new GEO_ORDER
> and GEO_STEERING target types and geo-sorting them based upon
> proximity to the client? Are you okay with the proposed ordering when
> all the steering types are mixed together?
>
> - Rawlin
>
>
Re: Steering Target Geo-Ordering
Posted by "Eric Friedrich (efriedri)" <ef...@cisco.com>.
Thanks Rawlin-
Thanks for bearing with me. I’m thinking through this and wanted to brainstorm a little.
What’s the benefit of going to the Boston edge cache to get the Seattle or Denver DeliveryServices?
- If you already have redundant origins in Boston, you’re protected against failure of a single origin.
- If both Boston origins fail, the client would be routed to Denver and hit the local Denver origins.
- If both Boston origins are down, then you wouldn’t be able to get Boston content in Denver anyway
- But clients in Boston would now need to hit the Denver edges, perhaps overloading that CG. If this is a failure case you want to address, maybe we could do something like the following:
Would it be possible to create multiple levels of steering delivery services?
- Top Level: Client Steering DS (tr.nbc.cdn.customer.com<http://tr.nbc.cdn.com>) - assigned to all CG. Does GeoOrdering based on client proximity to assigned cache groups of the target DS'
- Target 1: Boston Client Steering DS (tr.bos-nbc.cdn.customer.com<http://tr.boston-nbc.cdn.customer.com>) - assigned only to Boston CG. Ranked steering policy
- Target 1.1: Boston Live DS (tr.bos-bos-nbc.cdn.customer.com<http://tr.bos-bos-nbc.cdn.customer.com>), rank=1
- Target 1.2: Denver Live DS (tr.den-bos-nbc.cdn.customer.com<http://tr.den-bos-nbc.cdn.customer.com>), rank=2
- Target 1.3: Seattle Live DS (tr.sea-bos-nbc.cdn.customer.com<http://tr.sea-bos-nbc.cdn.customer.com>), rank=3
- Target 2: Denver Client Steering DS (tr.den-nbc.cdn.customer.com<http://tr.denver-nbc.cdn.customer.com>) - assigned only to Denver CG
- Target 3: Seattle Client Steering DS (tr.seat-nbc.cdn.customer.com<http://tr.seattle-nbc.cdn.customer.com>) - assigned only to Seattle CG
The goal would be this order of DS’s returned to the client
Location: [tr.bos-bos-nbc, tr.den-bos-nbc, tr.sea-bos-nbc, tr.den-den-nbc, tr.sea-den-nbc, r.bos-den-nbc,…. ]
Is this what are you ultimately looking for?
This lets us choose CG and DS based on the location of the client relative to the edge cache, rather than proximity of client to origin. Since the client is talking to the edge cache and not the origin, this seems like a better metric. Being able to compose Steering DS would also give us more flexibility for other use cases in the future as well
On Feb 27, 2018, at 4:40 PM, Rawlin Peters <ra...@gmail.com>> wrote:
Hey Eric,
In this example I'd think that all the target DSes would need to be
assigned to all 3 Cache Groups. That way the client could possibly use
any of the target DSes from the cachegroup they're routed to. But I
suppose that could change on a case-by-case basis where maybe the
target DSes are only assigned to cache groups that are close to their
origins. In that case I'd think a client in Seattle would possibly be
routed to an edge cache in Boston, which would optimize the connection
between the Boston edge cache and the Boston origin. But it might be
better to have the client connect to an edge cache in Seattle which
then retrieves the content from a Boston origin (although these are
both worst-case scenarios).
As far as the content on the origins themselves go, they'd have to be
interchangeable from the client's perspective, but I'm not sure if it
would have to be identical or not. I imagine that would really depend
on what the steering DS is providing the client.
- Rawlin
On Tue, Feb 27, 2018 at 10:37 AM, Eric Friedrich (efriedri)
<ef...@cisco.com>> wrote:
In this example, what would be the assignments of delivery services to edge Cache Groups? Are all 3DS’ assigned to all 3 Cache Groups?
I’ll also assume that the content on the origins, while interchangeable from a clients perspective, is not identical? (i.e. might contain regionalized content?)
—Eric
On Feb 23, 2018, at 5:40 PM, Rawlin Peters <ra...@gmail.com>> wrote:
Hey folks,
At Comcast we have a need to augment CLIENT_STEERING (also regular
STEERING while we're at it) to allow targets to be ordered/sorted
based upon the client's proximity to the origin of the target delivery
services. I'd like to get your feedback on my proposed design and
address any of your concerns.
For HTTP_LIVE targets for instance, we'd like edge caches to ideally
retrieve/serve data from an Origin that is close to the client and
fall back to Origins that are farther and farther away. This allows
for increased redundancy while ordering optimal Origins (Delivery
Services) for a client to choose from.
For example, I have 3 Origins in different locations: Seattle, Denver,
and Boston. I would create an HTTP_LIVE delivery service for each
origin and a CLIENT_STEERING delivery service with those delivery
services as targets. I would then like to have those targets ordered
based upon proximity to the client. So a client in Seattle would get
the list [Seattle, Denver, Boston], while a client in Boston would get
the list [Boston, Denver, Seattle].
To make things more complicated, I want to add a redundant origin in
each location and split traffic between them (like STEERING_WEIGHT
today) while taking into account the geo-ordering. I also want to be
able to force an ordering (like STEERING_ORDER today) among co-located
targets.
In order to accomplish this I propose to:
1. add two new steering types: GEO_ORDER and GEO_WEIGHT (by adding a
target of type GEO_*, a steering DS would then enable geo-ordering)
2. associate a Delivery Service Origin with a latitude/longitude,
thereby associating a lat/long to a GEO_* target
Item 1 is pretty straightforward and will also play nicely with the
current steering types (STEERING_ORDER and STEERING_WEIGHT). I've
completed a POC within Traffic Router that basically provides the
following ordering when mixing all 4 types in a single steering
delivery service:
- Negative STEERING_ORDER targets
- GEO_WEIGHT and GEO_ORDER targets, grouped by proximity to the
client, ordered by geo-order and the consistent-hashing from the
weightings
- STEERING_WEIGHT targets (consistent-hashed)
- Positive STEERING_ORDER targets
Item 2 is not as straightforward because the simple thing would be to
just add an Origin Lat/Long field to the Delivery Service and call it
a day. However I don't think we should do that, and I'll dive more
into that in a separate thread (coming soon).
Does anyone have questions/concerns about adding these new GEO_ORDER
and GEO_STEERING target types and geo-sorting them based upon
proximity to the client? Are you okay with the proposed ordering when
all the steering types are mixed together?
- Rawlin
Re: Steering Target Geo-Ordering
Posted by Rawlin Peters <ra...@gmail.com>.
Hey Eric,
In this example I'd think that all the target DSes would need to be
assigned to all 3 Cache Groups. That way the client could possibly use
any of the target DSes from the cachegroup they're routed to. But I
suppose that could change on a case-by-case basis where maybe the
target DSes are only assigned to cache groups that are close to their
origins. In that case I'd think a client in Seattle would possibly be
routed to an edge cache in Boston, which would optimize the connection
between the Boston edge cache and the Boston origin. But it might be
better to have the client connect to an edge cache in Seattle which
then retrieves the content from a Boston origin (although these are
both worst-case scenarios).
As far as the content on the origins themselves go, they'd have to be
interchangeable from the client's perspective, but I'm not sure if it
would have to be identical or not. I imagine that would really depend
on what the steering DS is providing the client.
- Rawlin
On Tue, Feb 27, 2018 at 10:37 AM, Eric Friedrich (efriedri)
<ef...@cisco.com> wrote:
> In this example, what would be the assignments of delivery services to edge Cache Groups? Are all 3DS’ assigned to all 3 Cache Groups?
>
> I’ll also assume that the content on the origins, while interchangeable from a clients perspective, is not identical? (i.e. might contain regionalized content?)
>
> —Eric
>
>
>
>
>> On Feb 23, 2018, at 5:40 PM, Rawlin Peters <ra...@gmail.com> wrote:
>>
>> Hey folks,
>>
>> At Comcast we have a need to augment CLIENT_STEERING (also regular
>> STEERING while we're at it) to allow targets to be ordered/sorted
>> based upon the client's proximity to the origin of the target delivery
>> services. I'd like to get your feedback on my proposed design and
>> address any of your concerns.
>>
>> For HTTP_LIVE targets for instance, we'd like edge caches to ideally
>> retrieve/serve data from an Origin that is close to the client and
>> fall back to Origins that are farther and farther away. This allows
>> for increased redundancy while ordering optimal Origins (Delivery
>> Services) for a client to choose from.
>>
>> For example, I have 3 Origins in different locations: Seattle, Denver,
>> and Boston. I would create an HTTP_LIVE delivery service for each
>> origin and a CLIENT_STEERING delivery service with those delivery
>> services as targets. I would then like to have those targets ordered
>> based upon proximity to the client. So a client in Seattle would get
>> the list [Seattle, Denver, Boston], while a client in Boston would get
>> the list [Boston, Denver, Seattle].
>>
>> To make things more complicated, I want to add a redundant origin in
>> each location and split traffic between them (like STEERING_WEIGHT
>> today) while taking into account the geo-ordering. I also want to be
>> able to force an ordering (like STEERING_ORDER today) among co-located
>> targets.
>>
>> In order to accomplish this I propose to:
>> 1. add two new steering types: GEO_ORDER and GEO_WEIGHT (by adding a
>> target of type GEO_*, a steering DS would then enable geo-ordering)
>> 2. associate a Delivery Service Origin with a latitude/longitude,
>> thereby associating a lat/long to a GEO_* target
>>
>> Item 1 is pretty straightforward and will also play nicely with the
>> current steering types (STEERING_ORDER and STEERING_WEIGHT). I've
>> completed a POC within Traffic Router that basically provides the
>> following ordering when mixing all 4 types in a single steering
>> delivery service:
>> - Negative STEERING_ORDER targets
>> - GEO_WEIGHT and GEO_ORDER targets, grouped by proximity to the
>> client, ordered by geo-order and the consistent-hashing from the
>> weightings
>> - STEERING_WEIGHT targets (consistent-hashed)
>> - Positive STEERING_ORDER targets
>>
>> Item 2 is not as straightforward because the simple thing would be to
>> just add an Origin Lat/Long field to the Delivery Service and call it
>> a day. However I don't think we should do that, and I'll dive more
>> into that in a separate thread (coming soon).
>>
>> Does anyone have questions/concerns about adding these new GEO_ORDER
>> and GEO_STEERING target types and geo-sorting them based upon
>> proximity to the client? Are you okay with the proposed ordering when
>> all the steering types are mixed together?
>>
>> - Rawlin
>
Re: Steering Target Geo-Ordering
Posted by "Eric Friedrich (efriedri)" <ef...@cisco.com>.
In this example, what would be the assignments of delivery services to edge Cache Groups? Are all 3DS’ assigned to all 3 Cache Groups?
I’ll also assume that the content on the origins, while interchangeable from a clients perspective, is not identical? (i.e. might contain regionalized content?)
—Eric
> On Feb 23, 2018, at 5:40 PM, Rawlin Peters <ra...@gmail.com> wrote:
>
> Hey folks,
>
> At Comcast we have a need to augment CLIENT_STEERING (also regular
> STEERING while we're at it) to allow targets to be ordered/sorted
> based upon the client's proximity to the origin of the target delivery
> services. I'd like to get your feedback on my proposed design and
> address any of your concerns.
>
> For HTTP_LIVE targets for instance, we'd like edge caches to ideally
> retrieve/serve data from an Origin that is close to the client and
> fall back to Origins that are farther and farther away. This allows
> for increased redundancy while ordering optimal Origins (Delivery
> Services) for a client to choose from.
>
> For example, I have 3 Origins in different locations: Seattle, Denver,
> and Boston. I would create an HTTP_LIVE delivery service for each
> origin and a CLIENT_STEERING delivery service with those delivery
> services as targets. I would then like to have those targets ordered
> based upon proximity to the client. So a client in Seattle would get
> the list [Seattle, Denver, Boston], while a client in Boston would get
> the list [Boston, Denver, Seattle].
>
> To make things more complicated, I want to add a redundant origin in
> each location and split traffic between them (like STEERING_WEIGHT
> today) while taking into account the geo-ordering. I also want to be
> able to force an ordering (like STEERING_ORDER today) among co-located
> targets.
>
> In order to accomplish this I propose to:
> 1. add two new steering types: GEO_ORDER and GEO_WEIGHT (by adding a
> target of type GEO_*, a steering DS would then enable geo-ordering)
> 2. associate a Delivery Service Origin with a latitude/longitude,
> thereby associating a lat/long to a GEO_* target
>
> Item 1 is pretty straightforward and will also play nicely with the
> current steering types (STEERING_ORDER and STEERING_WEIGHT). I've
> completed a POC within Traffic Router that basically provides the
> following ordering when mixing all 4 types in a single steering
> delivery service:
> - Negative STEERING_ORDER targets
> - GEO_WEIGHT and GEO_ORDER targets, grouped by proximity to the
> client, ordered by geo-order and the consistent-hashing from the
> weightings
> - STEERING_WEIGHT targets (consistent-hashed)
> - Positive STEERING_ORDER targets
>
> Item 2 is not as straightforward because the simple thing would be to
> just add an Origin Lat/Long field to the Delivery Service and call it
> a day. However I don't think we should do that, and I'll dive more
> into that in a separate thread (coming soon).
>
> Does anyone have questions/concerns about adding these new GEO_ORDER
> and GEO_STEERING target types and geo-sorting them based upon
> proximity to the client? Are you okay with the proposed ordering when
> all the steering types are mixed together?
>
> - Rawlin