You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficcontrol.apache.org by Jeremy Mitchell <mi...@gmail.com> on 2017/05/19 20:09:38 UTC
Re: Drop Server -> Delivery Service assignments?
Bringing up an old email regarding DS to cache assignments.
It sounds like we want to preserve DS/cache assignments (the
deliveryservice_server table) so you can assign a subset of cdn edge caches
to a DS if you want. However, what if we make the assignment of ALL cdn
edge caches to a new DS the default?
So when you create a DS for the Foo cdn thru the API, your new DS is
automagically assigned all the edge caches in the Foo cdn...
(at that point you could always remove caches from a DS)
...or the create DS API could have a boolean flag added to it. for example,
assignAllCaches: true|false. if you pass in true, all caches are assigned
to the DS, if false, none are.
thoughts? I'm trying to think of self-service and i'm guessing most people
will not want to assign caches to a DS...
jeremy
On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <mi...@gmail.com>
wrote:
> gotcha. makes sense.
>
> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <da...@gmail.com>
> wrote:
>
>> I believe Eric had a use case where he wanted to assign certain delivery
>> services to certain caches.
>> If we just use initial dispersion and max DNS answers then we don't have
>> the ability to control which servers are used, just how many.
>>
>>
>>
>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <mi...@gmail.com>
>> wrote:
>>
>>> So what I heard was that potentially we move in this direction:
>>>
>>> 1. No need to explicitly assign edge caches to a delivery service. a
>>> delivery service (by default) would employ ALL edge caches of the cdn in
>>> which the ds belongs to.
>>> 2. Ability to override this default behavior and explicitly assign
>>> specific edge caches to a delivery service (this is what we do today)
>>>
>>> But earlier Mark said "so the same functionality [as #2] can
>>> essentially be reached with assigning all caches, and setting the config
>>> params properly [initial dispersion and max dns answers]"
>>>
>>> so why not ditch #2 in favor of using initial dispersion and max dns
>>> answers to control cache assignments?
>>>
>>> it would be nice imo to get rid of the ds_server table. i'm always in
>>> favor of reducing complexity... :)
>>>
>>> jeremy
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
>>> efriedri@cisco.com> wrote:
>>>
>>>> Yes - I think its a reasonable default to have a delivery service
>>>> assigned to all caches as long as we still keep the ability to assign some
>>>> DS’s to just a subset of caches.
>>>>
>>>> —Eric
>>>>
>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <mt...@apache.org>
>>>> wrote:
>>>>
>>>> Good feedback, thanks. I think having _zero_ caches assigned to a
>>>> delivery service could be added fairly easily, and could function as a good
>>>> default to the assumption of _all_ caches being assigned. Would you agree
>>>> with that?
>>>>
>>>> Thanks,
>>>> Mark
>>>>
>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
>>>> efriedri@cisco.com> wrote:
>>>>
>>>>> Yeah,
>>>>> We have several some use cases where where cache or CG level
>>>>> allocation would be helpful.
>>>>>
>>>>> 1) Assigning new or trial delivery services to a subset of caches
>>>>> (considered beta cache nodes). This is helpful in trying out new DS configs
>>>>> before rolling them out to production.
>>>>>
>>>>> 2) Assigning DSs to caches can be a form of quota or resource
>>>>> reservation on the delivery service. Providers of content to the CDN can
>>>>> receive tiered service by either sharing caches or using dedicated caches.
>>>>>
>>>>> —Eric
>>>>>
>>>>>
>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <mt...@apache.org>
>>>>> wrote:
>>>>>
>>>>> Today, there isn't an actual link from Cache Group to DS -- I think
>>>>> what
>>>>> you are referring to is all done in the app layer to make a nice
>>>>> summary in
>>>>> the UI to allow operators to very quickly assign all caches in a cache
>>>>> group.
>>>>>
>>>>> Do you use that "feature" to assign caches from some cache groups, and
>>>>> not
>>>>> others?
>>>>>
>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
>>>>> efriedri@cisco.com> wrote:
>>>>>
>>>>>> Would we keep the Cache Group -> DS assignment?
>>>>>>
>>>>>> —Eric
>>>>>>
>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <mt...@apache.org>
>>>>>> wrote:
>>>>>> >
>>>>>> > Seeking community opinion on this.
>>>>>> >
>>>>>> > Should we drop the server -> delivery service assignments
>>>>>> completely?
>>>>>> >
>>>>>> > When the project began, selectively assigning caches was the only
>>>>>> way to selectively route traffic within a cache group -- let's say you have
>>>>>> a DNS-routed service with a relatively large library size, you might not
>>>>>> want assign all caches in a cache group to that service for fear you would
>>>>>> 'pollute' those caches. Or, if you were turning up a 'test' DS of which you
>>>>>> wanted to limit exposure on, you might apply the same logic. In recent
>>>>>> releases, Traffic Control supports "initial dispersion" for _both_ DNS and
>>>>>> HTTP-routed services, and "max DNS answers" for DNS-routed services, so the
>>>>>> same functionality can essentially be reached with assigning all caches,
>>>>>> and setting the config params properly.
>>>>>> >
>>>>>> > The motivation for this is to reduce the size & complexity of the
>>>>>> Traffic Router's config (aka CRConfig), as this config scales mostly
>>>>>> linearly with the number of caches and the number of delivery services in
>>>>>> your CDN.
>>>>>> >
>>>>>> > Another option might be to not assign caches as the default, and
>>>>>> individual cache assignments is available, but used only when necessary.
>>>>>> >
>>>>>> > Other thoughts? Clarifications needed?
>>>>>> >
>>>>>> > Thanks,
>>>>>> > Mark
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>
Re:
Posted by Robert Butts <ro...@apache.org>.
Erm, sorry for the (no subject), using the lists.apache.org website failed.
I replied to the real thread, please reply there, not here.
On Wed, Jan 2, 2019 at 11:06 AM Robert O Butts <ro...@apache.org> wrote:
> Reviving this, because I noticed a few things, when working on the DS
> Snapshots PR:
>
> (1) I noticed the DeliveryService-Servers query is _by far_ the slowest
> query we have, building the CRConfig. For a given machine and a given large
> CDN, selecting all DS data takes about ~10ms; the DSS query takes ~200ms.
> This is to trivially select the data, with indexes. If you ever need a
> nontrivial query with nontrivial joins, it quickly becomes seconds.
>
> (2) Writing code, I accidentally generated a CRConfig without
> DeliveryService-Servers. It was ~0.5mb (uncompressed). Compared to ~10mb
> for the same CDN with DSSes.
>
> These sizes and query times are because the DSSes are a cross product of
> DS and Servers.
>
> I think we need to do this, and not only in the interface, but implement
> it such that we (1) don't actually store the DSS mappings, for DSes and
> Servers which are "default-assigned", and (2) don't put them in the
> CRConfig. In terms of performance, I strongly suspect it will be faster for
> the Router to compute them, than the time it takes it to receive all that
> data over the network.
>
> We keep talking about "breaking up the huge CRConfig" - it looks to me
> like removing DSSes will make it so small we don't _need_ to break it up
> anytime soon.
>
> To @mtorluemke 's original point, doing this will significantly improve
> scalability, completely aside from the Self Service and interface benefits.
>
> It sounds like there was consensus here. We just need to do it. I'm
> suggesting, with the evidence above, this really ought to be a higher
> priority. The evidence suggests fixing this at the data level will
> significantly improve the scalability of Traffic Control.
>
> On 2017/06/01 07:14:47, Oren Shemesh <or...@qwilt.com> wrote:
> > Assuming that this suggestion implies that a DS can be assigned to a
> cache
> > set, I believe it is a good, scalable way to handle the issue of server
> > assignments.
> > Cache sets are not mutually exclusive, to you can create different sets
> > according to different needs, and expose only the sets (Not the
> individual
> > caches) to the people assigning DSes to cache sets.
> > e.g. you can have east-coast/west-coast sets, together with having
> > all/large/medium/small sets, together with having a set for 'caches that
> > sit in peering points of type X'.
> >
> > +1 !
> >
> > On Wed, May 31, 2017 at 12:34 AM, Durfey, Ryan <Ry...@comcast.com>
> > wrote:
> >
> > > I want to lobby for a grouping layer for cache assignments. Just like
> we
> > > want to group users and services into Tenant groups I think it makes
> sense
> > > to group caches into “Cache Sets” (I am avoiding the words “cache
> group”).
> > > Assigning individual caches to services is messy and prone to errors
> > > especially when you add new caches to the network. You have to figure
> out
> > > which services it should be assigned and you will end up with really
> > > inconsistent assignments. “Cache Sets” wouldn’t need to be mutually
> > > exclusive. Below are some example use cases. This way when a cache
> gets
> > > added to the network, it goes into the “All” set and is auto-assigned
> > > consistently to all services using this set.
> > >
> > >
> > >
> > > Delivery Service >> “Cache Set” >> Servers
> > >
> > >
> > >
> > > Examples Cache Sets
> > >
> > > - All (default)
> > > - Small Soak (1-4 test caches)
> > > - Med Soak 1 cache in each site
> > > - Large Soak 2 caches in each site
> > > - Site Set (all caches in a site)
> > >
> > >
> > >
> > >
> > >
> > > *Ryan Durfey* M | 303-524-5099 <(303)%20524-5099>
> > >
> > > CDN Support (24x7): 866-405-2993 <(866)%20405-2993> or
> > > cdn_support@comcast.com
> > >
> > >
> > >
> > >
> > >
> > > *From: *Dave Neuman <ne...@apache.org>
> > > *Reply-To: *"users@trafficcontrol.incubator.apache.org" <
> > > users@trafficcontrol.incubator.apache.org>
> > > *Date: *Thursday, May 25, 2017 at 7:55 AM
> > > *To: *"users@trafficcontrol.incubator.apache.org"
> <users@trafficcontrol.
> > > incubator.apache.org>
> > > *Subject: *Re: Drop Server -> Delivery Service assignments?
> > >
> > >
> > >
> > > Sure, the clone server DS assignment should be in the API too, all of
> the
> > > UI functionality should be...
> > >
> > > I figured the action of assigning a server to a DS (whether it is one
> or
> > > all) would be its own API, we shouldn't tightly couple functionality
> like
> > > that.
> > >
> > >
> > >
> > > On Thu, May 25, 2017 at 5:43 AM, Nir Sopher <ni...@qwilt.com> wrote:
> > >
> > > Hi Dave,
> > >
> > >
> > >
> > > As far as I can see, the "clone server's DS assignment" flow is
> supported
> > > only in the UI and does not have a dedicate API endpoint. It should
> > > probably be added to support the suggested flow.
> > >
> > >
> > >
> > > May I suggest that in a similar manner, the API we will have a separate
> > > endpoint assigning all servers to the DS - so the CRUD of adding a DS
> would
> > > not be the one to actually write to the "DS/Server" table?
> > >
> > > The UI can just hide the separation and call both APIs when adding a
> DS.
> > >
> > >
> > >
> > > I believe this will leave the CRUD clearer and will not create a
> situation
> > > that the order of 2 CRUD operations on unrelated tables effects the
> system
> > > behavior: Take the below sequences for example, after sequence "A",
> "DS1"
> > > is deployed on "Server1". This is not the case after sequence "B"
> > >
> > >
> > >
> > > Sequence A:
> > >
> > > POST api/1.2/delivery-service => creating DS1
> > >
> > > POST api/1.2/server => creating Server1
> > >
> > >
> > >
> > > Sequence B:
> > >
> > > POST api/1.2/server => creating Server1
> > >
> > > POST api/1.2/delivery-service => creating DS1
> > >
> > >
> > >
> > > Nir
> > >
> > >
> > >
> > > On Wed, May 24, 2017 at 11:43 PM, Shmulik Asafi <sh...@qwilt.com>
> > > wrote:
> > >
> > > There are still 3 phases, even for the first version of the DS: (1)
> The DS
> > > owner defines the DS, (2) the DS owner asks for it to be deployed, (3)
> the
> > > CDN operator deploys it.
> > >
> > >
> > >
> > > So the automatic assignment itself is part of (3)
> > >
> > >
> > >
> > >
> > >
> > > On 24 May 2017 11:37 p.m., "Dave Neuman" <ne...@apache.org> wrote:
> > >
> > > This is all part of Step 1. This is the first time a Delivery service
> is
> > > created so there are no other versions of it...
> > >
> > >
> > >
> > > On Wed, May 24, 2017 at 2:33 PM, Shmulik Asafi <sh...@qwilt.com>
> wrote:
> > >
> > > Hi,
> > >
> > > Referencing back to Nir's slides in the summit on Self Service, there
> are
> > > three main phases to the flow relevant here:
> > >
> > > 1. The DS-owner user defines/modifies the DS (using configuration
> > > versions)
> > > 2. The DS-owner user declares which DS versions she wants to be
> > > deployed
> > > 3. The CDN-operator user/robot configures what DS versions to assign
> > > to which servers
> > >
> > > If we decide to automatically assign DS to all caches by default, in
> the
> > > Self Service flow it is part of phase 3.
> > >
> > >
> > >
> > > If we want to give the DS-owner an opt-out from automatic assignment to
> > > all caches; it should be a property of whatever tables are used to
> manage
> > > phase 2.
> > >
> > >
> > >
> > > In my view, an immediate implementation of this logic should not affect
> > > the DS table (which in Self Service becomes associated with "phase 1");
> > > only the assignment table (which in Self Service becomes associated
> with
> > > "phase 3").
> > >
> > >
> > >
> > > As long as the boundaries are mentally clear and the code is written in
> > > such a way that respects these future boundaries, I dont believe
> there's
> > > much risk in features like this to Self Service.
> > >
> > > Hope that made sense
> > >
> > > Nir - would you agree?
> > >
> > >
> > >
> > > On Wed, May 24, 2017 at 10:56 PM, Dave Neuman <ne...@apache.org>
> wrote:
> > >
> > > In the system today when you create a server you have the ability to
> clone
> > > the delivery service assignments from another cache or manually update
> the
> > > delivery service -> server mapping by going into individual DSs and
> > > assigning the cache to them. I don't think that needs to change with
> what
> > > Jeremy is proposing.
> > >
> > > I also don't think we should put a special value in the
> > > deliveryservice_server table that means "all caches". If you want to
> assign
> > > all caches to a DS put all of them in the table. If you add a new
> cache
> > > and want your DSs on there, clone the mappings from an existing cache.
> > >
> > >
> > >
> > >
> > >
> > > On Wed, May 24, 2017 at 1:43 PM, Nir Sopher <ni...@qwilt.com> wrote:
> > >
> > > Hi Jeremy,
> > >
> > >
> > >
> > > I’m ok with automatically assigning all edges to a new DS. The aspect
> of
> > > "self-service" can be dealt with in a different manner.
> > >
> > > However, how would you solve the problem when a new server is created?
> How
> > > would you know which DS it should be automatically be assigned?
> > >
> > > The model has no configuration marking the DS as one that should be
> > > deployed on all servers.
> > >
> > > My suggestion was to model it with "undef" value in the DS/Servers
> table.
> > >
> > > DS | Server
> > >
> > > ------+--------
> > >
> > > ds1 | server1 => Allow the CDN owner to deploy the ds on a specific
> > > cache
> > >
> > > ------+--------
> > >
> > > ds2 | *undef *=> Allow the CDN owner to deploy the ds on all caches
> > >
> > >
> > >
> > > This basically means that every place reading the DS/Server table, will
> > > have to use a wrapper, unfolding a the "server=undef" raw, to a raw per
> > > relevant server in the CDN.
> > >
> > > I'm not sure how much effort it requires.
> > >
> > >
> > >
> > > Nir
> > >
> > >
> > >
> > > On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <
> mitchell852@gmail.com>
> > > wrote:
> > >
> > > Nir,
> > >
> > >
> > >
> > > Thinking about self-service, do you envision the "DS owner" creates
> the DS
> > > and the "CDN owner" assigns caches to the DS? This seems like an
> > > unnecessary step to me and one that negates the idea of "self service".
> > >
> > >
> > >
> > > As the "DS owner", I want to create a DS and by default, employ ALL
> caches
> > > in my cdn. To be honest, in 99% of cases I really don't want to be
> bothered
> > > with assigning caches. I want the full power of the CDN behind my DS.
> > >
> > >
> > >
> > > So, I'm suggesting that when a new DS is created thru the API, ALL edge
> > > caches in the cdn are assigned to that DS.....and then, the CDN owner
> (or
> > > maybe DS owner) can remove caches from that DS if they need to.
> > >
> > >
> > >
> > > Jeremy
> > >
> > >
> > >
> > > On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com> wrote:
> > >
> > > @Jeremy
> > >
> > >
> > >
> > > The issue I was trying to point at, is that the "DS-Server" table is
> the
> > > one to provide the separation between the DS owner domain and the CDN
> owner
> > > domain (tenancy wise).
> > >
> > > As I see it, when considering multi tenancy, the DS has its owning
> tenant
> > > which is in charge of the specific record in the DS table, and
> specifically
> > > on adding the DS. The CDN is usually owned by another tenant that
> controls
> > > the deployment - currently by adding the ds/servers record to the
> DS/Server
> > > table.
> > >
> > >
> > >
> > > If the DS/Server table is dismissed or automatically written as a
> result
> > > of a ds addition, it removes the "server assignment" step and the CDN
> owner
> > > loses the control over the deployment decision.
> > >
> > >
> > >
> > > Before I open a discussion about how to solve this new tenancy issue, I
> > > would like to verify it is a real issue and there is not something I'm
> > > missing there.
> > >
> > > What do you think?
> > >
> > >
> > >
> > > Nir
> > >
> > >
> > >
> > > On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com>
> wrote:
> > >
> > > @Nir - maybe your proposal is something to consider more long-term but
> for
> > > the short-term i'm wondering if one of the following can / should be
> > > implemented:
> > >
> > >
> > >
> > > A) creating a DS thru the API (POST /api/*/deliveryservices) can have a
> > > default effect of automatically assigning all edges to the new DS (via
> the deliveryservice_server
> > > table)
> > >
> > >
> > >
> > > or
> > >
> > >
> > >
> > > B) the deliveryservice_server table simple goes away and it becomes
> > > implied that a DS employs all edge caches
> > >
> > >
> > >
> > > "A" would probably be the easiest to be honest.
> > >
> > >
> > >
> > > jeremy
> > >
> > >
> > >
> > > On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com> wrote:
> > >
> > > +1 for having a configuration allowing to apply the DS to the entire
> CDN.
> > >
> > >
> > >
> > > Note that adding a "deploy by default" bool field to the DS itself may
> > > cause tenancy issues, as I believe this field should be in the control
> of
> > > the "CDN owner" tenant, and not of the "DS owner".
> > >
> > >
> > >
> > > What about keeping the ds_server table, and allow "undef" for the
> server
> > > value - meaning "all servers in the CDN"?
> > >
> > > In the future the table may grow and have additional fields to
> white-list
> > > servers by.
> > >
> > > For example:
> > >
> > >
> > >
> > > DS | Server | cache-group | profile
> > >
> > > ------+--------+-------------+--------
> > >
> > > ds1 | s1 | null | null => ds to be applied on s1
> > >
> > > ------+--------+-------------+--------
> > >
> > > ds2 | null | null | null => ds to be applied on all
> > > servers (in ds CDN)
> > >
> > > ------+--------+-------------+--------
> > >
> > > ds3 | null | group1 | null => ds to be applied on all
> > > servers in cache group "group1"
> > >
> > > ------+--------+-------------+--------
> > >
> > > ds4 | null | group1 | ATS-6.2 => ds to be applied on all
> > > servers in cache group "group1", with profile "ATS-6.2"
> > >
> > >
> > >
> > >
> > >
> > > During CR config snapshot and configuration pull (or in advance), the
> > > required calculation would be done, to provide the list of relevant
> DSs per
> > > server.
> > >
> > >
> > >
> > > Also note that moving in this direction is backward compatible as well
> as
> > > supports other non trivial extensions one may want in the future:
> Patterns
> > > (wildcards/regexs) support, as well as black list, using a "policy
> table"
> > > mechanism.
> > >
> > >
> > >
> > > In the below example "ds5" is to be deployed on all caches, except for
> > > caches with "ATS-5" profile
> > >
> > >
> > >
> > > DS | Server | cache-group | profile | deploy-decision
> > >
> > > ------+--------+-------------+---------+----------------
> > >
> > > ds5 | null | null | ATS-5 | false
> > >
> > > ------+--------+-------------+---------+----------------
> > >
> > > ds5 | null | null | null | true
> > >
> > > ------+--------+-------------+---------+----------------
> > >
> > >
> > >
> > >
> > >
> > > Nir
> > >
> > >
> > >
> > > On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <
> mitchell852@gmail.com>
> > > wrote:
> > >
> > > @dave - i'm not sure if you're suggesting:
> > >
> > >
> > >
> > > A) on DS create, by default create an entry in the
> deliveryservice_server
> > > table for each cdn edge cache. so if there are 200 cdn edge caches,
> you end
> > > up with 200 entries in the deliveryservice_server table for that DS.
> > >
> > > B) completely kill the deliveryservice_server table and it becomes
> implied
> > > that a DS uses ALL cdn edge caches (the way mids work today. mids don't
> > > have to be explicitly assigned to a DS)
> > >
> > >
> > >
> > > "A" would allow you to control the scope of caches by using initial
> > > dispersion OR control the scope of caches by simply removing the ones
> you
> > > don't want to receive traffic.
> > >
> > > "B" would only allow you to control the scope of caches by using
> initial
> > > dispersion (or so i think).
> > >
> > >
> > >
> > > jeremy
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>
> wrote:
> > >
> > > +1, I think assigning DSs to all caches by default is fine. Then we
> can
> > > use initial dispersion to make sure traffic is spread between an
> > > appropriate number of caches.
> > >
> > >
> > >
> > > On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <
> mitchell852@gmail.com>
> > > wrote:
> > >
> > > Bringing up an old email regarding DS to cache assignments.
> > >
> > >
> > >
> > > It sounds like we want to preserve DS/cache assignments (the
> > > deliveryservice_server table) so you can assign a subset of cdn edge
> caches
> > > to a DS if you want. However, what if we make the assignment of ALL cdn
> > > edge caches to a new DS the default?
> > >
> > >
> > >
> > > So when you create a DS for the Foo cdn thru the API, your new DS is
> > > automagically assigned all the edge caches in the Foo cdn...
> > >
> > >
> > >
> > > (at that point you could always remove caches from a DS)
> > >
> > >
> > >
> > > ...or the create DS API could have a boolean flag added to it. for
> > > example, assignAllCaches: true|false. if you pass in true, all caches
> are
> > > assigned to the DS, if false, none are.
> > >
> > >
> > >
> > > thoughts? I'm trying to think of self-service and i'm guessing most
> people
> > > will not want to assign caches to a DS...
> > >
> > >
> > >
> > > jeremy
> > >
> > >
> > >
> > > On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
> mitchell852@gmail.com>
> > > wrote:
> > >
> > > gotcha. makes sense.
> > >
> > >
> > >
> > > On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <david.neuman64@gmail.com
> >
> > > wrote:
> > >
> > > I believe Eric had a use case where he wanted to assign certain
> delivery
> > > services to certain caches.
> > >
> > > If we just use initial dispersion and max DNS answers then we don't
> have
> > > the ability to control which servers are used, just how many.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <mitchell852@gmail.com
> >
> > > wrote:
> > >
> > > So what I heard was that potentially we move in this direction:
> > >
> > >
> > >
> > > 1. No need to explicitly assign edge caches to a delivery service. a
> > > delivery service (by default) would employ ALL edge caches of the cdn
> in
> > > which the ds belongs to.
> > >
> > > 2. Ability to override this default behavior and explicitly assign
> > > specific edge caches to a delivery service (this is what we do today)
> > >
> > >
> > >
> > > But earlier Mark said "so the same functionality [as #2] can
> essentially
> > > be reached with assigning all caches, and setting the config params
> > > properly [initial dispersion and max dns answers]"
> > >
> > >
> > >
> > > so why not ditch #2 in favor of using initial dispersion and max dns
> > > answers to control cache assignments?
> > >
> > >
> > >
> > > it would be nice imo to get rid of the ds_server table. i'm always in
> > > favor of reducing complexity... :)
> > >
> > >
> > >
> > > jeremy
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
> > > efriedri@cisco.com> wrote:
> > >
> > > Yes - I think its a reasonable default to have a delivery service
> assigned
> > > to all caches as long as we still keep the ability to assign some
> DS’s to
> > > just a subset of caches.
> > >
> > >
> > >
> > > —Eric
> > >
> > >
> > >
> > > On Jan 4, 2017, at 4:56 PM, Mark Torluemke <mt...@apache.org>
> wrote:
> > >
> > >
> > >
> > > Good feedback, thanks. I think having _zero_ caches assigned to a
> delivery
> > > service could be added fairly easily, and could function as a good
> default
> > > to the assumption of _all_ caches being assigned. Would you agree with
> > > that?
> > >
> > >
> > >
> > > Thanks,
> > >
> > > Mark
> > >
> > >
> > >
> > > On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
> > > efriedri@cisco.com> wrote:
> > >
> > > Yeah,
> > >
> > > We have several some use cases where where cache or CG level
> allocation
> > > would be helpful.
> > >
> > >
> > >
> > > 1) Assigning new or trial delivery services to a subset of caches
> > > (considered beta cache nodes). This is helpful in trying out new DS
> configs
> > > before rolling them out to production.
> > >
> > >
> > >
> > > 2) Assigning DSs to caches can be a form of quota or resource
> reservation
> > > on the delivery service. Providers of content to the CDN can receive
> tiered
> > > service by either sharing caches or using dedicated caches.
> > >
> > >
> > >
> > > —Eric
> > >
> > >
> > >
> > >
> > >
> > > On Jan 4, 2017, at 2:04 PM, Mark Torluemke <mt...@apache.org>
> wrote:
> > >
> > >
> > >
> > > Today, there isn't an actual link from Cache Group to DS -- I think
> what
> > > you are referring to is all done in the app layer to make a nice
> summary in
> > > the UI to allow operators to very quickly assign all caches in a cache
> > > group.
> > >
> > > Do you use that "feature" to assign caches from some cache groups, and
> not
> > > others?
> > >
> > >
> > >
> > > On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
> > > efriedri@cisco.com> wrote:
> > >
> > > Would we keep the Cache Group -> DS assignment?
> > >
> > > —Eric
> > >
> > >
> > > > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <mt...@apache.org>
> > > wrote:
> > > >
> > > > Seeking community opinion on this.
> > > >
> > > > Should we drop the server -> delivery service assignments completely?
> > > >
> > > > When the project began, selectively assigning caches was the only
> way to
> > > selectively route traffic within a cache group -- let's say you have a
> > > DNS-routed service with a relatively large library size, you might not
> want
> > > assign all caches in a cache group to that service for fear you would
> > > 'pollute' those caches. Or, if you were turning up a 'test' DS of
> which you
> > > wanted to limit exposure on, you might apply the same logic. In recent
> > > releases, Traffic Control supports "initial dispersion" for _both_ DNS
> and
> > > HTTP-routed services, and "max DNS answers" for DNS-routed services,
> so the
> > > same functionality can essentially be reached with assigning all
> caches,
> > > and setting the config params properly.
> > > >
> > > > The motivation for this is to reduce the size & complexity of the
> > > Traffic Router's config (aka CRConfig), as this config scales mostly
> > > linearly with the number of caches and the number of delivery services
> in
> > > your CDN.
> > > >
> > > > Another option might be to not assign caches as the default, and
> > > individual cache assignments is available, but used only when
> necessary.
> > > >
> > > > Other thoughts? Clarifications needed?
> > > >
> > > > Thanks,
> > > > Mark
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > --
> > >
> > > *Shmulik Asafi*
> > > Qwilt | Work: +972-72-2221692 <+972%2072-222-1692>| Mobile:
> > > +972-54-6581595 <+972%2054-658-1595>| shmulika@qwilt.com <
> yoav@qwilt.com>
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> > --
> >
> > *Oren Shemesh*
> > Qwilt | Work: +972-72-2221637| Mobile: +972-50-2281168 | orens@qwilt.com
> > <yo...@qwilt.com>
> >
>
Posted by Robert O Butts <ro...@apache.org>.
Reviving this, because I noticed a few things, when working on the DS Snapshots PR:
(1) I noticed the DeliveryService-Servers query is _by far_ the slowest query we have, building the CRConfig. For a given machine and a given large CDN, selecting all DS data takes about ~10ms; the DSS query takes ~200ms. This is to trivially select the data, with indexes. If you ever need a nontrivial query with nontrivial joins, it quickly becomes seconds.
(2) Writing code, I accidentally generated a CRConfig without DeliveryService-Servers. It was ~0.5mb (uncompressed). Compared to ~10mb for the same CDN with DSSes.
These sizes and query times are because the DSSes are a cross product of DS and Servers.
I think we need to do this, and not only in the interface, but implement it such that we (1) don't actually store the DSS mappings, for DSes and Servers which are "default-assigned", and (2) don't put them in the CRConfig. In terms of performance, I strongly suspect it will be faster for the Router to compute them, than the time it takes it to receive all that data over the network.
We keep talking about "breaking up the huge CRConfig" - it looks to me like removing DSSes will make it so small we don't _need_ to break it up anytime soon.
To @mtorluemke 's original point, doing this will significantly improve scalability, completely aside from the Self Service and interface benefits.
It sounds like there was consensus here. We just need to do it. I'm suggesting, with the evidence above, this really ought to be a higher priority. The evidence suggests fixing this at the data level will significantly improve the scalability of Traffic Control.
On 2017/06/01 07:14:47, Oren Shemesh <or...@qwilt.com> wrote:
> Assuming that this suggestion implies that a DS can be assigned to a cache
> set, I believe it is a good, scalable way to handle the issue of server
> assignments.
> Cache sets are not mutually exclusive, to you can create different sets
> according to different needs, and expose only the sets (Not the individual
> caches) to the people assigning DSes to cache sets.
> e.g. you can have east-coast/west-coast sets, together with having
> all/large/medium/small sets, together with having a set for 'caches that
> sit in peering points of type X'.
>
> +1 !
>
> On Wed, May 31, 2017 at 12:34 AM, Durfey, Ryan <Ry...@comcast.com>
> wrote:
>
> > I want to lobby for a grouping layer for cache assignments. Just like we
> > want to group users and services into Tenant groups I think it makes sense
> > to group caches into “Cache Sets” (I am avoiding the words “cache group”).
> > Assigning individual caches to services is messy and prone to errors
> > especially when you add new caches to the network. You have to figure out
> > which services it should be assigned and you will end up with really
> > inconsistent assignments. “Cache Sets” wouldn’t need to be mutually
> > exclusive. Below are some example use cases. This way when a cache gets
> > added to the network, it goes into the “All” set and is auto-assigned
> > consistently to all services using this set.
> >
> >
> >
> > Delivery Service >> “Cache Set” >> Servers
> >
> >
> >
> > Examples Cache Sets
> >
> > - All (default)
> > - Small Soak (1-4 test caches)
> > - Med Soak 1 cache in each site
> > - Large Soak 2 caches in each site
> > - Site Set (all caches in a site)
> >
> >
> >
> >
> >
> > *Ryan Durfey* M | 303-524-5099 <(303)%20524-5099>
> >
> > CDN Support (24x7): 866-405-2993 <(866)%20405-2993> or
> > cdn_support@comcast.com
> >
> >
> >
> >
> >
> > *From: *Dave Neuman <ne...@apache.org>
> > *Reply-To: *"users@trafficcontrol.incubator.apache.org" <
> > users@trafficcontrol.incubator.apache.org>
> > *Date: *Thursday, May 25, 2017 at 7:55 AM
> > *To: *"users@trafficcontrol.incubator.apache.org" <users@trafficcontrol.
> > incubator.apache.org>
> > *Subject: *Re: Drop Server -> Delivery Service assignments?
> >
> >
> >
> > Sure, the clone server DS assignment should be in the API too, all of the
> > UI functionality should be...
> >
> > I figured the action of assigning a server to a DS (whether it is one or
> > all) would be its own API, we shouldn't tightly couple functionality like
> > that.
> >
> >
> >
> > On Thu, May 25, 2017 at 5:43 AM, Nir Sopher <ni...@qwilt.com> wrote:
> >
> > Hi Dave,
> >
> >
> >
> > As far as I can see, the "clone server's DS assignment" flow is supported
> > only in the UI and does not have a dedicate API endpoint. It should
> > probably be added to support the suggested flow.
> >
> >
> >
> > May I suggest that in a similar manner, the API we will have a separate
> > endpoint assigning all servers to the DS - so the CRUD of adding a DS would
> > not be the one to actually write to the "DS/Server" table?
> >
> > The UI can just hide the separation and call both APIs when adding a DS.
> >
> >
> >
> > I believe this will leave the CRUD clearer and will not create a situation
> > that the order of 2 CRUD operations on unrelated tables effects the system
> > behavior: Take the below sequences for example, after sequence "A", "DS1"
> > is deployed on "Server1". This is not the case after sequence "B"
> >
> >
> >
> > Sequence A:
> >
> > POST api/1.2/delivery-service => creating DS1
> >
> > POST api/1.2/server => creating Server1
> >
> >
> >
> > Sequence B:
> >
> > POST api/1.2/server => creating Server1
> >
> > POST api/1.2/delivery-service => creating DS1
> >
> >
> >
> > Nir
> >
> >
> >
> > On Wed, May 24, 2017 at 11:43 PM, Shmulik Asafi <sh...@qwilt.com>
> > wrote:
> >
> > There are still 3 phases, even for the first version of the DS: (1) The DS
> > owner defines the DS, (2) the DS owner asks for it to be deployed, (3) the
> > CDN operator deploys it.
> >
> >
> >
> > So the automatic assignment itself is part of (3)
> >
> >
> >
> >
> >
> > On 24 May 2017 11:37 p.m., "Dave Neuman" <ne...@apache.org> wrote:
> >
> > This is all part of Step 1. This is the first time a Delivery service is
> > created so there are no other versions of it...
> >
> >
> >
> > On Wed, May 24, 2017 at 2:33 PM, Shmulik Asafi <sh...@qwilt.com> wrote:
> >
> > Hi,
> >
> > Referencing back to Nir's slides in the summit on Self Service, there are
> > three main phases to the flow relevant here:
> >
> > 1. The DS-owner user defines/modifies the DS (using configuration
> > versions)
> > 2. The DS-owner user declares which DS versions she wants to be
> > deployed
> > 3. The CDN-operator user/robot configures what DS versions to assign
> > to which servers
> >
> > If we decide to automatically assign DS to all caches by default, in the
> > Self Service flow it is part of phase 3.
> >
> >
> >
> > If we want to give the DS-owner an opt-out from automatic assignment to
> > all caches; it should be a property of whatever tables are used to manage
> > phase 2.
> >
> >
> >
> > In my view, an immediate implementation of this logic should not affect
> > the DS table (which in Self Service becomes associated with "phase 1");
> > only the assignment table (which in Self Service becomes associated with
> > "phase 3").
> >
> >
> >
> > As long as the boundaries are mentally clear and the code is written in
> > such a way that respects these future boundaries, I dont believe there's
> > much risk in features like this to Self Service.
> >
> > Hope that made sense
> >
> > Nir - would you agree?
> >
> >
> >
> > On Wed, May 24, 2017 at 10:56 PM, Dave Neuman <ne...@apache.org> wrote:
> >
> > In the system today when you create a server you have the ability to clone
> > the delivery service assignments from another cache or manually update the
> > delivery service -> server mapping by going into individual DSs and
> > assigning the cache to them. I don't think that needs to change with what
> > Jeremy is proposing.
> >
> > I also don't think we should put a special value in the
> > deliveryservice_server table that means "all caches". If you want to assign
> > all caches to a DS put all of them in the table. If you add a new cache
> > and want your DSs on there, clone the mappings from an existing cache.
> >
> >
> >
> >
> >
> > On Wed, May 24, 2017 at 1:43 PM, Nir Sopher <ni...@qwilt.com> wrote:
> >
> > Hi Jeremy,
> >
> >
> >
> > I’m ok with automatically assigning all edges to a new DS. The aspect of
> > "self-service" can be dealt with in a different manner.
> >
> > However, how would you solve the problem when a new server is created? How
> > would you know which DS it should be automatically be assigned?
> >
> > The model has no configuration marking the DS as one that should be
> > deployed on all servers.
> >
> > My suggestion was to model it with "undef" value in the DS/Servers table.
> >
> > DS | Server
> >
> > ------+--------
> >
> > ds1 | server1 => Allow the CDN owner to deploy the ds on a specific
> > cache
> >
> > ------+--------
> >
> > ds2 | *undef *=> Allow the CDN owner to deploy the ds on all caches
> >
> >
> >
> > This basically means that every place reading the DS/Server table, will
> > have to use a wrapper, unfolding a the "server=undef" raw, to a raw per
> > relevant server in the CDN.
> >
> > I'm not sure how much effort it requires.
> >
> >
> >
> > Nir
> >
> >
> >
> > On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <mi...@gmail.com>
> > wrote:
> >
> > Nir,
> >
> >
> >
> > Thinking about self-service, do you envision the "DS owner" creates the DS
> > and the "CDN owner" assigns caches to the DS? This seems like an
> > unnecessary step to me and one that negates the idea of "self service".
> >
> >
> >
> > As the "DS owner", I want to create a DS and by default, employ ALL caches
> > in my cdn. To be honest, in 99% of cases I really don't want to be bothered
> > with assigning caches. I want the full power of the CDN behind my DS.
> >
> >
> >
> > So, I'm suggesting that when a new DS is created thru the API, ALL edge
> > caches in the cdn are assigned to that DS.....and then, the CDN owner (or
> > maybe DS owner) can remove caches from that DS if they need to.
> >
> >
> >
> > Jeremy
> >
> >
> >
> > On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com> wrote:
> >
> > @Jeremy
> >
> >
> >
> > The issue I was trying to point at, is that the "DS-Server" table is the
> > one to provide the separation between the DS owner domain and the CDN owner
> > domain (tenancy wise).
> >
> > As I see it, when considering multi tenancy, the DS has its owning tenant
> > which is in charge of the specific record in the DS table, and specifically
> > on adding the DS. The CDN is usually owned by another tenant that controls
> > the deployment - currently by adding the ds/servers record to the DS/Server
> > table.
> >
> >
> >
> > If the DS/Server table is dismissed or automatically written as a result
> > of a ds addition, it removes the "server assignment" step and the CDN owner
> > loses the control over the deployment decision.
> >
> >
> >
> > Before I open a discussion about how to solve this new tenancy issue, I
> > would like to verify it is a real issue and there is not something I'm
> > missing there.
> >
> > What do you think?
> >
> >
> >
> > Nir
> >
> >
> >
> > On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com> wrote:
> >
> > @Nir - maybe your proposal is something to consider more long-term but for
> > the short-term i'm wondering if one of the following can / should be
> > implemented:
> >
> >
> >
> > A) creating a DS thru the API (POST /api/*/deliveryservices) can have a
> > default effect of automatically assigning all edges to the new DS (via the deliveryservice_server
> > table)
> >
> >
> >
> > or
> >
> >
> >
> > B) the deliveryservice_server table simple goes away and it becomes
> > implied that a DS employs all edge caches
> >
> >
> >
> > "A" would probably be the easiest to be honest.
> >
> >
> >
> > jeremy
> >
> >
> >
> > On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com> wrote:
> >
> > +1 for having a configuration allowing to apply the DS to the entire CDN.
> >
> >
> >
> > Note that adding a "deploy by default" bool field to the DS itself may
> > cause tenancy issues, as I believe this field should be in the control of
> > the "CDN owner" tenant, and not of the "DS owner".
> >
> >
> >
> > What about keeping the ds_server table, and allow "undef" for the server
> > value - meaning "all servers in the CDN"?
> >
> > In the future the table may grow and have additional fields to white-list
> > servers by.
> >
> > For example:
> >
> >
> >
> > DS | Server | cache-group | profile
> >
> > ------+--------+-------------+--------
> >
> > ds1 | s1 | null | null => ds to be applied on s1
> >
> > ------+--------+-------------+--------
> >
> > ds2 | null | null | null => ds to be applied on all
> > servers (in ds CDN)
> >
> > ------+--------+-------------+--------
> >
> > ds3 | null | group1 | null => ds to be applied on all
> > servers in cache group "group1"
> >
> > ------+--------+-------------+--------
> >
> > ds4 | null | group1 | ATS-6.2 => ds to be applied on all
> > servers in cache group "group1", with profile "ATS-6.2"
> >
> >
> >
> >
> >
> > During CR config snapshot and configuration pull (or in advance), the
> > required calculation would be done, to provide the list of relevant DSs per
> > server.
> >
> >
> >
> > Also note that moving in this direction is backward compatible as well as
> > supports other non trivial extensions one may want in the future: Patterns
> > (wildcards/regexs) support, as well as black list, using a "policy table"
> > mechanism.
> >
> >
> >
> > In the below example "ds5" is to be deployed on all caches, except for
> > caches with "ATS-5" profile
> >
> >
> >
> > DS | Server | cache-group | profile | deploy-decision
> >
> > ------+--------+-------------+---------+----------------
> >
> > ds5 | null | null | ATS-5 | false
> >
> > ------+--------+-------------+---------+----------------
> >
> > ds5 | null | null | null | true
> >
> > ------+--------+-------------+---------+----------------
> >
> >
> >
> >
> >
> > Nir
> >
> >
> >
> > On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <mi...@gmail.com>
> > wrote:
> >
> > @dave - i'm not sure if you're suggesting:
> >
> >
> >
> > A) on DS create, by default create an entry in the deliveryservice_server
> > table for each cdn edge cache. so if there are 200 cdn edge caches, you end
> > up with 200 entries in the deliveryservice_server table for that DS.
> >
> > B) completely kill the deliveryservice_server table and it becomes implied
> > that a DS uses ALL cdn edge caches (the way mids work today. mids don't
> > have to be explicitly assigned to a DS)
> >
> >
> >
> > "A" would allow you to control the scope of caches by using initial
> > dispersion OR control the scope of caches by simply removing the ones you
> > don't want to receive traffic.
> >
> > "B" would only allow you to control the scope of caches by using initial
> > dispersion (or so i think).
> >
> >
> >
> > jeremy
> >
> >
> >
> >
> >
> >
> >
> > On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org> wrote:
> >
> > +1, I think assigning DSs to all caches by default is fine. Then we can
> > use initial dispersion to make sure traffic is spread between an
> > appropriate number of caches.
> >
> >
> >
> > On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <mi...@gmail.com>
> > wrote:
> >
> > Bringing up an old email regarding DS to cache assignments.
> >
> >
> >
> > It sounds like we want to preserve DS/cache assignments (the
> > deliveryservice_server table) so you can assign a subset of cdn edge caches
> > to a DS if you want. However, what if we make the assignment of ALL cdn
> > edge caches to a new DS the default?
> >
> >
> >
> > So when you create a DS for the Foo cdn thru the API, your new DS is
> > automagically assigned all the edge caches in the Foo cdn...
> >
> >
> >
> > (at that point you could always remove caches from a DS)
> >
> >
> >
> > ...or the create DS API could have a boolean flag added to it. for
> > example, assignAllCaches: true|false. if you pass in true, all caches are
> > assigned to the DS, if false, none are.
> >
> >
> >
> > thoughts? I'm trying to think of self-service and i'm guessing most people
> > will not want to assign caches to a DS...
> >
> >
> >
> > jeremy
> >
> >
> >
> > On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <mi...@gmail.com>
> > wrote:
> >
> > gotcha. makes sense.
> >
> >
> >
> > On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <da...@gmail.com>
> > wrote:
> >
> > I believe Eric had a use case where he wanted to assign certain delivery
> > services to certain caches.
> >
> > If we just use initial dispersion and max DNS answers then we don't have
> > the ability to control which servers are used, just how many.
> >
> >
> >
> >
> >
> >
> >
> > On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <mi...@gmail.com>
> > wrote:
> >
> > So what I heard was that potentially we move in this direction:
> >
> >
> >
> > 1. No need to explicitly assign edge caches to a delivery service. a
> > delivery service (by default) would employ ALL edge caches of the cdn in
> > which the ds belongs to.
> >
> > 2. Ability to override this default behavior and explicitly assign
> > specific edge caches to a delivery service (this is what we do today)
> >
> >
> >
> > But earlier Mark said "so the same functionality [as #2] can essentially
> > be reached with assigning all caches, and setting the config params
> > properly [initial dispersion and max dns answers]"
> >
> >
> >
> > so why not ditch #2 in favor of using initial dispersion and max dns
> > answers to control cache assignments?
> >
> >
> >
> > it would be nice imo to get rid of the ds_server table. i'm always in
> > favor of reducing complexity... :)
> >
> >
> >
> > jeremy
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
> > efriedri@cisco.com> wrote:
> >
> > Yes - I think its a reasonable default to have a delivery service assigned
> > to all caches as long as we still keep the ability to assign some DS’s to
> > just a subset of caches.
> >
> >
> >
> > —Eric
> >
> >
> >
> > On Jan 4, 2017, at 4:56 PM, Mark Torluemke <mt...@apache.org> wrote:
> >
> >
> >
> > Good feedback, thanks. I think having _zero_ caches assigned to a delivery
> > service could be added fairly easily, and could function as a good default
> > to the assumption of _all_ caches being assigned. Would you agree with
> > that?
> >
> >
> >
> > Thanks,
> >
> > Mark
> >
> >
> >
> > On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
> > efriedri@cisco.com> wrote:
> >
> > Yeah,
> >
> > We have several some use cases where where cache or CG level allocation
> > would be helpful.
> >
> >
> >
> > 1) Assigning new or trial delivery services to a subset of caches
> > (considered beta cache nodes). This is helpful in trying out new DS configs
> > before rolling them out to production.
> >
> >
> >
> > 2) Assigning DSs to caches can be a form of quota or resource reservation
> > on the delivery service. Providers of content to the CDN can receive tiered
> > service by either sharing caches or using dedicated caches.
> >
> >
> >
> > —Eric
> >
> >
> >
> >
> >
> > On Jan 4, 2017, at 2:04 PM, Mark Torluemke <mt...@apache.org> wrote:
> >
> >
> >
> > Today, there isn't an actual link from Cache Group to DS -- I think what
> > you are referring to is all done in the app layer to make a nice summary in
> > the UI to allow operators to very quickly assign all caches in a cache
> > group.
> >
> > Do you use that "feature" to assign caches from some cache groups, and not
> > others?
> >
> >
> >
> > On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
> > efriedri@cisco.com> wrote:
> >
> > Would we keep the Cache Group -> DS assignment?
> >
> > —Eric
> >
> >
> > > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <mt...@apache.org>
> > wrote:
> > >
> > > Seeking community opinion on this.
> > >
> > > Should we drop the server -> delivery service assignments completely?
> > >
> > > When the project began, selectively assigning caches was the only way to
> > selectively route traffic within a cache group -- let's say you have a
> > DNS-routed service with a relatively large library size, you might not want
> > assign all caches in a cache group to that service for fear you would
> > 'pollute' those caches. Or, if you were turning up a 'test' DS of which you
> > wanted to limit exposure on, you might apply the same logic. In recent
> > releases, Traffic Control supports "initial dispersion" for _both_ DNS and
> > HTTP-routed services, and "max DNS answers" for DNS-routed services, so the
> > same functionality can essentially be reached with assigning all caches,
> > and setting the config params properly.
> > >
> > > The motivation for this is to reduce the size & complexity of the
> > Traffic Router's config (aka CRConfig), as this config scales mostly
> > linearly with the number of caches and the number of delivery services in
> > your CDN.
> > >
> > > Another option might be to not assign caches as the default, and
> > individual cache assignments is available, but used only when necessary.
> > >
> > > Other thoughts? Clarifications needed?
> > >
> > > Thanks,
> > > Mark
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > --
> >
> > *Shmulik Asafi*
> > Qwilt | Work: +972-72-2221692 <+972%2072-222-1692>| Mobile:
> > +972-54-6581595 <+972%2054-658-1595>| shmulika@qwilt.com <yo...@qwilt.com>
> >
> >
> >
> >
> >
> >
> >
>
>
>
> --
>
> *Oren Shemesh*
> Qwilt | Work: +972-72-2221637| Mobile: +972-50-2281168 | orens@qwilt.com
> <yo...@qwilt.com>
>
Re: Drop Server -> Delivery Service assignments?
Posted by Oren Shemesh <or...@qwilt.com>.
Assuming that this suggestion implies that a DS can be assigned to a cache
set, I believe it is a good, scalable way to handle the issue of server
assignments.
Cache sets are not mutually exclusive, to you can create different sets
according to different needs, and expose only the sets (Not the individual
caches) to the people assigning DSes to cache sets.
e.g. you can have east-coast/west-coast sets, together with having
all/large/medium/small sets, together with having a set for 'caches that
sit in peering points of type X'.
+1 !
On Wed, May 31, 2017 at 12:34 AM, Durfey, Ryan <Ry...@comcast.com>
wrote:
> I want to lobby for a grouping layer for cache assignments. Just like we
> want to group users and services into Tenant groups I think it makes sense
> to group caches into “Cache Sets” (I am avoiding the words “cache group”).
> Assigning individual caches to services is messy and prone to errors
> especially when you add new caches to the network. You have to figure out
> which services it should be assigned and you will end up with really
> inconsistent assignments. “Cache Sets” wouldn’t need to be mutually
> exclusive. Below are some example use cases. This way when a cache gets
> added to the network, it goes into the “All” set and is auto-assigned
> consistently to all services using this set.
>
>
>
> Delivery Service >> “Cache Set” >> Servers
>
>
>
> Examples Cache Sets
>
> - All (default)
> - Small Soak (1-4 test caches)
> - Med Soak 1 cache in each site
> - Large Soak 2 caches in each site
> - Site Set (all caches in a site)
>
>
>
>
>
> *Ryan Durfey* M | 303-524-5099 <(303)%20524-5099>
>
> CDN Support (24x7): 866-405-2993 <(866)%20405-2993> or
> cdn_support@comcast.com
>
>
>
>
>
> *From: *Dave Neuman <ne...@apache.org>
> *Reply-To: *"users@trafficcontrol.incubator.apache.org" <
> users@trafficcontrol.incubator.apache.org>
> *Date: *Thursday, May 25, 2017 at 7:55 AM
> *To: *"users@trafficcontrol.incubator.apache.org" <users@trafficcontrol.
> incubator.apache.org>
> *Subject: *Re: Drop Server -> Delivery Service assignments?
>
>
>
> Sure, the clone server DS assignment should be in the API too, all of the
> UI functionality should be...
>
> I figured the action of assigning a server to a DS (whether it is one or
> all) would be its own API, we shouldn't tightly couple functionality like
> that.
>
>
>
> On Thu, May 25, 2017 at 5:43 AM, Nir Sopher <ni...@qwilt.com> wrote:
>
> Hi Dave,
>
>
>
> As far as I can see, the "clone server's DS assignment" flow is supported
> only in the UI and does not have a dedicate API endpoint. It should
> probably be added to support the suggested flow.
>
>
>
> May I suggest that in a similar manner, the API we will have a separate
> endpoint assigning all servers to the DS - so the CRUD of adding a DS would
> not be the one to actually write to the "DS/Server" table?
>
> The UI can just hide the separation and call both APIs when adding a DS.
>
>
>
> I believe this will leave the CRUD clearer and will not create a situation
> that the order of 2 CRUD operations on unrelated tables effects the system
> behavior: Take the below sequences for example, after sequence "A", "DS1"
> is deployed on "Server1". This is not the case after sequence "B"
>
>
>
> Sequence A:
>
> POST api/1.2/delivery-service => creating DS1
>
> POST api/1.2/server => creating Server1
>
>
>
> Sequence B:
>
> POST api/1.2/server => creating Server1
>
> POST api/1.2/delivery-service => creating DS1
>
>
>
> Nir
>
>
>
> On Wed, May 24, 2017 at 11:43 PM, Shmulik Asafi <sh...@qwilt.com>
> wrote:
>
> There are still 3 phases, even for the first version of the DS: (1) The DS
> owner defines the DS, (2) the DS owner asks for it to be deployed, (3) the
> CDN operator deploys it.
>
>
>
> So the automatic assignment itself is part of (3)
>
>
>
>
>
> On 24 May 2017 11:37 p.m., "Dave Neuman" <ne...@apache.org> wrote:
>
> This is all part of Step 1. This is the first time a Delivery service is
> created so there are no other versions of it...
>
>
>
> On Wed, May 24, 2017 at 2:33 PM, Shmulik Asafi <sh...@qwilt.com> wrote:
>
> Hi,
>
> Referencing back to Nir's slides in the summit on Self Service, there are
> three main phases to the flow relevant here:
>
> 1. The DS-owner user defines/modifies the DS (using configuration
> versions)
> 2. The DS-owner user declares which DS versions she wants to be
> deployed
> 3. The CDN-operator user/robot configures what DS versions to assign
> to which servers
>
> If we decide to automatically assign DS to all caches by default, in the
> Self Service flow it is part of phase 3.
>
>
>
> If we want to give the DS-owner an opt-out from automatic assignment to
> all caches; it should be a property of whatever tables are used to manage
> phase 2.
>
>
>
> In my view, an immediate implementation of this logic should not affect
> the DS table (which in Self Service becomes associated with "phase 1");
> only the assignment table (which in Self Service becomes associated with
> "phase 3").
>
>
>
> As long as the boundaries are mentally clear and the code is written in
> such a way that respects these future boundaries, I dont believe there's
> much risk in features like this to Self Service.
>
> Hope that made sense
>
> Nir - would you agree?
>
>
>
> On Wed, May 24, 2017 at 10:56 PM, Dave Neuman <ne...@apache.org> wrote:
>
> In the system today when you create a server you have the ability to clone
> the delivery service assignments from another cache or manually update the
> delivery service -> server mapping by going into individual DSs and
> assigning the cache to them. I don't think that needs to change with what
> Jeremy is proposing.
>
> I also don't think we should put a special value in the
> deliveryservice_server table that means "all caches". If you want to assign
> all caches to a DS put all of them in the table. If you add a new cache
> and want your DSs on there, clone the mappings from an existing cache.
>
>
>
>
>
> On Wed, May 24, 2017 at 1:43 PM, Nir Sopher <ni...@qwilt.com> wrote:
>
> Hi Jeremy,
>
>
>
> I’m ok with automatically assigning all edges to a new DS. The aspect of
> "self-service" can be dealt with in a different manner.
>
> However, how would you solve the problem when a new server is created? How
> would you know which DS it should be automatically be assigned?
>
> The model has no configuration marking the DS as one that should be
> deployed on all servers.
>
> My suggestion was to model it with "undef" value in the DS/Servers table.
>
> DS | Server
>
> ------+--------
>
> ds1 | server1 => Allow the CDN owner to deploy the ds on a specific
> cache
>
> ------+--------
>
> ds2 | *undef *=> Allow the CDN owner to deploy the ds on all caches
>
>
>
> This basically means that every place reading the DS/Server table, will
> have to use a wrapper, unfolding a the "server=undef" raw, to a raw per
> relevant server in the CDN.
>
> I'm not sure how much effort it requires.
>
>
>
> Nir
>
>
>
> On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
> Nir,
>
>
>
> Thinking about self-service, do you envision the "DS owner" creates the DS
> and the "CDN owner" assigns caches to the DS? This seems like an
> unnecessary step to me and one that negates the idea of "self service".
>
>
>
> As the "DS owner", I want to create a DS and by default, employ ALL caches
> in my cdn. To be honest, in 99% of cases I really don't want to be bothered
> with assigning caches. I want the full power of the CDN behind my DS.
>
>
>
> So, I'm suggesting that when a new DS is created thru the API, ALL edge
> caches in the cdn are assigned to that DS.....and then, the CDN owner (or
> maybe DS owner) can remove caches from that DS if they need to.
>
>
>
> Jeremy
>
>
>
> On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com> wrote:
>
> @Jeremy
>
>
>
> The issue I was trying to point at, is that the "DS-Server" table is the
> one to provide the separation between the DS owner domain and the CDN owner
> domain (tenancy wise).
>
> As I see it, when considering multi tenancy, the DS has its owning tenant
> which is in charge of the specific record in the DS table, and specifically
> on adding the DS. The CDN is usually owned by another tenant that controls
> the deployment - currently by adding the ds/servers record to the DS/Server
> table.
>
>
>
> If the DS/Server table is dismissed or automatically written as a result
> of a ds addition, it removes the "server assignment" step and the CDN owner
> loses the control over the deployment decision.
>
>
>
> Before I open a discussion about how to solve this new tenancy issue, I
> would like to verify it is a real issue and there is not something I'm
> missing there.
>
> What do you think?
>
>
>
> Nir
>
>
>
> On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com> wrote:
>
> @Nir - maybe your proposal is something to consider more long-term but for
> the short-term i'm wondering if one of the following can / should be
> implemented:
>
>
>
> A) creating a DS thru the API (POST /api/*/deliveryservices) can have a
> default effect of automatically assigning all edges to the new DS (via the deliveryservice_server
> table)
>
>
>
> or
>
>
>
> B) the deliveryservice_server table simple goes away and it becomes
> implied that a DS employs all edge caches
>
>
>
> "A" would probably be the easiest to be honest.
>
>
>
> jeremy
>
>
>
> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com> wrote:
>
> +1 for having a configuration allowing to apply the DS to the entire CDN.
>
>
>
> Note that adding a "deploy by default" bool field to the DS itself may
> cause tenancy issues, as I believe this field should be in the control of
> the "CDN owner" tenant, and not of the "DS owner".
>
>
>
> What about keeping the ds_server table, and allow "undef" for the server
> value - meaning "all servers in the CDN"?
>
> In the future the table may grow and have additional fields to white-list
> servers by.
>
> For example:
>
>
>
> DS | Server | cache-group | profile
>
> ------+--------+-------------+--------
>
> ds1 | s1 | null | null => ds to be applied on s1
>
> ------+--------+-------------+--------
>
> ds2 | null | null | null => ds to be applied on all
> servers (in ds CDN)
>
> ------+--------+-------------+--------
>
> ds3 | null | group1 | null => ds to be applied on all
> servers in cache group "group1"
>
> ------+--------+-------------+--------
>
> ds4 | null | group1 | ATS-6.2 => ds to be applied on all
> servers in cache group "group1", with profile "ATS-6.2"
>
>
>
>
>
> During CR config snapshot and configuration pull (or in advance), the
> required calculation would be done, to provide the list of relevant DSs per
> server.
>
>
>
> Also note that moving in this direction is backward compatible as well as
> supports other non trivial extensions one may want in the future: Patterns
> (wildcards/regexs) support, as well as black list, using a "policy table"
> mechanism.
>
>
>
> In the below example "ds5" is to be deployed on all caches, except for
> caches with "ATS-5" profile
>
>
>
> DS | Server | cache-group | profile | deploy-decision
>
> ------+--------+-------------+---------+----------------
>
> ds5 | null | null | ATS-5 | false
>
> ------+--------+-------------+---------+----------------
>
> ds5 | null | null | null | true
>
> ------+--------+-------------+---------+----------------
>
>
>
>
>
> Nir
>
>
>
> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
> @dave - i'm not sure if you're suggesting:
>
>
>
> A) on DS create, by default create an entry in the deliveryservice_server
> table for each cdn edge cache. so if there are 200 cdn edge caches, you end
> up with 200 entries in the deliveryservice_server table for that DS.
>
> B) completely kill the deliveryservice_server table and it becomes implied
> that a DS uses ALL cdn edge caches (the way mids work today. mids don't
> have to be explicitly assigned to a DS)
>
>
>
> "A" would allow you to control the scope of caches by using initial
> dispersion OR control the scope of caches by simply removing the ones you
> don't want to receive traffic.
>
> "B" would only allow you to control the scope of caches by using initial
> dispersion (or so i think).
>
>
>
> jeremy
>
>
>
>
>
>
>
> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org> wrote:
>
> +1, I think assigning DSs to all caches by default is fine. Then we can
> use initial dispersion to make sure traffic is spread between an
> appropriate number of caches.
>
>
>
> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
> Bringing up an old email regarding DS to cache assignments.
>
>
>
> It sounds like we want to preserve DS/cache assignments (the
> deliveryservice_server table) so you can assign a subset of cdn edge caches
> to a DS if you want. However, what if we make the assignment of ALL cdn
> edge caches to a new DS the default?
>
>
>
> So when you create a DS for the Foo cdn thru the API, your new DS is
> automagically assigned all the edge caches in the Foo cdn...
>
>
>
> (at that point you could always remove caches from a DS)
>
>
>
> ...or the create DS API could have a boolean flag added to it. for
> example, assignAllCaches: true|false. if you pass in true, all caches are
> assigned to the DS, if false, none are.
>
>
>
> thoughts? I'm trying to think of self-service and i'm guessing most people
> will not want to assign caches to a DS...
>
>
>
> jeremy
>
>
>
> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
> gotcha. makes sense.
>
>
>
> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <da...@gmail.com>
> wrote:
>
> I believe Eric had a use case where he wanted to assign certain delivery
> services to certain caches.
>
> If we just use initial dispersion and max DNS answers then we don't have
> the ability to control which servers are used, just how many.
>
>
>
>
>
>
>
> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
> So what I heard was that potentially we move in this direction:
>
>
>
> 1. No need to explicitly assign edge caches to a delivery service. a
> delivery service (by default) would employ ALL edge caches of the cdn in
> which the ds belongs to.
>
> 2. Ability to override this default behavior and explicitly assign
> specific edge caches to a delivery service (this is what we do today)
>
>
>
> But earlier Mark said "so the same functionality [as #2] can essentially
> be reached with assigning all caches, and setting the config params
> properly [initial dispersion and max dns answers]"
>
>
>
> so why not ditch #2 in favor of using initial dispersion and max dns
> answers to control cache assignments?
>
>
>
> it would be nice imo to get rid of the ds_server table. i'm always in
> favor of reducing complexity... :)
>
>
>
> jeremy
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
> efriedri@cisco.com> wrote:
>
> Yes - I think its a reasonable default to have a delivery service assigned
> to all caches as long as we still keep the ability to assign some DS’s to
> just a subset of caches.
>
>
>
> —Eric
>
>
>
> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <mt...@apache.org> wrote:
>
>
>
> Good feedback, thanks. I think having _zero_ caches assigned to a delivery
> service could be added fairly easily, and could function as a good default
> to the assumption of _all_ caches being assigned. Would you agree with
> that?
>
>
>
> Thanks,
>
> Mark
>
>
>
> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
> efriedri@cisco.com> wrote:
>
> Yeah,
>
> We have several some use cases where where cache or CG level allocation
> would be helpful.
>
>
>
> 1) Assigning new or trial delivery services to a subset of caches
> (considered beta cache nodes). This is helpful in trying out new DS configs
> before rolling them out to production.
>
>
>
> 2) Assigning DSs to caches can be a form of quota or resource reservation
> on the delivery service. Providers of content to the CDN can receive tiered
> service by either sharing caches or using dedicated caches.
>
>
>
> —Eric
>
>
>
>
>
> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <mt...@apache.org> wrote:
>
>
>
> Today, there isn't an actual link from Cache Group to DS -- I think what
> you are referring to is all done in the app layer to make a nice summary in
> the UI to allow operators to very quickly assign all caches in a cache
> group.
>
> Do you use that "feature" to assign caches from some cache groups, and not
> others?
>
>
>
> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
> efriedri@cisco.com> wrote:
>
> Would we keep the Cache Group -> DS assignment?
>
> —Eric
>
>
> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <mt...@apache.org>
> wrote:
> >
> > Seeking community opinion on this.
> >
> > Should we drop the server -> delivery service assignments completely?
> >
> > When the project began, selectively assigning caches was the only way to
> selectively route traffic within a cache group -- let's say you have a
> DNS-routed service with a relatively large library size, you might not want
> assign all caches in a cache group to that service for fear you would
> 'pollute' those caches. Or, if you were turning up a 'test' DS of which you
> wanted to limit exposure on, you might apply the same logic. In recent
> releases, Traffic Control supports "initial dispersion" for _both_ DNS and
> HTTP-routed services, and "max DNS answers" for DNS-routed services, so the
> same functionality can essentially be reached with assigning all caches,
> and setting the config params properly.
> >
> > The motivation for this is to reduce the size & complexity of the
> Traffic Router's config (aka CRConfig), as this config scales mostly
> linearly with the number of caches and the number of delivery services in
> your CDN.
> >
> > Another option might be to not assign caches as the default, and
> individual cache assignments is available, but used only when necessary.
> >
> > Other thoughts? Clarifications needed?
> >
> > Thanks,
> > Mark
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
>
> *Shmulik Asafi*
> Qwilt | Work: +972-72-2221692 <+972%2072-222-1692>| Mobile:
> +972-54-6581595 <+972%2054-658-1595>| shmulika@qwilt.com <yo...@qwilt.com>
>
>
>
>
>
>
>
--
*Oren Shemesh*
Qwilt | Work: +972-72-2221637| Mobile: +972-50-2281168 | orens@qwilt.com
<yo...@qwilt.com>
Re: Drop Server -> Delivery Service assignments?
Posted by "Durfey, Ryan" <Ry...@comcast.com>.
I want to lobby for a grouping layer for cache assignments. Just like we want to group users and services into Tenant groups I think it makes sense to group caches into “Cache Sets” (I am avoiding the words “cache group”). Assigning individual caches to services is messy and prone to errors especially when you add new caches to the network. You have to figure out which services it should be assigned and you will end up with really inconsistent assignments. “Cache Sets” wouldn’t need to be mutually exclusive. Below are some example use cases. This way when a cache gets added to the network, it goes into the “All” set and is auto-assigned consistently to all services using this set.
Delivery Service >> “Cache Set” >> Servers
Examples Cache Sets
* All (default)
* Small Soak (1-4 test caches)
* Med Soak 1 cache in each site
* Large Soak 2 caches in each site
* Site Set (all caches in a site)
Ryan Durfey M | 303-524-5099
CDN Support (24x7): 866-405-2993 or cdn_support@comcast.com<ma...@comcast.com>
From: Dave Neuman <ne...@apache.org>
Reply-To: "users@trafficcontrol.incubator.apache.org" <us...@trafficcontrol.incubator.apache.org>
Date: Thursday, May 25, 2017 at 7:55 AM
To: "users@trafficcontrol.incubator.apache.org" <us...@trafficcontrol.incubator.apache.org>
Subject: Re: Drop Server -> Delivery Service assignments?
Sure, the clone server DS assignment should be in the API too, all of the UI functionality should be...
I figured the action of assigning a server to a DS (whether it is one or all) would be its own API, we shouldn't tightly couple functionality like that.
On Thu, May 25, 2017 at 5:43 AM, Nir Sopher <ni...@qwilt.com>> wrote:
Hi Dave,
As far as I can see, the "clone server's DS assignment" flow is supported only in the UI and does not have a dedicate API endpoint. It should probably be added to support the suggested flow.
May I suggest that in a similar manner, the API we will have a separate endpoint assigning all servers to the DS - so the CRUD of adding a DS would not be the one to actually write to the "DS/Server" table?
The UI can just hide the separation and call both APIs when adding a DS.
I believe this will leave the CRUD clearer and will not create a situation that the order of 2 CRUD operations on unrelated tables effects the system behavior: Take the below sequences for example, after sequence "A", "DS1" is deployed on "Server1". This is not the case after sequence "B"
Sequence A:
POST api/1.2/delivery-service => creating DS1
POST api/1.2/server => creating Server1
Sequence B:
POST api/1.2/server => creating Server1
POST api/1.2/delivery-service => creating DS1
Nir
On Wed, May 24, 2017 at 11:43 PM, Shmulik Asafi <sh...@qwilt.com>> wrote:
There are still 3 phases, even for the first version of the DS: (1) The DS owner defines the DS, (2) the DS owner asks for it to be deployed, (3) the CDN operator deploys it.
So the automatic assignment itself is part of (3)
On 24 May 2017 11:37 p.m., "Dave Neuman" <ne...@apache.org>> wrote:
This is all part of Step 1. This is the first time a Delivery service is created so there are no other versions of it...
On Wed, May 24, 2017 at 2:33 PM, Shmulik Asafi <sh...@qwilt.com>> wrote:
Hi,
Referencing back to Nir's slides in the summit on Self Service, there are three main phases to the flow relevant here:
1. The DS-owner user defines/modifies the DS (using configuration versions)
2. The DS-owner user declares which DS versions she wants to be deployed
3. The CDN-operator user/robot configures what DS versions to assign to which servers
If we decide to automatically assign DS to all caches by default, in the Self Service flow it is part of phase 3.
If we want to give the DS-owner an opt-out from automatic assignment to all caches; it should be a property of whatever tables are used to manage phase 2.
In my view, an immediate implementation of this logic should not affect the DS table (which in Self Service becomes associated with "phase 1"); only the assignment table (which in Self Service becomes associated with "phase 3").
As long as the boundaries are mentally clear and the code is written in such a way that respects these future boundaries, I dont believe there's much risk in features like this to Self Service.
Hope that made sense
Nir - would you agree?
On Wed, May 24, 2017 at 10:56 PM, Dave Neuman <ne...@apache.org>> wrote:
In the system today when you create a server you have the ability to clone the delivery service assignments from another cache or manually update the delivery service -> server mapping by going into individual DSs and assigning the cache to them. I don't think that needs to change with what Jeremy is proposing.
I also don't think we should put a special value in the deliveryservice_server table that means "all caches". If you want to assign all caches to a DS put all of them in the table. If you add a new cache and want your DSs on there, clone the mappings from an existing cache.
On Wed, May 24, 2017 at 1:43 PM, Nir Sopher <ni...@qwilt.com>> wrote:
Hi Jeremy,
I’m ok with automatically assigning all edges to a new DS. The aspect of "self-service" can be dealt with in a different manner.
However, how would you solve the problem when a new server is created? How would you know which DS it should be automatically be assigned?
The model has no configuration marking the DS as one that should be deployed on all servers.
My suggestion was to model it with "undef" value in the DS/Servers table.
DS | Server
------+--------
ds1 | server1 => Allow the CDN owner to deploy the ds on a specific cache
------+--------
ds2 | undef => Allow the CDN owner to deploy the ds on all caches
This basically means that every place reading the DS/Server table, will have to use a wrapper, unfolding a the "server=undef" raw, to a raw per relevant server in the CDN.
I'm not sure how much effort it requires.
Nir
On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <mi...@gmail.com>> wrote:
Nir,
Thinking about self-service, do you envision the "DS owner" creates the DS and the "CDN owner" assigns caches to the DS? This seems like an unnecessary step to me and one that negates the idea of "self service".
As the "DS owner", I want to create a DS and by default, employ ALL caches in my cdn. To be honest, in 99% of cases I really don't want to be bothered with assigning caches. I want the full power of the CDN behind my DS.
So, I'm suggesting that when a new DS is created thru the API, ALL edge caches in the cdn are assigned to that DS.....and then, the CDN owner (or maybe DS owner) can remove caches from that DS if they need to.
Jeremy
On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com>> wrote:
@Jeremy
The issue I was trying to point at, is that the "DS-Server" table is the one to provide the separation between the DS owner domain and the CDN owner domain (tenancy wise).
As I see it, when considering multi tenancy, the DS has its owning tenant which is in charge of the specific record in the DS table, and specifically on adding the DS. The CDN is usually owned by another tenant that controls the deployment - currently by adding the ds/servers record to the DS/Server table.
If the DS/Server table is dismissed or automatically written as a result of a ds addition, it removes the "server assignment" step and the CDN owner loses the control over the deployment decision.
Before I open a discussion about how to solve this new tenancy issue, I would like to verify it is a real issue and there is not something I'm missing there.
What do you think?
Nir
On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com>> wrote:
@Nir - maybe your proposal is something to consider more long-term but for the short-term i'm wondering if one of the following can / should be implemented:
A) creating a DS thru the API (POST /api/*/deliveryservices) can have a default effect of automatically assigning all edges to the new DS (via the deliveryservice_server table)
or
B) the deliveryservice_server table simple goes away and it becomes implied that a DS employs all edge caches
"A" would probably be the easiest to be honest.
jeremy
On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com>> wrote:
+1 for having a configuration allowing to apply the DS to the entire CDN.
Note that adding a "deploy by default" bool field to the DS itself may cause tenancy issues, as I believe this field should be in the control of the "CDN owner" tenant, and not of the "DS owner".
What about keeping the ds_server table, and allow "undef" for the server value - meaning "all servers in the CDN"?
In the future the table may grow and have additional fields to white-list servers by.
For example:
DS | Server | cache-group | profile
------+--------+-------------+--------
ds1 | s1 | null | null => ds to be applied on s1
------+--------+-------------+--------
ds2 | null | null | null => ds to be applied on all servers (in ds CDN)
------+--------+-------------+--------
ds3 | null | group1 | null => ds to be applied on all servers in cache group "group1"
------+--------+-------------+--------
ds4 | null | group1 | ATS-6.2 => ds to be applied on all servers in cache group "group1", with profile "ATS-6.2"
During CR config snapshot and configuration pull (or in advance), the required calculation would be done, to provide the list of relevant DSs per server.
Also note that moving in this direction is backward compatible as well as supports other non trivial extensions one may want in the future: Patterns (wildcards/regexs) support, as well as black list, using a "policy table" mechanism.
In the below example "ds5" is to be deployed on all caches, except for caches with "ATS-5" profile
DS | Server | cache-group | profile | deploy-decision
------+--------+-------------+---------+----------------
ds5 | null | null | ATS-5 | false
------+--------+-------------+---------+----------------
ds5 | null | null | null | true
------+--------+-------------+---------+----------------
Nir
On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <mi...@gmail.com>> wrote:
@dave - i'm not sure if you're suggesting:
A) on DS create, by default create an entry in the deliveryservice_server table for each cdn edge cache. so if there are 200 cdn edge caches, you end up with 200 entries in the deliveryservice_server table for that DS.
B) completely kill the deliveryservice_server table and it becomes implied that a DS uses ALL cdn edge caches (the way mids work today. mids don't have to be explicitly assigned to a DS)
"A" would allow you to control the scope of caches by using initial dispersion OR control the scope of caches by simply removing the ones you don't want to receive traffic.
"B" would only allow you to control the scope of caches by using initial dispersion (or so i think).
jeremy
On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>> wrote:
+1, I think assigning DSs to all caches by default is fine. Then we can use initial dispersion to make sure traffic is spread between an appropriate number of caches.
On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <mi...@gmail.com>> wrote:
Bringing up an old email regarding DS to cache assignments.
It sounds like we want to preserve DS/cache assignments (the deliveryservice_server table) so you can assign a subset of cdn edge caches to a DS if you want. However, what if we make the assignment of ALL cdn edge caches to a new DS the default?
So when you create a DS for the Foo cdn thru the API, your new DS is automagically assigned all the edge caches in the Foo cdn...
(at that point you could always remove caches from a DS)
...or the create DS API could have a boolean flag added to it. for example, assignAllCaches: true|false. if you pass in true, all caches are assigned to the DS, if false, none are.
thoughts? I'm trying to think of self-service and i'm guessing most people will not want to assign caches to a DS...
jeremy
On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <mi...@gmail.com>> wrote:
gotcha. makes sense.
On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <da...@gmail.com>> wrote:
I believe Eric had a use case where he wanted to assign certain delivery services to certain caches.
If we just use initial dispersion and max DNS answers then we don't have the ability to control which servers are used, just how many.
On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <mi...@gmail.com>> wrote:
So what I heard was that potentially we move in this direction:
1. No need to explicitly assign edge caches to a delivery service. a delivery service (by default) would employ ALL edge caches of the cdn in which the ds belongs to.
2. Ability to override this default behavior and explicitly assign specific edge caches to a delivery service (this is what we do today)
But earlier Mark said "so the same functionality [as #2] can essentially be reached with assigning all caches, and setting the config params properly [initial dispersion and max dns answers]"
so why not ditch #2 in favor of using initial dispersion and max dns answers to control cache assignments?
it would be nice imo to get rid of the ds_server table. i'm always in favor of reducing complexity... :)
jeremy
On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <ef...@cisco.com>> wrote:
Yes - I think its a reasonable default to have a delivery service assigned to all caches as long as we still keep the ability to assign some DS’s to just a subset of caches.
—Eric
On Jan 4, 2017, at 4:56 PM, Mark Torluemke <mt...@apache.org>> wrote:
Good feedback, thanks. I think having _zero_ caches assigned to a delivery service could be added fairly easily, and could function as a good default to the assumption of _all_ caches being assigned. Would you agree with that?
Thanks,
Mark
On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <ef...@cisco.com>> wrote:
Yeah,
We have several some use cases where where cache or CG level allocation would be helpful.
1) Assigning new or trial delivery services to a subset of caches (considered beta cache nodes). This is helpful in trying out new DS configs before rolling them out to production.
2) Assigning DSs to caches can be a form of quota or resource reservation on the delivery service. Providers of content to the CDN can receive tiered service by either sharing caches or using dedicated caches.
—Eric
On Jan 4, 2017, at 2:04 PM, Mark Torluemke <mt...@apache.org>> wrote:
Today, there isn't an actual link from Cache Group to DS -- I think what
you are referring to is all done in the app layer to make a nice summary in
the UI to allow operators to very quickly assign all caches in a cache
group.
Do you use that "feature" to assign caches from some cache groups, and not
others?
On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <ef...@cisco.com>> wrote:
Would we keep the Cache Group -> DS assignment?
—Eric
> On Jan 3, 2017, at 9:22 AM, Mark Torluemke <mt...@apache.org>> wrote:
>
> Seeking community opinion on this.
>
> Should we drop the server -> delivery service assignments completely?
>
> When the project began, selectively assigning caches was the only way to selectively route traffic within a cache group -- let's say you have a DNS-routed service with a relatively large library size, you might not want assign all caches in a cache group to that service for fear you would 'pollute' those caches. Or, if you were turning up a 'test' DS of which you wanted to limit exposure on, you might apply the same logic. In recent releases, Traffic Control supports "initial dispersion" for _both_ DNS and HTTP-routed services, and "max DNS answers" for DNS-routed services, so the same functionality can essentially be reached with assigning all caches, and setting the config params properly.
>
> The motivation for this is to reduce the size & complexity of the Traffic Router's config (aka CRConfig), as this config scales mostly linearly with the number of caches and the number of delivery services in your CDN.
>
> Another option might be to not assign caches as the default, and individual cache assignments is available, but used only when necessary.
>
> Other thoughts? Clarifications needed?
>
> Thanks,
> Mark
--
Shmulik Asafi
Qwilt | Work: +972-72-2221692<tel:+972%2072-222-1692>| Mobile: +972-54-6581595<tel:+972%2054-658-1595>| shmulika@qwilt.com<ma...@qwilt.com>
Re: Drop Server -> Delivery Service assignments?
Posted by Dave Neuman <ne...@apache.org>.
Sure, the clone server DS assignment should be in the API too, all of the
UI functionality should be...
I figured the action of assigning a server to a DS (whether it is one or
all) would be its own API, we shouldn't tightly couple functionality like
that.
On Thu, May 25, 2017 at 5:43 AM, Nir Sopher <ni...@qwilt.com> wrote:
> Hi Dave,
>
> As far as I can see, the "clone server's DS assignment" flow is supported
> only in the UI and does not have a dedicate API endpoint. It should
> probably be added to support the suggested flow.
>
> May I suggest that in a similar manner, the API we will have a separate
> endpoint assigning all servers to the DS - so the CRUD of adding a DS would
> not be the one to actually write to the "DS/Server" table?
> The UI can just hide the separation and call both APIs when adding a DS.
>
> I believe this will leave the CRUD clearer and will not create a situation
> that the order of 2 CRUD operations on unrelated tables effects the system
> behavior: Take the below sequences for example, after sequence "A", "DS1"
> is deployed on "Server1". This is not the case after sequence "B"
>
> Sequence A:
> POST api/1.2/delivery-service => creating DS1
> POST api/1.2/server => creating Server1
>
> Sequence B:
> POST api/1.2/server => creating Server1
> POST api/1.2/delivery-service => creating DS1
>
> Nir
>
> On Wed, May 24, 2017 at 11:43 PM, Shmulik Asafi <sh...@qwilt.com>
> wrote:
>
>> There are still 3 phases, even for the first version of the DS: (1) The
>> DS owner defines the DS, (2) the DS owner asks for it to be deployed, (3)
>> the CDN operator deploys it.
>>
>> So the automatic assignment itself is part of (3)
>>
>>
>> On 24 May 2017 11:37 p.m., "Dave Neuman" <ne...@apache.org> wrote:
>>
>>> This is all part of Step 1. This is the first time a Delivery service is
>>> created so there are no other versions of it...
>>>
>>> On Wed, May 24, 2017 at 2:33 PM, Shmulik Asafi <sh...@qwilt.com>
>>> wrote:
>>>
>>>> Hi,
>>>> Referencing back to Nir's slides in the summit on Self Service, there
>>>> are three main phases to the flow relevant here:
>>>>
>>>> 1. The DS-owner user defines/modifies the DS (using configuration
>>>> versions)
>>>> 2. The DS-owner user declares which DS versions she wants to be
>>>> deployed
>>>> 3. The CDN-operator user/robot configures what DS versions to
>>>> assign to which servers
>>>>
>>>> If we decide to automatically assign DS to all caches by default, in
>>>> the Self Service flow it is part of phase 3.
>>>>
>>>> If we want to give the DS-owner an opt-out from automatic assignment to
>>>> all caches; it should be a property of whatever tables are used to manage
>>>> phase 2.
>>>>
>>>> In my view, an immediate implementation of this logic should not affect
>>>> the DS table (which in Self Service becomes associated with "phase 1");
>>>> only the assignment table (which in Self Service becomes associated with
>>>> "phase 3").
>>>>
>>>> As long as the boundaries are mentally clear and the code is written in
>>>> such a way that respects these future boundaries, I dont believe there's
>>>> much risk in features like this to Self Service.
>>>> Hope that made sense
>>>> Nir - would you agree?
>>>>
>>>> On Wed, May 24, 2017 at 10:56 PM, Dave Neuman <ne...@apache.org>
>>>> wrote:
>>>>
>>>>> In the system today when you create a server you have the ability to
>>>>> clone the delivery service assignments from another cache or manually
>>>>> update the delivery service -> server mapping by going into individual DSs
>>>>> and assigning the cache to them. I don't think that needs to change with
>>>>> what Jeremy is proposing.
>>>>> I also don't think we should put a special value in the
>>>>> deliveryservice_server table that means "all caches". If you want to assign
>>>>> all caches to a DS put all of them in the table. If you add a new cache
>>>>> and want your DSs on there, clone the mappings from an existing cache.
>>>>>
>>>>>
>>>>> On Wed, May 24, 2017 at 1:43 PM, Nir Sopher <ni...@qwilt.com> wrote:
>>>>>
>>>>>> Hi Jeremy,
>>>>>>
>>>>>> I’m ok with automatically assigning all edges to a new DS. The aspect
>>>>>> of "self-service" can be dealt with in a different manner.
>>>>>> However, how would you solve the problem when a new server is
>>>>>> created? How would you know which DS it should be automatically be
>>>>>> assigned?
>>>>>> The model has no configuration marking the DS as one that should be
>>>>>> deployed on all servers.
>>>>>> My suggestion was to model it with "undef" value in the DS/Servers
>>>>>> table.
>>>>>> DS | Server
>>>>>> ------+--------
>>>>>> ds1 | server1 => Allow the CDN owner to deploy the ds on a
>>>>>> specific cache
>>>>>> ------+--------
>>>>>> ds2 | *undef *=> Allow the CDN owner to deploy the ds on all
>>>>>> caches
>>>>>>
>>>>>> This basically means that every place reading the DS/Server table,
>>>>>> will have to use a wrapper, unfolding a the "server=undef" raw, to a raw
>>>>>> per relevant server in the CDN.
>>>>>> I'm not sure how much effort it requires.
>>>>>>
>>>>>> Nir
>>>>>>
>>>>>> On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <
>>>>>> mitchell852@gmail.com> wrote:
>>>>>>
>>>>>>> Nir,
>>>>>>>
>>>>>>> Thinking about self-service, do you envision the "DS owner" creates
>>>>>>> the DS and the "CDN owner" assigns caches to the DS? This seems like an
>>>>>>> unnecessary step to me and one that negates the idea of "self service".
>>>>>>>
>>>>>>> As the "DS owner", I want to create a DS and by default, employ ALL
>>>>>>> caches in my cdn. To be honest, in 99% of cases I really don't want to be
>>>>>>> bothered with assigning caches. I want the full power of the CDN behind my
>>>>>>> DS.
>>>>>>>
>>>>>>> So, I'm suggesting that when a new DS is created thru the API, ALL
>>>>>>> edge caches in the cdn are assigned to that DS.....and then, the CDN owner
>>>>>>> (or maybe DS owner) can remove caches from that DS if they need to.
>>>>>>>
>>>>>>> Jeremy
>>>>>>>
>>>>>>> On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com> wrote:
>>>>>>>
>>>>>>>> @Jeremy
>>>>>>>>
>>>>>>>> The issue I was trying to point at, is that the "DS-Server" table
>>>>>>>> is the one to provide the separation between the DS owner domain and the
>>>>>>>> CDN owner domain (tenancy wise).
>>>>>>>> As I see it, when considering multi tenancy, the DS has its owning
>>>>>>>> tenant which is in charge of the specific record in the DS table, and
>>>>>>>> specifically on adding the DS. The CDN is usually owned by another tenant
>>>>>>>> that controls the deployment - currently by adding the ds/servers record to
>>>>>>>> the DS/Server table.
>>>>>>>>
>>>>>>>> If the DS/Server table is dismissed or automatically written as a
>>>>>>>> result of a ds addition, it removes the "server assignment" step and the
>>>>>>>> CDN owner loses the control over the deployment decision.
>>>>>>>>
>>>>>>>> Before I open a discussion about how to solve this new tenancy
>>>>>>>> issue, I would like to verify it is a real issue and there is not something
>>>>>>>> I'm missing there.
>>>>>>>> What do you think?
>>>>>>>>
>>>>>>>> Nir
>>>>>>>>
>>>>>>>> On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> @Nir - maybe your proposal is something to consider more long-term
>>>>>>>>> but for the short-term i'm wondering if one of the following can / should
>>>>>>>>> be implemented:
>>>>>>>>>
>>>>>>>>> A) creating a DS thru the API (POST /api/*/deliveryservices) can
>>>>>>>>> have a default effect of automatically assigning all edges to the new DS
>>>>>>>>> (via the deliveryservice_server table)
>>>>>>>>>
>>>>>>>>> or
>>>>>>>>>
>>>>>>>>> B) the deliveryservice_server table simple goes away and it
>>>>>>>>> becomes implied that a DS employs all edge caches
>>>>>>>>>
>>>>>>>>> "A" would probably be the easiest to be honest.
>>>>>>>>>
>>>>>>>>> jeremy
>>>>>>>>>
>>>>>>>>> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> +1 for having a configuration allowing to apply the DS to the
>>>>>>>>>> entire CDN.
>>>>>>>>>>
>>>>>>>>>> Note that adding a "deploy by default" bool field to the DS
>>>>>>>>>> itself may cause tenancy issues, as I believe this field should be in the
>>>>>>>>>> control of the "CDN owner" tenant, and not of the "DS owner".
>>>>>>>>>>
>>>>>>>>>> What about keeping the ds_server table, and allow "undef" for the
>>>>>>>>>> server value - meaning "all servers in the CDN"?
>>>>>>>>>> In the future the table may grow and have additional fields to
>>>>>>>>>> white-list servers by.
>>>>>>>>>> For example:
>>>>>>>>>>
>>>>>>>>>> DS | Server | cache-group | profile
>>>>>>>>>> ------+--------+-------------+--------
>>>>>>>>>> ds1 | s1 | null | null => ds to be applied
>>>>>>>>>> on s1
>>>>>>>>>> ------+--------+-------------+--------
>>>>>>>>>> ds2 | null | null | null => ds to be applied
>>>>>>>>>> on all servers (in ds CDN)
>>>>>>>>>> ------+--------+-------------+--------
>>>>>>>>>> ds3 | null | group1 | null => ds to be applied
>>>>>>>>>> on all servers in cache group "group1"
>>>>>>>>>> ------+--------+-------------+--------
>>>>>>>>>> ds4 | null | group1 | ATS-6.2 => ds to be applied
>>>>>>>>>> on all servers in cache group "group1", with profile "ATS-6.2"
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> During CR config snapshot and configuration pull (or in advance),
>>>>>>>>>> the required calculation would be done, to provide the list of relevant DSs
>>>>>>>>>> per server.
>>>>>>>>>>
>>>>>>>>>> Also note that moving in this direction is backward compatible as
>>>>>>>>>> well as supports other non trivial extensions one may want in the future:
>>>>>>>>>> Patterns (wildcards/regexs) support, as well as black list, using a "policy
>>>>>>>>>> table" mechanism.
>>>>>>>>>>
>>>>>>>>>> In the below example "ds5" is to be deployed on all caches,
>>>>>>>>>> except for caches with "ATS-5" profile
>>>>>>>>>>
>>>>>>>>>> DS | Server | cache-group | profile | deploy-decision
>>>>>>>>>> ------+--------+-------------+---------+----------------
>>>>>>>>>> ds5 | null | null | ATS-5 | false
>>>>>>>>>> ------+--------+-------------+---------+----------------
>>>>>>>>>> ds5 | null | null | null | true
>>>>>>>>>> ------+--------+-------------+---------+----------------
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Nir
>>>>>>>>>>
>>>>>>>>>> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <
>>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> @dave - i'm not sure if you're suggesting:
>>>>>>>>>>>
>>>>>>>>>>> A) on DS create, by default create an entry in the
>>>>>>>>>>> deliveryservice_server table for each cdn edge cache. so if there are 200
>>>>>>>>>>> cdn edge caches, you end up with 200 entries in the deliveryservice_server
>>>>>>>>>>> table for that DS.
>>>>>>>>>>> B) completely kill the deliveryservice_server table and it
>>>>>>>>>>> becomes implied that a DS uses ALL cdn edge caches (the way
>>>>>>>>>>> mids work today. mids don't have to be explicitly assigned to a DS)
>>>>>>>>>>>
>>>>>>>>>>> "A" would allow you to control the scope of caches by using
>>>>>>>>>>> initial dispersion OR control the scope of caches by simply
>>>>>>>>>>> removing the ones you don't want to receive traffic.
>>>>>>>>>>> "B" would only allow you to control the scope of caches by
>>>>>>>>>>> using initial dispersion (or so i think).
>>>>>>>>>>>
>>>>>>>>>>> jeremy
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> +1, I think assigning DSs to all caches by default is fine.
>>>>>>>>>>>> Then we can use initial dispersion to make sure traffic is spread between
>>>>>>>>>>>> an appropriate number of caches.
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <
>>>>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Bringing up an old email regarding DS to cache assignments.
>>>>>>>>>>>>>
>>>>>>>>>>>>> It sounds like we want to preserve DS/cache assignments (the
>>>>>>>>>>>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
>>>>>>>>>>>>> to a DS if you want. However, what if we make the assignment of ALL cdn
>>>>>>>>>>>>> edge caches to a new DS the default?
>>>>>>>>>>>>>
>>>>>>>>>>>>> So when you create a DS for the Foo cdn thru the API, your new
>>>>>>>>>>>>> DS is automagically assigned all the edge caches in the Foo cdn...
>>>>>>>>>>>>>
>>>>>>>>>>>>> (at that point you could always remove caches from a DS)
>>>>>>>>>>>>>
>>>>>>>>>>>>> ...or the create DS API could have a boolean flag added to it.
>>>>>>>>>>>>> for example, assignAllCaches: true|false. if you pass in true, all caches
>>>>>>>>>>>>> are assigned to the DS, if false, none are.
>>>>>>>>>>>>>
>>>>>>>>>>>>> thoughts? I'm trying to think of self-service and i'm guessing
>>>>>>>>>>>>> most people will not want to assign caches to a DS...
>>>>>>>>>>>>>
>>>>>>>>>>>>> jeremy
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
>>>>>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> gotcha. makes sense.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <
>>>>>>>>>>>>>> david.neuman64@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I believe Eric had a use case where he wanted to assign
>>>>>>>>>>>>>>> certain delivery services to certain caches.
>>>>>>>>>>>>>>> If we just use initial dispersion and max DNS answers then
>>>>>>>>>>>>>>> we don't have the ability to control which servers are used, just how many.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
>>>>>>>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> So what I heard was that potentially we move in this
>>>>>>>>>>>>>>>> direction:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1. No need to explicitly assign edge caches to a delivery
>>>>>>>>>>>>>>>> service. a delivery service (by default) would employ ALL edge caches of
>>>>>>>>>>>>>>>> the cdn in which the ds belongs to.
>>>>>>>>>>>>>>>> 2. Ability to override this default behavior and explicitly
>>>>>>>>>>>>>>>> assign specific edge caches to a delivery service (this is what we do today)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> But earlier Mark said "so the same functionality [as #2]
>>>>>>>>>>>>>>>> can essentially be reached with assigning all caches, and setting the
>>>>>>>>>>>>>>>> config params properly [initial dispersion and max dns answers]"
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> so why not ditch #2 in favor of using initial dispersion
>>>>>>>>>>>>>>>> and max dns answers to control cache assignments?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> it would be nice imo to get rid of the ds_server table. i'm
>>>>>>>>>>>>>>>> always in favor of reducing complexity... :)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> jeremy
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
>>>>>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Yes - I think its a reasonable default to have a delivery
>>>>>>>>>>>>>>>>> service assigned to all caches as long as we still keep the ability to
>>>>>>>>>>>>>>>>> assign some DS’s to just a subset of caches.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <
>>>>>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Good feedback, thanks. I think having _zero_ caches
>>>>>>>>>>>>>>>>> assigned to a delivery service could be added fairly easily, and could
>>>>>>>>>>>>>>>>> function as a good default to the assumption of _all_ caches being
>>>>>>>>>>>>>>>>> assigned. Would you agree with that?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Mark
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri)
>>>>>>>>>>>>>>>>> <ef...@cisco.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Yeah,
>>>>>>>>>>>>>>>>>> We have several some use cases where where cache or CG
>>>>>>>>>>>>>>>>>> level allocation would be helpful.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1) Assigning new or trial delivery services to a subset
>>>>>>>>>>>>>>>>>> of caches (considered beta cache nodes). This is helpful in trying out new
>>>>>>>>>>>>>>>>>> DS configs before rolling them out to production.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or
>>>>>>>>>>>>>>>>>> resource reservation on the delivery service. Providers of content to the
>>>>>>>>>>>>>>>>>> CDN can receive tiered service by either sharing caches or using dedicated
>>>>>>>>>>>>>>>>>> caches.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <
>>>>>>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Today, there isn't an actual link from Cache Group to DS
>>>>>>>>>>>>>>>>>> -- I think what
>>>>>>>>>>>>>>>>>> you are referring to is all done in the app layer to make
>>>>>>>>>>>>>>>>>> a nice summary in
>>>>>>>>>>>>>>>>>> the UI to allow operators to very quickly assign all
>>>>>>>>>>>>>>>>>> caches in a cache
>>>>>>>>>>>>>>>>>> group.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Do you use that "feature" to assign caches from some
>>>>>>>>>>>>>>>>>> cache groups, and not
>>>>>>>>>>>>>>>>>> others?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri)
>>>>>>>>>>>>>>>>>> <ef...@cisco.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
>>>>>>>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>>> > Seeking community opinion on this.
>>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>>> > Should we drop the server -> delivery service
>>>>>>>>>>>>>>>>>>> assignments completely?
>>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>>> > When the project began, selectively assigning caches
>>>>>>>>>>>>>>>>>>> was the only way to selectively route traffic within a cache group -- let's
>>>>>>>>>>>>>>>>>>> say you have a DNS-routed service with a relatively large library size, you
>>>>>>>>>>>>>>>>>>> might not want assign all caches in a cache group to that service for fear
>>>>>>>>>>>>>>>>>>> you would 'pollute' those caches. Or, if you were turning up a 'test' DS of
>>>>>>>>>>>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
>>>>>>>>>>>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
>>>>>>>>>>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
>>>>>>>>>>>>>>>>>>> services, so the same functionality can essentially be reached with
>>>>>>>>>>>>>>>>>>> assigning all caches, and setting the config params properly.
>>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>>> > The motivation for this is to reduce the size &
>>>>>>>>>>>>>>>>>>> complexity of the Traffic Router's config (aka CRConfig), as this config
>>>>>>>>>>>>>>>>>>> scales mostly linearly with the number of caches and the number of delivery
>>>>>>>>>>>>>>>>>>> services in your CDN.
>>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>>> > Another option might be to not assign caches as the
>>>>>>>>>>>>>>>>>>> default, and individual cache assignments is available, but used only when
>>>>>>>>>>>>>>>>>>> necessary.
>>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>>> > Other thoughts? Clarifications needed?
>>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>>> > Thanks,
>>>>>>>>>>>>>>>>>>> > Mark
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Shmulik Asafi*
>>>> Qwilt | Work: +972-72-2221692 <+972%2072-222-1692>| Mobile:
>>>> +972-54-6581595 <+972%2054-658-1595>| shmulika@qwilt.com
>>>> <yo...@qwilt.com>
>>>>
>>>
>>>
>
Re: Drop Server -> Delivery Service assignments?
Posted by Nir Sopher <ni...@qwilt.com>.
Hi Dave,
As far as I can see, the "clone server's DS assignment" flow is supported
only in the UI and does not have a dedicate API endpoint. It should
probably be added to support the suggested flow.
May I suggest that in a similar manner, the API we will have a separate
endpoint assigning all servers to the DS - so the CRUD of adding a DS would
not be the one to actually write to the "DS/Server" table?
The UI can just hide the separation and call both APIs when adding a DS.
I believe this will leave the CRUD clearer and will not create a situation
that the order of 2 CRUD operations on unrelated tables effects the system
behavior: Take the below sequences for example, after sequence "A", "DS1"
is deployed on "Server1". This is not the case after sequence "B"
Sequence A:
POST api/1.2/delivery-service => creating DS1
POST api/1.2/server => creating Server1
Sequence B:
POST api/1.2/server => creating Server1
POST api/1.2/delivery-service => creating DS1
Nir
On Wed, May 24, 2017 at 11:43 PM, Shmulik Asafi <sh...@qwilt.com> wrote:
> There are still 3 phases, even for the first version of the DS: (1) The DS
> owner defines the DS, (2) the DS owner asks for it to be deployed, (3) the
> CDN operator deploys it.
>
> So the automatic assignment itself is part of (3)
>
>
> On 24 May 2017 11:37 p.m., "Dave Neuman" <ne...@apache.org> wrote:
>
>> This is all part of Step 1. This is the first time a Delivery service is
>> created so there are no other versions of it...
>>
>> On Wed, May 24, 2017 at 2:33 PM, Shmulik Asafi <sh...@qwilt.com>
>> wrote:
>>
>>> Hi,
>>> Referencing back to Nir's slides in the summit on Self Service, there
>>> are three main phases to the flow relevant here:
>>>
>>> 1. The DS-owner user defines/modifies the DS (using configuration
>>> versions)
>>> 2. The DS-owner user declares which DS versions she wants to be
>>> deployed
>>> 3. The CDN-operator user/robot configures what DS versions to assign
>>> to which servers
>>>
>>> If we decide to automatically assign DS to all caches by default, in the
>>> Self Service flow it is part of phase 3.
>>>
>>> If we want to give the DS-owner an opt-out from automatic assignment to
>>> all caches; it should be a property of whatever tables are used to manage
>>> phase 2.
>>>
>>> In my view, an immediate implementation of this logic should not affect
>>> the DS table (which in Self Service becomes associated with "phase 1");
>>> only the assignment table (which in Self Service becomes associated with
>>> "phase 3").
>>>
>>> As long as the boundaries are mentally clear and the code is written in
>>> such a way that respects these future boundaries, I dont believe there's
>>> much risk in features like this to Self Service.
>>> Hope that made sense
>>> Nir - would you agree?
>>>
>>> On Wed, May 24, 2017 at 10:56 PM, Dave Neuman <ne...@apache.org> wrote:
>>>
>>>> In the system today when you create a server you have the ability to
>>>> clone the delivery service assignments from another cache or manually
>>>> update the delivery service -> server mapping by going into individual DSs
>>>> and assigning the cache to them. I don't think that needs to change with
>>>> what Jeremy is proposing.
>>>> I also don't think we should put a special value in the
>>>> deliveryservice_server table that means "all caches". If you want to assign
>>>> all caches to a DS put all of them in the table. If you add a new cache
>>>> and want your DSs on there, clone the mappings from an existing cache.
>>>>
>>>>
>>>> On Wed, May 24, 2017 at 1:43 PM, Nir Sopher <ni...@qwilt.com> wrote:
>>>>
>>>>> Hi Jeremy,
>>>>>
>>>>> I’m ok with automatically assigning all edges to a new DS. The aspect
>>>>> of "self-service" can be dealt with in a different manner.
>>>>> However, how would you solve the problem when a new server is created?
>>>>> How would you know which DS it should be automatically be assigned?
>>>>> The model has no configuration marking the DS as one that should be
>>>>> deployed on all servers.
>>>>> My suggestion was to model it with "undef" value in the DS/Servers
>>>>> table.
>>>>> DS | Server
>>>>> ------+--------
>>>>> ds1 | server1 => Allow the CDN owner to deploy the ds on a specific
>>>>> cache
>>>>> ------+--------
>>>>> ds2 | *undef *=> Allow the CDN owner to deploy the ds on all
>>>>> caches
>>>>>
>>>>> This basically means that every place reading the DS/Server table,
>>>>> will have to use a wrapper, unfolding a the "server=undef" raw, to a raw
>>>>> per relevant server in the CDN.
>>>>> I'm not sure how much effort it requires.
>>>>>
>>>>> Nir
>>>>>
>>>>> On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <
>>>>> mitchell852@gmail.com> wrote:
>>>>>
>>>>>> Nir,
>>>>>>
>>>>>> Thinking about self-service, do you envision the "DS owner" creates
>>>>>> the DS and the "CDN owner" assigns caches to the DS? This seems like an
>>>>>> unnecessary step to me and one that negates the idea of "self service".
>>>>>>
>>>>>> As the "DS owner", I want to create a DS and by default, employ ALL
>>>>>> caches in my cdn. To be honest, in 99% of cases I really don't want to be
>>>>>> bothered with assigning caches. I want the full power of the CDN behind my
>>>>>> DS.
>>>>>>
>>>>>> So, I'm suggesting that when a new DS is created thru the API, ALL
>>>>>> edge caches in the cdn are assigned to that DS.....and then, the CDN owner
>>>>>> (or maybe DS owner) can remove caches from that DS if they need to.
>>>>>>
>>>>>> Jeremy
>>>>>>
>>>>>> On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com> wrote:
>>>>>>
>>>>>>> @Jeremy
>>>>>>>
>>>>>>> The issue I was trying to point at, is that the "DS-Server" table is
>>>>>>> the one to provide the separation between the DS owner domain and the CDN
>>>>>>> owner domain (tenancy wise).
>>>>>>> As I see it, when considering multi tenancy, the DS has its owning
>>>>>>> tenant which is in charge of the specific record in the DS table, and
>>>>>>> specifically on adding the DS. The CDN is usually owned by another tenant
>>>>>>> that controls the deployment - currently by adding the ds/servers record to
>>>>>>> the DS/Server table.
>>>>>>>
>>>>>>> If the DS/Server table is dismissed or automatically written as a
>>>>>>> result of a ds addition, it removes the "server assignment" step and the
>>>>>>> CDN owner loses the control over the deployment decision.
>>>>>>>
>>>>>>> Before I open a discussion about how to solve this new tenancy
>>>>>>> issue, I would like to verify it is a real issue and there is not something
>>>>>>> I'm missing there.
>>>>>>> What do you think?
>>>>>>>
>>>>>>> Nir
>>>>>>>
>>>>>>> On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> @Nir - maybe your proposal is something to consider more long-term
>>>>>>>> but for the short-term i'm wondering if one of the following can / should
>>>>>>>> be implemented:
>>>>>>>>
>>>>>>>> A) creating a DS thru the API (POST /api/*/deliveryservices) can
>>>>>>>> have a default effect of automatically assigning all edges to the new DS
>>>>>>>> (via the deliveryservice_server table)
>>>>>>>>
>>>>>>>> or
>>>>>>>>
>>>>>>>> B) the deliveryservice_server table simple goes away and it
>>>>>>>> becomes implied that a DS employs all edge caches
>>>>>>>>
>>>>>>>> "A" would probably be the easiest to be honest.
>>>>>>>>
>>>>>>>> jeremy
>>>>>>>>
>>>>>>>> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> +1 for having a configuration allowing to apply the DS to the
>>>>>>>>> entire CDN.
>>>>>>>>>
>>>>>>>>> Note that adding a "deploy by default" bool field to the DS itself
>>>>>>>>> may cause tenancy issues, as I believe this field should be in the control
>>>>>>>>> of the "CDN owner" tenant, and not of the "DS owner".
>>>>>>>>>
>>>>>>>>> What about keeping the ds_server table, and allow "undef" for the
>>>>>>>>> server value - meaning "all servers in the CDN"?
>>>>>>>>> In the future the table may grow and have additional fields to
>>>>>>>>> white-list servers by.
>>>>>>>>> For example:
>>>>>>>>>
>>>>>>>>> DS | Server | cache-group | profile
>>>>>>>>> ------+--------+-------------+--------
>>>>>>>>> ds1 | s1 | null | null => ds to be applied on
>>>>>>>>> s1
>>>>>>>>> ------+--------+-------------+--------
>>>>>>>>> ds2 | null | null | null => ds to be applied on
>>>>>>>>> all servers (in ds CDN)
>>>>>>>>> ------+--------+-------------+--------
>>>>>>>>> ds3 | null | group1 | null => ds to be applied on
>>>>>>>>> all servers in cache group "group1"
>>>>>>>>> ------+--------+-------------+--------
>>>>>>>>> ds4 | null | group1 | ATS-6.2 => ds to be applied on
>>>>>>>>> all servers in cache group "group1", with profile "ATS-6.2"
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> During CR config snapshot and configuration pull (or in advance),
>>>>>>>>> the required calculation would be done, to provide the list of relevant DSs
>>>>>>>>> per server.
>>>>>>>>>
>>>>>>>>> Also note that moving in this direction is backward compatible as
>>>>>>>>> well as supports other non trivial extensions one may want in the future:
>>>>>>>>> Patterns (wildcards/regexs) support, as well as black list, using a "policy
>>>>>>>>> table" mechanism.
>>>>>>>>>
>>>>>>>>> In the below example "ds5" is to be deployed on all caches, except
>>>>>>>>> for caches with "ATS-5" profile
>>>>>>>>>
>>>>>>>>> DS | Server | cache-group | profile | deploy-decision
>>>>>>>>> ------+--------+-------------+---------+----------------
>>>>>>>>> ds5 | null | null | ATS-5 | false
>>>>>>>>> ------+--------+-------------+---------+----------------
>>>>>>>>> ds5 | null | null | null | true
>>>>>>>>> ------+--------+-------------+---------+----------------
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Nir
>>>>>>>>>
>>>>>>>>> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <
>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> @dave - i'm not sure if you're suggesting:
>>>>>>>>>>
>>>>>>>>>> A) on DS create, by default create an entry in the
>>>>>>>>>> deliveryservice_server table for each cdn edge cache. so if there are 200
>>>>>>>>>> cdn edge caches, you end up with 200 entries in the deliveryservice_server
>>>>>>>>>> table for that DS.
>>>>>>>>>> B) completely kill the deliveryservice_server table and it
>>>>>>>>>> becomes implied that a DS uses ALL cdn edge caches (the way mids
>>>>>>>>>> work today. mids don't have to be explicitly assigned to a DS)
>>>>>>>>>>
>>>>>>>>>> "A" would allow you to control the scope of caches by using
>>>>>>>>>> initial dispersion OR control the scope of caches by simply
>>>>>>>>>> removing the ones you don't want to receive traffic.
>>>>>>>>>> "B" would only allow you to control the scope of caches by using
>>>>>>>>>> initial dispersion (or so i think).
>>>>>>>>>>
>>>>>>>>>> jeremy
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1, I think assigning DSs to all caches by default is fine.
>>>>>>>>>>> Then we can use initial dispersion to make sure traffic is spread between
>>>>>>>>>>> an appropriate number of caches.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <
>>>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Bringing up an old email regarding DS to cache assignments.
>>>>>>>>>>>>
>>>>>>>>>>>> It sounds like we want to preserve DS/cache assignments (the
>>>>>>>>>>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
>>>>>>>>>>>> to a DS if you want. However, what if we make the assignment of ALL cdn
>>>>>>>>>>>> edge caches to a new DS the default?
>>>>>>>>>>>>
>>>>>>>>>>>> So when you create a DS for the Foo cdn thru the API, your new
>>>>>>>>>>>> DS is automagically assigned all the edge caches in the Foo cdn...
>>>>>>>>>>>>
>>>>>>>>>>>> (at that point you could always remove caches from a DS)
>>>>>>>>>>>>
>>>>>>>>>>>> ...or the create DS API could have a boolean flag added to it.
>>>>>>>>>>>> for example, assignAllCaches: true|false. if you pass in true, all caches
>>>>>>>>>>>> are assigned to the DS, if false, none are.
>>>>>>>>>>>>
>>>>>>>>>>>> thoughts? I'm trying to think of self-service and i'm guessing
>>>>>>>>>>>> most people will not want to assign caches to a DS...
>>>>>>>>>>>>
>>>>>>>>>>>> jeremy
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
>>>>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> gotcha. makes sense.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <
>>>>>>>>>>>>> david.neuman64@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I believe Eric had a use case where he wanted to assign
>>>>>>>>>>>>>> certain delivery services to certain caches.
>>>>>>>>>>>>>> If we just use initial dispersion and max DNS answers then we
>>>>>>>>>>>>>> don't have the ability to control which servers are used, just how many.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
>>>>>>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> So what I heard was that potentially we move in this
>>>>>>>>>>>>>>> direction:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1. No need to explicitly assign edge caches to a delivery
>>>>>>>>>>>>>>> service. a delivery service (by default) would employ ALL edge caches of
>>>>>>>>>>>>>>> the cdn in which the ds belongs to.
>>>>>>>>>>>>>>> 2. Ability to override this default behavior and explicitly
>>>>>>>>>>>>>>> assign specific edge caches to a delivery service (this is what we do today)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> But earlier Mark said "so the same functionality [as #2]
>>>>>>>>>>>>>>> can essentially be reached with assigning all caches, and setting the
>>>>>>>>>>>>>>> config params properly [initial dispersion and max dns answers]"
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> so why not ditch #2 in favor of using initial dispersion
>>>>>>>>>>>>>>> and max dns answers to control cache assignments?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> it would be nice imo to get rid of the ds_server table. i'm
>>>>>>>>>>>>>>> always in favor of reducing complexity... :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> jeremy
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
>>>>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yes - I think its a reasonable default to have a delivery
>>>>>>>>>>>>>>>> service assigned to all caches as long as we still keep the ability to
>>>>>>>>>>>>>>>> assign some DS’s to just a subset of caches.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <
>>>>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Good feedback, thanks. I think having _zero_ caches
>>>>>>>>>>>>>>>> assigned to a delivery service could be added fairly easily, and could
>>>>>>>>>>>>>>>> function as a good default to the assumption of _all_ caches being
>>>>>>>>>>>>>>>> assigned. Would you agree with that?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Mark
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
>>>>>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Yeah,
>>>>>>>>>>>>>>>>> We have several some use cases where where cache or CG
>>>>>>>>>>>>>>>>> level allocation would be helpful.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1) Assigning new or trial delivery services to a subset of
>>>>>>>>>>>>>>>>> caches (considered beta cache nodes). This is helpful in trying out new DS
>>>>>>>>>>>>>>>>> configs before rolling them out to production.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or
>>>>>>>>>>>>>>>>> resource reservation on the delivery service. Providers of content to the
>>>>>>>>>>>>>>>>> CDN can receive tiered service by either sharing caches or using dedicated
>>>>>>>>>>>>>>>>> caches.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <
>>>>>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Today, there isn't an actual link from Cache Group to DS
>>>>>>>>>>>>>>>>> -- I think what
>>>>>>>>>>>>>>>>> you are referring to is all done in the app layer to make
>>>>>>>>>>>>>>>>> a nice summary in
>>>>>>>>>>>>>>>>> the UI to allow operators to very quickly assign all
>>>>>>>>>>>>>>>>> caches in a cache
>>>>>>>>>>>>>>>>> group.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Do you use that "feature" to assign caches from some cache
>>>>>>>>>>>>>>>>> groups, and not
>>>>>>>>>>>>>>>>> others?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri)
>>>>>>>>>>>>>>>>> <ef...@cisco.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
>>>>>>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>> > Seeking community opinion on this.
>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>> > Should we drop the server -> delivery service
>>>>>>>>>>>>>>>>>> assignments completely?
>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>> > When the project began, selectively assigning caches
>>>>>>>>>>>>>>>>>> was the only way to selectively route traffic within a cache group -- let's
>>>>>>>>>>>>>>>>>> say you have a DNS-routed service with a relatively large library size, you
>>>>>>>>>>>>>>>>>> might not want assign all caches in a cache group to that service for fear
>>>>>>>>>>>>>>>>>> you would 'pollute' those caches. Or, if you were turning up a 'test' DS of
>>>>>>>>>>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
>>>>>>>>>>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
>>>>>>>>>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
>>>>>>>>>>>>>>>>>> services, so the same functionality can essentially be reached with
>>>>>>>>>>>>>>>>>> assigning all caches, and setting the config params properly.
>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>> > The motivation for this is to reduce the size &
>>>>>>>>>>>>>>>>>> complexity of the Traffic Router's config (aka CRConfig), as this config
>>>>>>>>>>>>>>>>>> scales mostly linearly with the number of caches and the number of delivery
>>>>>>>>>>>>>>>>>> services in your CDN.
>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>> > Another option might be to not assign caches as the
>>>>>>>>>>>>>>>>>> default, and individual cache assignments is available, but used only when
>>>>>>>>>>>>>>>>>> necessary.
>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>> > Other thoughts? Clarifications needed?
>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>> > Thanks,
>>>>>>>>>>>>>>>>>> > Mark
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> *Shmulik Asafi*
>>> Qwilt | Work: +972-72-2221692 <+972%2072-222-1692>| Mobile:
>>> +972-54-6581595 <+972%2054-658-1595>| shmulika@qwilt.com
>>> <yo...@qwilt.com>
>>>
>>
>>
Re: Drop Server -> Delivery Service assignments?
Posted by Shmulik Asafi <sh...@qwilt.com>.
There are still 3 phases, even for the first version of the DS: (1) The DS
owner defines the DS, (2) the DS owner asks for it to be deployed, (3) the
CDN operator deploys it.
So the automatic assignment itself is part of (3)
On 24 May 2017 11:37 p.m., "Dave Neuman" <ne...@apache.org> wrote:
> This is all part of Step 1. This is the first time a Delivery service is
> created so there are no other versions of it...
>
> On Wed, May 24, 2017 at 2:33 PM, Shmulik Asafi <sh...@qwilt.com> wrote:
>
>> Hi,
>> Referencing back to Nir's slides in the summit on Self Service, there are
>> three main phases to the flow relevant here:
>>
>> 1. The DS-owner user defines/modifies the DS (using configuration
>> versions)
>> 2. The DS-owner user declares which DS versions she wants to be
>> deployed
>> 3. The CDN-operator user/robot configures what DS versions to assign
>> to which servers
>>
>> If we decide to automatically assign DS to all caches by default, in the
>> Self Service flow it is part of phase 3.
>>
>> If we want to give the DS-owner an opt-out from automatic assignment to
>> all caches; it should be a property of whatever tables are used to manage
>> phase 2.
>>
>> In my view, an immediate implementation of this logic should not affect
>> the DS table (which in Self Service becomes associated with "phase 1");
>> only the assignment table (which in Self Service becomes associated with
>> "phase 3").
>>
>> As long as the boundaries are mentally clear and the code is written in
>> such a way that respects these future boundaries, I dont believe there's
>> much risk in features like this to Self Service.
>> Hope that made sense
>> Nir - would you agree?
>>
>> On Wed, May 24, 2017 at 10:56 PM, Dave Neuman <ne...@apache.org> wrote:
>>
>>> In the system today when you create a server you have the ability to
>>> clone the delivery service assignments from another cache or manually
>>> update the delivery service -> server mapping by going into individual DSs
>>> and assigning the cache to them. I don't think that needs to change with
>>> what Jeremy is proposing.
>>> I also don't think we should put a special value in the
>>> deliveryservice_server table that means "all caches". If you want to assign
>>> all caches to a DS put all of them in the table. If you add a new cache
>>> and want your DSs on there, clone the mappings from an existing cache.
>>>
>>>
>>> On Wed, May 24, 2017 at 1:43 PM, Nir Sopher <ni...@qwilt.com> wrote:
>>>
>>>> Hi Jeremy,
>>>>
>>>> I’m ok with automatically assigning all edges to a new DS. The aspect
>>>> of "self-service" can be dealt with in a different manner.
>>>> However, how would you solve the problem when a new server is created?
>>>> How would you know which DS it should be automatically be assigned?
>>>> The model has no configuration marking the DS as one that should be
>>>> deployed on all servers.
>>>> My suggestion was to model it with "undef" value in the DS/Servers
>>>> table.
>>>> DS | Server
>>>> ------+--------
>>>> ds1 | server1 => Allow the CDN owner to deploy the ds on a specific
>>>> cache
>>>> ------+--------
>>>> ds2 | *undef *=> Allow the CDN owner to deploy the ds on all caches
>>>>
>>>> This basically means that every place reading the DS/Server table, will
>>>> have to use a wrapper, unfolding a the "server=undef" raw, to a raw per
>>>> relevant server in the CDN.
>>>> I'm not sure how much effort it requires.
>>>>
>>>> Nir
>>>>
>>>> On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <mitchell852@gmail.com
>>>> > wrote:
>>>>
>>>>> Nir,
>>>>>
>>>>> Thinking about self-service, do you envision the "DS owner" creates
>>>>> the DS and the "CDN owner" assigns caches to the DS? This seems like an
>>>>> unnecessary step to me and one that negates the idea of "self service".
>>>>>
>>>>> As the "DS owner", I want to create a DS and by default, employ ALL
>>>>> caches in my cdn. To be honest, in 99% of cases I really don't want to be
>>>>> bothered with assigning caches. I want the full power of the CDN behind my
>>>>> DS.
>>>>>
>>>>> So, I'm suggesting that when a new DS is created thru the API, ALL
>>>>> edge caches in the cdn are assigned to that DS.....and then, the CDN owner
>>>>> (or maybe DS owner) can remove caches from that DS if they need to.
>>>>>
>>>>> Jeremy
>>>>>
>>>>> On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com> wrote:
>>>>>
>>>>>> @Jeremy
>>>>>>
>>>>>> The issue I was trying to point at, is that the "DS-Server" table is
>>>>>> the one to provide the separation between the DS owner domain and the CDN
>>>>>> owner domain (tenancy wise).
>>>>>> As I see it, when considering multi tenancy, the DS has its owning
>>>>>> tenant which is in charge of the specific record in the DS table, and
>>>>>> specifically on adding the DS. The CDN is usually owned by another tenant
>>>>>> that controls the deployment - currently by adding the ds/servers record to
>>>>>> the DS/Server table.
>>>>>>
>>>>>> If the DS/Server table is dismissed or automatically written as a
>>>>>> result of a ds addition, it removes the "server assignment" step and the
>>>>>> CDN owner loses the control over the deployment decision.
>>>>>>
>>>>>> Before I open a discussion about how to solve this new tenancy issue,
>>>>>> I would like to verify it is a real issue and there is not something I'm
>>>>>> missing there.
>>>>>> What do you think?
>>>>>>
>>>>>> Nir
>>>>>>
>>>>>> On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> @Nir - maybe your proposal is something to consider more long-term
>>>>>>> but for the short-term i'm wondering if one of the following can / should
>>>>>>> be implemented:
>>>>>>>
>>>>>>> A) creating a DS thru the API (POST /api/*/deliveryservices) can
>>>>>>> have a default effect of automatically assigning all edges to the new DS
>>>>>>> (via the deliveryservice_server table)
>>>>>>>
>>>>>>> or
>>>>>>>
>>>>>>> B) the deliveryservice_server table simple goes away and it becomes
>>>>>>> implied that a DS employs all edge caches
>>>>>>>
>>>>>>> "A" would probably be the easiest to be honest.
>>>>>>>
>>>>>>> jeremy
>>>>>>>
>>>>>>> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com> wrote:
>>>>>>>
>>>>>>>> +1 for having a configuration allowing to apply the DS to the
>>>>>>>> entire CDN.
>>>>>>>>
>>>>>>>> Note that adding a "deploy by default" bool field to the DS itself
>>>>>>>> may cause tenancy issues, as I believe this field should be in the control
>>>>>>>> of the "CDN owner" tenant, and not of the "DS owner".
>>>>>>>>
>>>>>>>> What about keeping the ds_server table, and allow "undef" for the
>>>>>>>> server value - meaning "all servers in the CDN"?
>>>>>>>> In the future the table may grow and have additional fields to
>>>>>>>> white-list servers by.
>>>>>>>> For example:
>>>>>>>>
>>>>>>>> DS | Server | cache-group | profile
>>>>>>>> ------+--------+-------------+--------
>>>>>>>> ds1 | s1 | null | null => ds to be applied on
>>>>>>>> s1
>>>>>>>> ------+--------+-------------+--------
>>>>>>>> ds2 | null | null | null => ds to be applied on
>>>>>>>> all servers (in ds CDN)
>>>>>>>> ------+--------+-------------+--------
>>>>>>>> ds3 | null | group1 | null => ds to be applied on
>>>>>>>> all servers in cache group "group1"
>>>>>>>> ------+--------+-------------+--------
>>>>>>>> ds4 | null | group1 | ATS-6.2 => ds to be applied on
>>>>>>>> all servers in cache group "group1", with profile "ATS-6.2"
>>>>>>>>
>>>>>>>>
>>>>>>>> During CR config snapshot and configuration pull (or in advance),
>>>>>>>> the required calculation would be done, to provide the list of relevant DSs
>>>>>>>> per server.
>>>>>>>>
>>>>>>>> Also note that moving in this direction is backward compatible as
>>>>>>>> well as supports other non trivial extensions one may want in the future:
>>>>>>>> Patterns (wildcards/regexs) support, as well as black list, using a "policy
>>>>>>>> table" mechanism.
>>>>>>>>
>>>>>>>> In the below example "ds5" is to be deployed on all caches, except
>>>>>>>> for caches with "ATS-5" profile
>>>>>>>>
>>>>>>>> DS | Server | cache-group | profile | deploy-decision
>>>>>>>> ------+--------+-------------+---------+----------------
>>>>>>>> ds5 | null | null | ATS-5 | false
>>>>>>>> ------+--------+-------------+---------+----------------
>>>>>>>> ds5 | null | null | null | true
>>>>>>>> ------+--------+-------------+---------+----------------
>>>>>>>>
>>>>>>>>
>>>>>>>> Nir
>>>>>>>>
>>>>>>>> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <
>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> @dave - i'm not sure if you're suggesting:
>>>>>>>>>
>>>>>>>>> A) on DS create, by default create an entry in the
>>>>>>>>> deliveryservice_server table for each cdn edge cache. so if there are 200
>>>>>>>>> cdn edge caches, you end up with 200 entries in the deliveryservice_server
>>>>>>>>> table for that DS.
>>>>>>>>> B) completely kill the deliveryservice_server table and it becomes
>>>>>>>>> implied that a DS uses ALL cdn edge caches (the way mids work
>>>>>>>>> today. mids don't have to be explicitly assigned to a DS)
>>>>>>>>>
>>>>>>>>> "A" would allow you to control the scope of caches by using
>>>>>>>>> initial dispersion OR control the scope of caches by simply
>>>>>>>>> removing the ones you don't want to receive traffic.
>>>>>>>>> "B" would only allow you to control the scope of caches by using
>>>>>>>>> initial dispersion (or so i think).
>>>>>>>>>
>>>>>>>>> jeremy
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> +1, I think assigning DSs to all caches by default is fine. Then
>>>>>>>>>> we can use initial dispersion to make sure traffic is spread between an
>>>>>>>>>> appropriate number of caches.
>>>>>>>>>>
>>>>>>>>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <
>>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Bringing up an old email regarding DS to cache assignments.
>>>>>>>>>>>
>>>>>>>>>>> It sounds like we want to preserve DS/cache assignments (the
>>>>>>>>>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
>>>>>>>>>>> to a DS if you want. However, what if we make the assignment of ALL cdn
>>>>>>>>>>> edge caches to a new DS the default?
>>>>>>>>>>>
>>>>>>>>>>> So when you create a DS for the Foo cdn thru the API, your new
>>>>>>>>>>> DS is automagically assigned all the edge caches in the Foo cdn...
>>>>>>>>>>>
>>>>>>>>>>> (at that point you could always remove caches from a DS)
>>>>>>>>>>>
>>>>>>>>>>> ...or the create DS API could have a boolean flag added to it.
>>>>>>>>>>> for example, assignAllCaches: true|false. if you pass in true, all caches
>>>>>>>>>>> are assigned to the DS, if false, none are.
>>>>>>>>>>>
>>>>>>>>>>> thoughts? I'm trying to think of self-service and i'm guessing
>>>>>>>>>>> most people will not want to assign caches to a DS...
>>>>>>>>>>>
>>>>>>>>>>> jeremy
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
>>>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> gotcha. makes sense.
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <
>>>>>>>>>>>> david.neuman64@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I believe Eric had a use case where he wanted to assign
>>>>>>>>>>>>> certain delivery services to certain caches.
>>>>>>>>>>>>> If we just use initial dispersion and max DNS answers then we
>>>>>>>>>>>>> don't have the ability to control which servers are used, just how many.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
>>>>>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> So what I heard was that potentially we move in this
>>>>>>>>>>>>>> direction:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1. No need to explicitly assign edge caches to a delivery
>>>>>>>>>>>>>> service. a delivery service (by default) would employ ALL edge caches of
>>>>>>>>>>>>>> the cdn in which the ds belongs to.
>>>>>>>>>>>>>> 2. Ability to override this default behavior and explicitly
>>>>>>>>>>>>>> assign specific edge caches to a delivery service (this is what we do today)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> But earlier Mark said "so the same functionality [as #2] can
>>>>>>>>>>>>>> essentially be reached with assigning all caches, and setting the config
>>>>>>>>>>>>>> params properly [initial dispersion and max dns answers]"
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> so why not ditch #2 in favor of using initial dispersion and
>>>>>>>>>>>>>> max dns answers to control cache assignments?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> it would be nice imo to get rid of the ds_server table. i'm
>>>>>>>>>>>>>> always in favor of reducing complexity... :)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> jeremy
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
>>>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yes - I think its a reasonable default to have a delivery
>>>>>>>>>>>>>>> service assigned to all caches as long as we still keep the ability to
>>>>>>>>>>>>>>> assign some DS’s to just a subset of caches.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <
>>>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Good feedback, thanks. I think having _zero_ caches assigned
>>>>>>>>>>>>>>> to a delivery service could be added fairly easily, and could function as a
>>>>>>>>>>>>>>> good default to the assumption of _all_ caches being assigned. Would you
>>>>>>>>>>>>>>> agree with that?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Mark
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
>>>>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yeah,
>>>>>>>>>>>>>>>> We have several some use cases where where cache or CG
>>>>>>>>>>>>>>>> level allocation would be helpful.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1) Assigning new or trial delivery services to a subset of
>>>>>>>>>>>>>>>> caches (considered beta cache nodes). This is helpful in trying out new DS
>>>>>>>>>>>>>>>> configs before rolling them out to production.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or
>>>>>>>>>>>>>>>> resource reservation on the delivery service. Providers of content to the
>>>>>>>>>>>>>>>> CDN can receive tiered service by either sharing caches or using dedicated
>>>>>>>>>>>>>>>> caches.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <
>>>>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Today, there isn't an actual link from Cache Group to DS --
>>>>>>>>>>>>>>>> I think what
>>>>>>>>>>>>>>>> you are referring to is all done in the app layer to make a
>>>>>>>>>>>>>>>> nice summary in
>>>>>>>>>>>>>>>> the UI to allow operators to very quickly assign all caches
>>>>>>>>>>>>>>>> in a cache
>>>>>>>>>>>>>>>> group.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Do you use that "feature" to assign caches from some cache
>>>>>>>>>>>>>>>> groups, and not
>>>>>>>>>>>>>>>> others?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
>>>>>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
>>>>>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>> > Seeking community opinion on this.
>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>> > Should we drop the server -> delivery service
>>>>>>>>>>>>>>>>> assignments completely?
>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>> > When the project began, selectively assigning caches was
>>>>>>>>>>>>>>>>> the only way to selectively route traffic within a cache group -- let's say
>>>>>>>>>>>>>>>>> you have a DNS-routed service with a relatively large library size, you
>>>>>>>>>>>>>>>>> might not want assign all caches in a cache group to that service for fear
>>>>>>>>>>>>>>>>> you would 'pollute' those caches. Or, if you were turning up a 'test' DS of
>>>>>>>>>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
>>>>>>>>>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
>>>>>>>>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
>>>>>>>>>>>>>>>>> services, so the same functionality can essentially be reached with
>>>>>>>>>>>>>>>>> assigning all caches, and setting the config params properly.
>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>> > The motivation for this is to reduce the size &
>>>>>>>>>>>>>>>>> complexity of the Traffic Router's config (aka CRConfig), as this config
>>>>>>>>>>>>>>>>> scales mostly linearly with the number of caches and the number of delivery
>>>>>>>>>>>>>>>>> services in your CDN.
>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>> > Another option might be to not assign caches as the
>>>>>>>>>>>>>>>>> default, and individual cache assignments is available, but used only when
>>>>>>>>>>>>>>>>> necessary.
>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>> > Other thoughts? Clarifications needed?
>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>> > Thanks,
>>>>>>>>>>>>>>>>> > Mark
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>
>>
>> --
>> *Shmulik Asafi*
>> Qwilt | Work: +972-72-2221692 <+972%2072-222-1692>| Mobile:
>> +972-54-6581595 <+972%2054-658-1595>| shmulika@qwilt.com <yo...@qwilt.com>
>>
>
>
Re: Drop Server -> Delivery Service assignments?
Posted by Dave Neuman <ne...@apache.org>.
This is all part of Step 1. This is the first time a Delivery service is
created so there are no other versions of it...
On Wed, May 24, 2017 at 2:33 PM, Shmulik Asafi <sh...@qwilt.com> wrote:
> Hi,
> Referencing back to Nir's slides in the summit on Self Service, there are
> three main phases to the flow relevant here:
>
> 1. The DS-owner user defines/modifies the DS (using configuration
> versions)
> 2. The DS-owner user declares which DS versions she wants to be
> deployed
> 3. The CDN-operator user/robot configures what DS versions to assign
> to which servers
>
> If we decide to automatically assign DS to all caches by default, in the
> Self Service flow it is part of phase 3.
>
> If we want to give the DS-owner an opt-out from automatic assignment to
> all caches; it should be a property of whatever tables are used to manage
> phase 2.
>
> In my view, an immediate implementation of this logic should not affect
> the DS table (which in Self Service becomes associated with "phase 1");
> only the assignment table (which in Self Service becomes associated with
> "phase 3").
>
> As long as the boundaries are mentally clear and the code is written in
> such a way that respects these future boundaries, I dont believe there's
> much risk in features like this to Self Service.
> Hope that made sense
> Nir - would you agree?
>
> On Wed, May 24, 2017 at 10:56 PM, Dave Neuman <ne...@apache.org> wrote:
>
>> In the system today when you create a server you have the ability to
>> clone the delivery service assignments from another cache or manually
>> update the delivery service -> server mapping by going into individual DSs
>> and assigning the cache to them. I don't think that needs to change with
>> what Jeremy is proposing.
>> I also don't think we should put a special value in the
>> deliveryservice_server table that means "all caches". If you want to assign
>> all caches to a DS put all of them in the table. If you add a new cache
>> and want your DSs on there, clone the mappings from an existing cache.
>>
>>
>> On Wed, May 24, 2017 at 1:43 PM, Nir Sopher <ni...@qwilt.com> wrote:
>>
>>> Hi Jeremy,
>>>
>>> I’m ok with automatically assigning all edges to a new DS. The aspect of
>>> "self-service" can be dealt with in a different manner.
>>> However, how would you solve the problem when a new server is created?
>>> How would you know which DS it should be automatically be assigned?
>>> The model has no configuration marking the DS as one that should be
>>> deployed on all servers.
>>> My suggestion was to model it with "undef" value in the DS/Servers table.
>>> DS | Server
>>> ------+--------
>>> ds1 | server1 => Allow the CDN owner to deploy the ds on a specific
>>> cache
>>> ------+--------
>>> ds2 | *undef *=> Allow the CDN owner to deploy the ds on all caches
>>>
>>> This basically means that every place reading the DS/Server table, will
>>> have to use a wrapper, unfolding a the "server=undef" raw, to a raw per
>>> relevant server in the CDN.
>>> I'm not sure how much effort it requires.
>>>
>>> Nir
>>>
>>> On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <mi...@gmail.com>
>>> wrote:
>>>
>>>> Nir,
>>>>
>>>> Thinking about self-service, do you envision the "DS owner" creates the
>>>> DS and the "CDN owner" assigns caches to the DS? This seems like an
>>>> unnecessary step to me and one that negates the idea of "self service".
>>>>
>>>> As the "DS owner", I want to create a DS and by default, employ ALL
>>>> caches in my cdn. To be honest, in 99% of cases I really don't want to be
>>>> bothered with assigning caches. I want the full power of the CDN behind my
>>>> DS.
>>>>
>>>> So, I'm suggesting that when a new DS is created thru the API, ALL edge
>>>> caches in the cdn are assigned to that DS.....and then, the CDN owner (or
>>>> maybe DS owner) can remove caches from that DS if they need to.
>>>>
>>>> Jeremy
>>>>
>>>> On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com> wrote:
>>>>
>>>>> @Jeremy
>>>>>
>>>>> The issue I was trying to point at, is that the "DS-Server" table is
>>>>> the one to provide the separation between the DS owner domain and the CDN
>>>>> owner domain (tenancy wise).
>>>>> As I see it, when considering multi tenancy, the DS has its owning
>>>>> tenant which is in charge of the specific record in the DS table, and
>>>>> specifically on adding the DS. The CDN is usually owned by another tenant
>>>>> that controls the deployment - currently by adding the ds/servers record to
>>>>> the DS/Server table.
>>>>>
>>>>> If the DS/Server table is dismissed or automatically written as a
>>>>> result of a ds addition, it removes the "server assignment" step and the
>>>>> CDN owner loses the control over the deployment decision.
>>>>>
>>>>> Before I open a discussion about how to solve this new tenancy issue,
>>>>> I would like to verify it is a real issue and there is not something I'm
>>>>> missing there.
>>>>> What do you think?
>>>>>
>>>>> Nir
>>>>>
>>>>> On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> @Nir - maybe your proposal is something to consider more long-term
>>>>>> but for the short-term i'm wondering if one of the following can / should
>>>>>> be implemented:
>>>>>>
>>>>>> A) creating a DS thru the API (POST /api/*/deliveryservices) can have
>>>>>> a default effect of automatically assigning all edges to the new DS (via
>>>>>> the deliveryservice_server table)
>>>>>>
>>>>>> or
>>>>>>
>>>>>> B) the deliveryservice_server table simple goes away and it becomes
>>>>>> implied that a DS employs all edge caches
>>>>>>
>>>>>> "A" would probably be the easiest to be honest.
>>>>>>
>>>>>> jeremy
>>>>>>
>>>>>> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com> wrote:
>>>>>>
>>>>>>> +1 for having a configuration allowing to apply the DS to the entire
>>>>>>> CDN.
>>>>>>>
>>>>>>> Note that adding a "deploy by default" bool field to the DS itself
>>>>>>> may cause tenancy issues, as I believe this field should be in the control
>>>>>>> of the "CDN owner" tenant, and not of the "DS owner".
>>>>>>>
>>>>>>> What about keeping the ds_server table, and allow "undef" for the
>>>>>>> server value - meaning "all servers in the CDN"?
>>>>>>> In the future the table may grow and have additional fields to
>>>>>>> white-list servers by.
>>>>>>> For example:
>>>>>>>
>>>>>>> DS | Server | cache-group | profile
>>>>>>> ------+--------+-------------+--------
>>>>>>> ds1 | s1 | null | null => ds to be applied on s1
>>>>>>> ------+--------+-------------+--------
>>>>>>> ds2 | null | null | null => ds to be applied on
>>>>>>> all servers (in ds CDN)
>>>>>>> ------+--------+-------------+--------
>>>>>>> ds3 | null | group1 | null => ds to be applied on
>>>>>>> all servers in cache group "group1"
>>>>>>> ------+--------+-------------+--------
>>>>>>> ds4 | null | group1 | ATS-6.2 => ds to be applied on
>>>>>>> all servers in cache group "group1", with profile "ATS-6.2"
>>>>>>>
>>>>>>>
>>>>>>> During CR config snapshot and configuration pull (or in advance),
>>>>>>> the required calculation would be done, to provide the list of relevant DSs
>>>>>>> per server.
>>>>>>>
>>>>>>> Also note that moving in this direction is backward compatible as
>>>>>>> well as supports other non trivial extensions one may want in the future:
>>>>>>> Patterns (wildcards/regexs) support, as well as black list, using a "policy
>>>>>>> table" mechanism.
>>>>>>>
>>>>>>> In the below example "ds5" is to be deployed on all caches, except
>>>>>>> for caches with "ATS-5" profile
>>>>>>>
>>>>>>> DS | Server | cache-group | profile | deploy-decision
>>>>>>> ------+--------+-------------+---------+----------------
>>>>>>> ds5 | null | null | ATS-5 | false
>>>>>>> ------+--------+-------------+---------+----------------
>>>>>>> ds5 | null | null | null | true
>>>>>>> ------+--------+-------------+---------+----------------
>>>>>>>
>>>>>>>
>>>>>>> Nir
>>>>>>>
>>>>>>> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <
>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>
>>>>>>>> @dave - i'm not sure if you're suggesting:
>>>>>>>>
>>>>>>>> A) on DS create, by default create an entry in the
>>>>>>>> deliveryservice_server table for each cdn edge cache. so if there are 200
>>>>>>>> cdn edge caches, you end up with 200 entries in the deliveryservice_server
>>>>>>>> table for that DS.
>>>>>>>> B) completely kill the deliveryservice_server table and it becomes
>>>>>>>> implied that a DS uses ALL cdn edge caches (the way mids work
>>>>>>>> today. mids don't have to be explicitly assigned to a DS)
>>>>>>>>
>>>>>>>> "A" would allow you to control the scope of caches by using initial
>>>>>>>> dispersion OR control the scope of caches by simply removing the
>>>>>>>> ones you don't want to receive traffic.
>>>>>>>> "B" would only allow you to control the scope of caches by using
>>>>>>>> initial dispersion (or so i think).
>>>>>>>>
>>>>>>>> jeremy
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> +1, I think assigning DSs to all caches by default is fine. Then
>>>>>>>>> we can use initial dispersion to make sure traffic is spread between an
>>>>>>>>> appropriate number of caches.
>>>>>>>>>
>>>>>>>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <
>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Bringing up an old email regarding DS to cache assignments.
>>>>>>>>>>
>>>>>>>>>> It sounds like we want to preserve DS/cache assignments (the
>>>>>>>>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
>>>>>>>>>> to a DS if you want. However, what if we make the assignment of ALL cdn
>>>>>>>>>> edge caches to a new DS the default?
>>>>>>>>>>
>>>>>>>>>> So when you create a DS for the Foo cdn thru the API, your new DS
>>>>>>>>>> is automagically assigned all the edge caches in the Foo cdn...
>>>>>>>>>>
>>>>>>>>>> (at that point you could always remove caches from a DS)
>>>>>>>>>>
>>>>>>>>>> ...or the create DS API could have a boolean flag added to it.
>>>>>>>>>> for example, assignAllCaches: true|false. if you pass in true, all caches
>>>>>>>>>> are assigned to the DS, if false, none are.
>>>>>>>>>>
>>>>>>>>>> thoughts? I'm trying to think of self-service and i'm guessing
>>>>>>>>>> most people will not want to assign caches to a DS...
>>>>>>>>>>
>>>>>>>>>> jeremy
>>>>>>>>>>
>>>>>>>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
>>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> gotcha. makes sense.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <
>>>>>>>>>>> david.neuman64@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I believe Eric had a use case where he wanted to assign certain
>>>>>>>>>>>> delivery services to certain caches.
>>>>>>>>>>>> If we just use initial dispersion and max DNS answers then we
>>>>>>>>>>>> don't have the ability to control which servers are used, just how many.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
>>>>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> So what I heard was that potentially we move in this direction:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. No need to explicitly assign edge caches to a delivery
>>>>>>>>>>>>> service. a delivery service (by default) would employ ALL edge caches of
>>>>>>>>>>>>> the cdn in which the ds belongs to.
>>>>>>>>>>>>> 2. Ability to override this default behavior and explicitly
>>>>>>>>>>>>> assign specific edge caches to a delivery service (this is what we do today)
>>>>>>>>>>>>>
>>>>>>>>>>>>> But earlier Mark said "so the same functionality [as #2] can
>>>>>>>>>>>>> essentially be reached with assigning all caches, and setting the config
>>>>>>>>>>>>> params properly [initial dispersion and max dns answers]"
>>>>>>>>>>>>>
>>>>>>>>>>>>> so why not ditch #2 in favor of using initial dispersion and
>>>>>>>>>>>>> max dns answers to control cache assignments?
>>>>>>>>>>>>>
>>>>>>>>>>>>> it would be nice imo to get rid of the ds_server table. i'm
>>>>>>>>>>>>> always in favor of reducing complexity... :)
>>>>>>>>>>>>>
>>>>>>>>>>>>> jeremy
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
>>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yes - I think its a reasonable default to have a delivery
>>>>>>>>>>>>>> service assigned to all caches as long as we still keep the ability to
>>>>>>>>>>>>>> assign some DS’s to just a subset of caches.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <
>>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Good feedback, thanks. I think having _zero_ caches assigned
>>>>>>>>>>>>>> to a delivery service could be added fairly easily, and could function as a
>>>>>>>>>>>>>> good default to the assumption of _all_ caches being assigned. Would you
>>>>>>>>>>>>>> agree with that?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Mark
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
>>>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yeah,
>>>>>>>>>>>>>>> We have several some use cases where where cache or CG
>>>>>>>>>>>>>>> level allocation would be helpful.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1) Assigning new or trial delivery services to a subset of
>>>>>>>>>>>>>>> caches (considered beta cache nodes). This is helpful in trying out new DS
>>>>>>>>>>>>>>> configs before rolling them out to production.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or
>>>>>>>>>>>>>>> resource reservation on the delivery service. Providers of content to the
>>>>>>>>>>>>>>> CDN can receive tiered service by either sharing caches or using dedicated
>>>>>>>>>>>>>>> caches.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <
>>>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Today, there isn't an actual link from Cache Group to DS --
>>>>>>>>>>>>>>> I think what
>>>>>>>>>>>>>>> you are referring to is all done in the app layer to make a
>>>>>>>>>>>>>>> nice summary in
>>>>>>>>>>>>>>> the UI to allow operators to very quickly assign all caches
>>>>>>>>>>>>>>> in a cache
>>>>>>>>>>>>>>> group.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Do you use that "feature" to assign caches from some cache
>>>>>>>>>>>>>>> groups, and not
>>>>>>>>>>>>>>> others?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
>>>>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
>>>>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > Seeking community opinion on this.
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > Should we drop the server -> delivery service assignments
>>>>>>>>>>>>>>>> completely?
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > When the project began, selectively assigning caches was
>>>>>>>>>>>>>>>> the only way to selectively route traffic within a cache group -- let's say
>>>>>>>>>>>>>>>> you have a DNS-routed service with a relatively large library size, you
>>>>>>>>>>>>>>>> might not want assign all caches in a cache group to that service for fear
>>>>>>>>>>>>>>>> you would 'pollute' those caches. Or, if you were turning up a 'test' DS of
>>>>>>>>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
>>>>>>>>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
>>>>>>>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
>>>>>>>>>>>>>>>> services, so the same functionality can essentially be reached with
>>>>>>>>>>>>>>>> assigning all caches, and setting the config params properly.
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > The motivation for this is to reduce the size &
>>>>>>>>>>>>>>>> complexity of the Traffic Router's config (aka CRConfig), as this config
>>>>>>>>>>>>>>>> scales mostly linearly with the number of caches and the number of delivery
>>>>>>>>>>>>>>>> services in your CDN.
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > Another option might be to not assign caches as the
>>>>>>>>>>>>>>>> default, and individual cache assignments is available, but used only when
>>>>>>>>>>>>>>>> necessary.
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > Other thoughts? Clarifications needed?
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > Thanks,
>>>>>>>>>>>>>>>> > Mark
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>
>
>
> --
> *Shmulik Asafi*
> Qwilt | Work: +972-72-2221692 <+972%2072-222-1692>| Mobile:
> +972-54-6581595 <+972%2054-658-1595>| shmulika@qwilt.com <yo...@qwilt.com>
>
Re: Drop Server -> Delivery Service assignments?
Posted by Shmulik Asafi <sh...@qwilt.com>.
Hi,
Referencing back to Nir's slides in the summit on Self Service, there are
three main phases to the flow relevant here:
1. The DS-owner user defines/modifies the DS (using configuration
versions)
2. The DS-owner user declares which DS versions she wants to be deployed
3. The CDN-operator user/robot configures what DS versions to assign to
which servers
If we decide to automatically assign DS to all caches by default, in the
Self Service flow it is part of phase 3.
If we want to give the DS-owner an opt-out from automatic assignment to all
caches; it should be a property of whatever tables are used to manage phase
2.
In my view, an immediate implementation of this logic should not affect the
DS table (which in Self Service becomes associated with "phase 1"); only
the assignment table (which in Self Service becomes associated with "phase
3").
As long as the boundaries are mentally clear and the code is written in
such a way that respects these future boundaries, I dont believe there's
much risk in features like this to Self Service.
Hope that made sense
Nir - would you agree?
On Wed, May 24, 2017 at 10:56 PM, Dave Neuman <ne...@apache.org> wrote:
> In the system today when you create a server you have the ability to clone
> the delivery service assignments from another cache or manually update the
> delivery service -> server mapping by going into individual DSs and
> assigning the cache to them. I don't think that needs to change with what
> Jeremy is proposing.
> I also don't think we should put a special value in the
> deliveryservice_server table that means "all caches". If you want to assign
> all caches to a DS put all of them in the table. If you add a new cache
> and want your DSs on there, clone the mappings from an existing cache.
>
>
> On Wed, May 24, 2017 at 1:43 PM, Nir Sopher <ni...@qwilt.com> wrote:
>
>> Hi Jeremy,
>>
>> I’m ok with automatically assigning all edges to a new DS. The aspect of
>> "self-service" can be dealt with in a different manner.
>> However, how would you solve the problem when a new server is created?
>> How would you know which DS it should be automatically be assigned?
>> The model has no configuration marking the DS as one that should be
>> deployed on all servers.
>> My suggestion was to model it with "undef" value in the DS/Servers table.
>> DS | Server
>> ------+--------
>> ds1 | server1 => Allow the CDN owner to deploy the ds on a specific
>> cache
>> ------+--------
>> ds2 | *undef *=> Allow the CDN owner to deploy the ds on all caches
>>
>> This basically means that every place reading the DS/Server table, will
>> have to use a wrapper, unfolding a the "server=undef" raw, to a raw per
>> relevant server in the CDN.
>> I'm not sure how much effort it requires.
>>
>> Nir
>>
>> On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <mi...@gmail.com>
>> wrote:
>>
>>> Nir,
>>>
>>> Thinking about self-service, do you envision the "DS owner" creates the
>>> DS and the "CDN owner" assigns caches to the DS? This seems like an
>>> unnecessary step to me and one that negates the idea of "self service".
>>>
>>> As the "DS owner", I want to create a DS and by default, employ ALL
>>> caches in my cdn. To be honest, in 99% of cases I really don't want to be
>>> bothered with assigning caches. I want the full power of the CDN behind my
>>> DS.
>>>
>>> So, I'm suggesting that when a new DS is created thru the API, ALL edge
>>> caches in the cdn are assigned to that DS.....and then, the CDN owner (or
>>> maybe DS owner) can remove caches from that DS if they need to.
>>>
>>> Jeremy
>>>
>>> On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com> wrote:
>>>
>>>> @Jeremy
>>>>
>>>> The issue I was trying to point at, is that the "DS-Server" table is
>>>> the one to provide the separation between the DS owner domain and the CDN
>>>> owner domain (tenancy wise).
>>>> As I see it, when considering multi tenancy, the DS has its owning
>>>> tenant which is in charge of the specific record in the DS table, and
>>>> specifically on adding the DS. The CDN is usually owned by another tenant
>>>> that controls the deployment - currently by adding the ds/servers record to
>>>> the DS/Server table.
>>>>
>>>> If the DS/Server table is dismissed or automatically written as a
>>>> result of a ds addition, it removes the "server assignment" step and the
>>>> CDN owner loses the control over the deployment decision.
>>>>
>>>> Before I open a discussion about how to solve this new tenancy issue, I
>>>> would like to verify it is a real issue and there is not something I'm
>>>> missing there.
>>>> What do you think?
>>>>
>>>> Nir
>>>>
>>>> On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com>
>>>> wrote:
>>>>
>>>>> @Nir - maybe your proposal is something to consider more long-term but
>>>>> for the short-term i'm wondering if one of the following can / should be
>>>>> implemented:
>>>>>
>>>>> A) creating a DS thru the API (POST /api/*/deliveryservices) can have
>>>>> a default effect of automatically assigning all edges to the new DS (via
>>>>> the deliveryservice_server table)
>>>>>
>>>>> or
>>>>>
>>>>> B) the deliveryservice_server table simple goes away and it becomes
>>>>> implied that a DS employs all edge caches
>>>>>
>>>>> "A" would probably be the easiest to be honest.
>>>>>
>>>>> jeremy
>>>>>
>>>>> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com> wrote:
>>>>>
>>>>>> +1 for having a configuration allowing to apply the DS to the entire
>>>>>> CDN.
>>>>>>
>>>>>> Note that adding a "deploy by default" bool field to the DS itself
>>>>>> may cause tenancy issues, as I believe this field should be in the control
>>>>>> of the "CDN owner" tenant, and not of the "DS owner".
>>>>>>
>>>>>> What about keeping the ds_server table, and allow "undef" for the
>>>>>> server value - meaning "all servers in the CDN"?
>>>>>> In the future the table may grow and have additional fields to
>>>>>> white-list servers by.
>>>>>> For example:
>>>>>>
>>>>>> DS | Server | cache-group | profile
>>>>>> ------+--------+-------------+--------
>>>>>> ds1 | s1 | null | null => ds to be applied on s1
>>>>>> ------+--------+-------------+--------
>>>>>> ds2 | null | null | null => ds to be applied on
>>>>>> all servers (in ds CDN)
>>>>>> ------+--------+-------------+--------
>>>>>> ds3 | null | group1 | null => ds to be applied on
>>>>>> all servers in cache group "group1"
>>>>>> ------+--------+-------------+--------
>>>>>> ds4 | null | group1 | ATS-6.2 => ds to be applied on
>>>>>> all servers in cache group "group1", with profile "ATS-6.2"
>>>>>>
>>>>>>
>>>>>> During CR config snapshot and configuration pull (or in advance), the
>>>>>> required calculation would be done, to provide the list of relevant DSs per
>>>>>> server.
>>>>>>
>>>>>> Also note that moving in this direction is backward compatible as
>>>>>> well as supports other non trivial extensions one may want in the future:
>>>>>> Patterns (wildcards/regexs) support, as well as black list, using a "policy
>>>>>> table" mechanism.
>>>>>>
>>>>>> In the below example "ds5" is to be deployed on all caches, except
>>>>>> for caches with "ATS-5" profile
>>>>>>
>>>>>> DS | Server | cache-group | profile | deploy-decision
>>>>>> ------+--------+-------------+---------+----------------
>>>>>> ds5 | null | null | ATS-5 | false
>>>>>> ------+--------+-------------+---------+----------------
>>>>>> ds5 | null | null | null | true
>>>>>> ------+--------+-------------+---------+----------------
>>>>>>
>>>>>>
>>>>>> Nir
>>>>>>
>>>>>> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <
>>>>>> mitchell852@gmail.com> wrote:
>>>>>>
>>>>>>> @dave - i'm not sure if you're suggesting:
>>>>>>>
>>>>>>> A) on DS create, by default create an entry in the
>>>>>>> deliveryservice_server table for each cdn edge cache. so if there are 200
>>>>>>> cdn edge caches, you end up with 200 entries in the deliveryservice_server
>>>>>>> table for that DS.
>>>>>>> B) completely kill the deliveryservice_server table and it becomes
>>>>>>> implied that a DS uses ALL cdn edge caches (the way mids work
>>>>>>> today. mids don't have to be explicitly assigned to a DS)
>>>>>>>
>>>>>>> "A" would allow you to control the scope of caches by using initial
>>>>>>> dispersion OR control the scope of caches by simply removing the
>>>>>>> ones you don't want to receive traffic.
>>>>>>> "B" would only allow you to control the scope of caches by using
>>>>>>> initial dispersion (or so i think).
>>>>>>>
>>>>>>> jeremy
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +1, I think assigning DSs to all caches by default is fine. Then
>>>>>>>> we can use initial dispersion to make sure traffic is spread between an
>>>>>>>> appropriate number of caches.
>>>>>>>>
>>>>>>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <
>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Bringing up an old email regarding DS to cache assignments.
>>>>>>>>>
>>>>>>>>> It sounds like we want to preserve DS/cache assignments (the
>>>>>>>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
>>>>>>>>> to a DS if you want. However, what if we make the assignment of ALL cdn
>>>>>>>>> edge caches to a new DS the default?
>>>>>>>>>
>>>>>>>>> So when you create a DS for the Foo cdn thru the API, your new DS
>>>>>>>>> is automagically assigned all the edge caches in the Foo cdn...
>>>>>>>>>
>>>>>>>>> (at that point you could always remove caches from a DS)
>>>>>>>>>
>>>>>>>>> ...or the create DS API could have a boolean flag added to it. for
>>>>>>>>> example, assignAllCaches: true|false. if you pass in true, all caches are
>>>>>>>>> assigned to the DS, if false, none are.
>>>>>>>>>
>>>>>>>>> thoughts? I'm trying to think of self-service and i'm guessing
>>>>>>>>> most people will not want to assign caches to a DS...
>>>>>>>>>
>>>>>>>>> jeremy
>>>>>>>>>
>>>>>>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> gotcha. makes sense.
>>>>>>>>>>
>>>>>>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <
>>>>>>>>>> david.neuman64@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I believe Eric had a use case where he wanted to assign certain
>>>>>>>>>>> delivery services to certain caches.
>>>>>>>>>>> If we just use initial dispersion and max DNS answers then we
>>>>>>>>>>> don't have the ability to control which servers are used, just how many.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
>>>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> So what I heard was that potentially we move in this direction:
>>>>>>>>>>>>
>>>>>>>>>>>> 1. No need to explicitly assign edge caches to a delivery
>>>>>>>>>>>> service. a delivery service (by default) would employ ALL edge caches of
>>>>>>>>>>>> the cdn in which the ds belongs to.
>>>>>>>>>>>> 2. Ability to override this default behavior and explicitly
>>>>>>>>>>>> assign specific edge caches to a delivery service (this is what we do today)
>>>>>>>>>>>>
>>>>>>>>>>>> But earlier Mark said "so the same functionality [as #2] can
>>>>>>>>>>>> essentially be reached with assigning all caches, and setting the config
>>>>>>>>>>>> params properly [initial dispersion and max dns answers]"
>>>>>>>>>>>>
>>>>>>>>>>>> so why not ditch #2 in favor of using initial dispersion and
>>>>>>>>>>>> max dns answers to control cache assignments?
>>>>>>>>>>>>
>>>>>>>>>>>> it would be nice imo to get rid of the ds_server table. i'm
>>>>>>>>>>>> always in favor of reducing complexity... :)
>>>>>>>>>>>>
>>>>>>>>>>>> jeremy
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Yes - I think its a reasonable default to have a delivery
>>>>>>>>>>>>> service assigned to all caches as long as we still keep the ability to
>>>>>>>>>>>>> assign some DS’s to just a subset of caches.
>>>>>>>>>>>>>
>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <
>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Good feedback, thanks. I think having _zero_ caches assigned
>>>>>>>>>>>>> to a delivery service could be added fairly easily, and could function as a
>>>>>>>>>>>>> good default to the assumption of _all_ caches being assigned. Would you
>>>>>>>>>>>>> agree with that?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Mark
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
>>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yeah,
>>>>>>>>>>>>>> We have several some use cases where where cache or CG
>>>>>>>>>>>>>> level allocation would be helpful.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1) Assigning new or trial delivery services to a subset of
>>>>>>>>>>>>>> caches (considered beta cache nodes). This is helpful in trying out new DS
>>>>>>>>>>>>>> configs before rolling them out to production.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or resource
>>>>>>>>>>>>>> reservation on the delivery service. Providers of content to the CDN can
>>>>>>>>>>>>>> receive tiered service by either sharing caches or using dedicated caches.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <
>>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Today, there isn't an actual link from Cache Group to DS -- I
>>>>>>>>>>>>>> think what
>>>>>>>>>>>>>> you are referring to is all done in the app layer to make a
>>>>>>>>>>>>>> nice summary in
>>>>>>>>>>>>>> the UI to allow operators to very quickly assign all caches
>>>>>>>>>>>>>> in a cache
>>>>>>>>>>>>>> group.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Do you use that "feature" to assign caches from some cache
>>>>>>>>>>>>>> groups, and not
>>>>>>>>>>>>>> others?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
>>>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
>>>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> > Seeking community opinion on this.
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> > Should we drop the server -> delivery service assignments
>>>>>>>>>>>>>>> completely?
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> > When the project began, selectively assigning caches was
>>>>>>>>>>>>>>> the only way to selectively route traffic within a cache group -- let's say
>>>>>>>>>>>>>>> you have a DNS-routed service with a relatively large library size, you
>>>>>>>>>>>>>>> might not want assign all caches in a cache group to that service for fear
>>>>>>>>>>>>>>> you would 'pollute' those caches. Or, if you were turning up a 'test' DS of
>>>>>>>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
>>>>>>>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
>>>>>>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
>>>>>>>>>>>>>>> services, so the same functionality can essentially be reached with
>>>>>>>>>>>>>>> assigning all caches, and setting the config params properly.
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> > The motivation for this is to reduce the size & complexity
>>>>>>>>>>>>>>> of the Traffic Router's config (aka CRConfig), as this config scales mostly
>>>>>>>>>>>>>>> linearly with the number of caches and the number of delivery services in
>>>>>>>>>>>>>>> your CDN.
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> > Another option might be to not assign caches as the
>>>>>>>>>>>>>>> default, and individual cache assignments is available, but used only when
>>>>>>>>>>>>>>> necessary.
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> > Other thoughts? Clarifications needed?
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> > Thanks,
>>>>>>>>>>>>>>> > Mark
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>>
>
--
*Shmulik Asafi*
Qwilt | Work: +972-72-2221692| Mobile: +972-54-6581595| shmulika@qwilt.com
<yo...@qwilt.com>
Re: Drop Server -> Delivery Service assignments?
Posted by Dave Neuman <ne...@apache.org>.
In the system today when you create a server you have the ability to clone
the delivery service assignments from another cache or manually update the
delivery service -> server mapping by going into individual DSs and
assigning the cache to them. I don't think that needs to change with what
Jeremy is proposing.
I also don't think we should put a special value in the
deliveryservice_server table that means "all caches". If you want to assign
all caches to a DS put all of them in the table. If you add a new cache
and want your DSs on there, clone the mappings from an existing cache.
On Wed, May 24, 2017 at 1:43 PM, Nir Sopher <ni...@qwilt.com> wrote:
> Hi Jeremy,
>
> I’m ok with automatically assigning all edges to a new DS. The aspect of
> "self-service" can be dealt with in a different manner.
> However, how would you solve the problem when a new server is created? How
> would you know which DS it should be automatically be assigned?
> The model has no configuration marking the DS as one that should be
> deployed on all servers.
> My suggestion was to model it with "undef" value in the DS/Servers table.
> DS | Server
> ------+--------
> ds1 | server1 => Allow the CDN owner to deploy the ds on a specific
> cache
> ------+--------
> ds2 | *undef *=> Allow the CDN owner to deploy the ds on all caches
>
> This basically means that every place reading the DS/Server table, will
> have to use a wrapper, unfolding a the "server=undef" raw, to a raw per
> relevant server in the CDN.
> I'm not sure how much effort it requires.
>
> Nir
>
> On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
>> Nir,
>>
>> Thinking about self-service, do you envision the "DS owner" creates the
>> DS and the "CDN owner" assigns caches to the DS? This seems like an
>> unnecessary step to me and one that negates the idea of "self service".
>>
>> As the "DS owner", I want to create a DS and by default, employ ALL
>> caches in my cdn. To be honest, in 99% of cases I really don't want to be
>> bothered with assigning caches. I want the full power of the CDN behind my
>> DS.
>>
>> So, I'm suggesting that when a new DS is created thru the API, ALL edge
>> caches in the cdn are assigned to that DS.....and then, the CDN owner (or
>> maybe DS owner) can remove caches from that DS if they need to.
>>
>> Jeremy
>>
>> On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com> wrote:
>>
>>> @Jeremy
>>>
>>> The issue I was trying to point at, is that the "DS-Server" table is the
>>> one to provide the separation between the DS owner domain and the CDN owner
>>> domain (tenancy wise).
>>> As I see it, when considering multi tenancy, the DS has its owning
>>> tenant which is in charge of the specific record in the DS table, and
>>> specifically on adding the DS. The CDN is usually owned by another tenant
>>> that controls the deployment - currently by adding the ds/servers record to
>>> the DS/Server table.
>>>
>>> If the DS/Server table is dismissed or automatically written as a result
>>> of a ds addition, it removes the "server assignment" step and the CDN owner
>>> loses the control over the deployment decision.
>>>
>>> Before I open a discussion about how to solve this new tenancy issue, I
>>> would like to verify it is a real issue and there is not something I'm
>>> missing there.
>>> What do you think?
>>>
>>> Nir
>>>
>>> On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com>
>>> wrote:
>>>
>>>> @Nir - maybe your proposal is something to consider more long-term but
>>>> for the short-term i'm wondering if one of the following can / should be
>>>> implemented:
>>>>
>>>> A) creating a DS thru the API (POST /api/*/deliveryservices) can have a
>>>> default effect of automatically assigning all edges to the new DS (via the deliveryservice_server
>>>> table)
>>>>
>>>> or
>>>>
>>>> B) the deliveryservice_server table simple goes away and it becomes
>>>> implied that a DS employs all edge caches
>>>>
>>>> "A" would probably be the easiest to be honest.
>>>>
>>>> jeremy
>>>>
>>>> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com> wrote:
>>>>
>>>>> +1 for having a configuration allowing to apply the DS to the entire
>>>>> CDN.
>>>>>
>>>>> Note that adding a "deploy by default" bool field to the DS itself may
>>>>> cause tenancy issues, as I believe this field should be in the control of
>>>>> the "CDN owner" tenant, and not of the "DS owner".
>>>>>
>>>>> What about keeping the ds_server table, and allow "undef" for the
>>>>> server value - meaning "all servers in the CDN"?
>>>>> In the future the table may grow and have additional fields to
>>>>> white-list servers by.
>>>>> For example:
>>>>>
>>>>> DS | Server | cache-group | profile
>>>>> ------+--------+-------------+--------
>>>>> ds1 | s1 | null | null => ds to be applied on s1
>>>>> ------+--------+-------------+--------
>>>>> ds2 | null | null | null => ds to be applied on all
>>>>> servers (in ds CDN)
>>>>> ------+--------+-------------+--------
>>>>> ds3 | null | group1 | null => ds to be applied on all
>>>>> servers in cache group "group1"
>>>>> ------+--------+-------------+--------
>>>>> ds4 | null | group1 | ATS-6.2 => ds to be applied on all
>>>>> servers in cache group "group1", with profile "ATS-6.2"
>>>>>
>>>>>
>>>>> During CR config snapshot and configuration pull (or in advance), the
>>>>> required calculation would be done, to provide the list of relevant DSs per
>>>>> server.
>>>>>
>>>>> Also note that moving in this direction is backward compatible as well
>>>>> as supports other non trivial extensions one may want in the future:
>>>>> Patterns (wildcards/regexs) support, as well as black list, using a "policy
>>>>> table" mechanism.
>>>>>
>>>>> In the below example "ds5" is to be deployed on all caches, except for
>>>>> caches with "ATS-5" profile
>>>>>
>>>>> DS | Server | cache-group | profile | deploy-decision
>>>>> ------+--------+-------------+---------+----------------
>>>>> ds5 | null | null | ATS-5 | false
>>>>> ------+--------+-------------+---------+----------------
>>>>> ds5 | null | null | null | true
>>>>> ------+--------+-------------+---------+----------------
>>>>>
>>>>>
>>>>> Nir
>>>>>
>>>>> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <
>>>>> mitchell852@gmail.com> wrote:
>>>>>
>>>>>> @dave - i'm not sure if you're suggesting:
>>>>>>
>>>>>> A) on DS create, by default create an entry in the
>>>>>> deliveryservice_server table for each cdn edge cache. so if there are 200
>>>>>> cdn edge caches, you end up with 200 entries in the deliveryservice_server
>>>>>> table for that DS.
>>>>>> B) completely kill the deliveryservice_server table and it becomes
>>>>>> implied that a DS uses ALL cdn edge caches (the way mids work today.
>>>>>> mids don't have to be explicitly assigned to a DS)
>>>>>>
>>>>>> "A" would allow you to control the scope of caches by using initial
>>>>>> dispersion OR control the scope of caches by simply removing the
>>>>>> ones you don't want to receive traffic.
>>>>>> "B" would only allow you to control the scope of caches by using
>>>>>> initial dispersion (or so i think).
>>>>>>
>>>>>> jeremy
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> +1, I think assigning DSs to all caches by default is fine. Then we
>>>>>>> can use initial dispersion to make sure traffic is spread between an
>>>>>>> appropriate number of caches.
>>>>>>>
>>>>>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <
>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>
>>>>>>>> Bringing up an old email regarding DS to cache assignments.
>>>>>>>>
>>>>>>>> It sounds like we want to preserve DS/cache assignments (the
>>>>>>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
>>>>>>>> to a DS if you want. However, what if we make the assignment of ALL cdn
>>>>>>>> edge caches to a new DS the default?
>>>>>>>>
>>>>>>>> So when you create a DS for the Foo cdn thru the API, your new DS
>>>>>>>> is automagically assigned all the edge caches in the Foo cdn...
>>>>>>>>
>>>>>>>> (at that point you could always remove caches from a DS)
>>>>>>>>
>>>>>>>> ...or the create DS API could have a boolean flag added to it. for
>>>>>>>> example, assignAllCaches: true|false. if you pass in true, all caches are
>>>>>>>> assigned to the DS, if false, none are.
>>>>>>>>
>>>>>>>> thoughts? I'm trying to think of self-service and i'm guessing most
>>>>>>>> people will not want to assign caches to a DS...
>>>>>>>>
>>>>>>>> jeremy
>>>>>>>>
>>>>>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> gotcha. makes sense.
>>>>>>>>>
>>>>>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <
>>>>>>>>> david.neuman64@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I believe Eric had a use case where he wanted to assign certain
>>>>>>>>>> delivery services to certain caches.
>>>>>>>>>> If we just use initial dispersion and max DNS answers then we
>>>>>>>>>> don't have the ability to control which servers are used, just how many.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
>>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> So what I heard was that potentially we move in this direction:
>>>>>>>>>>>
>>>>>>>>>>> 1. No need to explicitly assign edge caches to a delivery
>>>>>>>>>>> service. a delivery service (by default) would employ ALL edge caches of
>>>>>>>>>>> the cdn in which the ds belongs to.
>>>>>>>>>>> 2. Ability to override this default behavior and explicitly
>>>>>>>>>>> assign specific edge caches to a delivery service (this is what we do today)
>>>>>>>>>>>
>>>>>>>>>>> But earlier Mark said "so the same functionality [as #2] can
>>>>>>>>>>> essentially be reached with assigning all caches, and setting the config
>>>>>>>>>>> params properly [initial dispersion and max dns answers]"
>>>>>>>>>>>
>>>>>>>>>>> so why not ditch #2 in favor of using initial dispersion and
>>>>>>>>>>> max dns answers to control cache assignments?
>>>>>>>>>>>
>>>>>>>>>>> it would be nice imo to get rid of the ds_server table. i'm
>>>>>>>>>>> always in favor of reducing complexity... :)
>>>>>>>>>>>
>>>>>>>>>>> jeremy
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Yes - I think its a reasonable default to have a delivery
>>>>>>>>>>>> service assigned to all caches as long as we still keep the ability to
>>>>>>>>>>>> assign some DS’s to just a subset of caches.
>>>>>>>>>>>>
>>>>>>>>>>>> —Eric
>>>>>>>>>>>>
>>>>>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <
>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Good feedback, thanks. I think having _zero_ caches assigned to
>>>>>>>>>>>> a delivery service could be added fairly easily, and could function as a
>>>>>>>>>>>> good default to the assumption of _all_ caches being assigned. Would you
>>>>>>>>>>>> agree with that?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Mark
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Yeah,
>>>>>>>>>>>>> We have several some use cases where where cache or CG level
>>>>>>>>>>>>> allocation would be helpful.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1) Assigning new or trial delivery services to a subset of
>>>>>>>>>>>>> caches (considered beta cache nodes). This is helpful in trying out new DS
>>>>>>>>>>>>> configs before rolling them out to production.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or resource
>>>>>>>>>>>>> reservation on the delivery service. Providers of content to the CDN can
>>>>>>>>>>>>> receive tiered service by either sharing caches or using dedicated caches.
>>>>>>>>>>>>>
>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <
>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Today, there isn't an actual link from Cache Group to DS -- I
>>>>>>>>>>>>> think what
>>>>>>>>>>>>> you are referring to is all done in the app layer to make a
>>>>>>>>>>>>> nice summary in
>>>>>>>>>>>>> the UI to allow operators to very quickly assign all caches in
>>>>>>>>>>>>> a cache
>>>>>>>>>>>>> group.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Do you use that "feature" to assign caches from some cache
>>>>>>>>>>>>> groups, and not
>>>>>>>>>>>>> others?
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
>>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
>>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > Seeking community opinion on this.
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > Should we drop the server -> delivery service assignments
>>>>>>>>>>>>>> completely?
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > When the project began, selectively assigning caches was
>>>>>>>>>>>>>> the only way to selectively route traffic within a cache group -- let's say
>>>>>>>>>>>>>> you have a DNS-routed service with a relatively large library size, you
>>>>>>>>>>>>>> might not want assign all caches in a cache group to that service for fear
>>>>>>>>>>>>>> you would 'pollute' those caches. Or, if you were turning up a 'test' DS of
>>>>>>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
>>>>>>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
>>>>>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
>>>>>>>>>>>>>> services, so the same functionality can essentially be reached with
>>>>>>>>>>>>>> assigning all caches, and setting the config params properly.
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > The motivation for this is to reduce the size & complexity
>>>>>>>>>>>>>> of the Traffic Router's config (aka CRConfig), as this config scales mostly
>>>>>>>>>>>>>> linearly with the number of caches and the number of delivery services in
>>>>>>>>>>>>>> your CDN.
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > Another option might be to not assign caches as the
>>>>>>>>>>>>>> default, and individual cache assignments is available, but used only when
>>>>>>>>>>>>>> necessary.
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > Other thoughts? Clarifications needed?
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > Thanks,
>>>>>>>>>>>>>> > Mark
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>
>
Re: [EXTERNAL] Re: Drop Server -> Delivery Service assignments?
Posted by "Schmidt, Andrew (Contractor)" <An...@comcast.com>.
Rawlin,
This sounds good. TR would still need to know the mappings between the service groups and the servers so that would need to be in the config somewhere.
Andy
From: Nir Sopher <ni...@qwilt.com>
Reply-To: "users@trafficcontrol.apache.org" <us...@trafficcontrol.apache.org>
Date: Wednesday, January 23, 2019 at 4:55 PM
To: "users@trafficcontrol.apache.org" <us...@trafficcontrol.apache.org>
Subject: Re: [EXTERNAL] Re: Drop Server -> Delivery Service assignments?
+1 for topologies.
In self service, the customer should not be aware of the cache group structure. He should rather be familiar with the topology the ISP wants to expose.
Still, I would avoid from pushing business layer logic into the router. As I see it, all translations should be done in OPs.
Keep in mind that this is required even for the ability to debug the system. Leys say the admin user would like to see which cache is assigned with each ds. If the logic is in the router, only a developer will be able to check it. If the logic is in the ops, admin will be able to see it easily.
Nir
On Thu, Jan 24, 2019, 01:24 Rawlin Peters <ra...@gmail.com> wrote:
The Flexible Cachegroup (a.k.a BYOT - Bring Your Own Topology) stuff
I'm planning to start working on again soon will alleviate a lot of
these issues I think. Basically the plan is you will group servers
into Service Groups, then you will compose those Service Groups into 1
or more Topologies. Delivery Services would then be assigned to a
Topology rather than assigned to specific edge caches.
I haven't ironed out all the details yet, but the CRConfig could be
changed like this:
- instead of a Server listing all the DSes it has assigned, it would
just list the "edge" Service Groups its assigned to
- the CRConfig would list the Topologies and which "edge" service
groups they're composed of
- the DS entry would just contain the one Topology it's assigned to
Rather than scaling the size of the CRConfig at (S*D) (servers times
deliveryservices), it would scale more linearly with the number of
Servers, Topologies, and Service Groups, respectively. So adding an
edge cache with all DSes assigned to the CDN wouldn't mean adding D
delivery service assignments to the CRConfig, and adding a new
delivery service assigned to all servers wouldn't mean adding S server
assignments to the CRConfig. I imagine this would keep the CRConfig
small enough.
This is very similar to Andy's proposal of assigning delivery services
to cachegroups, and from the perspective of Traffic Router, it's
almost as if the DSes _were_ actually assigned to cachegroups rather
than to servers.
Related to Rob's proposal for "default-assigned" delivery services,
you could set a Topology as the "default" Topology which would be used
when a DS hasn't been assigned to a Topology. We could even tie the
idea of a "default" Topology to tenancy and be able to choose a
default Topology per tenant. Then self-service users wouldn't have to
deal with DS-Server assignments directly and would just pick from the
set of "authorized" Topologies (or just take the "default") for that
specific tenant.
- Rawlin
On Tue, Jan 22, 2019 at 9:01 PM Schmidt, Andrew (Contractor)
<An...@comcast.com>> wrote:
>
> It sounds to me like assigning Delivery Services to cache groups would solve all of these problems. We could create special cache groups for testing if necessary or even cache groups with just a one cache in it if needed. When new caches get added or removed from a cache group then no changes would need to made to DS config. We could also do a lot in the TP UI for self-service. We could have a default option in the UI which assigns all cache groups and then we could change that in the future if we want to change the model. For instance, I could imagine charging customers per the number of cache groups they want, which would then tie the cache group assignments to roles and capabilities. At worse, the self-service user would need to select from their authorized cache groups when creating a DS. The cache group assignments could be a part of the delivery service configuration, which would then allow us to completely separate the DS configurations from everything else in the CRConfig.
>
>
>
> Andy
>
>
>
> From: Nir Sopher <ni...@qwilt.com>>
> Reply-To: "users@trafficcontrol.apache.org<ma...@trafficcontrol.apache.org>" <us...@trafficcontrol.apache.org>>
> Date: Tuesday, January 22, 2019 at 11:55 AM
> To: "users@trafficcontrol.apache.org<ma...@trafficcontrol.apache.org>" <us...@trafficcontrol.apache.org>>
> Subject: [EXTERNAL] Re: Drop Server -> Delivery Service assignments?
>
>
>
> Hi Rob,
>
>
>
> I strongly agree with the first part of your suggestion. I think that giving the user only fine grain control on the ds/servers mapping, and holding such a table is in the db, is not scalable nor maintable. I would suggest to use some kind of rule table, allowing the assignment of ds to all caches, specific cache groups, or specific cache.
>
>
>
> I see less value in removing the dss mapping from the cr config. It indeed makes the cr config smaller, but this is not the creteria I believe we should discuss arround "breaking up the huge CRConfig". From self service point of view, we need to break the cr config in a way that reduces the dependency between different elements cfg chage, and specifically delivery services - so you can easily change the configuration and footprint for a single ds. Pushing the dss logic into the router might not fit with this effort.
>
> Anyhow, if we decide to go in this direction in the cr-config level as well, I think it would be important to preserve the ability to have specific dss mapping in the cr config. Maybe build the router cr-config digestion in layers - where the higher layer have some logic for creating the dss, and the lower layer work with dss mapping as an input.
>
>
>
> Nir
>
>
>
> On Wed, Jan 2, 2019, 20:08 Robert O Butts <ro...@apache.org> wrote:
>
> Reviving this, because I noticed a few things, when working on the DS Snapshots PR:
>
> (1) I noticed the DeliveryService-Servers query is _by far_ the slowest query we have, building the CRConfig. For a given machine and a given large CDN, selecting all DS data takes about ~10ms; the DSS query takes ~200ms. This is to trivially select the data, with indexes. If you ever need a nontrivial query with nontrivial joins, it quickly becomes seconds.
>
> (2) Writing code, I accidentally generated a CRConfig without DeliveryService-Servers. It was ~0.5mb (uncompressed). Compared to ~10mb for the same CDN with DSSes.
>
> These sizes and query times are because the DSSes are a cross product of DS and Servers.
>
> I think we need to do this, and not only in the interface, but implement it such that we (1) don't actually store the DSS mappings, for DSes and Servers which are "default-assigned", and (2) don't put them in the CRConfig. In terms of performance, I strongly suspect it will be faster for the Router to compute them, than the time it takes it to receive all that data over the network.
>
> We keep talking about "breaking up the huge CRConfig" - it looks to me like removing DSSes will make it so small we don't _need_ to break it up anytime soon.
>
> To @mtorluemke 's original point, doing this will significantly improve scalability, completely aside from the Self Service and interface benefits.
>
> It sounds like there was consensus here. We just need to do it. I'm suggesting, with the evidence above, this really ought to be a higher priority. The evidence suggests fixing this at the data level will significantly improve the scalability of Traffic Control.
>
> On 2017/05/24 19:43:53, Nir Sopher <ni...@qwilt.com>> wrote:
> > Hi Jeremy,
> >
> > I’m ok with automatically assigning all edges to a new DS. The aspect of
> > "self-service" can be dealt with in a different manner.
> > However, how would you solve the problem when a new server is created? How
> > would you know which DS it should be automatically be assigned?
> > The model has no configuration marking the DS as one that should be
> > deployed on all servers.
> > My suggestion was to model it with "undef" value in the DS/Servers table.
> > DS | Server
> > ------+--------
> > ds1 | server1 => Allow the CDN owner to deploy the ds on a specific cache
> > ------+--------
> > ds2 | *undef *=> Allow the CDN owner to deploy the ds on all caches
> >
> > This basically means that every place reading the DS/Server table, will
> > have to use a wrapper, unfolding a the "server=undef" raw, to a raw per
> > relevant server in the CDN.
> > I'm not sure how much effort it requires.
> >
> > Nir
> >
> > On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <mi...@gmail.com>>
> > wrote:
> >
> > > Nir,
> > >
> > > Thinking about self-service, do you envision the "DS owner" creates the DS
> > > and the "CDN owner" assigns caches to the DS? This seems like an
> > > unnecessary step to me and one that negates the idea of "self service".
> > >
> > > As the "DS owner", I want to create a DS and by default, employ ALL caches
> > > in my cdn. To be honest, in 99% of cases I really don't want to be bothered
> > > with assigning caches. I want the full power of the CDN behind my DS.
> > >
> > > So, I'm suggesting that when a new DS is created thru the API, ALL edge
> > > caches in the cdn are assigned to that DS.....and then, the CDN owner (or
> > > maybe DS owner) can remove caches from that DS if they need to.
> > >
> > > Jeremy
> > >
> > > On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com>> wrote:
> > >
> > >> @Jeremy
> > >>
> > >> The issue I was trying to point at, is that the "DS-Server" table is the
> > >> one to provide the separation between the DS owner domain and the CDN owner
> > >> domain (tenancy wise).
> > >> As I see it, when considering multi tenancy, the DS has its owning tenant
> > >> which is in charge of the specific record in the DS table, and specifically
> > >> on adding the DS. The CDN is usually owned by another tenant that controls
> > >> the deployment - currently by adding the ds/servers record to the DS/Server
> > >> table.
> > >>
> > >> If the DS/Server table is dismissed or automatically written as a result
> > >> of a ds addition, it removes the "server assignment" step and the CDN owner
> > >> loses the control over the deployment decision.
> > >>
> > >> Before I open a discussion about how to solve this new tenancy issue, I
> > >> would like to verify it is a real issue and there is not something I'm
> > >> missing there.
> > >> What do you think?
> > >>
> > >> Nir
> > >>
> > >> On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com>> wrote:
> > >>
> > >>> @Nir - maybe your proposal is something to consider more long-term but
> > >>> for the short-term i'm wondering if one of the following can / should be
> > >>> implemented:
> > >>>
> > >>> A) creating a DS thru the API (POST /api/*/deliveryservices) can have a
> > >>> default effect of automatically assigning all edges to the new DS (via the deliveryservice_server
> > >>> table)
> > >>>
> > >>> or
> > >>>
> > >>> B) the deliveryservice_server table simple goes away and it becomes
> > >>> implied that a DS employs all edge caches
> > >>>
> > >>> "A" would probably be the easiest to be honest.
> > >>>
> > >>> jeremy
> > >>>
> > >>> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com>> wrote:
> > >>>
> > >>>> +1 for having a configuration allowing to apply the DS to the entire
> > >>>> CDN.
> > >>>>
> > >>>> Note that adding a "deploy by default" bool field to the DS itself may
> > >>>> cause tenancy issues, as I believe this field should be in the control of
> > >>>> the "CDN owner" tenant, and not of the "DS owner".
> > >>>>
> > >>>> What about keeping the ds_server table, and allow "undef" for the
> > >>>> server value - meaning "all servers in the CDN"?
> > >>>> In the future the table may grow and have additional fields to
> > >>>> white-list servers by.
> > >>>> For example:
> > >>>>
> > >>>> DS | Server | cache-group | profile
> > >>>> ------+--------+-------------+--------
> > >>>> ds1 | s1 | null | null => ds to be applied on s1
> > >>>> ------+--------+-------------+--------
> > >>>> ds2 | null | null | null => ds to be applied on all
> > >>>> servers (in ds CDN)
> > >>>> ------+--------+-------------+--------
> > >>>> ds3 | null | group1 | null => ds to be applied on all
> > >>>> servers in cache group "group1"
> > >>>> ------+--------+-------------+--------
> > >>>> ds4 | null | group1 | ATS-6.2 => ds to be applied on all
> > >>>> servers in cache group "group1", with profile "ATS-6.2"
> > >>>>
> > >>>>
> > >>>> During CR config snapshot and configuration pull (or in advance), the
> > >>>> required calculation would be done, to provide the list of relevant DSs per
> > >>>> server.
> > >>>>
> > >>>> Also note that moving in this direction is backward compatible as well
> > >>>> as supports other non trivial extensions one may want in the future:
> > >>>> Patterns (wildcards/regexs) support, as well as black list, using a "policy
> > >>>> table" mechanism.
> > >>>>
> > >>>> In the below example "ds5" is to be deployed on all caches, except for
> > >>>> caches with "ATS-5" profile
> > >>>>
> > >>>> DS | Server | cache-group | profile | deploy-decision
> > >>>> ------+--------+-------------+---------+----------------
> > >>>> ds5 | null | null | ATS-5 | false
> > >>>> ------+--------+-------------+---------+----------------
> > >>>> ds5 | null | null | null | true
> > >>>> ------+--------+-------------+---------+----------------
> > >>>>
> > >>>>
> > >>>> Nir
> > >>>>
> > >>>> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <mi...@gmail.com>
> > >>>> > wrote:
> > >>>>
> > >>>>> @dave - i'm not sure if you're suggesting:
> > >>>>>
> > >>>>> A) on DS create, by default create an entry in the
> > >>>>> deliveryservice_server table for each cdn edge cache. so if there are 200
> > >>>>> cdn edge caches, you end up with 200 entries in the deliveryservice_server
> > >>>>> table for that DS.
> > >>>>> B) completely kill the deliveryservice_server table and it becomes
> > >>>>> implied that a DS uses ALL cdn edge caches (the way mids work today.
> > >>>>> mids don't have to be explicitly assigned to a DS)
> > >>>>>
> > >>>>> "A" would allow you to control the scope of caches by using initial
> > >>>>> dispersion OR control the scope of caches by simply removing the ones
> > >>>>> you don't want to receive traffic.
> > >>>>> "B" would only allow you to control the scope of caches by using
> > >>>>> initial dispersion (or so i think).
> > >>>>>
> > >>>>> jeremy
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> +1, I think assigning DSs to all caches by default is fine. Then we
> > >>>>>> can use initial dispersion to make sure traffic is spread between an
> > >>>>>> appropriate number of caches.
> > >>>>>>
> > >>>>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <
> > >>>>>> mitchell852@gmail.com<ma...@gmail.com>> wrote:
> > >>>>>>
> > >>>>>>> Bringing up an old email regarding DS to cache assignments.
> > >>>>>>>
> > >>>>>>> It sounds like we want to preserve DS/cache assignments (the
> > >>>>>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
> > >>>>>>> to a DS if you want. However, what if we make the assignment of ALL cdn
> > >>>>>>> edge caches to a new DS the default?
> > >>>>>>>
> > >>>>>>> So when you create a DS for the Foo cdn thru the API, your new DS is
> > >>>>>>> automagically assigned all the edge caches in the Foo cdn...
> > >>>>>>>
> > >>>>>>> (at that point you could always remove caches from a DS)
> > >>>>>>>
> > >>>>>>> ...or the create DS API could have a boolean flag added to it. for
> > >>>>>>> example, assignAllCaches: true|false. if you pass in true, all caches are
> > >>>>>>> assigned to the DS, if false, none are.
> > >>>>>>>
> > >>>>>>> thoughts? I'm trying to think of self-service and i'm guessing most
> > >>>>>>> people will not want to assign caches to a DS...
> > >>>>>>>
> > >>>>>>> jeremy
> > >>>>>>>
> > >>>>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
> > >>>>>>> mitchell852@gmail.com<ma...@gmail.com>> wrote:
> > >>>>>>>
> > >>>>>>>> gotcha. makes sense.
> > >>>>>>>>
> > >>>>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <
> > >>>>>>>> david.neuman64@gmail.com<ma...@gmail.com>> wrote:
> > >>>>>>>>
> > >>>>>>>>> I believe Eric had a use case where he wanted to assign certain
> > >>>>>>>>> delivery services to certain caches.
> > >>>>>>>>> If we just use initial dispersion and max DNS answers then we
> > >>>>>>>>> don't have the ability to control which servers are used, just how many.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
> > >>>>>>>>> mitchell852@gmail.com<ma...@gmail.com>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> So what I heard was that potentially we move in this direction:
> > >>>>>>>>>>
> > >>>>>>>>>> 1. No need to explicitly assign edge caches to a delivery
> > >>>>>>>>>> service. a delivery service (by default) would employ ALL edge caches of
> > >>>>>>>>>> the cdn in which the ds belongs to.
> > >>>>>>>>>> 2. Ability to override this default behavior and explicitly
> > >>>>>>>>>> assign specific edge caches to a delivery service (this is what we do today)
> > >>>>>>>>>>
> > >>>>>>>>>> But earlier Mark said "so the same functionality [as #2] can
> > >>>>>>>>>> essentially be reached with assigning all caches, and setting the config
> > >>>>>>>>>> params properly [initial dispersion and max dns answers]"
> > >>>>>>>>>>
> > >>>>>>>>>> so why not ditch #2 in favor of using initial dispersion and max
> > >>>>>>>>>> dns answers to control cache assignments?
> > >>>>>>>>>>
> > >>>>>>>>>> it would be nice imo to get rid of the ds_server table. i'm
> > >>>>>>>>>> always in favor of reducing complexity... :)
> > >>>>>>>>>>
> > >>>>>>>>>> jeremy
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
> > >>>>>>>>>> efriedri@cisco.com<ma...@cisco.com>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Yes - I think its a reasonable default to have a delivery
> > >>>>>>>>>>> service assigned to all caches as long as we still keep the ability to
> > >>>>>>>>>>> assign some DS’s to just a subset of caches.
> > >>>>>>>>>>>
> > >>>>>>>>>>> —Eric
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <
> > >>>>>>>>>>> mtorluemke@apache.org<ma...@apache.org>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Good feedback, thanks. I think having _zero_ caches assigned to
> > >>>>>>>>>>> a delivery service could be added fairly easily, and could function as a
> > >>>>>>>>>>> good default to the assumption of _all_ caches being assigned. Would you
> > >>>>>>>>>>> agree with that?
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks,
> > >>>>>>>>>>> Mark
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
> > >>>>>>>>>>> efriedri@cisco.com<ma...@cisco.com>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Yeah,
> > >>>>>>>>>>>> We have several some use cases where where cache or CG level
> > >>>>>>>>>>>> allocation would be helpful.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> 1) Assigning new or trial delivery services to a subset of
> > >>>>>>>>>>>> caches (considered beta cache nodes). This is helpful in trying out new DS
> > >>>>>>>>>>>> configs before rolling them out to production.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or resource
> > >>>>>>>>>>>> reservation on the delivery service. Providers of content to the CDN can
> > >>>>>>>>>>>> receive tiered service by either sharing caches or using dedicated caches.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> —Eric
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <
> > >>>>>>>>>>>> mtorluemke@apache.org<ma...@apache.org>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Today, there isn't an actual link from Cache Group to DS -- I
> > >>>>>>>>>>>> think what
> > >>>>>>>>>>>> you are referring to is all done in the app layer to make a
> > >>>>>>>>>>>> nice summary in
> > >>>>>>>>>>>> the UI to allow operators to very quickly assign all caches in
> > >>>>>>>>>>>> a cache
> > >>>>>>>>>>>> group.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Do you use that "feature" to assign caches from some cache
> > >>>>>>>>>>>> groups, and not
> > >>>>>>>>>>>> others?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
> > >>>>>>>>>>>> efriedri@cisco.com<ma...@cisco.com>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> —Eric
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
> > >>>>>>>>>>>>> mtorluemke@apache.org<ma...@apache.org>> wrote:
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > Seeking community opinion on this.
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > Should we drop the server -> delivery service assignments
> > >>>>>>>>>>>>> completely?
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > When the project began, selectively assigning caches was the
> > >>>>>>>>>>>>> only way to selectively route traffic within a cache group -- let's say you
> > >>>>>>>>>>>>> have a DNS-routed service with a relatively large library size, you might
> > >>>>>>>>>>>>> not want assign all caches in a cache group to that service for fear you
> > >>>>>>>>>>>>> would 'pollute' those caches. Or, if you were turning up a 'test' DS of
> > >>>>>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
> > >>>>>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
> > >>>>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
> > >>>>>>>>>>>>> services, so the same functionality can essentially be reached with
> > >>>>>>>>>>>>> assigning all caches, and setting the config params properly.
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > The motivation for this is to reduce the size & complexity
> > >>>>>>>>>>>>> of the Traffic Router's config (aka CRConfig), as this config scales mostly
> > >>>>>>>>>>>>> linearly with the number of caches and the number of delivery services in
> > >>>>>>>>>>>>> your CDN.
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > Another option might be to not assign caches as the default,
> > >>>>>>>>>>>>> and individual cache assignments is available, but used only when necessary.
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > Other thoughts? Clarifications needed?
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > Thanks,
> > >>>>>>>>>>>>> > Mark
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >
> >
Re: [EXTERNAL] Re: Drop Server -> Delivery Service assignments?
Posted by Nir Sopher <ni...@qwilt.com>.
+1 for topologies.
In self service, the customer should not be aware of the cache group
structure. He should rather be familiar with the topology the ISP wants to
expose.
Still, I would avoid from pushing business layer logic into the router. As
I see it, all translations should be done in OPs.
Keep in mind that this is required even for the ability to debug the
system. Leys say the admin user would like to see which cache is assigned
with each ds. If the logic is in the router, only a developer will be able
to check it. If the logic is in the ops, admin will be able to see it
easily.
Nir
On Thu, Jan 24, 2019, 01:24 Rawlin Peters <rawlin.peters@gmail.com wrote:
> The Flexible Cachegroup (a.k.a BYOT - Bring Your Own Topology) stuff
> I'm planning to start working on again soon will alleviate a lot of
> these issues I think. Basically the plan is you will group servers
> into Service Groups, then you will compose those Service Groups into 1
> or more Topologies. Delivery Services would then be assigned to a
> Topology rather than assigned to specific edge caches.
>
> I haven't ironed out all the details yet, but the CRConfig could be
> changed like this:
> - instead of a Server listing all the DSes it has assigned, it would
> just list the "edge" Service Groups its assigned to
> - the CRConfig would list the Topologies and which "edge" service
> groups they're composed of
> - the DS entry would just contain the one Topology it's assigned to
>
> Rather than scaling the size of the CRConfig at (S*D) (servers times
> deliveryservices), it would scale more linearly with the number of
> Servers, Topologies, and Service Groups, respectively. So adding an
> edge cache with all DSes assigned to the CDN wouldn't mean adding D
> delivery service assignments to the CRConfig, and adding a new
> delivery service assigned to all servers wouldn't mean adding S server
> assignments to the CRConfig. I imagine this would keep the CRConfig
> small enough.
>
> This is very similar to Andy's proposal of assigning delivery services
> to cachegroups, and from the perspective of Traffic Router, it's
> almost as if the DSes _were_ actually assigned to cachegroups rather
> than to servers.
>
> Related to Rob's proposal for "default-assigned" delivery services,
> you could set a Topology as the "default" Topology which would be used
> when a DS hasn't been assigned to a Topology. We could even tie the
> idea of a "default" Topology to tenancy and be able to choose a
> default Topology per tenant. Then self-service users wouldn't have to
> deal with DS-Server assignments directly and would just pick from the
> set of "authorized" Topologies (or just take the "default") for that
> specific tenant.
>
> - Rawlin
>
> On Tue, Jan 22, 2019 at 9:01 PM Schmidt, Andrew (Contractor)
> <An...@comcast.com> wrote:
> >
> > It sounds to me like assigning Delivery Services to cache groups would
> solve all of these problems. We could create special cache groups for
> testing if necessary or even cache groups with just a one cache in it if
> needed. When new caches get added or removed from a cache group then no
> changes would need to made to DS config. We could also do a lot in the TP
> UI for self-service. We could have a default option in the UI which assigns
> all cache groups and then we could change that in the future if we want to
> change the model. For instance, I could imagine charging customers per the
> number of cache groups they want, which would then tie the cache group
> assignments to roles and capabilities. At worse, the self-service user
> would need to select from their authorized cache groups when creating a DS.
> The cache group assignments could be a part of the delivery service
> configuration, which would then allow us to completely separate the DS
> configurations from everything else in the CRConfig.
> >
> >
> >
> > Andy
> >
> >
> >
> > From: Nir Sopher <ni...@qwilt.com>
> > Reply-To: "users@trafficcontrol.apache.org" <
> users@trafficcontrol.apache.org>
> > Date: Tuesday, January 22, 2019 at 11:55 AM
> > To: "users@trafficcontrol.apache.org" <us...@trafficcontrol.apache.org>
> > Subject: [EXTERNAL] Re: Drop Server -> Delivery Service assignments?
> >
> >
> >
> > Hi Rob,
> >
> >
> >
> > I strongly agree with the first part of your suggestion. I think that
> giving the user only fine grain control on the ds/servers mapping, and
> holding such a table is in the db, is not scalable nor maintable. I would
> suggest to use some kind of rule table, allowing the assignment of ds to
> all caches, specific cache groups, or specific cache.
> >
> >
> >
> > I see less value in removing the dss mapping from the cr config. It
> indeed makes the cr config smaller, but this is not the creteria I believe
> we should discuss arround "breaking up the huge CRConfig". From self
> service point of view, we need to break the cr config in a way that reduces
> the dependency between different elements cfg chage, and specifically
> delivery services - so you can easily change the configuration and
> footprint for a single ds. Pushing the dss logic into the router might not
> fit with this effort.
> >
> > Anyhow, if we decide to go in this direction in the cr-config level as
> well, I think it would be important to preserve the ability to have
> specific dss mapping in the cr config. Maybe build the router cr-config
> digestion in layers - where the higher layer have some logic for creating
> the dss, and the lower layer work with dss mapping as an input.
> >
> >
> >
> > Nir
> >
> >
> >
> > On Wed, Jan 2, 2019, 20:08 Robert O Butts <rob@apache.org wrote:
> >
> > Reviving this, because I noticed a few things, when working on the DS
> Snapshots PR:
> >
> > (1) I noticed the DeliveryService-Servers query is _by far_ the slowest
> query we have, building the CRConfig. For a given machine and a given large
> CDN, selecting all DS data takes about ~10ms; the DSS query takes ~200ms.
> This is to trivially select the data, with indexes. If you ever need a
> nontrivial query with nontrivial joins, it quickly becomes seconds.
> >
> > (2) Writing code, I accidentally generated a CRConfig without
> DeliveryService-Servers. It was ~0.5mb (uncompressed). Compared to ~10mb
> for the same CDN with DSSes.
> >
> > These sizes and query times are because the DSSes are a cross product of
> DS and Servers.
> >
> > I think we need to do this, and not only in the interface, but implement
> it such that we (1) don't actually store the DSS mappings, for DSes and
> Servers which are "default-assigned", and (2) don't put them in the
> CRConfig. In terms of performance, I strongly suspect it will be faster for
> the Router to compute them, than the time it takes it to receive all that
> data over the network.
> >
> > We keep talking about "breaking up the huge CRConfig" - it looks to me
> like removing DSSes will make it so small we don't _need_ to break it up
> anytime soon.
> >
> > To @mtorluemke 's original point, doing this will significantly improve
> scalability, completely aside from the Self Service and interface benefits.
> >
> > It sounds like there was consensus here. We just need to do it. I'm
> suggesting, with the evidence above, this really ought to be a higher
> priority. The evidence suggests fixing this at the data level will
> significantly improve the scalability of Traffic Control.
> >
> > On 2017/05/24 19:43:53, Nir Sopher <ni...@qwilt.com> wrote:
> > > Hi Jeremy,
> > >
> > > I’m ok with automatically assigning all edges to a new DS. The aspect
> of
> > > "self-service" can be dealt with in a different manner.
> > > However, how would you solve the problem when a new server is created?
> How
> > > would you know which DS it should be automatically be assigned?
> > > The model has no configuration marking the DS as one that should be
> > > deployed on all servers.
> > > My suggestion was to model it with "undef" value in the DS/Servers
> table.
> > > DS | Server
> > > ------+--------
> > > ds1 | server1 => Allow the CDN owner to deploy the ds on a specific
> cache
> > > ------+--------
> > > ds2 | *undef *=> Allow the CDN owner to deploy the ds on all caches
> > >
> > > This basically means that every place reading the DS/Server table, will
> > > have to use a wrapper, unfolding a the "server=undef" raw, to a raw per
> > > relevant server in the CDN.
> > > I'm not sure how much effort it requires.
> > >
> > > Nir
> > >
> > > On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <
> mitchell852@gmail.com>
> > > wrote:
> > >
> > > > Nir,
> > > >
> > > > Thinking about self-service, do you envision the "DS owner" creates
> the DS
> > > > and the "CDN owner" assigns caches to the DS? This seems like an
> > > > unnecessary step to me and one that negates the idea of "self
> service".
> > > >
> > > > As the "DS owner", I want to create a DS and by default, employ ALL
> caches
> > > > in my cdn. To be honest, in 99% of cases I really don't want to be
> bothered
> > > > with assigning caches. I want the full power of the CDN behind my DS.
> > > >
> > > > So, I'm suggesting that when a new DS is created thru the API, ALL
> edge
> > > > caches in the cdn are assigned to that DS.....and then, the CDN
> owner (or
> > > > maybe DS owner) can remove caches from that DS if they need to.
> > > >
> > > > Jeremy
> > > >
> > > > On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com> wrote:
> > > >
> > > >> @Jeremy
> > > >>
> > > >> The issue I was trying to point at, is that the "DS-Server" table
> is the
> > > >> one to provide the separation between the DS owner domain and the
> CDN owner
> > > >> domain (tenancy wise).
> > > >> As I see it, when considering multi tenancy, the DS has its owning
> tenant
> > > >> which is in charge of the specific record in the DS table, and
> specifically
> > > >> on adding the DS. The CDN is usually owned by another tenant that
> controls
> > > >> the deployment - currently by adding the ds/servers record to the
> DS/Server
> > > >> table.
> > > >>
> > > >> If the DS/Server table is dismissed or automatically written as a
> result
> > > >> of a ds addition, it removes the "server assignment" step and the
> CDN owner
> > > >> loses the control over the deployment decision.
> > > >>
> > > >> Before I open a discussion about how to solve this new tenancy
> issue, I
> > > >> would like to verify it is a real issue and there is not something
> I'm
> > > >> missing there.
> > > >> What do you think?
> > > >>
> > > >> Nir
> > > >>
> > > >> On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com>
> wrote:
> > > >>
> > > >>> @Nir - maybe your proposal is something to consider more long-term
> but
> > > >>> for the short-term i'm wondering if one of the following can /
> should be
> > > >>> implemented:
> > > >>>
> > > >>> A) creating a DS thru the API (POST /api/*/deliveryservices) can
> have a
> > > >>> default effect of automatically assigning all edges to the new DS
> (via the deliveryservice_server
> > > >>> table)
> > > >>>
> > > >>> or
> > > >>>
> > > >>> B) the deliveryservice_server table simple goes away and it becomes
> > > >>> implied that a DS employs all edge caches
> > > >>>
> > > >>> "A" would probably be the easiest to be honest.
> > > >>>
> > > >>> jeremy
> > > >>>
> > > >>> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com>
> wrote:
> > > >>>
> > > >>>> +1 for having a configuration allowing to apply the DS to the
> entire
> > > >>>> CDN.
> > > >>>>
> > > >>>> Note that adding a "deploy by default" bool field to the DS
> itself may
> > > >>>> cause tenancy issues, as I believe this field should be in the
> control of
> > > >>>> the "CDN owner" tenant, and not of the "DS owner".
> > > >>>>
> > > >>>> What about keeping the ds_server table, and allow "undef" for the
> > > >>>> server value - meaning "all servers in the CDN"?
> > > >>>> In the future the table may grow and have additional fields to
> > > >>>> white-list servers by.
> > > >>>> For example:
> > > >>>>
> > > >>>> DS | Server | cache-group | profile
> > > >>>> ------+--------+-------------+--------
> > > >>>> ds1 | s1 | null | null => ds to be applied
> on s1
> > > >>>> ------+--------+-------------+--------
> > > >>>> ds2 | null | null | null => ds to be applied
> on all
> > > >>>> servers (in ds CDN)
> > > >>>> ------+--------+-------------+--------
> > > >>>> ds3 | null | group1 | null => ds to be applied
> on all
> > > >>>> servers in cache group "group1"
> > > >>>> ------+--------+-------------+--------
> > > >>>> ds4 | null | group1 | ATS-6.2 => ds to be applied
> on all
> > > >>>> servers in cache group "group1", with profile "ATS-6.2"
> > > >>>>
> > > >>>>
> > > >>>> During CR config snapshot and configuration pull (or in advance),
> the
> > > >>>> required calculation would be done, to provide the list of
> relevant DSs per
> > > >>>> server.
> > > >>>>
> > > >>>> Also note that moving in this direction is backward compatible as
> well
> > > >>>> as supports other non trivial extensions one may want in the
> future:
> > > >>>> Patterns (wildcards/regexs) support, as well as black list, using
> a "policy
> > > >>>> table" mechanism.
> > > >>>>
> > > >>>> In the below example "ds5" is to be deployed on all caches,
> except for
> > > >>>> caches with "ATS-5" profile
> > > >>>>
> > > >>>> DS | Server | cache-group | profile | deploy-decision
> > > >>>> ------+--------+-------------+---------+----------------
> > > >>>> ds5 | null | null | ATS-5 | false
> > > >>>> ------+--------+-------------+---------+----------------
> > > >>>> ds5 | null | null | null | true
> > > >>>> ------+--------+-------------+---------+----------------
> > > >>>>
> > > >>>>
> > > >>>> Nir
> > > >>>>
> > > >>>> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <
> mitchell852@gmail.com
> > > >>>> > wrote:
> > > >>>>
> > > >>>>> @dave - i'm not sure if you're suggesting:
> > > >>>>>
> > > >>>>> A) on DS create, by default create an entry in the
> > > >>>>> deliveryservice_server table for each cdn edge cache. so if
> there are 200
> > > >>>>> cdn edge caches, you end up with 200 entries in the
> deliveryservice_server
> > > >>>>> table for that DS.
> > > >>>>> B) completely kill the deliveryservice_server table and it
> becomes
> > > >>>>> implied that a DS uses ALL cdn edge caches (the way mids work
> today.
> > > >>>>> mids don't have to be explicitly assigned to a DS)
> > > >>>>>
> > > >>>>> "A" would allow you to control the scope of caches by using
> initial
> > > >>>>> dispersion OR control the scope of caches by simply removing the
> ones
> > > >>>>> you don't want to receive traffic.
> > > >>>>> "B" would only allow you to control the scope of caches by using
> > > >>>>> initial dispersion (or so i think).
> > > >>>>>
> > > >>>>> jeremy
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>
> > > >>>>> wrote:
> > > >>>>>
> > > >>>>>> +1, I think assigning DSs to all caches by default is fine.
> Then we
> > > >>>>>> can use initial dispersion to make sure traffic is spread
> between an
> > > >>>>>> appropriate number of caches.
> > > >>>>>>
> > > >>>>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <
> > > >>>>>> mitchell852@gmail.com> wrote:
> > > >>>>>>
> > > >>>>>>> Bringing up an old email regarding DS to cache assignments.
> > > >>>>>>>
> > > >>>>>>> It sounds like we want to preserve DS/cache assignments (the
> > > >>>>>>> deliveryservice_server table) so you can assign a subset of
> cdn edge caches
> > > >>>>>>> to a DS if you want. However, what if we make the assignment
> of ALL cdn
> > > >>>>>>> edge caches to a new DS the default?
> > > >>>>>>>
> > > >>>>>>> So when you create a DS for the Foo cdn thru the API, your new
> DS is
> > > >>>>>>> automagically assigned all the edge caches in the Foo cdn...
> > > >>>>>>>
> > > >>>>>>> (at that point you could always remove caches from a DS)
> > > >>>>>>>
> > > >>>>>>> ...or the create DS API could have a boolean flag added to it.
> for
> > > >>>>>>> example, assignAllCaches: true|false. if you pass in true, all
> caches are
> > > >>>>>>> assigned to the DS, if false, none are.
> > > >>>>>>>
> > > >>>>>>> thoughts? I'm trying to think of self-service and i'm guessing
> most
> > > >>>>>>> people will not want to assign caches to a DS...
> > > >>>>>>>
> > > >>>>>>> jeremy
> > > >>>>>>>
> > > >>>>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
> > > >>>>>>> mitchell852@gmail.com> wrote:
> > > >>>>>>>
> > > >>>>>>>> gotcha. makes sense.
> > > >>>>>>>>
> > > >>>>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <
> > > >>>>>>>> david.neuman64@gmail.com> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> I believe Eric had a use case where he wanted to assign
> certain
> > > >>>>>>>>> delivery services to certain caches.
> > > >>>>>>>>> If we just use initial dispersion and max DNS answers then we
> > > >>>>>>>>> don't have the ability to control which servers are used,
> just how many.
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
> > > >>>>>>>>> mitchell852@gmail.com> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> So what I heard was that potentially we move in this
> direction:
> > > >>>>>>>>>>
> > > >>>>>>>>>> 1. No need to explicitly assign edge caches to a delivery
> > > >>>>>>>>>> service. a delivery service (by default) would employ ALL
> edge caches of
> > > >>>>>>>>>> the cdn in which the ds belongs to.
> > > >>>>>>>>>> 2. Ability to override this default behavior and explicitly
> > > >>>>>>>>>> assign specific edge caches to a delivery service (this is
> what we do today)
> > > >>>>>>>>>>
> > > >>>>>>>>>> But earlier Mark said "so the same functionality [as #2] can
> > > >>>>>>>>>> essentially be reached with assigning all caches, and
> setting the config
> > > >>>>>>>>>> params properly [initial dispersion and max dns answers]"
> > > >>>>>>>>>>
> > > >>>>>>>>>> so why not ditch #2 in favor of using initial dispersion
> and max
> > > >>>>>>>>>> dns answers to control cache assignments?
> > > >>>>>>>>>>
> > > >>>>>>>>>> it would be nice imo to get rid of the ds_server table. i'm
> > > >>>>>>>>>> always in favor of reducing complexity... :)
> > > >>>>>>>>>>
> > > >>>>>>>>>> jeremy
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
> > > >>>>>>>>>> efriedri@cisco.com> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>> Yes - I think its a reasonable default to have a delivery
> > > >>>>>>>>>>> service assigned to all caches as long as we still keep
> the ability to
> > > >>>>>>>>>>> assign some DS’s to just a subset of caches.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> —Eric
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <
> > > >>>>>>>>>>> mtorluemke@apache.org> wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Good feedback, thanks. I think having _zero_ caches
> assigned to
> > > >>>>>>>>>>> a delivery service could be added fairly easily, and could
> function as a
> > > >>>>>>>>>>> good default to the assumption of _all_ caches being
> assigned. Would you
> > > >>>>>>>>>>> agree with that?
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Thanks,
> > > >>>>>>>>>>> Mark
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
> > > >>>>>>>>>>> efriedri@cisco.com> wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> Yeah,
> > > >>>>>>>>>>>> We have several some use cases where where cache or CG
> level
> > > >>>>>>>>>>>> allocation would be helpful.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> 1) Assigning new or trial delivery services to a subset of
> > > >>>>>>>>>>>> caches (considered beta cache nodes). This is helpful in
> trying out new DS
> > > >>>>>>>>>>>> configs before rolling them out to production.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or
> resource
> > > >>>>>>>>>>>> reservation on the delivery service. Providers of content
> to the CDN can
> > > >>>>>>>>>>>> receive tiered service by either sharing caches or using
> dedicated caches.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> —Eric
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <
> > > >>>>>>>>>>>> mtorluemke@apache.org> wrote:
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Today, there isn't an actual link from Cache Group to DS
> -- I
> > > >>>>>>>>>>>> think what
> > > >>>>>>>>>>>> you are referring to is all done in the app layer to make
> a
> > > >>>>>>>>>>>> nice summary in
> > > >>>>>>>>>>>> the UI to allow operators to very quickly assign all
> caches in
> > > >>>>>>>>>>>> a cache
> > > >>>>>>>>>>>> group.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Do you use that "feature" to assign caches from some cache
> > > >>>>>>>>>>>> groups, and not
> > > >>>>>>>>>>>> others?
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri)
> <
> > > >>>>>>>>>>>> efriedri@cisco.com> wrote:
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> —Eric
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
> > > >>>>>>>>>>>>> mtorluemke@apache.org> wrote:
> > > >>>>>>>>>>>>> >
> > > >>>>>>>>>>>>> > Seeking community opinion on this.
> > > >>>>>>>>>>>>> >
> > > >>>>>>>>>>>>> > Should we drop the server -> delivery service
> assignments
> > > >>>>>>>>>>>>> completely?
> > > >>>>>>>>>>>>> >
> > > >>>>>>>>>>>>> > When the project began, selectively assigning caches
> was the
> > > >>>>>>>>>>>>> only way to selectively route traffic within a cache
> group -- let's say you
> > > >>>>>>>>>>>>> have a DNS-routed service with a relatively large
> library size, you might
> > > >>>>>>>>>>>>> not want assign all caches in a cache group to that
> service for fear you
> > > >>>>>>>>>>>>> would 'pollute' those caches. Or, if you were turning up
> a 'test' DS of
> > > >>>>>>>>>>>>> which you wanted to limit exposure on, you might apply
> the same logic. In
> > > >>>>>>>>>>>>> recent releases, Traffic Control supports "initial
> dispersion" for _both_
> > > >>>>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for
> DNS-routed
> > > >>>>>>>>>>>>> services, so the same functionality can essentially be
> reached with
> > > >>>>>>>>>>>>> assigning all caches, and setting the config params
> properly.
> > > >>>>>>>>>>>>> >
> > > >>>>>>>>>>>>> > The motivation for this is to reduce the size &
> complexity
> > > >>>>>>>>>>>>> of the Traffic Router's config (aka CRConfig), as this
> config scales mostly
> > > >>>>>>>>>>>>> linearly with the number of caches and the number of
> delivery services in
> > > >>>>>>>>>>>>> your CDN.
> > > >>>>>>>>>>>>> >
> > > >>>>>>>>>>>>> > Another option might be to not assign caches as the
> default,
> > > >>>>>>>>>>>>> and individual cache assignments is available, but used
> only when necessary.
> > > >>>>>>>>>>>>> >
> > > >>>>>>>>>>>>> > Other thoughts? Clarifications needed?
> > > >>>>>>>>>>>>> >
> > > >>>>>>>>>>>>> > Thanks,
> > > >>>>>>>>>>>>> > Mark
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >
> > >
>
Re: [EXTERNAL] Re: Drop Server -> Delivery Service assignments?
Posted by Rawlin Peters <ra...@gmail.com>.
The Flexible Cachegroup (a.k.a BYOT - Bring Your Own Topology) stuff
I'm planning to start working on again soon will alleviate a lot of
these issues I think. Basically the plan is you will group servers
into Service Groups, then you will compose those Service Groups into 1
or more Topologies. Delivery Services would then be assigned to a
Topology rather than assigned to specific edge caches.
I haven't ironed out all the details yet, but the CRConfig could be
changed like this:
- instead of a Server listing all the DSes it has assigned, it would
just list the "edge" Service Groups its assigned to
- the CRConfig would list the Topologies and which "edge" service
groups they're composed of
- the DS entry would just contain the one Topology it's assigned to
Rather than scaling the size of the CRConfig at (S*D) (servers times
deliveryservices), it would scale more linearly with the number of
Servers, Topologies, and Service Groups, respectively. So adding an
edge cache with all DSes assigned to the CDN wouldn't mean adding D
delivery service assignments to the CRConfig, and adding a new
delivery service assigned to all servers wouldn't mean adding S server
assignments to the CRConfig. I imagine this would keep the CRConfig
small enough.
This is very similar to Andy's proposal of assigning delivery services
to cachegroups, and from the perspective of Traffic Router, it's
almost as if the DSes _were_ actually assigned to cachegroups rather
than to servers.
Related to Rob's proposal for "default-assigned" delivery services,
you could set a Topology as the "default" Topology which would be used
when a DS hasn't been assigned to a Topology. We could even tie the
idea of a "default" Topology to tenancy and be able to choose a
default Topology per tenant. Then self-service users wouldn't have to
deal with DS-Server assignments directly and would just pick from the
set of "authorized" Topologies (or just take the "default") for that
specific tenant.
- Rawlin
On Tue, Jan 22, 2019 at 9:01 PM Schmidt, Andrew (Contractor)
<An...@comcast.com> wrote:
>
> It sounds to me like assigning Delivery Services to cache groups would solve all of these problems. We could create special cache groups for testing if necessary or even cache groups with just a one cache in it if needed. When new caches get added or removed from a cache group then no changes would need to made to DS config. We could also do a lot in the TP UI for self-service. We could have a default option in the UI which assigns all cache groups and then we could change that in the future if we want to change the model. For instance, I could imagine charging customers per the number of cache groups they want, which would then tie the cache group assignments to roles and capabilities. At worse, the self-service user would need to select from their authorized cache groups when creating a DS. The cache group assignments could be a part of the delivery service configuration, which would then allow us to completely separate the DS configurations from everything else in the CRConfig.
>
>
>
> Andy
>
>
>
> From: Nir Sopher <ni...@qwilt.com>
> Reply-To: "users@trafficcontrol.apache.org" <us...@trafficcontrol.apache.org>
> Date: Tuesday, January 22, 2019 at 11:55 AM
> To: "users@trafficcontrol.apache.org" <us...@trafficcontrol.apache.org>
> Subject: [EXTERNAL] Re: Drop Server -> Delivery Service assignments?
>
>
>
> Hi Rob,
>
>
>
> I strongly agree with the first part of your suggestion. I think that giving the user only fine grain control on the ds/servers mapping, and holding such a table is in the db, is not scalable nor maintable. I would suggest to use some kind of rule table, allowing the assignment of ds to all caches, specific cache groups, or specific cache.
>
>
>
> I see less value in removing the dss mapping from the cr config. It indeed makes the cr config smaller, but this is not the creteria I believe we should discuss arround "breaking up the huge CRConfig". From self service point of view, we need to break the cr config in a way that reduces the dependency between different elements cfg chage, and specifically delivery services - so you can easily change the configuration and footprint for a single ds. Pushing the dss logic into the router might not fit with this effort.
>
> Anyhow, if we decide to go in this direction in the cr-config level as well, I think it would be important to preserve the ability to have specific dss mapping in the cr config. Maybe build the router cr-config digestion in layers - where the higher layer have some logic for creating the dss, and the lower layer work with dss mapping as an input.
>
>
>
> Nir
>
>
>
> On Wed, Jan 2, 2019, 20:08 Robert O Butts <rob@apache.org wrote:
>
> Reviving this, because I noticed a few things, when working on the DS Snapshots PR:
>
> (1) I noticed the DeliveryService-Servers query is _by far_ the slowest query we have, building the CRConfig. For a given machine and a given large CDN, selecting all DS data takes about ~10ms; the DSS query takes ~200ms. This is to trivially select the data, with indexes. If you ever need a nontrivial query with nontrivial joins, it quickly becomes seconds.
>
> (2) Writing code, I accidentally generated a CRConfig without DeliveryService-Servers. It was ~0.5mb (uncompressed). Compared to ~10mb for the same CDN with DSSes.
>
> These sizes and query times are because the DSSes are a cross product of DS and Servers.
>
> I think we need to do this, and not only in the interface, but implement it such that we (1) don't actually store the DSS mappings, for DSes and Servers which are "default-assigned", and (2) don't put them in the CRConfig. In terms of performance, I strongly suspect it will be faster for the Router to compute them, than the time it takes it to receive all that data over the network.
>
> We keep talking about "breaking up the huge CRConfig" - it looks to me like removing DSSes will make it so small we don't _need_ to break it up anytime soon.
>
> To @mtorluemke 's original point, doing this will significantly improve scalability, completely aside from the Self Service and interface benefits.
>
> It sounds like there was consensus here. We just need to do it. I'm suggesting, with the evidence above, this really ought to be a higher priority. The evidence suggests fixing this at the data level will significantly improve the scalability of Traffic Control.
>
> On 2017/05/24 19:43:53, Nir Sopher <ni...@qwilt.com> wrote:
> > Hi Jeremy,
> >
> > I’m ok with automatically assigning all edges to a new DS. The aspect of
> > "self-service" can be dealt with in a different manner.
> > However, how would you solve the problem when a new server is created? How
> > would you know which DS it should be automatically be assigned?
> > The model has no configuration marking the DS as one that should be
> > deployed on all servers.
> > My suggestion was to model it with "undef" value in the DS/Servers table.
> > DS | Server
> > ------+--------
> > ds1 | server1 => Allow the CDN owner to deploy the ds on a specific cache
> > ------+--------
> > ds2 | *undef *=> Allow the CDN owner to deploy the ds on all caches
> >
> > This basically means that every place reading the DS/Server table, will
> > have to use a wrapper, unfolding a the "server=undef" raw, to a raw per
> > relevant server in the CDN.
> > I'm not sure how much effort it requires.
> >
> > Nir
> >
> > On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <mi...@gmail.com>
> > wrote:
> >
> > > Nir,
> > >
> > > Thinking about self-service, do you envision the "DS owner" creates the DS
> > > and the "CDN owner" assigns caches to the DS? This seems like an
> > > unnecessary step to me and one that negates the idea of "self service".
> > >
> > > As the "DS owner", I want to create a DS and by default, employ ALL caches
> > > in my cdn. To be honest, in 99% of cases I really don't want to be bothered
> > > with assigning caches. I want the full power of the CDN behind my DS.
> > >
> > > So, I'm suggesting that when a new DS is created thru the API, ALL edge
> > > caches in the cdn are assigned to that DS.....and then, the CDN owner (or
> > > maybe DS owner) can remove caches from that DS if they need to.
> > >
> > > Jeremy
> > >
> > > On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com> wrote:
> > >
> > >> @Jeremy
> > >>
> > >> The issue I was trying to point at, is that the "DS-Server" table is the
> > >> one to provide the separation between the DS owner domain and the CDN owner
> > >> domain (tenancy wise).
> > >> As I see it, when considering multi tenancy, the DS has its owning tenant
> > >> which is in charge of the specific record in the DS table, and specifically
> > >> on adding the DS. The CDN is usually owned by another tenant that controls
> > >> the deployment - currently by adding the ds/servers record to the DS/Server
> > >> table.
> > >>
> > >> If the DS/Server table is dismissed or automatically written as a result
> > >> of a ds addition, it removes the "server assignment" step and the CDN owner
> > >> loses the control over the deployment decision.
> > >>
> > >> Before I open a discussion about how to solve this new tenancy issue, I
> > >> would like to verify it is a real issue and there is not something I'm
> > >> missing there.
> > >> What do you think?
> > >>
> > >> Nir
> > >>
> > >> On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com> wrote:
> > >>
> > >>> @Nir - maybe your proposal is something to consider more long-term but
> > >>> for the short-term i'm wondering if one of the following can / should be
> > >>> implemented:
> > >>>
> > >>> A) creating a DS thru the API (POST /api/*/deliveryservices) can have a
> > >>> default effect of automatically assigning all edges to the new DS (via the deliveryservice_server
> > >>> table)
> > >>>
> > >>> or
> > >>>
> > >>> B) the deliveryservice_server table simple goes away and it becomes
> > >>> implied that a DS employs all edge caches
> > >>>
> > >>> "A" would probably be the easiest to be honest.
> > >>>
> > >>> jeremy
> > >>>
> > >>> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com> wrote:
> > >>>
> > >>>> +1 for having a configuration allowing to apply the DS to the entire
> > >>>> CDN.
> > >>>>
> > >>>> Note that adding a "deploy by default" bool field to the DS itself may
> > >>>> cause tenancy issues, as I believe this field should be in the control of
> > >>>> the "CDN owner" tenant, and not of the "DS owner".
> > >>>>
> > >>>> What about keeping the ds_server table, and allow "undef" for the
> > >>>> server value - meaning "all servers in the CDN"?
> > >>>> In the future the table may grow and have additional fields to
> > >>>> white-list servers by.
> > >>>> For example:
> > >>>>
> > >>>> DS | Server | cache-group | profile
> > >>>> ------+--------+-------------+--------
> > >>>> ds1 | s1 | null | null => ds to be applied on s1
> > >>>> ------+--------+-------------+--------
> > >>>> ds2 | null | null | null => ds to be applied on all
> > >>>> servers (in ds CDN)
> > >>>> ------+--------+-------------+--------
> > >>>> ds3 | null | group1 | null => ds to be applied on all
> > >>>> servers in cache group "group1"
> > >>>> ------+--------+-------------+--------
> > >>>> ds4 | null | group1 | ATS-6.2 => ds to be applied on all
> > >>>> servers in cache group "group1", with profile "ATS-6.2"
> > >>>>
> > >>>>
> > >>>> During CR config snapshot and configuration pull (or in advance), the
> > >>>> required calculation would be done, to provide the list of relevant DSs per
> > >>>> server.
> > >>>>
> > >>>> Also note that moving in this direction is backward compatible as well
> > >>>> as supports other non trivial extensions one may want in the future:
> > >>>> Patterns (wildcards/regexs) support, as well as black list, using a "policy
> > >>>> table" mechanism.
> > >>>>
> > >>>> In the below example "ds5" is to be deployed on all caches, except for
> > >>>> caches with "ATS-5" profile
> > >>>>
> > >>>> DS | Server | cache-group | profile | deploy-decision
> > >>>> ------+--------+-------------+---------+----------------
> > >>>> ds5 | null | null | ATS-5 | false
> > >>>> ------+--------+-------------+---------+----------------
> > >>>> ds5 | null | null | null | true
> > >>>> ------+--------+-------------+---------+----------------
> > >>>>
> > >>>>
> > >>>> Nir
> > >>>>
> > >>>> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <mitchell852@gmail.com
> > >>>> > wrote:
> > >>>>
> > >>>>> @dave - i'm not sure if you're suggesting:
> > >>>>>
> > >>>>> A) on DS create, by default create an entry in the
> > >>>>> deliveryservice_server table for each cdn edge cache. so if there are 200
> > >>>>> cdn edge caches, you end up with 200 entries in the deliveryservice_server
> > >>>>> table for that DS.
> > >>>>> B) completely kill the deliveryservice_server table and it becomes
> > >>>>> implied that a DS uses ALL cdn edge caches (the way mids work today.
> > >>>>> mids don't have to be explicitly assigned to a DS)
> > >>>>>
> > >>>>> "A" would allow you to control the scope of caches by using initial
> > >>>>> dispersion OR control the scope of caches by simply removing the ones
> > >>>>> you don't want to receive traffic.
> > >>>>> "B" would only allow you to control the scope of caches by using
> > >>>>> initial dispersion (or so i think).
> > >>>>>
> > >>>>> jeremy
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> +1, I think assigning DSs to all caches by default is fine. Then we
> > >>>>>> can use initial dispersion to make sure traffic is spread between an
> > >>>>>> appropriate number of caches.
> > >>>>>>
> > >>>>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <
> > >>>>>> mitchell852@gmail.com> wrote:
> > >>>>>>
> > >>>>>>> Bringing up an old email regarding DS to cache assignments.
> > >>>>>>>
> > >>>>>>> It sounds like we want to preserve DS/cache assignments (the
> > >>>>>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
> > >>>>>>> to a DS if you want. However, what if we make the assignment of ALL cdn
> > >>>>>>> edge caches to a new DS the default?
> > >>>>>>>
> > >>>>>>> So when you create a DS for the Foo cdn thru the API, your new DS is
> > >>>>>>> automagically assigned all the edge caches in the Foo cdn...
> > >>>>>>>
> > >>>>>>> (at that point you could always remove caches from a DS)
> > >>>>>>>
> > >>>>>>> ...or the create DS API could have a boolean flag added to it. for
> > >>>>>>> example, assignAllCaches: true|false. if you pass in true, all caches are
> > >>>>>>> assigned to the DS, if false, none are.
> > >>>>>>>
> > >>>>>>> thoughts? I'm trying to think of self-service and i'm guessing most
> > >>>>>>> people will not want to assign caches to a DS...
> > >>>>>>>
> > >>>>>>> jeremy
> > >>>>>>>
> > >>>>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
> > >>>>>>> mitchell852@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>>> gotcha. makes sense.
> > >>>>>>>>
> > >>>>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <
> > >>>>>>>> david.neuman64@gmail.com> wrote:
> > >>>>>>>>
> > >>>>>>>>> I believe Eric had a use case where he wanted to assign certain
> > >>>>>>>>> delivery services to certain caches.
> > >>>>>>>>> If we just use initial dispersion and max DNS answers then we
> > >>>>>>>>> don't have the ability to control which servers are used, just how many.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
> > >>>>>>>>> mitchell852@gmail.com> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> So what I heard was that potentially we move in this direction:
> > >>>>>>>>>>
> > >>>>>>>>>> 1. No need to explicitly assign edge caches to a delivery
> > >>>>>>>>>> service. a delivery service (by default) would employ ALL edge caches of
> > >>>>>>>>>> the cdn in which the ds belongs to.
> > >>>>>>>>>> 2. Ability to override this default behavior and explicitly
> > >>>>>>>>>> assign specific edge caches to a delivery service (this is what we do today)
> > >>>>>>>>>>
> > >>>>>>>>>> But earlier Mark said "so the same functionality [as #2] can
> > >>>>>>>>>> essentially be reached with assigning all caches, and setting the config
> > >>>>>>>>>> params properly [initial dispersion and max dns answers]"
> > >>>>>>>>>>
> > >>>>>>>>>> so why not ditch #2 in favor of using initial dispersion and max
> > >>>>>>>>>> dns answers to control cache assignments?
> > >>>>>>>>>>
> > >>>>>>>>>> it would be nice imo to get rid of the ds_server table. i'm
> > >>>>>>>>>> always in favor of reducing complexity... :)
> > >>>>>>>>>>
> > >>>>>>>>>> jeremy
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
> > >>>>>>>>>> efriedri@cisco.com> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Yes - I think its a reasonable default to have a delivery
> > >>>>>>>>>>> service assigned to all caches as long as we still keep the ability to
> > >>>>>>>>>>> assign some DS’s to just a subset of caches.
> > >>>>>>>>>>>
> > >>>>>>>>>>> —Eric
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <
> > >>>>>>>>>>> mtorluemke@apache.org> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Good feedback, thanks. I think having _zero_ caches assigned to
> > >>>>>>>>>>> a delivery service could be added fairly easily, and could function as a
> > >>>>>>>>>>> good default to the assumption of _all_ caches being assigned. Would you
> > >>>>>>>>>>> agree with that?
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks,
> > >>>>>>>>>>> Mark
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
> > >>>>>>>>>>> efriedri@cisco.com> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Yeah,
> > >>>>>>>>>>>> We have several some use cases where where cache or CG level
> > >>>>>>>>>>>> allocation would be helpful.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> 1) Assigning new or trial delivery services to a subset of
> > >>>>>>>>>>>> caches (considered beta cache nodes). This is helpful in trying out new DS
> > >>>>>>>>>>>> configs before rolling them out to production.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or resource
> > >>>>>>>>>>>> reservation on the delivery service. Providers of content to the CDN can
> > >>>>>>>>>>>> receive tiered service by either sharing caches or using dedicated caches.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> —Eric
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <
> > >>>>>>>>>>>> mtorluemke@apache.org> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Today, there isn't an actual link from Cache Group to DS -- I
> > >>>>>>>>>>>> think what
> > >>>>>>>>>>>> you are referring to is all done in the app layer to make a
> > >>>>>>>>>>>> nice summary in
> > >>>>>>>>>>>> the UI to allow operators to very quickly assign all caches in
> > >>>>>>>>>>>> a cache
> > >>>>>>>>>>>> group.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Do you use that "feature" to assign caches from some cache
> > >>>>>>>>>>>> groups, and not
> > >>>>>>>>>>>> others?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
> > >>>>>>>>>>>> efriedri@cisco.com> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> —Eric
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
> > >>>>>>>>>>>>> mtorluemke@apache.org> wrote:
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > Seeking community opinion on this.
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > Should we drop the server -> delivery service assignments
> > >>>>>>>>>>>>> completely?
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > When the project began, selectively assigning caches was the
> > >>>>>>>>>>>>> only way to selectively route traffic within a cache group -- let's say you
> > >>>>>>>>>>>>> have a DNS-routed service with a relatively large library size, you might
> > >>>>>>>>>>>>> not want assign all caches in a cache group to that service for fear you
> > >>>>>>>>>>>>> would 'pollute' those caches. Or, if you were turning up a 'test' DS of
> > >>>>>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
> > >>>>>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
> > >>>>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
> > >>>>>>>>>>>>> services, so the same functionality can essentially be reached with
> > >>>>>>>>>>>>> assigning all caches, and setting the config params properly.
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > The motivation for this is to reduce the size & complexity
> > >>>>>>>>>>>>> of the Traffic Router's config (aka CRConfig), as this config scales mostly
> > >>>>>>>>>>>>> linearly with the number of caches and the number of delivery services in
> > >>>>>>>>>>>>> your CDN.
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > Another option might be to not assign caches as the default,
> > >>>>>>>>>>>>> and individual cache assignments is available, but used only when necessary.
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > Other thoughts? Clarifications needed?
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > Thanks,
> > >>>>>>>>>>>>> > Mark
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >
> >
Re: [EXTERNAL] Re: Drop Server -> Delivery Service assignments?
Posted by "Schmidt, Andrew (Contractor)" <An...@comcast.com>.
It sounds to me like assigning Delivery Services to cache groups would solve all of these problems. We could create special cache groups for testing if necessary or even cache groups with just a one cache in it if needed. When new caches get added or removed from a cache group then no changes would need to made to DS config. We could also do a lot in the TP UI for self-service. We could have a default option in the UI which assigns all cache groups and then we could change that in the future if we want to change the model. For instance, I could imagine charging customers per the number of cache groups they want, which would then tie the cache group assignments to roles and capabilities. At worse, the self-service user would need to select from their authorized cache groups when creating a DS. The cache group assignments could be a part of the delivery service configuration, which would then allow us to completely separate the DS configurations from everything else in the CRConfig.
Andy
From: Nir Sopher <ni...@qwilt.com>
Reply-To: "users@trafficcontrol.apache.org" <us...@trafficcontrol.apache.org>
Date: Tuesday, January 22, 2019 at 11:55 AM
To: "users@trafficcontrol.apache.org" <us...@trafficcontrol.apache.org>
Subject: [EXTERNAL] Re: Drop Server -> Delivery Service assignments?
Hi Rob,
I strongly agree with the first part of your suggestion. I think that giving the user only fine grain control on the ds/servers mapping, and holding such a table is in the db, is not scalable nor maintable. I would suggest to use some kind of rule table, allowing the assignment of ds to all caches, specific cache groups, or specific cache.
I see less value in removing the dss mapping from the cr config. It indeed makes the cr config smaller, but this is not the creteria I believe we should discuss arround "breaking up the huge CRConfig". From self service point of view, we need to break the cr config in a way that reduces the dependency between different elements cfg chage, and specifically delivery services - so you can easily change the configuration and footprint for a single ds. Pushing the dss logic into the router might not fit with this effort.
Anyhow, if we decide to go in this direction in the cr-config level as well, I think it would be important to preserve the ability to have specific dss mapping in the cr config. Maybe build the router cr-config digestion in layers - where the higher layer have some logic for creating the dss, and the lower layer work with dss mapping as an input.
Nir
On Wed, Jan 2, 2019, 20:08 Robert O Butts <ro...@apache.org> wrote:
Reviving this, because I noticed a few things, when working on the DS Snapshots PR:
(1) I noticed the DeliveryService-Servers query is _by far_ the slowest query we have, building the CRConfig. For a given machine and a given large CDN, selecting all DS data takes about ~10ms; the DSS query takes ~200ms. This is to trivially select the data, with indexes. If you ever need a nontrivial query with nontrivial joins, it quickly becomes seconds.
(2) Writing code, I accidentally generated a CRConfig without DeliveryService-Servers. It was ~0.5mb (uncompressed). Compared to ~10mb for the same CDN with DSSes.
These sizes and query times are because the DSSes are a cross product of DS and Servers.
I think we need to do this, and not only in the interface, but implement it such that we (1) don't actually store the DSS mappings, for DSes and Servers which are "default-assigned", and (2) don't put them in the CRConfig. In terms of performance, I strongly suspect it will be faster for the Router to compute them, than the time it takes it to receive all that data over the network.
We keep talking about "breaking up the huge CRConfig" - it looks to me like removing DSSes will make it so small we don't _need_ to break it up anytime soon.
To @mtorluemke 's original point, doing this will significantly improve scalability, completely aside from the Self Service and interface benefits.
It sounds like there was consensus here. We just need to do it. I'm suggesting, with the evidence above, this really ought to be a higher priority. The evidence suggests fixing this at the data level will significantly improve the scalability of Traffic Control.
On 2017/05/24 19:43:53, Nir Sopher <ni...@qwilt.com>> wrote:
> Hi Jeremy,
>
> I’m ok with automatically assigning all edges to a new DS. The aspect of
> "self-service" can be dealt with in a different manner.
> However, how would you solve the problem when a new server is created? How
> would you know which DS it should be automatically be assigned?
> The model has no configuration marking the DS as one that should be
> deployed on all servers.
> My suggestion was to model it with "undef" value in the DS/Servers table.
> DS | Server
> ------+--------
> ds1 | server1 => Allow the CDN owner to deploy the ds on a specific cache
> ------+--------
> ds2 | *undef *=> Allow the CDN owner to deploy the ds on all caches
>
> This basically means that every place reading the DS/Server table, will
> have to use a wrapper, unfolding a the "server=undef" raw, to a raw per
> relevant server in the CDN.
> I'm not sure how much effort it requires.
>
> Nir
>
> On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <mi...@gmail.com>>
> wrote:
>
> > Nir,
> >
> > Thinking about self-service, do you envision the "DS owner" creates the DS
> > and the "CDN owner" assigns caches to the DS? This seems like an
> > unnecessary step to me and one that negates the idea of "self service".
> >
> > As the "DS owner", I want to create a DS and by default, employ ALL caches
> > in my cdn. To be honest, in 99% of cases I really don't want to be bothered
> > with assigning caches. I want the full power of the CDN behind my DS.
> >
> > So, I'm suggesting that when a new DS is created thru the API, ALL edge
> > caches in the cdn are assigned to that DS.....and then, the CDN owner (or
> > maybe DS owner) can remove caches from that DS if they need to.
> >
> > Jeremy
> >
> > On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com>> wrote:
> >
> >> @Jeremy
> >>
> >> The issue I was trying to point at, is that the "DS-Server" table is the
> >> one to provide the separation between the DS owner domain and the CDN owner
> >> domain (tenancy wise).
> >> As I see it, when considering multi tenancy, the DS has its owning tenant
> >> which is in charge of the specific record in the DS table, and specifically
> >> on adding the DS. The CDN is usually owned by another tenant that controls
> >> the deployment - currently by adding the ds/servers record to the DS/Server
> >> table.
> >>
> >> If the DS/Server table is dismissed or automatically written as a result
> >> of a ds addition, it removes the "server assignment" step and the CDN owner
> >> loses the control over the deployment decision.
> >>
> >> Before I open a discussion about how to solve this new tenancy issue, I
> >> would like to verify it is a real issue and there is not something I'm
> >> missing there.
> >> What do you think?
> >>
> >> Nir
> >>
> >> On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com>> wrote:
> >>
> >>> @Nir - maybe your proposal is something to consider more long-term but
> >>> for the short-term i'm wondering if one of the following can / should be
> >>> implemented:
> >>>
> >>> A) creating a DS thru the API (POST /api/*/deliveryservices) can have a
> >>> default effect of automatically assigning all edges to the new DS (via the deliveryservice_server
> >>> table)
> >>>
> >>> or
> >>>
> >>> B) the deliveryservice_server table simple goes away and it becomes
> >>> implied that a DS employs all edge caches
> >>>
> >>> "A" would probably be the easiest to be honest.
> >>>
> >>> jeremy
> >>>
> >>> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com>> wrote:
> >>>
> >>>> +1 for having a configuration allowing to apply the DS to the entire
> >>>> CDN.
> >>>>
> >>>> Note that adding a "deploy by default" bool field to the DS itself may
> >>>> cause tenancy issues, as I believe this field should be in the control of
> >>>> the "CDN owner" tenant, and not of the "DS owner".
> >>>>
> >>>> What about keeping the ds_server table, and allow "undef" for the
> >>>> server value - meaning "all servers in the CDN"?
> >>>> In the future the table may grow and have additional fields to
> >>>> white-list servers by.
> >>>> For example:
> >>>>
> >>>> DS | Server | cache-group | profile
> >>>> ------+--------+-------------+--------
> >>>> ds1 | s1 | null | null => ds to be applied on s1
> >>>> ------+--------+-------------+--------
> >>>> ds2 | null | null | null => ds to be applied on all
> >>>> servers (in ds CDN)
> >>>> ------+--------+-------------+--------
> >>>> ds3 | null | group1 | null => ds to be applied on all
> >>>> servers in cache group "group1"
> >>>> ------+--------+-------------+--------
> >>>> ds4 | null | group1 | ATS-6.2 => ds to be applied on all
> >>>> servers in cache group "group1", with profile "ATS-6.2"
> >>>>
> >>>>
> >>>> During CR config snapshot and configuration pull (or in advance), the
> >>>> required calculation would be done, to provide the list of relevant DSs per
> >>>> server.
> >>>>
> >>>> Also note that moving in this direction is backward compatible as well
> >>>> as supports other non trivial extensions one may want in the future:
> >>>> Patterns (wildcards/regexs) support, as well as black list, using a "policy
> >>>> table" mechanism.
> >>>>
> >>>> In the below example "ds5" is to be deployed on all caches, except for
> >>>> caches with "ATS-5" profile
> >>>>
> >>>> DS | Server | cache-group | profile | deploy-decision
> >>>> ------+--------+-------------+---------+----------------
> >>>> ds5 | null | null | ATS-5 | false
> >>>> ------+--------+-------------+---------+----------------
> >>>> ds5 | null | null | null | true
> >>>> ------+--------+-------------+---------+----------------
> >>>>
> >>>>
> >>>> Nir
> >>>>
> >>>> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <mi...@gmail.com>
> >>>> > wrote:
> >>>>
> >>>>> @dave - i'm not sure if you're suggesting:
> >>>>>
> >>>>> A) on DS create, by default create an entry in the
> >>>>> deliveryservice_server table for each cdn edge cache. so if there are 200
> >>>>> cdn edge caches, you end up with 200 entries in the deliveryservice_server
> >>>>> table for that DS.
> >>>>> B) completely kill the deliveryservice_server table and it becomes
> >>>>> implied that a DS uses ALL cdn edge caches (the way mids work today.
> >>>>> mids don't have to be explicitly assigned to a DS)
> >>>>>
> >>>>> "A" would allow you to control the scope of caches by using initial
> >>>>> dispersion OR control the scope of caches by simply removing the ones
> >>>>> you don't want to receive traffic.
> >>>>> "B" would only allow you to control the scope of caches by using
> >>>>> initial dispersion (or so i think).
> >>>>>
> >>>>> jeremy
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>>
> >>>>> wrote:
> >>>>>
> >>>>>> +1, I think assigning DSs to all caches by default is fine. Then we
> >>>>>> can use initial dispersion to make sure traffic is spread between an
> >>>>>> appropriate number of caches.
> >>>>>>
> >>>>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <
> >>>>>> mitchell852@gmail.com<ma...@gmail.com>> wrote:
> >>>>>>
> >>>>>>> Bringing up an old email regarding DS to cache assignments.
> >>>>>>>
> >>>>>>> It sounds like we want to preserve DS/cache assignments (the
> >>>>>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
> >>>>>>> to a DS if you want. However, what if we make the assignment of ALL cdn
> >>>>>>> edge caches to a new DS the default?
> >>>>>>>
> >>>>>>> So when you create a DS for the Foo cdn thru the API, your new DS is
> >>>>>>> automagically assigned all the edge caches in the Foo cdn...
> >>>>>>>
> >>>>>>> (at that point you could always remove caches from a DS)
> >>>>>>>
> >>>>>>> ...or the create DS API could have a boolean flag added to it. for
> >>>>>>> example, assignAllCaches: true|false. if you pass in true, all caches are
> >>>>>>> assigned to the DS, if false, none are.
> >>>>>>>
> >>>>>>> thoughts? I'm trying to think of self-service and i'm guessing most
> >>>>>>> people will not want to assign caches to a DS...
> >>>>>>>
> >>>>>>> jeremy
> >>>>>>>
> >>>>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
> >>>>>>> mitchell852@gmail.com<ma...@gmail.com>> wrote:
> >>>>>>>
> >>>>>>>> gotcha. makes sense.
> >>>>>>>>
> >>>>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <
> >>>>>>>> david.neuman64@gmail.com<ma...@gmail.com>> wrote:
> >>>>>>>>
> >>>>>>>>> I believe Eric had a use case where he wanted to assign certain
> >>>>>>>>> delivery services to certain caches.
> >>>>>>>>> If we just use initial dispersion and max DNS answers then we
> >>>>>>>>> don't have the ability to control which servers are used, just how many.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
> >>>>>>>>> mitchell852@gmail.com<ma...@gmail.com>> wrote:
> >>>>>>>>>
> >>>>>>>>>> So what I heard was that potentially we move in this direction:
> >>>>>>>>>>
> >>>>>>>>>> 1. No need to explicitly assign edge caches to a delivery
> >>>>>>>>>> service. a delivery service (by default) would employ ALL edge caches of
> >>>>>>>>>> the cdn in which the ds belongs to.
> >>>>>>>>>> 2. Ability to override this default behavior and explicitly
> >>>>>>>>>> assign specific edge caches to a delivery service (this is what we do today)
> >>>>>>>>>>
> >>>>>>>>>> But earlier Mark said "so the same functionality [as #2] can
> >>>>>>>>>> essentially be reached with assigning all caches, and setting the config
> >>>>>>>>>> params properly [initial dispersion and max dns answers]"
> >>>>>>>>>>
> >>>>>>>>>> so why not ditch #2 in favor of using initial dispersion and max
> >>>>>>>>>> dns answers to control cache assignments?
> >>>>>>>>>>
> >>>>>>>>>> it would be nice imo to get rid of the ds_server table. i'm
> >>>>>>>>>> always in favor of reducing complexity... :)
> >>>>>>>>>>
> >>>>>>>>>> jeremy
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
> >>>>>>>>>> efriedri@cisco.com<ma...@cisco.com>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Yes - I think its a reasonable default to have a delivery
> >>>>>>>>>>> service assigned to all caches as long as we still keep the ability to
> >>>>>>>>>>> assign some DS’s to just a subset of caches.
> >>>>>>>>>>>
> >>>>>>>>>>> —Eric
> >>>>>>>>>>>
> >>>>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <
> >>>>>>>>>>> mtorluemke@apache.org<ma...@apache.org>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Good feedback, thanks. I think having _zero_ caches assigned to
> >>>>>>>>>>> a delivery service could be added fairly easily, and could function as a
> >>>>>>>>>>> good default to the assumption of _all_ caches being assigned. Would you
> >>>>>>>>>>> agree with that?
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Mark
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
> >>>>>>>>>>> efriedri@cisco.com<ma...@cisco.com>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Yeah,
> >>>>>>>>>>>> We have several some use cases where where cache or CG level
> >>>>>>>>>>>> allocation would be helpful.
> >>>>>>>>>>>>
> >>>>>>>>>>>> 1) Assigning new or trial delivery services to a subset of
> >>>>>>>>>>>> caches (considered beta cache nodes). This is helpful in trying out new DS
> >>>>>>>>>>>> configs before rolling them out to production.
> >>>>>>>>>>>>
> >>>>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or resource
> >>>>>>>>>>>> reservation on the delivery service. Providers of content to the CDN can
> >>>>>>>>>>>> receive tiered service by either sharing caches or using dedicated caches.
> >>>>>>>>>>>>
> >>>>>>>>>>>> —Eric
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <
> >>>>>>>>>>>> mtorluemke@apache.org<ma...@apache.org>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Today, there isn't an actual link from Cache Group to DS -- I
> >>>>>>>>>>>> think what
> >>>>>>>>>>>> you are referring to is all done in the app layer to make a
> >>>>>>>>>>>> nice summary in
> >>>>>>>>>>>> the UI to allow operators to very quickly assign all caches in
> >>>>>>>>>>>> a cache
> >>>>>>>>>>>> group.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Do you use that "feature" to assign caches from some cache
> >>>>>>>>>>>> groups, and not
> >>>>>>>>>>>> others?
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
> >>>>>>>>>>>> efriedri@cisco.com<ma...@cisco.com>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> —Eric
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
> >>>>>>>>>>>>> mtorluemke@apache.org<ma...@apache.org>> wrote:
> >>>>>>>>>>>>> >
> >>>>>>>>>>>>> > Seeking community opinion on this.
> >>>>>>>>>>>>> >
> >>>>>>>>>>>>> > Should we drop the server -> delivery service assignments
> >>>>>>>>>>>>> completely?
> >>>>>>>>>>>>> >
> >>>>>>>>>>>>> > When the project began, selectively assigning caches was the
> >>>>>>>>>>>>> only way to selectively route traffic within a cache group -- let's say you
> >>>>>>>>>>>>> have a DNS-routed service with a relatively large library size, you might
> >>>>>>>>>>>>> not want assign all caches in a cache group to that service for fear you
> >>>>>>>>>>>>> would 'pollute' those caches. Or, if you were turning up a 'test' DS of
> >>>>>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
> >>>>>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
> >>>>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
> >>>>>>>>>>>>> services, so the same functionality can essentially be reached with
> >>>>>>>>>>>>> assigning all caches, and setting the config params properly.
> >>>>>>>>>>>>> >
> >>>>>>>>>>>>> > The motivation for this is to reduce the size & complexity
> >>>>>>>>>>>>> of the Traffic Router's config (aka CRConfig), as this config scales mostly
> >>>>>>>>>>>>> linearly with the number of caches and the number of delivery services in
> >>>>>>>>>>>>> your CDN.
> >>>>>>>>>>>>> >
> >>>>>>>>>>>>> > Another option might be to not assign caches as the default,
> >>>>>>>>>>>>> and individual cache assignments is available, but used only when necessary.
> >>>>>>>>>>>>> >
> >>>>>>>>>>>>> > Other thoughts? Clarifications needed?
> >>>>>>>>>>>>> >
> >>>>>>>>>>>>> > Thanks,
> >>>>>>>>>>>>> > Mark
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >
>
Re: Drop Server -> Delivery Service assignments?
Posted by Nir Sopher <ni...@qwilt.com>.
Hi Rob,
I strongly agree with the first part of your suggestion. I think that
giving the user only fine grain control on the ds/servers mapping, and
holding such a table is in the db, is not scalable nor maintable. I would
suggest to use some kind of rule table, allowing the assignment of ds to
all caches, specific cache groups, or specific cache.
I see less value in removing the dss mapping from the cr config. It indeed
makes the cr config smaller, but this is not the creteria I believe we
should discuss arround "breaking up the huge CRConfig". From self service
point of view, we need to break the cr config in a way that reduces the
dependency between different elements cfg chage, and specifically delivery
services - so you can easily change the configuration and footprint for a
single ds. Pushing the dss logic into the router might not fit with this
effort.
Anyhow, if we decide to go in this direction in the cr-config level as
well, I think it would be important to preserve the ability to have
specific dss mapping in the cr config. Maybe build the router cr-config
digestion in layers - where the higher layer have some logic for
creating the dss, and the lower layer work with dss mapping as an input.
Nir
On Wed, Jan 2, 2019, 20:08 Robert O Butts <rob@apache.org wrote:
> Reviving this, because I noticed a few things, when working on the DS
> Snapshots PR:
>
> (1) I noticed the DeliveryService-Servers query is _by far_ the slowest
> query we have, building the CRConfig. For a given machine and a given large
> CDN, selecting all DS data takes about ~10ms; the DSS query takes ~200ms.
> This is to trivially select the data, with indexes. If you ever need a
> nontrivial query with nontrivial joins, it quickly becomes seconds.
>
> (2) Writing code, I accidentally generated a CRConfig without
> DeliveryService-Servers. It was ~0.5mb (uncompressed). Compared to ~10mb
> for the same CDN with DSSes.
>
> These sizes and query times are because the DSSes are a cross product of
> DS and Servers.
>
> I think we need to do this, and not only in the interface, but implement
> it such that we (1) don't actually store the DSS mappings, for DSes and
> Servers which are "default-assigned", and (2) don't put them in the
> CRConfig. In terms of performance, I strongly suspect it will be faster for
> the Router to compute them, than the time it takes it to receive all that
> data over the network.
>
> We keep talking about "breaking up the huge CRConfig" - it looks to me
> like removing DSSes will make it so small we don't _need_ to break it up
> anytime soon.
>
> To @mtorluemke 's original point, doing this will significantly improve
> scalability, completely aside from the Self Service and interface benefits.
>
> It sounds like there was consensus here. We just need to do it. I'm
> suggesting, with the evidence above, this really ought to be a higher
> priority. The evidence suggests fixing this at the data level will
> significantly improve the scalability of Traffic Control.
>
> On 2017/05/24 19:43:53, Nir Sopher <ni...@qwilt.com> wrote:
> > Hi Jeremy,
> >
> > I’m ok with automatically assigning all edges to a new DS. The aspect of
> > "self-service" can be dealt with in a different manner.
> > However, how would you solve the problem when a new server is created?
> How
> > would you know which DS it should be automatically be assigned?
> > The model has no configuration marking the DS as one that should be
> > deployed on all servers.
> > My suggestion was to model it with "undef" value in the DS/Servers table.
> > DS | Server
> > ------+--------
> > ds1 | server1 => Allow the CDN owner to deploy the ds on a specific
> cache
> > ------+--------
> > ds2 | *undef *=> Allow the CDN owner to deploy the ds on all caches
> >
> > This basically means that every place reading the DS/Server table, will
> > have to use a wrapper, unfolding a the "server=undef" raw, to a raw per
> > relevant server in the CDN.
> > I'm not sure how much effort it requires.
> >
> > Nir
> >
> > On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <mi...@gmail.com>
> > wrote:
> >
> > > Nir,
> > >
> > > Thinking about self-service, do you envision the "DS owner" creates
> the DS
> > > and the "CDN owner" assigns caches to the DS? This seems like an
> > > unnecessary step to me and one that negates the idea of "self service".
> > >
> > > As the "DS owner", I want to create a DS and by default, employ ALL
> caches
> > > in my cdn. To be honest, in 99% of cases I really don't want to be
> bothered
> > > with assigning caches. I want the full power of the CDN behind my DS.
> > >
> > > So, I'm suggesting that when a new DS is created thru the API, ALL edge
> > > caches in the cdn are assigned to that DS.....and then, the CDN owner
> (or
> > > maybe DS owner) can remove caches from that DS if they need to.
> > >
> > > Jeremy
> > >
> > > On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com> wrote:
> > >
> > >> @Jeremy
> > >>
> > >> The issue I was trying to point at, is that the "DS-Server" table is
> the
> > >> one to provide the separation between the DS owner domain and the CDN
> owner
> > >> domain (tenancy wise).
> > >> As I see it, when considering multi tenancy, the DS has its owning
> tenant
> > >> which is in charge of the specific record in the DS table, and
> specifically
> > >> on adding the DS. The CDN is usually owned by another tenant that
> controls
> > >> the deployment - currently by adding the ds/servers record to the
> DS/Server
> > >> table.
> > >>
> > >> If the DS/Server table is dismissed or automatically written as a
> result
> > >> of a ds addition, it removes the "server assignment" step and the CDN
> owner
> > >> loses the control over the deployment decision.
> > >>
> > >> Before I open a discussion about how to solve this new tenancy issue,
> I
> > >> would like to verify it is a real issue and there is not something I'm
> > >> missing there.
> > >> What do you think?
> > >>
> > >> Nir
> > >>
> > >> On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com>
> wrote:
> > >>
> > >>> @Nir - maybe your proposal is something to consider more long-term
> but
> > >>> for the short-term i'm wondering if one of the following can /
> should be
> > >>> implemented:
> > >>>
> > >>> A) creating a DS thru the API (POST /api/*/deliveryservices) can
> have a
> > >>> default effect of automatically assigning all edges to the new DS
> (via the deliveryservice_server
> > >>> table)
> > >>>
> > >>> or
> > >>>
> > >>> B) the deliveryservice_server table simple goes away and it becomes
> > >>> implied that a DS employs all edge caches
> > >>>
> > >>> "A" would probably be the easiest to be honest.
> > >>>
> > >>> jeremy
> > >>>
> > >>> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com> wrote:
> > >>>
> > >>>> +1 for having a configuration allowing to apply the DS to the entire
> > >>>> CDN.
> > >>>>
> > >>>> Note that adding a "deploy by default" bool field to the DS itself
> may
> > >>>> cause tenancy issues, as I believe this field should be in the
> control of
> > >>>> the "CDN owner" tenant, and not of the "DS owner".
> > >>>>
> > >>>> What about keeping the ds_server table, and allow "undef" for the
> > >>>> server value - meaning "all servers in the CDN"?
> > >>>> In the future the table may grow and have additional fields to
> > >>>> white-list servers by.
> > >>>> For example:
> > >>>>
> > >>>> DS | Server | cache-group | profile
> > >>>> ------+--------+-------------+--------
> > >>>> ds1 | s1 | null | null => ds to be applied on
> s1
> > >>>> ------+--------+-------------+--------
> > >>>> ds2 | null | null | null => ds to be applied on
> all
> > >>>> servers (in ds CDN)
> > >>>> ------+--------+-------------+--------
> > >>>> ds3 | null | group1 | null => ds to be applied on
> all
> > >>>> servers in cache group "group1"
> > >>>> ------+--------+-------------+--------
> > >>>> ds4 | null | group1 | ATS-6.2 => ds to be applied on
> all
> > >>>> servers in cache group "group1", with profile "ATS-6.2"
> > >>>>
> > >>>>
> > >>>> During CR config snapshot and configuration pull (or in advance),
> the
> > >>>> required calculation would be done, to provide the list of relevant
> DSs per
> > >>>> server.
> > >>>>
> > >>>> Also note that moving in this direction is backward compatible as
> well
> > >>>> as supports other non trivial extensions one may want in the future:
> > >>>> Patterns (wildcards/regexs) support, as well as black list, using a
> "policy
> > >>>> table" mechanism.
> > >>>>
> > >>>> In the below example "ds5" is to be deployed on all caches, except
> for
> > >>>> caches with "ATS-5" profile
> > >>>>
> > >>>> DS | Server | cache-group | profile | deploy-decision
> > >>>> ------+--------+-------------+---------+----------------
> > >>>> ds5 | null | null | ATS-5 | false
> > >>>> ------+--------+-------------+---------+----------------
> > >>>> ds5 | null | null | null | true
> > >>>> ------+--------+-------------+---------+----------------
> > >>>>
> > >>>>
> > >>>> Nir
> > >>>>
> > >>>> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <
> mitchell852@gmail.com
> > >>>> > wrote:
> > >>>>
> > >>>>> @dave - i'm not sure if you're suggesting:
> > >>>>>
> > >>>>> A) on DS create, by default create an entry in the
> > >>>>> deliveryservice_server table for each cdn edge cache. so if there
> are 200
> > >>>>> cdn edge caches, you end up with 200 entries in the
> deliveryservice_server
> > >>>>> table for that DS.
> > >>>>> B) completely kill the deliveryservice_server table and it becomes
> > >>>>> implied that a DS uses ALL cdn edge caches (the way mids work
> today.
> > >>>>> mids don't have to be explicitly assigned to a DS)
> > >>>>>
> > >>>>> "A" would allow you to control the scope of caches by using initial
> > >>>>> dispersion OR control the scope of caches by simply removing the
> ones
> > >>>>> you don't want to receive traffic.
> > >>>>> "B" would only allow you to control the scope of caches by using
> > >>>>> initial dispersion (or so i think).
> > >>>>>
> > >>>>> jeremy
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> +1, I think assigning DSs to all caches by default is fine. Then
> we
> > >>>>>> can use initial dispersion to make sure traffic is spread between
> an
> > >>>>>> appropriate number of caches.
> > >>>>>>
> > >>>>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <
> > >>>>>> mitchell852@gmail.com> wrote:
> > >>>>>>
> > >>>>>>> Bringing up an old email regarding DS to cache assignments.
> > >>>>>>>
> > >>>>>>> It sounds like we want to preserve DS/cache assignments (the
> > >>>>>>> deliveryservice_server table) so you can assign a subset of cdn
> edge caches
> > >>>>>>> to a DS if you want. However, what if we make the assignment of
> ALL cdn
> > >>>>>>> edge caches to a new DS the default?
> > >>>>>>>
> > >>>>>>> So when you create a DS for the Foo cdn thru the API, your new
> DS is
> > >>>>>>> automagically assigned all the edge caches in the Foo cdn...
> > >>>>>>>
> > >>>>>>> (at that point you could always remove caches from a DS)
> > >>>>>>>
> > >>>>>>> ...or the create DS API could have a boolean flag added to it.
> for
> > >>>>>>> example, assignAllCaches: true|false. if you pass in true, all
> caches are
> > >>>>>>> assigned to the DS, if false, none are.
> > >>>>>>>
> > >>>>>>> thoughts? I'm trying to think of self-service and i'm guessing
> most
> > >>>>>>> people will not want to assign caches to a DS...
> > >>>>>>>
> > >>>>>>> jeremy
> > >>>>>>>
> > >>>>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
> > >>>>>>> mitchell852@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>>> gotcha. makes sense.
> > >>>>>>>>
> > >>>>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <
> > >>>>>>>> david.neuman64@gmail.com> wrote:
> > >>>>>>>>
> > >>>>>>>>> I believe Eric had a use case where he wanted to assign certain
> > >>>>>>>>> delivery services to certain caches.
> > >>>>>>>>> If we just use initial dispersion and max DNS answers then we
> > >>>>>>>>> don't have the ability to control which servers are used, just
> how many.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
> > >>>>>>>>> mitchell852@gmail.com> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> So what I heard was that potentially we move in this
> direction:
> > >>>>>>>>>>
> > >>>>>>>>>> 1. No need to explicitly assign edge caches to a delivery
> > >>>>>>>>>> service. a delivery service (by default) would employ ALL
> edge caches of
> > >>>>>>>>>> the cdn in which the ds belongs to.
> > >>>>>>>>>> 2. Ability to override this default behavior and explicitly
> > >>>>>>>>>> assign specific edge caches to a delivery service (this is
> what we do today)
> > >>>>>>>>>>
> > >>>>>>>>>> But earlier Mark said "so the same functionality [as #2] can
> > >>>>>>>>>> essentially be reached with assigning all caches, and setting
> the config
> > >>>>>>>>>> params properly [initial dispersion and max dns answers]"
> > >>>>>>>>>>
> > >>>>>>>>>> so why not ditch #2 in favor of using initial dispersion and
> max
> > >>>>>>>>>> dns answers to control cache assignments?
> > >>>>>>>>>>
> > >>>>>>>>>> it would be nice imo to get rid of the ds_server table. i'm
> > >>>>>>>>>> always in favor of reducing complexity... :)
> > >>>>>>>>>>
> > >>>>>>>>>> jeremy
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
> > >>>>>>>>>> efriedri@cisco.com> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Yes - I think its a reasonable default to have a delivery
> > >>>>>>>>>>> service assigned to all caches as long as we still keep the
> ability to
> > >>>>>>>>>>> assign some DS’s to just a subset of caches.
> > >>>>>>>>>>>
> > >>>>>>>>>>> —Eric
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <
> > >>>>>>>>>>> mtorluemke@apache.org> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Good feedback, thanks. I think having _zero_ caches assigned
> to
> > >>>>>>>>>>> a delivery service could be added fairly easily, and could
> function as a
> > >>>>>>>>>>> good default to the assumption of _all_ caches being
> assigned. Would you
> > >>>>>>>>>>> agree with that?
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks,
> > >>>>>>>>>>> Mark
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
> > >>>>>>>>>>> efriedri@cisco.com> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Yeah,
> > >>>>>>>>>>>> We have several some use cases where where cache or CG
> level
> > >>>>>>>>>>>> allocation would be helpful.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> 1) Assigning new or trial delivery services to a subset of
> > >>>>>>>>>>>> caches (considered beta cache nodes). This is helpful in
> trying out new DS
> > >>>>>>>>>>>> configs before rolling them out to production.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or
> resource
> > >>>>>>>>>>>> reservation on the delivery service. Providers of content
> to the CDN can
> > >>>>>>>>>>>> receive tiered service by either sharing caches or using
> dedicated caches.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> —Eric
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <
> > >>>>>>>>>>>> mtorluemke@apache.org> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Today, there isn't an actual link from Cache Group to DS --
> I
> > >>>>>>>>>>>> think what
> > >>>>>>>>>>>> you are referring to is all done in the app layer to make a
> > >>>>>>>>>>>> nice summary in
> > >>>>>>>>>>>> the UI to allow operators to very quickly assign all caches
> in
> > >>>>>>>>>>>> a cache
> > >>>>>>>>>>>> group.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Do you use that "feature" to assign caches from some cache
> > >>>>>>>>>>>> groups, and not
> > >>>>>>>>>>>> others?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
> > >>>>>>>>>>>> efriedri@cisco.com> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> —Eric
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
> > >>>>>>>>>>>>> mtorluemke@apache.org> wrote:
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > Seeking community opinion on this.
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > Should we drop the server -> delivery service assignments
> > >>>>>>>>>>>>> completely?
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > When the project began, selectively assigning caches was
> the
> > >>>>>>>>>>>>> only way to selectively route traffic within a cache group
> -- let's say you
> > >>>>>>>>>>>>> have a DNS-routed service with a relatively large library
> size, you might
> > >>>>>>>>>>>>> not want assign all caches in a cache group to that
> service for fear you
> > >>>>>>>>>>>>> would 'pollute' those caches. Or, if you were turning up a
> 'test' DS of
> > >>>>>>>>>>>>> which you wanted to limit exposure on, you might apply the
> same logic. In
> > >>>>>>>>>>>>> recent releases, Traffic Control supports "initial
> dispersion" for _both_
> > >>>>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for
> DNS-routed
> > >>>>>>>>>>>>> services, so the same functionality can essentially be
> reached with
> > >>>>>>>>>>>>> assigning all caches, and setting the config params
> properly.
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > The motivation for this is to reduce the size &
> complexity
> > >>>>>>>>>>>>> of the Traffic Router's config (aka CRConfig), as this
> config scales mostly
> > >>>>>>>>>>>>> linearly with the number of caches and the number of
> delivery services in
> > >>>>>>>>>>>>> your CDN.
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > Another option might be to not assign caches as the
> default,
> > >>>>>>>>>>>>> and individual cache assignments is available, but used
> only when necessary.
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > Other thoughts? Clarifications needed?
> > >>>>>>>>>>>>> >
> > >>>>>>>>>>>>> > Thanks,
> > >>>>>>>>>>>>> > Mark
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >
> >
>
Re: Drop Server -> Delivery Service assignments?
Posted by Robert O Butts <ro...@apache.org>.
Reviving this, because I noticed a few things, when working on the DS Snapshots PR:
(1) I noticed the DeliveryService-Servers query is _by far_ the slowest query we have, building the CRConfig. For a given machine and a given large CDN, selecting all DS data takes about ~10ms; the DSS query takes ~200ms. This is to trivially select the data, with indexes. If you ever need a nontrivial query with nontrivial joins, it quickly becomes seconds.
(2) Writing code, I accidentally generated a CRConfig without DeliveryService-Servers. It was ~0.5mb (uncompressed). Compared to ~10mb for the same CDN with DSSes.
These sizes and query times are because the DSSes are a cross product of DS and Servers.
I think we need to do this, and not only in the interface, but implement it such that we (1) don't actually store the DSS mappings, for DSes and Servers which are "default-assigned", and (2) don't put them in the CRConfig. In terms of performance, I strongly suspect it will be faster for the Router to compute them, than the time it takes it to receive all that data over the network.
We keep talking about "breaking up the huge CRConfig" - it looks to me like removing DSSes will make it so small we don't _need_ to break it up anytime soon.
To @mtorluemke 's original point, doing this will significantly improve scalability, completely aside from the Self Service and interface benefits.
It sounds like there was consensus here. We just need to do it. I'm suggesting, with the evidence above, this really ought to be a higher priority. The evidence suggests fixing this at the data level will significantly improve the scalability of Traffic Control.
On 2017/05/24 19:43:53, Nir Sopher <ni...@qwilt.com> wrote:
> Hi Jeremy,
>
> I’m ok with automatically assigning all edges to a new DS. The aspect of
> "self-service" can be dealt with in a different manner.
> However, how would you solve the problem when a new server is created? How
> would you know which DS it should be automatically be assigned?
> The model has no configuration marking the DS as one that should be
> deployed on all servers.
> My suggestion was to model it with "undef" value in the DS/Servers table.
> DS | Server
> ------+--------
> ds1 | server1 => Allow the CDN owner to deploy the ds on a specific cache
> ------+--------
> ds2 | *undef *=> Allow the CDN owner to deploy the ds on all caches
>
> This basically means that every place reading the DS/Server table, will
> have to use a wrapper, unfolding a the "server=undef" raw, to a raw per
> relevant server in the CDN.
> I'm not sure how much effort it requires.
>
> Nir
>
> On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
> > Nir,
> >
> > Thinking about self-service, do you envision the "DS owner" creates the DS
> > and the "CDN owner" assigns caches to the DS? This seems like an
> > unnecessary step to me and one that negates the idea of "self service".
> >
> > As the "DS owner", I want to create a DS and by default, employ ALL caches
> > in my cdn. To be honest, in 99% of cases I really don't want to be bothered
> > with assigning caches. I want the full power of the CDN behind my DS.
> >
> > So, I'm suggesting that when a new DS is created thru the API, ALL edge
> > caches in the cdn are assigned to that DS.....and then, the CDN owner (or
> > maybe DS owner) can remove caches from that DS if they need to.
> >
> > Jeremy
> >
> > On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com> wrote:
> >
> >> @Jeremy
> >>
> >> The issue I was trying to point at, is that the "DS-Server" table is the
> >> one to provide the separation between the DS owner domain and the CDN owner
> >> domain (tenancy wise).
> >> As I see it, when considering multi tenancy, the DS has its owning tenant
> >> which is in charge of the specific record in the DS table, and specifically
> >> on adding the DS. The CDN is usually owned by another tenant that controls
> >> the deployment - currently by adding the ds/servers record to the DS/Server
> >> table.
> >>
> >> If the DS/Server table is dismissed or automatically written as a result
> >> of a ds addition, it removes the "server assignment" step and the CDN owner
> >> loses the control over the deployment decision.
> >>
> >> Before I open a discussion about how to solve this new tenancy issue, I
> >> would like to verify it is a real issue and there is not something I'm
> >> missing there.
> >> What do you think?
> >>
> >> Nir
> >>
> >> On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com> wrote:
> >>
> >>> @Nir - maybe your proposal is something to consider more long-term but
> >>> for the short-term i'm wondering if one of the following can / should be
> >>> implemented:
> >>>
> >>> A) creating a DS thru the API (POST /api/*/deliveryservices) can have a
> >>> default effect of automatically assigning all edges to the new DS (via the deliveryservice_server
> >>> table)
> >>>
> >>> or
> >>>
> >>> B) the deliveryservice_server table simple goes away and it becomes
> >>> implied that a DS employs all edge caches
> >>>
> >>> "A" would probably be the easiest to be honest.
> >>>
> >>> jeremy
> >>>
> >>> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com> wrote:
> >>>
> >>>> +1 for having a configuration allowing to apply the DS to the entire
> >>>> CDN.
> >>>>
> >>>> Note that adding a "deploy by default" bool field to the DS itself may
> >>>> cause tenancy issues, as I believe this field should be in the control of
> >>>> the "CDN owner" tenant, and not of the "DS owner".
> >>>>
> >>>> What about keeping the ds_server table, and allow "undef" for the
> >>>> server value - meaning "all servers in the CDN"?
> >>>> In the future the table may grow and have additional fields to
> >>>> white-list servers by.
> >>>> For example:
> >>>>
> >>>> DS | Server | cache-group | profile
> >>>> ------+--------+-------------+--------
> >>>> ds1 | s1 | null | null => ds to be applied on s1
> >>>> ------+--------+-------------+--------
> >>>> ds2 | null | null | null => ds to be applied on all
> >>>> servers (in ds CDN)
> >>>> ------+--------+-------------+--------
> >>>> ds3 | null | group1 | null => ds to be applied on all
> >>>> servers in cache group "group1"
> >>>> ------+--------+-------------+--------
> >>>> ds4 | null | group1 | ATS-6.2 => ds to be applied on all
> >>>> servers in cache group "group1", with profile "ATS-6.2"
> >>>>
> >>>>
> >>>> During CR config snapshot and configuration pull (or in advance), the
> >>>> required calculation would be done, to provide the list of relevant DSs per
> >>>> server.
> >>>>
> >>>> Also note that moving in this direction is backward compatible as well
> >>>> as supports other non trivial extensions one may want in the future:
> >>>> Patterns (wildcards/regexs) support, as well as black list, using a "policy
> >>>> table" mechanism.
> >>>>
> >>>> In the below example "ds5" is to be deployed on all caches, except for
> >>>> caches with "ATS-5" profile
> >>>>
> >>>> DS | Server | cache-group | profile | deploy-decision
> >>>> ------+--------+-------------+---------+----------------
> >>>> ds5 | null | null | ATS-5 | false
> >>>> ------+--------+-------------+---------+----------------
> >>>> ds5 | null | null | null | true
> >>>> ------+--------+-------------+---------+----------------
> >>>>
> >>>>
> >>>> Nir
> >>>>
> >>>> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <mitchell852@gmail.com
> >>>> > wrote:
> >>>>
> >>>>> @dave - i'm not sure if you're suggesting:
> >>>>>
> >>>>> A) on DS create, by default create an entry in the
> >>>>> deliveryservice_server table for each cdn edge cache. so if there are 200
> >>>>> cdn edge caches, you end up with 200 entries in the deliveryservice_server
> >>>>> table for that DS.
> >>>>> B) completely kill the deliveryservice_server table and it becomes
> >>>>> implied that a DS uses ALL cdn edge caches (the way mids work today.
> >>>>> mids don't have to be explicitly assigned to a DS)
> >>>>>
> >>>>> "A" would allow you to control the scope of caches by using initial
> >>>>> dispersion OR control the scope of caches by simply removing the ones
> >>>>> you don't want to receive traffic.
> >>>>> "B" would only allow you to control the scope of caches by using
> >>>>> initial dispersion (or so i think).
> >>>>>
> >>>>> jeremy
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>
> >>>>> wrote:
> >>>>>
> >>>>>> +1, I think assigning DSs to all caches by default is fine. Then we
> >>>>>> can use initial dispersion to make sure traffic is spread between an
> >>>>>> appropriate number of caches.
> >>>>>>
> >>>>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <
> >>>>>> mitchell852@gmail.com> wrote:
> >>>>>>
> >>>>>>> Bringing up an old email regarding DS to cache assignments.
> >>>>>>>
> >>>>>>> It sounds like we want to preserve DS/cache assignments (the
> >>>>>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
> >>>>>>> to a DS if you want. However, what if we make the assignment of ALL cdn
> >>>>>>> edge caches to a new DS the default?
> >>>>>>>
> >>>>>>> So when you create a DS for the Foo cdn thru the API, your new DS is
> >>>>>>> automagically assigned all the edge caches in the Foo cdn...
> >>>>>>>
> >>>>>>> (at that point you could always remove caches from a DS)
> >>>>>>>
> >>>>>>> ...or the create DS API could have a boolean flag added to it. for
> >>>>>>> example, assignAllCaches: true|false. if you pass in true, all caches are
> >>>>>>> assigned to the DS, if false, none are.
> >>>>>>>
> >>>>>>> thoughts? I'm trying to think of self-service and i'm guessing most
> >>>>>>> people will not want to assign caches to a DS...
> >>>>>>>
> >>>>>>> jeremy
> >>>>>>>
> >>>>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
> >>>>>>> mitchell852@gmail.com> wrote:
> >>>>>>>
> >>>>>>>> gotcha. makes sense.
> >>>>>>>>
> >>>>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <
> >>>>>>>> david.neuman64@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>> I believe Eric had a use case where he wanted to assign certain
> >>>>>>>>> delivery services to certain caches.
> >>>>>>>>> If we just use initial dispersion and max DNS answers then we
> >>>>>>>>> don't have the ability to control which servers are used, just how many.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
> >>>>>>>>> mitchell852@gmail.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> So what I heard was that potentially we move in this direction:
> >>>>>>>>>>
> >>>>>>>>>> 1. No need to explicitly assign edge caches to a delivery
> >>>>>>>>>> service. a delivery service (by default) would employ ALL edge caches of
> >>>>>>>>>> the cdn in which the ds belongs to.
> >>>>>>>>>> 2. Ability to override this default behavior and explicitly
> >>>>>>>>>> assign specific edge caches to a delivery service (this is what we do today)
> >>>>>>>>>>
> >>>>>>>>>> But earlier Mark said "so the same functionality [as #2] can
> >>>>>>>>>> essentially be reached with assigning all caches, and setting the config
> >>>>>>>>>> params properly [initial dispersion and max dns answers]"
> >>>>>>>>>>
> >>>>>>>>>> so why not ditch #2 in favor of using initial dispersion and max
> >>>>>>>>>> dns answers to control cache assignments?
> >>>>>>>>>>
> >>>>>>>>>> it would be nice imo to get rid of the ds_server table. i'm
> >>>>>>>>>> always in favor of reducing complexity... :)
> >>>>>>>>>>
> >>>>>>>>>> jeremy
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
> >>>>>>>>>> efriedri@cisco.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Yes - I think its a reasonable default to have a delivery
> >>>>>>>>>>> service assigned to all caches as long as we still keep the ability to
> >>>>>>>>>>> assign some DS’s to just a subset of caches.
> >>>>>>>>>>>
> >>>>>>>>>>> —Eric
> >>>>>>>>>>>
> >>>>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <
> >>>>>>>>>>> mtorluemke@apache.org> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Good feedback, thanks. I think having _zero_ caches assigned to
> >>>>>>>>>>> a delivery service could be added fairly easily, and could function as a
> >>>>>>>>>>> good default to the assumption of _all_ caches being assigned. Would you
> >>>>>>>>>>> agree with that?
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Mark
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
> >>>>>>>>>>> efriedri@cisco.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Yeah,
> >>>>>>>>>>>> We have several some use cases where where cache or CG level
> >>>>>>>>>>>> allocation would be helpful.
> >>>>>>>>>>>>
> >>>>>>>>>>>> 1) Assigning new or trial delivery services to a subset of
> >>>>>>>>>>>> caches (considered beta cache nodes). This is helpful in trying out new DS
> >>>>>>>>>>>> configs before rolling them out to production.
> >>>>>>>>>>>>
> >>>>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or resource
> >>>>>>>>>>>> reservation on the delivery service. Providers of content to the CDN can
> >>>>>>>>>>>> receive tiered service by either sharing caches or using dedicated caches.
> >>>>>>>>>>>>
> >>>>>>>>>>>> —Eric
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <
> >>>>>>>>>>>> mtorluemke@apache.org> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Today, there isn't an actual link from Cache Group to DS -- I
> >>>>>>>>>>>> think what
> >>>>>>>>>>>> you are referring to is all done in the app layer to make a
> >>>>>>>>>>>> nice summary in
> >>>>>>>>>>>> the UI to allow operators to very quickly assign all caches in
> >>>>>>>>>>>> a cache
> >>>>>>>>>>>> group.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Do you use that "feature" to assign caches from some cache
> >>>>>>>>>>>> groups, and not
> >>>>>>>>>>>> others?
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
> >>>>>>>>>>>> efriedri@cisco.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> —Eric
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
> >>>>>>>>>>>>> mtorluemke@apache.org> wrote:
> >>>>>>>>>>>>> >
> >>>>>>>>>>>>> > Seeking community opinion on this.
> >>>>>>>>>>>>> >
> >>>>>>>>>>>>> > Should we drop the server -> delivery service assignments
> >>>>>>>>>>>>> completely?
> >>>>>>>>>>>>> >
> >>>>>>>>>>>>> > When the project began, selectively assigning caches was the
> >>>>>>>>>>>>> only way to selectively route traffic within a cache group -- let's say you
> >>>>>>>>>>>>> have a DNS-routed service with a relatively large library size, you might
> >>>>>>>>>>>>> not want assign all caches in a cache group to that service for fear you
> >>>>>>>>>>>>> would 'pollute' those caches. Or, if you were turning up a 'test' DS of
> >>>>>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
> >>>>>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
> >>>>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
> >>>>>>>>>>>>> services, so the same functionality can essentially be reached with
> >>>>>>>>>>>>> assigning all caches, and setting the config params properly.
> >>>>>>>>>>>>> >
> >>>>>>>>>>>>> > The motivation for this is to reduce the size & complexity
> >>>>>>>>>>>>> of the Traffic Router's config (aka CRConfig), as this config scales mostly
> >>>>>>>>>>>>> linearly with the number of caches and the number of delivery services in
> >>>>>>>>>>>>> your CDN.
> >>>>>>>>>>>>> >
> >>>>>>>>>>>>> > Another option might be to not assign caches as the default,
> >>>>>>>>>>>>> and individual cache assignments is available, but used only when necessary.
> >>>>>>>>>>>>> >
> >>>>>>>>>>>>> > Other thoughts? Clarifications needed?
> >>>>>>>>>>>>> >
> >>>>>>>>>>>>> > Thanks,
> >>>>>>>>>>>>> > Mark
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >
>
Re: Drop Server -> Delivery Service assignments?
Posted by Nir Sopher <ni...@qwilt.com>.
Hi Jeremy,
I’m ok with automatically assigning all edges to a new DS. The aspect of
"self-service" can be dealt with in a different manner.
However, how would you solve the problem when a new server is created? How
would you know which DS it should be automatically be assigned?
The model has no configuration marking the DS as one that should be
deployed on all servers.
My suggestion was to model it with "undef" value in the DS/Servers table.
DS | Server
------+--------
ds1 | server1 => Allow the CDN owner to deploy the ds on a specific cache
------+--------
ds2 | *undef *=> Allow the CDN owner to deploy the ds on all caches
This basically means that every place reading the DS/Server table, will
have to use a wrapper, unfolding a the "server=undef" raw, to a raw per
relevant server in the CDN.
I'm not sure how much effort it requires.
Nir
On Wed, May 24, 2017 at 5:21 PM, Jeremy Mitchell <mi...@gmail.com>
wrote:
> Nir,
>
> Thinking about self-service, do you envision the "DS owner" creates the DS
> and the "CDN owner" assigns caches to the DS? This seems like an
> unnecessary step to me and one that negates the idea of "self service".
>
> As the "DS owner", I want to create a DS and by default, employ ALL caches
> in my cdn. To be honest, in 99% of cases I really don't want to be bothered
> with assigning caches. I want the full power of the CDN behind my DS.
>
> So, I'm suggesting that when a new DS is created thru the API, ALL edge
> caches in the cdn are assigned to that DS.....and then, the CDN owner (or
> maybe DS owner) can remove caches from that DS if they need to.
>
> Jeremy
>
> On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com> wrote:
>
>> @Jeremy
>>
>> The issue I was trying to point at, is that the "DS-Server" table is the
>> one to provide the separation between the DS owner domain and the CDN owner
>> domain (tenancy wise).
>> As I see it, when considering multi tenancy, the DS has its owning tenant
>> which is in charge of the specific record in the DS table, and specifically
>> on adding the DS. The CDN is usually owned by another tenant that controls
>> the deployment - currently by adding the ds/servers record to the DS/Server
>> table.
>>
>> If the DS/Server table is dismissed or automatically written as a result
>> of a ds addition, it removes the "server assignment" step and the CDN owner
>> loses the control over the deployment decision.
>>
>> Before I open a discussion about how to solve this new tenancy issue, I
>> would like to verify it is a real issue and there is not something I'm
>> missing there.
>> What do you think?
>>
>> Nir
>>
>> On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com> wrote:
>>
>>> @Nir - maybe your proposal is something to consider more long-term but
>>> for the short-term i'm wondering if one of the following can / should be
>>> implemented:
>>>
>>> A) creating a DS thru the API (POST /api/*/deliveryservices) can have a
>>> default effect of automatically assigning all edges to the new DS (via the deliveryservice_server
>>> table)
>>>
>>> or
>>>
>>> B) the deliveryservice_server table simple goes away and it becomes
>>> implied that a DS employs all edge caches
>>>
>>> "A" would probably be the easiest to be honest.
>>>
>>> jeremy
>>>
>>> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com> wrote:
>>>
>>>> +1 for having a configuration allowing to apply the DS to the entire
>>>> CDN.
>>>>
>>>> Note that adding a "deploy by default" bool field to the DS itself may
>>>> cause tenancy issues, as I believe this field should be in the control of
>>>> the "CDN owner" tenant, and not of the "DS owner".
>>>>
>>>> What about keeping the ds_server table, and allow "undef" for the
>>>> server value - meaning "all servers in the CDN"?
>>>> In the future the table may grow and have additional fields to
>>>> white-list servers by.
>>>> For example:
>>>>
>>>> DS | Server | cache-group | profile
>>>> ------+--------+-------------+--------
>>>> ds1 | s1 | null | null => ds to be applied on s1
>>>> ------+--------+-------------+--------
>>>> ds2 | null | null | null => ds to be applied on all
>>>> servers (in ds CDN)
>>>> ------+--------+-------------+--------
>>>> ds3 | null | group1 | null => ds to be applied on all
>>>> servers in cache group "group1"
>>>> ------+--------+-------------+--------
>>>> ds4 | null | group1 | ATS-6.2 => ds to be applied on all
>>>> servers in cache group "group1", with profile "ATS-6.2"
>>>>
>>>>
>>>> During CR config snapshot and configuration pull (or in advance), the
>>>> required calculation would be done, to provide the list of relevant DSs per
>>>> server.
>>>>
>>>> Also note that moving in this direction is backward compatible as well
>>>> as supports other non trivial extensions one may want in the future:
>>>> Patterns (wildcards/regexs) support, as well as black list, using a "policy
>>>> table" mechanism.
>>>>
>>>> In the below example "ds5" is to be deployed on all caches, except for
>>>> caches with "ATS-5" profile
>>>>
>>>> DS | Server | cache-group | profile | deploy-decision
>>>> ------+--------+-------------+---------+----------------
>>>> ds5 | null | null | ATS-5 | false
>>>> ------+--------+-------------+---------+----------------
>>>> ds5 | null | null | null | true
>>>> ------+--------+-------------+---------+----------------
>>>>
>>>>
>>>> Nir
>>>>
>>>> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <mitchell852@gmail.com
>>>> > wrote:
>>>>
>>>>> @dave - i'm not sure if you're suggesting:
>>>>>
>>>>> A) on DS create, by default create an entry in the
>>>>> deliveryservice_server table for each cdn edge cache. so if there are 200
>>>>> cdn edge caches, you end up with 200 entries in the deliveryservice_server
>>>>> table for that DS.
>>>>> B) completely kill the deliveryservice_server table and it becomes
>>>>> implied that a DS uses ALL cdn edge caches (the way mids work today.
>>>>> mids don't have to be explicitly assigned to a DS)
>>>>>
>>>>> "A" would allow you to control the scope of caches by using initial
>>>>> dispersion OR control the scope of caches by simply removing the ones
>>>>> you don't want to receive traffic.
>>>>> "B" would only allow you to control the scope of caches by using
>>>>> initial dispersion (or so i think).
>>>>>
>>>>> jeremy
>>>>>
>>>>>
>>>>>
>>>>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> +1, I think assigning DSs to all caches by default is fine. Then we
>>>>>> can use initial dispersion to make sure traffic is spread between an
>>>>>> appropriate number of caches.
>>>>>>
>>>>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <
>>>>>> mitchell852@gmail.com> wrote:
>>>>>>
>>>>>>> Bringing up an old email regarding DS to cache assignments.
>>>>>>>
>>>>>>> It sounds like we want to preserve DS/cache assignments (the
>>>>>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
>>>>>>> to a DS if you want. However, what if we make the assignment of ALL cdn
>>>>>>> edge caches to a new DS the default?
>>>>>>>
>>>>>>> So when you create a DS for the Foo cdn thru the API, your new DS is
>>>>>>> automagically assigned all the edge caches in the Foo cdn...
>>>>>>>
>>>>>>> (at that point you could always remove caches from a DS)
>>>>>>>
>>>>>>> ...or the create DS API could have a boolean flag added to it. for
>>>>>>> example, assignAllCaches: true|false. if you pass in true, all caches are
>>>>>>> assigned to the DS, if false, none are.
>>>>>>>
>>>>>>> thoughts? I'm trying to think of self-service and i'm guessing most
>>>>>>> people will not want to assign caches to a DS...
>>>>>>>
>>>>>>> jeremy
>>>>>>>
>>>>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>
>>>>>>>> gotcha. makes sense.
>>>>>>>>
>>>>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <
>>>>>>>> david.neuman64@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I believe Eric had a use case where he wanted to assign certain
>>>>>>>>> delivery services to certain caches.
>>>>>>>>> If we just use initial dispersion and max DNS answers then we
>>>>>>>>> don't have the ability to control which servers are used, just how many.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
>>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> So what I heard was that potentially we move in this direction:
>>>>>>>>>>
>>>>>>>>>> 1. No need to explicitly assign edge caches to a delivery
>>>>>>>>>> service. a delivery service (by default) would employ ALL edge caches of
>>>>>>>>>> the cdn in which the ds belongs to.
>>>>>>>>>> 2. Ability to override this default behavior and explicitly
>>>>>>>>>> assign specific edge caches to a delivery service (this is what we do today)
>>>>>>>>>>
>>>>>>>>>> But earlier Mark said "so the same functionality [as #2] can
>>>>>>>>>> essentially be reached with assigning all caches, and setting the config
>>>>>>>>>> params properly [initial dispersion and max dns answers]"
>>>>>>>>>>
>>>>>>>>>> so why not ditch #2 in favor of using initial dispersion and max
>>>>>>>>>> dns answers to control cache assignments?
>>>>>>>>>>
>>>>>>>>>> it would be nice imo to get rid of the ds_server table. i'm
>>>>>>>>>> always in favor of reducing complexity... :)
>>>>>>>>>>
>>>>>>>>>> jeremy
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Yes - I think its a reasonable default to have a delivery
>>>>>>>>>>> service assigned to all caches as long as we still keep the ability to
>>>>>>>>>>> assign some DS’s to just a subset of caches.
>>>>>>>>>>>
>>>>>>>>>>> —Eric
>>>>>>>>>>>
>>>>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <
>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Good feedback, thanks. I think having _zero_ caches assigned to
>>>>>>>>>>> a delivery service could be added fairly easily, and could function as a
>>>>>>>>>>> good default to the assumption of _all_ caches being assigned. Would you
>>>>>>>>>>> agree with that?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Mark
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Yeah,
>>>>>>>>>>>> We have several some use cases where where cache or CG level
>>>>>>>>>>>> allocation would be helpful.
>>>>>>>>>>>>
>>>>>>>>>>>> 1) Assigning new or trial delivery services to a subset of
>>>>>>>>>>>> caches (considered beta cache nodes). This is helpful in trying out new DS
>>>>>>>>>>>> configs before rolling them out to production.
>>>>>>>>>>>>
>>>>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or resource
>>>>>>>>>>>> reservation on the delivery service. Providers of content to the CDN can
>>>>>>>>>>>> receive tiered service by either sharing caches or using dedicated caches.
>>>>>>>>>>>>
>>>>>>>>>>>> —Eric
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <
>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Today, there isn't an actual link from Cache Group to DS -- I
>>>>>>>>>>>> think what
>>>>>>>>>>>> you are referring to is all done in the app layer to make a
>>>>>>>>>>>> nice summary in
>>>>>>>>>>>> the UI to allow operators to very quickly assign all caches in
>>>>>>>>>>>> a cache
>>>>>>>>>>>> group.
>>>>>>>>>>>>
>>>>>>>>>>>> Do you use that "feature" to assign caches from some cache
>>>>>>>>>>>> groups, and not
>>>>>>>>>>>> others?
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
>>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
>>>>>>>>>>>>>
>>>>>>>>>>>>> —Eric
>>>>>>>>>>>>>
>>>>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
>>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Seeking community opinion on this.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Should we drop the server -> delivery service assignments
>>>>>>>>>>>>> completely?
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > When the project began, selectively assigning caches was the
>>>>>>>>>>>>> only way to selectively route traffic within a cache group -- let's say you
>>>>>>>>>>>>> have a DNS-routed service with a relatively large library size, you might
>>>>>>>>>>>>> not want assign all caches in a cache group to that service for fear you
>>>>>>>>>>>>> would 'pollute' those caches. Or, if you were turning up a 'test' DS of
>>>>>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
>>>>>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
>>>>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
>>>>>>>>>>>>> services, so the same functionality can essentially be reached with
>>>>>>>>>>>>> assigning all caches, and setting the config params properly.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > The motivation for this is to reduce the size & complexity
>>>>>>>>>>>>> of the Traffic Router's config (aka CRConfig), as this config scales mostly
>>>>>>>>>>>>> linearly with the number of caches and the number of delivery services in
>>>>>>>>>>>>> your CDN.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Another option might be to not assign caches as the default,
>>>>>>>>>>>>> and individual cache assignments is available, but used only when necessary.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Other thoughts? Clarifications needed?
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Thanks,
>>>>>>>>>>>>> > Mark
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>
Re: Drop Server -> Delivery Service assignments?
Posted by Jeremy Mitchell <mi...@gmail.com>.
Nir,
Thinking about self-service, do you envision the "DS owner" creates the DS
and the "CDN owner" assigns caches to the DS? This seems like an
unnecessary step to me and one that negates the idea of "self service".
As the "DS owner", I want to create a DS and by default, employ ALL caches
in my cdn. To be honest, in 99% of cases I really don't want to be bothered
with assigning caches. I want the full power of the CDN behind my DS.
So, I'm suggesting that when a new DS is created thru the API, ALL edge
caches in the cdn are assigned to that DS.....and then, the CDN owner (or
maybe DS owner) can remove caches from that DS if they need to.
Jeremy
On Tue, May 23, 2017 at 1:08 AM, Nir Sopher <ni...@qwilt.com> wrote:
> @Jeremy
>
> The issue I was trying to point at, is that the "DS-Server" table is the
> one to provide the separation between the DS owner domain and the CDN owner
> domain (tenancy wise).
> As I see it, when considering multi tenancy, the DS has its owning tenant
> which is in charge of the specific record in the DS table, and specifically
> on adding the DS. The CDN is usually owned by another tenant that controls
> the deployment - currently by adding the ds/servers record to the DS/Server
> table.
>
> If the DS/Server table is dismissed or automatically written as a result
> of a ds addition, it removes the "server assignment" step and the CDN owner
> loses the control over the deployment decision.
>
> Before I open a discussion about how to solve this new tenancy issue, I
> would like to verify it is a real issue and there is not something I'm
> missing there.
> What do you think?
>
> Nir
>
> On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com> wrote:
>
>> @Nir - maybe your proposal is something to consider more long-term but
>> for the short-term i'm wondering if one of the following can / should be
>> implemented:
>>
>> A) creating a DS thru the API (POST /api/*/deliveryservices) can have a
>> default effect of automatically assigning all edges to the new DS (via the deliveryservice_server
>> table)
>>
>> or
>>
>> B) the deliveryservice_server table simple goes away and it becomes
>> implied that a DS employs all edge caches
>>
>> "A" would probably be the easiest to be honest.
>>
>> jeremy
>>
>> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com> wrote:
>>
>>> +1 for having a configuration allowing to apply the DS to the entire CDN.
>>>
>>> Note that adding a "deploy by default" bool field to the DS itself may
>>> cause tenancy issues, as I believe this field should be in the control of
>>> the "CDN owner" tenant, and not of the "DS owner".
>>>
>>> What about keeping the ds_server table, and allow "undef" for the server
>>> value - meaning "all servers in the CDN"?
>>> In the future the table may grow and have additional fields to
>>> white-list servers by.
>>> For example:
>>>
>>> DS | Server | cache-group | profile
>>> ------+--------+-------------+--------
>>> ds1 | s1 | null | null => ds to be applied on s1
>>> ------+--------+-------------+--------
>>> ds2 | null | null | null => ds to be applied on all
>>> servers (in ds CDN)
>>> ------+--------+-------------+--------
>>> ds3 | null | group1 | null => ds to be applied on all
>>> servers in cache group "group1"
>>> ------+--------+-------------+--------
>>> ds4 | null | group1 | ATS-6.2 => ds to be applied on all
>>> servers in cache group "group1", with profile "ATS-6.2"
>>>
>>>
>>> During CR config snapshot and configuration pull (or in advance), the
>>> required calculation would be done, to provide the list of relevant DSs per
>>> server.
>>>
>>> Also note that moving in this direction is backward compatible as well
>>> as supports other non trivial extensions one may want in the future:
>>> Patterns (wildcards/regexs) support, as well as black list, using a "policy
>>> table" mechanism.
>>>
>>> In the below example "ds5" is to be deployed on all caches, except for
>>> caches with "ATS-5" profile
>>>
>>> DS | Server | cache-group | profile | deploy-decision
>>> ------+--------+-------------+---------+----------------
>>> ds5 | null | null | ATS-5 | false
>>> ------+--------+-------------+---------+----------------
>>> ds5 | null | null | null | true
>>> ------+--------+-------------+---------+----------------
>>>
>>>
>>> Nir
>>>
>>> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <mi...@gmail.com>
>>> wrote:
>>>
>>>> @dave - i'm not sure if you're suggesting:
>>>>
>>>> A) on DS create, by default create an entry in the
>>>> deliveryservice_server table for each cdn edge cache. so if there are 200
>>>> cdn edge caches, you end up with 200 entries in the deliveryservice_server
>>>> table for that DS.
>>>> B) completely kill the deliveryservice_server table and it becomes
>>>> implied that a DS uses ALL cdn edge caches (the way mids work today.
>>>> mids don't have to be explicitly assigned to a DS)
>>>>
>>>> "A" would allow you to control the scope of caches by using initial
>>>> dispersion OR control the scope of caches by simply removing the ones
>>>> you don't want to receive traffic.
>>>> "B" would only allow you to control the scope of caches by using
>>>> initial dispersion (or so i think).
>>>>
>>>> jeremy
>>>>
>>>>
>>>>
>>>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org> wrote:
>>>>
>>>>> +1, I think assigning DSs to all caches by default is fine. Then we
>>>>> can use initial dispersion to make sure traffic is spread between an
>>>>> appropriate number of caches.
>>>>>
>>>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <
>>>>> mitchell852@gmail.com> wrote:
>>>>>
>>>>>> Bringing up an old email regarding DS to cache assignments.
>>>>>>
>>>>>> It sounds like we want to preserve DS/cache assignments (the
>>>>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
>>>>>> to a DS if you want. However, what if we make the assignment of ALL cdn
>>>>>> edge caches to a new DS the default?
>>>>>>
>>>>>> So when you create a DS for the Foo cdn thru the API, your new DS is
>>>>>> automagically assigned all the edge caches in the Foo cdn...
>>>>>>
>>>>>> (at that point you could always remove caches from a DS)
>>>>>>
>>>>>> ...or the create DS API could have a boolean flag added to it. for
>>>>>> example, assignAllCaches: true|false. if you pass in true, all caches are
>>>>>> assigned to the DS, if false, none are.
>>>>>>
>>>>>> thoughts? I'm trying to think of self-service and i'm guessing most
>>>>>> people will not want to assign caches to a DS...
>>>>>>
>>>>>> jeremy
>>>>>>
>>>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
>>>>>> mitchell852@gmail.com> wrote:
>>>>>>
>>>>>>> gotcha. makes sense.
>>>>>>>
>>>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <
>>>>>>> david.neuman64@gmail.com> wrote:
>>>>>>>
>>>>>>>> I believe Eric had a use case where he wanted to assign certain
>>>>>>>> delivery services to certain caches.
>>>>>>>> If we just use initial dispersion and max DNS answers then we don't
>>>>>>>> have the ability to control which servers are used, just how many.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
>>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> So what I heard was that potentially we move in this direction:
>>>>>>>>>
>>>>>>>>> 1. No need to explicitly assign edge caches to a delivery service.
>>>>>>>>> a delivery service (by default) would employ ALL edge caches of the cdn in
>>>>>>>>> which the ds belongs to.
>>>>>>>>> 2. Ability to override this default behavior and explicitly assign
>>>>>>>>> specific edge caches to a delivery service (this is what we do today)
>>>>>>>>>
>>>>>>>>> But earlier Mark said "so the same functionality [as #2] can
>>>>>>>>> essentially be reached with assigning all caches, and setting the config
>>>>>>>>> params properly [initial dispersion and max dns answers]"
>>>>>>>>>
>>>>>>>>> so why not ditch #2 in favor of using initial dispersion and max
>>>>>>>>> dns answers to control cache assignments?
>>>>>>>>>
>>>>>>>>> it would be nice imo to get rid of the ds_server table. i'm always
>>>>>>>>> in favor of reducing complexity... :)
>>>>>>>>>
>>>>>>>>> jeremy
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>
>>>>>>>>>> Yes - I think its a reasonable default to have a delivery service
>>>>>>>>>> assigned to all caches as long as we still keep the ability to assign some
>>>>>>>>>> DS’s to just a subset of caches.
>>>>>>>>>>
>>>>>>>>>> —Eric
>>>>>>>>>>
>>>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <mt...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Good feedback, thanks. I think having _zero_ caches assigned to a
>>>>>>>>>> delivery service could be added fairly easily, and could function as a good
>>>>>>>>>> default to the assumption of _all_ caches being assigned. Would you agree
>>>>>>>>>> with that?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Mark
>>>>>>>>>>
>>>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Yeah,
>>>>>>>>>>> We have several some use cases where where cache or CG level
>>>>>>>>>>> allocation would be helpful.
>>>>>>>>>>>
>>>>>>>>>>> 1) Assigning new or trial delivery services to a subset of
>>>>>>>>>>> caches (considered beta cache nodes). This is helpful in trying out new DS
>>>>>>>>>>> configs before rolling them out to production.
>>>>>>>>>>>
>>>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or resource
>>>>>>>>>>> reservation on the delivery service. Providers of content to the CDN can
>>>>>>>>>>> receive tiered service by either sharing caches or using dedicated caches.
>>>>>>>>>>>
>>>>>>>>>>> —Eric
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <
>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Today, there isn't an actual link from Cache Group to DS -- I
>>>>>>>>>>> think what
>>>>>>>>>>> you are referring to is all done in the app layer to make a nice
>>>>>>>>>>> summary in
>>>>>>>>>>> the UI to allow operators to very quickly assign all caches in a
>>>>>>>>>>> cache
>>>>>>>>>>> group.
>>>>>>>>>>>
>>>>>>>>>>> Do you use that "feature" to assign caches from some cache
>>>>>>>>>>> groups, and not
>>>>>>>>>>> others?
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
>>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
>>>>>>>>>>>>
>>>>>>>>>>>> —Eric
>>>>>>>>>>>>
>>>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
>>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>>> >
>>>>>>>>>>>> > Seeking community opinion on this.
>>>>>>>>>>>> >
>>>>>>>>>>>> > Should we drop the server -> delivery service assignments
>>>>>>>>>>>> completely?
>>>>>>>>>>>> >
>>>>>>>>>>>> > When the project began, selectively assigning caches was the
>>>>>>>>>>>> only way to selectively route traffic within a cache group -- let's say you
>>>>>>>>>>>> have a DNS-routed service with a relatively large library size, you might
>>>>>>>>>>>> not want assign all caches in a cache group to that service for fear you
>>>>>>>>>>>> would 'pollute' those caches. Or, if you were turning up a 'test' DS of
>>>>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
>>>>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
>>>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
>>>>>>>>>>>> services, so the same functionality can essentially be reached with
>>>>>>>>>>>> assigning all caches, and setting the config params properly.
>>>>>>>>>>>> >
>>>>>>>>>>>> > The motivation for this is to reduce the size & complexity of
>>>>>>>>>>>> the Traffic Router's config (aka CRConfig), as this config scales mostly
>>>>>>>>>>>> linearly with the number of caches and the number of delivery services in
>>>>>>>>>>>> your CDN.
>>>>>>>>>>>> >
>>>>>>>>>>>> > Another option might be to not assign caches as the default,
>>>>>>>>>>>> and individual cache assignments is available, but used only when necessary.
>>>>>>>>>>>> >
>>>>>>>>>>>> > Other thoughts? Clarifications needed?
>>>>>>>>>>>> >
>>>>>>>>>>>> > Thanks,
>>>>>>>>>>>> > Mark
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
Re: Drop Server -> Delivery Service assignments?
Posted by Nir Sopher <ni...@qwilt.com>.
@Jeremy
The issue I was trying to point at, is that the "DS-Server" table is the
one to provide the separation between the DS owner domain and the CDN owner
domain (tenancy wise).
As I see it, when considering multi tenancy, the DS has its owning tenant
which is in charge of the specific record in the DS table, and specifically
on adding the DS. The CDN is usually owned by another tenant that controls
the deployment - currently by adding the ds/servers record to the DS/Server
table.
If the DS/Server table is dismissed or automatically written as a result of
a ds addition, it removes the "server assignment" step and the CDN owner
loses the control over the deployment decision.
Before I open a discussion about how to solve this new tenancy issue, I
would like to verify it is a real issue and there is not something I'm
missing there.
What do you think?
Nir
On May 22, 2017 6:13 PM, "Jeremy Mitchell" <mi...@gmail.com> wrote:
> @Nir - maybe your proposal is something to consider more long-term but for
> the short-term i'm wondering if one of the following can / should be
> implemented:
>
> A) creating a DS thru the API (POST /api/*/deliveryservices) can have a
> default effect of automatically assigning all edges to the new DS (via the deliveryservice_server
> table)
>
> or
>
> B) the deliveryservice_server table simple goes away and it becomes
> implied that a DS employs all edge caches
>
> "A" would probably be the easiest to be honest.
>
> jeremy
>
> On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com> wrote:
>
>> +1 for having a configuration allowing to apply the DS to the entire CDN.
>>
>> Note that adding a "deploy by default" bool field to the DS itself may
>> cause tenancy issues, as I believe this field should be in the control of
>> the "CDN owner" tenant, and not of the "DS owner".
>>
>> What about keeping the ds_server table, and allow "undef" for the server
>> value - meaning "all servers in the CDN"?
>> In the future the table may grow and have additional fields to white-list
>> servers by.
>> For example:
>>
>> DS | Server | cache-group | profile
>> ------+--------+-------------+--------
>> ds1 | s1 | null | null => ds to be applied on s1
>> ------+--------+-------------+--------
>> ds2 | null | null | null => ds to be applied on all
>> servers (in ds CDN)
>> ------+--------+-------------+--------
>> ds3 | null | group1 | null => ds to be applied on all
>> servers in cache group "group1"
>> ------+--------+-------------+--------
>> ds4 | null | group1 | ATS-6.2 => ds to be applied on all
>> servers in cache group "group1", with profile "ATS-6.2"
>>
>>
>> During CR config snapshot and configuration pull (or in advance), the
>> required calculation would be done, to provide the list of relevant DSs per
>> server.
>>
>> Also note that moving in this direction is backward compatible as well as
>> supports other non trivial extensions one may want in the future: Patterns
>> (wildcards/regexs) support, as well as black list, using a "policy table"
>> mechanism.
>>
>> In the below example "ds5" is to be deployed on all caches, except for
>> caches with "ATS-5" profile
>>
>> DS | Server | cache-group | profile | deploy-decision
>> ------+--------+-------------+---------+----------------
>> ds5 | null | null | ATS-5 | false
>> ------+--------+-------------+---------+----------------
>> ds5 | null | null | null | true
>> ------+--------+-------------+---------+----------------
>>
>>
>> Nir
>>
>> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <mi...@gmail.com>
>> wrote:
>>
>>> @dave - i'm not sure if you're suggesting:
>>>
>>> A) on DS create, by default create an entry in the
>>> deliveryservice_server table for each cdn edge cache. so if there are 200
>>> cdn edge caches, you end up with 200 entries in the deliveryservice_server
>>> table for that DS.
>>> B) completely kill the deliveryservice_server table and it becomes
>>> implied that a DS uses ALL cdn edge caches (the way mids work today.
>>> mids don't have to be explicitly assigned to a DS)
>>>
>>> "A" would allow you to control the scope of caches by using initial
>>> dispersion OR control the scope of caches by simply removing the ones
>>> you don't want to receive traffic.
>>> "B" would only allow you to control the scope of caches by using
>>> initial dispersion (or so i think).
>>>
>>> jeremy
>>>
>>>
>>>
>>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org> wrote:
>>>
>>>> +1, I think assigning DSs to all caches by default is fine. Then we
>>>> can use initial dispersion to make sure traffic is spread between an
>>>> appropriate number of caches.
>>>>
>>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <mitchell852@gmail.com
>>>> > wrote:
>>>>
>>>>> Bringing up an old email regarding DS to cache assignments.
>>>>>
>>>>> It sounds like we want to preserve DS/cache assignments (the
>>>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
>>>>> to a DS if you want. However, what if we make the assignment of ALL cdn
>>>>> edge caches to a new DS the default?
>>>>>
>>>>> So when you create a DS for the Foo cdn thru the API, your new DS is
>>>>> automagically assigned all the edge caches in the Foo cdn...
>>>>>
>>>>> (at that point you could always remove caches from a DS)
>>>>>
>>>>> ...or the create DS API could have a boolean flag added to it. for
>>>>> example, assignAllCaches: true|false. if you pass in true, all caches are
>>>>> assigned to the DS, if false, none are.
>>>>>
>>>>> thoughts? I'm trying to think of self-service and i'm guessing most
>>>>> people will not want to assign caches to a DS...
>>>>>
>>>>> jeremy
>>>>>
>>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <
>>>>> mitchell852@gmail.com> wrote:
>>>>>
>>>>>> gotcha. makes sense.
>>>>>>
>>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <
>>>>>> david.neuman64@gmail.com> wrote:
>>>>>>
>>>>>>> I believe Eric had a use case where he wanted to assign certain
>>>>>>> delivery services to certain caches.
>>>>>>> If we just use initial dispersion and max DNS answers then we don't
>>>>>>> have the ability to control which servers are used, just how many.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
>>>>>>> mitchell852@gmail.com> wrote:
>>>>>>>
>>>>>>>> So what I heard was that potentially we move in this direction:
>>>>>>>>
>>>>>>>> 1. No need to explicitly assign edge caches to a delivery service.
>>>>>>>> a delivery service (by default) would employ ALL edge caches of the cdn in
>>>>>>>> which the ds belongs to.
>>>>>>>> 2. Ability to override this default behavior and explicitly assign
>>>>>>>> specific edge caches to a delivery service (this is what we do today)
>>>>>>>>
>>>>>>>> But earlier Mark said "so the same functionality [as #2] can
>>>>>>>> essentially be reached with assigning all caches, and setting the config
>>>>>>>> params properly [initial dispersion and max dns answers]"
>>>>>>>>
>>>>>>>> so why not ditch #2 in favor of using initial dispersion and max
>>>>>>>> dns answers to control cache assignments?
>>>>>>>>
>>>>>>>> it would be nice imo to get rid of the ds_server table. i'm always
>>>>>>>> in favor of reducing complexity... :)
>>>>>>>>
>>>>>>>> jeremy
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>
>>>>>>>>> Yes - I think its a reasonable default to have a delivery service
>>>>>>>>> assigned to all caches as long as we still keep the ability to assign some
>>>>>>>>> DS’s to just a subset of caches.
>>>>>>>>>
>>>>>>>>> —Eric
>>>>>>>>>
>>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <mt...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Good feedback, thanks. I think having _zero_ caches assigned to a
>>>>>>>>> delivery service could be added fairly easily, and could function as a good
>>>>>>>>> default to the assumption of _all_ caches being assigned. Would you agree
>>>>>>>>> with that?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Mark
>>>>>>>>>
>>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>
>>>>>>>>>> Yeah,
>>>>>>>>>> We have several some use cases where where cache or CG level
>>>>>>>>>> allocation would be helpful.
>>>>>>>>>>
>>>>>>>>>> 1) Assigning new or trial delivery services to a subset of caches
>>>>>>>>>> (considered beta cache nodes). This is helpful in trying out new DS configs
>>>>>>>>>> before rolling them out to production.
>>>>>>>>>>
>>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or resource
>>>>>>>>>> reservation on the delivery service. Providers of content to the CDN can
>>>>>>>>>> receive tiered service by either sharing caches or using dedicated caches.
>>>>>>>>>>
>>>>>>>>>> —Eric
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <mt...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Today, there isn't an actual link from Cache Group to DS -- I
>>>>>>>>>> think what
>>>>>>>>>> you are referring to is all done in the app layer to make a nice
>>>>>>>>>> summary in
>>>>>>>>>> the UI to allow operators to very quickly assign all caches in a
>>>>>>>>>> cache
>>>>>>>>>> group.
>>>>>>>>>>
>>>>>>>>>> Do you use that "feature" to assign caches from some cache
>>>>>>>>>> groups, and not
>>>>>>>>>> others?
>>>>>>>>>>
>>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
>>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
>>>>>>>>>>>
>>>>>>>>>>> —Eric
>>>>>>>>>>>
>>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
>>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>>> >
>>>>>>>>>>> > Seeking community opinion on this.
>>>>>>>>>>> >
>>>>>>>>>>> > Should we drop the server -> delivery service assignments
>>>>>>>>>>> completely?
>>>>>>>>>>> >
>>>>>>>>>>> > When the project began, selectively assigning caches was the
>>>>>>>>>>> only way to selectively route traffic within a cache group -- let's say you
>>>>>>>>>>> have a DNS-routed service with a relatively large library size, you might
>>>>>>>>>>> not want assign all caches in a cache group to that service for fear you
>>>>>>>>>>> would 'pollute' those caches. Or, if you were turning up a 'test' DS of
>>>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
>>>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
>>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
>>>>>>>>>>> services, so the same functionality can essentially be reached with
>>>>>>>>>>> assigning all caches, and setting the config params properly.
>>>>>>>>>>> >
>>>>>>>>>>> > The motivation for this is to reduce the size & complexity of
>>>>>>>>>>> the Traffic Router's config (aka CRConfig), as this config scales mostly
>>>>>>>>>>> linearly with the number of caches and the number of delivery services in
>>>>>>>>>>> your CDN.
>>>>>>>>>>> >
>>>>>>>>>>> > Another option might be to not assign caches as the default,
>>>>>>>>>>> and individual cache assignments is available, but used only when necessary.
>>>>>>>>>>> >
>>>>>>>>>>> > Other thoughts? Clarifications needed?
>>>>>>>>>>> >
>>>>>>>>>>> > Thanks,
>>>>>>>>>>> > Mark
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Drop Server -> Delivery Service assignments?
Posted by Jeremy Mitchell <mi...@gmail.com>.
@Nir - maybe your proposal is something to consider more long-term but for
the short-term i'm wondering if one of the following can / should be
implemented:
A) creating a DS thru the API (POST /api/*/deliveryservices) can have a
default effect of automatically assigning all edges to the new DS (via
the deliveryservice_server
table)
or
B) the deliveryservice_server table simple goes away and it becomes implied
that a DS employs all edge caches
"A" would probably be the easiest to be honest.
jeremy
On Sat, May 20, 2017 at 12:34 PM, Nir Sopher <ni...@qwilt.com> wrote:
> +1 for having a configuration allowing to apply the DS to the entire CDN.
>
> Note that adding a "deploy by default" bool field to the DS itself may
> cause tenancy issues, as I believe this field should be in the control of
> the "CDN owner" tenant, and not of the "DS owner".
>
> What about keeping the ds_server table, and allow "undef" for the server
> value - meaning "all servers in the CDN"?
> In the future the table may grow and have additional fields to white-list
> servers by.
> For example:
>
> DS | Server | cache-group | profile
> ------+--------+-------------+--------
> ds1 | s1 | null | null => ds to be applied on s1
> ------+--------+-------------+--------
> ds2 | null | null | null => ds to be applied on all
> servers (in ds CDN)
> ------+--------+-------------+--------
> ds3 | null | group1 | null => ds to be applied on all
> servers in cache group "group1"
> ------+--------+-------------+--------
> ds4 | null | group1 | ATS-6.2 => ds to be applied on all
> servers in cache group "group1", with profile "ATS-6.2"
>
>
> During CR config snapshot and configuration pull (or in advance), the
> required calculation would be done, to provide the list of relevant DSs per
> server.
>
> Also note that moving in this direction is backward compatible as well as
> supports other non trivial extensions one may want in the future: Patterns
> (wildcards/regexs) support, as well as black list, using a "policy table"
> mechanism.
>
> In the below example "ds5" is to be deployed on all caches, except for
> caches with "ATS-5" profile
>
> DS | Server | cache-group | profile | deploy-decision
> ------+--------+-------------+---------+----------------
> ds5 | null | null | ATS-5 | false
> ------+--------+-------------+---------+----------------
> ds5 | null | null | null | true
> ------+--------+-------------+---------+----------------
>
>
> Nir
>
> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
>> @dave - i'm not sure if you're suggesting:
>>
>> A) on DS create, by default create an entry in the deliveryservice_server
>> table for each cdn edge cache. so if there are 200 cdn edge caches, you end
>> up with 200 entries in the deliveryservice_server table for that DS.
>> B) completely kill the deliveryservice_server table and it becomes
>> implied that a DS uses ALL cdn edge caches (the way mids work today.
>> mids don't have to be explicitly assigned to a DS)
>>
>> "A" would allow you to control the scope of caches by using initial
>> dispersion OR control the scope of caches by simply removing the ones
>> you don't want to receive traffic.
>> "B" would only allow you to control the scope of caches by using initial
>> dispersion (or so i think).
>>
>> jeremy
>>
>>
>>
>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org> wrote:
>>
>>> +1, I think assigning DSs to all caches by default is fine. Then we can
>>> use initial dispersion to make sure traffic is spread between an
>>> appropriate number of caches.
>>>
>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <mi...@gmail.com>
>>> wrote:
>>>
>>>> Bringing up an old email regarding DS to cache assignments.
>>>>
>>>> It sounds like we want to preserve DS/cache assignments (the
>>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
>>>> to a DS if you want. However, what if we make the assignment of ALL cdn
>>>> edge caches to a new DS the default?
>>>>
>>>> So when you create a DS for the Foo cdn thru the API, your new DS is
>>>> automagically assigned all the edge caches in the Foo cdn...
>>>>
>>>> (at that point you could always remove caches from a DS)
>>>>
>>>> ...or the create DS API could have a boolean flag added to it. for
>>>> example, assignAllCaches: true|false. if you pass in true, all caches are
>>>> assigned to the DS, if false, none are.
>>>>
>>>> thoughts? I'm trying to think of self-service and i'm guessing most
>>>> people will not want to assign caches to a DS...
>>>>
>>>> jeremy
>>>>
>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <mitchell852@gmail.com
>>>> > wrote:
>>>>
>>>>> gotcha. makes sense.
>>>>>
>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <david.neuman64@gmail.com
>>>>> > wrote:
>>>>>
>>>>>> I believe Eric had a use case where he wanted to assign certain
>>>>>> delivery services to certain caches.
>>>>>> If we just use initial dispersion and max DNS answers then we don't
>>>>>> have the ability to control which servers are used, just how many.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
>>>>>> mitchell852@gmail.com> wrote:
>>>>>>
>>>>>>> So what I heard was that potentially we move in this direction:
>>>>>>>
>>>>>>> 1. No need to explicitly assign edge caches to a delivery service. a
>>>>>>> delivery service (by default) would employ ALL edge caches of the cdn in
>>>>>>> which the ds belongs to.
>>>>>>> 2. Ability to override this default behavior and explicitly assign
>>>>>>> specific edge caches to a delivery service (this is what we do today)
>>>>>>>
>>>>>>> But earlier Mark said "so the same functionality [as #2] can
>>>>>>> essentially be reached with assigning all caches, and setting the config
>>>>>>> params properly [initial dispersion and max dns answers]"
>>>>>>>
>>>>>>> so why not ditch #2 in favor of using initial dispersion and max
>>>>>>> dns answers to control cache assignments?
>>>>>>>
>>>>>>> it would be nice imo to get rid of the ds_server table. i'm always
>>>>>>> in favor of reducing complexity... :)
>>>>>>>
>>>>>>> jeremy
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>
>>>>>>>> Yes - I think its a reasonable default to have a delivery service
>>>>>>>> assigned to all caches as long as we still keep the ability to assign some
>>>>>>>> DS’s to just a subset of caches.
>>>>>>>>
>>>>>>>> —Eric
>>>>>>>>
>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <mt...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Good feedback, thanks. I think having _zero_ caches assigned to a
>>>>>>>> delivery service could be added fairly easily, and could function as a good
>>>>>>>> default to the assumption of _all_ caches being assigned. Would you agree
>>>>>>>> with that?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Mark
>>>>>>>>
>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>
>>>>>>>>> Yeah,
>>>>>>>>> We have several some use cases where where cache or CG level
>>>>>>>>> allocation would be helpful.
>>>>>>>>>
>>>>>>>>> 1) Assigning new or trial delivery services to a subset of caches
>>>>>>>>> (considered beta cache nodes). This is helpful in trying out new DS configs
>>>>>>>>> before rolling them out to production.
>>>>>>>>>
>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or resource
>>>>>>>>> reservation on the delivery service. Providers of content to the CDN can
>>>>>>>>> receive tiered service by either sharing caches or using dedicated caches.
>>>>>>>>>
>>>>>>>>> —Eric
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <mt...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Today, there isn't an actual link from Cache Group to DS -- I
>>>>>>>>> think what
>>>>>>>>> you are referring to is all done in the app layer to make a nice
>>>>>>>>> summary in
>>>>>>>>> the UI to allow operators to very quickly assign all caches in a
>>>>>>>>> cache
>>>>>>>>> group.
>>>>>>>>>
>>>>>>>>> Do you use that "feature" to assign caches from some cache groups,
>>>>>>>>> and not
>>>>>>>>> others?
>>>>>>>>>
>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>
>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
>>>>>>>>>>
>>>>>>>>>> —Eric
>>>>>>>>>>
>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>> >
>>>>>>>>>> > Seeking community opinion on this.
>>>>>>>>>> >
>>>>>>>>>> > Should we drop the server -> delivery service assignments
>>>>>>>>>> completely?
>>>>>>>>>> >
>>>>>>>>>> > When the project began, selectively assigning caches was the
>>>>>>>>>> only way to selectively route traffic within a cache group -- let's say you
>>>>>>>>>> have a DNS-routed service with a relatively large library size, you might
>>>>>>>>>> not want assign all caches in a cache group to that service for fear you
>>>>>>>>>> would 'pollute' those caches. Or, if you were turning up a 'test' DS of
>>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
>>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
>>>>>>>>>> services, so the same functionality can essentially be reached with
>>>>>>>>>> assigning all caches, and setting the config params properly.
>>>>>>>>>> >
>>>>>>>>>> > The motivation for this is to reduce the size & complexity of
>>>>>>>>>> the Traffic Router's config (aka CRConfig), as this config scales mostly
>>>>>>>>>> linearly with the number of caches and the number of delivery services in
>>>>>>>>>> your CDN.
>>>>>>>>>> >
>>>>>>>>>> > Another option might be to not assign caches as the default,
>>>>>>>>>> and individual cache assignments is available, but used only when necessary.
>>>>>>>>>> >
>>>>>>>>>> > Other thoughts? Clarifications needed?
>>>>>>>>>> >
>>>>>>>>>> > Thanks,
>>>>>>>>>> > Mark
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Drop Server -> Delivery Service assignments?
Posted by "Eric Friedrich (efriedri)" <ef...@cisco.com>.
John-
Can you comment on the schema changes you are planning for Device Groups?
—Eric
On Oct 18, 2017, at 1:08 PM, Nir Sopher <ni...@qwilt.com>> wrote:
Bringing this thread up again, following Eric's presentation for servers group.
The group can be a column in the suggested table
Nir
On Sat, May 20, 2017 at 9:34 PM, Nir Sopher <ni...@qwilt.com>> wrote:
+1 for having a configuration allowing to apply the DS to the entire CDN.
Note that adding a "deploy by default" bool field to the DS itself may cause tenancy issues, as I believe this field should be in the control of the "CDN owner" tenant, and not of the "DS owner".
What about keeping the ds_server table, and allow "undef" for the server value - meaning "all servers in the CDN"?
In the future the table may grow and have additional fields to white-list servers by.
For example:
DS | Server | cache-group | profile
------+--------+-------------+--------
ds1 | s1 | null | null => ds to be applied on s1
------+--------+-------------+--------
ds2 | null | null | null => ds to be applied on all servers (in ds CDN)
------+--------+-------------+--------
ds3 | null | group1 | null => ds to be applied on all servers in cache group "group1"
------+--------+-------------+--------
ds4 | null | group1 | ATS-6.2 => ds to be applied on all servers in cache group "group1", with profile "ATS-6.2"
During CR config snapshot and configuration pull (or in advance), the required calculation would be done, to provide the list of relevant DSs per server.
Also note that moving in this direction is backward compatible as well as supports other non trivial extensions one may want in the future: Patterns (wildcards/regexs) support, as well as black list, using a "policy table" mechanism.
In the below example "ds5" is to be deployed on all caches, except for caches with "ATS-5" profile
DS | Server | cache-group | profile | deploy-decision
------+--------+-------------+---------+----------------
ds5 | null | null | ATS-5 | false
------+--------+-------------+---------+----------------
ds5 | null | null | null | true
------+--------+-------------+---------+----------------
Nir
On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <mi...@gmail.com>> wrote:
@dave - i'm not sure if you're suggesting:
A) on DS create, by default create an entry in the deliveryservice_server table for each cdn edge cache. so if there are 200 cdn edge caches, you end up with 200 entries in the deliveryservice_server table for that DS.
B) completely kill the deliveryservice_server table and it becomes implied that a DS uses ALL cdn edge caches (the way mids work today. mids don't have to be explicitly assigned to a DS)
"A" would allow you to control the scope of caches by using initial dispersion OR control the scope of caches by simply removing the ones you don't want to receive traffic.
"B" would only allow you to control the scope of caches by using initial dispersion (or so i think).
jeremy
On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org>> wrote:
+1, I think assigning DSs to all caches by default is fine. Then we can use initial dispersion to make sure traffic is spread between an appropriate number of caches.
On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <mi...@gmail.com>> wrote:
Bringing up an old email regarding DS to cache assignments.
It sounds like we want to preserve DS/cache assignments (the deliveryservice_server table) so you can assign a subset of cdn edge caches to a DS if you want. However, what if we make the assignment of ALL cdn edge caches to a new DS the default?
So when you create a DS for the Foo cdn thru the API, your new DS is automagically assigned all the edge caches in the Foo cdn...
(at that point you could always remove caches from a DS)
...or the create DS API could have a boolean flag added to it. for example, assignAllCaches: true|false. if you pass in true, all caches are assigned to the DS, if false, none are.
thoughts? I'm trying to think of self-service and i'm guessing most people will not want to assign caches to a DS...
jeremy
On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <mi...@gmail.com>> wrote:
gotcha. makes sense.
On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <da...@gmail.com>> wrote:
I believe Eric had a use case where he wanted to assign certain delivery services to certain caches.
If we just use initial dispersion and max DNS answers then we don't have the ability to control which servers are used, just how many.
On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <mi...@gmail.com>> wrote:
So what I heard was that potentially we move in this direction:
1. No need to explicitly assign edge caches to a delivery service. a delivery service (by default) would employ ALL edge caches of the cdn in which the ds belongs to.
2. Ability to override this default behavior and explicitly assign specific edge caches to a delivery service (this is what we do today)
But earlier Mark said "so the same functionality [as #2] can essentially be reached with assigning all caches, and setting the config params properly [initial dispersion and max dns answers]"
so why not ditch #2 in favor of using initial dispersion and max dns answers to control cache assignments?
it would be nice imo to get rid of the ds_server table. i'm always in favor of reducing complexity... :)
jeremy
On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <ef...@cisco.com>> wrote:
Yes - I think its a reasonable default to have a delivery service assigned to all caches as long as we still keep the ability to assign some DS’s to just a subset of caches.
—Eric
On Jan 4, 2017, at 4:56 PM, Mark Torluemke <mt...@apache.org>> wrote:
Good feedback, thanks. I think having _zero_ caches assigned to a delivery service could be added fairly easily, and could function as a good default to the assumption of _all_ caches being assigned. Would you agree with that?
Thanks,
Mark
On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <ef...@cisco.com>> wrote:
Yeah,
We have several some use cases where where cache or CG level allocation would be helpful.
1) Assigning new or trial delivery services to a subset of caches (considered beta cache nodes). This is helpful in trying out new DS configs before rolling them out to production.
2) Assigning DSs to caches can be a form of quota or resource reservation on the delivery service. Providers of content to the CDN can receive tiered service by either sharing caches or using dedicated caches.
—Eric
On Jan 4, 2017, at 2:04 PM, Mark Torluemke <mt...@apache.org>> wrote:
Today, there isn't an actual link from Cache Group to DS -- I think what
you are referring to is all done in the app layer to make a nice summary in
the UI to allow operators to very quickly assign all caches in a cache
group.
Do you use that "feature" to assign caches from some cache groups, and not
others?
On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <ef...@cisco.com>> wrote:
Would we keep the Cache Group -> DS assignment?
—Eric
> On Jan 3, 2017, at 9:22 AM, Mark Torluemke <mt...@apache.org>> wrote:
>
> Seeking community opinion on this.
>
> Should we drop the server -> delivery service assignments completely?
>
> When the project began, selectively assigning caches was the only way to selectively route traffic within a cache group -- let's say you have a DNS-routed service with a relatively large library size, you might not want assign all caches in a cache group to that service for fear you would 'pollute' those caches. Or, if you were turning up a 'test' DS of which you wanted to limit exposure on, you might apply the same logic. In recent releases, Traffic Control supports "initial dispersion" for _both_ DNS and HTTP-routed services, and "max DNS answers" for DNS-routed services, so the same functionality can essentially be reached with assigning all caches, and setting the config params properly.
>
> The motivation for this is to reduce the size & complexity of the Traffic Router's config (aka CRConfig), as this config scales mostly linearly with the number of caches and the number of delivery services in your CDN.
>
> Another option might be to not assign caches as the default, and individual cache assignments is available, but used only when necessary.
>
> Other thoughts? Clarifications needed?
>
> Thanks,
> Mark
Re: Drop Server -> Delivery Service assignments?
Posted by Nir Sopher <ni...@qwilt.com>.
Bringing this thread up again, following Eric's presentation for servers
group.
The group can be a column in the suggested table
Nir
On Sat, May 20, 2017 at 9:34 PM, Nir Sopher <ni...@qwilt.com> wrote:
> +1 for having a configuration allowing to apply the DS to the entire CDN.
>
> Note that adding a "deploy by default" bool field to the DS itself may
> cause tenancy issues, as I believe this field should be in the control of
> the "CDN owner" tenant, and not of the "DS owner".
>
> What about keeping the ds_server table, and allow "undef" for the server
> value - meaning "all servers in the CDN"?
> In the future the table may grow and have additional fields to white-list
> servers by.
> For example:
>
> DS | Server | cache-group | profile
> ------+--------+-------------+--------
> ds1 | s1 | null | null => ds to be applied on s1
> ------+--------+-------------+--------
> ds2 | null | null | null => ds to be applied on all
> servers (in ds CDN)
> ------+--------+-------------+--------
> ds3 | null | group1 | null => ds to be applied on all
> servers in cache group "group1"
> ------+--------+-------------+--------
> ds4 | null | group1 | ATS-6.2 => ds to be applied on all
> servers in cache group "group1", with profile "ATS-6.2"
>
>
> During CR config snapshot and configuration pull (or in advance), the
> required calculation would be done, to provide the list of relevant DSs per
> server.
>
> Also note that moving in this direction is backward compatible as well as
> supports other non trivial extensions one may want in the future: Patterns
> (wildcards/regexs) support, as well as black list, using a "policy table"
> mechanism.
>
> In the below example "ds5" is to be deployed on all caches, except for
> caches with "ATS-5" profile
>
> DS | Server | cache-group | profile | deploy-decision
> ------+--------+-------------+---------+----------------
> ds5 | null | null | ATS-5 | false
> ------+--------+-------------+---------+----------------
> ds5 | null | null | null | true
> ------+--------+-------------+---------+----------------
>
>
> Nir
>
> On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
>> @dave - i'm not sure if you're suggesting:
>>
>> A) on DS create, by default create an entry in the deliveryservice_server
>> table for each cdn edge cache. so if there are 200 cdn edge caches, you end
>> up with 200 entries in the deliveryservice_server table for that DS.
>> B) completely kill the deliveryservice_server table and it becomes
>> implied that a DS uses ALL cdn edge caches (the way mids work today.
>> mids don't have to be explicitly assigned to a DS)
>>
>> "A" would allow you to control the scope of caches by using initial
>> dispersion OR control the scope of caches by simply removing the ones
>> you don't want to receive traffic.
>> "B" would only allow you to control the scope of caches by using initial
>> dispersion (or so i think).
>>
>> jeremy
>>
>>
>>
>> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org> wrote:
>>
>>> +1, I think assigning DSs to all caches by default is fine. Then we can
>>> use initial dispersion to make sure traffic is spread between an
>>> appropriate number of caches.
>>>
>>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <mi...@gmail.com>
>>> wrote:
>>>
>>>> Bringing up an old email regarding DS to cache assignments.
>>>>
>>>> It sounds like we want to preserve DS/cache assignments (the
>>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
>>>> to a DS if you want. However, what if we make the assignment of ALL cdn
>>>> edge caches to a new DS the default?
>>>>
>>>> So when you create a DS for the Foo cdn thru the API, your new DS is
>>>> automagically assigned all the edge caches in the Foo cdn...
>>>>
>>>> (at that point you could always remove caches from a DS)
>>>>
>>>> ...or the create DS API could have a boolean flag added to it. for
>>>> example, assignAllCaches: true|false. if you pass in true, all caches are
>>>> assigned to the DS, if false, none are.
>>>>
>>>> thoughts? I'm trying to think of self-service and i'm guessing most
>>>> people will not want to assign caches to a DS...
>>>>
>>>> jeremy
>>>>
>>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <mitchell852@gmail.com
>>>> > wrote:
>>>>
>>>>> gotcha. makes sense.
>>>>>
>>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <david.neuman64@gmail.com
>>>>> > wrote:
>>>>>
>>>>>> I believe Eric had a use case where he wanted to assign certain
>>>>>> delivery services to certain caches.
>>>>>> If we just use initial dispersion and max DNS answers then we don't
>>>>>> have the ability to control which servers are used, just how many.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <
>>>>>> mitchell852@gmail.com> wrote:
>>>>>>
>>>>>>> So what I heard was that potentially we move in this direction:
>>>>>>>
>>>>>>> 1. No need to explicitly assign edge caches to a delivery service. a
>>>>>>> delivery service (by default) would employ ALL edge caches of the cdn in
>>>>>>> which the ds belongs to.
>>>>>>> 2. Ability to override this default behavior and explicitly assign
>>>>>>> specific edge caches to a delivery service (this is what we do today)
>>>>>>>
>>>>>>> But earlier Mark said "so the same functionality [as #2] can
>>>>>>> essentially be reached with assigning all caches, and setting the config
>>>>>>> params properly [initial dispersion and max dns answers]"
>>>>>>>
>>>>>>> so why not ditch #2 in favor of using initial dispersion and max
>>>>>>> dns answers to control cache assignments?
>>>>>>>
>>>>>>> it would be nice imo to get rid of the ds_server table. i'm always
>>>>>>> in favor of reducing complexity... :)
>>>>>>>
>>>>>>> jeremy
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>
>>>>>>>> Yes - I think its a reasonable default to have a delivery service
>>>>>>>> assigned to all caches as long as we still keep the ability to assign some
>>>>>>>> DS’s to just a subset of caches.
>>>>>>>>
>>>>>>>> —Eric
>>>>>>>>
>>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <mt...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Good feedback, thanks. I think having _zero_ caches assigned to a
>>>>>>>> delivery service could be added fairly easily, and could function as a good
>>>>>>>> default to the assumption of _all_ caches being assigned. Would you agree
>>>>>>>> with that?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Mark
>>>>>>>>
>>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>
>>>>>>>>> Yeah,
>>>>>>>>> We have several some use cases where where cache or CG level
>>>>>>>>> allocation would be helpful.
>>>>>>>>>
>>>>>>>>> 1) Assigning new or trial delivery services to a subset of caches
>>>>>>>>> (considered beta cache nodes). This is helpful in trying out new DS configs
>>>>>>>>> before rolling them out to production.
>>>>>>>>>
>>>>>>>>> 2) Assigning DSs to caches can be a form of quota or resource
>>>>>>>>> reservation on the delivery service. Providers of content to the CDN can
>>>>>>>>> receive tiered service by either sharing caches or using dedicated caches.
>>>>>>>>>
>>>>>>>>> —Eric
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <mt...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Today, there isn't an actual link from Cache Group to DS -- I
>>>>>>>>> think what
>>>>>>>>> you are referring to is all done in the app layer to make a nice
>>>>>>>>> summary in
>>>>>>>>> the UI to allow operators to very quickly assign all caches in a
>>>>>>>>> cache
>>>>>>>>> group.
>>>>>>>>>
>>>>>>>>> Do you use that "feature" to assign caches from some cache groups,
>>>>>>>>> and not
>>>>>>>>> others?
>>>>>>>>>
>>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
>>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>>
>>>>>>>>>> Would we keep the Cache Group -> DS assignment?
>>>>>>>>>>
>>>>>>>>>> —Eric
>>>>>>>>>>
>>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
>>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>>> >
>>>>>>>>>> > Seeking community opinion on this.
>>>>>>>>>> >
>>>>>>>>>> > Should we drop the server -> delivery service assignments
>>>>>>>>>> completely?
>>>>>>>>>> >
>>>>>>>>>> > When the project began, selectively assigning caches was the
>>>>>>>>>> only way to selectively route traffic within a cache group -- let's say you
>>>>>>>>>> have a DNS-routed service with a relatively large library size, you might
>>>>>>>>>> not want assign all caches in a cache group to that service for fear you
>>>>>>>>>> would 'pollute' those caches. Or, if you were turning up a 'test' DS of
>>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
>>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
>>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
>>>>>>>>>> services, so the same functionality can essentially be reached with
>>>>>>>>>> assigning all caches, and setting the config params properly.
>>>>>>>>>> >
>>>>>>>>>> > The motivation for this is to reduce the size & complexity of
>>>>>>>>>> the Traffic Router's config (aka CRConfig), as this config scales mostly
>>>>>>>>>> linearly with the number of caches and the number of delivery services in
>>>>>>>>>> your CDN.
>>>>>>>>>> >
>>>>>>>>>> > Another option might be to not assign caches as the default,
>>>>>>>>>> and individual cache assignments is available, but used only when necessary.
>>>>>>>>>> >
>>>>>>>>>> > Other thoughts? Clarifications needed?
>>>>>>>>>> >
>>>>>>>>>> > Thanks,
>>>>>>>>>> > Mark
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Drop Server -> Delivery Service assignments?
Posted by Nir Sopher <ni...@qwilt.com>.
+1 for having a configuration allowing to apply the DS to the entire CDN.
Note that adding a "deploy by default" bool field to the DS itself may
cause tenancy issues, as I believe this field should be in the control of
the "CDN owner" tenant, and not of the "DS owner".
What about keeping the ds_server table, and allow "undef" for the server
value - meaning "all servers in the CDN"?
In the future the table may grow and have additional fields to white-list
servers by.
For example:
DS | Server | cache-group | profile
------+--------+-------------+--------
ds1 | s1 | null | null => ds to be applied on s1
------+--------+-------------+--------
ds2 | null | null | null => ds to be applied on all
servers (in ds CDN)
------+--------+-------------+--------
ds3 | null | group1 | null => ds to be applied on all
servers in cache group "group1"
------+--------+-------------+--------
ds4 | null | group1 | ATS-6.2 => ds to be applied on all
servers in cache group "group1", with profile "ATS-6.2"
During CR config snapshot and configuration pull (or in advance), the
required calculation would be done, to provide the list of relevant DSs per
server.
Also note that moving in this direction is backward compatible as well as
supports other non trivial extensions one may want in the future: Patterns
(wildcards/regexs) support, as well as black list, using a "policy table"
mechanism.
In the below example "ds5" is to be deployed on all caches, except for
caches with "ATS-5" profile
DS | Server | cache-group | profile | deploy-decision
------+--------+-------------+---------+----------------
ds5 | null | null | ATS-5 | false
------+--------+-------------+---------+----------------
ds5 | null | null | null | true
------+--------+-------------+---------+----------------
Nir
On Sat, May 20, 2017 at 1:42 AM, Jeremy Mitchell <mi...@gmail.com>
wrote:
> @dave - i'm not sure if you're suggesting:
>
> A) on DS create, by default create an entry in the deliveryservice_server
> table for each cdn edge cache. so if there are 200 cdn edge caches, you end
> up with 200 entries in the deliveryservice_server table for that DS.
> B) completely kill the deliveryservice_server table and it becomes implied
> that a DS uses ALL cdn edge caches (the way mids work today. mids don't
> have to be explicitly assigned to a DS)
>
> "A" would allow you to control the scope of caches by using initial
> dispersion OR control the scope of caches by simply removing the ones you
> don't want to receive traffic.
> "B" would only allow you to control the scope of caches by using initial
> dispersion (or so i think).
>
> jeremy
>
>
>
> On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org> wrote:
>
>> +1, I think assigning DSs to all caches by default is fine. Then we can
>> use initial dispersion to make sure traffic is spread between an
>> appropriate number of caches.
>>
>> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <mi...@gmail.com>
>> wrote:
>>
>>> Bringing up an old email regarding DS to cache assignments.
>>>
>>> It sounds like we want to preserve DS/cache assignments (the
>>> deliveryservice_server table) so you can assign a subset of cdn edge caches
>>> to a DS if you want. However, what if we make the assignment of ALL cdn
>>> edge caches to a new DS the default?
>>>
>>> So when you create a DS for the Foo cdn thru the API, your new DS is
>>> automagically assigned all the edge caches in the Foo cdn...
>>>
>>> (at that point you could always remove caches from a DS)
>>>
>>> ...or the create DS API could have a boolean flag added to it. for
>>> example, assignAllCaches: true|false. if you pass in true, all caches are
>>> assigned to the DS, if false, none are.
>>>
>>> thoughts? I'm trying to think of self-service and i'm guessing most
>>> people will not want to assign caches to a DS...
>>>
>>> jeremy
>>>
>>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <mi...@gmail.com>
>>> wrote:
>>>
>>>> gotcha. makes sense.
>>>>
>>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <da...@gmail.com>
>>>> wrote:
>>>>
>>>>> I believe Eric had a use case where he wanted to assign certain
>>>>> delivery services to certain caches.
>>>>> If we just use initial dispersion and max DNS answers then we don't
>>>>> have the ability to control which servers are used, just how many.
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <mitchell852@gmail.com
>>>>> > wrote:
>>>>>
>>>>>> So what I heard was that potentially we move in this direction:
>>>>>>
>>>>>> 1. No need to explicitly assign edge caches to a delivery service. a
>>>>>> delivery service (by default) would employ ALL edge caches of the cdn in
>>>>>> which the ds belongs to.
>>>>>> 2. Ability to override this default behavior and explicitly assign
>>>>>> specific edge caches to a delivery service (this is what we do today)
>>>>>>
>>>>>> But earlier Mark said "so the same functionality [as #2] can
>>>>>> essentially be reached with assigning all caches, and setting the config
>>>>>> params properly [initial dispersion and max dns answers]"
>>>>>>
>>>>>> so why not ditch #2 in favor of using initial dispersion and max dns
>>>>>> answers to control cache assignments?
>>>>>>
>>>>>> it would be nice imo to get rid of the ds_server table. i'm always in
>>>>>> favor of reducing complexity... :)
>>>>>>
>>>>>> jeremy
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
>>>>>> efriedri@cisco.com> wrote:
>>>>>>
>>>>>>> Yes - I think its a reasonable default to have a delivery service
>>>>>>> assigned to all caches as long as we still keep the ability to assign some
>>>>>>> DS’s to just a subset of caches.
>>>>>>>
>>>>>>> —Eric
>>>>>>>
>>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <mt...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Good feedback, thanks. I think having _zero_ caches assigned to a
>>>>>>> delivery service could be added fairly easily, and could function as a good
>>>>>>> default to the assumption of _all_ caches being assigned. Would you agree
>>>>>>> with that?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Mark
>>>>>>>
>>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>
>>>>>>>> Yeah,
>>>>>>>> We have several some use cases where where cache or CG level
>>>>>>>> allocation would be helpful.
>>>>>>>>
>>>>>>>> 1) Assigning new or trial delivery services to a subset of caches
>>>>>>>> (considered beta cache nodes). This is helpful in trying out new DS configs
>>>>>>>> before rolling them out to production.
>>>>>>>>
>>>>>>>> 2) Assigning DSs to caches can be a form of quota or resource
>>>>>>>> reservation on the delivery service. Providers of content to the CDN can
>>>>>>>> receive tiered service by either sharing caches or using dedicated caches.
>>>>>>>>
>>>>>>>> —Eric
>>>>>>>>
>>>>>>>>
>>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <mt...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Today, there isn't an actual link from Cache Group to DS -- I think
>>>>>>>> what
>>>>>>>> you are referring to is all done in the app layer to make a nice
>>>>>>>> summary in
>>>>>>>> the UI to allow operators to very quickly assign all caches in a
>>>>>>>> cache
>>>>>>>> group.
>>>>>>>>
>>>>>>>> Do you use that "feature" to assign caches from some cache groups,
>>>>>>>> and not
>>>>>>>> others?
>>>>>>>>
>>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
>>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>>
>>>>>>>>> Would we keep the Cache Group -> DS assignment?
>>>>>>>>>
>>>>>>>>> —Eric
>>>>>>>>>
>>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <
>>>>>>>>> mtorluemke@apache.org> wrote:
>>>>>>>>> >
>>>>>>>>> > Seeking community opinion on this.
>>>>>>>>> >
>>>>>>>>> > Should we drop the server -> delivery service assignments
>>>>>>>>> completely?
>>>>>>>>> >
>>>>>>>>> > When the project began, selectively assigning caches was the
>>>>>>>>> only way to selectively route traffic within a cache group -- let's say you
>>>>>>>>> have a DNS-routed service with a relatively large library size, you might
>>>>>>>>> not want assign all caches in a cache group to that service for fear you
>>>>>>>>> would 'pollute' those caches. Or, if you were turning up a 'test' DS of
>>>>>>>>> which you wanted to limit exposure on, you might apply the same logic. In
>>>>>>>>> recent releases, Traffic Control supports "initial dispersion" for _both_
>>>>>>>>> DNS and HTTP-routed services, and "max DNS answers" for DNS-routed
>>>>>>>>> services, so the same functionality can essentially be reached with
>>>>>>>>> assigning all caches, and setting the config params properly.
>>>>>>>>> >
>>>>>>>>> > The motivation for this is to reduce the size & complexity of
>>>>>>>>> the Traffic Router's config (aka CRConfig), as this config scales mostly
>>>>>>>>> linearly with the number of caches and the number of delivery services in
>>>>>>>>> your CDN.
>>>>>>>>> >
>>>>>>>>> > Another option might be to not assign caches as the default, and
>>>>>>>>> individual cache assignments is available, but used only when necessary.
>>>>>>>>> >
>>>>>>>>> > Other thoughts? Clarifications needed?
>>>>>>>>> >
>>>>>>>>> > Thanks,
>>>>>>>>> > Mark
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Drop Server -> Delivery Service assignments?
Posted by Jeremy Mitchell <mi...@gmail.com>.
@dave - i'm not sure if you're suggesting:
A) on DS create, by default create an entry in the deliveryservice_server
table for each cdn edge cache. so if there are 200 cdn edge caches, you end
up with 200 entries in the deliveryservice_server table for that DS.
B) completely kill the deliveryservice_server table and it becomes implied
that a DS uses ALL cdn edge caches (the way mids work today. mids don't
have to be explicitly assigned to a DS)
"A" would allow you to control the scope of caches by using initial
dispersion OR control the scope of caches by simply removing the ones you
don't want to receive traffic.
"B" would only allow you to control the scope of caches by using initial
dispersion (or so i think).
jeremy
On Fri, May 19, 2017 at 4:25 PM, Dave Neuman <ne...@apache.org> wrote:
> +1, I think assigning DSs to all caches by default is fine. Then we can
> use initial dispersion to make sure traffic is spread between an
> appropriate number of caches.
>
> On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
>> Bringing up an old email regarding DS to cache assignments.
>>
>> It sounds like we want to preserve DS/cache assignments (the
>> deliveryservice_server table) so you can assign a subset of cdn edge caches
>> to a DS if you want. However, what if we make the assignment of ALL cdn
>> edge caches to a new DS the default?
>>
>> So when you create a DS for the Foo cdn thru the API, your new DS is
>> automagically assigned all the edge caches in the Foo cdn...
>>
>> (at that point you could always remove caches from a DS)
>>
>> ...or the create DS API could have a boolean flag added to it. for
>> example, assignAllCaches: true|false. if you pass in true, all caches are
>> assigned to the DS, if false, none are.
>>
>> thoughts? I'm trying to think of self-service and i'm guessing most
>> people will not want to assign caches to a DS...
>>
>> jeremy
>>
>> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <mi...@gmail.com>
>> wrote:
>>
>>> gotcha. makes sense.
>>>
>>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <da...@gmail.com>
>>> wrote:
>>>
>>>> I believe Eric had a use case where he wanted to assign certain
>>>> delivery services to certain caches.
>>>> If we just use initial dispersion and max DNS answers then we don't
>>>> have the ability to control which servers are used, just how many.
>>>>
>>>>
>>>>
>>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <mi...@gmail.com>
>>>> wrote:
>>>>
>>>>> So what I heard was that potentially we move in this direction:
>>>>>
>>>>> 1. No need to explicitly assign edge caches to a delivery service. a
>>>>> delivery service (by default) would employ ALL edge caches of the cdn in
>>>>> which the ds belongs to.
>>>>> 2. Ability to override this default behavior and explicitly assign
>>>>> specific edge caches to a delivery service (this is what we do today)
>>>>>
>>>>> But earlier Mark said "so the same functionality [as #2] can
>>>>> essentially be reached with assigning all caches, and setting the config
>>>>> params properly [initial dispersion and max dns answers]"
>>>>>
>>>>> so why not ditch #2 in favor of using initial dispersion and max dns
>>>>> answers to control cache assignments?
>>>>>
>>>>> it would be nice imo to get rid of the ds_server table. i'm always in
>>>>> favor of reducing complexity... :)
>>>>>
>>>>> jeremy
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
>>>>> efriedri@cisco.com> wrote:
>>>>>
>>>>>> Yes - I think its a reasonable default to have a delivery service
>>>>>> assigned to all caches as long as we still keep the ability to assign some
>>>>>> DS’s to just a subset of caches.
>>>>>>
>>>>>> —Eric
>>>>>>
>>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <mt...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>> Good feedback, thanks. I think having _zero_ caches assigned to a
>>>>>> delivery service could be added fairly easily, and could function as a good
>>>>>> default to the assumption of _all_ caches being assigned. Would you agree
>>>>>> with that?
>>>>>>
>>>>>> Thanks,
>>>>>> Mark
>>>>>>
>>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
>>>>>> efriedri@cisco.com> wrote:
>>>>>>
>>>>>>> Yeah,
>>>>>>> We have several some use cases where where cache or CG level
>>>>>>> allocation would be helpful.
>>>>>>>
>>>>>>> 1) Assigning new or trial delivery services to a subset of caches
>>>>>>> (considered beta cache nodes). This is helpful in trying out new DS configs
>>>>>>> before rolling them out to production.
>>>>>>>
>>>>>>> 2) Assigning DSs to caches can be a form of quota or resource
>>>>>>> reservation on the delivery service. Providers of content to the CDN can
>>>>>>> receive tiered service by either sharing caches or using dedicated caches.
>>>>>>>
>>>>>>> —Eric
>>>>>>>
>>>>>>>
>>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <mt...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Today, there isn't an actual link from Cache Group to DS -- I think
>>>>>>> what
>>>>>>> you are referring to is all done in the app layer to make a nice
>>>>>>> summary in
>>>>>>> the UI to allow operators to very quickly assign all caches in a
>>>>>>> cache
>>>>>>> group.
>>>>>>>
>>>>>>> Do you use that "feature" to assign caches from some cache groups,
>>>>>>> and not
>>>>>>> others?
>>>>>>>
>>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
>>>>>>> efriedri@cisco.com> wrote:
>>>>>>>
>>>>>>>> Would we keep the Cache Group -> DS assignment?
>>>>>>>>
>>>>>>>> —Eric
>>>>>>>>
>>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <mt...@apache.org>
>>>>>>>> wrote:
>>>>>>>> >
>>>>>>>> > Seeking community opinion on this.
>>>>>>>> >
>>>>>>>> > Should we drop the server -> delivery service assignments
>>>>>>>> completely?
>>>>>>>> >
>>>>>>>> > When the project began, selectively assigning caches was the only
>>>>>>>> way to selectively route traffic within a cache group -- let's say you have
>>>>>>>> a DNS-routed service with a relatively large library size, you might not
>>>>>>>> want assign all caches in a cache group to that service for fear you would
>>>>>>>> 'pollute' those caches. Or, if you were turning up a 'test' DS of which you
>>>>>>>> wanted to limit exposure on, you might apply the same logic. In recent
>>>>>>>> releases, Traffic Control supports "initial dispersion" for _both_ DNS and
>>>>>>>> HTTP-routed services, and "max DNS answers" for DNS-routed services, so the
>>>>>>>> same functionality can essentially be reached with assigning all caches,
>>>>>>>> and setting the config params properly.
>>>>>>>> >
>>>>>>>> > The motivation for this is to reduce the size & complexity of the
>>>>>>>> Traffic Router's config (aka CRConfig), as this config scales mostly
>>>>>>>> linearly with the number of caches and the number of delivery services in
>>>>>>>> your CDN.
>>>>>>>> >
>>>>>>>> > Another option might be to not assign caches as the default, and
>>>>>>>> individual cache assignments is available, but used only when necessary.
>>>>>>>> >
>>>>>>>> > Other thoughts? Clarifications needed?
>>>>>>>> >
>>>>>>>> > Thanks,
>>>>>>>> > Mark
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Drop Server -> Delivery Service assignments?
Posted by Dave Neuman <ne...@apache.org>.
+1, I think assigning DSs to all caches by default is fine. Then we can
use initial dispersion to make sure traffic is spread between an
appropriate number of caches.
On Fri, May 19, 2017 at 4:09 PM, Jeremy Mitchell <mi...@gmail.com>
wrote:
> Bringing up an old email regarding DS to cache assignments.
>
> It sounds like we want to preserve DS/cache assignments (the
> deliveryservice_server table) so you can assign a subset of cdn edge caches
> to a DS if you want. However, what if we make the assignment of ALL cdn
> edge caches to a new DS the default?
>
> So when you create a DS for the Foo cdn thru the API, your new DS is
> automagically assigned all the edge caches in the Foo cdn...
>
> (at that point you could always remove caches from a DS)
>
> ...or the create DS API could have a boolean flag added to it. for
> example, assignAllCaches: true|false. if you pass in true, all caches are
> assigned to the DS, if false, none are.
>
> thoughts? I'm trying to think of self-service and i'm guessing most people
> will not want to assign caches to a DS...
>
> jeremy
>
> On Fri, Jan 6, 2017 at 11:46 AM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
>> gotcha. makes sense.
>>
>> On Fri, Jan 6, 2017 at 9:40 AM, David Neuman <da...@gmail.com>
>> wrote:
>>
>>> I believe Eric had a use case where he wanted to assign certain delivery
>>> services to certain caches.
>>> If we just use initial dispersion and max DNS answers then we don't have
>>> the ability to control which servers are used, just how many.
>>>
>>>
>>>
>>> On Fri, Jan 6, 2017 at 9:23 AM, Jeremy Mitchell <mi...@gmail.com>
>>> wrote:
>>>
>>>> So what I heard was that potentially we move in this direction:
>>>>
>>>> 1. No need to explicitly assign edge caches to a delivery service. a
>>>> delivery service (by default) would employ ALL edge caches of the cdn in
>>>> which the ds belongs to.
>>>> 2. Ability to override this default behavior and explicitly assign
>>>> specific edge caches to a delivery service (this is what we do today)
>>>>
>>>> But earlier Mark said "so the same functionality [as #2] can
>>>> essentially be reached with assigning all caches, and setting the config
>>>> params properly [initial dispersion and max dns answers]"
>>>>
>>>> so why not ditch #2 in favor of using initial dispersion and max dns
>>>> answers to control cache assignments?
>>>>
>>>> it would be nice imo to get rid of the ds_server table. i'm always in
>>>> favor of reducing complexity... :)
>>>>
>>>> jeremy
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jan 5, 2017 at 9:08 AM, Eric Friedrich (efriedri) <
>>>> efriedri@cisco.com> wrote:
>>>>
>>>>> Yes - I think its a reasonable default to have a delivery service
>>>>> assigned to all caches as long as we still keep the ability to assign some
>>>>> DS’s to just a subset of caches.
>>>>>
>>>>> —Eric
>>>>>
>>>>> On Jan 4, 2017, at 4:56 PM, Mark Torluemke <mt...@apache.org>
>>>>> wrote:
>>>>>
>>>>> Good feedback, thanks. I think having _zero_ caches assigned to a
>>>>> delivery service could be added fairly easily, and could function as a good
>>>>> default to the assumption of _all_ caches being assigned. Would you agree
>>>>> with that?
>>>>>
>>>>> Thanks,
>>>>> Mark
>>>>>
>>>>> On Wed, Jan 4, 2017 at 1:55 PM, Eric Friedrich (efriedri) <
>>>>> efriedri@cisco.com> wrote:
>>>>>
>>>>>> Yeah,
>>>>>> We have several some use cases where where cache or CG level
>>>>>> allocation would be helpful.
>>>>>>
>>>>>> 1) Assigning new or trial delivery services to a subset of caches
>>>>>> (considered beta cache nodes). This is helpful in trying out new DS configs
>>>>>> before rolling them out to production.
>>>>>>
>>>>>> 2) Assigning DSs to caches can be a form of quota or resource
>>>>>> reservation on the delivery service. Providers of content to the CDN can
>>>>>> receive tiered service by either sharing caches or using dedicated caches.
>>>>>>
>>>>>> —Eric
>>>>>>
>>>>>>
>>>>>> On Jan 4, 2017, at 2:04 PM, Mark Torluemke <mt...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>> Today, there isn't an actual link from Cache Group to DS -- I think
>>>>>> what
>>>>>> you are referring to is all done in the app layer to make a nice
>>>>>> summary in
>>>>>> the UI to allow operators to very quickly assign all caches in a cache
>>>>>> group.
>>>>>>
>>>>>> Do you use that "feature" to assign caches from some cache groups,
>>>>>> and not
>>>>>> others?
>>>>>>
>>>>>> On Tue, Jan 3, 2017 at 9:50 AM, Eric Friedrich (efriedri) <
>>>>>> efriedri@cisco.com> wrote:
>>>>>>
>>>>>>> Would we keep the Cache Group -> DS assignment?
>>>>>>>
>>>>>>> —Eric
>>>>>>>
>>>>>>> > On Jan 3, 2017, at 9:22 AM, Mark Torluemke <mt...@apache.org>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > Seeking community opinion on this.
>>>>>>> >
>>>>>>> > Should we drop the server -> delivery service assignments
>>>>>>> completely?
>>>>>>> >
>>>>>>> > When the project began, selectively assigning caches was the only
>>>>>>> way to selectively route traffic within a cache group -- let's say you have
>>>>>>> a DNS-routed service with a relatively large library size, you might not
>>>>>>> want assign all caches in a cache group to that service for fear you would
>>>>>>> 'pollute' those caches. Or, if you were turning up a 'test' DS of which you
>>>>>>> wanted to limit exposure on, you might apply the same logic. In recent
>>>>>>> releases, Traffic Control supports "initial dispersion" for _both_ DNS and
>>>>>>> HTTP-routed services, and "max DNS answers" for DNS-routed services, so the
>>>>>>> same functionality can essentially be reached with assigning all caches,
>>>>>>> and setting the config params properly.
>>>>>>> >
>>>>>>> > The motivation for this is to reduce the size & complexity of the
>>>>>>> Traffic Router's config (aka CRConfig), as this config scales mostly
>>>>>>> linearly with the number of caches and the number of delivery services in
>>>>>>> your CDN.
>>>>>>> >
>>>>>>> > Another option might be to not assign caches as the default, and
>>>>>>> individual cache assignments is available, but used only when necessary.
>>>>>>> >
>>>>>>> > Other thoughts? Clarifications needed?
>>>>>>> >
>>>>>>> > Thanks,
>>>>>>> > Mark
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>