TO - Queue server updates on delivery service servers only?

[Jeremy Mitchell <mi...@apache.org>]

Currently, you can queue updates:

- for a specific server
- for a specific cachegroup (all the servers in that cachegroup)
- for a specific cdn (all the servers in that cdn)

but you can't queue updates for:

- a specific delivery service (all the servers explicitly (edges) or
implicitly (mids) assigned to that DS)

Does it make sense to add this functionality? At least to the API?

I "think" when a delivery service change is made, it is common practice to
queue updates on the ENTIRE cdn that the DS belongs to. This seems like it
would unnecessarily queue updates for a lot of servers that don't belong to
the DS.

I know we may move away from the queue update approach at some point but
does this functionality make sense in the short term?

Jeremy

Re: TO - Queue server updates on delivery service servers only?

[Dave Neuman <ne...@apache.org>]

In the near future we plan to make the DS -> Server mapping optional.  This
means that we assign all DSs to all caches and rely on Traffic Router to do
the right thing using consistent hashing and dispersion.  If we assign a DS
to all servers then queueing updates for a DS is not really needed, since
you would need to queue the whole CDN anyway.

On Tue, Feb 7, 2017 at 2:03 AM, Nir Sopher <ni...@qwilt.com> wrote:

> Hi Jeremy,
>
> As I see it, we need to distinguish delivery-service changes that require a
> cr-config update, from those who doesn't.
>
> The simple case, where the changes does not effect on the cr-config, a
> "queue-update" of a delivery-service may be simple. Just going over the
> list of servers assigned to the DS and queue an update for each of them.
> Nevertheless, my understanding is that this solution requires further
> improvements in order to allow dealing with more than one delivery services
> at a time.
>
> The servers removal issue you discussed, is one of the cases in which the
> change also effects the cr-config - where I believe things get more
> complicated, as the cr-config is done on a CDN level and a solution dealing
> only with the queue-updates is more dangerous as it may cause
> inconsistencies and "black hole".
>
> Still, I believe these issues can be simply resolved along with TC-130
> <https://issues.apache.org/jira/browse/TC-130>applying the ideas
> discussed in
> the 2 "Streamlining TC management and operations sequences" threads.
> I would take the opportunity to thank Eric again for his inputs on these
> threads.
>
> When trying to highlight the foundations of the solution I would say:
>
>    1. Manage separately the configuration per delivery-service (in both
>    queue-update and cr-config)
>    2. Define some kind of "configuration generation", incremented on a DS
>    apply and managed per DS.
>    3. Keep track of each traffic-server - knowing/learning the held
>    configuration generation for each DS.
>
> Using these tools, when applying the configuration of a delivery-service,
> the system drives the traffic servers to
> 1. If the traffic-servers is assigned to the delivery-service - hold the
> latest generation.
> 2. O.w. hold no configuration for this delivery service (== "0"
> generation).
>
> As the system knows which generation is *currently* held by which server,
> as well as which generation *should* be held by which server, it can update
> only the servers needs to be updated.
> Specifically for the case you described, servers that are un-assigned from
> the service, would be updated as well to remove the cfg.
>
> What do you think?
> Nir
>
> On Fri, Feb 3, 2017 at 4:50 PM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
> > The more I think about this, the more I think it's not as straightforward
> > as I think to queue updates for a delivery service (meaning to queue
> > updates for all the caches employed by the ds).
> >
> > For example, imagine you had a delivery service that employed 10
> > caches....and you removed 2 caches from the ds...then queue'd updates for
> > the ds.....this would queue updates for the 8 remaining caches but really
> > the 2 caches you removed are the ones that need the updates...
> >
> > So unless told otherwise or someone has a solution to this problem (I
> can't
> > think of one currently), I'm gonna hold off on that issue.
> >
> > Jeremy
> >
> > On Wed, Feb 1, 2017 at 8:58 PM, Jeremy Mitchell <mi...@gmail.com>
> > wrote:
> >
> > > Created this issue. Would like input from more people to ensure this
> is a
> > > good idea and I'm not overlooking something...
> > >
> > > https://issues.apache.org/jira/browse/TC-129
> > >
> > > On Tue, Jan 31, 2017 at 10:47 AM, Eric Friedrich (efriedri) <
> > > efriedri@cisco.com> wrote:
> > >
> > >> Yes, when modifying a DS, it would be useful to have an API to queue
> on
> > >> just servers relevant to that DS.
> > >>
> > >> —Eric
> > >>
> > >> > On Jan 31, 2017, at 12:36 PM, Jeremy Mitchell <
> mitchell852@apache.org
> > >
> > >> wrote:
> > >> >
> > >> > Currently, you can queue updates:
> > >> >
> > >> > - for a specific server
> > >> > - for a specific cachegroup (all the servers in that cachegroup)
> > >> > - for a specific cdn (all the servers in that cdn)
> > >> >
> > >> > but you can't queue updates for:
> > >> >
> > >> > - a specific delivery service (all the servers explicitly (edges) or
> > >> > implicitly (mids) assigned to that DS)
> > >> >
> > >> > Does it make sense to add this functionality? At least to the API?
> > >> >
> > >> > I "think" when a delivery service change is made, it is common
> > practice
> > >> to
> > >> > queue updates on the ENTIRE cdn that the DS belongs to. This seems
> > like
> > >> it
> > >> > would unnecessarily queue updates for a lot of servers that don't
> > >> belong to
> > >> > the DS.
> > >> >
> > >> > I know we may move away from the queue update approach at some point
> > but
> > >> > does this functionality make sense in the short term?
> > >> >
> > >> > Jeremy
> > >>
> > >>
> > >
> >
>

Re: TO - Queue server updates on delivery service servers only?

[Nir Sopher <ni...@qwilt.com>]

Hi Jeremy,

As I see it, we need to distinguish delivery-service changes that require a
cr-config update, from those who doesn't.

The simple case, where the changes does not effect on the cr-config, a
"queue-update" of a delivery-service may be simple. Just going over the
list of servers assigned to the DS and queue an update for each of them.
Nevertheless, my understanding is that this solution requires further
improvements in order to allow dealing with more than one delivery services
at a time.

The servers removal issue you discussed, is one of the cases in which the
change also effects the cr-config - where I believe things get more
complicated, as the cr-config is done on a CDN level and a solution dealing
only with the queue-updates is more dangerous as it may cause
inconsistencies and "black hole".

Still, I believe these issues can be simply resolved along with TC-130
<https://issues.apache.org/jira/browse/TC-130>applying the ideas discussed in
the 2 "Streamlining TC management and operations sequences" threads.
I would take the opportunity to thank Eric again for his inputs on these
threads.

When trying to highlight the foundations of the solution I would say:

   1. Manage separately the configuration per delivery-service (in both
   queue-update and cr-config)
   2. Define some kind of "configuration generation", incremented on a DS
   apply and managed per DS.
   3. Keep track of each traffic-server - knowing/learning the held
   configuration generation for each DS.

Using these tools, when applying the configuration of a delivery-service,
the system drives the traffic servers to
1. If the traffic-servers is assigned to the delivery-service - hold the
latest generation.
2. O.w. hold no configuration for this delivery service (== "0" generation).

As the system knows which generation is *currently* held by which server,
as well as which generation *should* be held by which server, it can update
only the servers needs to be updated.
Specifically for the case you described, servers that are un-assigned from
the service, would be updated as well to remove the cfg.

What do you think?
Nir

On Fri, Feb 3, 2017 at 4:50 PM, Jeremy Mitchell <mi...@gmail.com>
wrote:

> The more I think about this, the more I think it's not as straightforward
> as I think to queue updates for a delivery service (meaning to queue
> updates for all the caches employed by the ds).
>
> For example, imagine you had a delivery service that employed 10
> caches....and you removed 2 caches from the ds...then queue'd updates for
> the ds.....this would queue updates for the 8 remaining caches but really
> the 2 caches you removed are the ones that need the updates...
>
> So unless told otherwise or someone has a solution to this problem (I can't
> think of one currently), I'm gonna hold off on that issue.
>
> Jeremy
>
> On Wed, Feb 1, 2017 at 8:58 PM, Jeremy Mitchell <mi...@gmail.com>
> wrote:
>
> > Created this issue. Would like input from more people to ensure this is a
> > good idea and I'm not overlooking something...
> >
> > https://issues.apache.org/jira/browse/TC-129
> >
> > On Tue, Jan 31, 2017 at 10:47 AM, Eric Friedrich (efriedri) <
> > efriedri@cisco.com> wrote:
> >
> >> Yes, when modifying a DS, it would be useful to have an API to queue on
> >> just servers relevant to that DS.
> >>
> >> —Eric
> >>
> >> > On Jan 31, 2017, at 12:36 PM, Jeremy Mitchell <mitchell852@apache.org
> >
> >> wrote:
> >> >
> >> > Currently, you can queue updates:
> >> >
> >> > - for a specific server
> >> > - for a specific cachegroup (all the servers in that cachegroup)
> >> > - for a specific cdn (all the servers in that cdn)
> >> >
> >> > but you can't queue updates for:
> >> >
> >> > - a specific delivery service (all the servers explicitly (edges) or
> >> > implicitly (mids) assigned to that DS)
> >> >
> >> > Does it make sense to add this functionality? At least to the API?
> >> >
> >> > I "think" when a delivery service change is made, it is common
> practice
> >> to
> >> > queue updates on the ENTIRE cdn that the DS belongs to. This seems
> like
> >> it
> >> > would unnecessarily queue updates for a lot of servers that don't
> >> belong to
> >> > the DS.
> >> >
> >> > I know we may move away from the queue update approach at some point
> but
> >> > does this functionality make sense in the short term?
> >> >
> >> > Jeremy
> >>
> >>
> >
>

Re: TO - Queue server updates on delivery service servers only?

[Jeremy Mitchell <mi...@gmail.com>]

The more I think about this, the more I think it's not as straightforward
as I think to queue updates for a delivery service (meaning to queue
updates for all the caches employed by the ds).

For example, imagine you had a delivery service that employed 10
caches....and you removed 2 caches from the ds...then queue'd updates for
the ds.....this would queue updates for the 8 remaining caches but really
the 2 caches you removed are the ones that need the updates...

So unless told otherwise or someone has a solution to this problem (I can't
think of one currently), I'm gonna hold off on that issue.

Jeremy

On Wed, Feb 1, 2017 at 8:58 PM, Jeremy Mitchell <mi...@gmail.com>
wrote:

> Created this issue. Would like input from more people to ensure this is a
> good idea and I'm not overlooking something...
>
> https://issues.apache.org/jira/browse/TC-129
>
> On Tue, Jan 31, 2017 at 10:47 AM, Eric Friedrich (efriedri) <
> efriedri@cisco.com> wrote:
>
>> Yes, when modifying a DS, it would be useful to have an API to queue on
>> just servers relevant to that DS.
>>
>> —Eric
>>
>> > On Jan 31, 2017, at 12:36 PM, Jeremy Mitchell <mi...@apache.org>
>> wrote:
>> >
>> > Currently, you can queue updates:
>> >
>> > - for a specific server
>> > - for a specific cachegroup (all the servers in that cachegroup)
>> > - for a specific cdn (all the servers in that cdn)
>> >
>> > but you can't queue updates for:
>> >
>> > - a specific delivery service (all the servers explicitly (edges) or
>> > implicitly (mids) assigned to that DS)
>> >
>> > Does it make sense to add this functionality? At least to the API?
>> >
>> > I "think" when a delivery service change is made, it is common practice
>> to
>> > queue updates on the ENTIRE cdn that the DS belongs to. This seems like
>> it
>> > would unnecessarily queue updates for a lot of servers that don't
>> belong to
>> > the DS.
>> >
>> > I know we may move away from the queue update approach at some point but
>> > does this functionality make sense in the short term?
>> >
>> > Jeremy
>>
>>
>

Re: TO - Queue server updates on delivery service servers only?

[Jeremy Mitchell <mi...@gmail.com>]

Created this issue. Would like input from more people to ensure this is a
good idea and I'm not overlooking something...

https://issues.apache.org/jira/browse/TC-129

On Tue, Jan 31, 2017 at 10:47 AM, Eric Friedrich (efriedri) <
efriedri@cisco.com> wrote:

> Yes, when modifying a DS, it would be useful to have an API to queue on
> just servers relevant to that DS.
>
> —Eric
>
> > On Jan 31, 2017, at 12:36 PM, Jeremy Mitchell <mi...@apache.org>
> wrote:
> >
> > Currently, you can queue updates:
> >
> > - for a specific server
> > - for a specific cachegroup (all the servers in that cachegroup)
> > - for a specific cdn (all the servers in that cdn)
> >
> > but you can't queue updates for:
> >
> > - a specific delivery service (all the servers explicitly (edges) or
> > implicitly (mids) assigned to that DS)
> >
> > Does it make sense to add this functionality? At least to the API?
> >
> > I "think" when a delivery service change is made, it is common practice
> to
> > queue updates on the ENTIRE cdn that the DS belongs to. This seems like
> it
> > would unnecessarily queue updates for a lot of servers that don't belong
> to
> > the DS.
> >
> > I know we may move away from the queue update approach at some point but
> > does this functionality make sense in the short term?
> >
> > Jeremy
>
>

Re: TO - Queue server updates on delivery service servers only?

["Eric Friedrich (efriedri)" <ef...@cisco.com>]

Yes, when modifying a DS, it would be useful to have an API to queue on just servers relevant to that DS. 

—Eric

> On Jan 31, 2017, at 12:36 PM, Jeremy Mitchell <mi...@apache.org> wrote:
> 
> Currently, you can queue updates:
> 
> - for a specific server
> - for a specific cachegroup (all the servers in that cachegroup)
> - for a specific cdn (all the servers in that cdn)
> 
> but you can't queue updates for:
> 
> - a specific delivery service (all the servers explicitly (edges) or
> implicitly (mids) assigned to that DS)
> 
> Does it make sense to add this functionality? At least to the API?
> 
> I "think" when a delivery service change is made, it is common practice to
> queue updates on the ENTIRE cdn that the DS belongs to. This seems like it
> would unnecessarily queue updates for a lot of servers that don't belong to
> the DS.
> 
> I know we may move away from the queue update approach at some point but
> does this functionality make sense in the short term?
> 
> Jeremy