You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficcontrol.apache.org by Jeremy Mitchell <mi...@apache.org> on 2018/02/02 20:43:31 UTC
Delivery Service Self-Service
As we move in the direction of self-service, there are a few obstacles that
need to be overcome and I'd like to discuss them a bit so grab a cup of
coffee...
When I say self-service, what I really mean is "delivery service
self-service" or the ability to manage your own delivery services (as
dictated by tenancy) and everything related to those delivery services.
"Everything" includes the following (afaik):
1. Create/Read/Update/Delete delivery services
2. Manage DS regexes
3. Manage DS SSL keys (if applicable)
4. Manage DS URL sig keys (if applicable)
5. Manage DS URI signing keys (if applicable)
6. Manage DS targets (steering* only)
7. Creating DS invalidate content jobs
8. Manage DS / cache assignments
If you can't do 1-7 yourself, it's not really self-service is it? #8 is
debatable.
I'll discuss each one:
1. Create/Read/Update/Delete delivery services
Ideally, you could CRUD your own delivery services but our system has some
limitations.
A) Our CDN is not properly insulated from bad DS configurations. If a user
enters a bad value, bad things could potentially happen to a cache or worse
the whole CDN.
B) Certain DS configuration changes requires queue updates and/or snapshot
for the change to take effect. We're not ready (nor will we ever be
probably) to let normal users queue updates and/or snapshot.
So in the interim, we're working on the ability to allow normal users to
create "delivery service requests" to facilitate creating/updating/deleting
a delivery service. These "requests" will have to be reviewed/fulfilled by
a higher level user (Ops or Admin) who can then queue/snapshot if needed.
2. Manage DS regexes
Here's an explanation of this:
http://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_ops/using.html#delivery-service-regexp
Currently, this requires the Operations role and for good reason. The
danger here involves the risk of a normal user entering a bad regex. For
example, it is my understanding that the regex in position zero needs to
always follow this format: .*\.foo\..*.
Maybe with some better API validation we could let normal users manage DS
regexes....or maybe these end up going away in favor of something
better/easier...not sure yet...
3. Manage DS SSL keys
SSL keys are only applicable where protocol > 0 (HTTPS, HTTP AND HTTPS, or
HTTP TO HTTPS) and currently, to manage them requires the Admin role. Why?
I'm not sure. Is their harm in letting normal users manage their own SSL
keys?
4. Manage DS URL sig keys
URL sig keys (
http://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_ops/using.html#generate-url-sig-keys)
are only applicable where signingAlgorithm = 'url_sig' and currently, to
manage them requires NO role apparently (only a tenancy check is
performed). This is the 1st one on the list that a "normal" user can do
currently.
5. Manage DS URI signing keys
URI signing keys are only applicable where signingAlgorithm = 'uri_signing'
and currently, to manage DS URI signing keys requires the Admin role. Is it
necessary to restrict this functionality to Admins only or can we allow
normal users to mange URI signing keys for their DS's?
6. Manage DS targets (steering* only)
Here's an explanation of this:
http://traffic-control-cdn.readthedocs.io/en/latest/admin/quick_howto/steering.html?highlight=steering
Currently, to manage DS targets requires the Admin or Steering role. Is
there any harm in allowing a normal user to "steer" their delivery service
to another delivery service as long as the target delivery service falls in
their tenancy?
7. Creating DS invalidate content jobs
http://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_ops/using.html?highlight=invalidate#invalidate-content
You can currently do this for your own DS's. This is the 2nd one on the
list that a "normal" user can do currently.
8. Manage DS / cache assignments
This is the debatable one. To provide true delivery service self-service
should a user have the ability to determine which caches are assigned to
their delivery service. I'm thinking NO. Currently, this action requires
the Ops role and I'm in favor of leaving it that way...
In summary, to provide true delivery service self-service I think we need
to do a few things:
1. Introduce DS requests until the day in which DS configurations can be
guaranteed and queue updates/snapshot becomes a thing of the past. (this is
in progress)
2. Revisit DS regexes or make their management more fool proof.
3. Tweak the roles of these actions. Currently, a lot of these things are
reserved for Ops/Admin. We have to change that or full DS self-service if
not possible. I'd like to make each of these things accessible to users
with the "Portal" role with the exception of #8.
Thoughts? Concerns? Funny jokes?
Jeremy
Re: Delivery Service Self-Service
Posted by "Durfey, Ryan" <Ry...@comcast.com>.
Awesome work Jeremy! Thank you and the team for keeping this moving forward.
1. Create/Read/Update/Delete delivery services
* Agree
2. Manage DS regexes
* Many customers CNAME to us, so we need this to be configurable at some level
* I think we should restrict users when entering additional host names to explicit full domain names, ie not allow them to enter regex matching patterns
* This restricts the overlap potential and allows for collision checks against existing explicit names and regexes
* Allow internal staff to enter regex matching patterns
3. Manage DS SSL keys (if applicable)
* Agree, let users manage keys
* Would like to allow users to upload their own certs as well
4. Manage DS URL sig keys (if applicable)
1. We are making the “sig-anchor” a configurable option in the next release so they need to be able to manipulate this
2. We also need to give them access to manipulate the URL signing bypass parameters which are regex values, we typically bypass signing for crossdomain.xml and clientaccesspolicy.xml or for a particular directory like where they store their thumbnails.
5. Manage DS URI signing keys (if applicable)
1. Agree we should let them manage the keys
6. Manage DS targets (steering* only)
1. Don’t have an opinion on this, haven’t used it.
7. Creating DS invalidate content jobs
1. I want to eventually add purge or true delete as an option but otherwise this is good.
8. Manage DS / cache assignments
1. Agree, should only be controlled by CDN operations
Ryan Durfey M | 303-524-5099
CDN Support (24x7): 866-405-2993 or cdn_support@comcast.com<ma...@comcast.com>
From: Jeremy Mitchell <mi...@apache.org>
Reply-To: "dev@trafficcontrol.incubator.apache.org" <de...@trafficcontrol.incubator.apache.org>
Date: Friday, February 2, 2018 at 1:50 PM
To: "dev@trafficcontrol.incubator.apache.org" <de...@trafficcontrol.incubator.apache.org>
Subject: Delivery Service Self-Service
As we move in the direction of self-service, there are a few obstacles that
need to be overcome and I'd like to discuss them a bit so grab a cup of
coffee...
When I say self-service, what I really mean is "delivery service
self-service" or the ability to manage your own delivery services (as
dictated by tenancy) and everything related to those delivery services.
"Everything" includes the following (afaik):
1. Create/Read/Update/Delete delivery services
2. Manage DS regexes
3. Manage DS SSL keys (if applicable)
4. Manage DS URL sig keys (if applicable)
5. Manage DS URI signing keys (if applicable)
6. Manage DS targets (steering* only)
7. Creating DS invalidate content jobs
8. Manage DS / cache assignments
If you can't do 1-7 yourself, it's not really self-service is it? #8 is
debatable.
I'll discuss each one:
1. Create/Read/Update/Delete delivery services
Ideally, you could CRUD your own delivery services but our system has some
limitations.
A) Our CDN is not properly insulated from bad DS configurations. If a user
enters a bad value, bad things could potentially happen to a cache or worse
the whole CDN.
B) Certain DS configuration changes requires queue updates and/or snapshot
for the change to take effect. We're not ready (nor will we ever be
probably) to let normal users queue updates and/or snapshot.
So in the interim, we're working on the ability to allow normal users to
create "delivery service requests" to facilitate creating/updating/deleting
a delivery service. These "requests" will have to be reviewed/fulfilled by
a higher level user (Ops or Admin) who can then queue/snapshot if needed.
2. Manage DS regexes
Here's an explanation of this:
http://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_ops/using.html#delivery-service-regexp
Currently, this requires the Operations role and for good reason. The
danger here involves the risk of a normal user entering a bad regex. For
example, it is my understanding that the regex in position zero needs to
always follow this format: .*\.foo\..*.
Maybe with some better API validation we could let normal users manage DS
regexes....or maybe these end up going away in favor of something
better/easier...not sure yet...
3. Manage DS SSL keys
SSL keys are only applicable where protocol > 0 (HTTPS, HTTP AND HTTPS, or
HTTP TO HTTPS) and currently, to manage them requires the Admin role. Why?
I'm not sure. Is their harm in letting normal users manage their own SSL
keys?
4. Manage DS URL sig keys
URL sig keys (
http://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_ops/using.html#generate-url-sig-keys)
are only applicable where signingAlgorithm = 'url_sig' and currently, to
manage them requires NO role apparently (only a tenancy check is
performed). This is the 1st one on the list that a "normal" user can do
currently.
5. Manage DS URI signing keys
URI signing keys are only applicable where signingAlgorithm = 'uri_signing'
and currently, to manage DS URI signing keys requires the Admin role. Is it
necessary to restrict this functionality to Admins only or can we allow
normal users to mange URI signing keys for their DS's?
6. Manage DS targets (steering* only)
Here's an explanation of this:
http://traffic-control-cdn.readthedocs.io/en/latest/admin/quick_howto/steering.html?highlight=steering
Currently, to manage DS targets requires the Admin or Steering role. Is
there any harm in allowing a normal user to "steer" their delivery service
to another delivery service as long as the target delivery service falls in
their tenancy?
7. Creating DS invalidate content jobs
http://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_ops/using.html?highlight=invalidate#invalidate-content
You can currently do this for your own DS's. This is the 2nd one on the
list that a "normal" user can do currently.
8. Manage DS / cache assignments
This is the debatable one. To provide true delivery service self-service
should a user have the ability to determine which caches are assigned to
their delivery service. I'm thinking NO. Currently, this action requires
the Ops role and I'm in favor of leaving it that way...
In summary, to provide true delivery service self-service I think we need
to do a few things:
1. Introduce DS requests until the day in which DS configurations can be
guaranteed and queue updates/snapshot becomes a thing of the past. (this is
in progress)
2. Revisit DS regexes or make their management more fool proof.
3. Tweak the roles of these actions. Currently, a lot of these things are
reserved for Ops/Admin. We have to change that or full DS self-service if
not possible. I'd like to make each of these things accessible to users
with the "Portal" role with the exception of #8.
Thoughts? Concerns? Funny jokes?
Jeremy
Re: Delivery Service Self-Service
Posted by Chris Lemmons <al...@gmail.com>.
Thoughts:
This is a great analysis! On point 2, I'm in favour of simplifying it
to a single name instead of a fancy regex, at least in the
user-editable settings.
Concerns:
"Bad neighbour" behaviour is the trickiest issue. For example, right
now, a user could, ignorantly or otherwise, create a URI Signing
profile that contains an invalid key that would prevent the remap from
being loaded. Or, they could create a very, very large number of keys
so that when an invalid signature comes through, it tanks performance
checking them all. Likewise, URI Signing has a trusted relationship
with the signer, so a malicious or incompetent signer can produce a
regex that spins the CPU trying to evaluate it. We'd need to close all
these, and probably more, avenues before it would be reasonable to
allow tenants to turn on or configure URI Signing.
Funny jokes:
Why was the Delivery Service arrested after invalidating all it's
content? It had a whole bunch of dirty cache.
On Fri, Feb 2, 2018 at 1:43 PM, Jeremy Mitchell <mi...@apache.org> wrote:
> As we move in the direction of self-service, there are a few obstacles that
> need to be overcome and I'd like to discuss them a bit so grab a cup of
> coffee...
>
> When I say self-service, what I really mean is "delivery service
> self-service" or the ability to manage your own delivery services (as
> dictated by tenancy) and everything related to those delivery services.
> "Everything" includes the following (afaik):
>
> 1. Create/Read/Update/Delete delivery services
> 2. Manage DS regexes
> 3. Manage DS SSL keys (if applicable)
> 4. Manage DS URL sig keys (if applicable)
> 5. Manage DS URI signing keys (if applicable)
> 6. Manage DS targets (steering* only)
> 7. Creating DS invalidate content jobs
> 8. Manage DS / cache assignments
>
> If you can't do 1-7 yourself, it's not really self-service is it? #8 is
> debatable.
>
> I'll discuss each one:
>
> 1. Create/Read/Update/Delete delivery services
>
> Ideally, you could CRUD your own delivery services but our system has some
> limitations.
>
> A) Our CDN is not properly insulated from bad DS configurations. If a user
> enters a bad value, bad things could potentially happen to a cache or worse
> the whole CDN.
> B) Certain DS configuration changes requires queue updates and/or snapshot
> for the change to take effect. We're not ready (nor will we ever be
> probably) to let normal users queue updates and/or snapshot.
>
> So in the interim, we're working on the ability to allow normal users to
> create "delivery service requests" to facilitate creating/updating/deleting
> a delivery service. These "requests" will have to be reviewed/fulfilled by
> a higher level user (Ops or Admin) who can then queue/snapshot if needed.
>
> 2. Manage DS regexes
>
> Here's an explanation of this:
> http://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_ops/using.html#delivery-service-regexp
>
> Currently, this requires the Operations role and for good reason. The
> danger here involves the risk of a normal user entering a bad regex. For
> example, it is my understanding that the regex in position zero needs to
> always follow this format: .*\.foo\..*.
>
> Maybe with some better API validation we could let normal users manage DS
> regexes....or maybe these end up going away in favor of something
> better/easier...not sure yet...
>
> 3. Manage DS SSL keys
>
> SSL keys are only applicable where protocol > 0 (HTTPS, HTTP AND HTTPS, or
> HTTP TO HTTPS) and currently, to manage them requires the Admin role. Why?
> I'm not sure. Is their harm in letting normal users manage their own SSL
> keys?
>
> 4. Manage DS URL sig keys
>
> URL sig keys (
> http://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_ops/using.html#generate-url-sig-keys)
> are only applicable where signingAlgorithm = 'url_sig' and currently, to
> manage them requires NO role apparently (only a tenancy check is
> performed). This is the 1st one on the list that a "normal" user can do
> currently.
>
> 5. Manage DS URI signing keys
>
> URI signing keys are only applicable where signingAlgorithm = 'uri_signing'
> and currently, to manage DS URI signing keys requires the Admin role. Is it
> necessary to restrict this functionality to Admins only or can we allow
> normal users to mange URI signing keys for their DS's?
>
> 6. Manage DS targets (steering* only)
>
> Here's an explanation of this:
> http://traffic-control-cdn.readthedocs.io/en/latest/admin/quick_howto/steering.html?highlight=steering
>
> Currently, to manage DS targets requires the Admin or Steering role. Is
> there any harm in allowing a normal user to "steer" their delivery service
> to another delivery service as long as the target delivery service falls in
> their tenancy?
>
> 7. Creating DS invalidate content jobs
>
> http://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_ops/using.html?highlight=invalidate#invalidate-content
>
> You can currently do this for your own DS's. This is the 2nd one on the
> list that a "normal" user can do currently.
>
> 8. Manage DS / cache assignments
>
> This is the debatable one. To provide true delivery service self-service
> should a user have the ability to determine which caches are assigned to
> their delivery service. I'm thinking NO. Currently, this action requires
> the Ops role and I'm in favor of leaving it that way...
>
> In summary, to provide true delivery service self-service I think we need
> to do a few things:
>
> 1. Introduce DS requests until the day in which DS configurations can be
> guaranteed and queue updates/snapshot becomes a thing of the past. (this is
> in progress)
> 2. Revisit DS regexes or make their management more fool proof.
> 3. Tweak the roles of these actions. Currently, a lot of these things are
> reserved for Ops/Admin. We have to change that or full DS self-service if
> not possible. I'd like to make each of these things accessible to users
> with the "Portal" role with the exception of #8.
>
> Thoughts? Concerns? Funny jokes?
>
> Jeremy
Re: Delivery Service Self-Service
Posted by Jeremy Mitchell <mi...@gmail.com>.
Thanks for all the input. Here's what I heard. For a refresher, these are
the 7 things that a "non-ops" user should be able to perform to achieve
"delivery service self-service":
1. CRUD delivery services - ✓
- non-ops users will be able to create "delivery service requests". ops
users will be able to fulfill those requests. so indirectly, non-ops users
can CRUD ds's. I will be creating documentation around this shortly.
2. Manage DS regexes - X
- this action should continue to be limited to ops users...however, maybe
we allow non-ops users to create cnames and cnames only that don't contain
regex values but conform to hostname format only. more research needed here.
3. Manage SSL keys - ✓
- non-ops users should be able to manage (add, generate) ssl keys for their
delivery services.
4. Manage URL sig keys - ✓
- non-ops users can already do this for their delivery services
5. Manage URI signing keys - X
- this action should continue to be limited to ops users as it could be
dangerous at the moment (i.e. bad neighbor behavior). need more research
here.
6. Manage steering targets - ✓
- Tenancy dictates which delivery services can be defined as a target
therefore no risk to allowing non-ops users to manage steering targets.
(they can only point a steering ds to their own ds's)
7. Invalidate DS content - ✓
- non-ops users can already do this for their delivery services
Not bad. 5 out of 7.
I will follow up later regarding "Manage DS regexes" and "Manage URI
signing keys" so we can figure out a way to provide that functionality to
non-ops users...
Thanks again,
Jeremy
On Mon, Feb 5, 2018 at 12:26 PM, Nir Sopher <ni...@qwilt.com> wrote:
> Hi Jeremy and all,
>
> Jeremy, thanks for putting it all together!
>
> I'll be short, as I mostly agree with most of point the Jeremy's pointed
> out.
>
> Regarding the "DS regex", like Ryan IIUC, I believe, the operator needs to
> be able to configure it and control it.
> First of all as it defines a resource in the operator's domain.
> Second, the definition of the regex requires some clear methodology to
> avoid collisions, or reuse/abuse.
> Following Rawlin remark, some reasonable default should be provided, but we
> must have the ability to change it.
> Note that path regexs are important as DS identifiers, supporting the SVA
> open-caching scheme.
>
> For the last point, the DS/Server assignment, I believe it should be in the
> hands of the operator.
> In the future it can be delegated the the user, subject to capacity
> management. The user should not be familiar with the actual caches, but can
> use some filters/tagging for defining the caches to be used.
>
> Nir
>
> On Mon, Feb 5, 2018 at 8:23 PM, Rawlin Peters <ra...@gmail.com>
> wrote:
>
> > Replies inline
> >
> > On Fri, Feb 2, 2018 at 1:43 PM, Jeremy Mitchell <mi...@apache.org>
> > wrote:
> > > 2. Manage DS regexes
> > >
> > > Here's an explanation of this:
> > > http://traffic-control-cdn.readthedocs.io/en/latest/
> > admin/traffic_ops/using.html#delivery-service-regexp
> > >
> > > Currently, this requires the Operations role and for good reason. The
> > > danger here involves the risk of a normal user entering a bad regex.
> For
> > > example, it is my understanding that the regex in position zero needs
> to
> > > always follow this format: .*\.foo\..*.
> > >
> > > Maybe with some better API validation we could let normal users manage
> DS
> > > regexes....or maybe these end up going away in favor of something
> > > better/easier...not sure yet...
> >
> > I think the approach that the Traffic Portal takes today is good. Just
> > giving a DS a default HOST regex with order = 0 using the xml_id will
> > probably cover most use cases for the DS. Then for the cases where
> > someone is CNAMEing to the DS FQDN, the DS owner should be able to add
> > a max number of HOST regexes with order > 0 matching the CNAME fqdn.
> > We should probably just call those "CNAME aliases" or something and
> > just expose them as a simple hostname list in the UI rather than as
> > HOST regexes with a specific ordering. For a list of aliases, I don't
> > think the order really matters at that point as long as they're
> > greater than 0 and sequential. That operation could be safe for a
> > regular DS owner assuming we validate that the alias is a valid
> > hostname (not a regex), unique, and not in use anywhere else in that
> > CDN.
> >
> > We might want to prohibit creating multiple HOST regexes with wildcard
> > characters (i.e. non-CNAME-alias)...I'm not even sure there's a valid
> > use case for that.
> >
> > I'm not totally familiar with the usage of PATH and HEADER regexes,
> > but they both seem like they should be secondary to the primary HOST
> > regex that's created by default (e.g. the request should be matched
> > against all primary HOST regexes first before checking against the
> > other types). Right now I can create a PATH regex that essentially
> > black-holes other DSes (which ones get black-holed depends on the
> > order DSes come in CRConfig.json, which seems non-deterministic). So
> > we don't want to allow a regular DS owner to modify the PATH and
> > HEADER types unless we modify TR to guarantee that primary-secondary
> > relationship between HOST and PATH/HEADER regexes.
> >
> > > 6. Manage DS targets (steering* only)
> > >
> > > Here's an explanation of this:
> > > http://traffic-control-cdn.readthedocs.io/en/latest/
> > admin/quick_howto/steering.html?highlight=steering
> > >
> > > Currently, to manage DS targets requires the Admin or Steering role. Is
> > > there any harm in allowing a normal user to "steer" their delivery
> > service
> > > to another delivery service as long as the target delivery service
> falls
> > in
> > > their tenancy?
> >
> > I think this should be alright as long as *all* target DSes of the
> > Steering DS fall in their tenancy.
> >
> > -Rawlin
> >
> > On Fri, Feb 2, 2018 at 1:43 PM, Jeremy Mitchell <mi...@apache.org>
> > wrote:
> > > As we move in the direction of self-service, there are a few obstacles
> > that
> > > need to be overcome and I'd like to discuss them a bit so grab a cup of
> > > coffee...
> > >
> > > When I say self-service, what I really mean is "delivery service
> > > self-service" or the ability to manage your own delivery services (as
> > > dictated by tenancy) and everything related to those delivery services.
> > > "Everything" includes the following (afaik):
> > >
> > > 1. Create/Read/Update/Delete delivery services
> > > 2. Manage DS regexes
> > > 3. Manage DS SSL keys (if applicable)
> > > 4. Manage DS URL sig keys (if applicable)
> > > 5. Manage DS URI signing keys (if applicable)
> > > 6. Manage DS targets (steering* only)
> > > 7. Creating DS invalidate content jobs
> > > 8. Manage DS / cache assignments
> > >
> > > If you can't do 1-7 yourself, it's not really self-service is it? #8 is
> > > debatable.
> > >
> > > I'll discuss each one:
> > >
> > > 1. Create/Read/Update/Delete delivery services
> > >
> > > Ideally, you could CRUD your own delivery services but our system has
> > some
> > > limitations.
> > >
> > > A) Our CDN is not properly insulated from bad DS configurations. If a
> > user
> > > enters a bad value, bad things could potentially happen to a cache or
> > worse
> > > the whole CDN.
> > > B) Certain DS configuration changes requires queue updates and/or
> > snapshot
> > > for the change to take effect. We're not ready (nor will we ever be
> > > probably) to let normal users queue updates and/or snapshot.
> > >
> > > So in the interim, we're working on the ability to allow normal users
> to
> > > create "delivery service requests" to facilitate
> > creating/updating/deleting
> > > a delivery service. These "requests" will have to be reviewed/fulfilled
> > by
> > > a higher level user (Ops or Admin) who can then queue/snapshot if
> needed.
> > >
> > > 2. Manage DS regexes
> > >
> > > Here's an explanation of this:
> > > http://traffic-control-cdn.readthedocs.io/en/latest/
> > admin/traffic_ops/using.html#delivery-service-regexp
> > >
> > > Currently, this requires the Operations role and for good reason. The
> > > danger here involves the risk of a normal user entering a bad regex.
> For
> > > example, it is my understanding that the regex in position zero needs
> to
> > > always follow this format: .*\.foo\..*.
> > >
> > > Maybe with some better API validation we could let normal users manage
> DS
> > > regexes....or maybe these end up going away in favor of something
> > > better/easier...not sure yet...
> > >
> > > 3. Manage DS SSL keys
> > >
> > > SSL keys are only applicable where protocol > 0 (HTTPS, HTTP AND HTTPS,
> > or
> > > HTTP TO HTTPS) and currently, to manage them requires the Admin role.
> > Why?
> > > I'm not sure. Is their harm in letting normal users manage their own
> SSL
> > > keys?
> > >
> > > 4. Manage DS URL sig keys
> > >
> > > URL sig keys (
> > > http://traffic-control-cdn.readthedocs.io/en/latest/
> > admin/traffic_ops/using.html#generate-url-sig-keys)
> > > are only applicable where signingAlgorithm = 'url_sig' and currently,
> to
> > > manage them requires NO role apparently (only a tenancy check is
> > > performed). This is the 1st one on the list that a "normal" user can do
> > > currently.
> > >
> > > 5. Manage DS URI signing keys
> > >
> > > URI signing keys are only applicable where signingAlgorithm =
> > 'uri_signing'
> > > and currently, to manage DS URI signing keys requires the Admin role.
> Is
> > it
> > > necessary to restrict this functionality to Admins only or can we allow
> > > normal users to mange URI signing keys for their DS's?
> > >
> > > 6. Manage DS targets (steering* only)
> > >
> > > Here's an explanation of this:
> > > http://traffic-control-cdn.readthedocs.io/en/latest/
> > admin/quick_howto/steering.html?highlight=steering
> > >
> > > Currently, to manage DS targets requires the Admin or Steering role. Is
> > > there any harm in allowing a normal user to "steer" their delivery
> > service
> > > to another delivery service as long as the target delivery service
> falls
> > in
> > > their tenancy?
> > >
> > > 7. Creating DS invalidate content jobs
> > >
> > > http://traffic-control-cdn.readthedocs.io/en/latest/
> > admin/traffic_ops/using.html?highlight=invalidate#invalidate-content
> > >
> > > You can currently do this for your own DS's. This is the 2nd one on the
> > > list that a "normal" user can do currently.
> > >
> > > 8. Manage DS / cache assignments
> > >
> > > This is the debatable one. To provide true delivery service
> self-service
> > > should a user have the ability to determine which caches are assigned
> to
> > > their delivery service. I'm thinking NO. Currently, this action
> requires
> > > the Ops role and I'm in favor of leaving it that way...
> > >
> > > In summary, to provide true delivery service self-service I think we
> need
> > > to do a few things:
> > >
> > > 1. Introduce DS requests until the day in which DS configurations can
> be
> > > guaranteed and queue updates/snapshot becomes a thing of the past.
> (this
> > is
> > > in progress)
> > > 2. Revisit DS regexes or make their management more fool proof.
> > > 3. Tweak the roles of these actions. Currently, a lot of these things
> are
> > > reserved for Ops/Admin. We have to change that or full DS self-service
> if
> > > not possible. I'd like to make each of these things accessible to users
> > > with the "Portal" role with the exception of #8.
> > >
> > > Thoughts? Concerns? Funny jokes?
> > >
> > > Jeremy
> >
>
Re: Delivery Service Self-Service
Posted by Nir Sopher <ni...@qwilt.com>.
Hi Jeremy and all,
Jeremy, thanks for putting it all together!
I'll be short, as I mostly agree with most of point the Jeremy's pointed
out.
Regarding the "DS regex", like Ryan IIUC, I believe, the operator needs to
be able to configure it and control it.
First of all as it defines a resource in the operator's domain.
Second, the definition of the regex requires some clear methodology to
avoid collisions, or reuse/abuse.
Following Rawlin remark, some reasonable default should be provided, but we
must have the ability to change it.
Note that path regexs are important as DS identifiers, supporting the SVA
open-caching scheme.
For the last point, the DS/Server assignment, I believe it should be in the
hands of the operator.
In the future it can be delegated the the user, subject to capacity
management. The user should not be familiar with the actual caches, but can
use some filters/tagging for defining the caches to be used.
Nir
On Mon, Feb 5, 2018 at 8:23 PM, Rawlin Peters <ra...@gmail.com>
wrote:
> Replies inline
>
> On Fri, Feb 2, 2018 at 1:43 PM, Jeremy Mitchell <mi...@apache.org>
> wrote:
> > 2. Manage DS regexes
> >
> > Here's an explanation of this:
> > http://traffic-control-cdn.readthedocs.io/en/latest/
> admin/traffic_ops/using.html#delivery-service-regexp
> >
> > Currently, this requires the Operations role and for good reason. The
> > danger here involves the risk of a normal user entering a bad regex. For
> > example, it is my understanding that the regex in position zero needs to
> > always follow this format: .*\.foo\..*.
> >
> > Maybe with some better API validation we could let normal users manage DS
> > regexes....or maybe these end up going away in favor of something
> > better/easier...not sure yet...
>
> I think the approach that the Traffic Portal takes today is good. Just
> giving a DS a default HOST regex with order = 0 using the xml_id will
> probably cover most use cases for the DS. Then for the cases where
> someone is CNAMEing to the DS FQDN, the DS owner should be able to add
> a max number of HOST regexes with order > 0 matching the CNAME fqdn.
> We should probably just call those "CNAME aliases" or something and
> just expose them as a simple hostname list in the UI rather than as
> HOST regexes with a specific ordering. For a list of aliases, I don't
> think the order really matters at that point as long as they're
> greater than 0 and sequential. That operation could be safe for a
> regular DS owner assuming we validate that the alias is a valid
> hostname (not a regex), unique, and not in use anywhere else in that
> CDN.
>
> We might want to prohibit creating multiple HOST regexes with wildcard
> characters (i.e. non-CNAME-alias)...I'm not even sure there's a valid
> use case for that.
>
> I'm not totally familiar with the usage of PATH and HEADER regexes,
> but they both seem like they should be secondary to the primary HOST
> regex that's created by default (e.g. the request should be matched
> against all primary HOST regexes first before checking against the
> other types). Right now I can create a PATH regex that essentially
> black-holes other DSes (which ones get black-holed depends on the
> order DSes come in CRConfig.json, which seems non-deterministic). So
> we don't want to allow a regular DS owner to modify the PATH and
> HEADER types unless we modify TR to guarantee that primary-secondary
> relationship between HOST and PATH/HEADER regexes.
>
> > 6. Manage DS targets (steering* only)
> >
> > Here's an explanation of this:
> > http://traffic-control-cdn.readthedocs.io/en/latest/
> admin/quick_howto/steering.html?highlight=steering
> >
> > Currently, to manage DS targets requires the Admin or Steering role. Is
> > there any harm in allowing a normal user to "steer" their delivery
> service
> > to another delivery service as long as the target delivery service falls
> in
> > their tenancy?
>
> I think this should be alright as long as *all* target DSes of the
> Steering DS fall in their tenancy.
>
> -Rawlin
>
> On Fri, Feb 2, 2018 at 1:43 PM, Jeremy Mitchell <mi...@apache.org>
> wrote:
> > As we move in the direction of self-service, there are a few obstacles
> that
> > need to be overcome and I'd like to discuss them a bit so grab a cup of
> > coffee...
> >
> > When I say self-service, what I really mean is "delivery service
> > self-service" or the ability to manage your own delivery services (as
> > dictated by tenancy) and everything related to those delivery services.
> > "Everything" includes the following (afaik):
> >
> > 1. Create/Read/Update/Delete delivery services
> > 2. Manage DS regexes
> > 3. Manage DS SSL keys (if applicable)
> > 4. Manage DS URL sig keys (if applicable)
> > 5. Manage DS URI signing keys (if applicable)
> > 6. Manage DS targets (steering* only)
> > 7. Creating DS invalidate content jobs
> > 8. Manage DS / cache assignments
> >
> > If you can't do 1-7 yourself, it's not really self-service is it? #8 is
> > debatable.
> >
> > I'll discuss each one:
> >
> > 1. Create/Read/Update/Delete delivery services
> >
> > Ideally, you could CRUD your own delivery services but our system has
> some
> > limitations.
> >
> > A) Our CDN is not properly insulated from bad DS configurations. If a
> user
> > enters a bad value, bad things could potentially happen to a cache or
> worse
> > the whole CDN.
> > B) Certain DS configuration changes requires queue updates and/or
> snapshot
> > for the change to take effect. We're not ready (nor will we ever be
> > probably) to let normal users queue updates and/or snapshot.
> >
> > So in the interim, we're working on the ability to allow normal users to
> > create "delivery service requests" to facilitate
> creating/updating/deleting
> > a delivery service. These "requests" will have to be reviewed/fulfilled
> by
> > a higher level user (Ops or Admin) who can then queue/snapshot if needed.
> >
> > 2. Manage DS regexes
> >
> > Here's an explanation of this:
> > http://traffic-control-cdn.readthedocs.io/en/latest/
> admin/traffic_ops/using.html#delivery-service-regexp
> >
> > Currently, this requires the Operations role and for good reason. The
> > danger here involves the risk of a normal user entering a bad regex. For
> > example, it is my understanding that the regex in position zero needs to
> > always follow this format: .*\.foo\..*.
> >
> > Maybe with some better API validation we could let normal users manage DS
> > regexes....or maybe these end up going away in favor of something
> > better/easier...not sure yet...
> >
> > 3. Manage DS SSL keys
> >
> > SSL keys are only applicable where protocol > 0 (HTTPS, HTTP AND HTTPS,
> or
> > HTTP TO HTTPS) and currently, to manage them requires the Admin role.
> Why?
> > I'm not sure. Is their harm in letting normal users manage their own SSL
> > keys?
> >
> > 4. Manage DS URL sig keys
> >
> > URL sig keys (
> > http://traffic-control-cdn.readthedocs.io/en/latest/
> admin/traffic_ops/using.html#generate-url-sig-keys)
> > are only applicable where signingAlgorithm = 'url_sig' and currently, to
> > manage them requires NO role apparently (only a tenancy check is
> > performed). This is the 1st one on the list that a "normal" user can do
> > currently.
> >
> > 5. Manage DS URI signing keys
> >
> > URI signing keys are only applicable where signingAlgorithm =
> 'uri_signing'
> > and currently, to manage DS URI signing keys requires the Admin role. Is
> it
> > necessary to restrict this functionality to Admins only or can we allow
> > normal users to mange URI signing keys for their DS's?
> >
> > 6. Manage DS targets (steering* only)
> >
> > Here's an explanation of this:
> > http://traffic-control-cdn.readthedocs.io/en/latest/
> admin/quick_howto/steering.html?highlight=steering
> >
> > Currently, to manage DS targets requires the Admin or Steering role. Is
> > there any harm in allowing a normal user to "steer" their delivery
> service
> > to another delivery service as long as the target delivery service falls
> in
> > their tenancy?
> >
> > 7. Creating DS invalidate content jobs
> >
> > http://traffic-control-cdn.readthedocs.io/en/latest/
> admin/traffic_ops/using.html?highlight=invalidate#invalidate-content
> >
> > You can currently do this for your own DS's. This is the 2nd one on the
> > list that a "normal" user can do currently.
> >
> > 8. Manage DS / cache assignments
> >
> > This is the debatable one. To provide true delivery service self-service
> > should a user have the ability to determine which caches are assigned to
> > their delivery service. I'm thinking NO. Currently, this action requires
> > the Ops role and I'm in favor of leaving it that way...
> >
> > In summary, to provide true delivery service self-service I think we need
> > to do a few things:
> >
> > 1. Introduce DS requests until the day in which DS configurations can be
> > guaranteed and queue updates/snapshot becomes a thing of the past. (this
> is
> > in progress)
> > 2. Revisit DS regexes or make their management more fool proof.
> > 3. Tweak the roles of these actions. Currently, a lot of these things are
> > reserved for Ops/Admin. We have to change that or full DS self-service if
> > not possible. I'd like to make each of these things accessible to users
> > with the "Portal" role with the exception of #8.
> >
> > Thoughts? Concerns? Funny jokes?
> >
> > Jeremy
>
Re: Delivery Service Self-Service
Posted by Rawlin Peters <ra...@gmail.com>.
Replies inline
On Fri, Feb 2, 2018 at 1:43 PM, Jeremy Mitchell <mi...@apache.org> wrote:
> 2. Manage DS regexes
>
> Here's an explanation of this:
> http://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_ops/using.html#delivery-service-regexp
>
> Currently, this requires the Operations role and for good reason. The
> danger here involves the risk of a normal user entering a bad regex. For
> example, it is my understanding that the regex in position zero needs to
> always follow this format: .*\.foo\..*.
>
> Maybe with some better API validation we could let normal users manage DS
> regexes....or maybe these end up going away in favor of something
> better/easier...not sure yet...
I think the approach that the Traffic Portal takes today is good. Just
giving a DS a default HOST regex with order = 0 using the xml_id will
probably cover most use cases for the DS. Then for the cases where
someone is CNAMEing to the DS FQDN, the DS owner should be able to add
a max number of HOST regexes with order > 0 matching the CNAME fqdn.
We should probably just call those "CNAME aliases" or something and
just expose them as a simple hostname list in the UI rather than as
HOST regexes with a specific ordering. For a list of aliases, I don't
think the order really matters at that point as long as they're
greater than 0 and sequential. That operation could be safe for a
regular DS owner assuming we validate that the alias is a valid
hostname (not a regex), unique, and not in use anywhere else in that
CDN.
We might want to prohibit creating multiple HOST regexes with wildcard
characters (i.e. non-CNAME-alias)...I'm not even sure there's a valid
use case for that.
I'm not totally familiar with the usage of PATH and HEADER regexes,
but they both seem like they should be secondary to the primary HOST
regex that's created by default (e.g. the request should be matched
against all primary HOST regexes first before checking against the
other types). Right now I can create a PATH regex that essentially
black-holes other DSes (which ones get black-holed depends on the
order DSes come in CRConfig.json, which seems non-deterministic). So
we don't want to allow a regular DS owner to modify the PATH and
HEADER types unless we modify TR to guarantee that primary-secondary
relationship between HOST and PATH/HEADER regexes.
> 6. Manage DS targets (steering* only)
>
> Here's an explanation of this:
> http://traffic-control-cdn.readthedocs.io/en/latest/admin/quick_howto/steering.html?highlight=steering
>
> Currently, to manage DS targets requires the Admin or Steering role. Is
> there any harm in allowing a normal user to "steer" their delivery service
> to another delivery service as long as the target delivery service falls in
> their tenancy?
I think this should be alright as long as *all* target DSes of the
Steering DS fall in their tenancy.
-Rawlin
On Fri, Feb 2, 2018 at 1:43 PM, Jeremy Mitchell <mi...@apache.org> wrote:
> As we move in the direction of self-service, there are a few obstacles that
> need to be overcome and I'd like to discuss them a bit so grab a cup of
> coffee...
>
> When I say self-service, what I really mean is "delivery service
> self-service" or the ability to manage your own delivery services (as
> dictated by tenancy) and everything related to those delivery services.
> "Everything" includes the following (afaik):
>
> 1. Create/Read/Update/Delete delivery services
> 2. Manage DS regexes
> 3. Manage DS SSL keys (if applicable)
> 4. Manage DS URL sig keys (if applicable)
> 5. Manage DS URI signing keys (if applicable)
> 6. Manage DS targets (steering* only)
> 7. Creating DS invalidate content jobs
> 8. Manage DS / cache assignments
>
> If you can't do 1-7 yourself, it's not really self-service is it? #8 is
> debatable.
>
> I'll discuss each one:
>
> 1. Create/Read/Update/Delete delivery services
>
> Ideally, you could CRUD your own delivery services but our system has some
> limitations.
>
> A) Our CDN is not properly insulated from bad DS configurations. If a user
> enters a bad value, bad things could potentially happen to a cache or worse
> the whole CDN.
> B) Certain DS configuration changes requires queue updates and/or snapshot
> for the change to take effect. We're not ready (nor will we ever be
> probably) to let normal users queue updates and/or snapshot.
>
> So in the interim, we're working on the ability to allow normal users to
> create "delivery service requests" to facilitate creating/updating/deleting
> a delivery service. These "requests" will have to be reviewed/fulfilled by
> a higher level user (Ops or Admin) who can then queue/snapshot if needed.
>
> 2. Manage DS regexes
>
> Here's an explanation of this:
> http://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_ops/using.html#delivery-service-regexp
>
> Currently, this requires the Operations role and for good reason. The
> danger here involves the risk of a normal user entering a bad regex. For
> example, it is my understanding that the regex in position zero needs to
> always follow this format: .*\.foo\..*.
>
> Maybe with some better API validation we could let normal users manage DS
> regexes....or maybe these end up going away in favor of something
> better/easier...not sure yet...
>
> 3. Manage DS SSL keys
>
> SSL keys are only applicable where protocol > 0 (HTTPS, HTTP AND HTTPS, or
> HTTP TO HTTPS) and currently, to manage them requires the Admin role. Why?
> I'm not sure. Is their harm in letting normal users manage their own SSL
> keys?
>
> 4. Manage DS URL sig keys
>
> URL sig keys (
> http://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_ops/using.html#generate-url-sig-keys)
> are only applicable where signingAlgorithm = 'url_sig' and currently, to
> manage them requires NO role apparently (only a tenancy check is
> performed). This is the 1st one on the list that a "normal" user can do
> currently.
>
> 5. Manage DS URI signing keys
>
> URI signing keys are only applicable where signingAlgorithm = 'uri_signing'
> and currently, to manage DS URI signing keys requires the Admin role. Is it
> necessary to restrict this functionality to Admins only or can we allow
> normal users to mange URI signing keys for their DS's?
>
> 6. Manage DS targets (steering* only)
>
> Here's an explanation of this:
> http://traffic-control-cdn.readthedocs.io/en/latest/admin/quick_howto/steering.html?highlight=steering
>
> Currently, to manage DS targets requires the Admin or Steering role. Is
> there any harm in allowing a normal user to "steer" their delivery service
> to another delivery service as long as the target delivery service falls in
> their tenancy?
>
> 7. Creating DS invalidate content jobs
>
> http://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_ops/using.html?highlight=invalidate#invalidate-content
>
> You can currently do this for your own DS's. This is the 2nd one on the
> list that a "normal" user can do currently.
>
> 8. Manage DS / cache assignments
>
> This is the debatable one. To provide true delivery service self-service
> should a user have the ability to determine which caches are assigned to
> their delivery service. I'm thinking NO. Currently, this action requires
> the Ops role and I'm in favor of leaving it that way...
>
> In summary, to provide true delivery service self-service I think we need
> to do a few things:
>
> 1. Introduce DS requests until the day in which DS configurations can be
> guaranteed and queue updates/snapshot becomes a thing of the past. (this is
> in progress)
> 2. Revisit DS regexes or make their management more fool proof.
> 3. Tweak the roles of these actions. Currently, a lot of these things are
> reserved for Ops/Admin. We have to change that or full DS self-service if
> not possible. I'd like to make each of these things accessible to users
> with the "Portal" role with the exception of #8.
>
> Thoughts? Concerns? Funny jokes?
>
> Jeremy