Replay, Authz, and 1.4

[Garrett Rooney <ro...@electricjellyfish.net>]

So Lundblad and myself, along with other random passers-by on
#svn-dev, have been talking about the replay authz stuff that we've
been trying to get in before 1.4, and we figured it should be
summarized and sent to the list.

Basically, the desired change to authz would be something like this. 
We add a new permission that controls the ability to execute
operations that recurse through an entire subtree, which includes
things like update, checkout, diff, merge, status -u, and replay. 
This is possible to implement, although it is tricky in some spots.

The problem with this is how is it configured in the authz config
file?  Well, first off there's the fact that just naively adding this
new option changes the semantics of current authz files.  So something
like this:

[/]
* = rw

Used to mean "everybody has full access to the tree", but now means
"everybody has full read write access, but can't actually checkout or
update anything because they never got this new access".  So we can
work around that by requiring the user to explicitly say this new
permission exists or something like that, but ick.

Alternatively, we could invert the sense of the new permission, but
then you're stuck with one permission type that's totally different
from the others, which is confusing.

Next, there's the issue that people now really need to carry around
this new permission all over the place, whenever they add a more
specific rule.  For example, if I want to keep people from writing to
a specific subdir, I used to be able to do this:

[/]
* = rw

[/foo]
* = r

Now I need something like:

[/]
* = rwc

[/foo]
* = rc

(assume c is our new 'reCursive' permission)

This kind of sucks, but our authz system doesn't have any concept of
inherited permissions, or negative permissions, or anything like that,
so it's the best we can do without big changes.

One potential solution here is to have c be implied by some other
stuff, for example if you have w access someplace you also get c for
free, but then you lack the ability to give someone the ability to
modify a directory without checking it out, which is bad because you
wouldn't be able to allow someone write access on say 'branches'
without also letting them check out branches.

Lundblad wants to allow things like +w or -w, inheriting the rest of
the perms from parent directories, but like I said, that's a bit of
work because we currently don't have anything like that, and it has
potential performance implications due to requiring us to look further
up the directory tree before we can be sure about a given answer.

Also note that all of this stuff really needs to play nicely with
wildcards, otherwise you have to set these permissions in a zillion
places, which is a non-starter.  Wildcards are not especially trivial.
 I've got a sort of working patch, but it's not done, and it will
require substantial review from a user experience perspective, to make
sure it does what we think users will want.

At this point in the conversation, we started to wonder if it was even
possible to implement a good authz based solution in a reasonable
amount of time, especially taking into account that you really don't
want to drop a massive authz change in right before a release.  So we
started to consider alternative plans.

The main alternative is read hooks of some sort, but let's face it,
those are just as complex as this authz stuff.  Alternatively, we
could just provide a work around for the problem, allowing users to
turn off replay if they're so inclined, so at least we're not any
worse off than we were in 1.3.x.

Anyway, at this point I'm not sure what we should be doing. 
Personally, I think I can implement solutions to the replay problem
for sites I'm responsible for, they won't be integrated with our
existing authz stuff, and they won't be especially nice, but they'll
work, and they'll allow us to move forward wihtout introducing massive
changes right before this release.  Alternatively, we can implement a
stopgap solution like defaulting replay to off, or allowing users to
turn it off if they're so inclined, but that means we decide to not
solve the update problem right now.  Finally we could back off and try
to find a more correct solution to this problem, but we need to
recognize that it's nontrivial, and requires some serious effort.

If anyone else has some feedback on this, I'd love to hear it.  I feel
like this is an important issue, and I'm really not sure what our
short or long term answer to it should be at this point.

-garrett

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Replay, Authz, and 1.4

[Marc Sherman <ms...@projectile.ca>]

Garrett Rooney wrote:
> So Lundblad and myself, along with other random passers-by on
> #svn-dev, have been talking about the replay authz stuff that we've
> been trying to get in before 1.4, and we figured it should be
> summarized and sent to the list.

I've been one of the people clamouring for this feature, but my interest
in it has just diminished greatly -- I'm leaving the company where I'm
currently svn admin, going (back) to a company that's stuck eternally on
cvs for political reasons, so my interest is now mostly academic. :)

> Basically, the desired change to authz would be something like this. 
> We add a new permission that controls the ability to execute
> operations that recurse through an entire subtree, which includes
> things like update, checkout, diff, merge, status -u, and replay. 
> This is possible to implement, although it is tricky in some spots.

I really like this simplification of the issue.  It solves all the
outstanding problems I've seen quite nicely.

> The problem with this is how is it configured in the authz config
> file? 

Nicolás Lichtmaier wrote:
> Have "r" mean "full read access including recursive reads", and add a
> new "a" (access?) meaning "just access, no recursive ops". So all
> the "*=rw" would still be full access and this new feature will be
> used just by those who explicitly activate it.
> 
> The thing is... does this look good? Is it clean to have a permision
> implying another? It's an esthetical issue.

I really like this suggestion.  The objection can be overcome by also
introducing "c" for reCursive access, and then simply defining "r" as a
shorthand for "ac".  It can even be sanely documented without invoking
the backwards compatibility crutch, since both "reAd" and "reCursion"
start with r. :)

Garrett Rooney wrote:
> Next, there's the issue that people now really need to carry around
> this new permission all over the place, whenever they add a more
> specific rule. 

With the r shorthand available, I'm not sure that I see this as a
problem.  It will be quite natural to just accidentally do the right thing.

> Lundblad wants to allow things like +w or -w, inheriting the rest of
> the perms from parent directories, but like I said, that's a bit of
> work because we currently don't have anything like that, and it has
> potential performance implications due to requiring us to look further
> up the directory tree before we can be sure about a given answer.

I think that would be nice syntactic sugar, but I don't see it as being
at all required (from a users perspective) for this new authz syntax to
be acceptably usable.

> At this point in the conversation, we started to wonder if it was even
> possible to implement a good authz based solution in a reasonable
> amount of time, especially taking into account that you really don't
> want to drop a massive authz change in right before a release.  So we
> started to consider alternative plans.

Greg Hudson wrote:
> I don't object to putting replay controls in the performance safety
> mechanism mentioned above, but I don't think there's any need to have
> that in the same release as replay goes into.

I agree 100%.  While this would be a great feature to add, I don't think
that replay alone makes it a release blocking requirement.

Garrett Rooney wrote:
> Perhaps a useful way to avoid some of the problem is simply to change
> the behavior of svnsync, have it only copy a revision at a time or
> something, so at least people who pick up the shiny new toy won't
> cause problems right off the bat the first time they try it, they'd
> have to at least write a script to do that  ;-) 

That seems quite wise.  Even just requiring a command line switch to
sync the entire repository in one go, with a warning about performance
in the --help string, would be more than enough, IMO.  As has already
been stated many times, people are already doing exactly what replay
does, in a much less efficient manner.  As long as you can avoid
introducing more _accidental_ DOS's caused by people trying out the tool
in it's default mode, you've met your responsibility to not make things
worse by introducing replay in 1.4 without authz recursion control being
available.

- Marc


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Replay, Authz, and 1.4

[Garrett Rooney <ro...@electricjellyfish.net>]

On 5/3/06, Greg Hudson <gh...@mit.edu> wrote:
> My position:
>
> 1. Checking out /tags, /branches, or their parents is a real problem,
> though apparently a tolerable one, and one we should address, but it
> isn't an authorization problem; it's a performance safety measure.  If
> it doesn't fit nicely into the authz file, we should invent a new
> mechanism rather than squeezing it into the existing one.

My only concern here is that any new mechanism to control that sort of
thing is likely to do things that are virtually identical to our
existing authz system.  Do we really want to have two things doing
almost the same thing?

> 2. You're hugely overblowing the severity of the replay problem.
> Checking out /tags or /branches or their parents is bad because has a
> multiplicative cost; a 1MB trunk with a hundred tags generates 100MB of
> data on a checkout of /tags.  Replay, on the other hand, is quite
> efficient; it gives you exactly as much data as you need.  Moreover,
> people who want to use that kind of functionality are already
> synthesizing it from other operations today, so it's unlikely that
> replay is going to create some kind of new load issue.  I don't object
> to putting replay controls in the performance safety mechanism mentioned
> above, but I don't think there's any need to have that in the same
> release as replay goes into.

The problem with replay is that the tool we provide that exposes it is
svnsync, which has the default behavior of sucking down all history in
a given repository.  Sure, we may not have the multiplicative
behavior, but we've still got a depth problem there, for a very large
repository it can be just as bad as checking out the root of the tree.

Perhaps a useful way to avoid some of the problem is simply to change
the behavior of svnsync, have it only copy a revision at a time or
something, so at least people who pick up the shiny new toy won't
cause problems right off the bat the first time they try it, they'd
have to at least write a script to do that ;-)

-garrett

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Replay, Authz, and 1.4

[Greg Hudson <gh...@MIT.EDU>]

My position:

1. Checking out /tags, /branches, or their parents is a real problem,
though apparently a tolerable one, and one we should address, but it
isn't an authorization problem; it's a performance safety measure.  If
it doesn't fit nicely into the authz file, we should invent a new
mechanism rather than squeezing it into the existing one.

2. You're hugely overblowing the severity of the replay problem.
Checking out /tags or /branches or their parents is bad because has a
multiplicative cost; a 1MB trunk with a hundred tags generates 100MB of
data on a checkout of /tags.  Replay, on the other hand, is quite
efficient; it gives you exactly as much data as you need.  Moreover,
people who want to use that kind of functionality are already
synthesizing it from other operations today, so it's unlikely that
replay is going to create some kind of new load issue.  I don't object
to putting replay controls in the performance safety mechanism mentioned
above, but I don't think there's any need to have that in the same
release as replay goes into.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Replay, Authz, and 1.4

[Nicolás Lichtmaier <ni...@reloco.com.ar>]

> [/]
> * = rw
>

Simple solution:

Have "r" mean "full read access including recursive reads", and add a 
new "a" (access?) meaning "just access, no recursive ops". So all the 
"*=rw" would still be full access and this new feature will be used just 
by those who explicitly activate it.

The thing is... does this look good? Is it clean to have a permision 
implying another? It's an esthetical issue.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Replay, Authz, and 1.4

[Danny van Heumen <da...@hccnet.nl>]

Garrett Rooney wrote:
> [/]
> * = rw
> 
> [/foo]
> * = r
> 
> Now I need something like:
> 
> [/]
> * = rwc
> 
> [/foo]
> * = rc
> 
> (assume c is our new 'reCursive' permission)

Just some thoughts that shot to mind when I was reading the messages:

1. Why use a letter for something like indicating 'recursive' access?
The way I see it we use letters for different types (levels?) of access.
- Read only (r) (btw: this is essentially saying you're allowed to
checkout, so we shouldn't need a new letter for that)
- Write (w)
- Replay
- Branching
- Merging
- etc.

For every one of these levels we could apply this recursively if we
needed to, so I see 'recursive' more as a modifier.

I think it would be obvious if we used a symbol for the recursive
attribute instead of a letter, better leave these for the different
levels of access.

2. If we have 'r' for recursive read access and 'a' for just read
access, we have to have 2 letters for every level of access, don't we?
('a' is probably from another post)

3. If you don't want to break the current authz style, you could use a
modifier to prevent recursion instead of one to activate recursion.
And I think it's more common to recursively add permission than
non-recursively so implicit recursion wouldn't be bad, would it?

Danny

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org