Atomic Renames Status

[Garrett Rooney <ro...@electricjellyfish.net>]

There's been a bunch of progress made on atomic renames lately, so I
figured I'd try and sum up where we are and where we're going for the
list.

The current status is we've got modifications to the bdb back end that
allow us to have a working svn_fs_move function.  There are various
hoops that need to be jumped through to allow that to work without
introducing various kinds of primary key conflicts, but I think we've
found them all now.  I know, famous last words.

So what does this mean to the rest of the system?  Well, not a whole
lot yet.  If you use a direct call to svn_fs_move to rename a file
inside your filesystem it will currently show up as a move in verbose
log output (the V letter instead of D, A, R, or M).  History tracing
through the move works properly, and updates and diffs that include
renames work as well as they ever have (diffs have some weirdness with
moved directories, but those issues exist with the current copy +
delete impl as well).  Due to the way our reporter code determines
what will be sent through the update/diff editors, there is actually
little work that needs to be done to make the svn client behave
properly in the face of renames.

It gets more complex when you start talking about editors that know
about copyfrom data.  For example, dump/load and replay both will
require more work to successfully function in the face of renames. 
I've got dump/load working with basic cases, but it needs a bunch more
work to be complete.  There is also the question of what kind of
modifications to the editor are allowed by our compat policy, see a
forthcoming mail about that.

There are a few other todo items remaining, naturally.

One of the work arounds for a primary key conflict problem involves
the addition of a new copy type, an alias that just points to another
copy in the table.  It would be good to be able to reuse the existing
soft copy mechanism, but when I originally tried that it didn't work. 
I need to revisit that and figure out why.  Additionally we don't want
to insert extra copy records willy nilly, which is what we're doing
now.  It should be possible to only do that in certain cases, but it
needs to be determined if it's possible to detect the cases where it's
required.

The current code makes svn_fs_copied_from treat renames just like
copies.  cmpilato has a philosophical objection to that, and we've
also discussed the possibility of moving the copyfrom info directly
into the log results, since the basic use case seems to be "run log
functions, get something that might be a copy, run copied_from to
check".  It seems like we could just skip some steps by returning that
info in the first place, and we can possibly make it more efficient
due to the way some of that is being stored in the copies table, which
we're already traversing.

None of this has been implemented in the fsfs back end, that needs to be done.

Dump and load need to work in all cases.  Getting that to happen at
all means we need to figure out how the editor should treat renames. 
I've got some ideas about it, but they need to be fleshed out far more
than the current impl on the branch.

We need to figure out what our backwards compat story is with some of
this stuff.  Just sending the moves back through log (the main place
they show up) might confuse existing code, so we might want to only do
so for clients that are new enough to understand it, falling back to
copy + delete for old clients.

Once all of this is done, we can start talking about how to make the
client actually perform these operations and THEN we can talk about
actually making use of this sort of data to improve things like update
and merge.

-garrett

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Atomic Renames Status

["C. Michael Pilato" <cm...@collab.net>]

Garrett Rooney wrote:
> There's been a bunch of progress made on atomic renames lately, so I
> figured I'd try and sum up where we are and where we're going for the
> list.

Thanks for the update, Garrett.  And good work!

-- 
C. Michael Pilato <cm...@collab.net>
CollabNet   <>   www.collab.net   <>   Distributed Development On Demand

Re: Atomic Renames Status

[Garrett Rooney <ro...@electricjellyfish.net>]

On 4/3/06, Daniel Rall <dl...@collab.net> wrote:
> On Fri, 31 Mar 2006, Garrett Rooney wrote:
> ...
> > So what does this mean to the rest of the system?  Well, not a whole
> > lot yet.  If you use a direct call to svn_fs_move to rename a file
> > inside your filesystem it will currently show up as a move in verbose
> > log output (the V letter instead of D, A, R, or M).
>
> This will have an impact on folks who wrap the command-line client (or
> even the svn_client_log functions?).

Yeah, that's why there's a big "XXX think about backwards
compatibility" in that part of the log code ;-)

Eventually I want it to be smart enough to only return that if the
client is new enough to understand it.

> ...
> > The current code makes svn_fs_copied_from treat renames just like
> > copies.  cmpilato has a philosophical objection to that, and we've
> > also discussed the possibility of moving the copyfrom info directly
> > into the log results, since the basic use case seems to be "run log
> > functions, get something that might be a copy, run copied_from to
> > check".  It seems like we could just skip some steps by returning that
> > info in the first place, and we can possibly make it more efficient
> > due to the way some of that is being stored in the copies table, which
> > we're already traversing.
> ...
>
> +1 on returning the info in the first place, rather than jumping
> through extra hoops to and fro the file system.

Well, we may still need some similar function to get at the data when
you aren't running log, but yeah, I think it makes sense to build it
into the log code.

-garrett

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Atomic Renames Status

[Daniel Rall <dl...@collab.net>]

On Fri, 31 Mar 2006, Garrett Rooney wrote:
...
> So what does this mean to the rest of the system?  Well, not a whole
> lot yet.  If you use a direct call to svn_fs_move to rename a file
> inside your filesystem it will currently show up as a move in verbose
> log output (the V letter instead of D, A, R, or M).

This will have an impact on folks who wrap the command-line client (or
even the svn_client_log functions?).

...
> The current code makes svn_fs_copied_from treat renames just like
> copies.  cmpilato has a philosophical objection to that, and we've
> also discussed the possibility of moving the copyfrom info directly
> into the log results, since the basic use case seems to be "run log
> functions, get something that might be a copy, run copied_from to
> check".  It seems like we could just skip some steps by returning that
> info in the first place, and we can possibly make it more efficient
> due to the way some of that is being stored in the copies table, which
> we're already traversing.
...

+1 on returning the info in the first place, rather than jumping
through extra hoops to and fro the file system.
-- 

Daniel Rall