Issues 3525 & 3526 - please don't forget about these bogus tree conflicts

[Johan Corveleyn <jc...@gmail.com>]

Hi devs,

I know a lot of you are mostly focused on wc-ng work right now (which
is a good thing: focus is good, wc-ng is good). But I'd like to
highlight two other important issues, which can IMHO be very confusing
to users, and which I think should definitely be fixed by 1.7 (or if
possible, sooner). I don't know if wc-ng makes fixing these easier,
but they should be fixed none the less.

* http://subversion.tigris.org/issues/show_bug.cgi?id=3525 - Locked
file which is scheduled for delete causes tree conflict
* http://subversion.tigris.org/issues/show_bug.cgi?id=3526 - Commit of
newly added file followed by move (or delete) of parent dir causes
tree conflict

The reason why they are so confusing is that you can get them all on
your own, even if you're the only user of the repository, using only a
single working copy. If you're the only one working in a particular
area of the tree, it can be very disconcerting to suddenly be
confronted with a tree conflict like this. Why on earth can I get
conflicts (of whatever nature) if I'm the only guy making changes???

Ever since tree conflicts arrived, I try to explain to my users how
they happen ("for instance, you edit a file, but in the meantime
someone else committed a move, ..."). But the above issues don't fall
into that category, so it's hard for me to maintain that svn is well
designed software. And users don't really care about the mixed
revision working copy (implementation detail).

(note: it's already disconcerting enough as it is that you can get
"out of date" errors even if you're the only user of a repo, working
with a single working copy. I consider this a bug actually, or in any
case very unexpected behavior. You'd expect that if you're the only
user working with stuff, that you can never ever get out of date. If
that wouldn't be the case, issue 3526 would not be an issue.)

So, please don't forget about these issues. If someone could mark
those two bugs as "Target milestone 1.7", that would give me some
reassurance that they will be looked at before 1.7 day ...

Thanks.
Kind regards,
Johan

Re: Issues 3525 & 3526 - please don't forget about these bogus tree conflicts

[Johan Corveleyn <jc...@gmail.com>]

Thanks, Stefan. Your answers were very enlightening, as always :-).

I really hope TC handling can be improved for cases like these
(resolving them automatically or avoiding them), if possible even for
1.7. I have no idea how you guys are progressing on this front, but I
guess a lot of the potential TC improvements depend on other core
subversion changes (like wc-ng, and later on editor v2 and such). So I
can appreciate that it's kind of a moving target right now ...

Anyway, I'll keep an eye on the dev-list and see how things go :-).

Cheers,
Johan

Re: Issues 3525 & 3526 - please don't forget about these bogus tree conflicts

[Stefan Sperling <st...@elego.de>]

On Sun, Mar 21, 2010 at 09:40:30AM +0100, Johan Corveleyn wrote:
> On Sat, Mar 20, 2010 at 10:44 PM, Stefan Sperling <st...@elego.de> wrote:
> > The only way to "fix" this self-inflicted tree conflict would
> > be to require an update of the *entire* working copy before any changes
> > can be committed.
> 
> You mean: "before you do any change", not "before any changes can be
> committed", no? The problem happens at "svn mv"-time, not at commit
> time.

Yes. I really meant to say "update right after committing before making
any further local changes to the working copy".

> > This is simply not possible to do because it is so
> > fundamentally different from the current, intended, behaviour.
> > Busy repositories would become very hard to work with.
> > As long as Subversion continues providing mixed-revision working
> > copies (possibly forever) it will not automatically bump the revision
> > of paths at commit time which aren't part of the commit.
> 
> Ok, I think I'm beginning to understand that the out-of-dateness is
> really unavoidable with the mixed-revision working copy.
> 
> But is there no way that this tree conflict can be avoided? That's the
> real difficulty for users now. In this case the "incoming edit" is
> just bumping the revision number of the directory. Everything else is
> already up to date AFAICS. Or is this incorrect?

It needs to bump the revision number of every path within the subtree,
not just the directory itself.

But I hope that once we've improved handling of local renames,
this particular situation should not cause a tree conflict anymore.
It will only flag a conflict if you actually delete the parent
directory, rather than renaming it. Not because we stop using
mixed-revision working copies, but because we'll be able to update
the directory at its renamed location.

Right now, svn cannot do that, because it has no good way of knowing
where in the working copy the added directory corresponding to the
deleted (renamed) directory exists. As far as svn is concerned, you
have deleted the directory before updating (even if you did a rename,
the update just sees a delete). The deletion is not based on the HEAD
revision but revision X, and more changes have been made within the directory
between rX and HEAD. Coincidentally, you made these changes from that
same working copy, but that doesn't matter. If you haven't updated the
directory before deleting it, with limited knowledge about renames svn
has to assume a tree conflict between your local delete and whatever
rX did to the directory.

> Maybe this is one of the TC cases for which an automatic resolution
> can be defined and implemented for 1.7?

This depends on what the update really does.
But I agree that we need to make it smarter in some cases.
One problem with the current TC detection code is that it flags
conflicts rather soon, upon being told that the server wants to
change a path, but without being aware of what kinds of edits
the server is really about to send for that path.
I think this is what's causing the "svn lock" TC problem you have found.

> > If you have users who really don't want to know about the details,
> > tell them to update their entire working copy before and after each
> > commit they make. That should avoid this problem, and is easy enough
> > to explain and understand.
> 
> Yes, I know. But up until now this is totally unrealistic because
> update is way too slow (windows, wc-1 locking, ... it takes over 4
> minutes to perform an update of an average-size WC in our company on
> an average client pc, even when there are no incoming changes; having
> an SSD on the client machine brings this down to 45 seconds, which is
> better but still a lot IMHO). Maybe with wc-ng this will become a more
> realistic usage pattern...

Some people have already done performance benchmarks that look very
promising. See:
1) http://svn.haxx.se/dev/archive-2010-02/0457.shtml
2) http://svn.haxx.se/dev/archive-2010-02/0239.shtml
   and http://svn.haxx.se/dev/archive-2010-02/0241.shtml
 
> Maybe a pragmatic way to avoid hitting this out-of-date + TC on
> moving/renaming a directory is just to update the dir that you're
> going to move before you move it. That's more or less manageable I
> think, as long as you don't forget it.

Yes. You are less likely to encounter tree conflicts when renaming
paths that are up-to-date. This holds true for self-inflicted tree
conflicts (you avoid the mixed-revision working copy problem) as well
as real tree conflicts (you narrow the time window in which conflicting
tree changes can be made by other people).

Stefan

Re: Issues 3525 & 3526 - please don't forget about these bogus tree conflicts

[Johan Corveleyn <jc...@gmail.com>]

On Sat, Mar 20, 2010 at 10:44 PM, Stefan Sperling <st...@elego.de> wrote:
> On Sat, Mar 20, 2010 at 10:10:06PM +0100, Johan Corveleyn wrote:
>>
>> * http://subversion.tigris.org/issues/show_bug.cgi?id=3525 - Locked
>> file which is scheduled for delete causes tree conflict
>
> This one is a real bug. It happens because the update editor driver
> calls open_file() on the deleted path (possibly to set/query the lock
> token?) and that function flags a tree conflict because it assumes
> the editor driver will attempt to edit the file later.
> I've set it's milestone to 1.7.0.

Thanks.

>
>> * http://subversion.tigris.org/issues/show_bug.cgi?id=3526 - Commit of
>> newly added file followed by move (or delete) of parent dir causes
>> tree conflict
>
> Sorry, but I think this issue should be closed as INVALID.

I don't really agree that the TC problem is invalid (but maybe my
out-of-dateness digression is). See below.

>
>> The reason why they are so confusing is that you can get them all on
>> your own, even if you're the only user of the repository, using only a
>> single working copy. If you're the only one working in a particular
>> area of the tree, it can be very disconcerting to suddenly be
>> confronted with a tree conflict like this. Why on earth can I get
>> conflicts (of whatever nature) if I'm the only guy making changes???
>
> Because of mixed-revision working copies.
> I know you understand the problem, and how it is caused by mixed revision
> working copies (we've discussed this before and you explain it in the issue).
>
> The only way to "fix" this self-inflicted tree conflict would
> be to require an update of the *entire* working copy before any changes
> can be committed.

You mean: "before you do any change", not "before any changes can be
committed", no? The problem happens at "svn mv"-time, not at commit
time.

> This is simply not possible to do because it is so
> fundamentally different from the current, intended, behaviour.
> Busy repositories would become very hard to work with.
> As long as Subversion continues providing mixed-revision working
> copies (possibly forever) it will not automatically bump the revision
> of paths at commit time which aren't part of the commit.

Ok, I think I'm beginning to understand that the out-of-dateness is
really unavoidable with the mixed-revision working copy.

But is there no way that this tree conflict can be avoided? That's the
real difficulty for users now. In this case the "incoming edit" is
just bumping the revision number of the directory. Everything else is
already up to date AFAICS. Or is this incorrect?

Maybe this is one of the TC cases for which an automatic resolution
can be defined and implemented for 1.7?

>> Ever since tree conflicts arrived, I try to explain to my users how
>> they happen ("for instance, you edit a file, but in the meantime
>> someone else committed a move, ..."). But the above issues don't fall
>> into that category, so it's hard for me to maintain that svn is well
>> designed software. And users don't really care about the mixed
>> revision working copy (implementation detail).
>
> It's not just an implementation detail. It's a feature :)
> Users should care about it, in my opinion.
>
> If you have users who really don't want to know about the details,
> tell them to update their entire working copy before and after each
> commit they make. That should avoid this problem, and is easy enough
> to explain and understand.

Yes, I know. But up until now this is totally unrealistic because
update is way too slow (windows, wc-1 locking, ... it takes over 4
minutes to perform an update of an average-size WC in our company on
an average client pc, even when there are no incoming changes; having
an SSD on the client machine brings this down to 45 seconds, which is
better but still a lot IMHO). Maybe with wc-ng this will become a more
realistic usage pattern...

Maybe a pragmatic way to avoid hitting this out-of-date + TC on
moving/renaming a directory is just to update the dir that you're
going to move before you move it. That's more or less manageable I
think, as long as you don't forget it.

Thanks.
Johan

Re: Issues 3525 & 3526 - please don't forget about these bogus tree conflicts

[Stefan Sperling <st...@elego.de>]

On Sat, Mar 20, 2010 at 10:10:06PM +0100, Johan Corveleyn wrote:
> Hi devs,
> 
> I know a lot of you are mostly focused on wc-ng work right now (which
> is a good thing: focus is good, wc-ng is good). But I'd like to
> highlight two other important issues, which can IMHO be very confusing
> to users, and which I think should definitely be fixed by 1.7 (or if
> possible, sooner). I don't know if wc-ng makes fixing these easier,
> but they should be fixed none the less.
> 
> * http://subversion.tigris.org/issues/show_bug.cgi?id=3525 - Locked
> file which is scheduled for delete causes tree conflict

This one is a real bug. It happens because the update editor driver
calls open_file() on the deleted path (possibly to set/query the lock
token?) and that function flags a tree conflict because it assumes
the editor driver will attempt to edit the file later.
I've set it's milestone to 1.7.0.

> * http://subversion.tigris.org/issues/show_bug.cgi?id=3526 - Commit of
> newly added file followed by move (or delete) of parent dir causes
> tree conflict

Sorry, but I think this issue should be closed as INVALID.

> The reason why they are so confusing is that you can get them all on
> your own, even if you're the only user of the repository, using only a
> single working copy. If you're the only one working in a particular
> area of the tree, it can be very disconcerting to suddenly be
> confronted with a tree conflict like this. Why on earth can I get
> conflicts (of whatever nature) if I'm the only guy making changes???

Because of mixed-revision working copies.
I know you understand the problem, and how it is caused by mixed revision
working copies (we've discussed this before and you explain it in the issue).

The only way to "fix" this self-inflicted tree conflict would
be to require an update of the *entire* working copy before any changes
can be committed. This is simply not possible to do because it is so
fundamentally different from the current, intended, behaviour.
Busy repositories would become very hard to work with.
As long as Subversion continues providing mixed-revision working
copies (possibly forever) it will not automatically bump the revision
of paths at commit time which aren't part of the commit.

> Ever since tree conflicts arrived, I try to explain to my users how
> they happen ("for instance, you edit a file, but in the meantime
> someone else committed a move, ..."). But the above issues don't fall
> into that category, so it's hard for me to maintain that svn is well
> designed software. And users don't really care about the mixed
> revision working copy (implementation detail).

It's not just an implementation detail. It's a feature :)
Users should care about it, in my opinion.

If you have users who really don't want to know about the details,
tell them to update their entire working copy before and after each
commit they make. That should avoid this problem, and is easy enough
to explain and understand.

Stefan