You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by David Carson <dc...@gmail.com> on 2011/09/09 18:17:15 UTC
cleaning up mergeinfo using svn 1.7
We have a repository that is seriously cluttered with the svn:mergeinfo
property. I don't believe we ever used the buggy 1.5.x versions that added
the property erroneously (although I could be wrong). However, due to a lax
policy regarding where merges should occur, the property is now attached to
files and directories all over the place. For example, when a user wants to
double-commit his one-file change from release 1.3 to 1.2, he merges the
change from the top-level of the 1.3 branch into the 1.2 branch and commits.
The change in 1.2 appears as one text change and *137 property changes*!
Note that one complication is the fact that some users have mistakenly
committed only the file changes and the top-level directory prop change, but
have left the rest of the property changes uncommitted.
I was hoping that the improvements in this area that are coming in 1.7 would
allow us to continue to use this polluted repository, because even though it
would not clean up the mess, it would not continue to create merges that
look much larger than they are (due to the numerous superfluous property
changes).
To that end, I installed 1.7.0-rc2 with a snapshot of our repository, so I
could experiment and see for myself. I did a simple merge from one branch
to another. I was disappointed to see that the same miscellaneous group of
files and directories below the top-level are having their snv:mergeinfo
modified, even though they do not participate in the change. I guess I am
not reading the documentation for 1.7 correctly, because I thought that this
was one of the major changes; namely, that any object which is not affected
by the merged changes would not have its svn:mergeinfo property changed.
In my test, all files and directories that *currently* have svn:mergeinfo
attached get the updated info from the merge being performed. This means
that the backport of my one-file change looks like a much larger change.
Here are my questions:
1. Assuming I don't clean up the repository, what help can I expect from
1.7 in terms of new merges? In other words, explain the behavior I should
expect, given the state of my repository now and the test I performed above.
2. What other methods are recommended if 1.7 will not do what I'm hoping
for? Are there any good cleanup scripts to help scrub the whole tree? (I
tried mergeinfo-sanitize, to see what it was, but it did not allow me to
authenticate??)
3. If I decide to simply prohibit users from making changes to
svn:mergeinfo on any object besides the top-level directory itself, what are
the unforeseen consequences that I need to worry about? For *this third
question, *I'm not so concerned about information that might be lost
during cleanup, although I know that is a huge issue. I'm more interested
in the limitations this might place on us in the future, assuming we can
clean up the tree completely so that *only* the top-level directory has
svn:mergeinfo attached to it. For example, if a user commits two fixes
using a single commit on 1.3, then wants to backport one of those fixes to
1.2, can the mergeinfo -- limited to only the top-level directory --
maintain all the information needed?
Thanks,
David
RE: cleaning up mergeinfo using svn 1.7
Posted by Bob Archer <Bo...@amsi.com>.
> We have a repository that is seriously cluttered with the svn:mergeinfo
> property. I don't believe we ever used the buggy 1.5.x versions that added
> the property erroneously (although I could be wrong). However, due to a lax
> policy regarding where merges should occur, the property is now attached to
> files and directories all over the place. For example, when a user wants to
> double-commit his one-file change from release 1.3 to 1.2, he merges the
> change from the top-level of the 1.3 branch into the 1.2 branch and
> commits. The change in 1.2 appears as one text change and 137 property
> changes!
>
> Note that one complication is the fact that some users have mistakenly
> committed only the file changes and the top-level directory prop change, but
> have left the rest of the property changes uncommitted.
It is possible that if you committed those property changes the mergeinfo would be elided. You may want to run a record-only merge, this should elide and extra mergeinfo that is on the child nodes. Afaik.
BOb
>
> I was hoping that the improvements in this area that are coming in 1.7 would
> allow us to continue to use this polluted repository, because even though it
> would not clean up the mess, it would not continue to create merges that
> look much larger than they are (due to the numerous superfluous property
> changes).
>
> To that end, I installed 1.7.0-rc2 with a snapshot of our repository, so I could
> experiment and see for myself. I did a simple merge from one branch to
> another. I was disappointed to see that the same miscellaneous group of
> files and directories below the top-level are having their snv:mergeinfo
> modified, even though they do not participate in the change. I guess I am
> not reading the documentation for 1.7 correctly, because I thought that this
> was one of the major changes; namely, that any object which is not affected
> by the merged changes would not have its svn:mergeinfo property
> changed.
>
> In my test, all files and directories that currently have svn:mergeinfo
> attached get the updated info from the merge being performed. This means
> that the backport of my one-file change looks like a much larger change.
>
> Here are my questions:
> 1. Assuming I don't clean up the repository, what help can I expect from 1.7
> in terms of new merges? In other words, explain the behavior I should
> expect, given the state of my repository now and the test I performed
> above.
> 2. What other methods are recommended if 1.7 will not do what I'm hoping
> for? Are there any good cleanup scripts to help scrub the whole tree? (I
> tried mergeinfo-sanitize, to see what it was, but it did not allow me to
> authenticate??) 3. If I decide to simply prohibit users from making changes to
> svn:mergeinfo on any object besides the top-level directory itself, what are
> the unforeseen consequences that I need to worry about? For this third
> question, I'm not so concerned about information that might be lost during
> cleanup, although I know that is a huge issue. I'm more interested in the
> limitations this might place on us in the future, assuming we can clean up the
> tree completely so that only the top-level directory has svn:mergeinfo
> attached to it. For example, if a user commits two fixes using a single commit
> on 1.3, then wants to backport one of those fixes to 1.2, can the mergeinfo --
> limited to only the top-level directory -- maintain all the information needed?
>
> Thanks,
> David
Re: cleaning up mergeinfo using svn 1.7
Posted by Stefan Sperling <st...@elego.de>.
On Sat, Sep 10, 2011 at 12:40:11AM +0200, Stefan Sperling wrote:
> It would help if you could present examples where elision does not
> work for you, so we can fix 'svn merge'.
The following quote from
https://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_client/mergeinfo.h
provides some background on how elision is supposed to work.
/* Elide any svn:mergeinfo set on TARGET_WCPATH to its nearest working
copy (or possibly repository) ancestor with equivalent mergeinfo.
If WC_ELISION_LIMIT_PATH is NULL check up to the root of the
working copy or the nearest switched parent for an elision
destination, if none is found check the repository, otherwise check
as far as WC_ELISION_LIMIT_PATH within the working copy.
TARGET_WCPATH and WC_ELISION_LIMIT_PATH, if it exists, must both be
absolute or relative to the working directory.
Elision occurs if:
A) TARGET_WCPATH has empty mergeinfo and no parent path with
explicit mergeinfo can be found in either the WC or the
repository (WC_ELISION_LIMIT_PATH must be NULL for this to
occur).
B) TARGET_WCPATH has empty mergeinfo and its nearest parent also
has empty mergeinfo.
C) TARGET_WCPATH has the same mergeinfo as its nearest parent
when that parent's mergeinfo is adjusted for the path
difference between the two, e.g.:
TARGET_WCPATH = A_COPY/D/H
TARGET_WCPATH's mergeinfo = '/A/D/H:3'
TARGET_WCPATH nearest parent = A_COPY
Parent's mergeinfo = '/A:3'
Path differece = 'D/H'
Parent's adjusted mergeinfo = '/A/D/H:3'
If Elision occurs remove the svn:mergeinfo property from
TARGET_WCPATH. */
svn_error_t *
svn_client__elide_mergeinfo(const char *target_wcpath,
const char *wc_elision_limit_path,
svn_client_ctx_t *ctx,
apr_pool_t *pool);
Re: cleaning up mergeinfo using svn 1.7
Posted by Stefan Sperling <st...@elego.de>.
On Mon, Sep 12, 2011 at 11:52:01AM -0400, David Carson wrote:
> > We could allow the existing elision code to be invoked without running
> > a merge. An option like --elide-mergeinfo could be added to the
> > 'svn mergeinfo' command for this purpose. Would this be useful?
> >
>
> Yes, I think such an option would probably be useful. What I really need,
> based on how much information is currently in my props, is something like:
> - try to elide (maybe using your new option)
> - output says you can't because you are missing a, b and c; suggests I use
> --record-only or do the missing merge if it is really missing
> Then I can iterate over this sequence until I succeed and the elision
> occurs.
Hi David,
I started some work on this but the results were not as fruitful as I
was hoping for. Please see this thread on the developer list for related
discussion: http://svn.haxx.se/dev/archive-2011-09/0243.shtml
This post by Paul Burba describes the current limitations of mergeinfo
elision and suggests a better algorithm:
http://svn.haxx.se/dev/archive-2011-09/0248.shtml
If you like, you could try to implement this algorithm in a wrapper script
to see how well it works.
Re: cleaning up mergeinfo using svn 1.7
Posted by David Carson <dc...@gmail.com>.
On Fri, Sep 9, 2011 at 6:40 PM, Stefan Sperling <st...@elego.de> wrote:
> On Fri, Sep 09, 2011 at 03:00:53PM -0400, David Carson wrote:
> > OK, so let's try again. I've merged a change from another branch, which
> is
> > a commit with a single file changed. The result is much better, but let
> me
> > verify that I understand my results:
> > - file a/b/c/foo.c shows the file changes -- OK
> > - file a/b/c/foo.c also contains svn:mergeinfo changes -- I assume this
> is
> > because it happens to be one of the many files that got contaminated with
> > the property prior to this test. So, since it is part of the changeset
> for
> > my merge, the prop has to be updated too.
> > - <top-level> directory (call it TLD/) contains svn:mergeinfo changes --
> OK
> > - TLD/a also contains svn:mergeinfo changes -- Same as above. Directory
> > TLD/a is one of the directories which was previously contaminated, and
> since
> > this path is 'involved' in the change, the prop has to be updated. I
> > wouldn't say the path really involved, except that it is the path on wich
> > foo.c sits. After all, TLD/a/ is not itself changed. Nonetheless, I can
> > live with this.
>
> This is because it's not about which "objects" (as you called them ealier)
> are affected by the merge, but which subtrees of the merge target are
> affected.
>
> In your case, it looks as if the target of the merge is the working copy
> root. An affected subtree of the root is a, an affected subtree
> of a is b, and so on, until you get to foo.c.
>
OK, then it seems to be working as I expect it to.
>
> If you merge from ^/branch/a/b/c/foo.c into wc-path/a/b/c/foo.c
> only foo.c should receive mergeinfo updates, because it is the
> merge target and has no children.
>
Ironically, that is what got us in this mess in the first place (merging at
various locations in the tree). I think what I want eventually is a tree
where ONLY the top-level directory has any svn:mergeinfo.
>
> > Now, are there any tools to aid in cleanup of the mess? Besides the
> large
> > number of files/dirs with the prop attached, the prop itself has a large
> > number of paths to contend with.
>
> There doesn't seem to be any officially sanctioned script.
>
> There are various scripts people have written and posted to this list.
> One is here: http://svn.haxx.se/dev/archive-2009-05/0438.shtml
> You could try the archive search on svn.haxx.se -- maybe you'll find
> something useful.
>
> However, I would say that the need for such a script implies a lack
> of functionality in Subversion itself. 'svn merge' does try to elide
> mergeinfo when subtree mergeinfo becomes redundant, but AFAIK the
> current implementation is a bit simplistic.
> It would help if you could present examples where elision does not
> work for you, so we can fix 'svn merge'.
>
> We could allow the existing elision code to be invoked without running
> a merge. An option like --elide-mergeinfo could be added to the
> 'svn mergeinfo' command for this purpose. Would this be useful?
>
Yes, I think such an option would probably be useful. What I really need,
based on how much information is currently in my props, is something like:
- try to elide (maybe using your new option)
- output says you can't because you are missing a, b and c; suggests I use
--record-only or do the missing merge if it is really missing
Then I can iterate over this sequence until I succeed and the elision
occurs.
Thanks.
Re: cleaning up mergeinfo using svn 1.7
Posted by Stefan Sperling <st...@elego.de>.
On Fri, Sep 09, 2011 at 03:00:53PM -0400, David Carson wrote:
> OK, so let's try again. I've merged a change from another branch, which is
> a commit with a single file changed. The result is much better, but let me
> verify that I understand my results:
> - file a/b/c/foo.c shows the file changes -- OK
> - file a/b/c/foo.c also contains svn:mergeinfo changes -- I assume this is
> because it happens to be one of the many files that got contaminated with
> the property prior to this test. So, since it is part of the changeset for
> my merge, the prop has to be updated too.
> - <top-level> directory (call it TLD/) contains svn:mergeinfo changes -- OK
> - TLD/a also contains svn:mergeinfo changes -- Same as above. Directory
> TLD/a is one of the directories which was previously contaminated, and since
> this path is 'involved' in the change, the prop has to be updated. I
> wouldn't say the path really involved, except that it is the path on wich
> foo.c sits. After all, TLD/a/ is not itself changed. Nonetheless, I can
> live with this.
This is because it's not about which "objects" (as you called them ealier)
are affected by the merge, but which subtrees of the merge target are affected.
In your case, it looks as if the target of the merge is the working copy
root. An affected subtree of the root is a, an affected subtree
of a is b, and so on, until you get to foo.c.
If you merge from ^/branch/a/b/c/foo.c into wc-path/a/b/c/foo.c
only foo.c should receive mergeinfo updates, because it is the
merge target and has no children.
> Now, are there any tools to aid in cleanup of the mess? Besides the large
> number of files/dirs with the prop attached, the prop itself has a large
> number of paths to contend with.
There doesn't seem to be any officially sanctioned script.
There are various scripts people have written and posted to this list.
One is here: http://svn.haxx.se/dev/archive-2009-05/0438.shtml
You could try the archive search on svn.haxx.se -- maybe you'll find
something useful.
However, I would say that the need for such a script implies a lack
of functionality in Subversion itself. 'svn merge' does try to elide
mergeinfo when subtree mergeinfo becomes redundant, but AFAIK the
current implementation is a bit simplistic.
It would help if you could present examples where elision does not
work for you, so we can fix 'svn merge'.
We could allow the existing elision code to be invoked without running
a merge. An option like --elide-mergeinfo could be added to the
'svn mergeinfo' command for this purpose. Would this be useful?
Re: cleaning up mergeinfo using svn 1.7
Posted by David Carson <dc...@gmail.com>.
On Fri, Sep 9, 2011 at 12:38 PM, Stefan Sperling <st...@elego.de> wrote:
> On Fri, Sep 09, 2011 at 12:17:15PM -0400, David Carson wrote:
> > To that end, I installed 1.7.0-rc2 with a snapshot of our repository, so
> I
> > could experiment and see for myself. I did a simple merge from one
> branch
> > to another. I was disappointed to see that the same miscellaneous group
> of
> > files and directories below the top-level are having their snv:mergeinfo
> > modified, even though they do not participate in the change. I guess I
> am
> > not reading the documentation for 1.7 correctly, because I thought that
> this
> > was one of the major changes; namely, that any object which is not
> affected
> > by the merged changes would not have its svn:mergeinfo property changed.
>
> Did the change you were merging contain svn:mergeinfo property changes?
> If so, those will be applied to the working copy.
>
> If the revision rN you are merging from 1.2 into 1.3 was itself the
> result of a merge, and this merge was done with a 1.6 client,
> this would explain the problem.
>
Bingo -- that makes sense. In my test, I updated a child branch with new
parent content and that range of revisions contained one that was itself a
merge which was performed using svn 1.6 (i.e., before taking the snapshot of
the repository).
OK, so let's try again. I've merged a change from another branch, which is
a commit with a single file changed. The result is much better, but let me
verify that I understand my results:
- file a/b/c/foo.c shows the file changes -- OK
- file a/b/c/foo.c also contains svn:mergeinfo changes -- I assume this is
because it happens to be one of the many files that got contaminated with
the property prior to this test. So, since it is part of the changeset for
my merge, the prop has to be updated too.
- <top-level> directory (call it TLD/) contains svn:mergeinfo changes -- OK
- TLD/a also contains svn:mergeinfo changes -- Same as above. Directory
TLD/a is one of the directories which was previously contaminated, and since
this path is 'involved' in the change, the prop has to be updated. I
wouldn't say the path really involved, except that it is the path on wich
foo.c sits. After all, TLD/a/ is not itself changed. Nonetheless, I can
live with this.
Now, are there any tools to aid in cleanup of the mess? Besides the large
number of files/dirs with the prop attached, the prop itself has a large
number of paths to contend with.
Re: cleaning up mergeinfo using svn 1.7
Posted by Stefan Sperling <st...@elego.de>.
On Fri, Sep 09, 2011 at 12:17:15PM -0400, David Carson wrote:
> To that end, I installed 1.7.0-rc2 with a snapshot of our repository, so I
> could experiment and see for myself. I did a simple merge from one branch
> to another. I was disappointed to see that the same miscellaneous group of
> files and directories below the top-level are having their snv:mergeinfo
> modified, even though they do not participate in the change. I guess I am
> not reading the documentation for 1.7 correctly, because I thought that this
> was one of the major changes; namely, that any object which is not affected
> by the merged changes would not have its svn:mergeinfo property changed.
Did the change you were merging contain svn:mergeinfo property changes?
If so, those will be applied to the working copy.
If the revision rN you are merging from 1.2 into 1.3 was itself the
result of a merge, and this merge was done with a 1.6 client,
this would explain the problem.