You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Stephen Armstrong <St...@nanometrics.ca> on 2008/01/30 21:58:41 UTC
Very slow merge for Feature Branch
Hello all,
Our company uses developer branches much like the practice of Feature
Branches described in the svn book
(http://svnbook.red-bean.com/en/1.4/svn.branchmerge.commonuses.html#svn.branchmerge.commonuses.patterns.feature).
We've been noticing that merges are taking quite a while on some larger
projects (15 - 20 min), regardless of how many changes were made.
Because of other practices, we've always got a working copy of our
branch, and of the trunk on disk, but we can't do the merge with a
command such as:
merge ./trunk ./branch ./trunk
It seems that when specifying a working copy for the merge, it only uses
it like an anchor, to get the URL for the repo. Is there a way we can do
the merge client-side? Some kind of offline-merge?
Thanks
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Very slow merge for Feature Branch
Posted by Stephen Armstrong <St...@nanometrics.ca>.
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Bicking, David (HHoldings, IT) wrote:
<blockquote
cite="mid:53F4EE95A373AB4887E261A376BC130D02D44B9A@AD1HFDEXC306.ad1.prod"
type="cite">
<blockquote type="cite">
<pre wrap="">-----Original Message-----
From: Stephen Armstrong [<a class="moz-txt-link-freetext" href="mailto:StephenArmstrong@nanometrics.ca">mailto:StephenArmstrong@nanometrics.ca</a>]</pre>
</blockquote>
<pre wrap=""><!---->-----------------< much history removed >-------------------
</pre>
<blockquote type="cite">
<pre wrap="">The command we were using is:
svn merge trunk@HEAD branch@HEAD trunk
Now, according to ancestry, branch has had most of it's files
modified since it was created months ago. But since last time
we ran this merge (and so synced all files) we've only
modified 1 file. It still takes a while (apparently it's only
7 min or so. Not 15-20 as I was led to believe), but as I've
said, a new branch with the same amount of changes seems to
happen almost instantly. I would assume this is because the
diff uses ancestry to eliminate any that haven't been
modified since branch creation and diff those that have.
It looks like getting our users to delete their branch and
recreate it after merging all finished changes into trunk
will solve the problem by keeping all branches young.
</pre>
</blockquote>
<pre wrap=""><!---->
I suspect your merges are scanning back to the branch point, when that
is not necessary. If I am correct, there are three better solutions.
1) when you merge, identify which revisions were merged in the commit
log. The next time you merge, view that information, and only merge the
revisions that aren't already merged (usually the subsequent ones).
2) Use svnmerge.py, which adds properties to the projects that keep
track of already merged revisions, thus making the merge (potentially)
as easy as "svnmerge merge".
3) Use Subversion 1.5 and the merge tracking feature.
I'm interested in hearing if any of those improve your experience!
--
David
</pre>
</blockquote>
We are already using a home-grown approach that I suspect is much like
#2. We've been waiting for svn 1.5 to come out, and when it does we'll
probably upgrade once it's had a chance to 'settle' in the community
for a couple months, and we've looked it over.<br>
<br>
My company switched from Clearcase to SVN a month or two before I
started, so the terminology we use comes from there I think.<br>
Rebase - merge in all changes that have been done on trunk into your
branch (keep your branch up-to-date with changes on trunk)<br>
Deliver - merge changes you have done to your branch into trunk<br>
<br>
Our rebase scripts use a property on the branch called 'lastrebase', so
to keep our branch up to date, the merge command is something like:<br>
merge <a class="moz-txt-link-freetext" href="http://.../project/trunk@LAST_REBASE">http://.../project/trunk@LAST_REBASE</a>
<a class="moz-txt-link-freetext" href="http://.../project/trunk@HEAD">http://.../project/trunk@HEAD</a> ./branchWC<br>
The script then bumps up the 'lastrebase' property and commits.<br>
<br>
Our deliver scripts require that you have just rebased your branch
before running the deliver script, so the only difference between your
branch and the trunk are the changes you made. The deliver script
doesn't do a merge based on revision ranges because if it did, the
range would include some commits to the branch that were done by the
rebase script, and so changes that were done on trunk, and merged into
your branch, would be re-merged into trunk. I think this is the reason
why suggestion #1 wouldn't work for us, but if I'm mis-understanding
it, let me know.<br>
<br>
Thanks<br>
Steve<br>
</body>
</html>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Re: Very slow merge for Feature Branch
Posted by "Bicking, David (HHoldings, IT)" <Da...@thehartford.com>.
> -----Original Message-----
> From: Stephen Armstrong [mailto:StephenArmstrong@nanometrics.ca]
>
-----------------< much history removed >-------------------
> The command we were using is:
>
> svn merge trunk@HEAD branch@HEAD trunk
>
> Now, according to ancestry, branch has had most of it's files
> modified since it was created months ago. But since last time
> we ran this merge (and so synced all files) we've only
> modified 1 file. It still takes a while (apparently it's only
> 7 min or so. Not 15-20 as I was led to believe), but as I've
> said, a new branch with the same amount of changes seems to
> happen almost instantly. I would assume this is because the
> diff uses ancestry to eliminate any that haven't been
> modified since branch creation and diff those that have.
>
> It looks like getting our users to delete their branch and
> recreate it after merging all finished changes into trunk
> will solve the problem by keeping all branches young.
I suspect your merges are scanning back to the branch point, when that
is not necessary. If I am correct, there are three better solutions.
1) when you merge, identify which revisions were merged in the commit
log. The next time you merge, view that information, and only merge the
revisions that aren't already merged (usually the subsequent ones).
2) Use svnmerge.py, which adds properties to the projects that keep
track of already merged revisions, thus making the merge (potentially)
as easy as "svnmerge merge".
3) Use Subversion 1.5 and the merge tracking feature.
I'm interested in hearing if any of those improve your experience!
--
David
*************************************************************************
This communication, including attachments, is
for the exclusive use of addressee and may contain proprietary,
confidential and/or privileged information. If you are not the intended
recipient, any use, copying, disclosure, dissemination or distribution is
strictly prohibited. If you are not the intended recipient, please notify
the sender immediately by return e-mail, delete this communication and
destroy all copies.
*************************************************************************
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Very slow merge for Feature Branch
Posted by Stephen Armstrong <St...@nanometrics.ca>.
Ryan Schmidt wrote:
> On Jan 31, 2008, at 12:36, Stephen Armstrong wrote:
>> Stephen Armstrong wrote:
>>> Ryan Schmidt wrote:
>>>> On Jan 30, 2008, at 15:58, Stephen Armstrong wrote:
>>>>
>>>>> Our company uses developer branches much like the practice of
>>>>> Feature Branches described in the svn book
>>>>> (http://svnbook.red-bean.com/en/1.4/svn.branchmerge.commonuses.html#svn.branchmerge.commonuses.patterns.feature).
>>>>> We've been noticing that merges are taking quite a while on some
>>>>> larger projects (15 - 20 min), regardless of how many changes were
>>>>> made.
>>>>
>>>> That seems to be a long time, but what is "larger"? How many files
>>>> and directories are we talking about? How much space on disk does a
>>>> working copy consume?
>>> When I checked out (and exported, so the .svn folders aren't
>>> interfering) trunk, it's 360 files, 58 folders, totaling 18.3MB.
>>>>> Because of other practices, we've always got a working copy of our
>>>>> branch, and of the trunk on disk, but we can't do the merge with a
>>>>> command such as:
>>>>>
>>>>> merge ./trunk ./branch ./trunk
>>>>>
>>>>> It seems that when specifying a working copy for the merge, it
>>>>> only uses it like an anchor, to get the URL for the repo. Is there
>>>>> a way we can do the merge client-side? Some kind of offline-merge?
>>>> I don't think you can use local working copies to do the merge.
>>> Are there known practices that will significantly increase a merge
>>> time? I'm grasping at straws thinking maybe there's alot of
>>> properties set on files in that project, and so all the PROPGET
>>> requests to the server are causing extra slow-down. Also, as
>>> comparison, another project that is 342 files, 85 folders, but only
>>> 1.5MB merges very quickly, so it seems like number of files isn't
>>> the cause, but possibly sheer size could be.
>> I'm replying to my own message to present more information.
>>
>> I think the slow-down is caused by how 'long' the branch has existed.
>> The two people working on this project have both had their developer
>> branches around for quite a while. A third user created a new branch
>> and did a merge, and it was almost instant. So the slowness must be
>> caused by merge's tracing back up the ancestry to where the branch
>> was originally created. Even though a.txt is the only file changed
>> _since the last merge_, hundreds of other files have been changed
>> since branch creation, so merge compares them to trunk, and finds
>> them identical.
>>
>> Am I right in how merge works? When it looks for ancestry, if the
>> file hasn't changed since branch creation, it doesn't compare it? So
>> what we're seeing here is the unneeded download and compare of many
>> identical files? (Unneeded for my case, but it makes sense for
>> subversion to be doing it)
> merge is just a diff and patch. Maybe you'd better show us the exact
> merge command you're typing...
The command we were using is:
svn merge trunk@HEAD branch@HEAD trunk
Now, according to ancestry, branch has had most of it's files modified
since it was created months ago. But since last time we ran this merge
(and so synced all files) we've only modified 1 file. It still takes a
while (apparently it's only 7 min or so. Not 15-20 as I was led to
believe), but as I've said, a new branch with the same amount of changes
seems to happen almost instantly. I would assume this is because the
diff uses ancestry to eliminate any that haven't been modified since
branch creation and diff those that have.
It looks like getting our users to delete their branch and recreate it
after merging all finished changes into trunk will solve the problem by
keeping all branches young.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Very slow merge for Feature Branch
Posted by Ryan Schmidt <su...@ryandesign.com>.
On Jan 31, 2008, at 12:36, Stephen Armstrong wrote:
> Stephen Armstrong wrote:
>> Ryan Schmidt wrote:
>>> On Jan 30, 2008, at 15:58, Stephen Armstrong wrote:
>>>
>>>> Our company uses developer branches much like the practice of
>>>> Feature Branches described in the svn book (http://svnbook.red-
>>>> bean.com/en/1.4/
>>>> svn.branchmerge.commonuses.html#svn.branchmerge.commonuses.patterns
>>>> .feature). We've been noticing that merges are taking quite a
>>>> while on some larger projects (15 - 20 min), regardless of how
>>>> many changes were made.
>>>
>>> That seems to be a long time, but what is "larger"? How many
>>> files and directories are we talking about? How much space on
>>> disk does a working copy consume?
>> When I checked out (and exported, so the .svn folders aren't
>> interfering) trunk, it's 360 files, 58 folders, totaling 18.3MB.
>>>> Because of other practices, we've always got a working copy of
>>>> our branch, and of the trunk on disk, but we can't do the merge
>>>> with a command such as:
>>>>
>>>> merge ./trunk ./branch ./trunk
>>>>
>>>> It seems that when specifying a working copy for the merge, it
>>>> only uses it like an anchor, to get the URL for the repo. Is
>>>> there a way we can do the merge client-side? Some kind of
>>>> offline-merge?
>>> I don't think you can use local working copies to do the merge.
>> Are there known practices that will significantly increase a merge
>> time? I'm grasping at straws thinking maybe there's alot of
>> properties set on files in that project, and so all the PROPGET
>> requests to the server are causing extra slow-down. Also, as
>> comparison, another project that is 342 files, 85 folders, but
>> only 1.5MB merges very quickly, so it seems like number of files
>> isn't the cause, but possibly sheer size could be.
> I'm replying to my own message to present more information.
>
> I think the slow-down is caused by how 'long' the branch has
> existed. The two people working on this project have both had their
> developer branches around for quite a while. A third user created a
> new branch and did a merge, and it was almost instant. So the
> slowness must be caused by merge's tracing back up the ancestry to
> where the branch was originally created. Even though a.txt is the
> only file changed _since the last merge_, hundreds of other files
> have been changed since branch creation, so merge compares them to
> trunk, and finds them identical.
>
> Am I right in how merge works? When it looks for ancestry, if the
> file hasn't changed since branch creation, it doesn't compare it?
> So what we're seeing here is the unneeded download and compare of
> many identical files? (Unneeded for my case, but it makes sense for
> subversion to be doing it)
merge is just a diff and patch. Maybe you'd better show us the exact
merge command you're typing...
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Very slow merge for Feature Branch
Posted by Stephen Armstrong <St...@nanometrics.ca>.
Stephen Armstrong wrote:
> Ryan Schmidt wrote:
>> On Jan 30, 2008, at 15:58, Stephen Armstrong wrote:
>>
>>> Our company uses developer branches much like the practice of
>>> Feature Branches described in the svn book
>>> (http://svnbook.red-bean.com/en/1.4/svn.branchmerge.commonuses.html#svn.branchmerge.commonuses.patterns.feature).
>>> We've been noticing that merges are taking quite a while on some
>>> larger projects (15 - 20 min), regardless of how many changes were
>>> made.
>>
>> That seems to be a long time, but what is "larger"? How many files
>> and directories are we talking about? How much space on disk does a
>> working copy consume?
> When I checked out (and exported, so the .svn folders aren't
> interfering) trunk, it's 360 files, 58 folders, totaling 18.3MB.
>>> Because of other practices, we've always got a working copy of our
>>> branch, and of the trunk on disk, but we can't do the merge with a
>>> command such as:
>>>
>>> merge ./trunk ./branch ./trunk
>>>
>>> It seems that when specifying a working copy for the merge, it only
>>> uses it like an anchor, to get the URL for the repo. Is there a way
>>> we can do the merge client-side? Some kind of offline-merge?
>> I don't think you can use local working copies to do the merge.
> Are there known practices that will significantly increase a merge
> time? I'm grasping at straws thinking maybe there's alot of properties
> set on files in that project, and so all the PROPGET requests to the
> server are causing extra slow-down. Also, as comparison, another
> project that is 342 files, 85 folders, but only 1.5MB merges very
> quickly, so it seems like number of files isn't the cause, but
> possibly sheer size could be.
I'm replying to my own message to present more information.
I think the slow-down is caused by how 'long' the branch has existed.
The two people working on this project have both had their developer
branches around for quite a while. A third user created a new branch and
did a merge, and it was almost instant. So the slowness must be caused
by merge's tracing back up the ancestry to where the branch was
originally created. Even though a.txt is the only file changed _since
the last merge_, hundreds of other files have been changed since branch
creation, so merge compares them to trunk, and finds them identical.
Am I right in how merge works? When it looks for ancestry, if the file
hasn't changed since branch creation, it doesn't compare it? So what
we're seeing here is the unneeded download and compare of many identical
files? (Unneeded for my case, but it makes sense for subversion to be
doing it)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Very slow merge for Feature Branch
Posted by Stephen Armstrong <St...@nanometrics.ca>.
Ryan Schmidt wrote:
> On Jan 30, 2008, at 15:58, Stephen Armstrong wrote:
>
>> Our company uses developer branches much like the practice of Feature
>> Branches described in the svn book
>> (http://svnbook.red-bean.com/en/1.4/svn.branchmerge.commonuses.html#svn.branchmerge.commonuses.patterns.feature).
>> We've been noticing that merges are taking quite a while on some
>> larger projects (15 - 20 min), regardless of how many changes were made.
>
> That seems to be a long time, but what is "larger"? How many files and
> directories are we talking about? How much space on disk does a
> working copy consume?
When I checked out (and exported, so the .svn folders aren't
interfering) trunk, it's 360 files, 58 folders, totaling 18.3MB.
>> Because of other practices, we've always got a working copy of our
>> branch, and of the trunk on disk, but we can't do the merge with a
>> command such as:
>>
>> merge ./trunk ./branch ./trunk
>>
>> It seems that when specifying a working copy for the merge, it only
>> uses it like an anchor, to get the URL for the repo. Is there a way
>> we can do the merge client-side? Some kind of offline-merge?
> I don't think you can use local working copies to do the merge.
Are there known practices that will significantly increase a merge time?
I'm grasping at straws thinking maybe there's alot of properties set on
files in that project, and so all the PROPGET requests to the server are
causing extra slow-down. Also, as comparison, another project that is
342 files, 85 folders, but only 1.5MB merges very quickly, so it seems
like number of files isn't the cause, but possibly sheer size could be.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Very slow merge for Feature Branch
Posted by Ryan Schmidt <su...@ryandesign.com>.
On Jan 30, 2008, at 15:58, Stephen Armstrong wrote:
> Our company uses developer branches much like the practice of
> Feature Branches described in the svn book (http://svnbook.red-
> bean.com/en/1.4/
> svn.branchmerge.commonuses.html#svn.branchmerge.commonuses.patterns.fe
> ature). We've been noticing that merges are taking quite a while on
> some larger projects (15 - 20 min), regardless of how many changes
> were made.
That seems to be a long time, but what is "larger"? How many files
and directories are we talking about? How much space on disk does a
working copy consume?
> Because of other practices, we've always got a working copy of our
> branch, and of the trunk on disk, but we can't do the merge with a
> command such as:
>
> merge ./trunk ./branch ./trunk
>
> It seems that when specifying a working copy for the merge, it only
> uses it like an anchor, to get the URL for the repo. Is there a way
> we can do the merge client-side? Some kind of offline-merge?
I don't think you can use local working copies to do the merge.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org