You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Luiz Daniel Lapolla <lu...@sfw.com.br> on 2004/03/04 21:56:41 UTC
Branching models
Hi,
We are improving our development process. We have been using CVSNT over
the last two years. All of our development goes directly on the mainline.
At each milestone a branch is created and development keeps going on
mainline.
This process led us into several problems with QA, broken builds and
concurrent development.
We are considering the adoption of a branch-by-task model:
- A new branch (across all files) is created for each Change Request,
based on the last stable milestone.
- All the development for one CR goes on it's own branch.
- After N cicles of testing and bugfixing the branch is scheduled to be
merged with it's baseline.
We are concerned with CVSNT performance, since our smaller project has
15,000 files and our development team throughput is around 150 change
requests per month.Recently, I read about subversion cheap copies and I'm hopping it canhandle all those branches that we are willing to create.Does this branch politic affects subversion performance? How it can becompared with CVS? Does anyone had a similar experience?Thanks in advanceDaniel
Re: Branching models
Posted by Mike Mason <mg...@thoughtworks.net>.
Luiz Daniel Lapolla wrote:
>Hi,
>
>We are improving our development process. We have been using CVSNT over
>the last two years. All of our development goes directly on the mainline.
>At each milestone a branch is created and development keeps going on
>mainline.
>
>This process led us into several problems with QA, broken builds and
>concurrent development.
>
>We are considering the adoption of a branch-by-task model:
> - A new branch (across all files) is created for each Change Request,
>based on the last stable milestone.
> - All the development for one CR goes on it's own branch.
> - After N cicles of testing and bugfixing the branch is scheduled to be
>merged with it's baseline.
>
>
The Perforce folks have a few things to say about using SCM effectively:
http://www.knowledgestorm.com/collateral/WTP/50369_07499_07194_bestpractices.pdf
Branching too much is probably not that helpful -- their advice is to
branch late, and because of conflicting policy for two codelines. How
big is a CR?
Cheers,
Mike.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Branching models
Posted by Philip Martin <ph...@codematters.co.uk>.
Philip Martin <ph...@codematters.co.uk> writes:
> If you can define some criteria to determine when a trunk commit
> should be allowed then it may be possible to use a pre-commit hook
> to reject commits that are not switched.
As an alternative, it may be possible to use mod_auth_auth_svn to
restrict non-switched commits.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Branching models
Posted by Philip Martin <ph...@codematters.co.uk>.
Chris Wein <cw...@mobilygen.com> writes:
> The answer to this is svn switch where you switch only the subset of
> files you are working. An svn update should update all non-switched
> files to the latest trunk revision without merging, with the
> developer then having to merge the trunk into the branch to resolve
> files changed both on the developer branch and between revs 100 and
> 110. Does this seem right so far?
Yes, although you may want to merge just the switched files.
> One concern I have is that the it is easy to forget to svn switch a file
> before editing it so that you inadvertently edit a trunk file instead of
> branching first. ClearCase handles this by forcing a switch before the
> file can be edited (hard to explain if you dont know clearcase). I am
> sure I can write a script to make it easier but I am not sure how I can
> remove the risk of inadvertent trunk editing.
It is possible to switch at any stage including after editing, so
don't bother trying to enforce the switch before editing. Concentrate
on making sure that the modified files are switched before commit (svn
status will indicate which files are modified and which files are
switched). If you can define some criteria to determine when a trunk
commit should be allowed then it may be possible to use a pre-commit
hook to reject commits that are not switched.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Branching models
Posted by Chris Wein <cw...@mobilygen.com>.
I am going to be implementing a similar scheme here. I come from a
ClearCase background where it happens essentially automatically with a
trivial configspec.
The one issue I am trying to sort out is uplevelling after the branch
has been done. That is, if the branch is made from revision 100 (for
example) and the fix takes a while, the developer will want to uplevel
from revision 100 to 110 (again, for example) which basically means
merging from 110 into the branch. If the entire tree is branched then
the merge from 110 will merge *all* the intermediate changes into the
developer's branch which is not the right thing to do. The answer to
this is svn switch where you switch only the subset of files you are
working. An svn update should update all non-switched files to the
latest trunk revision without merging, with the developer then having to
merge the trunk into the branch to resolve files changed both on the
developer branch and between revs 100 and 110. Does this seem right so
far?
One concern I have is that the it is easy to forget to svn switch a file
before editing it so that you inadvertently edit a trunk file instead of
branching first. ClearCase handles this by forcing a switch before the
file can be edited (hard to explain if you dont know clearcase). I am
sure I can write a script to make it easier but I am not sure how I can
remove the risk of inadvertent trunk editing.
How do people handle this?
On Thu, 2004-03-04 at 14:18, Jim Nutt wrote:
> On Thu, 4 Mar 2004 18:56:41 -0300
> "Luiz Daniel Lapolla" <lu...@sfw.com.br> wrote:
>
> > We are considering the adoption of a branch-by-task model:
> > - A new branch (across all files) is created for each Change
> > Request,
> > based on the last stable milestone.
> > - All the development for one CR goes on it's own branch.
> > - After N cicles of testing and bugfixing the branch is scheduled
> > to be
> > merged with it's baseline.
>
> This is the development model we're currently using. Performance seems
> to be good, but we've not really been using SVN long enough to get solid
> numbers.
>
> One caveat is that merging the whole branch back is a bit different than
> in CVS (you have to specify the starting revision of the branch). It's
> not that it's any more difficult, it's just that it's different enough
> that it's easy to make mistakes until you get the hang of it (the first
> few times I did it nearly drove me to distraction). Once you get used
> to it it though, it's not any harder than under CVS and has some
> advantages.
>
>
> --
> jim nutt
>
> home: jim@nuttz.org jabber: jimnutt@jabber.com
> work: jimnutt@vestek.com ms msg: jim@nuttz.org
> pgp id: 1ECBCC78
--
Chris Wein
Software Engineer
Mobilygen Corp.
E-Mail : cwein@mobilygen.com
Phone : 408-869-4035
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Branching models
Posted by Mike Mason <mg...@thoughtworks.net>.
Luiz Daniel Lapolla wrote:
>I'm aware of the performance issue for fetching an old revision in a
>dev-line whith lots of revisions.CVS has the same problem. Currently I'm
>using CVSNT and this issue is not a problem for us since almost all the
>ongoing work is very close to the head.
>
>
Subversion stores files as fully expanded versions, and as deltas
between the versions. It has a fancy algorithm internally so the maximum
number of deltas it needs to apply to get to a particular version
increases logarithmically with the number of versions stored. Basically,
retrieving an old version is pretty cheap.
Cheers,
Mike.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Branching models
Posted by Luiz Daniel Lapolla <lu...@sfw.com.br>.
The number seven I mentioned early refers to the number of branches created
during one day (of course there is an implicit commit but the only modified
file is the new branch parent directory).
In each branch the amount of files changed/created is small, ranging from a
dozen to maybe fifty (rarely). I have several small images but they are
modified rarely. The number of commits (on each branch) tends to be small,
in most cases one or two commits.
I'm aware of the performance issue for fetching an old revision in a
dev-line whith lots of revisions.CVS has the same problem. Currently I'm
using CVSNT and this issue is not a problem for us since almost all the
ongoing work is very close to the head.
----- Original Message -----
From: "C.A.T.Magic" <c....@gmx.at>
To: "Ben Collins-Sussman" <su...@collab.net>; "Luiz Daniel Lapolla"
<lu...@sfw.com.br>
Cc: <us...@subversion.tigris.org>
Sent: Friday, March 05, 2004 2:25 PM
Subject: Re: Branching models
>
> ----- Original Message -----
> From: "Ben Collins-Sussman" <su...@collab.net>
>
> > So make as many copies as you want. Is has nothing to do with
> > performance.
>
>
> I -think- it will depend on the amount of data changed
> on each revision, the size of the individual files
> and the size of the changesets.
>
> If the mentioned 15,000 files are purely text, html, .c
> (i.e. non-binary) files this is not a problem at all,
> since -real- changes on these files tend to be very small.
> I put about 60,000 .c and even many .png files into SVN
> and it still runs ok (up to ~ rev. 100 yet)
> but maybe I'll run a test on it overnight and push
> it up to some rev. 10000 to verify this :-)
>
> ( To give a worst-case example: if you'd put 15.000
> .png files into svn and RGB-color-correct all them 3 times
> a day and commit them, your DB would expand and slow
> down like hell -- binary files that change often should
> always be handled with care when using an SCM :-)
>
>
> ( note: the initial question was on 7 commits a day = 2555 a year )
>
> After reading through
> http://subversion.tigris.org/files/documents/15/17/svn-design.html#Schema
> I -think- (devs, please tell me if I got it wrong)
> that access to >recent revisions< of the repository will
> remain quite fast, independent from the amount of revisions
> in the DB.
> but access to -very-old- file revisions (e.g. rev. 1) get slightly
> slower with each new revision that is committed, since all
> data is stored as 'diffs' to the previous file, and all the,
> say 2555 diffs have to be applied in reverse order.
> Applying 2555 diffs on, say, 1MB files could get pretty slow -
> and that slowness gets multiplied by 15.000 files in the worst case.
>
>
> But note, that other revision control systems work mostly similar,
> and suffer from equal or even larger limitations.
>
>
> :-)
> =====
> c.a.t.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Branching models
Posted by "C.A.T.Magic" <c....@gmx.at>.
----- Original Message -----
From: "Ben Collins-Sussman" <su...@collab.net>
> So make as many copies as you want. Is has nothing to do with
> performance.
I -think- it will depend on the amount of data changed
on each revision, the size of the individual files
and the size of the changesets.
If the mentioned 15,000 files are purely text, html, .c
(i.e. non-binary) files this is not a problem at all,
since -real- changes on these files tend to be very small.
I put about 60,000 .c and even many .png files into SVN
and it still runs ok (up to ~ rev. 100 yet)
but maybe I'll run a test on it overnight and push
it up to some rev. 10000 to verify this :-)
( To give a worst-case example: if you'd put 15.000
.png files into svn and RGB-color-correct all them 3 times
a day and commit them, your DB would expand and slow
down like hell -- binary files that change often should
always be handled with care when using an SCM :-)
( note: the initial question was on 7 commits a day = 2555 a year )
After reading through
http://subversion.tigris.org/files/documents/15/17/svn-design.html#Schema
I -think- (devs, please tell me if I got it wrong)
that access to >recent revisions< of the repository will
remain quite fast, independent from the amount of revisions
in the DB.
but access to -very-old- file revisions (e.g. rev. 1) get slightly
slower with each new revision that is committed, since all
data is stored as 'diffs' to the previous file, and all the,
say 2555 diffs have to be applied in reverse order.
Applying 2555 diffs on, say, 1MB files could get pretty slow -
and that slowness gets multiplied by 15.000 files in the worst case.
But note, that other revision control systems work mostly similar,
and suffer from equal or even larger limitations.
:-)
=====
c.a.t.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Branching models
Posted by Luiz Daniel Lapolla <lu...@sfw.com.br>.
Ok, how I understood better the concept of cheap copies.
Thanks for the good work.
Daniel
----- Original Message -----
From: "Ben Collins-Sussman" <su...@collab.net>
To: "Luiz Daniel Lapolla" <lu...@sfw.com.br>
Cc: <us...@subversion.tigris.org>
Sent: Friday, March 05, 2004 1:13 PM
Subject: Re: Branching models
> On Fri, 2004-03-05 at 09:48, Luiz Daniel Lapolla wrote:
>
> > Can subversion cheap copies cope with all this branching without loosing
> > (too much) performance?
>
> When you branch in subversion, all you're doing is duplicating a single
> directory node. It's like making a hard-link. That's why it's O(1)
> time. When you start changing things underneath the branch-copy, then
> new nodes start getting generated "just in time"... any unchanged nodes
> are still shared between the two trees.
>
> So make as many copies as you want. Is has nothing to do with
> performance.
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Branching models
Posted by Ben Collins-Sussman <su...@collab.net>.
On Fri, 2004-03-05 at 09:48, Luiz Daniel Lapolla wrote:
> Can subversion cheap copies cope with all this branching without loosing
> (too much) performance?
When you branch in subversion, all you're doing is duplicating a single
directory node. It's like making a hard-link. That's why it's O(1)
time. When you start changing things underneath the branch-copy, then
new nodes start getting generated "just in time"... any unchanged nodes
are still shared between the two trees.
So make as many copies as you want. Is has nothing to do with
performance.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Branching models
Posted by Andreas Kostyrka <an...@kostyrka.org>.
On Fri, Mar 05, 2004 at 12:48:16PM -0300, Luiz Daniel Lapolla wrote:
> Thanks Jim,
>
> It's good to know I'm not alone in this matter. I'm still concerned about
> subversion behavior in the long run. I have no idea if branching 15,000
> files, 7 times per day will let my repository slower after two or three
> years of development. I already know that it's not a good idea to do this
> with CVSNT.
>
> Can subversion cheap copies cope with all this branching without loosing
> (too much) performance?
>
> Maybe someone from the dev team can make things clear?
What about trying it yourself? Write a small script that generates the
noise^H^H^H^H^H^Hprogram developement, and see it for yourself?
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Branching models
Posted by Luiz Daniel Lapolla <lu...@sfw.com.br>.
Thanks Jim,
It's good to know I'm not alone in this matter. I'm still concerned about
subversion behavior in the long run. I have no idea if branching 15,000
files, 7 times per day will let my repository slower after two or three
years of development. I already know that it's not a good idea to do this
with CVSNT.
Can subversion cheap copies cope with all this branching without loosing
(too much) performance?
Maybe someone from the dev team can make things clear?
Thanks in advance.
Daniel
----- Original Message -----
From: "Jim Nutt" <ji...@nuttz.org>
To: <us...@subversion.tigris.org>
Sent: Thursday, March 04, 2004 7:18 PM
Subject: Re: Branching models
On Thu, 4 Mar 2004 18:56:41 -0300
"Luiz Daniel Lapolla" <lu...@sfw.com.br> wrote:
> We are considering the adoption of a branch-by-task model:
> - A new branch (across all files) is created for each Change
> Request,
> based on the last stable milestone.
> - All the development for one CR goes on it's own branch.
> - After N cicles of testing and bugfixing the branch is scheduled
> to be
> merged with it's baseline.
This is the development model we're currently using. Performance seems
to be good, but we've not really been using SVN long enough to get solid
numbers.
One caveat is that merging the whole branch back is a bit different than
in CVS (you have to specify the starting revision of the branch). It's
not that it's any more difficult, it's just that it's different enough
that it's easy to make mistakes until you get the hang of it (the first
few times I did it nearly drove me to distraction). Once you get used
to it it though, it's not any harder than under CVS and has some
advantages.
--
jim nutt
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Branching models
Posted by Jim Nutt <ji...@nuttz.org>.
On Thu, 4 Mar 2004 18:56:41 -0300
"Luiz Daniel Lapolla" <lu...@sfw.com.br> wrote:
> We are considering the adoption of a branch-by-task model:
> - A new branch (across all files) is created for each Change
> Request,
> based on the last stable milestone.
> - All the development for one CR goes on it's own branch.
> - After N cicles of testing and bugfixing the branch is scheduled
> to be
> merged with it's baseline.
This is the development model we're currently using. Performance seems
to be good, but we've not really been using SVN long enough to get solid
numbers.
One caveat is that merging the whole branch back is a bit different than
in CVS (you have to specify the starting revision of the branch). It's
not that it's any more difficult, it's just that it's different enough
that it's easy to make mistakes until you get the hang of it (the first
few times I did it nearly drove me to distraction). Once you get used
to it it though, it's not any harder than under CVS and has some
advantages.
--
jim nutt
home: jim@nuttz.org jabber: jimnutt@jabber.com
work: jimnutt@vestek.com ms msg: jim@nuttz.org
pgp id: 1ECBCC78