You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Kean Johnston <jk...@sco.com> on 2005/11/08 01:51:32 UTC
FAQ: svn advocacy / advice
Good evening subverters,
We (SCO) are looking for a new revision control tool to
replace a historic SCCS tree, and hopefully, a ClearCase
tree as well. The source base is pretty large, and dates
back to the mid-70's. The current contender is cvs, becuase
it is such a well-known quantity, but I would much prefer
to use svn. However, both myself and others on the project
have a few concerns and questions, and I would appreciate
some advice.
I am sure much of this is covered in the doc, but I need
a fairly quick answer to justify spending the time doing
research. If these are all FAQ, I appologise but hope you
will indulge me. Yes/no answers are acceptable :)
1. Is fsfs the prefered solution? If so, since each revision
seems to be set of changes relative to the previous one,
considering the amount of history and size of tree and
number of revisions, does svn actually need to play back
from revision 1 or does it store some sort of "checkpoint"
along the way? If it does, is that automatic or manual?
Also, if it does, is that "checkpoint" then basically the
size of the repository at that point in time?
2. I know there is a cvs to svn converter, and I've seen some
threads about an SCCS to svn converter using cvs in between,
but does anyone know of a direct SCCS to svn converter? If
not, does svn's API lend itself to me writing such a tool?
Anyone have any WAG what the scope of such a project would
be?
3. Is it possible (even if slow) to check out a tree based on
property values? For example, could I say "check out all
files that have the property foo set to baz"? Or if not
check out, at least get a listing of such files which I
can then copy into a tag? (think: during the conversion
process I retain SCCS info in properties, and then want
to convert from a given SCCS SID to a propper svn tag).
4. Has anyone ever done scalability testing with svn? Is it
an appropriate tool for a 600 000+ source file repository?
Any / all advice greatly appreciated. Thank you for your time.
Kean
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FAQ: svn advocacy / advice
Posted by Daniel Rall <dl...@collab.net>.
On Mon, 07 Nov 2005, Garrett Rooney wrote:
> On 11/7/05, Kean Johnston <jk...@sco.com> wrote:
>
> > 1. Is fsfs the prefered solution? If so, since each revision
> > seems to be set of changes relative to the previous one,
> > considering the amount of history and size of tree and
> > number of revisions, does svn actually need to play back
> > from revision 1 or does it store some sort of "checkpoint"
> > along the way? If it does, is that automatic or manual?
> > Also, if it does, is that "checkpoint" then basically the
> > size of the repository at that point in time?
>
> fsfs is the default repository back end these days, so I suppose you
> could say it's "prefered". As for the deltas over large numbers of
> changes, fsfs uses a technique called "skip deltas", which ensures
> that you never have to apply more than log(n) deltas to get to any
> particular revision, see
> http://svn.collab.net/repos/svn/trunk/notes/skip-deltas for more
> details.
And for a comparison of SCCS's interleaved delta storage format and
Subversion's skip delta format (by the same author, Greg Hudson), see
<http://web.mit.edu/ghudson/thoughts/file-versioning>.
...
> > 3. Is it possible (even if slow) to check out a tree based on
> > property values? For example, could I say "check out all
> > files that have the property foo set to baz"? Or if not
> > check out, at least get a listing of such files which I
> > can then copy into a tag? (think: during the conversion
> > process I retain SCCS info in properties, and then want
> > to convert from a given SCCS SID to a propper svn tag).
>
> This is not currently possible.
You'd need to write some sort of tree crawler to do this.
Better yet, you'd avoid this problem by having the (theoretical)
converter perform this branch creation as part of the conversion
process (as cvs2svn does).
--
Daniel Rall
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FAQ: svn advocacy / advice
Posted by Kean Johnston <jk...@sco.com>.
> I hope that helps answer your questions,
It does indeed, thank you very much Garrett.
About the only change I know I will need to make to svn was the
thing I discussed a few weeks back about making the fsfs code store
its revision files in a hiearchy (similar to squid) rather than all
in a single directory as it currently does. I'll be submitting a
patch for aprroval soon :) I've been in X.org land the last few
weeks and havent had time to finish working on that.
Take care and thanks to you and all the other developers for a
truely amazing product.
Kean
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FAQ: svn advocacy / advice
Posted by Daniel Rall <dl...@collab.net>.
On Tue, 08 Nov 2005, Molle Bestefich wrote:
> Pedro Lamarão wrote:
> > http://gcc.gnu.org/svn/gcc/ ;-)
>
> 404 not found.
> Did you mean svn:// ?
Yeah, GCC's repository uses ra_svn. The repository info can be found
here:
http://gcc.gnu.org/svn.html
> (Odd repository. What's '/empty-dir' ?)
CVS uses an empty directory under CVSROOT (for diffs?), usually named
"Emptydir" -- could be an artifact from pre-conversion to Subversion.
(I didn't look to confirm whether this is in fact that case for GCC's
repo.)
--
Daniel Rall
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Switch to empty-dir [was: FAQ: svn advocacy / advice]
Posted by Julian Foad <ju...@btopenworld.com>.
Malcolm Rowe wrote:
> On Tue, Nov 08, 2005 at 04:06:46PM +0000, Molle Bestefich wrote:
>
>>(Odd repository. What's '/empty-dir' ?)
>
> A kludge to work around our broken handling of incomplete working copies
> (as seen, for example, in non-recursive checkouts).
>
> The user switches the directories in their wc that they're not interested
> in to the empty-dir directory. This effectively ignores them (and it's
> 'sticky'), but doesn't require the user to carry around a bunch of local
> 'svn:ignore' propsets that they're not planning to commit.
It may be a kludge, but it's one I hadn't thought of, and it sounds pretty cool
to me. We ought to have that as a FAQ answer. Any volunteers to write it?
- Julian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FAQ: svn advocacy / advice
Posted by Malcolm Rowe <ma...@farside.org.uk>.
On Tue, Nov 08, 2005 at 04:06:46PM +0000, Molle Bestefich wrote:
> Pedro Lamarão wrote:
> > http://gcc.gnu.org/svn/gcc/ ;-)
>
> 404 not found.
> Did you mean svn:// ?
>
Yes, he meant svn://.
> (Odd repository. What's '/empty-dir' ?)
>
A kludge to work around our broken handling of incomplete working copies
(as seen, for example, in non-recursive checkouts).
The user switches the directories in their wc that they're not interested
in to the empty-dir directory. This effectively ignores them (and it's
'sticky'), but doesn't require the user to carry around a bunch of local
'svn:ignore' propsets that they're not planning to commit.
Regards,
Malcolm
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FAQ: svn advocacy / advice
Posted by Daniel Berlin <db...@dberlin.org>.
On Tue, 2005-11-08 at 16:06 +0000, Molle Bestefich wrote:
> Pedro Lamarão wrote:
> > http://gcc.gnu.org/svn/gcc/ ;-)
>
> 404 not found.
> Did you mean svn:// ?
>
> (Odd repository. What's '/empty-dir' ?)
Not odd at all.
Some people want to do the equivalent of remove directories from their
checkout, but still have update work right.
The easiest way to do this is to switch the directories you don't want
to the empty dir :)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FAQ: svn advocacy / advice
Posted by Molle Bestefich <mo...@gmail.com>.
Pedro Lamarão wrote:
> http://gcc.gnu.org/svn/gcc/ ;-)
404 not found.
Did you mean svn:// ?
(Odd repository. What's '/empty-dir' ?)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FAQ: svn advocacy / advice
Posted by Pedro Lamarão <pe...@intersix.com.br>.
Garrett Rooney wrote:
>(...) and most recently GCC (don't know the
>URL, sorry).
>
>
http://gcc.gnu.org/svn/gcc/ ;-)
--
Pedro Lamarão
Desenvolvimento
Intersix Technologies S.A.
SP: (55 11 3803-9300)
RJ: (55 21 3852-3240)
www.intersix.com.br
Your Security is our Business
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FAQ: svn advocacy / advice
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On 11/7/05, Kean Johnston <jk...@sco.com> wrote:
> 1. Is fsfs the prefered solution? If so, since each revision
> seems to be set of changes relative to the previous one,
> considering the amount of history and size of tree and
> number of revisions, does svn actually need to play back
> from revision 1 or does it store some sort of "checkpoint"
> along the way? If it does, is that automatic or manual?
> Also, if it does, is that "checkpoint" then basically the
> size of the repository at that point in time?
fsfs is the default repository back end these days, so I suppose you
could say it's "prefered". As for the deltas over large numbers of
changes, fsfs uses a technique called "skip deltas", which ensures
that you never have to apply more than log(n) deltas to get to any
particular revision, see
http://svn.collab.net/repos/svn/trunk/notes/skip-deltas for more
details.
> 2. I know there is a cvs to svn converter, and I've seen some
> threads about an SCCS to svn converter using cvs in between,
> but does anyone know of a direct SCCS to svn converter? If
> not, does svn's API lend itself to me writing such a tool?
> Anyone have any WAG what the scope of such a project would
> be?
I don't know of such a tool, but I imagine writing one would be
possible, although depending on how much of an impedence missmatch
there is between sccs and svn it could be difficult. CVS, for
example, took a considerable amount of effort.
> 3. Is it possible (even if slow) to check out a tree based on
> property values? For example, could I say "check out all
> files that have the property foo set to baz"? Or if not
> check out, at least get a listing of such files which I
> can then copy into a tag? (think: during the conversion
> process I retain SCCS info in properties, and then want
> to convert from a given SCCS SID to a propper svn tag).
This is not currently possible.
> 4. Has anyone ever done scalability testing with svn? Is it
> an appropriate tool for a 600 000+ source file repository?
There are several large scale svn repositories in the open source
world, for example the ASF repository
(http://svn.apache.org/repos/asf), the KDE repository (don't know the
URL off the top of my head), and most recently GCC (don't know the
URL, sorry). I don't know if any of those scale up to 600K source
files, but they're pretty big. In my experience, very large working
copies aren't exactly speedy, but if you limit commands to the section
of the tree you care about as much as possible svn will scale up
reasonably well. On the repository side there's certainly nothing
preventing that sort of scalability, assuming you aren't doing
something silly like putting all of those files in a single directory
;-)
I hope that helps answer your questions,
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FAQ: svn advocacy / advice
Posted by Ivan Zhakov <ch...@gmail.com>.
On 11/10/05, Kean Johnston <jk...@sco.com> wrote:
> > I have recently moved my company to Subversion from SourceSafe.
> > 200-300 full time developers, for the most part on Windows. We have
> > big products with long history. And Subversion deals very nice. There
> Thanks thats a useful data point.
>
> > PS: If you interested we decide to use BDB as backend.
> Can I ask why? Seems like everyone is favouring fsfs these
> days.
Our arguments was:
1) BDB is faster in our case. We have 2-4 gb repositories, 20-50k
revisions, much binary data, big commits and heavy load.
2) BDB is transactional database, so any fail should be fixed by
transaction rollback.
3) BDB looks more tested.
4) FSFS have long post-commit phase -- this could produce timeout
problems on big commits
5) FSFS was much slower over HTTP/DAV
6) Initialy we planned to use Windows server so storing 50k files in
one directory would cause performance degradation. Anyway after
testing we switched to Linux server.
--
Ivan Zhakov
Re: FAQ: svn advocacy / advice
Posted by Kean Johnston <jk...@sco.com>.
> I have recently moved my company to Subversion from SourceSafe.
> 200-300 full time developers, for the most part on Windows. We have
> big products with long history. And Subversion deals very nice. There
Thanks thats a useful data point.
> PS: If you interested we decide to use BDB as backend.
Can I ask why? Seems like everyone is favouring fsfs these
days.
Kean
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FAQ: svn advocacy / advice
Posted by Ivan Zhakov <ch...@gmail.com>.
On 11/8/05, Kean Johnston <jk...@sco.com> wrote:
> Good evening subverters,
>
> We (SCO) are looking for a new revision control tool to
> replace a historic SCCS tree, and hopefully, a ClearCase
> tree as well. The source base is pretty large, and dates
> back to the mid-70's. The current contender is cvs, becuase
> it is such a well-known quantity, but I would much prefer
> to use svn. However, both myself and others on the project
> have a few concerns and questions, and I would appreciate
> some advice.
>
> I am sure much of this is covered in the doc, but I need
> a fairly quick answer to justify spending the time doing
> research. If these are all FAQ, I appologise but hope you
> will indulge me. Yes/no answers are acceptable :)
>
> 1. Is fsfs the prefered solution? If so, since each revision
> seems to be set of changes relative to the previous one,
> considering the amount of history and size of tree and
> number of revisions, does svn actually need to play back
> from revision 1 or does it store some sort of "checkpoint"
> along the way? If it does, is that automatic or manual?
> Also, if it does, is that "checkpoint" then basically the
> size of the repository at that point in time?
>
> 2. I know there is a cvs to svn converter, and I've seen some
> threads about an SCCS to svn converter using cvs in between,
> but does anyone know of a direct SCCS to svn converter? If
> not, does svn's API lend itself to me writing such a tool?
> Anyone have any WAG what the scope of such a project would
> be?
>
> 3. Is it possible (even if slow) to check out a tree based on
> property values? For example, could I say "check out all
> files that have the property foo set to baz"? Or if not
> check out, at least get a listing of such files which I
> can then copy into a tag? (think: during the conversion
> process I retain SCCS info in properties, and then want
> to convert from a given SCCS SID to a propper svn tag).
>
> 4. Has anyone ever done scalability testing with svn? Is it
> an appropriate tool for a 600 000+ source file repository?
>
> Any / all advice greatly appreciated. Thank you for your time.
I have recently moved my company to Subversion from SourceSafe.
200-300 full time developers, for the most part on Windows. We have
big products with long history. And Subversion deals very nice. There
is only one problem -- working copy littile bit slow on large working
copy, but I hope it will imrpove in 1.3.
PS: If you interested we decide to use BDB as backend.
--
Ivan Zhakov