You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Hari Kodungallur <hk...@gmail.com> on 2005/01/10 07:11:48 UTC
Performance Question
Hi All,
Is there any data available on the performance of Subversion in the
following scenarios?
* CVS vs SVN running on Apache
* CVS vs SVN running svnserve
* SVN (apache) vs SVN (svnserve)
I am interested basically in total time taken to do a fresh checkout
and updates.
RE: Performance Question
Posted by Arild Fines <ar...@broadpark.no>.
Ben Collins-Sussman wrote:
> Most nightly build systems using subversion just 'make clean; svn up;
> make'.
In Draco.NET[1], that's not even an option. A build is always done from a
fresh checkout to a temporary folder, which is deleted afterwards.
[1] http://draconet.sf.net
--
Arild
AnkhSVN, Subverting Visual Studio .NET: http://ankhsvn.tigris.org
Blog: http://ankhsvn.com/blog
IRC: irc://irc.freenode.net/ankhsvn
"Gentlemen, you can't fight in here! This is the war room!"
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Performance Question
Posted by Dale Worley <dw...@pingtel.com>.
-----Original Message-----
From: Ben Collins-Sussman [mailto:sussman@collab.net]
If working copies were cheap, nobody would ever revert changes or
switch to a branch. They'd just make a new workspace appear.
---------------------------------------------------------------------
Not so -- "svn revert" means "The change I made to this file is Not Good and
should be removed from the fix I am constructing", and "svn switch" means
"Transport these changes to be based on a different branch".
-----Original Message-----
From: Brass Tilde [mailto:brasstilde@insightbb.com]
That sounds an awful lot like you think svn should account for possible
deficiencies in the build system. Is that what you intended?
---------------------------------------------------------------------
I do adjust how I *use* Subversion to account for possible (known)
deficiencies in the build system.
Dale
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Performance Question
Posted by Ben Collins-Sussman <su...@collab.net>.
On Jan 11, 2005, at 7:27 AM, Tor Ringstad wrote:
> [Ben Collins-Sussman]
>> Why, because people are running 'svn checkout' every day? :-)
>
> Yes, that's exactly what I would do if wc's had been cheap. Several
> times a day. One clean wc for every single "activity".
>
> I do suspect, though, that this might be a way of thinking that is
> particular to people with ClearCase background. In ClearCase, wc's
> (aka "views") are essentially free, and using separate wc's per
> acitivity is encouraged.
>
>
You're absolutely right about Clearcase. Clearcase can instantly
create a workspace for you, no waiting. But systems like CVS and SVN
operate like Clearcase's "snapshot views": they're quite expensive to
create, so it's assumed they're long-lived things designed to live
(often) disconnected from the server. That's why so many svn commands
exist for re-using working copies, e.g. 'svn revert', and 'svn switch'.
If working copies were cheap, nobody would ever revert changes or
switch to a branch. They'd just make a new workspace appear.
So yah, what I meant to say is "reusing a working copy is the norm for
subversion."
Most nightly build systems using subversion just 'make clean; svn up;
make'.
:-)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Website release
Posted by John Browne <jk...@gmail.com>.
We use subversion to manage our sites, development and production. On
the production webserver we have a working copy of the production
sites checked out into their respective htdocs dirs (.svn folders
denied in httpd.conf). We have a cron job set up on the server that
runs an "svn update" every 15 minutes. On the development server, we
do the same thing, but for the development branch of each site. The
programmers work off their own local working copies, then commit their
changes when they are ready. They make it to the respective server at
the next 15-minute interval.
We have been using this system for a-while now, and it seems to work
well for us. Your mileage may vary.
On Fri, 14 Jan 2005 11:02:05 -0500 (EST), David Kramer
<da...@thekramers.net> wrote:
> On Tue, 11 Jan 2005, Kristian Hebert wrote:
>
> > Hello everyone,
> >
> > I hope this is the right mailinglist for my question.
> >
> > I was just wondering if there is any way to "release" a version of my
> > website to the server? I have all the files in svn, and I'm running the
> > webserver on the same server.
> > I guess I could log into the server and checkout all the files whenever I
> > want to release, but that seems very cumbersome, espesially if I want to
> > release to a testsite first. I haven't found anything in the manual or the
> > FAQ.
> > Am I blind? Or am I missing the obvious?
>
> Our web server points to a Subversion working copy of our website. I just
> implemented a php page this week to do an svn update. I just go to that
> page, it runs the svn update in $DOCUMENT_ROOT/.. , and prints the output
> and return value. No need to ssh in. Just remember to tell your
> webserver to hide .svn directories.
>
> If you don't want to do this, you can use the same basic technique to do
> an svn export.
> -------------------------------------------------------------------
> DDDD David Kramer http://thekramers.net
> DK KD
> DKK D
> DK KD Pretense and adversity are inversely proportional;
> DDDD Adversity reveals the true nature of all things.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Website release
Posted by David Kramer <da...@thekramers.net>.
On Tue, 11 Jan 2005, Kristian Hebert wrote:
> Hello everyone,
>
> I hope this is the right mailinglist for my question.
>
> I was just wondering if there is any way to "release" a version of my
> website to the server? I have all the files in svn, and I'm running the
> webserver on the same server.
> I guess I could log into the server and checkout all the files whenever I
> want to release, but that seems very cumbersome, espesially if I want to
> release to a testsite first. I haven't found anything in the manual or the
> FAQ.
> Am I blind? Or am I missing the obvious?
Our web server points to a Subversion working copy of our website. I just
implemented a php page this week to do an svn update. I just go to that
page, it runs the svn update in $DOCUMENT_ROOT/.. , and prints the output
and return value. No need to ssh in. Just remember to tell your
webserver to hide .svn directories.
If you don't want to do this, you can use the same basic technique to do
an svn export.
-------------------------------------------------------------------
DDDD David Kramer http://thekramers.net
DK KD
DKK D
DK KD Pretense and adversity are inversely proportional;
DDDD Adversity reveals the true nature of all things.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Website release
Posted by "Martin A. Brooks" <ma...@hinterlands.org>.
Kristian Hebert wrote:
>Am I blind? Or am I missing the obvious?
>
>
One easy way would be to have a post commit hook that exports the new
version of the website into your docroot.
Regards
Mart.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Website release
Posted by "Peter N. Lundblad" <pe...@famlundblad.se>.
On Tue, 11 Jan 2005, Peter N. Lundblad wrote:
> On Tue, 11 Jan 2005, Kristian Hebert wrote:
>
> > I hope this is the right mailinglist for my question.
> >
> Seems right.
>
> > I was just wondering if there is any way to "release" a version of my
> > website to the server? I have all the files in svn, and I'm running the
> > webserver on the same server.
>
> What about running svn export or svn update to the directory of the web
> server?
>
...from a post-commit hook I meant to say. Sorry.
I've seen people use branches in a creative way to support a test site and
a production site. I don't know how well that works, though. Maybe some
wrapper script that does the merge to the production site automagically
for you.
Regards,
//Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Website release
Posted by "Peter N. Lundblad" <pe...@famlundblad.se>.
On Tue, 11 Jan 2005, Kristian Hebert wrote:
> I hope this is the right mailinglist for my question.
>
Seems right.
> I was just wondering if there is any way to "release" a version of my
> website to the server? I have all the files in svn, and I'm running the
> webserver on the same server.
What about running svn export or svn update to the directory of the web
server?
Regards,
//Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Website release
Posted by Andy Peters <de...@latke.net>.
Kristian Hebert wrote:
> Hello everyone,
>
> I hope this is the right mailinglist for my question.
>
> I was just wondering if there is any way to "release" a version of my
> website to the server? I have all the files in svn, and I'm running the
> webserver on the same server.
> I guess I could log into the server and checkout all the files whenever I
> want to release, but that seems very cumbersome, espesially if I want to
> release to a testsite first. I haven't found anything in the manual or the
> FAQ.
> Am I blind? Or am I missing the obvious?
Seems to me that the easiest way to do this would be to, yes, log onto
the web server and do an svn checkout or svn update. You'd do the same
thing if you wanted to test the site on a development server.
-a
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Website release
Posted by Kristian Hebert <kr...@hebert.dk>.
Hello everyone,
I hope this is the right mailinglist for my question.
I was just wondering if there is any way to "release" a version of my
website to the server? I have all the files in svn, and I'm running the
webserver on the same server.
I guess I could log into the server and checkout all the files whenever I
want to release, but that seems very cumbersome, espesially if I want to
release to a testsite first. I haven't found anything in the manual or the
FAQ.
Am I blind? Or am I missing the obvious?
Please help
Kristian Hebert
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Performance Question
Posted by Dale Worley <dw...@pingtel.com>.
-----Original Message-----
From: news [mailto:news@sea.gmane.org]On Behalf Of Tor Ringstad
> If so, you're doing something very odd. The normal usage is that you
> check out a working copy once, then use it for months and months.
I would argue that this is "normal" usage not because it is the best
way, but because it is the only way most SCM's support well.
---------------------------------------------------------------------
I agree with Tor here -- If I had my way, I'd use a separate WC for every
activity. WCs tend to collect cruft, especially if the "make" structure is
not well-debugged, and one is often in the middle of several different
changes, sometimes on several branches of the product. At the moment, I've
got 7 WCs for our product, and 21 WCs sitting in the trash bin.
Dale
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Performance Question
Posted by Tor Ringstad <to...@tandberg.net>.
[Ben Collins-Sussman]
> Why, because people are running 'svn checkout' every day? :-)
Yes, that's exactly what I would do if wc's had been cheap. Several
times a day. One clean wc for every single "activity".
I do suspect, though, that this might be a way of thinking that is
particular to people with ClearCase background. In ClearCase, wc's
(aka "views") are essentially free, and using separate wc's per
acitivity is encouraged.
> If so, you're doing something very odd. The normal usage is that you
> check out a working copy once, then use it for months and months.
I would argue that this is "normal" usage not because it is the best
way, but because it is the only way most SCM's support well.
- Tor -
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Performance Question
Posted by Sean Laurent <se...@neuronfarm.com>.
On Tuesday 11 January 2005 11:19 am, Brass Tilde wrote:
> > > If you're talking about files that are the result of the build, then
> > > the build process should take care of cleaning those up.
> >
> > "Should" - yes. Purely hypothetically, there could be bugs in code. Also
> > in Makefiles... ;-D
>
> That sounds an awful lot like you think svn should account for possible
> deficiencies in the build system. Is that what you intended?
This is definitely something I encountered when working for a larger company
developing retail, shrink-wrap software. We used Perforce and developers
generally checked out working copies and used them for long periods of time.
However, we setup our automated build system, which ran nightly, to always
use a brand spanking new checkout every time.
Why?
For precisely the reason mentioned above. Although all of the members of our
development team were topnotch engineers, sometimes we made mistakes.
Sometimes the build process did _not_ cleanup properly. To improve quality
assurance and reduce the possibility of stray bugs getting introduced, we
wanted daily builds to reflect the codebase exactly.
While it took 30 minutes to pull down the codebase over the LAN, we observed
that doing a full checkout for every 'official' build of the software
definitely decreased inconsistencies between builds. Over the long term,
this process also reduced the occurence of the dreaded "unable to repro"
problem.
Were we using the SCM software to "account for possible deficiencies in the
build system"? Yup. Was this a "bad thing"? Nope.
-Sean
------------------------------
Whenever I feel blue, I start breathing again.
------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Performance Question
Posted by Brass Tilde <br...@insightbb.com>.
> > If you're talking about files that are the result of the build, then the
> > build process should take care of cleaning those up.
>
> "Should" - yes. Purely hypothetically, there could be bugs in code. Also
> in Makefiles... ;-D
That sounds an awful lot like you think svn should account for possible
deficiencies in the build system. Is that what you intended?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Performance Question
Posted by Peter Valdemar Mørch <no...@morch.com>.
Brass Tilde wrote:
> If you're talking about files that are the result of the build, then the
> build process should take care of cleaning those up.
"Should" - yes. Purely hypothetically, there could be bugs in code. Also
in Makefiles... ;-D
"Trust is good, control is better" - Lenin
Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Performance Question
Posted by Brass Tilde <br...@insightbb.com>.
> > That's still inefficient. The build system should run 'svn update' on
> > the same working copy every night. That's what most of the rest of us
do.
>
> I'm with Hari on this one. Daily / official builds should start with a
> pristine copy of the module that is about to be compiled, ensuring that
> there are no local files and/or modifications.
If the only thing ever done to the build working copy is an initial check
out and then subsequent updates, how would any local files or modifications
get there? Does svn update not delete files from a working copy when those
files are deleting from the revision from which the wc is being updated?
If you're talking about files that are the result of the build, then the
build process should take care of cleaning those up.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Performance Question
Posted by Tom Mornini <tm...@infomania.com>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Jan 10, 2005, at 8:26 AM, Peter Valdemar Mørch wrote:
> Ben Collins-Sussman wrote:
>> On Jan 10, 2005, at 8:58 AM, Hari Kodungallur wrote:
>>> Currently, our daily/nightly build (not individual developers) does
>>> fresh checkout everyday. The repository is rather large and hence the
>>> concern.
>> That's still inefficient. The build system should run 'svn update'
>> on the same working copy every night. That's what most of the rest
>> of us do.
snip...
> It does bring up an interesting question: If the output of
> `svn st --no-ignore`
> after an
> `svn update`
> is completely empty, is it then guaranteed in all cases that "." will
> equal what one gets with a fresh checkout, so that
>
> `diff -r . ../newcheckout`
>
> is empty?
>
> Are there other things that could be overlooked? E.g. keyword
> expansion differences?
>
> What is *the method* of getting a pristine checkout from a possibly
> dirty working copy?
Just a stab at this...
svn revert -R .
svn update
- --
- -- Tom Mornini
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (Darwin)
iD8DBQFB4rDVzaKkuvKwKvcRAgyHAKCGahGp+I/wM4Q8qA2Xq2n7s2LnHwCeJYz9
ku44t9yO3Gr/o3VxI+TkuQk=
=Tgp7
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Performance Question
Posted by Martin Tomes <li...@tomes.org>.
Peter Valdemar Mørch wrote:
> Ben Collins-Sussman wrote:
>> On Jan 10, 2005, at 8:58 AM, Hari Kodungallur wrote:
>>> Currently, our daily/nightly build (not individual developers) does
>>> fresh checkout everyday. The repository is rather large and hence the
>>> concern.
>>
>> That's still inefficient. The build system should run 'svn update' on
>> the same working copy every night. That's what most of the rest of us
>> do.
>
> What is *the method* of getting a pristine checkout from a possibly
> dirty working copy?
My guess is that you would have to do an svn revert --recursive to undo
any changes to files under revision control and a script which deleted
any files which aren't under revision control using svn status
--no-ignore (as you suggested) to list those files.
--
Martin Tomes
echo 'martin at tomes x org x uk'\
| sed -e 's/ x /\./g' -e 's/ at /@/'
The Subversion Wiki is at http://www.subversionary.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Performance Question
Posted by Peter Valdemar Mørch <no...@morch.com>.
Ben Collins-Sussman wrote:
> On Jan 10, 2005, at 8:58 AM, Hari Kodungallur wrote:
>> Currently, our daily/nightly build (not individual developers) does
>> fresh checkout everyday. The repository is rather large and hence the
>> concern.
>
> That's still inefficient. The build system should run 'svn update' on
> the same working copy every night. That's what most of the rest of us do.
I'm with Hari on this one. Daily / official builds should start with a
pristine copy of the module that is about to be compiled, ensuring that
there are no local files and/or modifications.
Yes one could start with a `svn status` to see if there are any local
files or modifications, but then there is svn:ignore.
One could then use `svn st --no-ignore`. What other things might bite?
It does bring up an interesting question: If the output of
`svn st --no-ignore`
after an
`svn update`
is completely empty, is it then guaranteed in all cases that "." will
equal what one gets with a fresh checkout, so that
`diff -r . ../newcheckout`
is empty?
Are there other things that could be overlooked? E.g. keyword expansion
differences?
What is *the method* of getting a pristine checkout from a possibly
dirty working copy?
Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Performance Question
Posted by Ben Collins-Sussman <su...@collab.net>.
On Jan 10, 2005, at 8:58 AM, Hari Kodungallur wrote:
>>
>> Why, because people are running 'svn checkout' every day? :-)
>>
>> If so, you're doing something very odd. The normal usage is that you
>> check out a working copy once, then use it for months and months.
>>
>> (And that's when you start seeing SVN speed advantages over CVS:
>> 'diff' and 'status' require no network access at all. It takes only 3
>> seconds to create a branch or tag, regardless of the size of your
>> tree.
>> When updating and committing, diffs are sent in both directions.)
>>
>
> Currently, our daily/nightly build (not individual developers) does
> fresh checkout everyday. The repository is rather large and hence the
> concern.
>
>
That's still inefficient. The build system should run 'svn update' on
the same working copy every night. That's what most of the rest of us
do.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Performance Question
Posted by Ben Collins-Sussman <su...@collab.net>.
On Jan 10, 2005, at 1:11 AM, Hari Kodungallur wrote:
>
> --> whether that is consistent with what is expected of SVN and/or if
> I there is something that I can do to tune the performance.
Yes, this is well-known. There's not much you can do.
Our collective experience is that 'svn co' is about 1.5x slower than
cvs on a linux client, and 3x slower on a win32 client.
However, svn does many other things faster than cvs.
> --> if running the svnserve protocol has any effect.
Yes, svnserve is much faster than http. It's a stateful protocol,
rather than a stateless one.
> --> what the performance numbers are when using FSFS vs Berkeley DB.
>
Experience shows that FSFS is often a bit faster than BDB.
> Basically, we would like to move a large repository from CVS to SVN,
> but we are concerned about the fresh checkout time.
>
Why, because people are running 'svn checkout' every day? :-)
If so, you're doing something very odd. The normal usage is that you
check out a working copy once, then use it for months and months.
(And that's when you start seeing SVN speed advantages over CVS:
'diff' and 'status' require no network access at all. It takes only 3
seconds to create a branch or tag, regardless of the size of your tree.
When updating and committing, diffs are sent in both directions.)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org