You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by NN Ott <no...@gmail.com> on 2011/02/14 22:43:58 UTC
svn 1.7 improvements: Large repo? (w/ 40,000 files)
Hello,
I notice that svn v1.7 is slated to have a new transfer protocol as well as
new WC module.
( http://subversion.apache.org/roadmap.html )
Right now, I have a repo with ~40,000 source and xml files.
Full checkouts take forever.
My machine is pretty beefy, so that's not the bottle neck. Network speed
to the remote svn server is OK, but not terrible, and certainly
not the bottle neck.
Assuming I upgrade my client to 1.7 when it comes out, my questions are:
- Will either of the above improvements help me if the server _does_ get
upgraded to v1.7?
- Will either of the above improvements help me if the server _does not_ get
upgraded to v1.7?
Thanks
Re: svn 1.7 improvements: Large repo? (w/ 40,000 files)
Posted by "Hyrum K. Wright" <hy...@mail.utexas.edu>.
On Mon, Feb 14, 2011 at 11:05 PM, NN Ott <no...@gmail.com> wrote:
...
>> All that said, I never heard that there will be a new protocol.
>
> It's on: http://subversion.apache.org/roadmap.html
> Details sound pretty cool. Particularly the pipelining and parallel request
> items.
FYI, http://subversion.apache.org/docs/release-notes/1.7.html is a bit
more specific and comprehensive about the stuff coming in 1.7. (It,
just like the roadmap, shouldn't be considered canonical until 1.7.0
is actually released.)
-Hyrum
RE: svn 1.7 improvements: Large repo? (w/ 40,000 files)
Posted by Bob Archer <Bo...@amsi.com>.
> > My machine is pretty beefy, so that's not the bottle neck.
> Network
> > speed to the remote svn server is OK, but not terrible, and
> > certainly not the bottle neck.
> It could be if it is a windows machine. Trying checking out to a
> RAM disk. I have found the performance is much much faster.
>
>
> Any recommended ram disk software?
I've used this one:
http://memory.dataram.com/products-and-services/software/ramdisk
It is free for up to 4GB disks.
There is also a topic on this over at superuser.com:
http://superuser.com/questions/34388/whats-the-best-ramdisk-for-windows
BOb
>
>
> All that said, I never heard that there will be a new protocol.
>
>
> It's on: http://subversion.apache.org/roadmap.html
>
> Details sound pretty cool.
> Particularly the pipelining and parallel request items.
>
>
>
Re: svn 1.7 improvements: Large repo? (w/ 40,000 files)
Posted by NN Ott <no...@gmail.com>.
>
>
> >
> > My machine is pretty beefy, so that's not the bottle neck. Network
> > speed to the remote svn server is OK, but not terrible, and
> > certainly not the bottle neck.
>
> It could be if it is a windows machine. Trying checking out to a RAM disk.
> I have found the performance is much much faster.
>
>
>
Any recommended ram disk software?
> All that said, I never heard that there will be a new protocol.
>
>
>
It's on: http://subversion.apache.org/roadmap.html
Details sound pretty cool. Particularly the pipelining and parallel request
items.
Re: What Subversion components am I using?
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Yes, the shared libs that svnserve uses. (see ldd(1))
See also:
http://subversion.apache.org/docs/community-guide/mailing-lists#fresh-post
Eric wrote on Mon, Feb 14, 2011 at 21:58:28 -0500:
>
> Subversion installation is version 1.4.2 on CentOS 5.2.
>
> Client is TortoiseSVN on Windows XP.
>
> I am using ONLY the svn+ssh protocol. I am not using the web
> interface, nor am I accessing the repository locally on the Linux
> box.
>
> Am I using any component of the Subversion installation on the
> CentOS 5.2 machine other than svnserve and the repository?
>
> Thanks...
>
What Subversion components am I using?
Posted by Eric <sp...@internetsmallfry.com>.
Subversion installation is version 1.4.2 on CentOS 5.2.
Client is TortoiseSVN on Windows XP.
I am using ONLY the svn+ssh protocol. I am not using the web
interface, nor am I accessing the repository locally on the Linux box.
Am I using any component of the Subversion installation on the CentOS
5.2 machine other than svnserve and the repository?
Thanks...
Re: svn 1.7 improvements: Large repo? (w/ 40,000 files)
Posted by NN Ott <no...@gmail.com>.
On Tue, Feb 15, 2011 at 10:44 AM, Stefan Sperling <st...@elego.de> wrote:
> On Tue, Feb 15, 2011 at 10:41:13AM -0500, NN Ott wrote:
> > Are things still on track for a q1 release?
>
> We're trying.
>
OK, here's hoping for smooth sailing. :-)
Re: svn 1.7 improvements: Large repo? (w/ 40,000 files)
Posted by Stefan Sperling <st...@elego.de>.
On Tue, Feb 15, 2011 at 10:41:13AM -0500, NN Ott wrote:
> Are things still on track for a q1 release?
We're trying.
Re: svn 1.7 improvements: Large repo? (w/ 40,000 files)
Posted by NN Ott <no...@gmail.com>.
On Tue, Feb 15, 2011 at 12:31 AM, Daniel Shahaf <d....@daniel.shahaf.name>wrote:
> > > - Will either of the above improvements help me if the server
> > > _does_ get upgraded to v1.7?
> > >
> > > - Will either of the above improvements help me if the server _does
> > > not_ get upgraded to v1.7?
> >
> > All that said, I never heard that there will be a new protocol.
> >
>
> http://subversion.apache.org/docs/release-notes/1.7#httpv2
>
Are things still on track for a q1 release?
Re: svn 1.7 improvements: Large repo? (w/ 40,000 files)
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
> > - Will either of the above improvements help me if the server
> > _does_ get upgraded to v1.7?
> >
> > - Will either of the above improvements help me if the server _does
> > not_ get upgraded to v1.7?
>
> All that said, I never heard that there will be a new protocol.
>
http://subversion.apache.org/docs/release-notes/1.7#httpv2
RE: svn 1.7 improvements: Large repo? (w/ 40,000 files)
Posted by Bob Archer <Bo...@amsi.com>.
> I notice that svn v1.7 is slated to have a new transfer protocol as
> well as new WC module.
>
> ( http://subversion.apache.org/roadmap.html )
>
> Right now, I have a repo with ~40,000 source and xml files.
>
> Full checkouts take forever.
That's why you shouldn't be doing "full" checkouts very often. You should check out the portion of your repository that you need to work on. Also, you also shouldn't need to checkout very much. Once you have a working copy you are mostly doing "update" which just brings down the modified files since your last checkout/update.
>
> My machine is pretty beefy, so that's not the bottle neck. Network
> speed to the remote svn server is OK, but not terrible, and
> certainly not the bottle neck.
It could be if it is a windows machine. Trying checking out to a RAM disk. I have found the performance is much much faster.
> Assuming I upgrade my client to 1.7 when it comes out, my questions
> are:
>
> - Will either of the above improvements help me if the server
> _does_ get upgraded to v1.7?
>
> - Will either of the above improvements help me if the server _does
> not_ get upgraded to v1.7?
Most of the wc perf improvements will probably work with a 1.6 server. But, I expect there are some that won't.
All that said, I never heard that there will be a new protocol.
BOb
Re: svn 1.7 improvements: Large repo? (w/ 40,000 files)
Posted by NN Ott <no...@gmail.com>.
On Mon, Feb 14, 2011 at 4:52 PM, ankush chadha
<an...@yahoo.com>wrote:
> Is there a reason to checkout 40k files again and again? Can it be
> substituted by an svn update (after first checkout)?
>
>
Well, I obviously try use update/switch to avoid a fresh checkout. :)
Using cmd-line & tortisesvn on windows.
I need to do a full checkout periodically, and it's a real headache. Not
a show-stopper, just a headache.
Thus, I am wondering if relief will come with v1.7.
(Also, are they still on track for a Q1/2011 release?)
Thanks
Re: svn 1.7 improvements: Large repo? (w/ 40,000 files)
Posted by ankush chadha <an...@yahoo.com>.
Is there a reason to checkout 40k files again and again? Can it be substituted
by an svn update (after first checkout)?
If you are using svnkit library to do svn checkouts, then you may want to try
there new version (with jna). There were performance issues in the old version
of svnkit.
I remember setting svnkit.symlinks to false (I use windows and no funky stuff)
and it reduced the svn checkout time by a crazy number.
Ankush
________________________________
From: NN Ott <no...@gmail.com>
To: users@subversion.apache.org
Sent: Mon, February 14, 2011 4:43:58 PM
Subject: svn 1.7 improvements: Large repo? (w/ 40,000 files)
Hello,
I notice that svn v1.7 is slated to have a new transfer protocol as well as new
WC module.
( http://subversion.apache.org/roadmap.html )
Right now, I have a repo with ~40,000 source and xml files.
Full checkouts take forever.
My machine is pretty beefy, so that's not the bottle neck. Network speed
to the remote svn server is OK, but not terrible, and certainly not the bottle
neck.
Assuming I upgrade my client to 1.7 when it comes out, my questions are:
- Will either of the above improvements help me if the server _does_ get
upgraded to v1.7?
- Will either of the above improvements help me if the server _does not_ get
upgraded to v1.7?
Thanks