You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by solo turn <so...@yahoo.com> on 2002/08/11 12:53:50 UTC
performance after 1.0: less metadata files on client?
yesterday we tried to check out our repository via http to a working
copy on a windows network drive. the result was not to good: more
than 6 hours, compared to 15-20 minutes with source safe.
the problem seems to be related to the structure of the metadata:
- while visual source safe uses one file per directory
- svn uses 5 files and 11 directories per directory
- svn uses 3 files per file
we store java code which tends to have many directories. we expect
soon to reach 1000, this then would mean 16000 additional files&dirs
through svn (without using rights&props). additionally we have 4000
source files, which means another 12000 files.
could this be a bottleneck on the long run? or do you somehow plan to
switch later to reiser4 where you attach all this files as properties
of files in reiserfs (http://www.reiserfs.org/v4/v4.html)?
ford.
__________________________________________________
Do You Yahoo!?
HotJobs - Search Thousands of New Jobs
http://www.hotjobs.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: performance after 1.0: less metadata files on client?
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On Sun, Aug 11, 2002 at 01:31:04PM -0400, Mix wrote:
> Garrett Rooney wrote:
> >this problem will likely be aleviated somewhat when we start caching
> >the contents of .svn/entries. at the moment that file is being parsed
> >many many more times than it should, which would be very bad if it is
> >located on a network filesystem. work is being done on caching this
> >information, so it would be interesting to see how much that helps the
> >performance problems you're seeing.
>
> The .svn/entries file is parsed multiple times on 'checkout'???
any time svn_wc_entry() is called that file is being parsed. checkout
uses the update editor, and that is calling svn_wc_entry() for each
file added (from my five second glance at the code) so yes, the
.svn/entries file is being parsed multiple times on checkout.
-garrett
--
garrett rooney Remember, any design flaw you're
rooneg@electricjellyfish.net sufficiently snide about becomes
http://electricjellyfish.net/ a feature. -- Dan Sugalski
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: Re: performance after 1.0: less metadata files on client?
Posted by Bill Tutt <ra...@lyra.org>.
> From: Mix [mailto:mixtim@acm.org]
>
> Garrett Rooney wrote:
> > this problem will likely be aleviated somewhat when we start caching
> > the contents of .svn/entries. at the moment that file is being
parsed
> > many many more times than it should, which would be very bad if it
is
> > located on a network filesystem. work is being done on caching this
> > information, so it would be interesting to see how much that helps
the
> > performance problems you're seeing.
>
> The .svn/entries file is parsed multiple times on 'checkout'???
>
Yep. That's why we want to cache it. :) Yes this is annoying at present,
we've known about it for awhile, but we've been busy fixing other areas
of the system.
Bill
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: performance after 1.0: less metadata files on client?
Posted by Mix <mi...@acm.org>.
Garrett Rooney wrote:
> this problem will likely be aleviated somewhat when we start caching
> the contents of .svn/entries. at the moment that file is being parsed
> many many more times than it should, which would be very bad if it is
> located on a network filesystem. work is being done on caching this
> information, so it would be interesting to see how much that helps the
> performance problems you're seeing.
The .svn/entries file is parsed multiple times on 'checkout'???
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: performance after 1.0: less metadata files on client?
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
On Sun, Aug 11, 2002 at 01:05:17PM -0400, Mix wrote:
> Bryan O'Sullivan wrote:
> >>yesterday we tried to check out our repository via http to a working
> >>copy on a windows network drive. the result was not to good: more
> >>than 6 hours, compared to 15-20 minutes with source safe.
> >
> >That's pretty horrifying, but I'd be leery of drawing many conclusions
> >without also testing on a local filesystem on your Windows box. That
> >way, you can discount any possible overhead from SMB. It would be
> >interesting to see if there's a performance difference between the two.
>
> Well, in many environments the only writable area you have is a network
> drive. Checking out from a remote cvs repository and having your working
> copy be on an NFS mount isn't that bad. Its horrible with svn because of
> those darn .svn directories that hold so much stuff. In this instance
> svn is noticably slower than cvs, sometimes painfully slow. So its not
> just a windows issue, its a network filesystem issue in general since
> svn takes the view that we can just write out whatever we want in a
> working copy since "disk space is cheap". Unfortunately network
> bandwidth often isn't.
this problem will likely be aleviated somewhat when we start caching
the contents of .svn/entries. at the moment that file is being parsed
many many more times than it should, which would be very bad if it is
located on a network filesystem. work is being done on caching this
information, so it would be interesting to see how much that helps the
performance problems you're seeing.
-garrett
--
garrett rooney Remember, any design flaw you're
rooneg@electricjellyfish.net sufficiently snide about becomes
http://electricjellyfish.net/ a feature. -- Dan Sugalski
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: performance after 1.0: less metadata files on client?
Posted by Mix <mi...@acm.org>.
Bryan O'Sullivan wrote:
>>yesterday we tried to check out our repository via http to a working
>>copy on a windows network drive. the result was not to good: more
>>than 6 hours, compared to 15-20 minutes with source safe.
>
> That's pretty horrifying, but I'd be leery of drawing many conclusions
> without also testing on a local filesystem on your Windows box. That
> way, you can discount any possible overhead from SMB. It would be
> interesting to see if there's a performance difference between the two.
Well, in many environments the only writable area you have is a network
drive. Checking out from a remote cvs repository and having your working
copy be on an NFS mount isn't that bad. Its horrible with svn because of
those darn .svn directories that hold so much stuff. In this instance
svn is noticably slower than cvs, sometimes painfully slow. So its not
just a windows issue, its a network filesystem issue in general since
svn takes the view that we can just write out whatever we want in a
working copy since "disk space is cheap". Unfortunately network
bandwidth often isn't.
Mix
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: performance after 1.0: less metadata files on client?
Posted by Bryan O'Sullivan <bo...@serpentine.com>.
On Sun, 2002-08-11 at 05:53, solo turn wrote:
> yesterday we tried to check out our repository via http to a working
> copy on a windows network drive. the result was not to good: more
> than 6 hours, compared to 15-20 minutes with source safe.
That's pretty horrifying, but I'd be leery of drawing many conclusions
without also testing on a local filesystem on your Windows box. That
way, you can discount any possible overhead from SMB. It would be
interesting to see if there's a performance difference between the two.
> or do you somehow plan to
> switch later to reiser4 where you attach all this files as properties
> of files in reiserfs (http://www.reiserfs.org/v4/v4.html)?
That would make running on Windows a tad difficult ...
<b
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: performance after 1.0: less metadata files on client?
Posted by Barry Scott <ba...@ntlworld.com>.
You also want to test both VSS and SVN with virus protection enabled.
VSS is slowed by a factor 10 or more when virus protection is active
on the client. And with Norton anti-virus you cannot configure it to
not scan the VSS files expect by telling Norton to not scan network
drives (corporate edition only).
Oh and if you use VSS via its COM automation interface expect even
lower performance. The GUI has interfaces that are many times faster
then the automation interface.
BArry
> -----Original Message-----
> From: solo turn [mailto:soloturn99@yahoo.com]
> Sent: 11 August 2002 13:54
> To: svn
> Subject: performance after 1.0: less metadata files on client?
>
>
> yesterday we tried to check out our repository via http to a working
> copy on a windows network drive. the result was not to good: more
> than 6 hours, compared to 15-20 minutes with source safe.
>
> the problem seems to be related to the structure of the metadata:
> - while visual source safe uses one file per directory
> - svn uses 5 files and 11 directories per directory
> - svn uses 3 files per file
>
> we store java code which tends to have many directories. we expect
> soon to reach 1000, this then would mean 16000 additional files&dirs
> through svn (without using rights&props). additionally we have 4000
> source files, which means another 12000 files.
>
> could this be a bottleneck on the long run? or do you somehow plan to
> switch later to reiser4 where you attach all this files as properties
> of files in reiserfs (http://www.reiserfs.org/v4/v4.html)?
>
> ford.
>
>
>
>
> __________________________________________________
> Do You Yahoo!?
> HotJobs - Search Thousands of New Jobs
> http://www.hotjobs.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org