Re: Directory size problem?

[kfogel <kf...@collab.net>]

Hal Mahaffey <hm...@aol.com> writes:
> already had a case where a single directory contained build manifests for
> 15,000 builds.  Performance accessing these files (in CVS) was noticeably
> painful.   We solved it by indexing the files in subdirs based on the 1st
> character of the build name: A...Za...z.
>
> Has anyone noticed performance problems with repositories with 10,000+
> revisions?

I don't believe we've seen any actual reports of it being a problem.

> What "older native filesystems" have this problem?  We were using Veritas v3.5
> on EMC Symms and noticed the problem.  Are there any recommended filesystems?
>
> If there *is* a problem with larger repositories, has the Subversion community
> considered indexing the 'db/revs' directory with, say, the first digit (so
> revision 2423 would go in db/revs/2)?  Just a thought.

Yep, we've considered that and other similar approaches.  We decided
not to do anything until we hear reports of the current arrangement
being a problem, that's all.

-Karl

-- 
www.collab.net  <>  CollabNet  |  Distributed Development On Demand

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Directory size problem?

[Nico Kadel-Garcia <nk...@comcast.net>]

kfogel wrote:
> Hal Mahaffey <hm...@aol.com> writes:
>> already had a case where a single directory contained build
>> manifests for 15,000 builds.  Performance accessing these files (in
>> CVS) was noticeably painful.   We solved it by indexing the files in
>> subdirs based on the 1st character of the build name: A...Za...z.
>>
>> Has anyone noticed performance problems with repositories with
>> 10,000+ revisions?
>
> I don't believe we've seen any actual reports of it being a problem.
>
>> What "older native filesystems" have this problem?  We were using
>> Veritas v3.5 on EMC Symms and noticed the problem.  Are there any
>> recommended filesystems?
>>
>> If there *is* a problem with larger repositories, has the Subversion
>> community considered indexing the 'db/revs' directory with, say, the
>> first digit (so revision 2423 would go in db/revs/2)?  Just a
>> thought.
>
> Yep, we've considered that and other similar approaches.  We decided
> not to do anything until we hear reports of the current arrangement
> being a problem, that's all.

What a wise person!

I wouldn't expect problems with ext3 filesystems, even with 2.4 Linux 
kernels, until you hit better than 5000 files in a directory. Certain 
operations, such as "ls -l" operations bog down quite badly: 2.6 kernels 
fixed this by implementing "htrees" for ext3.

Your Mileage May Vary for other filesystems: Reiserfs is famous for being 
able to handle things like news spools, but I find it unstable in extended 
use. 


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org