You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by "Harvey, Edward" <Ed...@patni.com> on 2008/05/09 20:23:38 UTC
Another comment on performance
I have been benchmarking many different systems and platforms over the last week or two. My findings, I think, probably won't have much effect here, given the present direction of WC rewrite, but I figure it's at least worth mention.
The benchmark is: (Before beginning) Create a large directory of small files. Reboot to completely clear any caching. And then time how long it takes to scan through those files. Repeat with varying numbers of threads, on varying hardware types, rebooting each time to invalidate cache.
Let's take a typical PC as the measuring stick. One sata disk, plenty of memory & core2 duo processor. I'll call the speed of this machine 1.0x. (1m 04s to scan 100,000 files)
I found that if scanning a large directory tree on the one-disk pc, if you use multiple threads, it hurts performance. Test was repeated with 2 threads, 5 threads, and 10 threads. Each test successively worse than the previous. 1m22s , 1m34s , 1m38s. Each sample was taken 3 times, and never varied more than 1s.
I ran the same tests on a typical RAID5 server - 8 sata 7200rpm disks, raid5 + hotspare = capacity & speed of 6 disks, with caching raid controller card.
I found with 1 thread, the server performance scanning a large directory of files is about 1.4x (0m 44s to scan 100,000 files)
I found with 5-10 threads, the server performance is about 2x. (10 threads, 0m 32s to scan 100,000 files)
The point was simply to see how threading helps or hinders performance, by trying to fully leverage the hardware available.
Conclusion:
* Typical PC performs the best for file-scanning test with a single thread.
* Typical RAID5 server performs the best for file-scanning test with multiple threads.
Single Disk, 1 thread 1.0x (3 samples)
Single Disk, 2 threads 0.79x (3 samples)
Single Disk, 5 threads 0.68x (3 samples)
Single Disk, 10 threads 0.65x (3 samples)
RAID5 Disks, 1 thread 1.45x (2 samples)
RAID5 Disks, 2 threads 1.82x (1 samples)
RAID5 Disks, 5 threads 1.91x (1 samples)
RAID5 Disks, 10 threads 2.00x (2 samples)
This e-mail message may contain proprietary, confidential or legally privileged information for the sole use of the person or entity to whom this message was originally addressed. Any review, e-transmission dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you have received this e-mail in error kindly delete this e-mail from your records. If it appears that this mail has been forwarded to you without proper authority, please notify us immediately at netadmin@patni.com and delete this mail.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Another comment on performance
Posted by Listman <li...@burble.net>.
On May 9, 2008, at 6:04 PM- May 9, 2008, Branko Čibej wrote:
> Harvey, Edward wrote:
>> I found that if scanning a large directory tree on the one-disk pc,
>> if you use multiple threads, it hurts performance. Test was
>> repeated with 2 threads, 5 threads, and 10 threads. Each test
>> successively worse than the previous. 1m22s , 1m34s , 1m38s. Each
>> sample was taken 3 times, and never varied more than 1s.
>>
>
> That makes perfect sense. Double the number of threads and you'll
> (try to) double the rate of seeks -- and seek time is a bottleneck.
>
SSD disks would be a nice thing to have here ;) (the seek time is 0
as the lookup is direct)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Another comment on performance
Posted by Branko Čibej <br...@xbc.nu>.
Harvey, Edward wrote:
> I found that if scanning a large directory tree on the one-disk pc, if you use multiple threads, it hurts performance. Test was repeated with 2 threads, 5 threads, and 10 threads. Each test successively worse than the previous. 1m22s , 1m34s , 1m38s. Each sample was taken 3 times, and never varied more than 1s.
>
That makes perfect sense. Double the number of threads and you'll (try
to) double the rate of seeks -- and seek time is a bottleneck.
> I ran the same tests on a typical RAID5 server - 8 sata 7200rpm disks, raid5 + hotspare = capacity & speed of 6 disks, with caching raid controller card.
>
> I found with 1 thread, the server performance scanning a large directory of files is about 1.4x (0m 44s to scan 100,000 files)
>
> I found with 5-10 threads, the server performance is about 2x. (10 threads, 0m 32s to scan 100,000 files)
>
Yup -- with more disks, you get more spindles to distribute the seeks on
in parallel.
(But do note that "it all boils down to seeks" is way too naïve an
interpretation.)
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org