You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Blair Zajac <bl...@orcaware.com> on 2004/01/31 00:22:51 UTC
svntest and tmpfs slow as disk
I'm now running svntest on a dual 3.06 Ghz system with 2 Gbytes of
RAM and using the ramdisk, but it's just as fast as running
the tests to disk. Has anybody else seen this. Everything looks
set up ok:
Here's a mount point:
% mount |grep tmpfs
tmpfs on /export/home1/blair/Code/SCMs/Subversion/smoke-test/obj-svn-shared/subversion/tests
type tmpfs (rw,nosuid,nodev,size=600m,user=blair)
Here's one of the test scripts:
% ps -p 29834 -fww
UID PID PPID C STIME TTY TIME CMD
blair 29834 21128 1 16:17 pts/0 00:00:00
/opt/i386-linux/python/bin/python2
/export/home1/blair/Code/SCMs/Subversion/smoke-test/svn/subversion/tests/clients/cmdline/update_tests.py
--verbose
It's cwd is in the tmpfs
% lsof -p 29834
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
python2 29834 blair cwd DIR 0,9 100 87961871
/export/home1/blair/Code/SCMs/Subversion/smoke-test/obj-svn-shared/subversion/tests/clients/cmdline
% grep tmpfs /etc/fstab
none /dev/shm tmpfs defaults 0 0
tmpfs /export/home1/blair/Code/SCMs/Subversion/smoke-test/obj-svn-shared/subversion/tests tmpfs defaults,user,noauto,exec,size=600m
I would expect these tests to scream, but they don't. Look at the
top output:
16:19:50 up 24 days, 6:20, 6 users, load average: 0.18, 0.21, 0.16
137 processes: 132 sleeping, 1 running, 0 zombie, 4 stopped
CPU0 states: 15.0% user 19.0% system 15.0% nice 0.0% iowait 64.0% idle
CPU1 states: 22.0% user 21.0% system 22.0% nice 0.0% iowait 55.0% idle
Mem: 2064440k av, 1762560k used, 301880k free, 0k shrd, 230028k buff
970980k actv, 74820k in_d, 46316k in_c
Swap: 1048536k av, 337900k used, 710636k free 1126500k cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
1937 blair 35 10 3768 3768 2596 S N 4.9 0.1 0:00 1 lt-svn
395 blair 35 10 3512 3512 1784 S N 2.9 0.1 0:00 0 python2
Only a load of 0.2. Running a timing test with a ext3 and tmpfs
show the same time for a full svntest (1 hour). This is on RedHat 9.
Very odd. Anybody has any ideas?
Best,
Blair
--
Blair Zajac <bl...@orcaware.com>
Plots of your system's performance - http://www.orcaware.com/orca/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntest and tmpfs slow as disk
Posted by Jani Averbach <ja...@jaa.iki.fi>.
On 2004-02-17 12:52-0800, Blair Zajac wrote:
> Jani Averbach wrote:
> >
> > On 2004-01-30 16:22-0800, Blair Zajac wrote:
> >
> > > I'm now running svntest on a dual 3.06 Ghz system with 2 Gbytes of
> > > RAM and using the ramdisk, but it's just as fast as running
> > > the tests to disk. Has anybody else seen this.
> >
> > I have definetly seen that with 2.6.x kernels and
> > __with ra_dav__. With 2.6.x kernels the tests almost stalled (dual
> > amd64 system, 2G RAM), and take _a_lot_of_ time. With ordinary
> > hd-based fs it will finish just fine. With 2.4.x kernels the tests are
> > behaving normally with tmpfs vs. hd-based fs.
Well, I think that I have found a reason for that.
It seems that my system is running out of randomness (/dev/random is
empty, the server is unattended), and that lack of randomness is
blocking the test suite. After I have switched apr to use
/dev/hw_random, everything seems to run with 2.6 kernel as fast as (or
faster) than with 2.4 kernel.
BR, Jani
--
Jani Averbach
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntest and tmpfs slow as disk
Posted by Blair Zajac <bl...@orcaware.com>.
Jani Averbach wrote:
>
> On 2004-01-30 16:22-0800, Blair Zajac wrote:
>
> > I'm now running svntest on a dual 3.06 Ghz system with 2 Gbytes of
> > RAM and using the ramdisk, but it's just as fast as running
> > the tests to disk. Has anybody else seen this.
>
> I have definetly seen that with 2.6.x kernels and
> __with ra_dav__. With 2.6.x kernels the tests almost stalled (dual
> amd64 system, 2G RAM), and take _a_lot_of_ time. With ordinary
> hd-based fs it will finish just fine. With 2.4.x kernels the tests are
> behaving normally with tmpfs vs. hd-based fs.
>
> > Only a load of 0.2.
>
> Same symptoms here, except that:
>
> > Running a timing test with a ext3 and tmpfs show the same time for a
> > full svntest (1 hour).
>
> In my system, ra_dav on tmpfs takes a much more longer than on ext3.
>
> > Very odd. Anybody has any ideas?
> >
>
> And one point more: ra_local and ra_svn work just fine with tmpfs, but
> ra_dav not -- but also ra_dav works ok with 2.4.x kernels. So I would
> point my finger to the 2.6.x kernels, and probably to the scheduler?
Odd that 2.6.x would behave differently there.
This system is running RedHat 9 with the latest kernel update from
RedHat, which is kernel-2.4.20-28.9.
Best,
Blair
--
Blair Zajac <bl...@orcaware.com>
Plots of your system's performance - http://www.orcaware.com/orca/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntest and tmpfs slow as disk
Posted by Jani Averbach <ja...@jaa.iki.fi>.
On 2004-01-30 16:22-0800, Blair Zajac wrote:
> I'm now running svntest on a dual 3.06 Ghz system with 2 Gbytes of
> RAM and using the ramdisk, but it's just as fast as running
> the tests to disk. Has anybody else seen this.
I have definetly seen that with 2.6.x kernels and
__with ra_dav__. With 2.6.x kernels the tests almost stalled (dual
amd64 system, 2G RAM), and take _a_lot_of_ time. With ordinary
hd-based fs it will finish just fine. With 2.4.x kernels the tests are
behaving normally with tmpfs vs. hd-based fs.
> Only a load of 0.2.
Same symptoms here, except that:
> Running a timing test with a ext3 and tmpfs show the same time for a
> full svntest (1 hour).
In my system, ra_dav on tmpfs takes a much more longer than on ext3.
> Very odd. Anybody has any ideas?
>
And one point more: ra_local and ra_svn work just fine with tmpfs, but
ra_dav not -- but also ra_dav works ok with 2.4.x kernels. So I would
point my finger to the 2.6.x kernels, and probably to the scheduler?
BR, Jani
--
Jani Averbach
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntest and tmpfs slow as disk
Posted by Philip Martin <ph...@codematters.co.uk>.
Blair Zajac <bl...@orcaware.com> writes:
> Regarding the different build directories, do you have all three
> builds proceeding from the building apr/apr-util/httpd/svn and
> running make test, or just the make test bit?
Just the "make" and "make check" for Subversion, I rarely build
Apache/APR.
> What changes did you make to svntest.sh?
None, I don't use it.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntest and tmpfs slow as disk
Posted by Blair Zajac <bl...@orcaware.com>.
Philip Martin wrote:
>
> Blair Zajac <bl...@orcaware.com> writes:
>
> > I'm now running svntest on a dual 3.06 Ghz system with 2 Gbytes of
> > RAM and using the ramdisk, but it's just as fast as running
> > the tests to disk.
>
> Have the physical times got better, or the tmpfs times got worse?
I don't have any old systems to compare this with, as this is my
first install of svntest.
>
> > Has anybody else seen this.
>
> No, I see tmpfs as significantly faster. Running update_tests.py on
> an old dual P3 I see the physical disk take about 230s and the tmpfs
> disk about 120s. That's wall clock time, both use about 60s of CPU.
> The figures are much the same for ra_local and ra_svn, while ra_dav
> takes about 10s more wall clock time.
>
> One of the limiting factors is the command line client's timestamp
> sleep; that's why the wall clock time is always going to be
> significantly greater than the CPU time. On the other hand I can use
> different build directories to run different RA layers in parallel
I was thinking about this and maybe the identical wall clock times
for ram vs ext3 filesystem performance is that the ext3 filesystem
is on a battery backed RAID filesystem which lets the OS think that
writes happen immediately??? So maybe on a normal filesystem without
the immediate writes I would see similar time differences as other
people.
Regarding the different build directories, do you have all three
builds proceeding from the building apr/apr-util/httpd/svn and
running make test, or just the make test bit? What changes did
you make to svntest.sh?
Best,
Blair
--
Blair Zajac <bl...@orcaware.com>
Plots of your system's performance - http://www.orcaware.com/orca/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svntest and tmpfs slow as disk
Posted by Philip Martin <ph...@codematters.co.uk>.
Blair Zajac <bl...@orcaware.com> writes:
> I'm now running svntest on a dual 3.06 Ghz system with 2 Gbytes of
> RAM and using the ramdisk, but it's just as fast as running
> the tests to disk.
Have the physical times got better, or the tmpfs times got worse?
> Has anybody else seen this.
No, I see tmpfs as significantly faster. Running update_tests.py on
an old dual P3 I see the physical disk take about 230s and the tmpfs
disk about 120s. That's wall clock time, both use about 60s of CPU.
The figures are much the same for ra_local and ra_svn, while ra_dav
takes about 10s more wall clock time.
One of the limiting factors is the command line client's timestamp
sleep; that's why the wall clock time is always going to be
significantly greater than the CPU time. On the other hand I can use
different build directories to run different RA layers in parallel
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org