You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Roland Schwingel <Ro...@onevision.de> on 2003/10/07 07:34:56 UTC
Subversion 0.30.0 stability problems : not able to check in bigger pieces
Hi....
As you might know I got problems with subversion concerning speed and
stability... I found something new (and unfortunately bad)... At least for
me...
I am running Suse Linux 8.0 on a Dual AMD 1600MP machine with 512MB Ram.
Kernel 2.4.20.
apache2 2.0.47
berkeley db 4.1.25
subversion 0.30.0
I tried the following (checkin apache 2.0.47 source) to test stability
about 10 times and it never finished....
rm -rf /path/to/test_repos
svnadmin create /path/to/test_repos
# recover ist just to be sure due to my recent problems, but it doesn't
matter if I use it here or not
svnadmin recover /path/to/test_repos
svn co http://svnserver/svn/test_repos svn_test
cd svn_test
tar xvfz /path/to/httpd-2.0.47.tar.gz
svn add *
svn ci -m test
It starts to check in first very fast, later it gets slower and slower and
finally gets stuck...
...
Adding httpd-2.0.47/test/test_parser.c
Adding httpd-2.0.47/test/test_select.c
Adding httpd-2.0.47/test/time-sem.c
Adding httpd-2.0.47/test/zb.c
Transmitting file data
.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................svn:
RA layer request failed
svn: Commit failed (details follow):
svn: PUT of
/svn/test_repos/!svn/wrk/e19f3a86-13c9-0310-a9ec-de66ee5e4426/httpd-2.0.47
/docs/manual/ssl/ssl_compat.xml.meta: timed out waiting for server
(http://svnserver)
The number of dots it paints varies, sometimes more sometimes less (and
therefore never fails at the same file to checkin). I am the only one who
uses svn here so there is no concurrency.
And sometimes something also appears to segfault...
from apache's error_log:
[Tue Oct 07 09:11:18 2003] [notice] child pid 23267 exit signal
Segmentation fault (11)
It appears soon before "svn: RA layer request failed" Not always but about
20% of my tests....
Yet it appears to me that 0.30.0 is not really useable...
Is someone else also having these problems?
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Subversion 0.30.0 stability problems : not able to check in bigger
pieces
Posted by Roland Schwingel <Ro...@onevision.de>.
Hi Brian
Brian Mathis <bm...@directedge.com> wrote on 07.10.2003 18:59:06:
> This sounds to me very much to be a memory problem. You are probably
> running out of it. How large is the data you are checking in?
It is the plain vanilla apache 2.0.47 distribution (~33 MB, 2637 files) I
want to check in (just for testing svn's stability)
> The fact that things are slowing down sounds like it is probably using
> the pagefile.
There is no swapping the whole time. Memory consumption of svn/http is
between
6-17MB. It appears to be a locking issue of the BDB. See my reply to Ray's
mail.
Thanks also for helping,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Subversion 0.30.0 stability problems : not able to check in bigger
pieces
Posted by Brian Mathis <bm...@directedge.com>.
This sounds to me very much to be a memory problem. You are probably
running out of it. How large is the data you are checking in?
The fact that things are slowing down sounds like it is probably using
the pagefile.
My suggestion is to open 2 windows to the server, run 'top' in 1 and run
the checkout in the other. Watch the memory usage and see what happens.
When you start top, press capital 'M' to sort by memory usage.
Roland Schwingel wrote:
> Hi....
>
> As you might know I got problems with subversion concerning speed and
> stability... I found something new (and unfortunately bad)... At least for
> me...
>
> I am running Suse Linux 8.0 on a Dual AMD 1600MP machine with 512MB Ram.
> Kernel 2.4.20.
> apache2 2.0.47
> berkeley db 4.1.25
> subversion 0.30.0
>
> I tried the following (checkin apache 2.0.47 source) to test stability
> about 10 times and it never finished....
>
> rm -rf /path/to/test_repos
> svnadmin create /path/to/test_repos
> # recover ist just to be sure due to my recent problems, but it doesn't
> matter if I use it here or not
> svnadmin recover /path/to/test_repos
> svn co http://svnserver/svn/test_repos svn_test
> cd svn_test
> tar xvfz /path/to/httpd-2.0.47.tar.gz
> svn add *
> svn ci -m test
>
> It starts to check in first very fast, later it gets slower and slower and
> finally gets stuck...
> ...
> Adding httpd-2.0.47/test/test_parser.c
> Adding httpd-2.0.47/test/test_select.c
> Adding httpd-2.0.47/test/time-sem.c
> Adding httpd-2.0.47/test/zb.c
> Transmitting file data
> .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................svn:
> RA layer request failed
> svn: Commit failed (details follow):
> svn: PUT of
> /svn/test_repos/!svn/wrk/e19f3a86-13c9-0310-a9ec-de66ee5e4426/httpd-2.0.47
> /docs/manual/ssl/ssl_compat.xml.meta: timed out waiting for server
> (http://svnserver)
>
> The number of dots it paints varies, sometimes more sometimes less (and
> therefore never fails at the same file to checkin). I am the only one who
> uses svn here so there is no concurrency.
>
> And sometimes something also appears to segfault...
> from apache's error_log:
> [Tue Oct 07 09:11:18 2003] [notice] child pid 23267 exit signal
> Segmentation fault (11)
> It appears soon before "svn: RA layer request failed" Not always but about
> 20% of my tests....
>
> Yet it appears to me that 0.30.0 is not really useable...
> Is someone else also having these problems?
>
> Roland
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
--
Brian Mathis
http://www.directedge.com/b/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Subversion 0.30.0 stability problems : not able to check in bigger pieces
Posted by David Good <dg...@fsautomation.com>.
On Tue, Oct 07, 2003 at 10:43:32AM +0200, Roland Schwingel <Ro...@onevision.de> wrote:
>
>
>
>
> Hi...
>
> Jan Evert van Grootheest <j....@euronext.nl> wrote on 07.10.2003
> 09:59:45:
> > Roland,
> >
> > On linux, signal 11 usually indicates memory (hardware) problems...
> > I'd suggest using memtest (or was that memtest86) for a couple of days.
> The machine runs very stable. It has a uptime of 191 days. I can do
> memtest86 next weekend, but I do not suspect *any* hardware trouble. It
> runs sendmail, cvs and so on and everything stable as a rock. It has a
> loadfactor of max. ~0.5 and is very responsive. Hard drives could be faster
> for my taste but thats not the reason for this.
Signal 11 is the "segmentation violation" signal. While I suppose it
*could* mean you have hardware memory problems, I've never seen that to
be the case. Usually, you get a segmentation violation when you try to
dereference an invalid pointer.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Subversion 0.30.0 stability problems : not able to check in bigger
pieces
Posted by Jan Evert van Grootheest <j....@euronext.nl>.
Roland,
I also read the Linux kernel mailing list. These are just the first
questions that will get asked in reply to a mail like yours (to rule out
users being dumb).
Oh, I forgot about the (linux) logfiles. I guess there are no clues in
there either...
-- Jan Evert
Roland Schwingel wrote:
>
>
>
> Hi...
>
> Jan Evert van Grootheest <j....@euronext.nl> wrote on 07.10.2003
> 09:59:45:
>
>>Roland,
>>
>>On linux, signal 11 usually indicates memory (hardware) problems...
>>I'd suggest using memtest (or was that memtest86) for a couple of days.
>
> The machine runs very stable. It has a uptime of 191 days. I can do
> memtest86 next weekend, but I do not suspect *any* hardware trouble. It
> runs sendmail, cvs and so on and everything stable as a rock. It has a
> loadfactor of max. ~0.5 and is very responsive. Hard drives could be faster
> for my taste but thats not the reason for this.
>
>
>>Other standard questions: are you overclocking? is your PSU good enough?
>>is your video card (if running something graphical) not using too much
>>power? what's the temperature of the CPU, disks etc?
>
> Of course no overclocking, enough powersupply (2 redundant regulated
> systems each powerful enough for a 4 CPU machine). Machine runs in a
> climanted server room, no gui. Nothing in /var/log/messages or
> /var/log/warn that might suspect installation/hardware problems. CPUs are
> about 53 degrees Celsius. Harddisks are mounted in cooled frames,
> temperature is ~30 degrees Celsius. Voltage for the CPUs/System (12V,5V,3.3
> V etc.) are all in their nominal levels. The machine never crashed since I
> assembled it.
>
> Signal 11 can also be a software problem. Maybe it is a not very well
> caught sideeffect of the lost connection.
> I will set up svn on a different system and make a crosstest but I think
> this will not help.
>
> But thanks for the hint and I will do the memtest and test a second
> (single-CPU) machine.
>
> Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Subversion 0.30.0 stability problems : not able to check in bigger
pieces
Posted by Roland Schwingel <Ro...@onevision.de>.
Hi...
Jan Evert van Grootheest <j....@euronext.nl> wrote on 07.10.2003
09:59:45:
> Roland,
>
> On linux, signal 11 usually indicates memory (hardware) problems...
> I'd suggest using memtest (or was that memtest86) for a couple of days.
The machine runs very stable. It has a uptime of 191 days. I can do
memtest86 next weekend, but I do not suspect *any* hardware trouble. It
runs sendmail, cvs and so on and everything stable as a rock. It has a
loadfactor of max. ~0.5 and is very responsive. Hard drives could be faster
for my taste but thats not the reason for this.
> Other standard questions: are you overclocking? is your PSU good enough?
> is your video card (if running something graphical) not using too much
> power? what's the temperature of the CPU, disks etc?
Of course no overclocking, enough powersupply (2 redundant regulated
systems each powerful enough for a 4 CPU machine). Machine runs in a
climanted server room, no gui. Nothing in /var/log/messages or
/var/log/warn that might suspect installation/hardware problems. CPUs are
about 53 degrees Celsius. Harddisks are mounted in cooled frames,
temperature is ~30 degrees Celsius. Voltage for the CPUs/System (12V,5V,3.3
V etc.) are all in their nominal levels. The machine never crashed since I
assembled it.
Signal 11 can also be a software problem. Maybe it is a not very well
caught sideeffect of the lost connection.
I will set up svn on a different system and make a crosstest but I think
this will not help.
But thanks for the hint and I will do the memtest and test a second
(single-CPU) machine.
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Subversion 0.30.0 stability problems : not able to check in bigger
pieces
Posted by Jan Evert van Grootheest <j....@euronext.nl>.
Roland,
On linux, signal 11 usually indicates memory (hardware) problems...
I'd suggest using memtest (or was that memtest86) for a couple of days.
Other standard questions: are you overclocking? is your PSU good enough?
is your video card (if running something graphical) not using too much
power? what's the temperature of the CPU, disks etc?
-- Jan Evert
Roland Schwingel wrote:
>
>
>
> Hi....
>
> As you might know I got problems with subversion concerning speed and
> stability... I found something new (and unfortunately bad)... At least for
> me...
>
> I am running Suse Linux 8.0 on a Dual AMD 1600MP machine with 512MB Ram.
> Kernel 2.4.20.
> apache2 2.0.47
> berkeley db 4.1.25
> subversion 0.30.0
>
> I tried the following (checkin apache 2.0.47 source) to test stability
> about 10 times and it never finished....
>
> rm -rf /path/to/test_repos
> svnadmin create /path/to/test_repos
> # recover ist just to be sure due to my recent problems, but it doesn't
> matter if I use it here or not
> svnadmin recover /path/to/test_repos
> svn co http://svnserver/svn/test_repos svn_test
> cd svn_test
> tar xvfz /path/to/httpd-2.0.47.tar.gz
> svn add *
> svn ci -m test
>
> It starts to check in first very fast, later it gets slower and slower and
> finally gets stuck...
> ...
> Adding httpd-2.0.47/test/test_parser.c
> Adding httpd-2.0.47/test/test_select.c
> Adding httpd-2.0.47/test/time-sem.c
> Adding httpd-2.0.47/test/zb.c
> Transmitting file data
> .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................svn:
> RA layer request failed
> svn: Commit failed (details follow):
> svn: PUT of
> /svn/test_repos/!svn/wrk/e19f3a86-13c9-0310-a9ec-de66ee5e4426/httpd-2.0.47
> /docs/manual/ssl/ssl_compat.xml.meta: timed out waiting for server
> (http://svnserver)
>
> The number of dots it paints varies, sometimes more sometimes less (and
> therefore never fails at the same file to checkin). I am the only one who
> uses svn here so there is no concurrency.
>
> And sometimes something also appears to segfault...
> from apache's error_log:
> [Tue Oct 07 09:11:18 2003] [notice] child pid 23267 exit signal
> Segmentation fault (11)
> It appears soon before "svn: RA layer request failed" Not always but about
> 20% of my tests....
>
> Yet it appears to me that 0.30.0 is not really useable...
> Is someone else also having these problems?
>
> Roland
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Subversion 0.30.0 stability problems : not able to check in bigger pieces
Posted by Robert Nürnberg <nu...@esono.de>.
> Then checkin works.... Slow but it works. So I assume that
> Ray is right, that this might be an issue of BDB 4.1.25 with 2 CPUs
Don't be too sure. I mentioned before that we had similar problems, but
with a 1 CPU server. Since we did not find a fix we decided to a windows
style approach in solving the problem: Reinstall everything. After that
things worked. We also moved to version 0.30 so that may have helped as
well. Just thought I mention this in case the problem reoccurs.
-rob
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Subversion 0.30.0 stability problems : not able to check in bigger pieces - just for the records
Posted by "C. Michael Pilato" <cm...@collab.net>.
Roland Schwingel <Ro...@onevision.de> writes:
> In 2 CPU mode svn 0.31.0 and bdb 4.0.14 works well. No problems.
> In 1 CPU mode svn 0.31.0 and bdb 4.0.14 works well. No problems.
> In 2 CPU mode svn 0.31.0 and bdb 4.1.25 showed up the same errors. Starting
> with the first checkin
> In 1 CPU mode svn 0.31.0 and bdb 4.1.25 worked nearly well. In 1 of 20
> cases it failed as well..
Roland, you rock. Thank you for taking to time to drill into this
weirdness a bit. While I certainly can't explain your findings
(beyond an initial guess of "there must be a bug in 4.1.25 or our use
of it"), I sincerely appreciate someone putting real faces on the
vague notions of instability that have been wafting through the list
as of late.
> So I am running now with svn 0.31.0 and bdb 4.0.14.
This, it would seem, is probably a good idea. :-)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Subversion 0.30.0 stability problems : not able to check in bigger
pieces - just for the records
Posted by Roland Schwingel <Ro...@onevision.de>.
Hi.....
Just for the records
> On weekend I will disable one cpu on my dual machine and make again a
test
> with BDB 4.1.25 (and a run with memtest86)
Did this on weekend. 11hours memntest86 did not show up any memory
problems.
Made some tests with 1 and 2 CPUs and berkeley db 4.0.14 and 4.1.25.
Each test with a brand new repos and a testcheckin of apache.
As desribed in my initial report. But each loop was done 20 times.
In 2 CPU mode svn 0.31.0 and bdb 4.0.14 works well. No problems.
In 1 CPU mode svn 0.31.0 and bdb 4.0.14 works well. No problems.
In 2 CPU mode svn 0.31.0 and bdb 4.1.25 showed up the same errors. Starting
with the first checkin
In 1 CPU mode svn 0.31.0 and bdb 4.1.25 worked nearly well. In 1 of 20
cases it failed as well..
So I am running now with svn 0.31.0 and bdb 4.0.14.
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Subversion 0.30.0 stability problems : not able to check in bigger
pieces
Posted by Roland Schwingel <Ro...@onevision.de>.
Hi...
Ray Miller <ra...@sysdev.oucs.ox.ac.uk> wrote on 07.10.2003 10:59:29:
> On Tue, Oct 07, 2003 at 10:48:02AM +0200, Roland Schwingel wrote:
> >
> > Thanks for the tip. I will serach the dev list archive. I will also
make
> > a second installation on a single CPU machine.
>
> Thanks - I had also planned to try that, so this will save me some
> trouble. I will be interested to hear how your single CPU test goes.
Made 2 tests...
installed everything on a second machine
AMD Duron 800MB single CPU, 256MB Ram, Suse Linux 8.2
Kernel 2.4.20
apache2 2.0.47
berkley 4.1.25
subversion 0.30.0
And here it works... I did the cycle 5 times... But it is slow. Checking in
of apache 2.0.47
(all local on the same machine, but using http - as shown in my initial
mail) takes about 8 minutes
Only small disk activity, no swapping, CPU ~90% idle (from what top says)
during checkin
(the phase where dots are painted, I think each for a file) before and
afterwards there are phases which
much higher load. Moderate memory consumtion of svn/httpd processes (6-17
MB)
Afterwards I went back to my 2 cpu machine and recompiled everything with
berkeley 4.0.14
Then checkin works.... Slow but it works. So I assume that Ray is right,
that this might be an
issue of BDB 4.1.25 with 2 CPUs. From what I have seen at sleepycat.com
(and the dev mailinglist)
there is a version 4.2.xx scheduled for october with changes in locking
code. Maybe this
helps subversion (and me). I am also eager to get subversion 0.31.0 where
issue 1490 and 1499 appaer
to be fixed (from what I have seen in the branches area of svn.collab.net).
Maybe this helps here to.
On weekend I will disable one cpu on my dual machine and make again a test
with BDB 4.1.25
(and a run with memtest86)
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Subversion 0.30.0 stability problems : not able to check in bigger pieces
Posted by Ray Miller <ra...@sysdev.oucs.ox.ac.uk>.
On Tue, Oct 07, 2003 at 10:48:02AM +0200, Roland Schwingel wrote:
>
> Thanks for the tip. I will serach the dev list archive. I will also make
> a second installation on a single CPU machine.
Thanks - I had also planned to try that, so this will save me some
trouble. I will be interested to hear how your single CPU test goes.
--
Ray Miller, Unix Systems Programmer & Team Leader
Systems Development & Support, Oxford University Computing Services
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Subversion 0.30.0 stability problems : not able to check in bigger
pieces
Posted by Roland Schwingel <Ro...@onevision.de>.
Hello Ray....
Ray Miller <ra...@herald.ox.ac.uk> wrote on 07.10.2003 09:53:03:
> On Tue, Oct 07, 2003 at 09:34:56AM +0200, Roland Schwingel wrote:
> >
> > Yet it appears to me that 0.30.0 is not really useable...
> > Is someone else also having these problems?
>
> I saw something very similar with 0.28.2 on a dual CPU system. I
> didn't report this as I hadn't got round to trying to reproduce it
> with the latest release. There have been a few messages on the dev
> list about problems with Berkeley DB 4.1 on multi-processor systems.
> I wonder if this is the same problem.
Thanks for the tip. I will serach the dev list archive. I will also make
a second installation on a single CPU machine.
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Subversion 0.30.0 stability problems : not able to check in bigger pieces
Posted by Ray Miller <ra...@herald.ox.ac.uk>.
On Tue, Oct 07, 2003 at 09:34:56AM +0200, Roland Schwingel wrote:
>
> Yet it appears to me that 0.30.0 is not really useable...
> Is someone else also having these problems?
I saw something very similar with 0.28.2 on a dual CPU system. I
didn't report this as I hadn't got round to trying to reproduce it
with the latest release. There have been a few messages on the dev
list about problems with Berkeley DB 4.1 on multi-processor systems.
I wonder if this is the same problem.
--
Ray Miller, Unix Systems Programmer & Team Leader
Systems Development & Support, Oxford University Computing Services
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Subversion 0.30.0 stability problems : not able to check in bigger
pieces
Posted by Roland Schwingel <Ro...@onevision.de>.
Hi Seth...
> > I am running Suse Linux 8.0 on a Dual AMD 1600MP machine with 512MB
Ram.
> > Kernel 2.4.20.
> > apache2 2.0.47
> > berkeley db 4.1.25
> > subversion 0.30.0
>
> As another thing to try: The INSTALL file for subversion recommends
> Berkeley DB 4.0.14, perhaps the issues you are having are related to
> using the more recent 4.1.25.
It appears that you are right (like Ray). See my mail I just posted as
reply to his mail...
Thanks also for helping,
Roland
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Subversion 0.30.0 stability problems : not able to check in bigger pieces
Posted by Seth Falcon <sf...@fhcrc.org>.
On Tue, Oct 07, 2003 at 09:34:56AM +0200, Roland Schwingel wrote:
> As you might know I got problems with subversion concerning speed and
> stability... I found something new (and unfortunately bad)... At least for
> me...
>
> I am running Suse Linux 8.0 on a Dual AMD 1600MP machine with 512MB Ram.
> Kernel 2.4.20.
> apache2 2.0.47
> berkeley db 4.1.25
> subversion 0.30.0
As another thing to try: The INSTALL file for subversion recommends
Berkeley DB 4.0.14, perhaps the issues you are having are related to
using the more recent 4.1.25.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org