You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by "Martin J. Evans" <ma...@easysoft.com> on 2004/06/03 17:42:50 UTC
svn client fails with Berkeley DB error Cannot allocate memory - repository broken
Hi subversion developers,
I've posted this problem to the users list and although I've had some
suggestions I am stuck with what looks like a serious bug in subversion.
The basic problem I'm seeing is I have a small perl script which when run using
a HTTP (remotely) or FILE (locally) URL returns:
Berkeley DB error while opening 'transactions' table for filesystem
/var/subversion/distribution/linux-x86/db:
Cannot allocate memory
and the repository is then broken. I am using subversion 1.0.2 and cannot
upgrade currently but others in the users list seem to see a similar problem
with 1.0.4. I can reproduce at will. I have tried doubling the lock values in
DB_CONFIG - it made no difference. The machine subversion is running on is
Linux with kernel 2.4.23, dual Xeon 2.4GHz and 2Gb of memory. subversion was
compiled with gcc 2.95.3 and uses glibc 2.2.4 and BDB 4.2.52.
I can replicate the problem at will now by doing this:
1. get hold of subversion sources
cd /tmp
svn co http://svn.collab.net/repos/svn/trunk subversion
cd subversion
find . -name .svn -exec rm -fr {} \;
2. create test repository
cd dir where repositories create (/var/subversion/distribution for me).
svnadmin create test-martin
3. import subversion tree into test repository.
cd /tmp/subversion
svn import . file:///var/subversion/distribution/test-martin -m "test"
4. prove the db is OK
cd /var/subversion/distribution
svnadmin recover test-martin
sh-2.05$ svnadmin recover test-martin/
Please wait; recovering the repository may take some time...
Recovery completed.
The latest repos revision is 1.
5. run the attached perl program with:
/tmp/x.pl -u file:///var/subversion/distribution/test-martin
Couldn't open a repository: Unable to open an ra_local session to URL: Unable
to open repository
'file:///var/subversion/distribution/test-martin/doc/book/README': Berkeley DB
error while opening 'uuids' table for filesystem
/var/subversion/distribution/test-martin/db:
Cannot allocate memory at /tmp/x.pl line 17
6. demonstrate DB is now broken and unrecoverable
cd /var/subversion/distribution
svnadmin recover test-martin
sh-2.05$ svnadmin recover test-martin
Please wait; recovering the repository may take some time...
svn: DB_RUNRECOVERY: Fatal error, run database recovery
This is repeatable at will for me.
I've followed the excellent hint from C. Michael Pilato and used db_stat before
and after running the perl script. The results were:
before:
2000 Maximum number of locks possible.
2000 Maximum number of lockers possible.
2000 Maximum number of lock objects possible.
9 Maximum number of locks at any one time.
17 Maximum number of lockers at any one time.
9 Maximum number of lock objects at any one time.
after:
2000 Maximum number of locks possible.
2000 Maximum number of lockers possible.
2000 Maximum number of lock objects possible.
883 Maximum number of locks at any one time.
1760 Maximum number of lockers at any one time.
19 Maximum number of lock objects at any one time.
I have never managed to recover a repository broken in this way but am open to
suggestions. All the arguments for changing the attached perl script to make it
succeed are irrelevant to me - I can make the script work fine via http by
reducing the number of max_client in apache or by introducing sleeps into it
(speed was never important for it). The error appears to happen at the
server end and it leaves my repository broken.
What I particularly don't like about it is that I can make this
happen using a http URL which implies to me I could break other public
repositories and they could break my public repositories. Needless to say I've
not tried this.
I am part of a team which proposed a move to subversion but am not looking
forward to explaining why anyone can break a repository. As such if anyone has
any suggestions to try I would be delighted to hear from you.
Thank you.
Martin
--
Martin J. Evans
Easysoft Ltd, UK
Development
Re: svn client fails with Berkeley DB error Cannot allocate memory - repository broken
Posted by "Martin J. Evans" <ma...@easysoft.com>.
Ben,
Thanks for trying.
I'd love to hear from anyone who has reproduced this problem or who knows why it
breaks my repositories.
Martin
--
Martin J. Evans
Development, Easysoft Ltd
Quoting Ben Collins-Sussman <su...@collab.net>:
> On Thu, 2004-06-03 at 12:42, Martin J. Evans wrote:
>
> > 5. run the attached perl program with:
> >
> > /tmp/x.pl -u file:///var/subversion/distribution/test-martin
>
> Well, I tried to reproduce, but I think my perl bindings are either too
> old or too new (from /trunk). I get an assertion failure on a
> path-join.
>
> Someone who's actively using perl bindings should try to reproduce.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn client fails with Berkeley DB error Cannot allocate memory - repository broken
Posted by Ben Reser <be...@reser.org>.
On Fri, Jun 04, 2004 at 03:13:09PM -0500, Ben Collins-Sussman wrote:
> On Thu, 2004-06-03 at 12:42, Martin J. Evans wrote:
>
> > 5. run the attached perl program with:
> >
> > /tmp/x.pl -u file:///var/subversion/distribution/test-martin
>
> Well, I tried to reproduce, but I think my perl bindings are either too
> old or too new (from /trunk). I get an assertion failure on a
> path-join.
>
> Someone who's actively using perl bindings should try to reproduce.
You probably used . as the path. :) Which isn't valid input to the
subversion libraries. You have to use full paths. This has come up
before with the Perl bindings. Perl apps can of course use the Cwd
module and abs_path() to work around this.
--
Ben Reser <be...@reser.org>
http://ben.reser.org
"Conscience is the inner voice which warns us somebody may be looking."
- H.L. Mencken
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn client fails with Berkeley DB error Cannot allocate memory
- repository broken
Posted by Ben Collins-Sussman <su...@collab.net>.
On Thu, 2004-06-03 at 12:42, Martin J. Evans wrote:
> 5. run the attached perl program with:
>
> /tmp/x.pl -u file:///var/subversion/distribution/test-martin
Well, I tried to reproduce, but I think my perl bindings are either too
old or too new (from /trunk). I get an assertion failure on a
path-join.
Someone who's actively using perl bindings should try to reproduce.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn client fails with Berkeley DB error Cannot allocate memory - repository broken
Posted by Ben Reser <be...@reser.org>.
On Thu, Jun 03, 2004 at 06:42:50PM +0100, Martin J. Evans wrote:
> Hi subversion developers,
>
> I've posted this problem to the users list and although I've had some
> suggestions I am stuck with what looks like a serious bug in subversion.
>
> The basic problem I'm seeing is I have a small perl script which when run using
> a HTTP (remotely) or FILE (locally) URL returns:
>
> Berkeley DB error while opening 'transactions' table for filesystem
> /var/subversion/distribution/linux-x86/db:
> Cannot allocate memory
>
> and the repository is then broken. I am using subversion 1.0.2 and cannot
> upgrade currently but others in the users list seem to see a similar problem
> with 1.0.4. I can reproduce at will. I have tried doubling the lock values in
> DB_CONFIG - it made no difference. The machine subversion is running on is
> Linux with kernel 2.4.23, dual Xeon 2.4GHz and 2Gb of memory. subversion was
> compiled with gcc 2.95.3 and uses glibc 2.2.4 and BDB 4.2.52.
>
> I can replicate the problem at will now by doing this:
>
> 1. get hold of subversion sources
> cd /tmp
> svn co http://svn.collab.net/repos/svn/trunk subversion
> cd subversion
> find . -name .svn -exec rm -fr {} \;
>
> 2. create test repository
> cd dir where repositories create (/var/subversion/distribution for me).
> svnadmin create test-martin
>
> 3. import subversion tree into test repository.
> cd /tmp/subversion
> svn import . file:///var/subversion/distribution/test-martin -m "test"
>
> 4. prove the db is OK
>
> cd /var/subversion/distribution
> svnadmin recover test-martin
>
> sh-2.05$ svnadmin recover test-martin/
> Please wait; recovering the repository may take some time...
>
> Recovery completed.
> The latest repos revision is 1.
>
> 5. run the attached perl program with:
>
> /tmp/x.pl -u file:///var/subversion/distribution/test-martin
>
> Couldn't open a repository: Unable to open an ra_local session to URL: Unable
> to open repository
> 'file:///var/subversion/distribution/test-martin/doc/book/README': Berkeley DB
> error while opening 'uuids' table for filesystem
> /var/subversion/distribution/test-martin/db:
> Cannot allocate memory at /tmp/x.pl line 17
I get a slightly different error:
Couldn't open a repository: Unable to open an ra_local session to URL:
Unable to open repository
'file:///home/breser/mje-test/test-martin/contrib/client-side': Berkeley
DB error while opening 'revisions' table for filesystem
/home/breser/mje-test/test-martin/db:
Cannot allocate memory at x.pl line 15
You'll note that my error is opening revisions, not uuids. Also I'm
using bdb 4.1 not 4.2.
But...
> 6. demonstrate DB is now broken and unrecoverable
>
> cd /var/subversion/distribution
> svnadmin recover test-martin
>
> sh-2.05$ svnadmin recover test-martin
> Please wait; recovering the repository may take some time...
> svn: DB_RUNRECOVERY: Fatal error, run database recovery
My repostiory doesn't get broken, nor do I have any problem running
recover. I can run that script forever without any problems.
Might be worthwhile to write a small client that does the same thing in
C and statically link and run gdb over it...
> I have never managed to recover a repository broken in this way but am
> open to suggestions. All the arguments for changing the attached perl
> script to make it succeed are irrelevant to me - I can make the script
> work fine via http by reducing the number of max_client in apache or
> by introducing sleeps into it (speed was never important for it). The
> error appears to happen at the server end and it leaves my repository
> broken.
Agred, this is either a bug in bdb or a bug in the way we're using it.
Either way it may very well depend upon during what operation you run
out of memory, which may make it difficult for us to reproduce.
> What I particularly don't like about it is that I can make this happen
> using a http URL which implies to me I could break other public
> repositories and they could break my public repositories. Needless to
> say I've not tried this.
Based on my testing it may or may not happen on a server. If you can
write a program to the C API that produces the same problem I'd be very
interested in the code path it's taking...
--
Ben Reser <be...@reser.org>
http://ben.reser.org
"Conscience is the inner voice which warns us somebody may be looking."
- H.L. Mencken
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org