You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Thomas -lists <th...@mysnip.de> on 2005/03/20 13:51:56 UTC
daily "Cannot allocate memory"-problem
Hey there,
I searched the list archives but couldn't find this problem described
so please point me to posts if that was done somewhere already.
For now I already had to "svnadmin recover" my repository for 3 times
(each day).
It had stopped working with this error:
"Berkeley DB error while opening 'nodes' table for filesystem
/home/subversion/repos/db:
Cannot allocate memory"
Its always the same "table" and error-message.
After running svnadmin recover it worked fine again.
I have only imported one project with around 350 files and one branch
with the same number of
files.
Does that message mean that the table has crashed? How could I avoid that?
That is running on gentoo-linux with these software-versions:
subversion-1.1.3
db-4.1.25_p1
I would really appreciate any pointers what to look for and where.
(oh, I'm using TRAC on the repository and just read a message that I
should use the fsfs-type then for the repos, is that right?)
Thanks in advance,
Thomas
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: daily "Cannot allocate memory"-problem
Posted by Eric Gillespie <ep...@pretzelnet.org>.
Ben Collins-Sussman <su...@collab.net> writes:
> I'm getting a bit annoyed that the automatic answer to every
> BDB problem is "switch to FSFS". :-)
I think the explanation for this is in the *50 lines* that
immediately follow this paragraph.
--
Eric Gillespie <*> epg@pretzelnet.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: daily "Cannot allocate memory"-problem
Posted by John Szakmeister <jo...@szakmeister.net>.
On Tuesday 22 March 2005 05:40, Dirk Schenkewitz wrote:
> Ben Collins-Sussman wrote:
> > On Mar 20, 2005, at 3:25 PM, Erik Huelsmann wrote:
> >> On Sun, 20 Mar 2005 21:21:43 +0000, Martin A. Brooks
> >>
> >> <ma...@hinterlands.org> wrote:
> >>> Ben Collins-Sussman wrote:
> >>>> I'm getting a bit annoyed that the automatic answer to every BDB
> >>>> problem is "switch to FSFS". :-)
> >>>
> >>> That's easy to fix, just make it so BDB doesn't cause about 80% of
> >>> the problems reported to this list :)
> >>
> >> If you all start using fsfs, that should happen soon enough....
> >
> > Yes, there are problems with the way Subversion utilizes BDB. Yes,
> > we're working to fix the usage, and Sleepycat is working on changing
> > BDB to address some of these things as well.
> >
> > Regardless (as has been said in the past): reading this list gives a
> > skewed view of reality. If I were reading this list, I would think
> > BDB is some horrible, unusable thing. My guess, though, is that 95%
> > of people using BDB repositories have no problems whatsoever. The 5%
> > who have problems tend to dominate the list.
>
> Is there any statistical knowledge or anything better than a guess
> about how many people use BDB and how many use FSF?
Unfortunately, no. :-(
> Since I'm on the list, I have not observed one single complaint against
> FSFS, while lots of problems seem to arise from BDB. That is, as far as
> I observed, the problems where the backend is the culprit divide into
> 100%(BDB):0%(FSFS). Maybe I missed something, please correct me if so.
There have been problems, and they've been corrected quickly. However, a
new one just cropped up this morning (see "File corruption causing
SVNadmin seg fault", msgid
<CB...@gmail.com>).
> I choose FSFS because I didn't want to install and take care of another
> big software packet (and because I heard that BDB was a bit unstable
> when used by other people) - now I'm glad. :-)
>
> It looks like FSFS does a very good job - my congrats for creating it!
Don't get me wrong, FSFS has proven to be stable thus far... but it's also
relatively new. The reason that we may not be seeing as many problems is
because it just isn't used as much, and it's operating fine in whatever
conditions it happens to be in. *shrug* All of this is really just
speculation anyways, unless we find a way to get more solid numbers about
the installed user base.
FWIW, I've been using BDB for years now, and I've had no corruptions, no
dataloss, and I've only had to 'svnadmin recover' a repository once...
and that was due to a problem in the way I set it up.
-John
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: daily "Cannot allocate memory"-problem
Posted by Chip Turner <ct...@redhat.com>.
Ben Collins-Sussman <su...@collab.net> writes:
>>> That's easy to fix, just make it so BDB doesn't cause about 80% of the
>>> problems reported to this list :)
>>
>> If you all start using fsfs, that should happen soon enough....
>>
>
> Yes, there are problems with the way Subversion utilizes BDB. Yes,
> we're working to fix the usage, and Sleepycat is working on changing
> BDB to address some of these things as well.
>
> Regardless (as has been said in the past): reading this list gives a
> skewed view of reality. If I were reading this list, I would think
> BDB is some horrible, unusable thing. My guess, though, is that 95%
> of people using BDB repositories have no problems whatsoever. The 5%
> who have problems tend to dominate the list.
I would be highly, highly surprised if the split is 95/5. The minute
you start doing anything complex (svn+ssh with multiple users, for
instance) you -will- have bdb trouble unless you happened to read the
fine print inside the svn book. And even then, following every best
practice from shell scripts wrapping svnserve w/umask to sticky bits
on dirs to the exact version of bdb suggested... problems still arise.
The simple fact is, bdb is one of -the- prohibiting factors from
subversion being more reliable (and, more importantly, as you hint,
being -perceived- as being more reliable).
The simple truth of the matter is, if people use fsfs from the very
beginning, they will have a much smoother, hassle-free experience with
subversion than if they use bdb. Whether it is svn's fault or
sleepycat's fault or solar flares, this remains as a fact -- fsfs is
easier to get running and less prone to peculiar behavior. It would
be one thing if bdb would have troubles getting running, but once
running, behaved properly, or if it had sensible error messages
instead of just freezing every svnserve, svnadmin, etc, but it
doesn't... it's a minefield for an unsophisticated user and it begins
blowing up when they try to take the single repo they have working and
share it among coworkers.
Subversion is so excellent and impressive otherwise; I just find it a
real shame that people get a bad taste in their mouth from using the
bdb backend, especially now that there is such an effective
alternative.
Chip
--
Chip Turner cturner@redhat.com
Red Hat, Inc.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: daily "Cannot allocate memory"-problem
Posted by tekHedd <te...@byteheaven.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Dirk Schenkewitz wrote:
| :-)
|
| tekHedd wrote:
|> OK with BDB 4.1, but the projects are pretty small so far. I am feeling
|> too deal with getting the latest bdb and so on, especially since the
That should be "too lazy to deal". Or proofread, apparently.
| I nearly asked this myself, but then I remembered about this "comparison
| sheet" (FSFS vs. BDB): http://svn.collab.net/repos/svn/trunk/notes/fsfs
| That's all I know. If anyone knows more, I would also like to know.
Thanks much for the pointer. Just what I needed.
~From my point of view, FSFS is probably the right choice for our
repository, but it's a bit young. I may just risk it. It's only the
source code to all our new products, after all. :)
Again, thanks,
tomS
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFCQdweDoEO2NB2KMURAisOAJ4lt2cjne72feeh5bSKwKAFj6y+4QCgj0g6
3EwX+tn/hP9UoyJdl8CA6tM=
=9tgC
-----END PGP SIGNATURE-----
Re: daily "Cannot allocate memory"-problem
Posted by Dirk Schenkewitz <sc...@docomolab-euro.com>.
:-)
tekHedd wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> |>>>> I'm getting a bit annoyed that the automatic answer to every BDB
> |>>>> problem is "switch to FSFS". :-)
>
> I am just switching to svn for new projects, and it seems to be working
> OK with BDB 4.1, but the projects are pretty small so far. I am feeling
> too deal with getting the latest bdb and so on, especially since the
> server is not my problem.
>
> My question is: is there a good reason that I should _not_ use FSFS?
I nearly asked this myself, but then I remembered about this "comparison
sheet" (FSFS vs. BDB): http://svn.collab.net/repos/svn/trunk/notes/fsfs
That's all I know. If anyone knows more, I would also like to know.
> I will initially be using svn+ssh using a shared account and
> certificates, but later plan to integrate more with apache...
Being a newbie myself, I'm on thin ice here, but I believe that you can
select any on these completely independent of the backend.
> Regards,
> tomS
Have fun
Dirk
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: daily "Cannot allocate memory"-problem
Posted by tekHedd <te...@byteheaven.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
|>>>> I'm getting a bit annoyed that the automatic answer to every BDB
|>>>> problem is "switch to FSFS". :-)
I am just switching to svn for new projects, and it seems to be working
OK with BDB 4.1, but the projects are pretty small so far. I am feeling
too deal with getting the latest bdb and so on, especially since the
server is not my problem.
My question is: is there a good reason that I should _not_ use FSFS?
I will initially be using svn+ssh using a shared account and
certificates, but later plan to integrate more with apache...
Regards,
tomS
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFCQKirDoEO2NB2KMURAlMsAJ4kLEMtL+ZB0lcOUxClMaHvdOYcqQCff6uj
YQAdkUTlriPTKGhFGqwbMHo=
=TEwy
-----END PGP SIGNATURE-----
Re: daily "Cannot allocate memory"-problem
Posted by Dirk Schenkewitz <sc...@docomolab-euro.com>.
Ben Collins-Sussman wrote:
>
> On Mar 20, 2005, at 3:25 PM, Erik Huelsmann wrote:
>
>> On Sun, 20 Mar 2005 21:21:43 +0000, Martin A. Brooks
>> <ma...@hinterlands.org> wrote:
>>
>>> Ben Collins-Sussman wrote:
>>>
>>>> I'm getting a bit annoyed that the automatic answer to every BDB
>>>> problem is "switch to FSFS". :-)
>>>
>>>
>>> That's easy to fix, just make it so BDB doesn't cause about 80% of
>>> the problems reported to this list :)
>>
>>
>> If you all start using fsfs, that should happen soon enough....
>>
>
> Yes, there are problems with the way Subversion utilizes BDB. Yes,
> we're working to fix the usage, and Sleepycat is working on changing
> BDB to address some of these things as well.
>
> Regardless (as has been said in the past): reading this list gives a
> skewed view of reality. If I were reading this list, I would think BDB
> is some horrible, unusable thing. My guess, though, is that 95% of
> people using BDB repositories have no problems whatsoever. The 5% who
> have problems tend to dominate the list.
Is there any statistical knowledge or anything better than a guess about
how many people use BDB and how many use FSF?
Since I'm on the list, I have not observed one single complaint against
FSFS, while lots of problems seem to arise from BDB. That is, as far as
I observed, the problems where the backend is the culprit divide into
100%(BDB):0%(FSFS). Maybe I missed something, please correct me if so.
I choose FSFS because I didn't want to install and take care of another
big software packet (and because I heard that BDB was a bit unstable when
used by other people) - now I'm glad. :-)
It looks like FSFS does a very good job - my congrats for creating it!
Dirk
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: daily "Cannot allocate memory"-problem
Posted by Ben Collins-Sussman <su...@collab.net>.
On Mar 20, 2005, at 3:25 PM, Erik Huelsmann wrote:
> On Sun, 20 Mar 2005 21:21:43 +0000, Martin A. Brooks
> <ma...@hinterlands.org> wrote:
>> Ben Collins-Sussman wrote:
>>
>>> I'm getting a bit annoyed that the automatic answer to every BDB
>>> problem is "switch to FSFS". :-)
>>
>> That's easy to fix, just make it so BDB doesn't cause about 80% of the
>> problems reported to this list :)
>
> If you all start using fsfs, that should happen soon enough....
>
Yes, there are problems with the way Subversion utilizes BDB. Yes,
we're working to fix the usage, and Sleepycat is working on changing
BDB to address some of these things as well.
Regardless (as has been said in the past): reading this list gives a
skewed view of reality. If I were reading this list, I would think BDB
is some horrible, unusable thing. My guess, though, is that 95% of
people using BDB repositories have no problems whatsoever. The 5% who
have problems tend to dominate the list.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: daily "Cannot allocate memory"-problem
Posted by Erik Huelsmann <eh...@gmail.com>.
On Sun, 20 Mar 2005 21:21:43 +0000, Martin A. Brooks
<ma...@hinterlands.org> wrote:
> Ben Collins-Sussman wrote:
>
> > I'm getting a bit annoyed that the automatic answer to every BDB
> > problem is "switch to FSFS". :-)
>
> That's easy to fix, just make it so BDB doesn't cause about 80% of the
> problems reported to this list :)
If you all start using fsfs, that should happen soon enough....
bye,
Erik.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: daily "Cannot allocate memory"-problem
Posted by "Martin A. Brooks" <ma...@hinterlands.org>.
Ben Collins-Sussman wrote:
> I'm getting a bit annoyed that the automatic answer to every BDB
> problem is "switch to FSFS". :-)
That's easy to fix, just make it so BDB doesn't cause about 80% of the
problems reported to this list :)
> Second, make sure that you've allocated enough BDB lock objects, have
> a big enough log buffer, and a big enough cache. If you were to
> 'svnadmin create' a BDB repository today using the latest /trunk svn
> code, here are the default values you'd see in DB_CONFIG. Make sure
> your own DB_CONFIG has at least these values, and after tweaking, run
> 'svnadmin recover' again to make the changes stick. See if that
> clears things up.
Or just use fsfs.
*removes troll outfit and runs away terribly, terribly fast*
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: daily "Cannot allocate memory"-problem
Posted by Ben Collins-Sussman <su...@collab.net>.
On Mar 20, 2005, at 7:51 AM, Thomas -lists wrote:
> Hey there,
>
> I searched the list archives but couldn't find this problem described
> so please point me to posts if that was done somewhere already.
>
> For now I already had to "svnadmin recover" my repository for 3 times
> (each day).
>
> It had stopped working with this error:
> "Berkeley DB error while opening 'nodes' table for filesystem
> /home/subversion/repos/db:
> Cannot allocate memory"
I'm getting a bit annoyed that the automatic answer to every BDB
problem is "switch to FSFS". :-)
First of all, upgrade to DB 4.2, it has many fewer problems than DB 4.1.
Second, make sure that you've allocated enough BDB lock objects, have a
big enough log buffer, and a big enough cache. If you were to
'svnadmin create' a BDB repository today using the latest /trunk svn
code, here are the default values you'd see in DB_CONFIG. Make sure
your own DB_CONFIG has at least these values, and after tweaking, run
'svnadmin recover' again to make the changes stick. See if that clears
things up.
### Lock subsystem
#
# Make sure you read the documentation at:
#
# http://www.sleepycat.com/docs/ref/lock/max.html
#
# before tweaking these values.
set_lk_max_locks 2000
set_lk_max_lockers 2000
set_lk_max_objects 2000
### Log file subsystem
#
# Make sure you read the documentation at:
#
# http://www.sleepycat.com/docs/api_c/env_set_lg_bsize.html
# http://www.sleepycat.com/docs/api_c/env_set_lg_max.html
# http://www.sleepycat.com/docs/ref/log/limits.html
#
# Increase the size of the in-memory log buffer from the default
# of 32 Kbytes to 256 Kbytes. Decrease the log file size from
# 10 Mbytes to 1 Mbyte. This will help reduce the amount of disk
# space required for hot backups. The size of the log file must be
# at least four times the size of the in-memory log buffer.
#
# Note: Decreasing the in-memory buffer size below 256 Kbytes
# will hurt commit performance. For details, see this post from
# Daniel Berlin <da...@dberlin.org>:
#
# http://subversion.tigris.org/servlets/ReadMsg?list=dev&msgId=161960
set_lg_bsize 262144
set_lg_max 1048576
#
# If you see "log region out of memory" errors, bump lg_regionmax.
# See http://www.sleepycat.com/docs/ref/log/config.html and
# http://svn.haxx.se/users/archive-2004-10/1001.shtml for more.
set_lg_regionmax 131072
#
# The default cache size in BDB is only 256k. As explained in
# http://svn.haxx.se/dev/archive-2004-12/0369.shtml, this is too
# small for most applications. Bump this number if "db_stat -m"
# shows too many cache misses.
set_cachesize 0 1048576 1
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: daily "Cannot allocate memory"-problem
Posted by "Martin A. Brooks" <ma...@hinterlands.org>.
Thomas -lists wrote:
> For now I already had to "svnadmin recover" my repository for 3 times
> (each day).
Convert the backend to fsfs. Problem solved.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org