You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Nicolás Lichtmaier <ni...@panoptico.reloco.com.ar> on 2005/02/07 22:31:53 UTC
Berkeley DB problem: "Could not allocate memory"
I'm having trouble with subversion. Some large commits fail with:
[Mon Feb 07 19:00:06 2005] [error] [client 10.53.3.67] (20014)Error
string not specified yet: Berkeley
DB error while opening 'uuids' table for filesystem
/var/lib/svn/pv/db:\nCannot allocate memory
[Mon Feb 07 19:00:06 2005] [error] [client 10.53.3.67] Could not fetch
resource information. [500, #0]
[Mon Feb 07 19:00:06 2005] [error] [client 10.53.3.67] Could not open
the requested SVN filesystem [50
0, #160029]
[Mon Feb 07 19:00:06 2005] [error] [client 10.53.3.67] Could not open
the requested SVN filesystem [50
0, #160029]
Is there anything I can do to help debug this problem. Should I rebuild
subversion with --enable-maintainer-mode ? Is that option ok for a
production server?
I've tried increading the "cache", as was said in a previous thread in
this mailing list, by adding "set_cachesize 0 8388608 1" to db/DB_CONFIG
and running "svnadmin recover". According to the users the problem only
got worse.
This is fairly big repository:
-rw-r--r-- 1 apache root 18M feb 7 19:19 representations
-rw-r--r-- 1 apache root 27M feb 7 19:19 nodes
-rw-r--r-- 1 apache root 30M feb 7 19:19 changes
-rw-r--r-- 1 apache root 5.3G feb 7 19:19 strings
I'm using versión 1.1.3 (r12730). The system is RedHat AS 3.
Thanks for any help you can give me.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Berkeley DB problem: "Could not allocate memory"
Posted by Nicolás Lichtmaier <ni...@panoptico.reloco.com.ar>.
>>184 Number of current locks.
>>380 Number of current lockers.
>>7 Number of current lock objects.
>>
>>
>Your repository should never have an lock accumulation when there are
>no attached processes. If you did this db_stat during such a time,
>then the fact that you have lock accumulation at all means that some
>processes didn't cleanly detach from the Berkeley DB environment.
>When this occurs, you can start seeing "Cannot allocate memory" errors
>well before the reported lock accumulation nears the maximums.
>
>
This repository is used only from an Apache httpd installation through
mod_dav_svn, and I see no traces of Apache coredumping or something
similar...
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Berkeley DB problem: "Could not allocate memory"
Posted by "C. Michael Pilato" <cm...@collab.net>.
Nicolás Lichtmaier <ni...@reloco.com.ar> writes:
> The problem is happening right now. I've ran "db_stat -c -h" and I see
> the maximums "at any one time" haven't reached the maximuns. Doesn't
> that eliminates the posibility of having ran out of locks?
[...]
> 184 Number of current locks.
> 380 Number of current lockers.
> 7 Number of current lock objects.
Your repository should never have an lock accumulation when there are
no attached processes. If you did this db_stat during such a time,
then the fact that you have lock accumulation at all means that some
processes didn't cleanly detach from the Berkeley DB environment.
When this occurs, you can start seeing "Cannot allocate memory" errors
well before the reported lock accumulation nears the maximums.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Berkeley DB problem: "Could not allocate memory"
Posted by Nicolás Lichtmaier <ni...@panoptico.reloco.com.ar>.
> I though that the "cannot allocate memory" problems were often fixed
> by increasing not the db cachesize, but the log buffer size and
> log_regionmax. Isn't that what our chats with Keith Bostic mentioned?
>
> I'm talking about the variables
>
> set_lg_bsize 262144
> set_lg_max 1048576
> set_lg_regionmax 131072
>
> If you 'svnadmin create' a repository, look at the docs surrounding
> those variables.
I've diffed the current DB_CONFIG with one from a newly created
repository and the only new parameter seems to be "set_lg_regionmax".
The comments surrounding it talk about "log region out of memory" erros,
which I am not seeing. Anyway, I'll put this "set_lg_regionmax 131072"
as stated in the file and see what happens.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Berkeley DB problem: "Could not allocate memory"
Posted by Ben Collins-Sussman <su...@collab.net>.
>>>
>>>
>>> I've tried increading the "cache", as was said in a previous thread
>>> in
>>> this mailing list, by adding "set_cachesize 0 8388608 1" to
>>> db/DB_CONFIG
>>> and running "svnadmin recover". According to the users the problem
>>> only
>>> got worse.
>>>
I though that the "cannot allocate memory" problems were often fixed by
increasing not the db cachesize, but the log buffer size and
log_regionmax. Isn't that what our chats with Keith Bostic mentioned?
I'm talking about the variables
set_lg_bsize 262144
set_lg_max 1048576
set_lg_regionmax 131072
If you 'svnadmin create' a repository, look at the docs surrounding
those variables.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Berkeley DB problem: "Could not allocate memory"
Posted by Nicolás Lichtmaier <ni...@panoptico.reloco.com.ar>.
>> I'm having trouble with subversion. Some large commits fail with:
>>
>> [Mon Feb 07 19:00:06 2005] [error] [client 10.53.3.67] (20014)Error
>> string not specified yet: Berkeley
>> DB error while opening 'uuids' table for filesystem
>> /var/lib/svn/pv/db:\nCannot allocate memory
>> [Mon Feb 07 19:00:06 2005] [error] [client 10.53.3.67] Could not fetch
>> resource information. [500, #0]
>> [Mon Feb 07 19:00:06 2005] [error] [client 10.53.3.67] Could not open
>> the requested SVN filesystem [50
>> 0, #160029]
>> [Mon Feb 07 19:00:06 2005] [error] [client 10.53.3.67] Could not open
>> the requested SVN filesystem [50
>> 0, #160029]
>>
>> Is there anything I can do to help debug this problem. Should I rebuild
>> subversion with --enable-maintainer-mode ? Is that option ok for a
>> production server?
>>
>> I've tried increading the "cache", as was said in a previous thread in
>> this mailing list, by adding "set_cachesize 0 8388608 1" to db/DB_CONFIG
>> and running "svnadmin recover". According to the users the problem only
>> got worse.
>>
>> This is fairly big repository:
>>
>> -rw-r--r-- 1 apache root 18M feb 7 19:19 representations
>> -rw-r--r-- 1 apache root 27M feb 7 19:19 nodes
>> -rw-r--r-- 1 apache root 30M feb 7 19:19 changes
>> -rw-r--r-- 1 apache root 5.3G feb 7 19:19 strings
>>
>> I'm using versión 1.1.3 (r12730). The system is RedHat AS 3.
>>
>> Thanks for any help you can give me.
>
>
> It's more likely running out of available locks (yes, horrible error
> msg).
>
> Run "db_stat -c -h /path/to/repos/db" to check if there are far more
> current locks than there ought to be (zero when the repos is
> quiescent), and shutdown access and recover if so.
The problem is happening right now. I've ran "db_stat -c -h" and I see
the maximums "at any one time" haven't reached the maximuns. Doesn't
that eliminates the posibility of having ran out of locks?
# db_stat -c -h .
152018 Last allocated locker ID.
2147M Current maximum unused locker ID.
9 Number of lock modes.
2000 Maximum number of locks possible.
2000 Maximum number of lockers possible.
2000 Maximum number of lock objects possible.
184 Number of current locks.
719 Maximum number of locks at any one time.
380 Number of current lockers.
1384 Maximum number of lockers at any one time.
7 Number of current lock objects.
39 Maximum number of lock objects at any one time.
3405795 Total number of locks requested.
3405611 Total number of locks released.
0 Total number of lock requests failing because DB_LOCK_NOWAIT was
set.
3 Total number of locks not immediately available due to conflicts.
0 Number of deadlocks.
0 Lock timeout value.
0 Number of locks that have timed out.
0 Transaction timeout value.
0 Number of transactions that have timed out.
872KB The size of the lock region..
3901 The number of region locks granted after waiting.
4578236 The number of region locks granted without waiting.
In any case... what should be done to improve Subversion's error
reporting? Is there anything one can do to help? Subversion is nice when
it works, but when it doesn't it's imposible to fix it (i.e. to support
it). Every error in subversion is "misterious" and users are often
presented with very scary failure messages...
Thanks!
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Berkeley DB problem: "Could not allocate memory"
Posted by Max Bowsher <ma...@ukf.net>.
Nicolás Lichtmaier wrote:
> I'm having trouble with subversion. Some large commits fail with:
>
> [Mon Feb 07 19:00:06 2005] [error] [client 10.53.3.67] (20014)Error
> string not specified yet: Berkeley
> DB error while opening 'uuids' table for filesystem
> /var/lib/svn/pv/db:\nCannot allocate memory
> [Mon Feb 07 19:00:06 2005] [error] [client 10.53.3.67] Could not fetch
> resource information. [500, #0]
> [Mon Feb 07 19:00:06 2005] [error] [client 10.53.3.67] Could not open
> the requested SVN filesystem [50
> 0, #160029]
> [Mon Feb 07 19:00:06 2005] [error] [client 10.53.3.67] Could not open
> the requested SVN filesystem [50
> 0, #160029]
>
> Is there anything I can do to help debug this problem. Should I rebuild
> subversion with --enable-maintainer-mode ? Is that option ok for a
> production server?
>
> I've tried increading the "cache", as was said in a previous thread in
> this mailing list, by adding "set_cachesize 0 8388608 1" to db/DB_CONFIG
> and running "svnadmin recover". According to the users the problem only
> got worse.
>
> This is fairly big repository:
>
> -rw-r--r-- 1 apache root 18M feb 7 19:19 representations
> -rw-r--r-- 1 apache root 27M feb 7 19:19 nodes
> -rw-r--r-- 1 apache root 30M feb 7 19:19 changes
> -rw-r--r-- 1 apache root 5.3G feb 7 19:19 strings
>
> I'm using versión 1.1.3 (r12730). The system is RedHat AS 3.
>
> Thanks for any help you can give me.
It's more likely running out of available locks (yes, horrible error msg).
Run "db_stat -c -h /path/to/repos/db" to check if there are far more current
locks than there ought to be (zero when the repos is quiescent), and
shutdown access and recover if so.
Max.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org