You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by John Peacock <jp...@rowman.com> on 2003/12/17 12:03:19 UTC
Dead repository with 0.35.0-to-be
I have a scheduled update running every night that updates my APC (Archive of
Perl Changes) repository with any patches applied. I just recreated it last
weekend (using 0.34.0) and the autoupdates have run fine for several days (using
0.35.0-to-be).
Last night, however, I was also dumping an old copy of the APC (from 0.27.0
days) into a new repository (on a different drive) when the auto-update started.
I see lines these lines in the log:
> 2003-12-17 00:04:35.946815500 svn: Unable to open an ra_local session to URL
> 2003-12-17 00:04:35.946878500 svn: Unable to open repository 'file:///var/svn-repository/perl5/branches/maint-5.6'
> 2003-12-17 00:04:35.946909500 svn: Berkeley DB error while opening 'changes' table for filesystem /var/svn-repository/perl5/db:
> 2003-12-17 00:04:35.946912500 Cannot allocate memory
and now the repository itself is totally dead:
> $ svnadmin recover /var/svn-repository/perl5
> Please wait; recovering the repository may take some time...
> svn: DB_RUNRECOVERY: Fatal error, run database recovery
Even running Berkeley's db_recover yields little more than several screens of this:
> db_recover: PANIC: fatal region error detected; run recovery
> db_recover: PANIC: fatal region error detected; run recovery
> ...
> db_recover: PANIC: fatal region error detected; run recovery
> db_recover: DB_ENV->open: DB_RUNRECOVERY: Fatal error, run database recovery
So it is completely hosed.
-rw-rw---- 1 jpeacock svn 11M Dec 17 07:01 changes
-rw-rw---- 1 jpeacock svn 24K Dec 17 07:01 copies
-rw-rw---- 1 jpeacock svn 1.5K Dec 14 09:01 DB_CONFIG
-rw-rw---- 1 jpeacock svn 1.5K Dec 14 09:00 DB_CONFIG~
-rw-rw-r-- 1 jpeacock jpeacock 1023K Dec 16 01:29 log.0000009750
-rw-rw-r-- 1 jpeacock jpeacock 503K Dec 17 01:00 log.0000009751
-rw-rw---- 1 jpeacock svn 19M Dec 17 07:01 nodes
-rw-rw---- 1 jpeacock svn 20M Dec 17 07:01 representations
-rw-rw---- 1 jpeacock svn 400K Dec 17 07:01 revisions
-rw-rw---- 1 jpeacock svn 4.2G Dec 17 07:01 strings
-rw-rw---- 1 jpeacock svn 18M Dec 17 07:01 transactions
-rw-rw---- 1 jpeacock svn 8.0K Dec 17 07:01 uuids
Should I wipe it out and restart, or does anyone want to take a crack at fixing it?
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4720 Boston Way
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5747
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Dead repository with 0.35.0-to-be
Posted by kf...@collab.net.
Ben Collins-Sussman <su...@collab.net> writes:
> mod_dav_svn doesn't fork; IIRC, Karl's solution (last week) was to do
> the deltification as part of the pool-cleanup operation for the whole
> request. httpd might kill off an idle server process, but not one
> that's still "cleaning up" after a request, right?
Indeed, the pool clean up is The Documented Way to do stuff
server-side after the response has been sent to the client.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Dead repository with 0.35.0-to-be
Posted by Ben Collins-Sussman <su...@collab.net>.
On Wed, 2003-12-17 at 08:51, Branko Čibej wrote:
> Is that so? I remember us talking about how this is *not* the right way
> to to do things in mod_dav_svn, because you cna't predict when httpd
> will kill off a server process.
mod_dav_svn doesn't fork; IIRC, Karl's solution (last week) was to do
the deltification as part of the pool-cleanup operation for the whole
request. httpd might kill off an idle server process, but not one
that's still "cleaning up" after a request, right?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Dead repository with 0.35.0-to-be
Posted by Branko Čibej <br...@xbc.nu>.
Tobias Ringstrom wrote:
> Branko Čibej wrote:
>
>> That's with asynchronous deltification turned on, right? Someone please
>> tell me that we're closing the FS environment before forking for
>> deltification, and that the forked process daemonizes itself before
>> opening the environment again.
>
>
> We don't fork. The deltafication for ra_svn and ra_dav are performed
> by the same thread/process that handles the request, but after the
> connection to the client has been closed.
Is that so? I remember us talking about how this is *not* the right way
to to do things in mod_dav_svn, because you cna't predict when httpd
will kill off a server process.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Dead repository with 0.35.0-to-be
Posted by Tobias Ringstrom <to...@ringstrom.mine.nu>.
Branko Čibej wrote:
> That's with asynchronous deltification turned on, right? Someone please
> tell me that we're closing the FS environment before forking for
> deltification, and that the forked process daemonizes itself before
> opening the environment again.
We don't fork. The deltafication for ra_svn and ra_dav are performed by
the same thread/process that handles the request, but after the
connection to the client has been closed.
/Tobias
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Dead repository with 0.35.0-to-be
Posted by John Peacock <jp...@rowman.com>.
John Peacock wrote:
> I should point out that it doesn't appear (from the log file at least)
> that any transactions got processed at that point (i.e. it was just
> starting up).
No one have any ideas how my repository got so badly borked? I'm going to
rebuild this evening, I suppose.
> Another thing to note is that I am running BerkeleyDB.4.2.52...
>
I should also have pointed out that I was using ra_local (for performance
reasons)...
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4720 Boston Way
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5747
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Dead repository with 0.35.0-to-be
Posted by John Peacock <jp...@rowman.com>.
Branko Čibej wrote:
> That's with asynchronous deltification turned on, right? Someone please
> tell me that we're closing the FS environment before forking for
> deltification, and that the forked process daemonizes itself before
> opening the environment again.
I should point out that it doesn't appear (from the log file at least) that any
transactions got processed at that point (i.e. it was just starting up).
I should say that my machine has 512MB installed, but was probably down to less
than 100MB (or lower) when the process started. Subversion should not behave
quite so badly with low memory situations (unless it was Berkeley that freaked).
Another thing to note is that I am running BerkeleyDB.4.2.52...
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4720 Boston Way
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5747
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Dead repository with 0.35.0-to-be
Posted by Branko Čibej <br...@xbc.nu>.
John Peacock wrote:
> I have a scheduled update running every night that updates my APC
> (Archive of Perl Changes) repository with any patches applied. I just
> recreated it last weekend (using 0.34.0) and the autoupdates have run
> fine for several days (using 0.35.0-to-be).
>
> Last night, however, I was also dumping an old copy of the APC (from
> 0.27.0 days) into a new repository (on a different drive) when the
> auto-update started. I see lines these lines in the log:
>
>> 2003-12-17 00:04:35.946815500 svn: Unable to open an ra_local session
>> to URL
>> 2003-12-17 00:04:35.946878500 svn: Unable to open repository
>> 'file:///var/svn-repository/perl5/branches/maint-5.6'
>> 2003-12-17 00:04:35.946909500 svn: Berkeley DB error while opening
>> 'changes' table for filesystem /var/svn-repository/perl5/db:
>> 2003-12-17 00:04:35.946912500 Cannot allocate memory
>
>
> and now the repository itself is totally dead:
>
>> $ svnadmin recover /var/svn-repository/perl5
>> Please wait; recovering the repository may take some time...
>> svn: DB_RUNRECOVERY: Fatal error, run database recovery
>
>
> Even running Berkeley's db_recover yields little more than several
> screens of this:
>
>> db_recover: PANIC: fatal region error detected; run recovery
>> db_recover: PANIC: fatal region error detected; run recovery
>> ...
>> db_recover: PANIC: fatal region error detected; run recovery
>> db_recover: DB_ENV->open: DB_RUNRECOVERY: Fatal error, run database
>> recovery
>
>
> So it is completely hosed.
That's with asynchronous deltification turned on, right? Someone please
tell me that we're closing the FS environment before forking for
deltification, and that the forked process daemonizes itself before
opening the environment again.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org