You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by kf...@collab.net on 2003/08/01 15:46:57 UTC
Re: MacOS X problems---Corrupt repository
"Hamilton Link" <he...@sandia.gov> writes:
> I don't suppose there are any other ways in which the INSTALL file for
> svn is increasingly out of date? It specifically says (still) that
> 4.0.14 is the Berkeley DB version to use. Since I certainly have never
> been able to definitively say what is and isn't required to run svn,
> do people on the dev team occasionally give a quick peek at the FAQ
> and INSTALL, etc. files?
Generally, 4.0.14 is still the recommended Berkeley, but on Mac OS/X
4.1.24 or higher is required -- 4.0.14 will not work.
We'd like to recommend 4.1.x for Subversion in general, but
experiments have shown that it's slower. The problem may be more with
the way we use Berkeley than with Berkeley itself. I can't quantify
this precisely only because I don't have the threads or issues handy
right now. Some people use 4.1.x anyway; as far as I know, they don't
have correctness issues.
I'm cross-posting this to the 'users' list, where I think it would be
useful for it to be seen. If people follow up, please trim 'dev' out
of the recipients, unless there's some reason it really should go to
dev, thanks.
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: MacOS X problems---Corrupt repository
Posted by Nicholas Riley <nj...@uiuc.edu>.
On Fri, Aug 01, 2003 at 10:46:57AM -0500, kfogel@collab.net wrote:
> "Hamilton Link" <he...@sandia.gov> writes:
> > I don't suppose there are any other ways in which the INSTALL file for
> > svn is increasingly out of date? It specifically says (still) that
> > 4.0.14 is the Berkeley DB version to use. Since I certainly have never
> > been able to definitively say what is and isn't required to run svn,
> > do people on the dev team occasionally give a quick peek at the FAQ
> > and INSTALL, etc. files?
>
> Generally, 4.0.14 is still the recommended Berkeley, but on Mac OS/X
> 4.1.24 or higher is required -- 4.0.14 will not work.
There are patches to 4.0.14 which were circulated about a year ago,
and cause it to work properly on Mac OS X. This is the setup I've run
since last October without problems. I haven't tried 4.1.25 recently,
but the last time I did, it repeatedly blew up in my face - corrupting
itself with a single commit.
The patch is available here:
<http://www.contactor.se/~dast/svn/archive-2002-10/0715.shtml>
--
=Nicholas Riley <nj...@uiuc.edu> | <http://www.uiuc.edu/ph/www/njriley>
Pablo Research Group, Department of Computer Science and
Medical Scholars Program, University of Illinois at Urbana-Champaign
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BDB recovery, WAS: RE: MacOS X problems---Corrupt repository
Posted by mark benedetto king <mb...@lowlatency.com>.
On Mon, Aug 11, 2003 at 09:36:02AM -0500, Ben Collins-Sussman wrote:
> "Sander Striker" <st...@apache.org> writes:
>
> > This is why Sleepycat recommends using a single program to access
> > the db and to proxy everything through that. We do have an issue on
> > this IIRC.
>
> Yeah, deep down, this is my secret wish. Post-1.0, I'd love to see a
> single daemon access the berkeley db environment -- much like the way
> 'mysqld' works.
>
This should become much simpler with a vtable-ized fs. I suspect that
Greg Hudson's svn-protocol marshalling routines (and methodology) would
be easily re-usable in such an endeavor.
--ben
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: BDB recovery, WAS: RE: MacOS X problems---Corrupt repository
Posted by Ben Collins-Sussman <su...@collab.net>.
"Sander Striker" <st...@apache.org> writes:
> This is why Sleepycat recommends using a single program to access
> the db and to proxy everything through that. We do have an issue on
> this IIRC.
Yeah, deep down, this is my secret wish. Post-1.0, I'd love to see a
single daemon access the berkeley db environment -- much like the way
'mysqld' works.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
BDB recovery, WAS: RE: MacOS X problems---Corrupt repository
Posted by Sander Striker <st...@apache.org>.
> From: Jack Repenning [mailto:jrepenning@collab.net]
> Sent: Monday, August 04, 2003 8:48 PM
> At 8:39 PM +0200 8/4/03, Sander Striker wrote:
>>
>> What we really need is a mandatory --force flag to svnadmin recover
>> and a "WARNING: ... Are you sure [yes/no]?" when --force isn't passed.
>
> Are you really saying there's nothing better that can be done? BDB
> doesn't record the PID of the locking processes, for example?
The philosophy behind recover is that all processes have died that
were accessing the db, and assuming that the locks that are there are
stale.
This is why Sleepycat recommends using a single program to access
the db and to proxy everything through that. We do have an issue on
this IIRC.
Sander
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: MacOS X problems---Corrupt repository
Posted by Jack Repenning <jr...@collab.net>.
At 8:39 PM +0200 8/4/03, Sander Striker wrote:
>
>What we really need is a mandatory --force flag to svnadmin recover
>and a "WARNING: ... Are you sure [yes/no]?" when --force isn't passed.
Are you really saying there's nothing better that can be done? BDB
doesn't record the PID of the locking processes, for example?
--
-==-
Jack Repenning
CollabNet, Inc.
8000 Marina Boulevard, Suite 600
Brisbane, California 94005
o: 650.228.2562
c: 408.835-8090
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: MacOS X problems---Corrupt repository
Posted by Sander Striker <st...@apache.org>.
> From: Jack Repenning [mailto:jrepenning@collab.net]
> Sent: Monday, August 04, 2003 8:12 PM
> (Moved from users to dev)
>
> At 7:56 PM -0400 8/3/03, Garrett Rooney wrote:
> >It's not a simple problem, because svnadmin recover needs to be able
> >to handle the case where the client has crashed while holding the
> >lock, so at least some of the time it needs to forcefully break the
> >lock. This is a bad thing (tm) if the other process actually is
> >still running. The current code doesn't handle this especially well
> >(it just breaks the lock all the time, rather than trying to acquire
> >it normally), so we should definately be doing something else, the
> >question is what?
>
> The danger of corrupting a repo during "recovery" is a really huge
> gotcha! Do we have an issue open to cover this? I can't spot one.
I updated the FAQ. That should help a bit.
What we really need is a mandatory --force flag to svnadmin recover
and a "WARNING: ... Are you sure [yes/no]?" when --force isn't passed.
Sander
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: MacOS X problems---Corrupt repository
Posted by Jack Repenning <jr...@collab.net>.
(Moved from users to dev)
At 7:56 PM -0400 8/3/03, Garrett Rooney wrote:
>It's not a simple problem, because svnadmin recover needs to be able
>to handle the case where the client has crashed while holding the
>lock, so at least some of the time it needs to forcefully break the
>lock. This is a bad thing (tm) if the other process actually is
>still running. The current code doesn't handle this especially well
>(it just breaks the lock all the time, rather than trying to acquire
>it normally), so we should definately be doing something else, the
>question is what?
The danger of corrupting a repo during "recovery" is a really huge
gotcha! Do we have an issue open to cover this? I can't spot one.
--
-==-
Jack Repenning
CollabNet, Inc.
8000 Marina Boulevard, Suite 600
Brisbane, California 94005
o: 650.228.2562
c: 408.835-8090
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: MacOS X problems---Corrupt repository
Posted by Garrett Rooney <ro...@electricjellyfish.net>.
Robert Spier wrote:
>>That's correct. It is positively critical that you never run
>>'svnadmin recover' on a repository that is in use by any processes.
>
>
> Isn't that why it gets an exclusive lock?
>
> $ svnadmin recover /tmp/t
> Acquiring exclusive lock on repository db.
>
> Or is that not going to protect you from shooting yourself in the
> foot?[1]
It's not a simple problem, because svnadmin recover needs to be able to
handle the case where the client has crashed while holding the lock, so
at least some of the time it needs to forcefully break the lock. This
is a bad thing (tm) if the other process actually is still running. The
current code doesn't handle this especially well (it just breaks the
lock all the time, rather than trying to acquire it normally), so we
should definately be doing something else, the question is what?
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: MacOS X problems---Corrupt repository
Posted by Robert Spier <rs...@pobox.com>.
> That's correct. It is positively critical that you never run
> 'svnadmin recover' on a repository that is in use by any processes.
Isn't that why it gets an exclusive lock?
$ svnadmin recover /tmp/t
Acquiring exclusive lock on repository db.
Or is that not going to protect you from shooting yourself in the
foot?[1]
-R
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: MacOS X problems---Corrupt repository
Posted by Michael Wood <mw...@its.uct.ac.za>.
On Sun, Aug 03, 2003 at 12:47:17AM -0500, cmpilato@collab.net wrote:
> "Sander Striker" <st...@apache.org> writes:
>
> > > My recollection is that Sander Striker was having discussions with
> > > sleepycat folk, and that some deep issue was revealed. In almost all
> > > of these cases, a berkeley db table had switched 'type' for some
> > > reason; the sleepycat people revealed that this might happen if
> > > recovery is happening whilst something else changes the db. (Is this
> > > accurate, Sander?)
> >
> > Let me dig up the mail. Note that a big chunk of communication was
> > done by cmpilato BTW.
> >
> > What it basically comes down to is that a recover will recreate the
> > environment. Which means that if you modify the db while recovering
> > you get an inconsistent environment (wrong allocation counts etc).
>
> That's correct. It is positively critical that you never run
> 'svnadmin recover' on a repository that is in use by any processes.
Perhaps the FAQ should be updated to recommend running hotbackup.py
before recovery?
--
Michael Wood <mw...@its.uct.ac.za>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: MacOS X problems---Corrupt repository
Posted by Michael Wood <mw...@its.uct.ac.za>.
On Sun, Aug 03, 2003 at 12:47:17AM -0500, cmpilato@collab.net wrote:
> "Sander Striker" <st...@apache.org> writes:
>
> > > My recollection is that Sander Striker was having discussions with
> > > sleepycat folk, and that some deep issue was revealed. In almost all
> > > of these cases, a berkeley db table had switched 'type' for some
> > > reason; the sleepycat people revealed that this might happen if
> > > recovery is happening whilst something else changes the db. (Is this
> > > accurate, Sander?)
> >
> > Let me dig up the mail. Note that a big chunk of communication was
> > done by cmpilato BTW.
> >
> > What it basically comes down to is that a recover will recreate the
> > environment. Which means that if you modify the db while recovering
> > you get an inconsistent environment (wrong allocation counts etc).
>
> That's correct. It is positively critical that you never run
> 'svnadmin recover' on a repository that is in use by any processes.
Perhaps the FAQ should be updated to recommend running hotbackup.py
before recovery?
--
Michael Wood <mw...@its.uct.ac.za>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: MacOS X problems---Corrupt repository
Posted by cm...@collab.net.
"Sander Striker" <st...@apache.org> writes:
> > My recollection is that Sander Striker was having discussions with
> > sleepycat folk, and that some deep issue was revealed. In almost all
> > of these cases, a berkeley db table had switched 'type' for some
> > reason; the sleepycat people revealed that this might happen if
> > recovery is happening whilst something else changes the db. (Is this
> > accurate, Sander?)
>
> Let me dig up the mail. Note that a big chunk of communication was
> done by cmpilato BTW.
>
> What it basically comes down to is that a recover will recreate the
> environment. Which means that if you modify the db while recovering
> you get an inconsistent environment (wrong allocation counts etc).
That's correct. It is positively critical that you never run
'svnadmin recover' on a repository that is in use by any processes.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: MacOS X problems---Corrupt repository
Posted by cm...@collab.net.
"Sander Striker" <st...@apache.org> writes:
> > My recollection is that Sander Striker was having discussions with
> > sleepycat folk, and that some deep issue was revealed. In almost all
> > of these cases, a berkeley db table had switched 'type' for some
> > reason; the sleepycat people revealed that this might happen if
> > recovery is happening whilst something else changes the db. (Is this
> > accurate, Sander?)
>
> Let me dig up the mail. Note that a big chunk of communication was
> done by cmpilato BTW.
>
> What it basically comes down to is that a recover will recreate the
> environment. Which means that if you modify the db while recovering
> you get an inconsistent environment (wrong allocation counts etc).
That's correct. It is positively critical that you never run
'svnadmin recover' on a repository that is in use by any processes.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: MacOS X problems---Corrupt repository
Posted by Sander Striker <st...@apache.org>.
> From: sussman@collab.net [mailto:sussman@collab.net]
> Sent: Friday, August 01, 2003 6:58 PM
> Blair Zajac <bl...@orcaware.com> writes:
>
> > To put this back on the radar screen, there has been a number of issues
> > with people using 4.1.25 on Linux/FreeBSD boxes that have had repositories
> > that end up being corrupted where the only way to recover is to restore
> > a backup copy.
>
> My recollection is that Sander Striker was having discussions with
> sleepycat folk, and that some deep issue was revealed. In almost all
> of these cases, a berkeley db table had switched 'type' for some
> reason; the sleepycat people revealed that this might happen if
> recovery is happening whilst something else changes the db. (Is this
> accurate, Sander?)
Let me dig up the mail. Note that a big chunk of communication was done by
cmpilato BTW.
What it basically comes down to is that a recover will recreate the environment.
Which means that if you modify the db while recovering you get an inconsistent
environment (wrong allocation counts etc).
Sander
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
RE: MacOS X problems---Corrupt repository
Posted by Sander Striker <st...@apache.org>.
> From: sussman@collab.net [mailto:sussman@collab.net]
> Sent: Friday, August 01, 2003 6:58 PM
> Blair Zajac <bl...@orcaware.com> writes:
>
> > To put this back on the radar screen, there has been a number of issues
> > with people using 4.1.25 on Linux/FreeBSD boxes that have had repositories
> > that end up being corrupted where the only way to recover is to restore
> > a backup copy.
>
> My recollection is that Sander Striker was having discussions with
> sleepycat folk, and that some deep issue was revealed. In almost all
> of these cases, a berkeley db table had switched 'type' for some
> reason; the sleepycat people revealed that this might happen if
> recovery is happening whilst something else changes the db. (Is this
> accurate, Sander?)
Let me dig up the mail. Note that a big chunk of communication was done by
cmpilato BTW.
What it basically comes down to is that a recover will recreate the environment.
Which means that if you modify the db while recovering you get an inconsistent
environment (wrong allocation counts etc).
Sander
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: MacOS X problems---Corrupt repository
Posted by Ben Collins-Sussman <su...@collab.net>.
Blair Zajac <bl...@orcaware.com> writes:
> To put this back on the radar screen, there has been a number of issues
> with people using 4.1.25 on Linux/FreeBSD boxes that have had repositories
> that end up being corrupted where the only way to recover is to restore
> a backup copy.
My recollection is that Sander Striker was having discussions with
sleepycat folk, and that some deep issue was revealed. In almost all
of these cases, a berkeley db table had switched 'type' for some
reason; the sleepycat people revealed that this might happen if
recovery is happening whilst something else changes the db. (Is this
accurate, Sander?)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: MacOS X problems---Corrupt repository
Posted by Ben Collins-Sussman <su...@collab.net>.
Blair Zajac <bl...@orcaware.com> writes:
> To put this back on the radar screen, there has been a number of issues
> with people using 4.1.25 on Linux/FreeBSD boxes that have had repositories
> that end up being corrupted where the only way to recover is to restore
> a backup copy.
My recollection is that Sander Striker was having discussions with
sleepycat folk, and that some deep issue was revealed. In almost all
of these cases, a berkeley db table had switched 'type' for some
reason; the sleepycat people revealed that this might happen if
recovery is happening whilst something else changes the db. (Is this
accurate, Sander?)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: MacOS X problems---Corrupt repository
Posted by Blair Zajac <bl...@orcaware.com>.
kfogel@collab.net wrote:
>
> "Hamilton Link" <he...@sandia.gov> writes:
> > I don't suppose there are any other ways in which the INSTALL file for
> > svn is increasingly out of date? It specifically says (still) that
> > 4.0.14 is the Berkeley DB version to use. Since I certainly have never
> > been able to definitively say what is and isn't required to run svn,
> > do people on the dev team occasionally give a quick peek at the FAQ
> > and INSTALL, etc. files?
>
> Generally, 4.0.14 is still the recommended Berkeley, but on Mac OS/X
> 4.1.24 or higher is required -- 4.0.14 will not work.
>
> We'd like to recommend 4.1.x for Subversion in general, but
> experiments have shown that it's slower. The problem may be more with
> the way we use Berkeley than with Berkeley itself. I can't quantify
> this precisely only because I don't have the threads or issues handy
> right now. Some people use 4.1.x anyway; as far as I know, they don't
> have correctness issues.
To put this back on the radar screen, there has been a number of issues
with people using 4.1.25 on Linux/FreeBSD boxes that have had repositories
that end up being corrupted where the only way to recover is to restore
a backup copy.
I'm hoping svn.collab.net will be moved again to 4.1.25 at some point to
get the experts on it.
See
http://www.contactor.se/~dast/svn/archive-2003-04/0734.shtml
Best,
Blair
--
Blair Zajac <bl...@orcaware.com>
Plots of your system's performance - http://www.orcaware.com/orca/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: MacOS X problems---Corrupt repository
Posted by Blair Zajac <bl...@orcaware.com>.
kfogel@collab.net wrote:
>
> "Hamilton Link" <he...@sandia.gov> writes:
> > I don't suppose there are any other ways in which the INSTALL file for
> > svn is increasingly out of date? It specifically says (still) that
> > 4.0.14 is the Berkeley DB version to use. Since I certainly have never
> > been able to definitively say what is and isn't required to run svn,
> > do people on the dev team occasionally give a quick peek at the FAQ
> > and INSTALL, etc. files?
>
> Generally, 4.0.14 is still the recommended Berkeley, but on Mac OS/X
> 4.1.24 or higher is required -- 4.0.14 will not work.
>
> We'd like to recommend 4.1.x for Subversion in general, but
> experiments have shown that it's slower. The problem may be more with
> the way we use Berkeley than with Berkeley itself. I can't quantify
> this precisely only because I don't have the threads or issues handy
> right now. Some people use 4.1.x anyway; as far as I know, they don't
> have correctness issues.
To put this back on the radar screen, there has been a number of issues
with people using 4.1.25 on Linux/FreeBSD boxes that have had repositories
that end up being corrupted where the only way to recover is to restore
a backup copy.
I'm hoping svn.collab.net will be moved again to 4.1.25 at some point to
get the experts on it.
See
http://www.contactor.se/~dast/svn/archive-2003-04/0734.shtml
Best,
Blair
--
Blair Zajac <bl...@orcaware.com>
Plots of your system's performance - http://www.orcaware.com/orca/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org