Hot to restore repo from hot-backup.py

[Eric Gillespie <ep...@pretzelnet.org>]

I just went to restore my repo from the backup repo-137.  I copied
it over to where mod_dav_svn looks, and did a fresh checkout.  Oh
crap!  I got revision 136!  Now i'm panicking because it looks like
hot-backup.py backs up one revision behind and i've now lost a
revision :(.

Bill Tutt told me not to worry, my revision is in the backup, i
just need to replay my logs.  No one knew how to do that, however.
I poked around in /usr/local/BerkeleyDB.4.0 (which has no man pages)
and found some HTML docs.  db_recover looked promising, so i gave
it a whirl.  >phew< that did it.

So you may want to document in the handbook that to restore from
one of these hot backups you need to copy it into place and then
run:

/usr/local/BerkeleyDB.4.0/bin/db_recover -ve -h ${repo}/db

-- 
Eric Gillespie, Jr. <*> epg@pretzelnet.org

Build a fire for a man, and he'll be warm for a day.  Set a man on
fire, and he'll be warm for the rest of his life. -Terry Pratchett

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Hot to restore repo from hot-backup.py

[Philip Martin <ph...@codematters.co.uk>]

Eric Gillespie <ep...@pretzelnet.org> writes:

> Well, assuming, this doesn't turn out to be a bug, and you really
> do need to run db_recover on the backups, you need to explicitly
> list the steps to restore from the hot-backup.

It's documented at

http://www.sleepycat.com/docs/ref/transapp/archival.html
http://www.sleepycat.com/docs/ref/transapp/recovery.html

Running db_recover is part of the procedure.

-- 
Philip Martin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Hot to restore repo from hot-backup.py

[Ben Collins-Sussman <su...@collab.net>]

Eric Gillespie <ep...@pretzelnet.org> writes:

> > This is what puzzles me.  There's no such thing as "how to restore" a
> > backup;  you just start using it.  No special knowledge required.  Or
> > so I *thought*
> 
> Um, it doesn't look like you thought that.  You just told me i had
> to "copy out" logfiles.  

Hold on, we're crossing streams here -- I'm describing the procedure
for making a hot backup, not the procedure for restoring a backup.  I
never thought there *was* a procedure for restoring.  :-)

Read the hot-backup.py script.  It's super-simple and commented; it's
following Berkeley DB's exact instructions on how to safely back up a
DB environment while the environment is (possibly) being used.


> In any case, grab one of these hot backup directories and checkout
> from it.  You'll see what i'm talking about.

No dice, actually.  svn.collab.net is full of hot backup directories.
Many, many times I've copied one to my personal box to play with;  and
its HEAD has always been the one I expected.  So I'm wondering why
your hot backup required some catching-up.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Hot to restore repo from hot-backup.py

[Eric Gillespie <ep...@pretzelnet.org>]

Ben Collins-Sussman <su...@collab.net> writes:

> No, it just documents your example of 'db_recover' as a way of making
> Berkeley DB replay transactions;  but the Handbook says that this is
> only necessary when the repository locks up... because, well, that's
> the only time we've ever had to use it!

Well, assuming, this doesn't turn out to be a bug, and you really
do need to run db_recover on the backups, you need to explicitly
list the steps to restore from the hot-backup.

> It means run 'db_archive -l', examine the list of logfiles it prints
> out, and then copy them out in that order.

What do you mean "copy them out"?  Copy them to where?  It's possible
my running db_recover just did this for me.

> This is what puzzles me.  There's no such thing as "how to restore" a
> backup;  you just start using it.  No special knowledge required.  Or
> so I *thought*

Um, it doesn't look like you thought that.  You just told me i had
to "copy out" logfiles.  In any case, grab one of these hot backup
directories and checkout from it.  You'll see what i'm talking
about.

--  
Eric Gillespie <*> epg@pretzelnet.org

Build a fire for a man, and he'll be warm for a day.  Set a man on
fire, and he'll be warm for the rest of his life. -Terry Pratchett

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

RE: Re: Hot to restore repo from hot-backup.py

[Bill Tutt <ra...@lyra.org>]

This is usually the case with most ACID stores. Just because a
transaction's commit() has returned successfully just means that the
data has been inserted into the transaction log. It doesn't guarantee
that the data has been inserted into the main database file yet.

Bill
----
Do you want a dangerous fugitive staying in your flat?
No.
Well, don't upset him and he'll be a nice fugitive staying in your flat.
 

> -----Original Message-----
> From: mark benedetto king [mailto:bking@inquira.com]
> Sent: Wednesday, July 17, 2002 4:40 PM
> To: Ben Collins-Sussman
> Cc: Eric Gillespie; dev@subversion.tigris.org
> Subject: Re: Hot to restore repo from hot-backup.py
> 
> On Wed, Jul 17, 2002 at 05:36:43PM -0500, Ben Collins-Sussman wrote:
> > But it seems that your backup had unplayed transactions in it, which
I
> > don't get.  Can someone more knowledgable explain?  Perhaps
> > Mr. Bostic?  :-)
> >
> 
> Well, I'm not nearly as authoritative as Mr. Bostic but wouldn't it
> make sense that a transaction can be considered complete as soon as
> the log is stabilized and the in-memory datastructures are updated
> (but before then on-disk database is updated)?
> 
> --ben
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Hot to restore repo from hot-backup.py

[mark benedetto king <bk...@inquira.com>]

On Wed, Jul 17, 2002 at 05:36:43PM -0500, Ben Collins-Sussman wrote:
> But it seems that your backup had unplayed transactions in it, which I
> don't get.  Can someone more knowledgable explain?  Perhaps
> Mr. Bostic?  :-)
> 

Well, I'm not nearly as authoritative as Mr. Bostic but wouldn't it
make sense that a transaction can be considered complete as soon as
the log is stabilized and the in-memory datastructures are updated
(but before then on-disk database is updated)?

--ben


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Hot to restore repo from hot-backup.py

[Ben Collins-Sussman <su...@collab.net>]

Eric Gillespie <ep...@pretzelnet.org> writes:

> Ben Collins-Sussman <su...@collab.net> writes:
> 
> > We've already got this information in the Handbook (in the "Repository
> > Maintenance" section), although it only mentions running this command
> > to unwedge a "frozen" repository.
> 
> It already documents how to restore from the backup?  I just looked
> and couldn't find that anywhere.

No, it just documents your example of 'db_recover' as a way of making
Berkeley DB replay transactions;  but the Handbook says that this is
only necessary when the repository locks up... because, well, that's
the only time we've ever had to use it!

But your use-case opens a whole new set of questions...

> > To do a "hot backup" of a db environment, all you need to do is
> > 1. copy the whole db/ area somewhere, and then 2. re-copy the logfiles
> > out again in a certain order.  Does this reveal some kind of
> > explanation?
> 
> Don't ask me.  What does "re-copy the logfiles in a certain order"
> mean? 

It means run 'db_archive -l', examine the list of logfiles it prints
out, and then copy them out in that order.

> I asked on the #svn channel how to restore, and no one knew.

This is what puzzles me.  There's no such thing as "how to restore" a
backup;  you just start using it.  No special knowledge required.  Or
so I *thought*

But it seems that your backup had unplayed transactions in it, which I
don't get.  Can someone more knowledgable explain?  Perhaps
Mr. Bostic?  :-)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Hot to restore repo from hot-backup.py

[Eric Gillespie <ep...@pretzelnet.org>]

Ben Collins-Sussman <su...@collab.net> writes:

> We've already got this information in the Handbook (in the "Repository
> Maintenance" section), although it only mentions running this command
> to unwedge a "frozen" repository.

It already documents how to restore from the backup?  I just looked
and couldn't find that anywhere.

> I'm surprised at your use-case though;  here we have a situation where
> the repository isn't wedged at all, yet db_recover reveals unplayed
> transactions?  How is that possible?

No idea.  Bill Tutt knew there were unplayed transactions in the
backup already; he's the one who told me that.  That's when i
decided to try db_recover.

> To do a "hot backup" of a db environment, all you need to do is
> 1. copy the whole db/ area somewhere, and then 2. re-copy the logfiles
> out again in a certain order.  Does this reveal some kind of
> explanation?

Don't ask me.  What does "re-copy the logfiles in a certain order"
mean?  I asked on the #svn channel how to restore, and no one knew.
Bill Tutt mentioned replaying log files, but he didn't know details.
So i poked around the db documentation and figured i'd give the
db_recover command a whirl.  It worked.

--  
Eric Gillespie <*> epg@pretzelnet.org

Build a fire for a man, and he'll be warm for a day.  Set a man on
fire, and he'll be warm for the rest of his life. -Terry Pratchett

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Hot to restore repo from hot-backup.py

[Ben Collins-Sussman <su...@collab.net>]

Eric Gillespie <ep...@pretzelnet.org> writes:

> 
> /usr/local/BerkeleyDB.4.0/bin/db_recover -ve -h ${repo}/db

We've already got this information in the Handbook (in the "Repository
Maintenance" section), although it only mentions running this command
to unwedge a "frozen" repository.

I'm surprised at your use-case though;  here we have a situation where
the repository isn't wedged at all, yet db_recover reveals unplayed
transactions?  How is that possible?

To do a "hot backup" of a db environment, all you need to do is
1. copy the whole db/ area somewhere, and then 2. re-copy the logfiles
out again in a certain order.  Does this reveal some kind of
explanation?


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org