You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Thomas Harold <th...@nybeta.com> on 2013/08/10 16:53:43 UTC
Re: Backup strategy sanity check
On 7/25/2013 7:30 AM, Stefan Sperling wrote:
> On Wed, Jul 24, 2013 at 03:22:11PM -0400, Thomas Harold wrote:
>> What we might do once 1.8 server is stable is switch to doing the
>> new "incremental" style hotcopy on Mon-Sat evenings and do a full
>> hotcopy on Sun.
>
> In Subversion 1.8, a full hotcopy is implemented as an incremental
> hotcopy into an emtpy target repository. There is no point in copying
> everything again on a Sunday because that just repeats work that's
> already been done.
>
During our upgrade process, we tried this (running svnadmin hotcopy
without blowing away the target directory first). We ran into an issue
that svnadmin hotcopy will not backup if the target directory is a
different format.
Old style of doing hotcopy backups:
1) rm -rf /backup/svnhotcopy/reponame
2) svnadmin hotcopy /var/svn/reponame /backup/svnhotcopy/reponame
Foolproof and always does the right thing. Downside is the extra disk
writes from redoing everything from scratch. (Our solution was to only
hotcopy repositories that had changes within the last N days. Which
reduced our nightly backup workload.)
With the 'svnadmin hotcopy --incremental' backups, we have to do extra
checking in the script (comparing reponame/db/format versions) in order
to make sure that the hotcopy runs correctly.
It would have been nice if --incremental would automatically upgrade the
target repository (and fallback to a full backup) if the versions mismatch.
Re: hotcopy --incremental auto-upgrade Re: Backup strategy sanity
check
Posted by Thomas Harold <th...@nybeta.com>.
On 8/10/2013 6:28 PM, Daniel Shahaf wrote:
> Daniel Shahaf wrote on Sun, Aug 11, 2013 at 01:25:24 +0300:
>> Thomas Harold wrote on Sat, Aug 10, 2013 at 10:53:43 -0400:
>>> With the 'svnadmin hotcopy --incremental' backups, we have to do extra
>>> checking in the script (comparing reponame/db/format versions) in order
>>> to make sure that the hotcopy runs correctly.
>>>
>>
>> If you have to do that, please do it correctly:
>>
>> - Check reponame/format
>> - Check reponame/db/fs-type
>> - Check reponame/db/format
>
> ... in this order.
>
Will make a note of what to check for future.
...
Maybe add --incremental-or-force (or some similar name) as an option.
It will, if the format/fs-type/db-format do not match, fall back to
doing a full blown hotcopy backup instead of incremental.
But it's a 'nice-to-have' and not 'required' because things changing
like this tend to be a once every 2-3 years thing. So we just work
around it.
Re: hotcopy --incremental auto-upgrade Re: Backup strategy sanity
check
Posted by Daniel Shahaf <da...@elego.de>.
Daniel Shahaf wrote on Sun, Aug 11, 2013 at 01:25:24 +0300:
> Thomas Harold wrote on Sat, Aug 10, 2013 at 10:53:43 -0400:
> > With the 'svnadmin hotcopy --incremental' backups, we have to do extra
> > checking in the script (comparing reponame/db/format versions) in order
> > to make sure that the hotcopy runs correctly.
> >
>
> If you have to do that, please do it correctly:
>
> - Check reponame/format
> - Check reponame/db/fs-type
> - Check reponame/db/format
... in this order.
Daniel
>
> (And 1.9 will have 'svnadmin info' which will give you the same information)
>
> > It would have been nice if --incremental would automatically upgrade the
> > target repository (and fallback to a full backup) if the versions
> > mismatch.
>
> Hmm. Interesting idea, but replacing failure modes with automagical
> behaviour is generally looked at with skepticism (is this error _really_
> always safe to not tell the admin about?). For the sake of argument,
> why shouldn't admins who want this behaviour opt-in to it by having
> their scripts do
>
> svnadmin upgrade $dest
> svnadmin hotcopy --incremental $src $dest
>
> ? (Note that 'upgrade' is idempotent, and will exit without error for
> already-most-recent-format repositories.)
>
> Cheers,
>
> Daniel
AW: hotcopy --incremental auto-upgrade Re: Backup strategy sanity
check
Posted by Markus Schaber <m....@codesys.com>.
Hi,
Von: Les Mikesell [mailto:lesmikesell@gmail.com]
>
> On Sat, Aug 10, 2013 at 5:25 PM, Daniel Shahaf <da...@elego.de> wrote:
> >
> >> It would have been nice if --incremental would automatically upgrade
> >> the target repository (and fallback to a full backup) if the versions
> >> mismatch.
> >
> > Hmm. Interesting idea, but replacing failure modes with automagical
> > behaviour is generally looked at with skepticism (is this error
> > _really_ always safe to not tell the admin about?). For the sake of
> > argument, why shouldn't admins who want this behaviour opt-in to it by
> > having their scripts do
> >
> > svnadmin upgrade $dest
> > svnadmin hotcopy --incremental $src $dest
> >
> > ? (Note that 'upgrade' is idempotent, and will exit without error for
> > already-most-recent-format repositories.)
>
> Which case is worse for an unattended script? Leaving you with no
> backups or one that needs a newer program to be able to use? And which case
> might an early user of subversion have been trained to expect?
I'd oppose auto-upgrading the destination repository to whatever toolchain version one happens to use - what happens when the source repository was not yet upgraded? Or it was upgraded from 1.5 format to 1.6 format, while the hotcopy is executing using 1.8?
However, I'd be fine with "svnadmin hotcopy" automatically falling back to a full copy (or do whatever else is necessary to create a backup) when the destination has a different version than the source repository, maybe guarded by an extra --allow-upgrades command line parameter.
Best regards
Markus Schaber
CODESYS(r) a trademark of 3S-Smart Software Solutions GmbH
Inspiring Automation Solutions
3S-Smart Software Solutions GmbH
Dipl.-Inf. Markus Schaber | Product Development Core Technology
Memminger Str. 151 | 87439 Kempten | Germany
Tel. +49-831-54031-979 | Fax +49-831-54031-50
E-Mail: m.schaber@codesys.com | Web: http://www.codesys.com | CODESYS store: http://store.codesys.com
CODESYS forum: http://forum.codesys.com
Managing Directors: Dipl.Inf. Dieter Hess, Dipl.Inf. Manfred Werner | Trade register: Kempten HRB 6186 | Tax ID No.: DE 167014915
Re: hotcopy --incremental auto-upgrade Re: Backup strategy sanity
check
Posted by Stefan Sperling <st...@elego.de>.
On Mon, Aug 12, 2013 at 08:36:42AM -0500, Les Mikesell wrote:
> On Sat, Aug 10, 2013 at 5:25 PM, Daniel Shahaf <da...@elego.de> wrote:
> >
> >> It would have been nice if --incremental would automatically upgrade the
> >> target repository (and fallback to a full backup) if the versions
> >> mismatch.
> >
> > Hmm. Interesting idea, but replacing failure modes with automagical
> > behaviour is generally looked at with skepticism (is this error _really_
> > always safe to not tell the admin about?). For the sake of argument,
> > why shouldn't admins who want this behaviour opt-in to it by having
> > their scripts do
> >
> > svnadmin upgrade $dest
> > svnadmin hotcopy --incremental $src $dest
> >
> > ? (Note that 'upgrade' is idempotent, and will exit without error for
> > already-most-recent-format repositories.)
The current implementation is conservative on purpose. If there is
huge demand I'll consider the idea of making incremental hotcopy deal
with upgraded repositories in some automated way. Perhaps there is
a good way of doing this. Perhaps not.
For now, users who copy data between repositories of different formats
should consider using 'svnadmin dump --incremental /svnadmin load'
instead of hotcopy.
> Which case is worse for an unattended script? Leaving you with no
> backups or one that needs a newer program to be able to use? And
> which case might an early user of subversion have been trained to
> expect?
Early users of Subversion don't have scripts that use incremental
hotcopy anyway. And if people don't notice that their backup scripts
are failing for some reason (I know this happens, I've seen it happen
many times), then they've got bigger problems which we cannot solve
for them.
Re: hotcopy --incremental auto-upgrade Re: Backup strategy sanity check
Posted by Les Mikesell <le...@gmail.com>.
On Sat, Aug 10, 2013 at 5:25 PM, Daniel Shahaf <da...@elego.de> wrote:
>
>> It would have been nice if --incremental would automatically upgrade the
>> target repository (and fallback to a full backup) if the versions
>> mismatch.
>
> Hmm. Interesting idea, but replacing failure modes with automagical
> behaviour is generally looked at with skepticism (is this error _really_
> always safe to not tell the admin about?). For the sake of argument,
> why shouldn't admins who want this behaviour opt-in to it by having
> their scripts do
>
> svnadmin upgrade $dest
> svnadmin hotcopy --incremental $src $dest
>
> ? (Note that 'upgrade' is idempotent, and will exit without error for
> already-most-recent-format repositories.)
Which case is worse for an unattended script? Leaving you with no
backups or one that needs a newer program to be able to use? And
which case might an early user of subversion have been trained to
expect?
--
Les Mikesell
lesmikesell@gmail.com
hotcopy --incremental auto-upgrade Re: Backup strategy sanity check
Posted by Daniel Shahaf <da...@elego.de>.
Thomas Harold wrote on Sat, Aug 10, 2013 at 10:53:43 -0400:
> With the 'svnadmin hotcopy --incremental' backups, we have to do extra
> checking in the script (comparing reponame/db/format versions) in order
> to make sure that the hotcopy runs correctly.
>
If you have to do that, please do it correctly:
- Check reponame/format
- Check reponame/db/fs-type
- Check reponame/db/format
(And 1.9 will have 'svnadmin info' which will give you the same information)
> It would have been nice if --incremental would automatically upgrade the
> target repository (and fallback to a full backup) if the versions
> mismatch.
Hmm. Interesting idea, but replacing failure modes with automagical
behaviour is generally looked at with skepticism (is this error _really_
always safe to not tell the admin about?). For the sake of argument,
why shouldn't admins who want this behaviour opt-in to it by having
their scripts do
svnadmin upgrade $dest
svnadmin hotcopy --incremental $src $dest
? (Note that 'upgrade' is idempotent, and will exit without error for
already-most-recent-format repositories.)
Cheers,
Daniel