You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by "Magnus Naeslund(k)" <ma...@kite.se> on 2006/08/16 11:51:49 UTC
Upgrading to 1.3.2: "svnadmin: File already exists"
Since this is a data-loss bug (in my view), I'm sending this to dev-list
aswell as the users-list. If I'm wrong in doing this, I apologize.
------------------------------------------------------------------------
When I was dumping and loading a repository (upgrading from 1.2.3 to
1.3.2) I got this nasty error message:
svnadmin: File already exists: filesystem 'repo1-restore1/db',
transaction '8684-1', path 'branches/someproject/v6.2/xproject/sl/docs/sl'
Command exited with non-zero status 1
The standard format dump file is about 7.9GB big, 1.7GB when dumped with
--deltas, the repo contains about 11000 revisions.
None of the dump files will load into a freshly created repo.
I don't see any problems with the paths, even if the sl/docs/sl looks
odd, that's the way it is.
So.. Does anyone know of this kind of problem?
So far I've only seen people converting from different other scm system
having this error. This repo is not converted.
From the error message I would think that the dump is wrongly ordered
somehow, that the sl/docs/sl directory maybe was created, deleted and
the created again, and due to wrong ordering it gets created twice in a row.
I'll dig into the dump file soon, but I was just wondering if this is a
known bug.
Regards,
Magnus
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Upgrading to 1.3.2: "svnadmin: File already exists"
Posted by Martin Furter <mf...@rola.ch>.
On Thu, 17 Aug 2006, Magnus Naeslund(k) wrote:
> C. Michael Pilato wrote:
>>
>> I'm not aware of such a bug.
>>
>> There are a couple of things you could do to facilitate someone here helping
>> to debug the situation:
>>
>> * Provide the output of 'svn log -v -q' from your original repository.
>>
>> * Provide the dumpfile, or at least maybe slosh it through grep so we
>> can see the revisions, and the paths and operations in those revisions:
>>
>> $ cat dumpfile | \
>> grep -E '(Revision-number:|Node-path:|Node-action:|Node-copyfrom-)'
>>
>
> I'll see if I can do this, the problem is that I work for a bank, and the bank has strict policys about source code.
> I definately cannot provide the dump file, that's for sure, but I'll see about the logs and/or filtrated dumpfile thing.
Maybe svndumptool.py can help here (though it only works for dump files
without deltas).
It can show a log (almost identical to svn log) of the contents of the
dumpfile, and some of the other commands may also be useful.
And if you want to tweak a dumpfile you can use the svndump classes
directly.
Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Upgrading to 1.3.2: "svnadmin: File already exists"
Posted by "Magnus Naeslund(k)" <ma...@kite.se>.
C. Michael Pilato wrote:
>
> I'm not aware of such a bug.
>
> There are a couple of things you could do to facilitate someone here helping
> to debug the situation:
>
> * Provide the output of 'svn log -v -q' from your original repository.
>
> * Provide the dumpfile, or at least maybe slosh it through grep so we
> can see the revisions, and the paths and operations in those revisions:
>
> $ cat dumpfile | \
> grep -E '(Revision-number:|Node-path:|Node-action:|Node-copyfrom-)'
>
I'll see if I can do this, the problem is that I work for a bank, and the bank has strict policys about source code.
I definately cannot provide the dump file, that's for sure, but I'll see about the logs and/or filtrated dumpfile thing.
Thanks,
Magnus
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Upgrading to 1.3.2: "svnadmin: File already exists"
Posted by "C. Michael Pilato" <cm...@collab.net>.
Magnus Naeslund(k) wrote:
> Since this is a data-loss bug (in my view), I'm sending this to dev-list
> aswell as the users-list. If I'm wrong in doing this, I apologize.
>
> ------------------------------------------------------------------------
>
> When I was dumping and loading a repository (upgrading from 1.2.3 to
> 1.3.2) I got this nasty error message:
>
> svnadmin: File already exists: filesystem 'repo1-restore1/db',
> transaction '8684-1', path 'branches/someproject/v6.2/xproject/sl/docs/sl'
> Command exited with non-zero status 1
>
> The standard format dump file is about 7.9GB big, 1.7GB when dumped with
> --deltas, the repo contains about 11000 revisions.
> None of the dump files will load into a freshly created repo.
>
> I don't see any problems with the paths, even if the sl/docs/sl looks
> odd, that's the way it is.
>
> So.. Does anyone know of this kind of problem?
> So far I've only seen people converting from different other scm system
> having this error. This repo is not converted.
>
> From the error message I would think that the dump is wrongly ordered
> somehow, that the sl/docs/sl directory maybe was created, deleted and
> the created again, and due to wrong ordering it gets created twice in a
> row.
>
> I'll dig into the dump file soon, but I was just wondering if this is a
> known bug.
I'm not aware of such a bug.
There are a couple of things you could do to facilitate someone here helping
to debug the situation:
* Provide the output of 'svn log -v -q' from your original repository.
* Provide the dumpfile, or at least maybe slosh it through grep so we
can see the revisions, and the paths and operations in those revisions:
$ cat dumpfile | \
grep -E '(Revision-number:|Node-path:|Node-action:|Node-copyfrom-)'
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Re: Upgrading to 1.3.2: "svnadmin: File already exists"
Posted by Ben Collins-Sussman <su...@red-bean.com>.
On 8/17/06, Sigfred Håversen <bs...@mumak.com> wrote:
> Any backup scheme that does not use 'svnadmin dump' for long term storage is
> _very flawed_. To put words into your mouth (sorry): any implicit of not using
> 'svnadmin dump' runs _very_ contrary to what was adviced before 1.x, speaking
> as a user since about 0.32. Please correct me if I am mistaken in this so
> that I may change my backup strategies.
See my email about backup best practices. I recommend an occasional
dump for long-term storage, but hotcopy for everyday backups.
The only reason we recommended dumping on every release before 1.0 is
because we were changing the db schema on nearly every release! It
was *required* to upgrade. Now that 1.0 has been released, we've made
a promise that no dump/load cycle is ever required to upgrade... until
we reach 2.0.
> As of 'svnadmin dump' is slow: I thought that relability and integrity of the
> repository is what is important.
It's not the only important thing, it's one of many important things.
If the disk crashes and you have to restore from backup, would you
rather tell your users to wait 10 hours for a dumpfile to load, or
wait 3 minutes for a hotcopy to be restored? Downtime matters too.
Mix the strategies.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Upgrading to 1.3.2: "svnadmin: File already exists"
Posted by Sigfred Håversen <bs...@mumak.com>.
On Thursday 17 August 2006 17:54, Ben Collins-Sussman wrote:
> On 8/17/06, Magnus Naeslund(k) <ma...@kite.se> wrote:
> > Ben Collins-Sussman wrote:
> > >> When I was dumping and loading a repository (upgrading from 1.2.3 to
> > >> 1.3.2) I got this nasty error message:
> > >
> > > Random side question: why were you dumping/loading? Changing from
> > > BDB to FSFS or something?
> >
> > I always dump/reload to:
> > 1) See that dump/reload works
> > 2) Let subversion have an opportunity to optimize the repository
>
> Optimize? Not sure what you mean there. If you dump an FSFS
> repository and then reload it into a new FSFS repository, I'd expect
> the two repositories to be byte-for-byte identical. It's not like
> there's some sort of defragmentation going on. :-)
I think that posters main point is 1) while 2) is nice to have.
> > I know that I didn't need to dump/reload for 1.3.x, but i do this as a
> > routine. I was lucky I did this, otherwise a restore from the backup
> > would have shown this error = really bad situation.
>
> I'm not sure I understand the reasoning here. You might have
> uncovered a bug in the code which creates a dumpfile; I doubt you've
> uncovered a deep corruption within the structure of your repository.
> Possible, but not as likely.
Test that backups works. I do that many times a year. Shit happens.
> (Do you use 'svnadmin dump' as a backup? That's awfully slow. You
> might just want to copy the repository.)
A 'svnadmin load' will work
a) for older repositories on newer versions of Subversion with schema changes
or new db backends for that matter, and
b) is OS independent.
Any backup scheme that does not use 'svnadmin dump' for long term storage is
_very flawed_. To put words into your mouth (sorry): any implicit of not using
'svnadmin dump' runs _very_ contrary to what was adviced before 1.x, speaking
as a user since about 0.32. Please correct me if I am mistaken in this so
that I may change my backup strategies.
As of 'svnadmin dump' is slow: I thought that relability and integrity of the
repository is what is important.
/Sigfred
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Upgrading to 1.3.2: "svnadmin: File already exists"
Posted by Ben Collins-Sussman <su...@red-bean.com>.
On 8/17/06, Jared Hardy <ja...@gmail.com> wrote:
> Should I give up all these options now, in favor of hot-backup.py? Is
> there any way to recover from these dump errors?
There's no right or wrong answers here. It's all about tradeoffs and
making choices.
Yes, dumping your repository guarantees it's in a portable format.
But it's also equivalent to running 'svn checkout' on *every* revision
in the repository, so it takes a long, long time. Is the portable
output format worth all the extra time? Maybe so, maybe not.
When people ask me for best practices, I generally recommend:
* do a hotcopy once a week as your "full" backup. Very quick.
* do incremental backups every night by dumping only the latest revisions.
* do a portable full-backup once a month by dumping.
Adjust timings as necessary, based on the activity level of your
repository. It's all about finding a balance between portability and
speed. Portability is nice, but generally it's not something that
matters when you're trying to restore a backup after a crash.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Upgrading to 1.3.2: "svnadmin: File already exists"
Posted by Jared Hardy <ja...@gmail.com>.
> > I know that I didn't need to dump/reload for 1.3.x, but i do this as a routine.
> > I was lucky I did this, otherwise a restore from the backup would have shown this error = really bad situation.
[snip]
>
> (Do you use 'svnadmin dump' as a backup? That's awfully slow. You
> might just want to copy the repository.)
I thought using 'svnadmin dump' as a backup was an accepted practice?
I have been using a script like this for a few months now:
nice svnadmin dump /svn/myrepo | gzip | split -b 256m --verbose -
/bak/myrepo.dump.gz.
I use split so that I can burn pieces of the dump to optical disk
sets, for longer term backups. I used 'svnadmin dump' instead of
hot-backup.py because I thought the dump files would be more portable,
in case I wanted to load it onto a different server platform. I use
pg_dumpall on my Postgres SQL server for similar reasons. Also, I
thought, if the problem that lead to the need for a restore was caused
by a particular revision or file commit, I could craft a dumpfilter
before restore, to prevent the problem the next time around. Last, as
others have noted in this thread, I would gain any benefits of having
the whole repo on the latest FSFS storage/delta model, upon any
restore.
Should I give up all these options now, in favor of hot-backup.py? Is
there any way to recover from these dump errors?
I am currently on svn 1.2 on the server, because of problems I had
setting up SVN 1.3 and Trac 0.94 on my SuSE ES9 server (problems not
worth discussing in this thread, I'm just noting my current version
and platform), but I would like to upgrade to 1.4 as soon as it's
deemed stable (with Trac 0.96).
Thanks,
Jared
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Upgrading to 1.3.2: "svnadmin: File already exists"
Posted by "Magnus Naeslund(k)" <ma...@kite.se>.
Ben Collins-Sussman wrote:
>
> Optimize? Not sure what you mean there. If you dump an FSFS
> repository and then reload it into a new FSFS repository, I'd expect
> the two repositories to be byte-for-byte identical. It's not like
> there's some sort of defragmentation going on. :-)
>
Well one example would if subversion would change delta algoritm.
But this part of the discussion is not important.
>> I know that I didn't need to dump/reload for 1.3.x, but i do this as a routine.
>> I was lucky I did this, otherwise a restore from the backup would have shown this error = really bad situation.
>
> I'm not sure I understand the reasoning here. You might have
> uncovered a bug in the code which creates a dumpfile; I doubt you've
> uncovered a deep corruption within the structure of your repository.
> Possible, but not as likely.
>
Well I'm not sure, but I like to verify that dump/restore works when I upgrade to another svn version.
Maybe you see that as silly, but I'm glad I tried it, as I found an actual problem (whatever reason for it).
> (Do you use 'svnadmin dump' as a backup? That's awfully slow. You
> might just want to copy the repository.)
Yes I use that format because I thought it was the most flexible format.
This might not be optimal as our repository grows, but I'll examine that later.
Regards,
Magnus
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Upgrading to 1.3.2: "svnadmin: File already exists"
Posted by Ben Collins-Sussman <su...@red-bean.com>.
Ah yes, thanks for the counterexamples, guys.
On 8/17/06, Mark Phippard <ma...@softlanding.com> wrote:
> Mark Phippard <ma...@softlanding.com> wrote on 08/17/2006 12:56:20 PM:
>
> > Greg Hudson <gh...@MIT.EDU> wrote on 08/17/2006 12:45:00 PM:
> >
> > > On Aug 17, 2006, at 11:54 AM, Ben Collins-Sussman wrote:
> > > >
> > > > Optimize? Not sure what you mean there. If you dump an FSFS
> > > > repository and then reload it into a new FSFS repository, I'd expect
> > > > the two repositories to be byte-for-byte identical. It's not like
> > > > there's some sort of defragmentation going on. :-)
> > >
> > > Counterexample: If you're upgrading to 1.4, you want to dump and load
> > > your repository (regardless of format) to re-encode all the diffs in
> > > svndiff1.
> >
> > Likewise, in 1.3 a dump/load would convert history from vdelta to xdelta
>
> > which was supposed to speed up blame and some other operations.
>
> I meant to say 1.2, but my point was just that there is some precedent for
> this.
>
> Mark
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Upgrading to 1.3.2: "svnadmin: File already exists"
Posted by Mark Phippard <ma...@softlanding.com>.
Mark Phippard <ma...@softlanding.com> wrote on 08/17/2006 12:56:20 PM:
> Greg Hudson <gh...@MIT.EDU> wrote on 08/17/2006 12:45:00 PM:
>
> > On Aug 17, 2006, at 11:54 AM, Ben Collins-Sussman wrote:
> > >
> > > Optimize? Not sure what you mean there. If you dump an FSFS
> > > repository and then reload it into a new FSFS repository, I'd expect
> > > the two repositories to be byte-for-byte identical. It's not like
> > > there's some sort of defragmentation going on. :-)
> >
> > Counterexample: If you're upgrading to 1.4, you want to dump and load
> > your repository (regardless of format) to re-encode all the diffs in
> > svndiff1.
>
> Likewise, in 1.3 a dump/load would convert history from vdelta to xdelta
> which was supposed to speed up blame and some other operations.
I meant to say 1.2, but my point was just that there is some precedent for
this.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Upgrading to 1.3.2: "svnadmin: File already exists"
Posted by Mark Phippard <ma...@softlanding.com>.
Greg Hudson <gh...@MIT.EDU> wrote on 08/17/2006 12:45:00 PM:
> On Aug 17, 2006, at 11:54 AM, Ben Collins-Sussman wrote:
> >
> > Optimize? Not sure what you mean there. If you dump an FSFS
> > repository and then reload it into a new FSFS repository, I'd expect
> > the two repositories to be byte-for-byte identical. It's not like
> > there's some sort of defragmentation going on. :-)
>
> Counterexample: If you're upgrading to 1.4, you want to dump and load
> your repository (regardless of format) to re-encode all the diffs in
> svndiff1.
Likewise, in 1.3 a dump/load would convert history from vdelta to xdelta
which was supposed to speed up blame and some other operations.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Upgrading to 1.3.2: "svnadmin: File already exists"
Posted by Greg Hudson <gh...@MIT.EDU>.
On Aug 17, 2006, at 11:54 AM, Ben Collins-Sussman wrote:
>
> Optimize? Not sure what you mean there. If you dump an FSFS
> repository and then reload it into a new FSFS repository, I'd expect
> the two repositories to be byte-for-byte identical. It's not like
> there's some sort of defragmentation going on. :-)
Counterexample: If you're upgrading to 1.4, you want to dump and load
your repository (regardless of format) to re-encode all the diffs in
svndiff1.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Upgrading to 1.3.2: "svnadmin: File already exists"
Posted by Ben Collins-Sussman <su...@red-bean.com>.
On 8/17/06, Magnus Naeslund(k) <ma...@kite.se> wrote:
> Ben Collins-Sussman wrote:
> >> When I was dumping and loading a repository (upgrading from 1.2.3 to
> >> 1.3.2) I got this nasty error message:
> >
> > Random side question: why were you dumping/loading? Changing from
> > BDB to FSFS or something?
>
> I always dump/reload to:
> 1) See that dump/reload works
> 2) Let subversion have an opportunity to optimize the repository
Optimize? Not sure what you mean there. If you dump an FSFS
repository and then reload it into a new FSFS repository, I'd expect
the two repositories to be byte-for-byte identical. It's not like
there's some sort of defragmentation going on. :-)
>
> I know that I didn't need to dump/reload for 1.3.x, but i do this as a routine.
> I was lucky I did this, otherwise a restore from the backup would have shown this error = really bad situation.
I'm not sure I understand the reasoning here. You might have
uncovered a bug in the code which creates a dumpfile; I doubt you've
uncovered a deep corruption within the structure of your repository.
Possible, but not as likely.
(Do you use 'svnadmin dump' as a backup? That's awfully slow. You
might just want to copy the repository.)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Upgrading to 1.3.2: "svnadmin: File already exists"
Posted by "Magnus Naeslund(k)" <ma...@kite.se>.
Ben Collins-Sussman wrote:
>> When I was dumping and loading a repository (upgrading from 1.2.3 to
>> 1.3.2) I got this nasty error message:
>
> Random side question: why were you dumping/loading? Changing from
> BDB to FSFS or something?
I always dump/reload to:
1) See that dump/reload works
2) Let subversion have an opportunity to optimize the repository
I know that I didn't need to dump/reload for 1.3.x, but i do this as a routine.
I was lucky I did this, otherwise a restore from the backup would have shown this error = really bad situation.
Regards,
Magnus
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Upgrading to 1.3.2: "svnadmin: File already exists"
Posted by Ben Collins-Sussman <su...@red-bean.com>.
> When I was dumping and loading a repository (upgrading from 1.2.3 to
> 1.3.2) I got this nasty error message:
Random side question: why were you dumping/loading? Changing from
BDB to FSFS or something?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org