You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by "Melikian, Chris" <Ch...@uk.fid-intl.com> on 2007/06/28 15:55:20 UTC
Lack of Subversion repository recovery tools
Now I really like Subversion. Been using it for 2 years as a user and
administrator and we've had a lot of success in implementing svn as the
version control system of choice for our continuous integration
environments against other more heavyweight tools such as ClearCase etc.
It's fast, lightweight, easy to get going etc
On the other hand, I've become very disillusioned with the lack of
documentation and tools on how to recover from corrupted revisions. The
problem we have is that we are trying to an "svnadmin dump" of our
repository in order to shrink it with svndumpfilter and we are getting
"Error: Invalid diff stream: insn 1245 cannot be decoded"
errors. We are using svn 1.3.2 on Solaris with FSFS as the back end and
svnserve as our repository access front end.
There are many, many emails listing the above error and very few
suggestions on how to solve this. There seems to be no way with the
supplied tools to correct this error. Fair enough if the revision is
corrupted but we cannot even mark the revision as bad or delete it and
so the repository stays in a corrupted state. The only tool we've found
is from the extremely helpful John Szakmeister with his fsfsverify.py
script but it hasn't solved our problem.
The other option is to perform incremental dumps in blocks skipping the
corrupted revisions. ie
svnadmin dump -r 1:4745 myrepo > dump_1_4745.txt <----- skip 4746,
the broken rev
svnadmin dump -r 4747:10000 --incremental myrepo > dump_4747_10000.txt
svnadmin dump -r 10001:20000 --incremental myrepo > dump_10000_20000.txt
svnadmin dump -r 20001:30000 --incremental myrepo > dump_10000_20000.txt
...etc...
The problem with the above is that svnadmin verify will only tell us the
first revision which is broken and then stops. So we don't know if there
is a broken revision after 4746 using the example above. Plus, if we
want to keep integrity of revisions, we'd have to put dummy commits into
the new repository where the broken revisions were. Not nice.
I think Subversion is an excellent tool but more time needs to be spent
on the administration tools to underpin the repository and make it rock
solid before new functionality is introduced.
If the answer is to submit a problem report, then I'm perfectly willing
to do so.
By the way, if anyone does know how to fix the above then please let me
know. We've been spending a long time on this.
Chris Melikian
Important: Fidelity Investments International (Reg. No.1448245),
Fidelity Investment Services Limited (Reg. No. 2016555), Fidelity
Pensions Management (Reg. No. 2015142) and Financial Administration
Services Limited (Reg. No. 1629709, a Fidelity Group company) are all
registered in England and Wales, are authorised and regulated in the UK
by the Financial Services Authority and have their registered offices at
Oakhill House, 130 Tonbridge Road, Hildenborough, Tonbridge, Kent TN11
9DZ. Tel 01732 361144. Fidelity only gives information on products and
does not give investment advice to private clients based on individual
circumstances. Any comments or statements made are not necessarily those
of Fidelity. The information transmitted is intended only for the person
or entity to which it is addressed and may contain confidential and/or
privileged material. If you received this in error, please contact the
sender and delete the material from any computer. All e-mails sent from
or to Fidelity may be subject to our monitoring procedures. Direct link
to Fidelity's website -
http://www.fidelity-international.com/world/index.html
Re: Re: Lack of Subversion repository recovery tools
Posted by Jing Xue <ji...@digizenstudio.com>.
On Fri, Jun 29, 2007 at 04:07:51PM -0300, Andreas Hasenack wrote:
> On Fri, Jun 29, 2007 at 02:52:04PM -0400, Jing Xue wrote:
> >
> > Sorry if I'm missing something very obvious here, but assuming a backup
> > implementation that ensures timely and completely verified backups, why
> > would one insist on repairing the repository? AFAIK, a hotcopy backup is a
> > good-to-go repository itself.
>
> Following this rationale, you could as well as toss all your /sbin/fsck*
> files. :)
The essential value of fsck to me is that, in the event of a corruption,
it allows me a much better chance (by trying to repair as much as
possible) to get to a point where I can comfortably examine the damage,
and decide whether I'd like to restore a backup and actually do it.
Without fsck, I might be very well shut out of my system due to even the
least amount of corruption.
Such is not the case in svn. It's extremely easy to restore a repository
from a backup. I would bother to try to rely on either a program's or my
guess work to repair the repository only in one case, that is some one
checked in something after the last backup, and nobody else updated
their work copies since then, _and_ the guy lost his own copy as well.
Does that happen? Sure. Often enough to warrant a repair tool? I don't
know. 8-)
--
Jing Xue
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Toby Thain <to...@smartgames.ca>.
On 29-Jun-07, at 6:45 PM, Andreas Hasenack wrote:
> On Fri, Jun 29, 2007 at 05:17:36PM -0300, Toby Thain wrote:
>> When was the last time your fsck reported corruption, and did you
>> find the
>> root cause?
>
> Sure, an unexpected shutdown for example. I'm glad such a minor event
> didn't force me to reinstall the whole operating system or restore
> from
> backups. You don't expect me to have a diesel generator around
> here, do
> you? :)
Yes, it will usually be an unexpected shutdown.
Which is why fsck is not a very good analogy: Because it's certainly
possible (viz. ZFS, Reiser3, etc) to design a database/filesystem/
repository whose integrity is not risked by "unexpected shutdown" -
in a word, transactional. (Obviating fsck!) And I suspect this is not
lost on the Subversion designers, given that Svn is transactional by
design, a property which must be based on the underlying data store.
Which is why I said that some structural design effort up front - if,
for example, it is desired behaviour to "stay consistent in the face
of unexpected interruption" - is likely to be a better investment
than a post-facto scavenger.
(Of course, if the design is *not* transactional (and I have no
evidence that Subversion+FSFS/BDB is not), then there could be
identifiable failure modes in the particular case of *interruption*
which could be addressed by an fsck-like scavenger. However, the
thread has not been discussing this special case, but rather
*arbitrary* hardware faults. "Unexpected interruption" is perhaps the
simplest and least destructive unexpected event of all.)
Another difference between software and hardware faults: A software
problem may *never* be encountered in your use of Subversion (of
course it has >0 bugs, but you may simply never step on one); but a
hardware problem is statistically *inevitable*.
Bottom line: Carefully characterise what you want to defend against.
--Toby
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Fri, Jun 29, 2007 at 05:17:36PM -0300, Toby Thain wrote:
> When was the last time your fsck reported corruption, and did you find the
> root cause?
Sure, an unexpected shutdown for example. I'm glad such a minor event
didn't force me to reinstall the whole operating system or restore from
backups. You don't expect me to have a diesel generator around here, do
you? :)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Toby Thain <to...@smartgames.ca>.
On 29-Jun-07, at 4:07 PM, Andreas Hasenack wrote:
> On Fri, Jun 29, 2007 at 02:52:04PM -0400, Jing Xue wrote:
>>
>> Quoting "Melikian, Chris" <Ch...@uk.fid-intl.com>:
>>
>>> Hi Ulrich,
>>>
>>> I completely agree that you need to use backups but I think that
>>> subversion should also have some tools to repair revisions so
>>> that you
>>> can at least get the repository back into a consistent state
>>> even if
>>> there is data loss.
>>>
>>> Surely, any production-ready version control system has such a tool?
>>
>> Sorry if I'm missing something very obvious here, but assuming a
>> backup
>> implementation that ensures timely and completely verified
>> backups, why
>> would one insist on repairing the repository? AFAIK, a hotcopy
>> backup is a
>> good-to-go repository itself.
>
> Following this rationale, you could as well as toss all your /sbin/
> fsck*
> files. :)
ZFS has no fsck. The reasons why are quite relevant to this thread.
When was the last time your fsck reported corruption, and did you
find the root cause?
--Toby
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Re: Lack of Subversion repository recovery tools
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Fri, Jun 29, 2007 at 02:52:04PM -0400, Jing Xue wrote:
>
> Quoting "Melikian, Chris" <Ch...@uk.fid-intl.com>:
>
>> Hi Ulrich,
>>
>> I completely agree that you need to use backups but I think that
>> subversion should also have some tools to repair revisions so that you
>> can at least get the repository back into a consistent state even if
>> there is data loss.
>>
>> Surely, any production-ready version control system has such a tool?
>
> Sorry if I'm missing something very obvious here, but assuming a backup
> implementation that ensures timely and completely verified backups, why
> would one insist on repairing the repository? AFAIK, a hotcopy backup is a
> good-to-go repository itself.
Following this rationale, you could as well as toss all your /sbin/fsck*
files. :)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Re: Lack of Subversion repository recovery tools
Posted by Jing Xue <ji...@digizenstudio.com>.
Quoting "Melikian, Chris" <Ch...@uk.fid-intl.com>:
> Hi Ulrich,
>
> I completely agree that you need to use backups but I think that
> subversion should also have some tools to repair revisions so that
> you can at least get the repository back into a consistent state
> even if there is data loss.
>
> Surely, any production-ready version control system has such a tool?
Sorry if I'm missing something very obvious here, but assuming a
backup implementation that ensures timely and completely verified
backups, why would one insist on repairing the repository? AFAIK, a
hotcopy backup is a good-to-go repository itself.
--
Jing Xue
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Re: Lack of Subversion repository recovery tools
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Fri, Jun 29, 2007 at 02:27:34PM +0200, Erik Huelsmann wrote:
>> Come on, no need to get sarcastic. Or are you going to blame intel's
>> core2duo recent bugs too? :)
>
> Yes :-) Seriously though, I wasn't joking; how do you expect
> Subversion to recover from things you can't foresee now? We'll only be
"recover" is a bad word actually. What I mean is "bring it back to a
consistent state where I can use svnadmin dump and get something out of
it". If an entry is corrupted, trash it and move on to the next piece of
data. If it depended on the previous one that was trashed, loose it too,
and so on. If nothing is left after this, well, we tried.
> able to tell something is wrong, not in which way it needs to be
> changed to be correct....
If it's wrong, trash it and move on to the next piece. That's what fsck
does. You may end up with just 10% of the data, due to the internal
deltas, but hey, it may contain that crucial record you needed...
> Ok, fair enough, but the request from the OP doesn't sound like he's
> asking for this. He's asking for functional tooling, as I interpret
> it. So, while it would be much clearer that recover doesn't do
> anything, it isn't a solution to his request.
Agreed. I will check the bug database and request it if it's not there
yet.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Re: Lack of Subversion repository recovery tools
Posted by Erik Huelsmann <eh...@gmail.com>.
On 6/29/07, Andreas Hasenack <an...@mandriva.com.br> wrote:
> On Fri, Jun 29, 2007 at 12:31:02PM +0200, Erik Huelsmann wrote:
> > On 6/29/07, Melikian, Chris <Ch...@uk.fid-intl.com> wrote:
> >> Hi Ulrich,
> >>
> >> I completely agree that you need to use backups but I think that
> >> subversion should also have some tools to repair revisions so that you can
> >> at least get the repository back into a consistent state even if there is
> >> data loss.
> >>
> >> Surely, any production-ready version control system has such a tool?
> >
> > And, given hardware failure (RAM chips?) how do you determine
>
> Come on, no need to get sarcastic. Or are you going to blame intel's
> core2duo recent bugs too? :)
Yes :-) Seriously though, I wasn't joking; how do you expect
Subversion to recover from things you can't foresee now? We'll only be
able to tell something is wrong, not in which way it needs to be
changed to be correct....
> At least patch "svnadmin recover" then to say it does nothing with the
> FSFS backend.
Ok, fair enough, but the request from the OP doesn't sound like he's
asking for this. He's asking for functional tooling, as I interpret
it. So, while it would be much clearer that recover doesn't do
anything, it isn't a solution to his request.
bye,
Erik.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Toby Thain <to...@smartgames.ca>.
On 29-Jun-07, at 9:11 AM, Andreas Hasenack wrote:
> On Fri, Jun 29, 2007 at 12:31:02PM +0200, Erik Huelsmann wrote:
>> On 6/29/07, Melikian, Chris <Ch...@uk.fid-intl.com> wrote:
>>> Hi Ulrich,
>>>
>>> I completely agree that you need to use backups but I think that
>>> subversion should also have some tools to repair revisions so
>>> that you can
>>> at least get the repository back into a consistent state even if
>>> there is
>>> data loss.
>>>
>>> Surely, any production-ready version control system has such a tool?
>>
>> And, given hardware failure (RAM chips?) how do you determine
>
> Come on, no need to get sarcastic.
Uh, bad RAM is a common failure mode.
--Toby
> Or are you going to blame intel's
> core2duo recent bugs too? :)
>
>
> At least patch "svnadmin recover" then to say it does nothing with the
> FSFS backend.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Re: Lack of Subversion repository recovery tools
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Fri, Jun 29, 2007 at 12:31:02PM +0200, Erik Huelsmann wrote:
> On 6/29/07, Melikian, Chris <Ch...@uk.fid-intl.com> wrote:
>> Hi Ulrich,
>>
>> I completely agree that you need to use backups but I think that
>> subversion should also have some tools to repair revisions so that you can
>> at least get the repository back into a consistent state even if there is
>> data loss.
>>
>> Surely, any production-ready version control system has such a tool?
>
> And, given hardware failure (RAM chips?) how do you determine
Come on, no need to get sarcastic. Or are you going to blame intel's
core2duo recent bugs too? :)
At least patch "svnadmin recover" then to say it does nothing with the
FSFS backend.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Blair Zajac <bl...@orcaware.com>.
Hi Chris,
The only tool I am aware of which tries to fix FSFS repositories is this one:
http://www.szakmeister.net/blog/?page_id=16
I haven't had the need to use it yet, but my understanding is that it looks for
specific errors that can occur in a FSFS repository. It may not fix all errors.
Let us know how it works for you.
Regards,
Blair
Melikian, Chris wrote:
> Hi Erik,
>
> No of course things happen which we have no control over. That's part of
> the fun/headache of IT right? ;-)
>
> What I am saying I that I am surprised there are no tools available to
> be able to recover a repository into a usable state. A simple(no such
> thing I know) tool is all that is needed to be able to mark a revision
> as bad and let and svnadmin dump/verify and the other tools work as they
> should.
>
> Do you think this is unreasonable?
>
> Cheers,
>
> Chris Melikian
>
> Important: Fidelity Investments International (Reg. No.1448245),
> Fidelity Investment Services Limited (Reg. No. 2016555), Fidelity
> Pensions Management (Reg. No. 2015142) and Financial Administration
> Services Limited (Reg. No. 1629709, a Fidelity Group company) are all
> registered in England and Wales, are authorised and regulated in the UK
> by the Financial Services Authority and have their registered offices at
> Oakhill House, 130 Tonbridge Road, Hildenborough, Tonbridge, Kent TN11
> 9DZ. Tel 01732 361144. Fidelity only gives information on products and
> does not give investment advice to private clients based on individual
> circumstances. Any comments or statements made are not necessarily those
> of Fidelity. The information transmitted is intended only for the person
> or entity to which it is addressed and may contain confidential and/or
> privileged material. If you received this in error, please contact the
> sender and delete the material from any computer. All e-mails sent from
> or to Fidelity may be subject to our monitoring procedures. Direct link
> to Fidelity's website -
> http://www.fidelity-international.com/world/index.html
>
>
> -----Original Message-----
> From: Erik Huelsmann [mailto:ehuels@gmail.com]
> Sent: 29 June 2007 11:31
> To: Melikian, Chris
> Cc: Ulrich Eckhardt; users@subversion.tigris.org
> Subject: Re: Re: Lack of Subversion repository recovery tools
>
> On 6/29/07, Melikian, Chris <Ch...@uk.fid-intl.com> wrote:
>> Hi Ulrich,
>>
>> I completely agree that you need to use backups but I think that
> subversion should also have some tools to repair revisions so that you
> can at least get the repository back into a consistent state even if
> there is data loss.
>> Surely, any production-ready version control system has such a tool?
>
> And, given hardware failure (RAM chips?) how do you determine
> consistency? Subversion provides tools to asses
> consistency/correctness and prevents getting into an inconsistent
> state under 'normal operation', but you can't expect it to be able to
> recover from situations which aren't predefined, right?
>
> Or would 'cutting the repository off at the last valid revision' also
> match your requirement? (it sure doesn't sound like any help to me!)
>
> bye,
>
> Erik.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Re: Lack of Subversion repository recovery tools
Posted by "Melikian, Chris" <Ch...@uk.fid-intl.com>.
Thanks Erik. My point is that as my repository is corrupted I cannot use
svndumpfilter to shrink my repository. In fact I will have a nightmare
task if my only option is to
1 Use svnadmin dump to dump the repository up to the point before the
first broken revision
2 I don't know if there are more errors at higher revision levels as
svnadmin verify stops after the first error it finds
3 So, I have to repeat "svnadmin dump --incremental -r LOW:HIGH"
commands hoping that there are no errors within the LOW/HIGH revision
numbers.
4 Skip revisions which are broken
5 Repeat from 3 until all the revisions are dumped
Then after that I have to "svnadmin --force-uuid load" the revisions
using dumpfilter into a new repository. I have to do a "dummy" commit
for each of the revisions which are broken so that I keep the repository
revision numbers the same. Then hopefully, &deity; willing, I can get my
users to access the repository without noticing a difference.
Is the above my only hope with FSFS?
Cheers,
Chris Melikian
Important: Fidelity Investments International (Reg. No.1448245),
Fidelity Investment Services Limited (Reg. No. 2016555), Fidelity
Pensions Management (Reg. No. 2015142) and Financial Administration
Services Limited (Reg. No. 1629709, a Fidelity Group company) are all
registered in England and Wales, are authorised and regulated in the UK
by the Financial Services Authority and have their registered offices at
Oakhill House, 130 Tonbridge Road, Hildenborough, Tonbridge, Kent TN11
9DZ. Tel 01732 361144. Fidelity only gives information on products and
does not give investment advice to private clients based on individual
circumstances. Any comments or statements made are not necessarily those
of Fidelity. The information transmitted is intended only for the person
or entity to which it is addressed and may contain confidential and/or
privileged material. If you received this in error, please contact the
sender and delete the material from any computer. All e-mails sent from
or to Fidelity may be subject to our monitoring procedures. Direct link
to Fidelity's website -
http://www.fidelity-international.com/world/index.html
-----Original Message-----
From: Erik Huelsmann [mailto:ehuels@gmail.com]
Sent: 30 June 2007 20:25
To: Melikian, Chris
Cc: users@subversion.tigris.org
Subject: Re: Re: Lack of Subversion repository recovery tools
Hi Chris,
> No of course things happen which we have no control over. That's part
of
> the fun/headache of IT right? ;-)
Yes :-)
> What I am saying I that I am surprised there are no tools available to
> be able to recover a repository into a usable state. A simple(no such
> thing I know) tool is all that is needed to be able to mark a revision
> as bad and let and svnadmin dump/verify and the other tools work as
they
> should.
>
> Do you think this is unreasonable?
Absolutely not! It's a very reasonable request! The only thing is that
this is the default in-production behaviour already: if a damaged
revision is not required to construct a later revision, it's simply
skipped and no damage will be detected.
With FSFS all changes in one revision are stored in one 'revision'
file. With skip-deltas [1] the chances become bigger that a large
number of those revision files have to be consulted, both on larger
trees (because there are more elements which have dependencies on
earlier revisions) and on larger numbers of commits (because chances
increase that different files in the same checkout will have been
modified in different revisions).
This makes FSFS more sensitive to broken revision files than the BDB
backend, which stores all file changes in one big database. If part of
that database gets damaged, you may only notice when you're checking
out the damaged file(s), but *only* when the revision you're checking
out actually depends on the damaged revisions.
So, we actually already try our hardest - even in production - to give
you all the output we can give you. With skip-deltas it's almost
impossible to actually mark a given revision as 'bad' and just proceed
with what we do have.
I hope that clarifies things a bit. The above is also an answer to the
'--try-harder' proposal earlier in the thread: we already do.
bye,
Erik.
[1] http://svn.collab.net/repos/svn/trunk/notes/skip-deltas
PS: Possibly, you can't help it, but could you please post without the
extremely long disclaimer? It has no legal value, especially on an
open and archived mailing list like this.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Wed, Jul 04, 2007 at 04:06:43PM +0200, Erik Huelsmann wrote:
> On 7/4/07, Andreas Hasenack <an...@mandriva.com.br> wrote:
>
> [ ... ]
>
>> > But that's not what you asked. You asked Subversion to verify all
>> > revisions. And it reported an error in revision 1. Which is correct,
>> > because revision 1 was corrupted.
>>
>> And it stopped there, even though I asked it to verify all revisions.
>
> You basically asked it: "Is the repository in good condition?" And it
> answered "Sorry, no." And annotated that with "Because your revision 1
> is broken."
>
>> svnadmin dump also stops there, and I believe this was the original
>> complaint.
>
> Well that's a good thing :-) They're implemented using exactly the same
> code!
>
>> The OP asked if there was a way for svnadmin verify to mark
>> that revision as bad, skip it and let svnadmin dump dump the rest. It
>> would be a waste if all the other 150k revisions didn't depend on this
>> one and just because of this couldn't be dumped.
>
> Ah! But this isn't true: you can dump them, but not with a single
> command...
A *lot* of commands. Something so manual that it calls for a tool :)
> As I'm on holidays, I can't verify it myself, but I've been told that
> the trunk (1.5-dev) version of svnadmin verify supports
> revision(range)s, so that this functionality is essentially available
> with that version (all be it not with 1 Subversion command).
But only then will you know if the next revision depends on the
previous. And you have to keep doing it.
I did this once, and it only worked after a lot of trial and error using
a dump&reload sequence, and it would have NOT worked without the help
from the already mentioned fsverify contributed script.
So, I believe a tool for FSFS recovery is indeed called for. And with
"recovery" I basically mean what I mentioned above in the line that
starts with "The OP asked if there ...". I know there is no such thing
as magic: lost data is lost.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Erik Huelsmann <eh...@gmail.com>.
On 7/4/07, Andreas Hasenack <an...@mandriva.com.br> wrote:
[ ... ]
> > But that's not what you asked. You asked Subversion to verify all
> > revisions. And it reported an error in revision 1. Which is correct,
> > because revision 1 was corrupted.
>
> And it stopped there, even though I asked it to verify all revisions.
You basically asked it: "Is the repository in good condition?" And it
answered "Sorry, no." And annotated that with "Because your revision 1
is broken."
> svnadmin dump also stops there, and I believe this was the original
> complaint.
Well that's a good thing :-) They're implemented using exactly the same code!
> The OP asked if there was a way for svnadmin verify to mark
> that revision as bad, skip it and let svnadmin dump dump the rest. It
> would be a waste if all the other 150k revisions didn't depend on this
> one and just because of this couldn't be dumped.
Ah! But this isn't true: you can dump them, but not with a single command...
As I'm on holidays, I can't verify it myself, but I've been told that
the trunk (1.5-dev) version of svnadmin verify supports
revision(range)s, so that this functionality is essentially available
with that version (all be it not with 1 Subversion command).
HTH,
Erik.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Re: Lack of Subversion repository recovery tools
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Wed, Jul 04, 2007 at 01:51:52PM +0100, Melikian, Chris wrote:
>
> Andreas Hasenack said:-
> > svnadmin dump also stops there, and I believe this was the original
> > complaint. The OP asked if there was a way for svnadmin verify to mark
> > that revision as bad, skip it and let svnadmin dump dump the rest. It
> > would be a waste if all the other 150k revisions didn't depend on this
> > one and just because of this couldn't be dumped.
> >
>
> That is a 100% correct description of my gripe. I'm stuck with a
> repository I can't dump because a revision is corrupt and can't be fixed
> with fsfsverify, a 3rd party tool with no support (officially but John
> has been very helpful!).
>
> I'm seriously doubting the production readiness of subversion with fsfs.
> I'm going to look into subversion with bdb but I'm facing increasing
If you go towards BDB, make sure you understand some of the DB_CONFIG
settings. The openldap faq is a good place to start:
http://www.openldap.org/faq/data/cache/1072.html
http://www.openldap.org/faq/data/cache/893.html
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Re: Lack of Subversion repository recovery tools
Posted by "Melikian, Chris" <Ch...@uk.fid-intl.com>.
________________________________
From: Talden [mailto:talden@gmail.com]
Sent: 05 July 2007 01:32
To: Toby Thain
Cc: Les Mikesell; users
Subject: Re: Lack of Subversion repository recovery tools
It seems a couple of tools might be needed.
1. Cheap verification. Most likely achieved by supporting
verify revision ranges so that recent revs can be verified to detect
errors earlier. This sounds like it's on the way.
2. Repo scraper. A dump option that dumps revisions while they
good, skips revisions when they're bad and keeps content in subsequent
good revisions that is recoverable - eg skips rather than diffs. Let
users know revision timestamp info to suggest that some or all data
newer than these times is missing.
Cheap verification dramatically reduces the urgency of providing
what could be a fairly complex recovery tool.
If repo content can be verified more often then users will get
more opportunity to use their backups productively.
--
Talden
The above would be great. Where a revision has corrupted, that hasn't
stopped us from being able to check in a subsequent revision of the same
code in the same part of the svn tree. SVN is clever enough to allow
you to check in again and successfully update the revision on top of the
broken one. It's only the one revision which is broken so I wouldn't
expect the revision tree for a particular file to be removed just
because one of its revisions was corrupted.
Chris Melikian
Important: Fidelity Investments International (Reg. No.1448245),
Fidelity Investment Services Limited (Reg. No. 2016555), Fidelity
Pensions Management (Reg. No. 2015142) and Financial Administration
Services Limited (Reg. No. 1629709, a Fidelity Group company) are all
registered in England and Wales, are authorised and regulated in the UK
by the Financial Services Authority and have their registered offices at
Oakhill House, 130 Tonbridge Road, Hildenborough, Tonbridge, Kent TN11
9DZ. Tel 01732 361144. Fidelity only gives information on products and
does not give investment advice to private clients based on individual
circumstances. Any comments or statements made are not necessarily those
of Fidelity. The information transmitted is intended only for the person
or entity to which it is addressed and may contain confidential and/or
privileged material. If you received this in error, please contact the
sender and delete the material from any computer. All e-mails sent from
or to Fidelity may be subject to our monitoring procedures. Direct link
to Fidelity's website -
http://www.fidelity-international.com/world/index.html
Re: Lack of Subversion repository recovery tools
Posted by Talden <ta...@gmail.com>.
The point really is that it's always possible there are bugs in any given
release that might produce corruption - whether it's currently affected one
user or one thousand.
The releases are tested and that's great but when the times comes to say
"whoops we missed a bug" it's not acceptable to say to users with assets
bound up in a repository "never mind that bugs gone now".
It seems a couple of tools might be needed.
1. Cheap verification. Most likely achieved by supporting verify revision
ranges so that recent revs can be verified to detect errors earlier. This
sounds like it's on the way.
2. Repo scraper. A dump option that dumps revisions while they good, skips
revisions when they're bad and keeps content in subsequent good revisions
that is recoverable - eg skips rather than diffs. Let users know revision
timestamp info to suggest that some or all data newer than these times is
missing.
Cheap verification dramatically reduces the urgency of providing what could
be a fairly complex recovery tool.
If repo content can be verified more often then users will get more
opportunity to use their backups productively.
--
Talden
On 7/5/07, Toby Thain <to...@smartgames.ca> wrote:
>
>
> On 4-Jul-07, at 2:06 PM, Les Mikesell wrote:
>
> > Melikian, Chris wrote:
> >> That is a 100% correct description of my gripe. I'm stuck with a
> >> repository I can't dump because a revision is corrupt and can't be
> >> fixed
> >> with fsfsverify, a 3rd party tool with no support (officially but
> >> John
> >> has been very helpful!).
> >> I'm seriously doubting the production readiness of subversion with
> >> fsfs.
> >> I'm going to look into subversion with bdb but I'm facing increasing
> >> pressure to move to ClearCase which I am very loathed to do. I can't
> >> defend subversion/fsfs as it stands.
> >
> > Do you happen to know why you have a corrupt revision? If it is a
> > hardware problem or underlying filesystem issue there's not much
> > reason to think that any other software would handle it any better.
>
> Exactly.
>
> David is one of the few who seems to have experienced a real-life
> corruption, was that definitely due to a software bug? Or is nobody
> sure?
>
> --Toby
>
> >
> > --
> > Les Mikesell
> > lesmikesell@gmail.com
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> > For additional commands, e-mail: users-help@subversion.tigris.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
>
RE: Lack of Subversion repository recovery tools
Posted by Hernán Martínez Foffani <he...@ecc.es>.
>>> Are you using svnsync or any of the incremental backup procedures,
>>> and if so, do they detect the problem on the next run?
>>>
>>
>> Hi Les,
>>
>> No, we've only used svnadmin dump and verify.
>
> I'd think the best approach would be for these tools to include an
> integrity check on the repository and a way to rebuild it from the
> backed up copy if there are problems.
Uh... I've got lost here.
I always thought that if "svnadmin dump" finished its
work without any errors that would mean that the repository
is in good shape and that you can always rebuild it the
hole thing by using "svnadmin load".
Do you mean there's a chance that a succeful dump could
lead in an aborted load?
-H.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Les Mikesell <le...@gmail.com>.
Melikian, Chris wrote:
>
>> Are you using svnsync or any of the incremental backup
>> procedures, and
>> if so, do they detect the problem on the next run?
>>
>
> Hi Les,
>
> No, we've only used svnadmin dump and verify.
I'd think the best approach would be for these tools to include an
integrity check on the repository and a way to rebuild it from the
backed up copy if there are problems.
--
Les Mikesell
lesmikesell@gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Lack of Subversion repository recovery tools
Posted by "Melikian, Chris" <Ch...@uk.fid-intl.com>.
> -----Original Message-----
> From: Les Mikesell [mailto:lesmikesell@gmail.com]
> Sent: 05 July 2007 13:54
> To: Melikian, Chris
> Cc: Toby Thain; users
> Subject: Re: Lack of Subversion repository recovery tools
> Are you using svnsync or any of the incremental backup
> procedures, and
> if so, do they detect the problem on the next run?
>
> --
> Les Mikesell
> lesmikesell@gmail.com
>
Hi Les,
No, we've only used svnadmin dump and verify.
Cheers,
Chris Melikian
Mail zone XTW2A
Important: Fidelity Investments International (Reg. No.1448245),
Fidelity Investment Services Limited (Reg. No. 2016555), Fidelity
Pensions Management (Reg. No. 2015142) and Financial Administration
Services Limited (Reg. No. 1629709, a Fidelity Group company) are all
registered in England and Wales, are authorised and regulated in the UK
by the Financial Services Authority and have their registered offices at
Oakhill House, 130 Tonbridge Road, Hildenborough, Tonbridge, Kent TN11
9DZ. Tel 01732 361144. Fidelity only gives information on products and
does not give investment advice to private clients based on individual
circumstances. Any comments or statements made are not necessarily those
of Fidelity. The information transmitted is intended only for the person
or entity to which it is addressed and may contain confidential and/or
privileged material. If you received this in error, please contact the
sender and delete the material from any computer. All e-mails sent from
or to Fidelity may be subject to our monitoring procedures. Direct link
to Fidelity's website -
http://www.fidelity-international.com/world/index.html
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Les Mikesell <le...@gmail.com>.
Melikian, Chris wrote:
>>
>> David is one of the few who seems to have experienced a real-life
>> corruption, was that definitely due to a software bug? Or is nobody
>> sure?
>>
>> --Toby
>
> Who's David?
>
> Our corruptions occurred in our continuous integration environment when
> cruise/ant was committing binaries into svn. It's possible either that
> the cruise control process was stopped or that the svn server was
> restarted but I suspect the former.
>
> What I have found is that the problem almost always occurs on
> large(12Mb) binary check-ins.
Are you using svnsync or any of the incremental backup procedures, and
if so, do they detect the problem on the next run?
--
Les Mikesell
lesmikesell@gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Re: Lack of Subversion repository recovery tools
Posted by "Melikian, Chris" <Ch...@uk.fid-intl.com>.
> -----Original Message-----
> From: Toby Thain [mailto:toby@smartgames.ca]
> Sent: 04 July 2007 18:56
> To: Les Mikesell
> Cc: users
> Subject: Re: Lack of Subversion repository recovery tools
>
> ...snip...
>
> Exactly.
>
> David is one of the few who seems to have experienced a real-life
> corruption, was that definitely due to a software bug? Or is nobody
> sure?
>
> --Toby
Who's David?
Our corruptions occurred in our continuous integration environment when
cruise/ant was committing binaries into svn. It's possible either that
the cruise control process was stopped or that the svn server was
restarted but I suspect the former.
What I have found is that the problem almost always occurs on
large(12Mb) binary check-ins.
Cheers,
Chris Melikian
Important: Fidelity Investments International (Reg. No.1448245),
Fidelity Investment Services Limited (Reg. No. 2016555), Fidelity
Pensions Management (Reg. No. 2015142) and Financial Administration
Services Limited (Reg. No. 1629709, a Fidelity Group company) are all
registered in England and Wales, are authorised and regulated in the UK
by the Financial Services Authority and have their registered offices at
Oakhill House, 130 Tonbridge Road, Hildenborough, Tonbridge, Kent TN11
9DZ. Tel 01732 361144. Fidelity only gives information on products and
does not give investment advice to private clients based on individual
circumstances. Any comments or statements made are not necessarily those
of Fidelity. The information transmitted is intended only for the person
or entity to which it is addressed and may contain confidential and/or
privileged material. If you received this in error, please contact the
sender and delete the material from any computer. All e-mails sent from
or to Fidelity may be subject to our monitoring procedures. Direct link
to Fidelity's website -
http://www.fidelity-international.com/world/index.html
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Toby Thain <to...@smartgames.ca>.
On 4-Jul-07, at 2:06 PM, Les Mikesell wrote:
> Melikian, Chris wrote:
>> That is a 100% correct description of my gripe. I'm stuck with a
>> repository I can't dump because a revision is corrupt and can't be
>> fixed
>> with fsfsverify, a 3rd party tool with no support (officially but
>> John
>> has been very helpful!).
>> I'm seriously doubting the production readiness of subversion with
>> fsfs.
>> I'm going to look into subversion with bdb but I'm facing increasing
>> pressure to move to ClearCase which I am very loathed to do. I can't
>> defend subversion/fsfs as it stands.
>
> Do you happen to know why you have a corrupt revision? If it is a
> hardware problem or underlying filesystem issue there's not much
> reason to think that any other software would handle it any better.
Exactly.
David is one of the few who seems to have experienced a real-life
corruption, was that definitely due to a software bug? Or is nobody
sure?
--Toby
>
> --
> Les Mikesell
> lesmikesell@gmail.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Les Mikesell <le...@gmail.com>.
Melikian, Chris wrote:
>
> That is a 100% correct description of my gripe. I'm stuck with a
> repository I can't dump because a revision is corrupt and can't be fixed
> with fsfsverify, a 3rd party tool with no support (officially but John
> has been very helpful!).
>
> I'm seriously doubting the production readiness of subversion with fsfs.
> I'm going to look into subversion with bdb but I'm facing increasing
> pressure to move to ClearCase which I am very loathed to do. I can't
> defend subversion/fsfs as it stands.
Do you happen to know why you have a corrupt revision? If it is a
hardware problem or underlying filesystem issue there's not much reason
to think that any other software would handle it any better.
--
Les Mikesell
lesmikesell@gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Re: Lack of Subversion repository recovery tools
Posted by "Melikian, Chris" <Ch...@uk.fid-intl.com>.
Andreas Hasenack said:-
> svnadmin dump also stops there, and I believe this was the original
> complaint. The OP asked if there was a way for svnadmin verify to mark
> that revision as bad, skip it and let svnadmin dump dump the rest. It
> would be a waste if all the other 150k revisions didn't depend on this
> one and just because of this couldn't be dumped.
>
That is a 100% correct description of my gripe. I'm stuck with a
repository I can't dump because a revision is corrupt and can't be fixed
with fsfsverify, a 3rd party tool with no support (officially but John
has been very helpful!).
I'm seriously doubting the production readiness of subversion with fsfs.
I'm going to look into subversion with bdb but I'm facing increasing
pressure to move to ClearCase which I am very loathed to do. I can't
defend subversion/fsfs as it stands.
Cheers, Chris.
Important: Fidelity Investments International (Reg. No.1448245),
Fidelity Investment Services Limited (Reg. No. 2016555), Fidelity
Pensions Management (Reg. No. 2015142) and Financial Administration
Services Limited (Reg. No. 1629709, a Fidelity Group company) are all
registered in England and Wales, are authorised and regulated in the UK
by the Financial Services Authority and have their registered offices at
Oakhill House, 130 Tonbridge Road, Hildenborough, Tonbridge, Kent TN11
9DZ. Tel 01732 361144. Fidelity only gives information on products and
does not give investment advice to private clients based on individual
circumstances. Any comments or statements made are not necessarily those
of Fidelity. The information transmitted is intended only for the person
or entity to which it is addressed and may contain confidential and/or
privileged material. If you received this in error, please contact the
sender and delete the material from any computer. All e-mails sent from
or to Fidelity may be subject to our monitoring procedures. Direct link
to Fidelity's website -
http://www.fidelity-international.com/world/index.html
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Wed, Jul 04, 2007 at 03:40:58AM -0500, Ryan Schmidt wrote:
>> I guess I don't understand why dir2 depends on dir1 since both are
>> subdirectories of /. It's not /dir1/dir2: it's /dir1; /dir2.
>
> Of course dir2 does not depend on dir1 in this scenario.
>
> If you had asked Subversion to check out only dir2, à la...
>
> svn co file://$(pwd)/test-repo/dir2
>
> ...it would have succeeded. That's what Erik is trying to say when he says
> "if a damaged revision is not required to construct a later revision, it's
ok
> simply skipped and no damage will be detected." If the revision creating
> dir1 is corrupted, but you only ask to check out the unrelated dir2, then
> no error is reported and it works fine.
>
> But that's not what you asked. You asked Subversion to verify all
> revisions. And it reported an error in revision 1. Which is correct,
> because revision 1 was corrupted.
And it stopped there, even though I asked it to verify all revisions.
svnadmin dump also stops there, and I believe this was the original
complaint. The OP asked if there was a way for svnadmin verify to mark
that revision as bad, skip it and let svnadmin dump dump the rest. It
would be a waste if all the other 150k revisions didn't depend on this
one and just because of this couldn't be dumped.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Ryan Schmidt <su...@ryandesign.com>.
On Jul 3, 2007, at 10:41, Andreas Hasenack wrote:
> On Jul 3, 2007, at 10:19, Erik Huelsmann wrote:
>
>> On 7/2/07, Andreas Hasenack wrote:
>>
>>>>> Do you think this is unreasonable?
>>>>
>>>> Absolutely not! It's a very reasonable request! The only thing
>>>> is that
>>>> this is the default in-production behaviour already: if a damaged
>>>> revision is not required to construct a later revision, it's simply
>>>> skipped and no damage will be detected.
>>>
>>> Is it? Maybe I don't have the deltas inner workings correctly in my
>>> mind, but consider this simple test:
>>>
>>> [andreas@pandora x]$ svnadmin create test-repo
>>> [andreas@pandora x]$ svn mkdir file://$(pwd)/test-repo/dir1 -m "-
>>> created dir 1"
>>>
>>> Commit da revisão 1.
>>> [andreas@pandora x]$ svn mkdir file://$(pwd)/test-repo/dir2 -m "-
>>> created dir 2"
>>>
>>> Commit da revisão 2.
>>> [andreas@pandora x]$ echo corrupt > test-repo/db/revs/1
>>> [andreas@pandora x]$ svnadmin verify test-repo/
>>> * Verified revision 0.
>>> svnadmin: Can't set position pointer in file 'test-repo/db/revs/
>>> 1': Invalid argument
>>>
>>> Does the dir2 creation depend on dir1? If yes, then this error is
>>> valid.
>>
>> Ofcourse you get an error there. I said "if a damaged revision is not
>> required to construct a later revision, ...". What you're doing there
>> is subsequently checking out *all* revisions, meaning that the
>> damaged
>> revision itself and all revisions based on it are checked out. And
>> still you expect no errors?
>
> I guess I don't understand why dir2 depends on dir1 since both are
> subdirectories of /. It's not /dir1/dir2: it's /dir1; /dir2.
Of course dir2 does not depend on dir1 in this scenario.
If you had asked Subversion to check out only dir2, à la...
svn co file://$(pwd)/test-repo/dir2
...it would have succeeded. That's what Erik is trying to say when he
says "if a damaged revision is not required to construct a later
revision, it's simply skipped and no damage will be detected." If the
revision creating dir1 is corrupted, but you only ask to check out
the unrelated dir2, then no error is reported and it works fine.
But that's not what you asked. You asked Subversion to verify all
revisions. And it reported an error in revision 1. Which is correct,
because revision 1 was corrupted.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Re: Lack of Subversion repository recovery tools
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Tue, Jul 03, 2007 at 05:19:23PM +0200, Erik Huelsmann wrote:
>> Does the dir2 creation depend on dir1? If yes, then this error is valid.
>
> Ofcourse you get an error there. I said "if a damaged revision is not
> required to construct a later revision, ...". What you're doing there
> is subsequently checking out *all* revisions, meaning that the damaged
> revision itself and all revisions based on it are checked out. And
> still you expect no errors?
I guess I don't understand why dir2 depends on dir1 since both are
subdirectories of /. It's not /dir1/dir2: it's /dir1; /dir2.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Re: Lack of Subversion repository recovery tools
Posted by Erik Huelsmann <eh...@gmail.com>.
On 7/2/07, Andreas Hasenack <an...@mandriva.com.br> wrote:
> On Sat, Jun 30, 2007 at 09:24:59PM +0200, Erik Huelsmann wrote:
> > Hi Chris,
> >
> >> No of course things happen which we have no control over. That's part of
> >> the fun/headache of IT right? ;-)
> >
> > Yes :-)
> >
> >> What I am saying I that I am surprised there are no tools available to
> >> be able to recover a repository into a usable state. A simple(no such
> >> thing I know) tool is all that is needed to be able to mark a revision
> >> as bad and let and svnadmin dump/verify and the other tools work as they
> >> should.
> >>
> >> Do you think this is unreasonable?
> >
> > Absolutely not! It's a very reasonable request! The only thing is that
> > this is the default in-production behaviour already: if a damaged
> > revision is not required to construct a later revision, it's simply
> > skipped and no damage will be detected.
>
> Is it? Maybe I don't have the deltas inner workings correctly in my
> mind, but consider this simple test:
>
> [andreas@pandora x]$ svnadmin create test-repo
> [andreas@pandora x]$ svn mkdir file://$(pwd)/test-repo/dir1 -m "- created dir 1"
>
> Commit da revisão 1.
> [andreas@pandora x]$ svn mkdir file://$(pwd)/test-repo/dir2 -m "- created dir 2"
>
> Commit da revisão 2.
> [andreas@pandora x]$ echo corrupt > test-repo/db/revs/1
> [andreas@pandora x]$ svnadmin verify test-repo/
> * Verified revision 0.
> svnadmin: Can't set position pointer in file 'test-repo/db/revs/1': Invalid argument
>
> Does the dir2 creation depend on dir1? If yes, then this error is valid.
Ofcourse you get an error there. I said "if a damaged revision is not
required to construct a later revision, ...". What you're doing there
is subsequently checking out *all* revisions, meaning that the damaged
revision itself and all revisions based on it are checked out. And
still you expect no errors?
bye,
Erik.
Re: Re: Lack of Subversion repository recovery tools
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Sat, Jun 30, 2007 at 09:24:59PM +0200, Erik Huelsmann wrote:
> Hi Chris,
>
>> No of course things happen which we have no control over. That's part of
>> the fun/headache of IT right? ;-)
>
> Yes :-)
>
>> What I am saying I that I am surprised there are no tools available to
>> be able to recover a repository into a usable state. A simple(no such
>> thing I know) tool is all that is needed to be able to mark a revision
>> as bad and let and svnadmin dump/verify and the other tools work as they
>> should.
>>
>> Do you think this is unreasonable?
>
> Absolutely not! It's a very reasonable request! The only thing is that
> this is the default in-production behaviour already: if a damaged
> revision is not required to construct a later revision, it's simply
> skipped and no damage will be detected.
Is it? Maybe I don't have the deltas inner workings correctly in my
mind, but consider this simple test:
[andreas@pandora x]$ svnadmin create test-repo
[andreas@pandora x]$ svn mkdir file://$(pwd)/test-repo/dir1 -m "- created dir 1"
Commit da revisão 1.
[andreas@pandora x]$ svn mkdir file://$(pwd)/test-repo/dir2 -m "- created dir 2"
Commit da revisão 2.
[andreas@pandora x]$ echo corrupt > test-repo/db/revs/1
[andreas@pandora x]$ svnadmin verify test-repo/
* Verified revision 0.
svnadmin: Can't set position pointer in file 'test-repo/db/revs/1': Invalid argument
Does the dir2 creation depend on dir1? If yes, then this error is valid.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Re: Lack of Subversion repository recovery tools
Posted by Erik Huelsmann <eh...@gmail.com>.
Hi Chris,
> No of course things happen which we have no control over. That's part of
> the fun/headache of IT right? ;-)
Yes :-)
> What I am saying I that I am surprised there are no tools available to
> be able to recover a repository into a usable state. A simple(no such
> thing I know) tool is all that is needed to be able to mark a revision
> as bad and let and svnadmin dump/verify and the other tools work as they
> should.
>
> Do you think this is unreasonable?
Absolutely not! It's a very reasonable request! The only thing is that
this is the default in-production behaviour already: if a damaged
revision is not required to construct a later revision, it's simply
skipped and no damage will be detected.
With FSFS all changes in one revision are stored in one 'revision'
file. With skip-deltas [1] the chances become bigger that a large
number of those revision files have to be consulted, both on larger
trees (because there are more elements which have dependencies on
earlier revisions) and on larger numbers of commits (because chances
increase that different files in the same checkout will have been
modified in different revisions).
This makes FSFS more sensitive to broken revision files than the BDB
backend, which stores all file changes in one big database. If part of
that database gets damaged, you may only notice when you're checking
out the damaged file(s), but *only* when the revision you're checking
out actually depends on the damaged revisions.
So, we actually already try our hardest - even in production - to give
you all the output we can give you. With skip-deltas it's almost
impossible to actually mark a given revision as 'bad' and just proceed
with what we do have.
I hope that clarifies things a bit. The above is also an answer to the
'--try-harder' proposal earlier in the thread: we already do.
bye,
Erik.
[1] http://svn.collab.net/repos/svn/trunk/notes/skip-deltas
PS: Possibly, you can't help it, but could you please post without the
extremely long disclaimer? It has no legal value, especially on an
open and archived mailing list like this.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Re: Lack of Subversion repository recovery tools
Posted by "Melikian, Chris" <Ch...@uk.fid-intl.com>.
Hi Erik,
No of course things happen which we have no control over. That's part of
the fun/headache of IT right? ;-)
What I am saying I that I am surprised there are no tools available to
be able to recover a repository into a usable state. A simple(no such
thing I know) tool is all that is needed to be able to mark a revision
as bad and let and svnadmin dump/verify and the other tools work as they
should.
Do you think this is unreasonable?
Cheers,
Chris Melikian
Important: Fidelity Investments International (Reg. No.1448245),
Fidelity Investment Services Limited (Reg. No. 2016555), Fidelity
Pensions Management (Reg. No. 2015142) and Financial Administration
Services Limited (Reg. No. 1629709, a Fidelity Group company) are all
registered in England and Wales, are authorised and regulated in the UK
by the Financial Services Authority and have their registered offices at
Oakhill House, 130 Tonbridge Road, Hildenborough, Tonbridge, Kent TN11
9DZ. Tel 01732 361144. Fidelity only gives information on products and
does not give investment advice to private clients based on individual
circumstances. Any comments or statements made are not necessarily those
of Fidelity. The information transmitted is intended only for the person
or entity to which it is addressed and may contain confidential and/or
privileged material. If you received this in error, please contact the
sender and delete the material from any computer. All e-mails sent from
or to Fidelity may be subject to our monitoring procedures. Direct link
to Fidelity's website -
http://www.fidelity-international.com/world/index.html
-----Original Message-----
From: Erik Huelsmann [mailto:ehuels@gmail.com]
Sent: 29 June 2007 11:31
To: Melikian, Chris
Cc: Ulrich Eckhardt; users@subversion.tigris.org
Subject: Re: Re: Lack of Subversion repository recovery tools
On 6/29/07, Melikian, Chris <Ch...@uk.fid-intl.com> wrote:
> Hi Ulrich,
>
> I completely agree that you need to use backups but I think that
subversion should also have some tools to repair revisions so that you
can at least get the repository back into a consistent state even if
there is data loss.
>
> Surely, any production-ready version control system has such a tool?
And, given hardware failure (RAM chips?) how do you determine
consistency? Subversion provides tools to asses
consistency/correctness and prevents getting into an inconsistent
state under 'normal operation', but you can't expect it to be able to
recover from situations which aren't predefined, right?
Or would 'cutting the repository off at the last valid revision' also
match your requirement? (it sure doesn't sound like any help to me!)
bye,
Erik.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Re: Lack of Subversion repository recovery tools
Posted by Erik Huelsmann <eh...@gmail.com>.
On 6/29/07, Melikian, Chris <Ch...@uk.fid-intl.com> wrote:
> Hi Ulrich,
>
> I completely agree that you need to use backups but I think that subversion should also have some tools to repair revisions so that you can at least get the repository back into a consistent state even if there is data loss.
>
> Surely, any production-ready version control system has such a tool?
And, given hardware failure (RAM chips?) how do you determine
consistency? Subversion provides tools to asses
consistency/correctness and prevents getting into an inconsistent
state under 'normal operation', but you can't expect it to be able to
recover from situations which aren't predefined, right?
Or would 'cutting the repository off at the last valid revision' also
match your requirement? (it sure doesn't sound like any help to me!)
bye,
Erik.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Re: Lack of Subversion repository recovery tools
Posted by "Melikian, Chris" <Ch...@uk.fid-intl.com>.
Hi Ulrich,
I completely agree that you need to use backups but I think that subversion should also have some tools to repair revisions so that you can at least get the repository back into a consistent state even if there is data loss.
Surely, any production-ready version control system has such a tool?
Chris Melikian
Important: Fidelity Investments International (Reg. No.1448245), Fidelity Investment Services Limited (Reg. No. 2016555), Fidelity Pensions Management (Reg. No. 2015142) and Financial Administration Services Limited (Reg. No. 1629709, a Fidelity Group company) are all registered in England and Wales, are authorised and regulated in the UK by the Financial Services Authority and have their registered offices at Oakhill House, 130 Tonbridge Road, Hildenborough, Tonbridge, Kent TN11 9DZ. Tel 01732 361144. Fidelity only gives information on products and does not give investment advice to private clients based on individual circumstances. Any comments or statements made are not necessarily those of Fidelity. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. If you received this in error, please contact the sender and delete the material from any computer. All e-mails sent from or to Fidelity may be subject to our monitoring procedures. Direct link to Fidelity's website - http://www.fidelity-international.com/world/index.html
-----Original Message-----
From: Ulrich Eckhardt [mailto:eckhardt@satorlaser.com]
Sent: 29 June 2007 08:00
To: users@subversion.tigris.org
Subject: Re: Lack of Subversion repository recovery tools
On Thursday 28 June 2007 17:55, Melikian, Chris wrote:
> On the other hand, I've become very disillusioned with the lack of
> documentation and tools on how to recover from corrupted revisions.
Huh? To me that was always clear:
- Create new repository.
- Load last backup into repository.
- Dump the revisions since the backup incrementally.
- Load the incremental revisions into new repository.
- Copy over configuration from old repository.
Just in case there is any confusion about the point: _Subversion is NOT a
replacement for backups_! If you don't do backups, you better start doing so
now, Subversion can't do it for you.
Uli
--
Sator Laser GmbH
Geschäftsführer: Ronald Boers, Amtsgericht Hamburg HR B62 932
**************************************************************************************
Visit our website at <http://www.satorlaser.de/>
**************************************************************************************
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich.
**************************************************************************************
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Ulrich Eckhardt <ec...@satorlaser.com>.
On Thursday 28 June 2007 17:55, Melikian, Chris wrote:
> On the other hand, I've become very disillusioned with the lack of
> documentation and tools on how to recover from corrupted revisions.
Huh? To me that was always clear:
- Create new repository.
- Load last backup into repository.
- Dump the revisions since the backup incrementally.
- Load the incremental revisions into new repository.
- Copy over configuration from old repository.
Just in case there is any confusion about the point: _Subversion is NOT a
replacement for backups_! If you don't do backups, you better start doing so
now, Subversion can't do it for you.
Uli
--
Sator Laser GmbH
Geschäftsführer: Ronald Boers, Amtsgericht Hamburg HR B62 932
**************************************************************************************
Visit our website at <http://www.satorlaser.de/>
**************************************************************************************
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht verantwortlich.
**************************************************************************************
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by da...@jpmorgan.com.
Toby wrote:
>> On 29-Jun-07, at 12:26 PM, david.x.grierson@jpmorgan.com wrote:
>> Consider the following scenario: Let's say you're running version
>> 1.4.7
>> and the latest version is 1.5.3.
>>
>> 1.4.7 contains a defect which under certain circumstances can cause
>> the
>> repository to become corrupted.
>>
>> This defect was identified in version 1.4.8 but wasn't actually fixed
>> until version 1.5.0 because it required a re-design of aspects of the
>> repository format which wouldn't have been appropriate to issue in
>> a point
>> release of the product.
>
>Very unlikely.
>
>In the more likely scenario of a corruption bug in 1.4.7, if it were
>serious, it would merit a 1.4.8.
Okay - just ran into the "read length line" error for the first time
today.
A developer in one of the teams that use our SubVersion repositories
couldn't checkout a directory which appeared to have become corrupted over
30 revisions before the HEAD.
I had to dig around for 1) information on this corruption 2) whether any
tools for correcting the corruption existed or would I have to start
hacking around with the revision file and a copy of the revision format
description trying to figure out what was up?
There appears to have been enough of a need for a fixer tool that John
Szakmeister has developed a tool for fixing the corruption:
http://www.szakmeister.net/blog/?page_id=16
This appears to be a perfect example of this kind of situation:
*) The developers are not 100% sure as to what is causing the corruption
because they don't have a mechanism for reproducing it - consequently they
are unable to say whether it's actually fixed yet AFAIK.
*) The corruption is serious enough that the revision in question (and any
dependent revisions) are broken until the corruption is fixed.
All that's required is a "this is a known issue - here's a
path/URL/description" to be spat out by a "svnadmin verify" command and
take the fsfsverify tool John's written into the distribution while they
look for a fix. This would allow them to state that there is a supported
workaround while they are looking for the full resolution to the issue.
This would have certainly made my life easier and slighly less scarey
today.
John - if you're still reading this forum - thanks so much for fsfsverify
it's saved my day!
Dg.
--
David Grierson
JPMorgan - IB Architecture - Source Code Management Consultant
GDP 228-5574 / DDI +44 141 228 5574 / Email david.x.grierson@jpmorgan.com
Sentinel House 2nd floor, 103 Waterloo Street, Glasgow G2 7BW
This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates.
This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you.
Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to UK legal entities.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Toby Thain <to...@smartgames.ca>.
On 29-Jun-07, at 12:26 PM, david.x.grierson@jpmorgan.com wrote:
> Toby wrote:
>> On 29-Jun-07, at 11:35 AM, david.x.grierson@jpmorgan.com wrote:
>>> ...
>>> check-state against the FSF database to ensure that it's in a
>>> consistent
>>> state - and knew what kind of common issues might exist and what the
>>> regular fixes for those might be.
>>
>> And this is why I single out "software causes", because if you KNOW
>> what the common "issues" are, THEY GET FIXED, and the administrative
>> remedy is to install the fix. Additionally, in the case of a known
>> bug, it may be possible to characterise the damage caused, and make a
>> tool (if necessary) to repair this *specific* damage.
>
> Except that upgrading is time costly (testing, validation, arranging
> downtime, etc. etc.) and so some organisations (e.g. investment
> banks) are
> upgrade unfriendly so keeping on the latest & greatest isn't always an
> option.
>
> Consider the following scenario: Let's say you're running version
> 1.4.7
> and the latest version is 1.5.3.
>
> 1.4.7 contains a defect which under certain circumstances can cause
> the
> repository to become corrupted.
>
> This defect was identified in version 1.4.8 but wasn't actually fixed
> until version 1.5.0 because it required a re-design of aspects of the
> repository format which wouldn't have been appropriate to issue in
> a point
> release of the product.
Very unlikely.
In the more likely scenario of a corruption bug in 1.4.7, if it were
serious, it would merit a 1.4.8.
--Toby
>
> If an organisation were to experience this corruption whilst
> running 1.4.7
> - what should they do?
>
> Option A) Upgrade their servers to (at least) 1.5.0 to get the
> defect fix
> and go through all of the pain associated with the upgrade (testing
> of all
> new features, dump/load of repositories, etc. etc.)
>
> Option B) Have a fixer tool included in version 1.4.8 which
> understands
> repository formats which may experience this corruption (or
> downloadable
> as a support tool for circumstances such as these).
>
> I'd say it's more mature for any SCM vendor (be they open-source,
> commercial or otherwise) to acknowledge such an issue, release a tool
> which can repair any damage which *could* occur while they work on
> a full
> fix.
>
> Dg.
>
>
> This communication is for informational purposes only. It is not
> intended as an offer or solicitation for the purchase or sale of
> any financial instrument or as an official confirmation of any
> transaction. All market prices, data and other information are not
> warranted as to completeness or accuracy and are subject to change
> without notice. Any comments or statements made herein do not
> necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
> and affiliates.
>
> This transmission may contain information that is privileged,
> confidential, legally privileged, and/or exempt from disclosure
> under applicable law. If you are not the intended recipient, you
> are hereby notified that any disclosure, copying, distribution, or
> use of the information contained herein (including any reliance
> thereon) is STRICTLY PROHIBITED. Although this transmission and any
> attachments are believed to be free of any virus or other defect
> that might affect any computer system into which it is received and
> opened, it is the responsibility of the recipient to ensure that it
> is virus free and no responsibility is accepted by JPMorgan Chase &
> Co., its subsidiaries and affiliates, as applicable, for any loss
> or damage arising in any way from its use. If you received this
> transmission in error, please immediately contact the sender and
> destroy the material in its entirety, whether in electronic or hard
> copy format. Thank you.
> Please refer to http://www.jpmorgan.com/pages/disclosures for
> disclosures relating to UK legal entities.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by da...@jpmorgan.com.
Toby wrote:
> On 29-Jun-07, at 11:35 AM, david.x.grierson@jpmorgan.com wrote:
> > ...
> > check-state against the FSF database to ensure that it's in a
> > consistent
> > state - and knew what kind of common issues might exist and what the
> > regular fixes for those might be.
>
> And this is why I single out "software causes", because if you KNOW
> what the common "issues" are, THEY GET FIXED, and the administrative
> remedy is to install the fix. Additionally, in the case of a known
> bug, it may be possible to characterise the damage caused, and make a
> tool (if necessary) to repair this *specific* damage.
Except that upgrading is time costly (testing, validation, arranging
downtime, etc. etc.) and so some organisations (e.g. investment banks) are
upgrade unfriendly so keeping on the latest & greatest isn't always an
option.
Consider the following scenario: Let's say you're running version 1.4.7
and the latest version is 1.5.3.
1.4.7 contains a defect which under certain circumstances can cause the
repository to become corrupted.
This defect was identified in version 1.4.8 but wasn't actually fixed
until version 1.5.0 because it required a re-design of aspects of the
repository format which wouldn't have been appropriate to issue in a point
release of the product.
If an organisation were to experience this corruption whilst running 1.4.7
- what should they do?
Option A) Upgrade their servers to (at least) 1.5.0 to get the defect fix
and go through all of the pain associated with the upgrade (testing of all
new features, dump/load of repositories, etc. etc.)
Option B) Have a fixer tool included in version 1.4.8 which understands
repository formats which may experience this corruption (or downloadable
as a support tool for circumstances such as these).
I'd say it's more mature for any SCM vendor (be they open-source,
commercial or otherwise) to acknowledge such an issue, release a tool
which can repair any damage which *could* occur while they work on a full
fix.
Dg.
This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates.
This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you.
Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to UK legal entities.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Toby Thain <to...@smartgames.ca>.
On 29-Jun-07, at 11:35 AM, david.x.grierson@jpmorgan.com wrote:
> ...
> check-state against the FSF database to ensure that it's in a
> consistent
> state - and knew what kind of common issues might exist and what the
> regular fixes for those might be.
And this is why I single out "software causes", because if you KNOW
what the common "issues" are, THEY GET FIXED, and the administrative
remedy is to install the fix. Additionally, in the case of a known
bug, it may be possible to characterise the damage caused, and make a
tool (if necessary) to repair this *specific* damage.
A scavenger against issues caused by *unknown* bugs is pure
guesswork, so a cost/benefit analysis seems to apply.
--Toby
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by da...@jpmorgan.com.
Erik Wrote:
>On 6/29/07, david.x.grierson@jpmorgan.com <da...@jpmorgan.com>
wrote:
>> Eventually (a week or so later) they were unable to modify the objects
>> which had become corrupted. Fortunately the SCM system had tools for
going
>> through the repository identifying the components which were corrupted
and
>> then recovering their contents appropriately.
>
>Subversion protects you of problems like these: it sends over a delta
>and the md5 of the resulting fulltext if you apply the delta to the
>base-fulltext. Any additional or replaced bytes would result in md5
>checksum errors.
This would be exactly what I would hope for.
>> If you're in a job where you're able to turn around to management and
say
>> "sorry we've lost the last week's worth of development & it'll all just
>> have to be done again because our SCM system doesn't let us fix things"
>> then you're in a position of real power.
>
>How would you want the SCM tool to address an issue like this?
>
>(I'm quite serious here: how would your SCM tool know the NUL
>characters don't belong there, more importantly, how would it know
>which characters DO belong there?)
A completely fair point and one I agree completely with.
My point, however, was that *if* SVN wasn't quite so resilient to perform
the MD5 (which it is so it's not really a valid point but I'll plough on
regardless :-), and just accepted the NULLs then having to go to
management to tell them to restore an old backup because there were no
back-end manipulation tools is not a situation I'd like to be in.
> I do sympathise with the problems raised, but I'm not understanding
> what it is that you're asking for: Subversion WILL let you fix it as
> long as you know which characters should be there, you can always
> check out the wrong data and commit the correct data (which comes as
> close to 'repairing' the data as I can imagine...)
I guess the thing is that by performing the MD5 and transmitting that as
well you've avoided a situation where the client can unexpectedly corrupt
the data.
With other SCM solutions there are ways in which the data can be repaired
- for example manually editing the repository and then running some kind
of forced checksum update tool. This way any internal table of checksums
within the repository would accept that the forced modification was
correct and would then allow the repository to be marked "clean".
There could however be some kind of mechanism for even just performing a
check-state against the FSF database to ensure that it's in a consistent
state - and knew what kind of common issues might exist and what the
regular fixes for those might be.
Kind of like the way on HP-UX you had an inode editor with which you could
perform the lowest level manipulations of the filesystem to correct it -
or you could use fsck to have them automated *where possible* for you.
Dg.
--
David Grierson
JPMorgan - IB Architecture - Source Code Management Consultant
GDP 228-5574 / DDI +44 141 228 5574 / Email david.x.grierson@jpmorgan.com
Sentinel House 2nd floor, 103 Waterloo Street, Glasgow G2 7BW
This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates.
This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you.
Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to UK legal entities.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Erik Huelsmann <eh...@gmail.com>.
On 6/29/07, david.x.grierson@jpmorgan.com <da...@jpmorgan.com> wrote:
> The problem is that hardware faults and errors don't just occur on the
> server side where you have significant control over them - they can often
> occur on the client as well.
>
> I experienced a situation where a repository of another SCM tool was
> corrupted because the client network driver was erroneously inserting
> nulls into files while they were being transferred across the network.
>
> We were initially unaware of the issues because the SCM solution was
> (correctly) recording the contents of the files which were being presented
> to it as the new revision and developers were going on continuing their
> development.
>
> Eventually (a week or so later) they were unable to modify the objects
> which had become corrupted. Fortunately the SCM system had tools for going
> through the repository identifying the components which were corrupted and
> then recovering their contents appropriately.
Subversion protects you of problems like these: it sends over a delta
and the md5 of the resulting fulltext if you apply the delta to the
base-fulltext. Any additional or replaced bytes would result in md5
checksum errors.
> If you're in a job where you're able to turn around to management and say
> "sorry we've lost the last week's worth of development & it'll all just
> have to be done again because our SCM system doesn't let us fix things"
> then you're in a position of real power.
How would you want the SCM tool to address an issue like this?
(I'm quite serious here: how would your SCM tool know the NUL
characters don't belong there, more importantly, how would it know
which characters DO belong there?)
I do sympathise with the problems raised, but I'm not understanding
what it is that you're asking for: Subversion WILL let you fix it as
long as you know which characters should be there, you can always
check out the wrong data and commit the correct data (which comes as
close to 'repairing' the data as I can imagine...)
bye,
Erik.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Toby Thain <to...@smartgames.ca>.
On 29-Jun-07, at 10:44 AM, Andreas Hasenack wrote:
> On Fri, Jun 29, 2007 at 10:39:19AM -0300, Toby Thain wrote:
>>
>> On 29-Jun-07, at 9:40 AM, Andreas Hasenack wrote:
>>
>>> On Fri, Jun 29, 2007 at 09:26:57AM -0300, Toby Thain wrote:
>>>>> FSFS does not have tools
>>>>> to bring it back to a consistent state, which even the worst case
>>>>> scenario of fsck can do.
>>>>
>>>> But what was the root cause of the corruption? Since you are not
>>>> talking
>>>> about a hardware fault, you must be talking about a software fault.
>>>
>>> Why does it matter?
>>
>> IMHO it is relevant to the question of budgeting effort for a
>> recovery tool
>> or scavenger.
>>
>> Because if you are talking about recovering from software faults,
>> then you
>> have to bring the release methodology of the project into the
>> argument,
>
> So, either way, you would say that such a recovery tool is not needed,
> because:
> - if it's a hardware fault, nothing you can do about it
> - if it's a software bug, the project shouldn't have released the
> software in that state
No, it's more subtle than that. I'm not sure the list needs to hear
my rationale again, since the issue is so hypothetical.
But in a nutshell, IMHO:
- hw and sw causes must be treated separately;
- hw causes are boring; if you care about your data you'll be using
prophylactics such as good admin practices and/or ZFS;
- sw causes strongly invoke all the engineering questions I raised
earlier, and until you have examined and answered all those and
others, you cannot know whether a scavenger is justified. The effort
may be much better spent in proactive defenses such as test coverage,
regression tests, runtime consistency checks, preventive design,
prudent release engineering, etc.
If you don't feel this clarifies my position then I'll have to leave
it at that. :)
--Toby
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Fri, Jun 29, 2007 at 10:39:19AM -0300, Toby Thain wrote:
>
> On 29-Jun-07, at 9:40 AM, Andreas Hasenack wrote:
>
>> On Fri, Jun 29, 2007 at 09:26:57AM -0300, Toby Thain wrote:
>>>> FSFS does not have tools
>>>> to bring it back to a consistent state, which even the worst case
>>>> scenario of fsck can do.
>>>
>>> But what was the root cause of the corruption? Since you are not talking
>>> about a hardware fault, you must be talking about a software fault.
>>
>> Why does it matter?
>
> IMHO it is relevant to the question of budgeting effort for a recovery tool
> or scavenger.
>
> Because if you are talking about recovering from software faults, then you
> have to bring the release methodology of the project into the argument,
So, either way, you would say that such a recovery tool is not needed,
because:
- if it's a hardware fault, nothing you can do about it
- if it's a software bug, the project shouldn't have released the
software in that state
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Re: Lack of Subversion repository recovery tools
Posted by "Melikian, Chris" <Ch...@uk.fid-intl.com>.
>Back to the question... It may be that svndump simply needs an option
>to turn on a scavenging mode where it is much more tolerant of
>inconsistency (--tryharder).
Yep, something along those lines so that you can salvage something.
Or an
svnadmin --mark-corrupt-revisions-as-bad verify
command.
Cheers, Chris.
Important: Fidelity Investments International (Reg. No.1448245),
Fidelity Investment Services Limited (Reg. No. 2016555), Fidelity
Pensions Management (Reg. No. 2015142) and Financial Administration
Services Limited (Reg. No. 1629709, a Fidelity Group company) are all
registered in England and Wales, are authorised and regulated in the UK
by the Financial Services Authority and have their registered offices at
Oakhill House, 130 Tonbridge Road, Hildenborough, Tonbridge, Kent TN11
9DZ. Tel 01732 361144. Fidelity only gives information on products and
does not give investment advice to private clients based on individual
circumstances. Any comments or statements made are not necessarily those
of Fidelity. The information transmitted is intended only for the person
or entity to which it is addressed and may contain confidential and/or
privileged material. If you received this in error, please contact the
sender and delete the material from any computer. All e-mails sent from
or to Fidelity may be subject to our monitoring procedures. Direct link
to Fidelity's website -
http://www.fidelity-international.com/world/index.html
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Toby Thain <to...@smartgames.ca>.
On 29-Jun-07, at 11:00 AM, david.x.grierson@jpmorgan.com wrote:
> The problem is that hardware faults and errors don't just occur on the
> server side where you have significant control over them - they can
> often
> occur on the client as well.
>
> I experienced a situation where a repository of another SCM tool was
> corrupted because the client network driver was erroneously inserting
> nulls into files while they were being transferred across the network.
>
> We were initially unaware of the issues because the SCM solution was
> (correctly) recording the contents of the files which were being
> presented
> to it as the new revision and developers were going on continuing
> their
> development.
>
> Eventually (a week or so later) they were unable to modify the objects
> which had become corrupted. Fortunately the SCM system had tools
> for going
> through the repository identifying the components which were
> corrupted and
> then recovering their contents appropriately.
Subversion can recover from this situation without data loss using
its ordinary tools. It's not a situation where server side recovery
is indicated.
(I wonder if your anecdote is about VSS, which by design is just data
loss waiting to happen.)
--Toby
>
> If you're in a job where you're able to turn around to management
> and say
> "sorry we've lost the last week's worth of development & it'll all
> just
> have to be done again because our SCM system doesn't let us fix
> things"
> then you're in a position of real power.
>
> Dg.
> --
> David Grierson
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by da...@jpmorgan.com.
The problem is that hardware faults and errors don't just occur on the
server side where you have significant control over them - they can often
occur on the client as well.
I experienced a situation where a repository of another SCM tool was
corrupted because the client network driver was erroneously inserting
nulls into files while they were being transferred across the network.
We were initially unaware of the issues because the SCM solution was
(correctly) recording the contents of the files which were being presented
to it as the new revision and developers were going on continuing their
development.
Eventually (a week or so later) they were unable to modify the objects
which had become corrupted. Fortunately the SCM system had tools for going
through the repository identifying the components which were corrupted and
then recovering their contents appropriately.
If you're in a job where you're able to turn around to management and say
"sorry we've lost the last week's worth of development & it'll all just
have to be done again because our SCM system doesn't let us fix things"
then you're in a position of real power.
Dg.
--
David Grierson
JPMorgan - IB Architecture - Source Code Management Consultant
GDP 228-5574 / DDI +44 141 228 5574 / Email david.x.grierson@jpmorgan.com
Sentinel House 2nd floor, 103 Waterloo Street, Glasgow G2 7BW
This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates.
This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you.
Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to UK legal entities.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Toby Thain <to...@smartgames.ca>.
On 29-Jun-07, at 9:40 AM, Andreas Hasenack wrote:
> On Fri, Jun 29, 2007 at 09:26:57AM -0300, Toby Thain wrote:
>>> FSFS does not have tools
>>> to bring it back to a consistent state, which even the worst case
>>> scenario of fsck can do.
>>
>> But what was the root cause of the corruption? Since you are not
>> talking
>> about a hardware fault, you must be talking about a software fault.
>
> Why does it matter?
IMHO it is relevant to the question of budgeting effort for a
recovery tool or scavenger.
Because if you are talking about recovering from software faults,
then you have to bring the release methodology of the project into
the argument, such as: test coverage; regression testing; release
criteria; details of repository storage design (fault resilience,
this is also relevant to hw faults); built-in runtime consistency
checks... and so on. Depending on the rigour of all those project
aspects, it's possible to make some fairly confident statements about
the likelihood of software-caused corruption - especially as a stable
release ages.
Back to the question... It may be that svndump simply needs an option
to turn on a scavenging mode where it is much more tolerant of
inconsistency (--tryharder).
> Let's say the backups were whosed too. The poor
> admin will have nothing left, *nothing*, sort of rm -rf the repo and
> restart from scratch. In that scenario, saying "svn+fsfs has no
> recovery
> tool, you are doomed indeed" seems worse than "I tried the recovery
> procedures and was able to get back 30% of the data".
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Fri, Jun 29, 2007 at 09:26:57AM -0300, Toby Thain wrote:
>> FSFS does not have tools
>> to bring it back to a consistent state, which even the worst case
>> scenario of fsck can do.
>
> But what was the root cause of the corruption? Since you are not talking
> about a hardware fault, you must be talking about a software fault.
Why does it matter? Let's say the backups were whosed too. The poor
admin will have nothing left, *nothing*, sort of rm -rf the repo and
restart from scratch. In that scenario, saying "svn+fsfs has no recovery
tool, you are doomed indeed" seems worse than "I tried the recovery
procedures and was able to get back 30% of the data".
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Toby Thain <to...@smartgames.ca>.
On 28-Jun-07, at 7:10 PM, Andreas Hasenack wrote:
> On Thu, Jun 28, 2007 at 06:51:30PM -0300, Toby Thain wrote:
>>
>> On 28-Jun-07, at 5:02 PM, Andreas Hasenack wrote:
>>
>>> On Thu, Jun 28, 2007 at 01:57:16PM -0300, Toby Thain wrote:
>>>>> I have used BDB with very large repositories (~50Gb) and it
>>>>> survived
>>>>> many
>>>>> hardware failures (after running recovery and with proper log
>>>>> files
>>>>> archived). Now I'm also using FSFS, and had to spend about a week
>>>>> manually recovering from corruptions (caused by hardware failure).
>>>>
>>>> Prevention is better than cure... People really should start
>>>> looking more
>>>
>>> Is this the reason why there is no recovery for FSFS?
>>
>> It's not clear how much effort to "cure" hardware-caused problems is
>
> That's not what I meant/asked.
>
>> worthwhile, especially when a solution is available at a lower
>> layer. Any
>> discussion of this topic has to begin with asking, "where are the
>> errors
>> coming from." You mentioned hardware failure. There are two
>> surefire ways
>> of dealing with it: 1) ZFS 2) backups. A scavenging tool that is
>> supposed
>> to cope with arbitrary corruption (i.e. hardware problem) should
>> be a last
>> resort, IMHO? And this scenario is pretty much guaranteed to
>> result in
>> -some- loss. Hence, "Prevention is better than cure".
>
> You seem to imply tools like fsck are useless.
In the case of ZFS, nearly so, yes. In fact fsck does not exist there.
> FSFS does not have tools
> to bring it back to a consistent state, which even the worst case
> scenario of fsck can do.
But what was the root cause of the corruption? Since you are not
talking about a hardware fault, you must be talking about a software
fault.
--Toby
> Even worse, a corruption at revision 1 out of
> 150.000 will make svnadmin dump fail, even when the subsequent commits
> did not depend on that one in particular (i.e., no delta).
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Thu, Jun 28, 2007 at 06:51:30PM -0300, Toby Thain wrote:
>
> On 28-Jun-07, at 5:02 PM, Andreas Hasenack wrote:
>
>> On Thu, Jun 28, 2007 at 01:57:16PM -0300, Toby Thain wrote:
>>>> I have used BDB with very large repositories (~50Gb) and it survived
>>>> many
>>>> hardware failures (after running recovery and with proper log files
>>>> archived). Now I'm also using FSFS, and had to spend about a week
>>>> manually recovering from corruptions (caused by hardware failure).
>>>
>>> Prevention is better than cure... People really should start looking more
>>
>> Is this the reason why there is no recovery for FSFS?
>
> It's not clear how much effort to "cure" hardware-caused problems is
That's not what I meant/asked.
> worthwhile, especially when a solution is available at a lower layer. Any
> discussion of this topic has to begin with asking, "where are the errors
> coming from." You mentioned hardware failure. There are two surefire ways
> of dealing with it: 1) ZFS 2) backups. A scavenging tool that is supposed
> to cope with arbitrary corruption (i.e. hardware problem) should be a last
> resort, IMHO? And this scenario is pretty much guaranteed to result in
> -some- loss. Hence, "Prevention is better than cure".
You seem to imply tools like fsck are useless. FSFS does not have tools
to bring it back to a consistent state, which even the worst case
scenario of fsck can do. Even worse, a corruption at revision 1 out of
150.000 will make svnadmin dump fail, even when the subsequent commits
did not depend on that one in particular (i.e., no delta).
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Toby Thain <to...@smartgames.ca>.
On 28-Jun-07, at 5:02 PM, Andreas Hasenack wrote:
> On Thu, Jun 28, 2007 at 01:57:16PM -0300, Toby Thain wrote:
>>> I have used BDB with very large repositories (~50Gb) and it
>>> survived many
>>> hardware failures (after running recovery and with proper log files
>>> archived). Now I'm also using FSFS, and had to spend about a week
>>> manually recovering from corruptions (caused by hardware failure).
>>
>> Prevention is better than cure... People really should start
>> looking more
>
> Is this the reason why there is no recovery for FSFS?
It's not clear how much effort to "cure" hardware-caused problems is
worthwhile, especially when a solution is available at a lower layer.
Any discussion of this topic has to begin with asking, "where are the
errors coming from." You mentioned hardware failure. There are two
surefire ways of dealing with it: 1) ZFS 2) backups. A scavenging
tool that is supposed to cope with arbitrary corruption (i.e.
hardware problem) should be a last resort, IMHO? And this scenario is
pretty much guaranteed to result in -some- loss. Hence, "Prevention
is better than cure".
--Toby
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Thu, Jun 28, 2007 at 01:57:16PM -0300, Toby Thain wrote:
>> I have used BDB with very large repositories (~50Gb) and it survived many
>> hardware failures (after running recovery and with proper log files
>> archived). Now I'm also using FSFS, and had to spend about a week
>> manually recovering from corruptions (caused by hardware failure).
>
> Prevention is better than cure... People really should start looking more
Is this the reason why there is no recovery for FSFS?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Toby Thain <to...@smartgames.ca>.
On 28-Jun-07, at 1:18 PM, Andreas Hasenack wrote:
> On Thu, Jun 28, 2007 at 04:55:20PM +0100, Melikian, Chris wrote:
>> On the other hand, I've become very disillusioned with the lack of
>> documentation and tools on how to recover from corrupted
>> revisions. The
>> problem we have is that we are trying to an "svnadmin dump" of our
>> repository in order to shrink it with svndumpfilter and we are
>> getting
>>
>> "Error: Invalid diff stream: insn 1245 cannot be decoded"
>>
>> errors. We are using svn 1.3.2 on Solaris with FSFS as the back
>> end and
>> svnserve as our repository access front end.
>
> At least the BDB backend has recovery. FSFS doesn't.
>
> I have used BDB with very large repositories (~50Gb) and it
> survived many
> hardware failures (after running recovery and with proper log files
> archived). Now I'm also using FSFS, and had to spend about a week
> manually recovering from corruptions (caused by hardware failure).
Prevention is better than cure... People really should start looking
more seriously at ZFS (which won't deliver bad data to applications
in the case of hardware failure).
Of course this only applies to people who are really serious about
protecting their data.
--Toby
>
> So, BDB has more buttons to push and dials to adjust, but at least it
> had the recovery option which worked very well. But now I wouldn't
> recommend it anymore because subversion upstream focus more on FSFS.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Lack of Subversion repository recovery tools
Posted by Andreas Hasenack <an...@mandriva.com.br>.
On Thu, Jun 28, 2007 at 04:55:20PM +0100, Melikian, Chris wrote:
> On the other hand, I've become very disillusioned with the lack of
> documentation and tools on how to recover from corrupted revisions. The
> problem we have is that we are trying to an "svnadmin dump" of our
> repository in order to shrink it with svndumpfilter and we are getting
>
> "Error: Invalid diff stream: insn 1245 cannot be decoded"
>
> errors. We are using svn 1.3.2 on Solaris with FSFS as the back end and
> svnserve as our repository access front end.
At least the BDB backend has recovery. FSFS doesn't.
I have used BDB with very large repositories (~50Gb) and it survived many
hardware failures (after running recovery and with proper log files
archived). Now I'm also using FSFS, and had to spend about a week
manually recovering from corruptions (caused by hardware failure).
So, BDB has more buttons to push and dials to adjust, but at least it
had the recovery option which worked very well. But now I wouldn't
recommend it anymore because subversion upstream focus more on FSFS.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org