You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by John Szakmeister <jo...@szakmeister.net> on 2006/10/09 10:06:53 UTC
FSFS corruption on the rise...
Hello all,
I just wanted to put out there that the number of FSFS corruption cases I'm seeing has been increasing. I can't really say by how much, but it seems like a new one is popping up every couple of weeks, whereas it was months between before. The more serious issue is that I'm now seeing cases where there is a large swath of zeros being written into the file. The block size has varied, but it's always on a 4K boundary--just like the repeated block corruption. For instance, one file had 40K worth of zeros written into the front of the file. It seems to be very similar to the repeated block corruption, except this one appears to be destructive (I can't find the original data anywhere). I've also had one person claim that they caused their corruption to happen via svnserve. :-( The number of downloads for fsfsverify has jumped as well. I'm seeing roughly 20 a month, and generally only one of those makes it to the list, so I don't whether they had an issue and fixed it, or if they were just looking at the tool. Also, I've had several people report the corruption with APR >=0.9.7, so the issue is definitely not limited to those running <=0.9.6.
Only one person has been able to reproduce the corruption. They used tc to disrupt the network, and that seemed to trigger the issue. I tried doing the same with a couple machines at home, and was unsuccessful at reproducing the issue.
I wish I could be more involved with a solution--I've been swamped at home and at work, and just can't break away. However, I thought the community should at least know about what I'm seeing. I can try and dig up more details (I generally save all the corruption email messages) if someone needs it.
Thanks for listening!
-John
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FSFS corruption on the rise...
Posted by Mark Phippard <ma...@softlanding.com>.
John Szakmeister <jo...@szakmeister.net> wrote on 10/09/2006 08:44:06 PM:
>
> ----- Mark Phippard <ma...@softlanding.com> wrote:
> [snip]
> > I do not really know anything about it other than what I have read in
the
> > various mail exchanges. The way your message was written it implied
to me
> > that you were unaware of the potential fix. It seems like the main
issue
> > with calling this the fix is that no one has provided a repeatable
> > instance of the problem. If you are working with someone that can
repeat
> > it, perhaps you can assist them with getting a patched 1.4 that has
the
> > fix to see if it solves it?
>
> Sorry if my last message came across offensively Mark. I didn't intend
for it too.
No, not at all. I just knew you had been involved with this from the
start and thought maybe you somehow were away and missed this commit.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FSFS corruption on the rise...
Posted by John Szakmeister <jo...@szakmeister.net>.
----- Mark Phippard <ma...@softlanding.com> wrote:
[snip]
> I do not really know anything about it other than what I have read in the
> various mail exchanges. The way your message was written it implied to me
> that you were unaware of the potential fix. It seems like the main issue
> with calling this the fix is that no one has provided a repeatable
> instance of the problem. If you are working with someone that can repeat
> it, perhaps you can assist them with getting a patched 1.4 that has the
> fix to see if it solves it?
Sorry if my last message came across offensively Mark. I didn't intend for it too.
Yes, that's been the issue for a while (not being able to repeat it). I haven't talked to the guy who could repeat the problem in a little while, but it's probably worth a shot. I'll see if he can help out.
-John
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FSFS corruption on the rise...
Posted by Mark Phippard <ma...@softlanding.com>.
John Szakmeister <jo...@szakmeister.net> wrote on 10/09/2006 07:59:13 PM:
> ----- Mark Phippard <ma...@softlanding.com> wrote:
> > John Szakmeister <jo...@szakmeister.net> wrote on 10/09/2006 06:06:53
> > AM:
> >
> > > I wish I could be more involved with a solution--I've been swamped
> > at
> > home
> > > and at work, and just can't break away. However, I thought the
> > community
> > > should at least know about what I'm seeing. I can try and dig up
> > more
> > > details (I generally save all the corruption email messages) if
> > someone
> > needs it.
> >
> > Have you seen the potential fix that was committed to trunk? Take a
look
> > at r21738 which was committed on October 3.
> >
>
> I actually did. However, I think we should probably get it back-ported
and
> into 1.4.1 release soon. It's concerning that we're seeing actual
> corruption instead of the fully-recoverable repeated block problem.
I do not really know anything about it other than what I have read in the
various mail exchanges. The way your message was written it implied to me
that you were unaware of the potential fix. It seems like the main issue
with calling this the fix is that no one has provided a repeatable
instance of the problem. If you are working with someone that can repeat
it, perhaps you can assist them with getting a patched 1.4 that has the
fix to see if it solves it?
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FSFS corruption on the rise...
Posted by John Szakmeister <jo...@szakmeister.net>.
----- Mark Phippard <ma...@softlanding.com> wrote:
> John Szakmeister <jo...@szakmeister.net> wrote on 10/09/2006 06:06:53
> AM:
>
> > I wish I could be more involved with a solution--I've been swamped
> at
> home
> > and at work, and just can't break away. However, I thought the
> community
> > should at least know about what I'm seeing. I can try and dig up
> more
> > details (I generally save all the corruption email messages) if
> someone
> needs it.
>
> Have you seen the potential fix that was committed to trunk? Take a look
> at r21738 which was committed on October 3.
>
I actually did. However, I think we should probably get it back-ported and into 1.4.1 release soon. It's concerning that we're seeing actual corruption instead of the fully-recoverable repeated block problem.
-John
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FSFS corruption on the rise...
Posted by Mark Phippard <ma...@softlanding.com>.
John Szakmeister <jo...@szakmeister.net> wrote on 10/09/2006 06:06:53 AM:
> I wish I could be more involved with a solution--I've been swamped at
home
> and at work, and just can't break away. However, I thought the
community
> should at least know about what I'm seeing. I can try and dig up more
> details (I generally save all the corruption email messages) if someone
needs it.
Have you seen the potential fix that was committed to trunk? Take a look
at r21738 which was committed on October 3.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org