Corruption when using --threads

[Landon Bradshaw <la...@justleapin.com>]

We've been having a problem with our repository and finally had the time to
do some more testing ... the results are that we needed to drop "--threads"
from our svnserve invocation and just let it run via fork()

Environment: CentOS5
Revision: 1.4.2 (CentOS5) and 1.4.4 (Ubuntu)
DB-type: fsfs

Following these steps I could get a corruption within 10 revisions ...

Setup:
* create the repo
* create 2 subdirectories
* commit a set of text files into Dir1 (20 files all less than 2KB)
* commit a set of large binary files into Dir2 (6 files of 2.5MB to 16MB)

Stress:
* By randomly choosing a text file to change I created a new revision for
Dir1 containing 4 to 9 changed files
* By randomly swapping the filename of 2 files in Dir2 I created a large
changeset for Dir2
* Serial commits of Dir2 were continuously occurring while I was randomly
choosing a sleep time between serial commits of Dir1 (<16 seconds)

I would trigger 10 loops of the large swap/commit of Dir2 while triggering
20 loops of revisions from Dir1 and I would reliably get the corruption
before the 10 loops of the binary commits had finished ... "svn:
Decompression of svndiff data failed"

Removing the "--threads" from the svnserve options and I could get through
over 400 commits without an error ... restarting svnserve with that option
and again I would corrupt my repository within 10 commits ...

Is this worthy of posting a bug report? Does someone want my scripts to test
it out themselves?

...Landon

Re: Corruption when using --threads

[Erik Huelsmann <eh...@gmail.com>]

On Thu, Mar 20, 2008 at 5:46 PM, Landon Bradshaw
<la...@justleapin.com> wrote:
> To setup the text file directory (Dir1) I did the following:
> for i in 1 2 3 4 5 6 7 8 9; { touch chkfile00$i.txt; }
> for i in 10 11 12 13 14 15 16 17 18 19 20; { touch chkfile0$i.txt; }
> svn add *
> svn ci -m 'initial fileset'
>
> Then I used the following command after copying "txt_control.sh" into Dir1
> ./txt-control.sh 20
>
> For Dir2 I just grabbed some large binary files and named them all
> chkspace00#.bin and added them to subversion control ... then used the
> following command:
>  ./bin-control.sh 10
>
> These commands need to be running at the same time ... I was using 2
> different user accounts ...
>
> Please let me know if I missed anything to help recreate this problem ...

Hi,


I did all you mentioned, created a running svnserve -d --threads,
started both scripts, but before I could start txt-control,
bin-control had already finished: there's no waiting loop in it.

then, I introduced a 'sleep 1' on each bin iteration, but that didn't
work either.

Are you able to build 1.5.0-beta1? If so, could you try reproducing
this bug with it? I'm using the most recent 1.5.x code (more recent
than 1.5.0-beta1), which contains a lot of fixes since 1.4.x.

It would be extremely helpful if you could try...

Thanks in advance,


Erik.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Corruption when using --threads

[Landon Bradshaw <la...@justleapin.com>]

To setup the text file directory (Dir1) I did the following:
for i in 1 2 3 4 5 6 7 8 9; { touch chkfile00$i.txt; }
for i in 10 11 12 13 14 15 16 17 18 19 20; { touch chkfile0$i.txt; }
svn add *
svn ci -m 'initial fileset'

Then I used the following command after copying "txt_control.sh" into Dir1
./txt-control.sh 20

For Dir2 I just grabbed some large binary files and named them all
chkspace00#.bin and added them to subversion control ... then used the
following command:
./bin-control.sh 10

These commands need to be running at the same time ... I was using 2
different user accounts ...

Please let me know if I missed anything to help recreate this problem ...

On Thu, Mar 20, 2008 at 12:25 AM, Erik Huelsmann <eh...@gmail.com> wrote:

> On 3/20/08, Landon Bradshaw <la...@justleapin.com> wrote:
> > We've been having a problem with our repository and finally had the time
> to
> > do some more testing ... the results are that we needed to drop
> "--threads"
> > from our svnserve invocation and just let it run via fork()
> >
> > Environment: CentOS5
> > Revision: 1.4.2 (CentOS5) and 1.4.4 (Ubuntu)
> > DB-type: fsfs
> >
> > Following these steps I could get a corruption within 10 revisions ...
> >
> > Setup:
> > * create the repo
> > * create 2 subdirectories
> > * commit a set of text files into Dir1 (20 files all less than 2KB)
> > * commit a set of large binary files into Dir2 (6 files of 2.5MB to
> 16MB)
> >
> > Stress:
> > * By randomly choosing a text file to change I created a new revision
> for
> > Dir1 containing 4 to 9 changed files
> > * By randomly swapping the filename of 2 files in Dir2 I created a large
> > changeset for Dir2
> > * Serial commits of Dir2 were continuously occurring while I was
> randomly
> > choosing a sleep time between serial commits of Dir1 (<16 seconds)
> >
> > I would trigger 10 loops of the large swap/commit of Dir2 while
> triggering
> > 20 loops of revisions from Dir1 and I would reliably get the corruption
> > before the 10 loops of the binary commits had finished ... "svn:
> > Decompression of svndiff data failed"
> >
> > Removing the "--threads" from the svnserve options and I could get
> through
> > over 400 commits without an error ... restarting svnserve with that
> option
> > and again I would corrupt my repository within 10 commits ...
> >
> > Is this worthy of posting a bug report? Does someone want my scripts to
> test
> > it out themselves?
>
> If you can consistently reproduce, we'd - ofcourse - like your scripts!
>
> Bye,
>
> Erik.
>

Re: Corruption when using --threads

[Erik Huelsmann <eh...@gmail.com>]

On 3/20/08, Landon Bradshaw <la...@justleapin.com> wrote:
> We've been having a problem with our repository and finally had the time to
> do some more testing ... the results are that we needed to drop "--threads"
> from our svnserve invocation and just let it run via fork()
>
> Environment: CentOS5
> Revision: 1.4.2 (CentOS5) and 1.4.4 (Ubuntu)
> DB-type: fsfs
>
> Following these steps I could get a corruption within 10 revisions ...
>
> Setup:
> * create the repo
> * create 2 subdirectories
> * commit a set of text files into Dir1 (20 files all less than 2KB)
> * commit a set of large binary files into Dir2 (6 files of 2.5MB to 16MB)
>
> Stress:
> * By randomly choosing a text file to change I created a new revision for
> Dir1 containing 4 to 9 changed files
> * By randomly swapping the filename of 2 files in Dir2 I created a large
> changeset for Dir2
> * Serial commits of Dir2 were continuously occurring while I was randomly
> choosing a sleep time between serial commits of Dir1 (<16 seconds)
>
> I would trigger 10 loops of the large swap/commit of Dir2 while triggering
> 20 loops of revisions from Dir1 and I would reliably get the corruption
> before the 10 loops of the binary commits had finished ... "svn:
> Decompression of svndiff data failed"
>
> Removing the "--threads" from the svnserve options and I could get through
> over 400 commits without an error ... restarting svnserve with that option
> and again I would corrupt my repository within 10 commits ...
>
> Is this worthy of posting a bug report? Does someone want my scripts to test
> it out themselves?

If you can consistently reproduce, we'd - ofcourse - like your scripts!

Bye,

Erik.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Corruption when using --threads

[Troy Curtis Jr <tr...@gmail.com>]

On Wed, Mar 19, 2008 at 7:53 PM, Landon Bradshaw
<la...@justleapin.com> wrote:
> We've been having a problem with our repository and finally had the time to
> do some more testing ... the results are that we needed to drop "--threads"
> from our svnserve invocation and just let it run via fork()
>
> Environment: CentOS5
> Revision: 1.4.2 (CentOS5) and 1.4.4 (Ubuntu)
> DB-type: fsfs
>
> Following these steps I could get a corruption within 10 revisions ...
>
> Setup:
> * create the repo
> * create 2 subdirectories
>  * commit a set of text files into Dir1 (20 files all less than 2KB)
> * commit a set of large binary files into Dir2 (6 files of 2.5MB to 16MB)
>
> Stress:
> * By randomly choosing a text file to change I created a new revision for
> Dir1 containing 4 to 9 changed files
>  * By randomly swapping the filename of 2 files in Dir2 I created a large
> changeset for Dir2
> * Serial commits of Dir2 were continuously occurring while I was randomly
> choosing a sleep time between serial commits of Dir1 (<16 seconds)
>
> I would trigger 10 loops of the large swap/commit of Dir2 while triggering
> 20 loops of revisions from Dir1 and I would reliably get the corruption
> before the 10 loops of the binary commits had finished ... "svn:
> Decompression of svndiff data failed"
>
> Removing the "--threads" from the svnserve options and I could get through
> over 400 commits without an error ... restarting svnserve with that option
> and again I would corrupt my repository within 10 commits ...
>
> Is this worthy of posting a bug report? Does someone want my scripts to test
> it out themselves?
>
> ...Landon
>

I for one would like to see if I could duplicate that corruption.
Could you post your script?

Troy

-- 
"Beware of spyware. If you can, use the Firefox browser." - USA Today
Download now at http://getfirefox.com
Registered Linux User #354814 ( http://counter.li.org/)

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org