Specific commit breaks consistency of server repository

[Sergey Sharybin <se...@gmail.com>]

Hi,

We've discovered that one specific change consistently breaks corrupts
repository on the server side.

This seems to be specific to particular change, committing that change in
smaller chunks does not cause any corruption. There's nothing special in
the patch actually, it just adds some images, removes some other and
modifies some text files.

Consistently happens on FreeBSD and Debian Linux running Subversion 1.9.4.
I've also tested SVN version of Subversion r1765588 and previous release
1.9.3. Can reproduce corruption with all this versions.

Unfortunately, did not manage to reproduce anything like that on a smaller
repository, but it is a publicly available repository which we can share.

I've prepared [1] a demonstration case which can be run on a local Linux
machine. What is in that archive:

- Original repository. As far as `svnadmin verify` is concerned this
repository is in fully consistent state withotu any known issues.
- Modified checkout of that repository, which is used to apply changes and
commit changes to the repository.
- Script which is aimed to automatically reproduce the problem. This script
basically does:

1. Creates a new checkout of the local repository
2. RSync's changes from the modified checkout, this way we are always sure
changes are always applied correctly and everything.
3. Does some magic with `svn st`, `svn add` and `svn rm` to add new files
under versioning and remove some files from the versioning.
4. Commits changes to a local repository.
5. Updates checkout.

The `svn up` which happens at the end of the script will print:

svn: E160004: Filesystem is corrupt
svn: E200014: Checksum mismatch while reading representation:
   expected:  2a7aad86ce385afbf26edd935dcf3d00
     actual:  97ab96e4365a1225fea2a3b29e38fcab

This is an exact message which `svnadmin verify` will give when verifying
the repository after that commit.

For the record: File system is not corrupted, this error happens reliably
on several machines here.

Here is svnadmin info of the repository:

Path: .
UUID: c4de1f47-6596-e411-a384-0024e86c2797
Repository Format: 5
Compatible With Version: 1.9.0
Repository Capability: mergeinfo
Filesystem Type: fsfs
Filesystem Format: 7
FSFS Sharded: yes
FSFS Shard Size: 1000
FSFS Shards Packed: 2/2
FSFS Logical Addressing: no
Configuration File: db/fsfs.conf

I'll be happy to provide any extra information needed for troubleshooting :)

Thanks in advance.

[1] https://drive.google.com/open?id=0B6JCwGemQK3ZalpIT0lBNkxFMFE

-- 
With best regards, Sergey Sharybin

Re: Specific commit breaks consistency of server repository

[Daniel Shahaf <da...@apache.org>]

Daniel Shahaf wrote on Sun, Oct 23, 2016 at 16:50:03 +0000:
> Sergey Sharybin wrote on Fri, Oct 21, 2016 at 13:57:41 +0200:
> > We've discovered that one specific change consistently breaks corrupts
> > repository on the server side.
> 
> Sergey, thanks for the report.  Stefan has looked into it and concurs
> it's a bug.  We don't currently have a patch but we expect one to be in
> 1.9.5.

Fixed in r1766352, to be backported to 1.8.17 and 1.9.5.  See
https://issues.apache.org/jira/browse/SVN-4658

Re: Specific commit breaks consistency of server repository

[Daniel Shahaf <da...@apache.org>]

Sergey Sharybin wrote on Fri, Oct 21, 2016 at 13:57:41 +0200:
> We've discovered that one specific change consistently breaks corrupts
> repository on the server side.

Sergey, thanks for the report.  Stefan has looked into it and concurs
it's a bug.  We don't currently have a patch but we expect one to be in
1.9.5.

Quoting IRC:

<stefan2> [It] hits whenever you deltify against PLAIN reps >100kB *and* with no common sub-string to that base in the first 100k
<stefan2> or rather: the data is not actually corrupted but will not be reconstructed correctly