You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Dustin Lang <ds...@cmu.edu> on 2013/11/18 18:10:04 UTC
svnsync: properties not always copied?
[I am not on the list; please cc me in replies]
Hi,
I keep an svnsync'd mirror of my svn repository, and I run a very
stringent verification check: I do an "svnadmin dump" on the original and
the mirror, and demand that they have the same md5sum. It is paranoid,
but paranoia is good when it comes to backups!
Problem is, in some situation that I haven't nailed down, some properties
don't seem to get set in the svnsync'd version. I think it happens when I
make a tag, ie, copy a directory, that has properties set. For example,
in the "svnadmin dump":
--------------------------------------------------------
Node-path: tags/nova/2013-11-04-1
Node-kind: dir
Node-action: add
Node-copyfrom-rev: 24017
Node-copyfrom-path: trunk/src/astrometry
Prop-content-length: 92
Content-length: 92
K 10
svn:ignore
V 6
*.pyc
K 13
svn:mergeinfo
V 30
/tags/nova/2012-06-27-2:21009*
PROPS-END
--------------------------------------------------------
in the svnsync'd mirror:
--------------------------------------------------------
Node-path: tags/nova/2013-11-04-1
Node-kind: dir
Node-action: add
Node-copyfrom-rev: 24017
Node-copyfrom-path: trunk/src/astrometry
--------------------------------------------------------
This sounds a bit like closed issue 4184:
http://subversion.tigris.org/issues/show_bug.cgi?id=4184
I am using subversion-1.8.4.
For what it's worth, the mirror is on another machine and connects via
svn+ssh. This svn repo was created in 2005, and is format "3" (versions
1.5+).
Unfortunately, I was unable to create a simple test case demonstrating the
problem, so maybe it is some more complicated circumstance than the
properties and directory copy, or maybe it has to do with having an old
repo?
Any hints on what could trigger this would be much appreciated!
Thanks,
--dustin
RE: svnsync: properties not always copied?
Posted by Dustin Lang <ds...@cmu.edu>.
Dear Bert,
Thanks very much for your reply. I was also frustrated that I was not
able to create a small test case that reproduces the problem, hence my
plea for ideas of what could be triggering the problem.
> Your example 'svn:mergeinfo' missing could be caused by older Subversion
> clients that in some specific merge cases committed unmodified
> properties as modified.
You nailed it!
The machine where I normally make the tags like the one in the example I
sent is at svn 1.6.
I just tested, and when I tag with a v1.6 client, the svnadmin dumps
differ between the original and mirror, while when I tag with a v1.8.4
client, the svnadmin dumps are identical.
NICE!
Thanks very much,
--dustin
RE: svnsync: properties not always copied?
Posted by Bert Huijben <be...@qqmail.nl>.
> -----Original Message-----
> From: Dustin Lang [mailto:dstndstn@gmail.com] On Behalf Of Dustin Lang
> Sent: maandag 18 november 2013 18:10
> To: users@subversion.apache.org
> Subject: svnsync: properties not always copied?
>
>
> [I am not on the list; please cc me in replies]
>
> Hi,
>
> I keep an svnsync'd mirror of my svn repository, and I run a very
> stringent verification check: I do an "svnadmin dump" on the original and
> the mirror, and demand that they have the same md5sum. It is paranoid,
> but paranoia is good when it comes to backups!
>
> Problem is, in some situation that I haven't nailed down, some properties
> don't seem to get set in the svnsync'd version. I think it happens when I
> make a tag, ie, copy a directory, that has properties set. For example,
> in the "svnadmin dump":
Hi Dustin,
Your description of your problem sounds like something that might be very
important to fix, but it doesn't describe a way for us to reproduce the
problem. This makes it very hard for us to spend time on this issue.
Your example 'svn:mergeinfo' missing could be caused by older Subversion
clients that in some specific merge cases committed unmodified properties as
modified. When the replay code then calculates which property changes should
be reported, the no-change properties are not reported.
(All known cases of this problem should be fixed in Subversion 1.8.X
clients)
So there might be changes that are no real changes in the lower fs layers of
subversion that would be noted as 'no change' by the upper layers, which
could explain exactly what you are seeing: a difference between dumping via
the lower and higher layers.
Is there some way you can show us how you find the problem or how we can
reproduce the problem. That way we can try to find the root cause, and see
if there is something we can fix.
--
If you want to try reproducing the case I described here you need a
Subversion 1.7 or older client and merge directories that contain
properties. Subversion 1.7 will then just commit the properties that were
copied, as if they were locally changed into what they were before they were
copied.
As normal user you would probably never notice this problem... But we found
out about it when fixing some tree conflict cases in merging during 1.8
development.
--
Thanks,
Bert
>
> --------------------------------------------------------
> Node-path: tags/nova/2013-11-04-1
> Node-kind: dir
> Node-action: add
> Node-copyfrom-rev: 24017
> Node-copyfrom-path: trunk/src/astrometry
> Prop-content-length: 92
> Content-length: 92
>
> K 10
> svn:ignore
> V 6
> *.pyc
>
> K 13
> svn:mergeinfo
> V 30
> /tags/nova/2012-06-27-2:21009*
> PROPS-END
> --------------------------------------------------------
>
> in the svnsync'd mirror:
>
> --------------------------------------------------------
> Node-path: tags/nova/2013-11-04-1
> Node-kind: dir
> Node-action: add
> Node-copyfrom-rev: 24017
> Node-copyfrom-path: trunk/src/astrometry
> --------------------------------------------------------
>
> This sounds a bit like closed issue 4184:
> http://subversion.tigris.org/issues/show_bug.cgi?id=4184
>
> I am using subversion-1.8.4.
>
> For what it's worth, the mirror is on another machine and connects via
> svn+ssh. This svn repo was created in 2005, and is format "3" (versions
> 1.5+).
>
> Unfortunately, I was unable to create a simple test case demonstrating the
> problem, so maybe it is some more complicated circumstance than the
> properties and directory copy, or maybe it has to do with having an old
> repo?
>
> Any hints on what could trigger this would be much appreciated!
>
> Thanks,
> --dustin
Re: svnsync: properties not always copied?
Posted by Philip Martin <ph...@wandisco.com>.
Dustin Lang <ds...@cmu.edu> writes:
> (Patch:
> http://astrometry.net/svn/trunk/scripts/svnsync/sort-dels.diff, but I
> don't know if it still applies or is needed)
A similar fix is on trunk, 1.8 and 1.7.
--
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*
Re: svnsync: properties not always copied?
Posted by Dustin Lang <ds...@cmu.edu>.
Hi Mark,
> I do not want to sidetrack this discussion and it sounds like you are
> not having the issue I am going to mention, but I thought you could not
> rely on this any more due to the random hash ordering of APR in recent
> releases (which would yield different md5sum)?
At one point I put in a local patch to fix the order of deletes. I
thought I sent a note and the patch to the list. That was ages ago.
(Patch: http://astrometry.net/svn/trunk/scripts/svnsync/sort-dels.diff,
but I don't know if it still applies or is needed)
cheers,
--dustin
Re: svnsync: properties not always copied?
Posted by Mark Phippard <ma...@gmail.com>.
On Mon, Nov 18, 2013 at 12:10 PM, Dustin Lang <ds...@cmu.edu> wrote:
> I keep an svnsync'd mirror of my svn repository, and I run a very
> stringent verification check: I do an "svnadmin dump" on the original and
> the mirror, and demand that they have the same md5sum. It is paranoid, but
> paranoia is good when it comes to backups!
>
>
I do not want to sidetrack this discussion and it sounds like you are not
having the issue I am going to mention, but I thought you could not rely on
this any more due to the random hash ordering of APR in recent releases
(which would yield different md5sum)?
Did we "fix" all of our hashes to apply an ordering of the results? I
believe it was discussed but did not know if we ever implemented it
everywhere. The 1.8 Release Notes list this:
* fix dependency on APR hash order in several logic paths (r1338350 et al)
--
Thanks
Mark Phippard
http://markphip.blogspot.com/