You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Marius Gedminas <ma...@gedmin.as> on 2013/02/07 22:00:33 UTC
Discrepancies in svn mirror created with svnsync
I use svnsync to keep a mirror of svn.zope.org. Recently I've noticed
that at least three revisions differ between the original and the
mirror:
http://svn.zope.org/?rev=129030&view=rev
http://zope3.pov.lt/trac/changeset/129030
http://svn.zope.org/?rev=129031&view=rev
http://zope3.pov.lt/trac/changeset/129031
http://svn.zope.org/?rev=129032&view=rev
http://zope3.pov.lt/trac/changeset/129032
At the time of r129030 (2013-01-08) the server running svnsync was
running Ubuntu 12.04 LTS and had subversion 1.6.17dfsg-3ubuntu3. (It
still has that version.)
In case it matters: I originally created the mirror in late 2008 on a
different server, running Ubuntu 7.10, then moved it to this server in
late 2009 (running Ubuntu 9.04 at the time).
I do not have shell access on svn.zope.org and cannot tell you what
version of Subversion it was or is now running.
The cron script runs svnsync every 5 minutes. I occasionally get error
emails from it, usually due to intermittent network connectivity. I
keep an archive of those emails: none were sent on Jan 08, when the
broken commits got mirrored (unless they got corrupted later somehow?).
The mirror repository resides on a Software RAID 1 partition.
I've two questions, one driven by curiosity, and one by practical
concerns:
1) How could this have happened and why didn't svnsync notice?
2) How can I fix this without nuking my mirror and recreating it from
scratch?
The repository currently has 129184 revisions and occupies 3.3 gigs of
disk space.
P.S. I'm not subscribed to this list.
Marius Gedminas
--
The clothes have no emperor.
-- C.A.R. Hoare, commenting on ADA.
Re: Discrepancies in svn mirror created with svnsync
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Marius Gedminas wrote on Thu, Feb 14, 2013 at 08:28:22 +0200:
> On Wed, Feb 13, 2013 at 08:32:35PM +0000, Philip Martin wrote:
> > Stefan Sperling <st...@elego.de> writes:
> >
> > > It is possible that authz rules prohibit access to the path affected
> > > by the revision. In which case svnsync would silently omit that path.
> >
> > $ svn log -vqr129027 svn://svn.zope.org/repos/main/
> > ------------------------------------------------------------------------
> > r129027 | (no author) | (no date)
> > ------------------------------------------------------------------------
> >
> > That looks like a revision filtered by authz.
>
> I guess I'll have to patch svn-all-fast-export not to segfault when it
> encounters such empty revisions. (Unfortunately attempting to exclude
> them by revision number is not enough.)
>
+1. In general you cannot assume that any revprops will be present,
that any paths will have been changed, or that any path recorded as
changed has a different text or properties hash before v. after the
revision.
> Marius Gedminas
> --
> I never got into Linux. I swear to God, it's only lack of time. I'm past the
> years of my life where I can really dig into something like running a Linux
> system. I'm very sympathetic to the whole idea; Linux people always think the
> way I want to think.
> -- Steve Wozniak
Re: Discrepancies in svn mirror created with svnsync
Posted by Marius Gedminas <ma...@gedmin.as>.
On Wed, Feb 13, 2013 at 08:32:35PM +0000, Philip Martin wrote:
> Stefan Sperling <st...@elego.de> writes:
>
> > It is possible that authz rules prohibit access to the path affected
> > by the revision. In which case svnsync would silently omit that path.
>
> $ svn log -vqr129027 svn://svn.zope.org/repos/main/
> ------------------------------------------------------------------------
> r129027 | (no author) | (no date)
> ------------------------------------------------------------------------
>
> That looks like a revision filtered by authz.
Oh, interesting. I was checking with the web frontend where that
revision looks normal: http://svn.zope.org/?rev=129027&view=rev
When I try a checkout (with my ssh key), it fails with
$ svn co svn+ssh://svn.zope.org/repos/main/2github/trunk 2github
svn: E170001: Authorization failed
which pretty much seals the case, I guess.
I'm a bit disconcerned about lack of any warning when svnsync produces a
mirror that is different from the original repository. And also curious
how original first mirror managed to get a copy of that revision, but
not of r129030-129032. Authz rules added after the fact, probably.
I guess I'll have to patch svn-all-fast-export not to segfault when it
encounters such empty revisions. (Unfortunately attempting to exclude
them by revision number is not enough.)
Marius Gedminas
--
I never got into Linux. I swear to God, it's only lack of time. I'm past the
years of my life where I can really dig into something like running a Linux
system. I'm very sympathetic to the whole idea; Linux people always think the
way I want to think.
-- Steve Wozniak
Re: Discrepancies in svn mirror created with svnsync
Posted by Philip Martin <ph...@wandisco.com>.
Stefan Sperling <st...@elego.de> writes:
> It is possible that authz rules prohibit access to the path affected
> by the revision. In which case svnsync would silently omit that path.
$ svn log -vqr129027 svn://svn.zope.org/repos/main/
------------------------------------------------------------------------
r129027 | (no author) | (no date)
------------------------------------------------------------------------
That looks like a revision filtered by authz.
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: Discrepancies in svn mirror created with svnsync
Posted by Stefan Sperling <st...@elego.de>.
On Wed, Feb 13, 2013 at 09:19:13PM +0200, Marius Gedminas wrote:
> But then I ran svnsync again and it gave me broken revisions again:
>
> $ svnsync sync file:///$PWD/zope-mirror
> $ svnlook author -r 129030 zope-mirror/ | wc -c
> 1
>
> And then I noticed that r129027 also became broken in this new mirror,
> but was fine in the old one:
>
> $ svnlook author -r 129027 zope-mirror/ | wc -c
> 1
> $ svnlook author -r 129027 zope-mirror.BROKEN/ | wc -c
> 4
>
> I was doing this with a copy of the broken mirror on my laptop, which
> has svnsync version 1.7.5 (r1336830).
>
> I guess this answers your question from upthread:
>
> > It would be interesting to know if this problem also appears with
> > svnsync from 1.7.
>
> Sadly, yes.
It is possible that authz rules prohibit access to the path affected
by the revision. In which case svnsync would silently omit that path.
Re: Discrepancies in svn mirror created with svnsync
Posted by Marius Gedminas <ma...@gedmin.as>.
On Tue, Feb 12, 2013 at 06:54:14PM +0100, Stefan Sperling wrote:
> On Tue, Feb 12, 2013 at 05:18:01PM +0200, Marius Gedminas wrote:
> > Anyway, this takes care of prevention. What about recovery? Can I fix
> > the three missing revisions manually somehow?
>
> If you can obtain the original revision files from the master
> repository, you can try dropping them into the slave repository as-is.
> That should work, provided the revs are valid on the master.
I don't have shell (or file) access to the original server.
> > Or at least truncate my
> > mirror's history to that point so that I could run svnsync to get just
> > the last month and a half, instead of re-creating the mirror from
> > scratch?
>
> It's possible to reset the repository to some revision rN, yes.
>
> The official way of doing this is to dump the repository from r0
> to rN (using svnadmin dump) and loading this dump file into a freshly
> created repository (svnadmin load).
> Since you're creating an svnsync mirror you should probably run
> 'svnsync init' before loading the dump file, and after loading adjust
> the svn:sync-last-merged-rev revision property on r0 to say 'rN'.
Thanks, this worked (after a false start where I assumed svnadmin dump
-rN:M would be a half-open interval like it is for svn log -rN:M; in the
end I wasted an hour and had to repeat the dump & load):
$ mv zope-mirror zope-mirror.BROKEN
$ svnadmin create zope-mirror
$ svnadmin setuuid zope-mirror 62d5b8a3-27da-0310-9561-8e5933582275
$ cp zope-mirror.BROKEN/hooks/pre-revprop-change zope-mirror/hooks/
$ svnsync init file:///$PWD/zope-mirror svn://svn.zope.org/repos/main/
$ svnadmin dump zope-mirror.BROKEN -r0:129020 > DUMP
$ svnadmin load zope-mirror --bypass-prop-validation -q < DUMP
$ svnadmin setrevprop zope-mirror -r0 svn:sync-last-merged-rev <(svnlook youngest zope-mirror)
$ svnlook propget --revprop zope-mirror -r0 svn:sync-last-merged-rev
129320
But then I ran svnsync again and it gave me broken revisions again:
$ svnsync sync file:///$PWD/zope-mirror
$ svnlook author -r 129030 zope-mirror/ | wc -c
1
And then I noticed that r129027 also became broken in this new mirror,
but was fine in the old one:
$ svnlook author -r 129027 zope-mirror/ | wc -c
1
$ svnlook author -r 129027 zope-mirror.BROKEN/ | wc -c
4
I was doing this with a copy of the broken mirror on my laptop, which
has svnsync version 1.7.5 (r1336830).
I guess this answers your question from upthread:
> It would be interesting to know if this problem also appears with
> svnsync from 1.7.
Sadly, yes.
Marius Gedminas
--
No proper program contains an indication which as an operator-applied
occurrence identifies an operator-defining occurrence which as an
indication-applied occurrence identifies an indication-defining occurrence
different from the one identified by the given indication as an
indication-applied occurrence.
-- ALGOL 68 Report
Re: Discrepancies in svn mirror created with svnsync
Posted by Philip Martin <ph...@wandisco.com>.
Daniel Shahaf <d....@daniel.shahaf.name> writes:
>> The official way of doing this is to dump the repository from r0
>> to rN (using svnadmin dump) and loading this dump file into a freshly
>> created repository (svnadmin load).
>> Since you're creating an svnsync mirror you should probably run
>> 'svnsync init' before loading the dump file, and after loading adjust
>> the svn:sync-last-merged-rev revision property on r0 to say 'rN'.
>>
>
> Need to truncate rep-cache.db Stefan! (otherwise the invariant on it
> won't hold, which can cause lossy corruption)
>
> sqlite3 'delete from rep_cache if revision > N' rep-cache.db
No need to do that if dump/loading.
>> There is a shortcut if you want to avoid the dump/load cycle, but it
>> involves messing with data in the repository so please don't try this
>> at home and make backups first! Set the file 'db/current' to rN and
>> remove all files in db/revs and db/revprops between HEAD and rN.
>> Then run 'svnadmin recover', reset svn:sync-last-merged-rev to rN and
>> hope that everything still works. Better try this on a small test
>> repository first!
Truncating the rep-cache is necessary if you backdate db/current.
Running 'svnadmin recover' will do it for you if you use 1.7 but not if
you use 1.6. For 1.6 use the sqlite3 command or simply delete
db/rep-cache.db.
The other thing that can go wrong if you truncate a repository is that
you may end up with orphan exclusive locks (locks on files that don't
exist). This will prevent you adding those files in future until you
unlock the path. This is unlikely to affect an svnsync mirror since you
probably don't lock files in the mirror.
--
Certified & Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download
Re: Discrepancies in svn mirror created with svnsync
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Stefan Sperling wrote on Tue, Feb 12, 2013 at 18:54:14 +0100:
> On Tue, Feb 12, 2013 at 05:18:01PM +0200, Marius Gedminas wrote:
> > Anyway, this takes care of prevention. What about recovery? Can I fix
> > the three missing revisions manually somehow?
>
> If you can obtain the original revision files from the master
> repository, you can try dropping them into the slave repository as-is.
> That should work, provided the revs are valid on the master.
>
It is not guaranteed to work. It might work under some reasonable
assumptions, but I can think of two cases where it won't work off the
top of my head. That said: try it and see if dump/verify complain.
(bogus offsets due to mismatched enable-rep-caching settings in history,
and issue #4129)
> > Or at least truncate my
> > mirror's history to that point so that I could run svnsync to get just
> > the last month and a half, instead of re-creating the mirror from
> > scratch?
>
> It's possible to reset the repository to some revision rN, yes.
>
> The official way of doing this is to dump the repository from r0
> to rN (using svnadmin dump) and loading this dump file into a freshly
> created repository (svnadmin load).
> Since you're creating an svnsync mirror you should probably run
> 'svnsync init' before loading the dump file, and after loading adjust
> the svn:sync-last-merged-rev revision property on r0 to say 'rN'.
>
Need to truncate rep-cache.db Stefan! (otherwise the invariant on it
won't hold, which can cause lossy corruption)
sqlite3 'delete from rep_cache if revision > N' rep-cache.db
> There is a shortcut if you want to avoid the dump/load cycle, but it
> involves messing with data in the repository so please don't try this
> at home and make backups first! Set the file 'db/current' to rN and
> remove all files in db/revs and db/revprops between HEAD and rN.
> Then run 'svnadmin recover', reset svn:sync-last-merged-rev to rN and
> hope that everything still works. Better try this on a small test
> repository first!
Re: Discrepancies in svn mirror created with svnsync
Posted by Stefan Sperling <st...@elego.de>.
On Tue, Feb 12, 2013 at 05:18:01PM +0200, Marius Gedminas wrote:
> Anyway, this takes care of prevention. What about recovery? Can I fix
> the three missing revisions manually somehow?
If you can obtain the original revision files from the master
repository, you can try dropping them into the slave repository as-is.
That should work, provided the revs are valid on the master.
> Or at least truncate my
> mirror's history to that point so that I could run svnsync to get just
> the last month and a half, instead of re-creating the mirror from
> scratch?
It's possible to reset the repository to some revision rN, yes.
The official way of doing this is to dump the repository from r0
to rN (using svnadmin dump) and loading this dump file into a freshly
created repository (svnadmin load).
Since you're creating an svnsync mirror you should probably run
'svnsync init' before loading the dump file, and after loading adjust
the svn:sync-last-merged-rev revision property on r0 to say 'rN'.
There is a shortcut if you want to avoid the dump/load cycle, but it
involves messing with data in the repository so please don't try this
at home and make backups first! Set the file 'db/current' to rN and
remove all files in db/revs and db/revprops between HEAD and rN.
Then run 'svnadmin recover', reset svn:sync-last-merged-rev to rN and
hope that everything still works. Better try this on a small test
repository first!
Re: Discrepancies in svn mirror created with svnsync
Posted by Andreas Mohr <an...@lisas.de>.
Hi,
On Tue, Feb 12, 2013 at 05:18:01PM +0200, Marius Gedminas wrote:
> On Sat, Feb 09, 2013 at 11:31:07AM +0100, Andreas Mohr wrote:
> > Hi,
> >
> > On Fri, Feb 08, 2013 at 03:45:29PM +0100, Stefan Sperling wrote:
> > > So you should definitely wrap svnsync in a tool like lockfile (part of
> > > procmail), or upgrade to 1.7.
>
> I went with
>
> #!/bin/sh
> lockfile -r500 /stuff/zope-mirror/locks/ADHOC.lock || exit 1
> /usr/bin/sudo -u syncuser /usr/bin/svnsync sync file:///stuff/zope-mirror
> rm -f /stuff/zope-mirror/locks/ADHOC.lock
>
> > Be careful about "solutions" other than lockfile - some of these appear to be
> > terribly unsafe (some newer Ubuntu-introduced "atomic locking" package
> > comes to mind - which then executes anyway after a measly timeout!).
>
> Do you remember the name of the package?
Possibly it was the lockfile-progs one, but not sure, couldn't nail
it down any more, sorry.
There seem to be recommendations for flock, since that has clean
behaviour on shell trap etc.
Andreas Mohr
Re: Discrepancies in svn mirror created with svnsync
Posted by Marius Gedminas <ma...@gedmin.as>.
On Sat, Feb 09, 2013 at 11:31:07AM +0100, Andreas Mohr wrote:
> Hi,
>
> On Fri, Feb 08, 2013 at 03:45:29PM +0100, Stefan Sperling wrote:
> > I cannot tell you what happened here and why the revisions in the
> > mirro are empty. That sure is concerning.
> >
> > However there are known race conditions in svnsync in Subversion 1.6.
> > See http://subversion.apache.org/docs/release-notes/1.7.html#atomic-revprops
> >
> > So you should definitely wrap svnsync in a tool like lockfile (part of
> > procmail), or upgrade to 1.7.
I went with
#!/bin/sh
lockfile -r500 /stuff/zope-mirror/locks/ADHOC.lock || exit 1
/usr/bin/sudo -u syncuser /usr/bin/svnsync sync file:///stuff/zope-mirror
rm -f /stuff/zope-mirror/locks/ADHOC.lock
> Be careful about "solutions" other than lockfile - some of these appear to be
> terribly unsafe (some newer Ubuntu-introduced "atomic locking" package
> comes to mind - which then executes anyway after a measly timeout!).
Do you remember the name of the package?
Anyway, this takes care of prevention. What about recovery? Can I fix
the three missing revisions manually somehow? Or at least truncate my
mirror's history to that point so that I could run svnsync to get just
the last month and a half, instead of re-creating the mirror from
scratch?
Marius Gedminas
--
Your eyes are weary from staring at the CRT. You feel sleepy. Notice how
restful it is to watch the cursor blink. Close your eyes. The opinions
stated above are yours. You cannot imagine why you ever felt otherwise.
Re: fsfs.conf knob to have merge() fail rather than automagically
rebase Re: Discrepancies in svn mirror created with svnsync
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Stefan Sperling wrote on Sat, Feb 09, 2013 at 21:19:35 +0100:
> On Sat, Feb 09, 2013 at 03:28:02PM +0200, Daniel Shahaf wrote:
> > Under the commit finalization lock, if the txn-being-committed's base
> > revision is older than the current HEAD, by default the code will commit
> > it anyway provided that no conflicting changes were made in the interim.
> > I wonder if it would be a good idea to have a fsfs.conf knob that tells
> > the code to just give up if {txn's base revision} != {current HEAD}.
> > Then that knob would be recommended to be set on every repository which
> > is an svnsync slave repository.
>
> If it's recommended, why not have 'svnsync init' tweak the config file
> accordingly?
Because it works over libsvn_ra.
> And we could take this one step further by hiding the flag
> somewhere in the repository instead of putting it in the config file.
Sure, we could move the config file to a $REPOS_DIR/db/../foo.conf, or
an r0 revprop that FS backends be required to respect, etc. Not sure
what would be the best place.
Re: fsfs.conf knob to have merge() fail rather than automagically
rebase Re: Discrepancies in svn mirror created with svnsync
Posted by Stefan Sperling <st...@elego.de>.
On Sat, Feb 09, 2013 at 03:28:02PM +0200, Daniel Shahaf wrote:
> Under the commit finalization lock, if the txn-being-committed's base
> revision is older than the current HEAD, by default the code will commit
> it anyway provided that no conflicting changes were made in the interim.
> I wonder if it would be a good idea to have a fsfs.conf knob that tells
> the code to just give up if {txn's base revision} != {current HEAD}.
> Then that knob would be recommended to be set on every repository which
> is an svnsync slave repository.
If it's recommended, why not have 'svnsync init' tweak the config file
accordingly? And we could take this one step further by hiding the flag
somewhere in the repository instead of putting it in the config file.
fsfs.conf knob to have merge() fail rather than automagically
rebase Re: Discrepancies in svn mirror created with svnsync
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Andreas Mohr wrote on Sat, Feb 09, 2013 at 11:31:07 +0100:
> Hi,
>
> On Fri, Feb 08, 2013 at 03:45:29PM +0100, Stefan Sperling wrote:
> > I cannot tell you what happened here and why the revisions in the
> > mirro are empty. That sure is concerning.
> >
> > However there are known race conditions in svnsync in Subversion 1.6.
> > See http://subversion.apache.org/docs/release-notes/1.7.html#atomic-revprops
> >
> > So you should definitely wrap svnsync in a tool like lockfile (part of
> > procmail), or upgrade to 1.7.
>
> Note that directory creation/removal are the only FS mechanism which is
> guaranteed to be atomic, on UNIX (POSIX?) at least.
> Thus if lockfile isn't available/installable, as a manual mechanism
> you could fetch a tempfile name (obviously to be globally used by *all*
> svnsync script users!), and use that name to create a directory,
> run svnsync on success and then remove it.
> (or probably better use a static GUID value in the directory name)
>
> Be careful about "solutions" other than lockfile - some of these appear to be
> terribly unsafe (some newer Ubuntu-introduced "atomic locking" package
> comes to mind - which then executes anyway after a measly timeout!).
>
> Andreas Mohr
Under the commit finalization lock, if the txn-being-committed's base
revision is older than the current HEAD, by default the code will commit
it anyway provided that no conflicting changes were made in the interim.
I wonder if it would be a good idea to have a fsfs.conf knob that tells
the code to just give up if {txn's base revision} != {current HEAD}.
Then that knob would be recommended to be set on every repository which
is an svnsync slave repository.
Daniel
P.S. "merge() in the subject has absolutely no relation to 'svn merge';
it's the name of an internal FS function.
Re: Discrepancies in svn mirror created with svnsync
Posted by Andreas Mohr <an...@lisas.de>.
Hi,
On Fri, Feb 08, 2013 at 03:45:29PM +0100, Stefan Sperling wrote:
> I cannot tell you what happened here and why the revisions in the
> mirro are empty. That sure is concerning.
>
> However there are known race conditions in svnsync in Subversion 1.6.
> See http://subversion.apache.org/docs/release-notes/1.7.html#atomic-revprops
>
> So you should definitely wrap svnsync in a tool like lockfile (part of
> procmail), or upgrade to 1.7.
Note that directory creation/removal are the only FS mechanism which is
guaranteed to be atomic, on UNIX (POSIX?) at least.
Thus if lockfile isn't available/installable, as a manual mechanism
you could fetch a tempfile name (obviously to be globally used by *all*
svnsync script users!), and use that name to create a directory,
run svnsync on success and then remove it.
(or probably better use a static GUID value in the directory name)
Be careful about "solutions" other than lockfile - some of these appear to be
terribly unsafe (some newer Ubuntu-introduced "atomic locking" package
comes to mind - which then executes anyway after a measly timeout!).
Andreas Mohr
Re: Discrepancies in svn mirror created with svnsync
Posted by Lathan Bidwell <la...@andrews.edu>.
Zach, the email to unsubscribe is:
users-unsubscribe@subversion.apache.org
On 02/08/2013 09:47 AM, Zachary Burnham wrote:
> unsubscribe
>
> Z
>
> *
>
> Zachary Burnham* | Web Developer
> Energy Federation Inc | 1 Willow Street | Southborough, MA 01772
> 508.870.2277 x4467 | f 888.440.4219
> zburnham@efi.org <ma...@efi.org> | efi.org
> <http://www.efi.org/>
>
>
>
>
> On Feb 8, 2013, at 9:45 AM, Stefan Sperling <stsp@elego.de
> <ma...@elego.de>> wrote:
>
>> On Fri, Feb 08, 2013 at 03:45:16PM +0200, Marius Gedminas wrote:
>>> On Fri, Feb 08, 2013 at 11:44:40AM +0100, Andreas Krey wrote:
>>>> On Thu, 07 Feb 2013 23:00:33 +0000, Marius Gedminas wrote:
>>>> ...
>>>>> The cron script runs svnsync every 5 minutes.
>>>>
>>>> Do you make sure svnsync isn't started anew when the previous instance
>>>> hasn't terminated yet? (I don't know if that matters.)
>>>
>>> No. Here's my /etc/cron.d/zope-svn-mirror:
>>>
>>> # Mirror the Zope repository
>>> */5 * * * * root /usr/local/sbin/update-zope-mirror > /dev/null
>>>
>>> and here's my /usr/local/sbin/update-zope-mirror:
>>>
>>> #!/bin/sh
>>> /usr/bin/sudo -u syncuser /usr/bin/svnsync sync
>>> file:///stuff/zope-mirror
>>>
>>> It's possible that a temporal overlap happened.
>>>
>>> Marius Gedminas
>>
>> I cannot tell you what happened here and why the revisions in the
>> mirro are empty. That sure is concerning.
>>
>> However there are known race conditions in svnsync in Subversion 1.6.
>> See
>> http://subversion.apache.org/docs/release-notes/1.7.html#atomic-revprops
>>
>> So you should definitely wrap svnsync in a tool like lockfile (part of
>> procmail), or upgrade to 1.7.
>>
>> It would be interesting to know if this problem also appears with
>> svnsync from 1.7. Note that with file:// URLs the svnsync you are running
>> is both client and server from the mirror repository's point of view.
>> So the version running on svn.zope.org <http://svn.zope.org> doesn't
>> really matter (though I
>> checked, and it appears to be severely outdated...)
>
> ------------------------------------------------------------------------
>
> Spam <http://www.andrews.edu/spam/b.php?i=0aIVOMJDl&m=e6bd60b3199f&c=s>
> Not spam
> <http://www.andrews.edu/spam/b.php?i=0aIVOMJDl&m=e6bd60b3199f&c=n>
> Forget previous vote
> <http://www.andrews.edu/spam/b.php?i=0aIVOMJDl&m=e6bd60b3199f&c=f>
Re: Discrepancies in svn mirror created with svnsync
Posted by Zachary Burnham <zb...@efi.org>.
unsubscribe
Z
[cid:image002.jpg@01CD8139.6DE4E9D0]
Zachary Burnham | Web Developer
Energy Federation Inc | 1 Willow Street | Southborough, MA 01772
508.870.2277 x4467 | f 888.440.4219
zburnham@efi.org<ma...@efi.org> | efi.org<http://www.efi.org/>
On Feb 8, 2013, at 9:45 AM, Stefan Sperling <st...@elego.de>> wrote:
On Fri, Feb 08, 2013 at 03:45:16PM +0200, Marius Gedminas wrote:
On Fri, Feb 08, 2013 at 11:44:40AM +0100, Andreas Krey wrote:
On Thu, 07 Feb 2013 23:00:33 +0000, Marius Gedminas wrote:
...
The cron script runs svnsync every 5 minutes.
Do you make sure svnsync isn't started anew when the previous instance
hasn't terminated yet? (I don't know if that matters.)
No. Here's my /etc/cron.d/zope-svn-mirror:
# Mirror the Zope repository
*/5 * * * * root /usr/local/sbin/update-zope-mirror > /dev/null
and here's my /usr/local/sbin/update-zope-mirror:
#!/bin/sh
/usr/bin/sudo -u syncuser /usr/bin/svnsync sync file:///stuff/zope-mirror
It's possible that a temporal overlap happened.
Marius Gedminas
I cannot tell you what happened here and why the revisions in the
mirro are empty. That sure is concerning.
However there are known race conditions in svnsync in Subversion 1.6.
See http://subversion.apache.org/docs/release-notes/1.7.html#atomic-revprops
So you should definitely wrap svnsync in a tool like lockfile (part of
procmail), or upgrade to 1.7.
It would be interesting to know if this problem also appears with
svnsync from 1.7. Note that with file:// URLs the svnsync you are running
is both client and server from the mirror repository's point of view.
So the version running on svn.zope.org<http://svn.zope.org> doesn't really matter (though I
checked, and it appears to be severely outdated...)
Re: Discrepancies in svn mirror created with svnsync
Posted by Stefan Sperling <st...@elego.de>.
On Fri, Feb 08, 2013 at 03:45:16PM +0200, Marius Gedminas wrote:
> On Fri, Feb 08, 2013 at 11:44:40AM +0100, Andreas Krey wrote:
> > On Thu, 07 Feb 2013 23:00:33 +0000, Marius Gedminas wrote:
> > ...
> > > The cron script runs svnsync every 5 minutes.
> >
> > Do you make sure svnsync isn't started anew when the previous instance
> > hasn't terminated yet? (I don't know if that matters.)
>
> No. Here's my /etc/cron.d/zope-svn-mirror:
>
> # Mirror the Zope repository
> */5 * * * * root /usr/local/sbin/update-zope-mirror > /dev/null
>
> and here's my /usr/local/sbin/update-zope-mirror:
>
> #!/bin/sh
> /usr/bin/sudo -u syncuser /usr/bin/svnsync sync file:///stuff/zope-mirror
>
> It's possible that a temporal overlap happened.
>
> Marius Gedminas
I cannot tell you what happened here and why the revisions in the
mirro are empty. That sure is concerning.
However there are known race conditions in svnsync in Subversion 1.6.
See http://subversion.apache.org/docs/release-notes/1.7.html#atomic-revprops
So you should definitely wrap svnsync in a tool like lockfile (part of
procmail), or upgrade to 1.7.
It would be interesting to know if this problem also appears with
svnsync from 1.7. Note that with file:// URLs the svnsync you are running
is both client and server from the mirror repository's point of view.
So the version running on svn.zope.org doesn't really matter (though I
checked, and it appears to be severely outdated...)
Re: Discrepancies in svn mirror created with svnsync
Posted by Thorsten Schöning <ts...@am-soft.de>.
Guten Tag Marius Gedminas,
am Freitag, 8. Februar 2013 um 14:45 schrieben Sie:
> It's possible that a temporal overlap happened.
That shouldn't be a problem, as per default svnsync aquires exclusive
locks on the destination repo where it should mirror the data to and
subsequent calls to svnsync would throw an error about the former
created locks.
Mit freundlichen Grüßen,
Thorsten Schöning
--
Thorsten Schöning E-Mail:Thorsten.Schoening@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/
Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04
AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Re: Discrepancies in svn mirror created with svnsync
Posted by Marius Gedminas <ma...@gedmin.as>.
On Fri, Feb 08, 2013 at 11:44:40AM +0100, Andreas Krey wrote:
> On Thu, 07 Feb 2013 23:00:33 +0000, Marius Gedminas wrote:
> ...
> > The cron script runs svnsync every 5 minutes.
>
> Do you make sure svnsync isn't started anew when the previous instance
> hasn't terminated yet? (I don't know if that matters.)
No. Here's my /etc/cron.d/zope-svn-mirror:
# Mirror the Zope repository
*/5 * * * * root /usr/local/sbin/update-zope-mirror > /dev/null
and here's my /usr/local/sbin/update-zope-mirror:
#!/bin/sh
/usr/bin/sudo -u syncuser /usr/bin/svnsync sync file:///stuff/zope-mirror
It's possible that a temporal overlap happened.
Marius Gedminas
--
MCSE == Minesweeper Consultant / Solitaire Expert
Re: Discrepancies in svn mirror created with svnsync
Posted by Andreas Krey <a....@gmx.de>.
On Thu, 07 Feb 2013 23:00:33 +0000, Marius Gedminas wrote:
...
> The cron script runs svnsync every 5 minutes.
Do you make sure svnsync isn't started anew when the previous instance
hasn't terminated yet? (I don't know if that matters.)
Andreas
--
"Totally trivial. Famous last words."
From: Linus Torvalds <torvalds@*.org>
Date: Fri, 22 Jan 2010 07:29:21 -0800