Re: [RFC] Incremental dumps and mergeinfo (Was: Vetoing latest issue #3020 fix in 1.6.10)

[Julian Foad <ju...@wandisco.com>]

On Tue, 2010-08-03 at 14:35 -0400, Paul Burba wrote:
> Sweeping through my TODO list turned up this thread.
> 
> I'll commit this last patch tomorrow if there are no objections.

No objections to this improvement from me.

- Julian


> I pause a day only because I recall Mike saying he was planning to
> take a look at this.
> 
> Paul
> 
> On Tue, May 11, 2010 at 4:00 PM, Paul Burba <pt...@gmail.com> wrote:
[...]
> > At any rate, after my recent work we are left with one common(?) use
> > case still broken:
> >
> > Loading a sequence of incremental dumps when the fist load in the
> > sequence is into a *non-empty* repostitory.
> >
> > This is tested in svnadmin_tests.py 20 'don't filter mergeinfo revs
> > from incremental dump', specifically see '# PART 4: Load a a series of
> > incremental dumps to an non-empty repository."
> >
> > Basically the problem is this:
> >
> > 1) Given repository ReposA with X revisions in it.
> >
> >   Note: Previously this use case was also broken if ReposB
> >   was empty to start, but this now works (assuming none of
> >   the incremental dumps in step 3 were svndumpfiltered in
> >   such a way as to remove/renumber revisions).
> >
> > 2) Given repository ReposB with Y revisions in it.
> >
> > 3) Incrementally (but fully) dump ReposA
> >
> >   svnadmin dump ReposA -r0:100                 >  A.0.100.dump
> >   svnadmin dump ReposA -r101:200 --incremental >  A.101.200.dump
> >   svnadmin dump ReposA -r201:300 --incremental >  A.201.300.dump
> >   .
> >   .
> >   .
> >   svnadmin dump ReposA -rW:X     --incremental >  A.W.X.dump
> >
> > 4) Load the ReposA incremental dumps to ReposB
> >
> >   svnadmin load ReposB --parent-dir /some/subtree < A.0.100.dump
> >   svnadmin load ReposB --parent-dir /some/subtree < A.101.200.dump
> >   svnadmin load ReposB --parent-dir /some/subtree < A.201.300.dump
> >   .
> >   .
> >   .
> >   svnadmin load ReposB --parent-dir /some/subtree < A.W.X.dump
> >
> > The problem arises when one of the incremental loads contains
> > references to mergeinfo source revisions that predate the start of the
> > dump.  For example, say A.201.300.dump contains mergeinfo referencing
> > r82.  When this is loaded into ReposB the reference should be changed
> > to r(82+Y) to reflect the fact that ReposB has Y revisions in it
> > before we ever started loading parts of ReposA.  Currently no change
> > is made and the source rev stays at r82, which is obviously incorrect.
> >
> > The attached patch fixes this problem.  It takes all mergeinfo which
> > predates the first revision in the dump stream and adjusts it by the
> > difference between the head rev in ReposB and the current dump stream
> > revision.  Yes, this is a fairly fragile solution (as is our copyfrom
> > sources mapping, which this is based on).  If unrelated commits are
> > made to ReposB between loads of the incremental dumps, then the
> > mergeinfo source revs will be wrong; but it is *always* wrong right
> > now, at least this would fix this use case.  Opinions appreciated.
> >
> > And yes, there are still other potential problems: Think incremental
> > dumps that are dumpfiltered such that some revs are dropped or
> > renumbered.  But I'm not sure we really want to solve *every* problem
> > in this space, I'm not even sure we can (without bringing the
> > performance of svnadmin load to its knees).  I think this patch brings
> > us to "good enough" with the caveat that ill-advised use of svnadmin
> > dump/load and/or svndumpfilter can result in incorrect mergeinfo.
> >
> > Paul
> >