You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Daniel Shahaf <d....@daniel.shahaf.name> on 2011/01/13 17:28:44 UTC

Re: Trival merge of big text file: Dismal performance, 540x faster if binary.

Stefan Sperling wrote on Thu, Jan 13, 2011 at 13:07:34 +0000:
> On Thu, Jan 13, 2011 at 01:55:58PM +0100, Johan Corveleyn wrote:
> > Textual merging in svn makes use of a variant of the standard diff
> > algorithm, namely diff3. Just a couple of days ago, I finally
> > succeeded in making diff3 take advantage of those performance
> > improvements (haven't committed this to the branch yet, but maybe I'll
> > get to it tonight).
> > 
> > Would you be able to build an svn client from source? If so, could you
> > perhaps build a client from
> > http://svn.apache.org/repos/asf/subversion/branches/diff-optimizations-bytes
> > ?
> 
> Hey Johan,
> 
> I would be interested in doing testing and reviewing the changes
> on your branch. There might still be enough time to get them into 1.7.
> 
> I don't have any suitably large XML files though.
> If you and/or Andreas could provide some that would be great.
> 

How about taking periodic dumps of some large repository?  I count on
propchanges to give the "small change in the middle of the file" effect.

Another option:

    for i in 0 1 2 3 4 5 6 7 8 9; do
      cat $REPOS/db/revs/*/*$i
    done | tar -cf- > "`date`"

> Thanks,
> Stefan

Re: Trival merge of big text file: Dismal performance, 540x faster if binary.

Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Daniel Shahaf wrote on Thu, Jan 13, 2011 at 18:28:44 +0200:
> Stefan Sperling wrote on Thu, Jan 13, 2011 at 13:07:34 +0000:
> > On Thu, Jan 13, 2011 at 01:55:58PM +0100, Johan Corveleyn wrote:
> > > Textual merging in svn makes use of a variant of the standard diff
> > > algorithm, namely diff3. Just a couple of days ago, I finally
> > > succeeded in making diff3 take advantage of those performance
> > > improvements (haven't committed this to the branch yet, but maybe I'll
> > > get to it tonight).
> > > 
> > > Would you be able to build an svn client from source? If so, could you
> > > perhaps build a client from
> > > http://svn.apache.org/repos/asf/subversion/branches/diff-optimizations-bytes
> > > ?
> > 
> > Hey Johan,
> > 
> > I would be interested in doing testing and reviewing the changes
> > on your branch. There might still be enough time to get them into 1.7.
> > 
> > I don't have any suitably large XML files though.
> > If you and/or Andreas could provide some that would be great.
> > 
> 
> How about taking periodic dumps of some large repository?  I count on
> propchanges to give the "small change in the middle of the file" effect.
> 
> Another option:
> 
>     for i in 0 1 2 3 4 5 6 7 8 9; do
>       cat $REPOS/db/revs/*/*$i
>     done | tar -cf- > "`date`"

Without the tar.

> 
> > Thanks,
> > Stefan