You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Greg Stein <gs...@lyra.org> on 2003/11/12 00:00:01 UTC
Re: svn commit: rev 7695 - in trunk: doc/book subversion/include subversion/libsvn_fs subversion/libsvn_repos subversion/svnadmin subversion/tests/libsvn_fs
On Mon, Nov 10, 2003 at 07:01:20PM -0600, cmpilato@tigris.org wrote:
>...
> +++ trunk/subversion/svnadmin/main.c Mon Nov 10 19:01:18 2003
>...
> +/* This implements `svn_opt_subcommand_t'. */
> +static svn_error_t *
> +subcommand_deltify (apr_getopt_t *os, void *baton, apr_pool_t *pool)
> +{
>...
> + /* Loop over the requested revision range, performing the
> + predecessor deltification on paths changed in each. */
> + increment = (start > end) ? -1 : 1;
> + end = end + increment;
> + for (revision = start; revision != end; revision += increment)
> + {
Is there an "optimum" direction for the deltification? If so, then I'd
recommend reordering these to ensure they go in the best order.
For example, if deltify r1000 requires creating of a fulltext of r999,
then you'd want to do 1000 *before* deltifying 999.
>...
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 7695 - in trunk: doc/book subversion/include subversion/libsvn_fs subversion/libsvn_repos subversion/svnadmin subversion/tests/libsvn_fs
Posted by "C. Michael Pilato" <cm...@collab.net>.
Greg Hudson <gh...@MIT.EDU> writes:
> On Tue, 2003-11-11 at 22:57, C. Michael Pilato wrote:
> > > For example, if deltify r1000 requires creating of a fulltext of r999,
> > > then you'd want to do 1000 *before* deltifying 999.
> >
> > Yeah, I guess sometimes giving the user the flexibility just doesn't
> > pay off in the end.
>
> I think right now correctness demands that we deltify revisions in
> increasing order, even though that requires doing a lot of redundant
> work. If we want to optimize deltfying multiple revisions, I think that
> requires careful design.
Got some spare cycles to fix this svnadmin code? I'm buried in other
tasks. Pretty please?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 7695 - in trunk: doc/book subversion/include subversion/libsvn_fs subversion/libsvn_repos subversion/svnadmin subversion/tests/libsvn_fs
Posted by kf...@collab.net.
Greg Hudson <gh...@MIT.EDU> writes:
> > > For example, if deltify r1000 requires creating of a fulltext of r999,
> > > then you'd want to do 1000 *before* deltifying 999.
> >
> > Yeah, I guess sometimes giving the user the flexibility just doesn't
> > pay off in the end.
>
> I think right now correctness demands that we deltify revisions in
> increasing order, even though that requires doing a lot of redundant
> work. If we want to optimize deltfying multiple revisions, I think that
> requires careful design.
Why would it be a matter of correctness? A deltification should work
no matter what form the base text -- or target, for that matter -- is
stored in before the deltification.
It may be that it does nothing, if (say) the target is already
deltified. Or maybe it fully undeltifies the target and then
redeltifies, which is an awful lot of work... But correctness??
You're scaring me :-).
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 7695 - in trunk: doc/book subversion/include
subversion/libsvn_fs subversion/libsvn_repos subversion/svnadmin
subversion/tests/libsvn_fs
Posted by Greg Hudson <gh...@MIT.EDU>.
On Tue, 2003-11-11 at 22:57, C. Michael Pilato wrote:
> > For example, if deltify r1000 requires creating of a fulltext of r999,
> > then you'd want to do 1000 *before* deltifying 999.
>
> Yeah, I guess sometimes giving the user the flexibility just doesn't
> pay off in the end.
I think right now correctness demands that we deltify revisions in
increasing order, even though that requires doing a lot of redundant
work. If we want to optimize deltfying multiple revisions, I think that
requires careful design.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 7695 - in trunk: doc/book subversion/include subversion/libsvn_fs subversion/libsvn_repos subversion/svnadmin subversion/tests/libsvn_fs
Posted by "C. Michael Pilato" <cm...@collab.net>.
Greg Stein <gs...@lyra.org> writes:
> Is there an "optimum" direction for the deltification? If so, then I'd
> recommend reordering these to ensure they go in the best order.
>
> For example, if deltify r1000 requires creating of a fulltext of r999,
> then you'd want to do 1000 *before* deltifying 999.
Yeah, I guess sometimes giving the user the flexibility just doesn't
pay off in the end.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 7695 - in trunk: doc/book subversion/include subversion/libsvn_fs subversion/libsvn_repos subversion/svnadmin subversion/tests/libsvn_fs
Posted by kf...@collab.net.
"C. Michael Pilato" <cm...@collab.net> writes:
> A node is always deltified against some other node. Subsequent
> commits will not modify nodes that aren't part of that commit.
Because deltification can be a complex topic, I just want to clarify a
possibly misleading implication from the above:
Deltification normally doesn't modify (i.e., deltify) the nodes that
are part of a commit. It deltifies the predecessors of those nodes.
The nodes that are changed in the commit actually remain fulltext
(until later they become the predecessor of something, of course).
So when we're talking about deltification, shorthand language can be
misleading. If you commit a change to to get, say, revision 3 of foo,
it doesn't mean we deltify revision 3 of foo. It means we (ideally)
deltify the previous revision of foo, which is either foo:1 or foo:2
depending on when foo last changed.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 7695 - in trunk: doc/book subversion/include subversion/libsvn_fs subversion/libsvn_repos subversion/svnadmin subversion/tests/libsvn_fs
Posted by "C. Michael Pilato" <cm...@collab.net>.
Philip Martin <ph...@codematters.co.uk> writes:
> "C. Michael Pilato" <cm...@collab.net> writes:
>
> [explanation]
>
> Thanks.
>
> > That's why I would encourage folks to run 'svnadmin deltify -r REV'
> > per-commit. Or, if the wanna use a cronjob, they should remember the
> > ranges they've deltified, perhaps by setting a custom property on
> > revision 0 of the filesystem.
>
> I assume you mean a revprop on the repository not a property on
> revision 0.
There are no props on repositories, so actually, yes, I meant revision
0. It's your repository -- do whatcha want with it. Of course, you
don't have to do that. If you'd rather set a revision property
("deltified") on each revision as you deltify it, you can do that.
Wanna use some textfile to remember that? I don't care. Do whatcha
want. I was merely saying I would do on my own repository.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 7695 - in trunk: doc/book subversion/include
subversion/libsvn_fs subversion/libsvn_repos subversion/svnadmin
subversion/tests/libsvn_fs
Posted by Philip Martin <ph...@codematters.co.uk>.
"C. Michael Pilato" <cm...@collab.net> writes:
[explanation]
Thanks.
> That's why I would encourage folks to run 'svnadmin deltify -r REV'
> per-commit. Or, if the wanna use a cronjob, they should remember the
> ranges they've deltified, perhaps by setting a custom property on
> revision 0 of the filesystem.
I assume you mean a revprop on the repository not a property on
revision 0.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 7695 - in trunk: doc/book subversion/include subversion/libsvn_fs subversion/libsvn_repos subversion/svnadmin subversion/tests/libsvn_fs
Posted by "C. Michael Pilato" <cm...@collab.net>.
Philip Martin <ph...@codematters.co.uk> writes:
> "C. Michael Pilato" <cm...@collab.net> writes:
>
> > That's interesting ... I wonder why. Oh! If you go backwards, you
> > have to first *undeltify* your "against" nodes so you can deltify your
> > target nodes. In the opposite direction, your "against" nodes are
> > always fulltexts. That makes sense. And, fortunately, is the best
> > direction for later undeltification anyway.
>
> My understanding of deltification is a little vague. Once a node is
> deltified do subsequent commits or deltifications change it?
A node is always deltified against some other node. Subsequent
commits will not modify nodes that aren't part of that commit.
Subsequent deltifications of successors of the node may or may not
re-deltify that node against a different node, younger than itself,
and generally younger than the last node against which it was
deltified. I say "may or may not" because the node might not get hit in the
skip-delta process. I say "generally younger" because if you can
certainly have a situation where you cause a node N that is currently
deltified against its grand-successor J steps away to be redeltified
against a successor that is < J steps away.
> Can the fs determine that a node is already delitified?
Yes.
> It would be great if "svnadmin deltify -r1:HEAD" could simply skip
> over already deltified nodes,
Actually, it wouldn't. That would necessitate the removal of
skip-delta generation, and would ultimately make the retrieval of
older revisions of the node more costly.
> at present re-running deltification on an already deltified
> repository takes longer than the original deltification.
That's correct. Deltification works its very fastest when the node
you are deltifying is being deltified against a different *fulltext*
node.
That's why I would encourage folks to run 'svnadmin deltify -r REV'
per-commit. Or, if the wanna use a cronjob, they should remember the
ranges they've deltified, perhaps by setting a custom property on
revision 0 of the filesystem.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 7695 - in trunk: doc/book subversion/include
subversion/libsvn_fs subversion/libsvn_repos subversion/svnadmin
subversion/tests/libsvn_fs
Posted by Philip Martin <ph...@codematters.co.uk>.
"C. Michael Pilato" <cm...@collab.net> writes:
> That's interesting ... I wonder why. Oh! If you go backwards, you
> have to first *undeltify* your "against" nodes so you can deltify your
> target nodes. In the opposite direction, your "against" nodes are
> always fulltexts. That makes sense. And, fortunately, is the best
> direction for later undeltification anyway.
My understanding of deltification is a little vague. Once a node is
deltified do subsequent commits or deltifications change it? Can the
fs determine that a node is already delitified? It would be great if
"svnadmin deltify -r1:HEAD" could simply skip over already deltified
nodes, at present re-running deltification on an already deltified
repository takes longer than the original deltification.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 7695 - in trunk: doc/book subversion/include subversion/libsvn_fs subversion/libsvn_repos subversion/svnadmin subversion/tests/libsvn_fs
Posted by "C. Michael Pilato" <cm...@collab.net>.
Philip Martin <ph...@codematters.co.uk> writes:
> > Greg Stein said the stuff that start with '> >':
> > Is there an "optimum" direction for the deltification? If so, then I'd
> > recommend reordering these to ensure they go in the best order.
I can't think of a reason why there would be an optimum direction as
far as the cost of the operation goes. But the optimum direction for
the *result* is to go forward in time (the same order in which commits
happen).
> > For example, if deltify r1000 requires creating of a fulltext of r999,
> > then you'd want to do 1000 *before* deltifying 999.
No fulltexts are created as a result of deltification.
> Using this repository
>
> stress.pl -c -s0 -F10 -D10 -N2 -x100 -W -d -P3 -n40
>
> (that's 40 revisions, 100 files per revision, a small change in each
> 30k file) and comparing
>
> svnadmin deltify -r1:41 repostress
> svnadmin deltify -r41:4 repostress
>
> I find that forward is faster than backward, i.e. 41:1 takes about 50%
> more CPU than 1:41.
That's interesting ... I wonder why. Oh! If you go backwards, you
have to first *undeltify* your "against" nodes so you can deltify your
target nodes. In the opposite direction, your "against" nodes are
always fulltexts. That makes sense. And, fortunately, is the best
direction for later undeltification anyway.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 7695 - in trunk: doc/book subversion/include
subversion/libsvn_fs subversion/libsvn_repos subversion/svnadmin
subversion/tests/libsvn_fs
Posted by Philip Martin <ph...@codematters.co.uk>.
Greg Stein <gs...@lyra.org> writes:
> On Mon, Nov 10, 2003 at 07:01:20PM -0600, cmpilato@tigris.org wrote:
>>...
>> +++ trunk/subversion/svnadmin/main.c Mon Nov 10 19:01:18 2003
>>...
>> +/* This implements `svn_opt_subcommand_t'. */
>> +static svn_error_t *
>> +subcommand_deltify (apr_getopt_t *os, void *baton, apr_pool_t *pool)
>> +{
>>...
>> + /* Loop over the requested revision range, performing the
>> + predecessor deltification on paths changed in each. */
>> + increment = (start > end) ? -1 : 1;
>> + end = end + increment;
>> + for (revision = start; revision != end; revision += increment)
>> + {
>
> Is there an "optimum" direction for the deltification? If so, then I'd
> recommend reordering these to ensure they go in the best order.
>
> For example, if deltify r1000 requires creating of a fulltext of r999,
> then you'd want to do 1000 *before* deltifying 999.
Using this repository
stress.pl -c -s0 -F10 -D10 -N2 -x100 -W -d -P3 -n40
(that's 40 revisions, 100 files per revision, a small change in each
30k file) and comparing
svnadmin deltify -r1:41 repostress
svnadmin deltify -r41:4 repostress
I find that forward is faster than backward, i.e. 41:1 takes about 50%
more CPU than 1:41.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org