You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by "Leeuw van der, Tim" <ti...@nl.unisys.com> on 2003/11/14 12:24:20 UTC
Deltification (WAS: Re: Subversion 0.33.0 released).
I'd actually like to see this a per-repository configurable option.
If the post-commit hook for this cannot be automatically installed, which I suspect is not a feasible option due to platform dependancies and not knowing in advance what kind of scripting-languages are installed, then having no automatic deltification means a much larger barrier-to-entry for the inexperienced, new user who wants as much as possible everything automatic but does want version control.
Telling people: 'install a script as a hook, make up your own here's a sample, or svn will eat diskspace like there's no tomorrow' is not a good way to attract newbie type of users. The same sort of users who will access their repositories via 'file://' urls because you don't need to do any kind of setup for that, no servers to install etc.
And I think that that's a bad thing.
regards,
--Tim
-----Original Message-----
From: kfogel@newton.ch.collab.net [mailto:kfogel@newton.ch.collab.net]On
Behalf Of kfogel@collab.net
Sent: vrijdag 14 november 2003 7:03
To: Paul Gross
Cc: users@subversion.tigris.org
Subject: Re: Subversion 0.33.0 released.
Paul Gross <pa...@grosss.net> writes:
> making a post-commit hook to run it? I tried searching around, but I
> couldn't find a good explanation of what it is or why I need it.
This is something I probably should have explained more in the release
announcement. Well, hopefully people will see it in this thread. In
fact, I'll repeat the important part, just so people see it again:
2. In order to make commits more responsive, repository
deltification is no longer automatic. However, you may want
to run deltification as a background process in your repository
post-commit hook. For example, the new post-commit.tmpl file
recommends 'nice -2 svnadmin deltify "$REPOS" -r "$REV" &'.
What is deltification?
Basically, Subversion stores (or is capable of storing) successive
revisions of a file in compressed form, thusly: the youngest revisions
is stoed in full, then the previous revision is stored as a diff
against that, and the next previous is a diff against the previous,
and so on back to the first revision of the file.
Until 0.33, it did this automatically. Whenever you committed a
change to a file, Subversion would change the storage representation
of the previous revision to be a diff against the new version of the
file, to save space in the repository.
None of this is user visible, of course. When Subversion retrieves a
file from a revision, it undoes any deltification and just hands you
the fulltext.
But unfortunately, there *was* a sense in which it was user-visible:
for large files, or many files, the deltification process could take
so long that it visibly slowed down the commit. (There are various
other complexities here too, such as the fact that the extra database
hits were generating db logfiles, which the administrator would want
to be cleaning up somehow...).
Anyway, we stopped doing deltification as an automatic part of the
commit, and started considering other solutions.
The ideal solution would be to spin off a deltification thread in the
background at the end of the commit, and return immediately to the
user, so the user would see "Committed revision XXX" immediately and
not have to wait for the deltification to finish. However, it turns
out to be difficult to portably run processes in the background.
So for now, what we've done is foisted this responsibility onto the
repository administrator. But we've tried to make it very easy to set
up: we have a command 'svnadmin deltify', and a fairly portable
template for invoking it (in the background) in the post-commit hook.
In conclusion: repository administrators who don't have an infinite
amount of disk space will want to change their post-commit hooks to
run deltification :-).
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Deltification (WAS: Re: Subversion 0.33.0 released).
Posted by Jan Hendrik <ja...@bigfoot.com>.
Concerning Re: Deltification (WAS: Re: Subvers
Jason Voegele wrote on 14 Nov 2003, 11:26, at least in part:
> >> (BTW, I accidentally sent this message only to Tim the first time
> >> around. Is it possible to set the Reply-To: header to the mailing
> >> list?)
> >
> > We'd rather not, see http://www.unicom.com/pw/reply-to-harmful.html.
... and then several people on these lists complain about getting
postings twice: once to the list, then again with the "group" as that
guy calls it. Dunno his Elm(fire) mailer but Pegasus Mail is one of
the few mailers in the world complying to rules, but however it is
done on this list one ends up in manually
adding/deleting/correcting addresses. On the Pegasus mail list
they have FROM: for the private reply, TO: for the "group" and
REPLY-TO: for the list. Not sure, but it even can be they just add
the list address in case a REPLY-TO: already exists.
However that may be, on another list they have the following option:
set duplicates on
set duplicates off
Use `set duplicates off' if you want Mailman to not send you
messages
if your address is explicitly mentioned in the To: or Cc: fields
of
the message. This can reduce the number of duplicate
postings you
will receive.
Maybe that is again devil for some, too, but I think it is worth to be
thrown in here.
Jan Hendrik
---------------------------------------
Freedom quote:
Our policy is simple:
We are not going to betray our friends,
reward the enemies of freedom,
or permit fear and retreat to become American policies. ...
None of the four wars in my lifetime
came about because we were too strong.
It is weakness ... that invites adventurous adversaries
to make mistaken judgments.
-- Ronald Reagan
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Deltification (WAS: Re: Subversion 0.33.0 released).
Posted by kf...@collab.net.
Brian Denny <br...@briandenny.net> writes:
> Since there are some advantages to *not* deltifying, it would be nice
> to have some actual numbers showing repository growth with/without
> deltification. (Hmm, those numbers shouldn't be too hard to come up
> with, either).
Well, there are no disadvantages to deltifying if it is done right.
"Right" means: it shouldn't delay the user response time for commits,
and it shouldn't effectively make the repository larger because of
logfiles. We just haven't solved those problems yet.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Deltification (WAS: Re: Subversion 0.33.0 released).
Posted by Brian Denny <br...@briandenny.net>.
On Fri, Nov 14, 2003 at 11:26:57AM -0500, Jason Voegele wrote:
> I understand now the issues surrounding deltifacation and how the previous
> status quo was not sufficient. I would however like to voice my desire to
> have the "ideal" solution in 1.0.
What is the "ideal" solution?
Since there are some advantages to *not* deltifying, it would be nice
to have some actual numbers showing repository growth with/without
deltification. (Hmm, those numbers shouldn't be too hard to come up
with, either).
I also think that installing a post-commit hook is easy and should not
be a major "barrier to entry" for most users. Certainly it's a heckuva
lot easier than building subversion from source.
I'd think it would be possible for platform-specific installers to
install such a hook automatically if that is desired behavior.
-brian
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Deltification (WAS: Re: Subversion 0.33.0 released).
Posted by Jason Voegele <ja...@jvoegele.com>.
Thanks for the information, on both counts.
I understand now the issues surrounding deltifacation and how the previous
status quo was not sufficient. I would however like to voice my desire to
have the "ideal" solution in 1.0.
Again, I understand that a patch is preferred to my grumbling, but I don't
have the time for it right now. Perhaps next week I can begin to look
into it.
Thanks,
Jason Voegele
> "Jason Voegele" <ja...@jvoegele.com> writes:
>> I'm afraid I have to agree with Tim here. This solution to slow
>> deltification merely seems to pass the buck to the administrator, who
>> may in many cases be an end user simply wishing to replace CVS on
>> their local machine.
>
> It's important to understand that the previous status quo was not
> resulting in automatic space savings either. See this thread for
> detail:
>
> http://subversion.tigris.org/servlets/ReadMsg?list=dev&msgNo=49401
>
> We hope to automate it as a background process eventually, but that
> turned out to be quite a bit harder than one might expect... So for now
> we had to compromise.
>
>> (BTW, I accidentally sent this message only to Tim the first time
>> around. Is it possible to set the Reply-To: header to the mailing
>> list?)
>
> We'd rather not, see http://www.unicom.com/pw/reply-to-harmful.html.
--
Jason Voegele
"There is an essential core at the center of each man and woman that
remains unaltered no matter how life's externals may be transformed
or recombined. But it's smaller than we think."
-- Gene Wolfe, The Book of the Long Sun
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Deltification (WAS: Re: Subversion 0.33.0 released).
Posted by kf...@collab.net.
"Jason Voegele" <ja...@jvoegele.com> writes:
> I'm afraid I have to agree with Tim here. This solution to slow
> deltification merely seems to pass the buck to the administrator, who may
> in many cases be an end user simply wishing to replace CVS on their local
> machine.
It's important to understand that the previous status quo was not
resulting in automatic space savings either. See this thread for
detail:
http://subversion.tigris.org/servlets/ReadMsg?list=dev&msgNo=49401
We hope to automate it as a background process eventually, but that
turned out to be quite a bit harder than one might expect... So for
now we had to compromise.
> (BTW, I accidentally sent this message only to Tim the first time around.
> Is it possible to set the Reply-To: header to the mailing list?)
We'd rather not, see http://www.unicom.com/pw/reply-to-harmful.html.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Deltification (WAS: Re: Subversion 0.33.0 released).
Posted by Jason Voegele <ja...@jvoegele.com>.
Leeuw van der, Tim said:
> I'd actually like to see this a per-repository configurable option.
[snip]
> Telling people: 'install a script as a hook, make up your own here's a
> sample, or svn will eat diskspace like there's no tomorrow' is not a
> good way to attract newbie type of users. The same sort of users who
> will access their repositories via 'file://' urls because you don't need
> to do any kind of setup for that, no servers to install etc.
>
> And I think that that's a bad thing.
I'm afraid I have to agree with Tim here. This solution to slow
deltification merely seems to pass the buck to the administrator, who may
in many cases be an end user simply wishing to replace CVS on their local
machine.
I know, a patch speaks louder than words, but I at least wanted to express
my concern about this approach and, I hope, foster some further discussion
about other possible solutions.
(BTW, I accidentally sent this message only to Tim the first time around.
Is it possible to set the Reply-To: header to the mailing list?)
--
Jason Voegele
"There is an essential core at the center of each man and woman that
remains unaltered no matter how life's externals may be transformed
or recombined. But it's smaller than we think."
-- Gene Wolfe, The Book of the Long Sun
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org