You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Daniel Shahaf <da...@elego.de> on 2013/06/11 17:14:53 UTC
Re: svn 1.8 migration - directory deltification and revprop packing
C. Michael Pilato wrote on Tue, Jun 11, 2013 at 14:52:48 +0200:
> One advantage of being in a room full of Subversion developers, specifically
> the guy that implemented all this stuff, is that I can ask him directly
> about how to respond to this mail. :-) Hopefully I will accurately
> represent the answers Stefan Fuhrmann just gave me to your questions.
>
> On 06/10/2013 03:05 PM, Thomas Harold wrote:
> > a) Why are directory/property deltifications turned off by default?
>
> Stefan's jovial first answer was, "Because I'm a chicken." :-)
>
> Fortunately, he didn't stop there. He explained that there is no known
> correctness risk -- you're not going to damage your repositories by enabling
> the feature. But he wanted to phase in the feature to allow time to collect
> real-world data about the amount of space savings folks are actually
> observing in their repositories. The feature is on by default in his
> proposed next version of the FSFS format.
>
> > b) Is there a global setting file that can be used to enable
> > directory/property deltifications? Or will we have to update the fsfs.conf
> > file for each newly created repository in order to get this feature?
>
> In 1.8, you'll need to toggle this for each new repository manually.
>
> > c) Is it a safe assumption that in order to apply this change to an older
> > repository that we will need a dump/load cycle? Will we need a full dump or
> > will an delta style dump suffice (--deltas option of svnadmin dump command)?
>
> Not exactly. You can apply the change to an older repository sitting behind
> a server still running 1.8, and any new directory/property lists will be
> stored in a deltified fashion. If you want retroactive deltification of
> existing data, then yes, you'll need to dump and load your repositories.
> But as explained in the release notes, you can dump and re-load right back
> into a previous version of the repository format if you'd prefer to maintain
> compatibility with older server versions.
>
> As for the --deltas option, that has nothing in the world to do with the
> types of deltas we're discussing here. (As an aside, I would highly
> recommend that, unless you need your dumpfiles to be smaller, avoid the
> --deltas option. The performance penalty of using it isn't worth it.)
That's because we store skip-deltas but dumpfiles want deltas against
predecessor, right?
So, two thoughts:
- Is this still a problem with the new max-linear-deltification setting?
- Can we teach 'svnadmin dump' to just dump whatever delta is stored in
the filesystem, plus the base of that delta? For the common case of
loading the entire dump file, it's not really important what the delta
base is so long as it's older than the dumped revision.
Re: svn 1.8 migration - directory deltification and revprop packing
Posted by Daniel Shahaf <da...@elego.de>.
C. Michael Pilato wrote on Tue, Jun 11, 2013 at 17:32:59 +0200:
> On 06/11/2013 05:14 PM, Daniel Shahaf wrote:
> > C. Michael Pilato wrote on Tue, Jun 11, 2013 at 14:52:48 +0200:
> >> As for the --deltas option, that has nothing in the world to do with the
> >> types of deltas we're discussing here. (As an aside, I would highly
> >> recommend that, unless you need your dumpfiles to be smaller, avoid the
> >> --deltas option. The performance penalty of using it isn't worth it.)
> >
> > That's because we store skip-deltas but dumpfiles want deltas against
> > predecessor, right?
> >
> > So, two thoughts:
> >
> > - Is this still a problem with the new max-linear-deltification setting?
> >
> > - Can we teach 'svnadmin dump' to just dump whatever delta is stored in
> > the filesystem, plus the base of that delta? For the common case of
> > loading the entire dump file, it's not really important what the delta
> > base is so long as it's older than the dumped revision.
>
> To do so, we'd need to expose the deltas via the libsvn_fs API, which is
> currently not the case. Deltas in the repository filesystem are an
> implementation detail of that filesystem. They are not even guaranteed to
> be implemented in a fashion that is compatible with what is used in the
> repository dump stream format.
I imagined we could have an API similar to
svn_fs_try_process_file_contents(): if you have an svn_txdelta()-delta
precalculated, hand it, else return SVN_ERR_UNSUPPORTED_FEATURE.
Re: svn 1.8 migration - directory deltification and revprop packing
Posted by Daniel Shahaf <da...@elego.de>.
C. Michael Pilato wrote on Tue, Jun 11, 2013 at 17:32:59 +0200:
> On 06/11/2013 05:14 PM, Daniel Shahaf wrote:
> > C. Michael Pilato wrote on Tue, Jun 11, 2013 at 14:52:48 +0200:
> >> As for the --deltas option, that has nothing in the world to do with the
> >> types of deltas we're discussing here. (As an aside, I would highly
> >> recommend that, unless you need your dumpfiles to be smaller, avoid the
> >> --deltas option. The performance penalty of using it isn't worth it.)
> >
> > That's because we store skip-deltas but dumpfiles want deltas against
> > predecessor, right?
> >
> > So, two thoughts:
> >
> > - Is this still a problem with the new max-linear-deltification setting?
> >
> > - Can we teach 'svnadmin dump' to just dump whatever delta is stored in
> > the filesystem, plus the base of that delta? For the common case of
> > loading the entire dump file, it's not really important what the delta
> > base is so long as it's older than the dumped revision.
>
> To do so, we'd need to expose the deltas via the libsvn_fs API, which is
> currently not the case. Deltas in the repository filesystem are an
> implementation detail of that filesystem. They are not even guaranteed to
> be implemented in a fashion that is compatible with what is used in the
> repository dump stream format.
I imagined we could have an API similar to
svn_fs_try_process_file_contents(): if you have an svn_txdelta()-delta
precalculated, hand it, else return SVN_ERR_UNSUPPORTED_FEATURE.
Re: svn 1.8 migration - directory deltification and revprop packing
Posted by "C. Michael Pilato" <cm...@collab.net>.
On 06/11/2013 05:14 PM, Daniel Shahaf wrote:
> C. Michael Pilato wrote on Tue, Jun 11, 2013 at 14:52:48 +0200:
>> As for the --deltas option, that has nothing in the world to do with the
>> types of deltas we're discussing here. (As an aside, I would highly
>> recommend that, unless you need your dumpfiles to be smaller, avoid the
>> --deltas option. The performance penalty of using it isn't worth it.)
>
> That's because we store skip-deltas but dumpfiles want deltas against
> predecessor, right?
>
> So, two thoughts:
>
> - Is this still a problem with the new max-linear-deltification setting?
>
> - Can we teach 'svnadmin dump' to just dump whatever delta is stored in
> the filesystem, plus the base of that delta? For the common case of
> loading the entire dump file, it's not really important what the delta
> base is so long as it's older than the dumped revision.
To do so, we'd need to expose the deltas via the libsvn_fs API, which is
currently not the case. Deltas in the repository filesystem are an
implementation detail of that filesystem. They are not even guaranteed to
be implemented in a fashion that is compatible with what is used in the
repository dump stream format.
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Enterprise Cloud Development
Re: svn 1.8 migration - directory deltification and revprop packing
Posted by "C. Michael Pilato" <cm...@collab.net>.
On 06/11/2013 05:14 PM, Daniel Shahaf wrote:
> C. Michael Pilato wrote on Tue, Jun 11, 2013 at 14:52:48 +0200:
>> As for the --deltas option, that has nothing in the world to do with the
>> types of deltas we're discussing here. (As an aside, I would highly
>> recommend that, unless you need your dumpfiles to be smaller, avoid the
>> --deltas option. The performance penalty of using it isn't worth it.)
>
> That's because we store skip-deltas but dumpfiles want deltas against
> predecessor, right?
>
> So, two thoughts:
>
> - Is this still a problem with the new max-linear-deltification setting?
>
> - Can we teach 'svnadmin dump' to just dump whatever delta is stored in
> the filesystem, plus the base of that delta? For the common case of
> loading the entire dump file, it's not really important what the delta
> base is so long as it's older than the dumped revision.
To do so, we'd need to expose the deltas via the libsvn_fs API, which is
currently not the case. Deltas in the repository filesystem are an
implementation detail of that filesystem. They are not even guaranteed to
be implemented in a fashion that is compatible with what is used in the
repository dump stream format.
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Enterprise Cloud Development