You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@subversion.apache.org by st...@apache.org on 2012/10/14 06:28:34 UTC
svn commit: r1398004 - /subversion/site/publish/docs/release-notes/1.8.html
Author: stefan2
Date: Sun Oct 14 04:28:34 2012
New Revision: 1398004
URL: http://svn.apache.org/viewvc?rev=1398004&view=rev
Log:
Add sections for the 1.8 FSFS and server enhancements.
* site/publish/docs/release-notes/1.8.html
(fsfs-enhancements, new-tools): new sections
(svnserve): describe --client-speed parameter
Modified:
subversion/site/publish/docs/release-notes/1.8.html
Modified: subversion/site/publish/docs/release-notes/1.8.html
URL: http://svn.apache.org/viewvc/subversion/site/publish/docs/release-notes/1.8.html?rev=1398004&r1=1398003&r2=1398004&view=diff
==============================================================================
--- subversion/site/publish/docs/release-notes/1.8.html (original)
+++ subversion/site/publish/docs/release-notes/1.8.html Sun Oct 14 04:28:34 2012
@@ -40,6 +40,10 @@ describes what is planned for the releas
>HTTP client support based on neon has been removed</a></li>
<li><a href="#gpg-agent"
>In-memory password caching via GnuPG (<em>Unix client</em>)</a></li>
+ <li><a href="#fsfs-enhancements"
+ >FSFS size and performance enhancements</a></li>
+ <li><a href="#new-tools"
+ >New tools for administrators and infrastructure</a></li>
<li><a href="#enhancements"
>Many enhancements and bug fixes</a></li>
</ul>
@@ -458,6 +462,197 @@ href="http://svn.apache.org/viewvc/subve
>here</a>.</p>
</div> <!-- gpg-agent -->
+<div class="h3" id="fsfs-enhancements">
+<h3>FSFS size and performance enhancements
+ <a class="sectionlink" href="#fsfs-enhancements"
+ title="Link to this section">¶</a>
+</h3>
+
+<p>Subversion 1.8 introduces a number of improvements that can reduce
+the size of FSFS repositories, the total number of files on disk, the
+I/O overhead and gives the user fine-grained control over all of that.
+Some of these changes require a format bump, others are backward-compatible.
+</p>
+
+<div class="h4" id="fsfs-format-6">
+<h4>FSFS format bump
+ <a class="sectionlink" href="#fsfs-format-6"
+ title="Link to this section">¶</a>
+</h4>
+
+<p>The default repository format created by <tt>svnadmin create</tt> is
+now FSFS version 6 which is not be accessible to Subversion 1.7 or older.
+To create FSFS repositories compatible with Subversion 1.6 and 1.7, use
+the <tt>--pre-1.8-compatible</tt> parameter. Older repository formats
+remain fully supported by Subversion 1.8 but will not support
+<a href="#revprop-packing">revprop packing</a>.</p>
+
+<p>You may use <tt>svnadmin upgrade</tt> to upgrade existing repositories.
+However, to fully benefit from the latest <a href="#fsfs-deltification">
+repository size reductions</a>, it is recommended to create a new repository,
+adjust its settings and then dump / load or svnsync the content into it.
+</p>
+
+</div> <!-- fsfs-format-6 -->
+
+<div class="h4" id="revprop-packing">
+<h4>Packing revision properties
+ <a class="sectionlink" href="#revprop-packing"
+ title="Link to this section">¶</a>
+</h4>
+
+<p><a href="#fsfs-format-6">Format 6 repositories</a> will not only
+pack the revision files but also the revision property files. Upgrading
+exisiting packed repositories will automatically pack the revision
+properties. Using default settings, this will reduce the number of
+revprop files to less than 10 per 1000 revisions in typical usage
+scenarios.</p>
+
+<p>The <tt>db/fsfs.conf</tt> file allows you to fine-tune the
+revprop packing strategy (see the comments in that file for a
+detailed description of the available options). Please note that any
+change in this configuration file will not affect existing data.
+Moreover, upgrading repositories will not extend the existing
+configuration file, i.e. the server will use the default settings
+unless the new options are added explicitly.</p>
+
+<p>It is recommended also to activate <a href="#revprop-caching">
+revprop caching</a> when you use revprop packing. Otherwise, access
+to timestamps etc. will incur significant overhead.</p>
+
+</div> <!-- revprop-packing -->
+
+<div class="h4" id="revprop-caching">
+<h4>Caching revision properties
+ <a class="sectionlink" href="#revprop-caching"
+ title="Link to this section">¶</a>
+</h4>
+
+<p>Many operations will access revision properties to e.g. retrieve
+the commit time stamp. Therefore, thousands of revision property
+files may be read during a checkout. The UI to enable the revprop
+cache is similar to that for the other caches: <tt>svnserve</tt>
+now accepts the <tt>--cache-revprops yes</tt> parameter. For
+<tt>mod_dav_svn</tt>, the same looks like this in <tt>httpd.conf</tt>
+
+<pre>
+<IfModule dav_svn_module>
+ SVNCacheRevProps on
+</IfModule>
+</pre>
+
+<p>With <a href="#revprop-packing">revision property packing</a>
+enabled, revprop caching allows you to read hundreds of revprops
+at once from a single file. This can give e.g. <tt>svn log</tt>
+a significant performance boost.</p>
+
+</div> <!-- revprop-caching -->
+
+<div class="h4" id="fsfs-deltification">
+<h4>Directory and property storage reduction
+ <a class="sectionlink" href="#fsfs-deltification"
+ title="Link to this section">¶</a>
+</h4>
+
+<p>For each changed node in an FSFS repository, new versions of
+all parent directories will be created. Larger repositories tend
+to have relatively deep directory structures with at least one
+level (branches, modules or projects) having tens of entries.
+The total size of that data may well exceed the size of the actual
+change. Subversion 1.8 now supports directory deltification
+which eliminates most of that overhead.</p>
+
+<p>In <tt>db/fsfs.conf</tt>, you may now enable and disable
+directory deltification at any time and these settings will be
+applied in new revisions. For completeness, node properties may
+now be deltified as well although the reductions in repository
+size will usually be minimal. By default, directory and property
+deltification is off. Also, <tt>db/fsfs.conf</tt> now allows for
+a fine-grained control over how deltification will be applied.
+See the comments in that file for a detailed description of the
+individual options.
+</p>
+
+<p>Another optimization in 1.8 is that node properties with the
+same content will only be stored once and then referenced. This
+not only slightly reduces the size of the repository but greatly
+improves cache efficiency and can also significantly reduce I/O.
+</p>
+
+<p><b>It's backward compatible!</b> Please note that the mechanism
+to read deltified data is the same for file content as for directories
+and properties. Hence, Subversion 1.6 and 1.7 will be able to read
+them but will always write new revisions without that optimization.
+It is therefore perfectly legal to create a pre-1.8-compatible repository
+in 1.8, to enable various deltification options, to dump/load into
+the new repository using 1.8 tooling and to finally use it with a 1.6
+or 1.7 server.
+</p>
+
+</div> <!-- fsfs-deltification -->
+
+<div class="h4" id="fsfs-rep-sharing">
+<h4>Rep-sharing within revisions
+ <a class="sectionlink" href="#fsfs-rep-sharing"
+ title="Link to this section">¶</a>
+</h4>
+
+<p>When representation sharing has been enabled, Subversion 1.8
+will now be able to detect identical files with content within
+the same revision and only store them once. This is a common situation
+when you for instance import a non-incremental dump file or when
+users apply the same change to multiple branches in a single commit.
+</p>
+
+</div> <!-- fsfs-rep-sharing -->
+
+</div> <!-- fsfs-enhancements -->
+
+<div class="h3" id="new-tools">
+<h3>New tools and utilities
+ <a class="sectionlink" href="#new-tools"
+ title="Link to this section">¶</a>
+</h3>
+
+<div class="h4" id="benchmarking">
+<h4><tt>svn-bench</tt> Benchmarking client
+ <a class="sectionlink" href="#benchmarking"
+ title="Link to this section">¶</a>
+</h4>
+
+<p>Identifying bottlenecks in your Subversion infrastructure may be
+difficult because client-side influences (virus scanners, slow disks
+etc.) tend to mask other limitations. Therefore, Subversion 1.8
+introduces a very lightweight client that skips most of that local
+processing.</p>
+
+<p><tt>svn-bench</tt> provides commands similar to the standard
+command line interface, adding a <tt>null-</tt> prefix to them.
+Currently implemented are <tt>null-export</tt>, <tt>null-list</tt>
+and <tt>null-log</tt>. They support most of the parameters of their
+standard command counterparts.</p>
+
+<pre>
+$ time svn-bench null-export svn://localhost/tsvn_repos/trunk
+ 179 directories
+ 3,661 files
+ 84,902,674 bytes in files
+ 20,560 properties
+ 315,031 bytes in properties
+
+real 0m0.185s
+user 0m0.128s
+sys 0m0.040s
+</pre>
+
+<p>By running the client close to or directly on the server machine,
+you will be able to determine the raw server speed and compare that
+to what the client-side is seeing.</p>
+
+</div> <!-- benchmarking -->
+
+</div> <!-- new-tools -->
+
</div> <!-- new-features -->
<div class="h2" id="enhancements">
@@ -711,6 +906,32 @@ none, read-only, or read-write.</p>
</div> <!-- cmdline-admin -->
+<div class="h3" id="svnserve">
+<h3>Support for high-speed networks in 'svnserve'
+ <a class="sectionlink" href="#svnserve"
+ title="Link to this section">¶</a>
+</h3>
+
+<p>If your server has a 10 Gbit/s or faster network connection, the
+multi-stage data copying from caches to the network stack plus frequent
+status checks become a bottleneck. Today, most clients won't be able
+to generate enough aggregated traffic to make this an issue and an 8-core
+server should be able to send about 40Gb/s at 100% CPU load. Future
+clients may become able, however, to process 1Gb or even 10Gb per second
+and to create a seriously server loads.
+</p>
+
+<p>When the server is given the <tt>--client-speed N</tt> parameter,
+it will assume that the clients are able to process data rates of
+N Gbit/s. With N>0, the server will take various shortcuts to reduce
+the processing overhead. On the downside, it must employ strict time-outs
+to prevent clients from interfering each other: In any 1 second interval,
+a client must process incoming data at at least 2% of the specified
+speed or the operation may be aborted due to timeouts.
+</p>
+
+</div> <!-- svnserve -->
+
<div class="h3" id="cmdline">
<h3>Command-line client improvements (<em>client</em>)
<a class="sectionlink" href="#cmdline"
Re: svn commit: r1398004 - /subversion/site/publish/docs/release-notes/1.8.html
Posted by Stefan Fuhrmann <st...@wandisco.com>.
On Sun, Oct 14, 2012 at 7:31 PM, Daniel Shahaf <d....@daniel.shahaf.name>wrote:
> stefan2@apache.org wrote on Sun, Oct 14, 2012 at 04:28:34 -0000:
> > +However, to fully benefit from the latest <a href="#fsfs-deltification">
> > +repository size reductions</a>, it is recommended to create a new
> repository,
>
> For future reference: no space between '>' and 'r' please.
>
I'll try to remember that but I rarely touch html stuff -
so I may forget ...
> +clients may become able, however, to process 1Gb or even 10Gb per second
> > +and to create a seriously server loads.
>
> Adjective missing after 'seriously'?
>
> > +</p>
> > +
> > +<p>When the server is given the <tt>--client-speed N</tt> parameter,
> > +it will assume that the clients are able to process data rates of
> > +N Gbit/s. With N>0, the server will take various shortcuts to reduce
>
> Can N be fractional? If yes say so, if not I suggest changing N to be
> in Mbps to enable fine-tuning.
>
Both items changed in r1399497.
> <div class="h3" id="cmdline">
> > <h3>Command-line client improvements (<em>client</em>)
> > <a class="sectionlink" href="#cmdline"
> >
> >
>
> Looks good.
>
Thanks for the review!
-- Stefan^2.
--
*
Join us this October at Subversion Live
2012<http://www.wandisco.com/svn-live-2012>
for two days of best practice SVN training, networking, live demos,
committer meet and greet, and more! Space is limited, so get signed up
today<http://www.wandisco.com/svn-live-2012>
!
*
Re: svn commit: r1398004 - /subversion/site/publish/docs/release-notes/1.8.html
Posted by Stefan Fuhrmann <st...@wandisco.com>.
On Sun, Oct 14, 2012 at 7:31 PM, Daniel Shahaf <d....@daniel.shahaf.name>wrote:
> stefan2@apache.org wrote on Sun, Oct 14, 2012 at 04:28:34 -0000:
> > +However, to fully benefit from the latest <a href="#fsfs-deltification">
> > +repository size reductions</a>, it is recommended to create a new
> repository,
>
> For future reference: no space between '>' and 'r' please.
>
I'll try to remember that but I rarely touch html stuff -
so I may forget ...
> +clients may become able, however, to process 1Gb or even 10Gb per second
> > +and to create a seriously server loads.
>
> Adjective missing after 'seriously'?
>
> > +</p>
> > +
> > +<p>When the server is given the <tt>--client-speed N</tt> parameter,
> > +it will assume that the clients are able to process data rates of
> > +N Gbit/s. With N>0, the server will take various shortcuts to reduce
>
> Can N be fractional? If yes say so, if not I suggest changing N to be
> in Mbps to enable fine-tuning.
>
Both items changed in r1399497.
> <div class="h3" id="cmdline">
> > <h3>Command-line client improvements (<em>client</em>)
> > <a class="sectionlink" href="#cmdline"
> >
> >
>
> Looks good.
>
Thanks for the review!
-- Stefan^2.
--
*
Join us this October at Subversion Live
2012<http://www.wandisco.com/svn-live-2012>
for two days of best practice SVN training, networking, live demos,
committer meet and greet, and more! Space is limited, so get signed up
today<http://www.wandisco.com/svn-live-2012>
!
*
Re: svn commit: r1398004 -
/subversion/site/publish/docs/release-notes/1.8.html
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
stefan2@apache.org wrote on Sun, Oct 14, 2012 at 04:28:34 -0000:
> +However, to fully benefit from the latest <a href="#fsfs-deltification">
> +repository size reductions</a>, it is recommended to create a new repository,
For future reference: no space between '>' and 'r' please.
> +clients may become able, however, to process 1Gb or even 10Gb per second
> +and to create a seriously server loads.
Adjective missing after 'seriously'?
> +</p>
> +
> +<p>When the server is given the <tt>--client-speed N</tt> parameter,
> +it will assume that the clients are able to process data rates of
> +N Gbit/s. With N>0, the server will take various shortcuts to reduce
Can N be fractional? If yes say so, if not I suggest changing N to be
in Mbps to enable fine-tuning.
> <div class="h3" id="cmdline">
> <h3>Command-line client improvements (<em>client</em>)
> <a class="sectionlink" href="#cmdline"
>
>
Looks good.
Daniel
Re: svn commit: r1398004 -
/subversion/site/publish/docs/release-notes/1.8.html
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
stefan2@apache.org wrote on Sun, Oct 14, 2012 at 04:28:34 -0000:
> +However, to fully benefit from the latest <a href="#fsfs-deltification">
> +repository size reductions</a>, it is recommended to create a new repository,
For future reference: no space between '>' and 'r' please.
> +clients may become able, however, to process 1Gb or even 10Gb per second
> +and to create a seriously server loads.
Adjective missing after 'seriously'?
> +</p>
> +
> +<p>When the server is given the <tt>--client-speed N</tt> parameter,
> +it will assume that the clients are able to process data rates of
> +N Gbit/s. With N>0, the server will take various shortcuts to reduce
Can N be fractional? If yes say so, if not I suggest changing N to be
in Mbps to enable fine-tuning.
> <div class="h3" id="cmdline">
> <h3>Command-line client improvements (<em>client</em>)
> <a class="sectionlink" href="#cmdline"
>
>
Looks good.
Daniel