You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Bert Huijben <be...@vmoo.com> on 2008/10/05 00:23:16 UTC
FindFirstFile delay on merging resolved via profiling (r33447)
> -----Original Message-----
> From: rhuijben@tigris.org [mailto:rhuijben@tigris.org]
> Sent: zondag 5 oktober 2008 1:08
> To: svn@subversion.tigris.org
> Subject: svn commit: r33447 - trunk/subversion/libsvn_subr
>
> Author: rhuijben
> Date: Sat Oct 4 16:08:22 2008
> New Revision: 33447
>
> Log:
> Remove Windows specific side effects from svn_dirent_get_absolute.
> Passing
> the flag APR_FILEPATH_TRUENAME to apr_filepath_merge made it check and
> alter
> the casing of the passed path to the actual on-disk path. This makes
> the call
> several orders slower on windows than on other operating systems that
> don't
> check the path at all. This without actual profit as a case-changed
> path would
> fail on further processing.
>
> Subversion fixes the casing of all user provided paths in
> svn_opt__arg_canonicalize_path, so we shouldn't check it again when we
> make
> a path absolute.
>
> This patch saves several seconds on typical branch merges on Windows.
> (Removes
> a few thousand FindFirstFile calls on my test set involving only 21
> file
> changes on our 1.5.x branch.). The result is that it increases the
> merge
> performance on Windows by over 30%.
This two line patch on svn_dirent_get_absolute is the result of several hours of profiling and it makes the merge test we profiled 30% faster. (It probably increases the speed of many other subversion commands as well).
This very Windows specific slowdown was added in r13 when the call to _fullpath() from the Microsoft C++ library was replaced by the more generic apr_filepath_merge.
On all platforms except windows apr_filepath_merge can only make the path absolute, but on Windows it also allows the flag APR_FILEPATH_TRUENAME that actually verifies if the path exists by checking the exact casing of the path and all its ancestors.
!! It actually performs disk IO !!
On a typical test merge:
svn merge -c 31121,31211,31733,31846,32497,32800,32836,32837,33156,33178 http://svn.collab.net/repos/svn/trunk F:\wc\1.5.x
(F:\wc\1.5.x = 1.5.x branch @33444)
This svn_dirent_get_absolute involves several thousands of calls to FindFirstFile, while only 22 files receive updates.
Removing this flag shaves 7 seconds from the merge time on my quad core pc.
When I run it on a local mirror of the subversion repository the difference is well over 30%. (Over webdav most time is spent waiting on bits to arrive from the USA)
Lgo found some other performance issues around canonicalize (taking 20% of merge on Linux) and we are still looking for other issues, so I hope we will see some other performance improvements in the next few days.
Bert
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FindFirstFile delay on merging resolved via profiling (r33447)
Posted by Erik Huelsmann <eh...@gmail.com>.
On Mon, Oct 6, 2008 at 7:55 PM, Sander Rijken <sr...@d-90.nl> wrote:
>>
>> This two line patch on svn_dirent_get_absolute is the result of several
>> hours of profiling and it makes the merge test we profiled 30% faster. (It
>> probably increases the speed of many other subversion commands as well).
>>
>> This very Windows specific slowdown was added in r13 when the call to
>> _fullpath() from the Microsoft C++ library was replaced by the more generic
>> apr_filepath_merge.
>>
>> On all platforms except windows apr_filepath_merge can only make the path
>> absolute, but on Windows it also allows the flag APR_FILEPATH_TRUENAME that
>> actually verifies if the path exists by checking the exact casing of the
>> path and all its ancestors.
>>
>> !! It actually performs disk IO !!
>>
>> On a typical test merge:
>> svn merge -c 31121,31211,31733,31846,32497,32800,32836,32837,33156,33178
>> http://svn.collab.net/repos/svn/trunk F:\wc\1.5.x
>> (F:\wc\1.5.x = 1.5.x branch @33444)
>>
>> This svn_dirent_get_absolute involves several thousands of calls to
>> FindFirstFile, while only 22 files receive updates.
>>
>> Removing this flag shaves 7 seconds from the merge time on my quad core
>> pc.
>> When I run it on a local mirror of the subversion repository the
>> difference is well over 30%. (Over webdav most time is spent waiting on bits
>> to arrive from the USA)
>>
>
> Actually in the above merge example, 9 merges are performed (one of them has
> already been merged), and for each merge svn_sleep_for_timestamps is called
> (sleeping for 1.1 seconds)
> This means that going from 19 to 12 seconds, should actually be explained as
> going from 9 to 2 seconds, which is a far greater improvement.
>
> Do we need the sleep function to be called after merging every revision
> (most commands that call it only call it when they're finished)?
No, we need it only at the end.
Bye,
Erik.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: FindFirstFile delay on merging resolved via profiling (r33447)
Posted by Sander Rijken <sr...@d-90.nl>.
>
> This two line patch on svn_dirent_get_absolute is the result of
> several hours of profiling and it makes the merge test we profiled
> 30% faster. (It probably increases the speed of many other
> subversion commands as well).
>
> This very Windows specific slowdown was added in r13 when the call
> to _fullpath() from the Microsoft C++ library was replaced by the
> more generic apr_filepath_merge.
>
> On all platforms except windows apr_filepath_merge can only make the
> path absolute, but on Windows it also allows the flag
> APR_FILEPATH_TRUENAME that actually verifies if the path exists by
> checking the exact casing of the path and all its ancestors.
>
> !! It actually performs disk IO !!
>
> On a typical test merge:
> svn merge -c
> 31121,31211,31733,31846,32497,32800,32836,32837,33156,33178 http://svn.collab.net/repos/svn/trunk
> F:\wc\1.5.x
> (F:\wc\1.5.x = 1.5.x branch @33444)
>
> This svn_dirent_get_absolute involves several thousands of calls to
> FindFirstFile, while only 22 files receive updates.
>
> Removing this flag shaves 7 seconds from the merge time on my quad
> core pc.
> When I run it on a local mirror of the subversion repository the
> difference is well over 30%. (Over webdav most time is spent waiting
> on bits to arrive from the USA)
>
Actually in the above merge example, 9 merges are performed (one of
them has already been merged), and for each merge
svn_sleep_for_timestamps is called (sleeping for 1.1 seconds)
This means that going from 19 to 12 seconds, should actually be
explained as going from 9 to 2 seconds, which is a far greater
improvement.
Do we need the sleep function to be called after merging every
revision (most commands that call it only call it when they're
finished)?
--
Sander Rijken
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org