You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Stefan Fuhrmann <st...@alice-dsl.de> on 2010/03/28 12:50:49 UTC
[PATCH] delta_files() speedup 3/3: file write buffering
Hi devs,
this is part of the delta_files() speedup patch series:
http://svn.haxx.se/dev/archive-2010-03/0604.shtml
Under Windows, a significant part of the overall runtime
is spent in writing data to the (already buffered) APR
file implementation. It seems that the mutex serializing
the buffer access in apr_file_write is expensive.
Also, >50% of all write requests are 2 bytes or smaller
(i.e. line endings and empty lines). For them, the deep
call hierarchy constitutes a large overhead on register-
lacking x86.
This patch eliminates far over 90% of all write requests
bringing the portion of time spent in _svn_io_file_write_full
down from about 7 to below 3 percent on 32 bit Windows.
Performance gain is ~1% under Linux but due to the
larger async I/O and mutex overhead it is about 4%
under Windows:
s~$ time ~/1.7-928181/svn export --ignore-externals -q $REPO/trunk /dev/shm/t
real 0m3.727s
user 0m3.189s
sys 0m0.542s
~$ time ~/1.7-patched/svn export --ignore-externals -q $REPO/trunk /dev/shm/t
real 0m3.697s
user 0m3.148s
sys 0m0.558s
-- Stefan^2.
[[[
Buffer small write requests to APR file streams. For details see
http:// ...
* subversion/libsvn_subr/stream.c
(baton_apr): add write buffer members
(flush_buffer_apr): new function
(read_handler_apr): auto-allocate write buffer and re-direct
small writes there.
(read_handler_apr, close_handler_apr, mark_handler_apr,
seek_handler_apr): flush write buffer before calling i/o functions.
patch by stefanfuhrmann < at > alice-dsl.de
]]]
Re: [PATCH] delta_files() speedup 3/3: file write buffering
Posted by Stefan Fuhrmann <st...@alice-dsl.de>.
On Monday 29 March 2010 16:32:08 you wrote:
> [CC to dev@apr.apache.org]
>
> From: Stefan Fuhrmann [mailto:stefanfuhrmann@alice-dsl.de]
> > Under Windows, a significant part of the overall runtime
> > is spent in writing data to the (already buffered) APR
> > file implementation. It seems that the mutex serializing
> > the buffer access in apr_file_write is expensive.
> >
> > Also, >50% of all write requests are 2 bytes or smaller
> > (i.e. line endings and empty lines). For them, the deep
> > call hierarchy constitutes a large overhead on register-
> > lacking x86.
> >
> > This patch eliminates far over 90% of all write requests
> > bringing the portion of time spent in _svn_io_file_write_full
> > down from about 7 to below 3 percent on 32 bit Windows.
> >
> > Performance gain is ~1% under Linux but due to the
> > larger async I/O and mutex overhead it is about 4%
> > under Windows:
>
> Hi Stefan,
>
> Thanks for looking into this.
>
> I'm just wondering why APR does use the mutexes even though APR_XTHREAD
> isn't passed?
> Maybe because the mutex APR uses, a critical section, is more costly now
> than when this code was implemented several years ago?
> (On a singlecore machine a simple interlocked increment is very cheap, but
> on a multicore machine it requires at least some synchronization).
>
> Looking at the APR help it might be better to fix the buffering in APR,
> than to add yet another level of buffering. (Our buffer, APR's buffering
> and then the OS buffering).
>
> After a quick search through the apr code it seems that only windows tries
> to make file writes always thread safe, while other operating systems use
> APR_XTHREAD as a trigger.
>
>
> The Windows code also enables overlapped IO on APR_XTHREAD, so there could
> be some valid use where you want thread safe code, but not overlapped. But
> why only do this on Windows? (Maybe the APR team can answer this question).
Thank you for the feedback. I will try to come up with a patch
that reduces the APR file access overhead. Depending on
the results, I would then either send a patch to the APR team
or continue the discussion here.
-- Stefan^2.
Re: [PATCH] delta_files() speedup 3/3: file write buffering
Posted by Stefan Fuhrmann <st...@alice-dsl.de>.
On Monday 29 March 2010 16:32:08 you wrote:
> [CC to dev@apr.apache.org]
>
> From: Stefan Fuhrmann [mailto:stefanfuhrmann@alice-dsl.de]
> > Under Windows, a significant part of the overall runtime
> > is spent in writing data to the (already buffered) APR
> > file implementation. It seems that the mutex serializing
> > the buffer access in apr_file_write is expensive.
> >
> > Also, >50% of all write requests are 2 bytes or smaller
> > (i.e. line endings and empty lines). For them, the deep
> > call hierarchy constitutes a large overhead on register-
> > lacking x86.
> >
> > This patch eliminates far over 90% of all write requests
> > bringing the portion of time spent in _svn_io_file_write_full
> > down from about 7 to below 3 percent on 32 bit Windows.
> >
> > Performance gain is ~1% under Linux but due to the
> > larger async I/O and mutex overhead it is about 4%
> > under Windows:
>
> Hi Stefan,
>
> Thanks for looking into this.
>
> I'm just wondering why APR does use the mutexes even though APR_XTHREAD
> isn't passed?
> Maybe because the mutex APR uses, a critical section, is more costly now
> than when this code was implemented several years ago?
> (On a singlecore machine a simple interlocked increment is very cheap, but
> on a multicore machine it requires at least some synchronization).
>
> Looking at the APR help it might be better to fix the buffering in APR,
> than to add yet another level of buffering. (Our buffer, APR's buffering
> and then the OS buffering).
>
> After a quick search through the apr code it seems that only windows tries
> to make file writes always thread safe, while other operating systems use
> APR_XTHREAD as a trigger.
>
>
> The Windows code also enables overlapped IO on APR_XTHREAD, so there could
> be some valid use where you want thread safe code, but not overlapped. But
> why only do this on Windows? (Maybe the APR team can answer this question).
Thank you for the feedback. I will try to come up with a patch
that reduces the APR file access overhead. Depending on
the results, I would then either send a patch to the APR team
or continue the discussion here.
-- Stefan^2.
RE: [PATCH] delta_files() speedup 3/3: file write buffering
Posted by Bert Huijben <be...@qqmail.nl>.
[CC to dev@apr.apache.org]
> -----Original Message-----
> From: Stefan Fuhrmann [mailto:stefanfuhrmann@alice-dsl.de]
> Sent: zondag 28 maart 2010 14:51
> To: dev@subversion.apache.org
> Subject: [PATCH] delta_files() speedup 3/3: file write buffering
>
> Hi devs,
>
> this is part of the delta_files() speedup patch series:
> http://svn.haxx.se/dev/archive-2010-03/0604.shtml
>
> Under Windows, a significant part of the overall runtime
> is spent in writing data to the (already buffered) APR
> file implementation. It seems that the mutex serializing
> the buffer access in apr_file_write is expensive.
>
> Also, >50% of all write requests are 2 bytes or smaller
> (i.e. line endings and empty lines). For them, the deep
> call hierarchy constitutes a large overhead on register-
> lacking x86.
>
> This patch eliminates far over 90% of all write requests
> bringing the portion of time spent in _svn_io_file_write_full
> down from about 7 to below 3 percent on 32 bit Windows.
>
> Performance gain is ~1% under Linux but due to the
> larger async I/O and mutex overhead it is about 4%
> under Windows:
Hi Stefan,
Thanks for looking into this.
I'm just wondering why APR does use the mutexes even though APR_XTHREAD
isn't passed?
Maybe because the mutex APR uses, a critical section, is more costly now
than when this code was implemented several years ago?
(On a singlecore machine a simple interlocked increment is very cheap, but
on a multicore machine it requires at least some synchronization).
Looking at the APR help it might be better to fix the buffering in APR, than
to add yet another level of buffering. (Our buffer, APR's buffering and then
the OS buffering).
After a quick search through the apr code it seems that only windows tries
to make file writes always thread safe, while other operating systems use
APR_XTHREAD as a trigger.
The Windows code also enables overlapped IO on APR_XTHREAD, so there could
be some valid use where you want thread safe code, but not overlapped. But
why only do this on Windows? (Maybe the APR team can answer this question).
Bert
>
> s~$ time ~/1.7-928181/svn export --ignore-externals -q $REPO/trunk
> /dev/shm/t
> real 0m3.727s
> user 0m3.189s
> sys 0m0.542s
>
> ~$ time ~/1.7-patched/svn export --ignore-externals -q $REPO/trunk
> /dev/shm/t
> real 0m3.697s
> user 0m3.148s
> sys 0m0.558s
>
> -- Stefan^2.
>
> [[[
> Buffer small write requests to APR file streams. For details see
> http:// ...
>
> * subversion/libsvn_subr/stream.c
> (baton_apr): add write buffer members
> (flush_buffer_apr): new function
> (read_handler_apr): auto-allocate write buffer and re-direct
> small writes there.
> (read_handler_apr, close_handler_apr, mark_handler_apr,
> seek_handler_apr): flush write buffer before calling i/o functions.
>
> patch by stefanfuhrmann < at > alice-dsl.de
> ]]]
RE: [PATCH] delta_files() speedup 3/3: file write buffering
Posted by Bert Huijben <be...@qqmail.nl>.
[CC to dev@apr.apache.org]
> -----Original Message-----
> From: Stefan Fuhrmann [mailto:stefanfuhrmann@alice-dsl.de]
> Sent: zondag 28 maart 2010 14:51
> To: dev@subversion.apache.org
> Subject: [PATCH] delta_files() speedup 3/3: file write buffering
>
> Hi devs,
>
> this is part of the delta_files() speedup patch series:
> http://svn.haxx.se/dev/archive-2010-03/0604.shtml
>
> Under Windows, a significant part of the overall runtime
> is spent in writing data to the (already buffered) APR
> file implementation. It seems that the mutex serializing
> the buffer access in apr_file_write is expensive.
>
> Also, >50% of all write requests are 2 bytes or smaller
> (i.e. line endings and empty lines). For them, the deep
> call hierarchy constitutes a large overhead on register-
> lacking x86.
>
> This patch eliminates far over 90% of all write requests
> bringing the portion of time spent in _svn_io_file_write_full
> down from about 7 to below 3 percent on 32 bit Windows.
>
> Performance gain is ~1% under Linux but due to the
> larger async I/O and mutex overhead it is about 4%
> under Windows:
Hi Stefan,
Thanks for looking into this.
I'm just wondering why APR does use the mutexes even though APR_XTHREAD
isn't passed?
Maybe because the mutex APR uses, a critical section, is more costly now
than when this code was implemented several years ago?
(On a singlecore machine a simple interlocked increment is very cheap, but
on a multicore machine it requires at least some synchronization).
Looking at the APR help it might be better to fix the buffering in APR, than
to add yet another level of buffering. (Our buffer, APR's buffering and then
the OS buffering).
After a quick search through the apr code it seems that only windows tries
to make file writes always thread safe, while other operating systems use
APR_XTHREAD as a trigger.
The Windows code also enables overlapped IO on APR_XTHREAD, so there could
be some valid use where you want thread safe code, but not overlapped. But
why only do this on Windows? (Maybe the APR team can answer this question).
Bert
>
> s~$ time ~/1.7-928181/svn export --ignore-externals -q $REPO/trunk
> /dev/shm/t
> real 0m3.727s
> user 0m3.189s
> sys 0m0.542s
>
> ~$ time ~/1.7-patched/svn export --ignore-externals -q $REPO/trunk
> /dev/shm/t
> real 0m3.697s
> user 0m3.148s
> sys 0m0.558s
>
> -- Stefan^2.
>
> [[[
> Buffer small write requests to APR file streams. For details see
> http:// ...
>
> * subversion/libsvn_subr/stream.c
> (baton_apr): add write buffer members
> (flush_buffer_apr): new function
> (read_handler_apr): auto-allocate write buffer and re-direct
> small writes there.
> (read_handler_apr, close_handler_apr, mark_handler_apr,
> seek_handler_apr): flush write buffer before calling i/o functions.
>
> patch by stefanfuhrmann < at > alice-dsl.de
> ]]]