You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Philip Gladstone <pg...@cisco.com> on 2004/04/12 17:20:16 UTC
Performance of TransmitFile on Windows Servers with 2.0.49
I noticed that the performance of TransmitFile (used when EnableSendFile
On on Windows platforms) was significantly worse than EnableSendFile Off.
It turns out that the way that TransmitFile is called is *without* the
TF_WRITE_BEHIND flag. This means that TransmitFile does not complete (or
rather that the socket is not signalled) until *all the data* has been
ack'ed by the client. Windows clients send acks (roughly) every other
data packet, or after a 200ms timeout. Thus, about half the time, the
TransmitFile does not complete until this 200ms timeout has triggered.
This reduces the throughput on high speed networks significantly. On a
100Mbit LAN, the throughput drops from 11MBytes/sec to around
2.5MBytes/sec.
My question is: is there a good reason that the TF_WRITE_BEHIND flag is
not being used?
Philip
p.s. If you want to try this, don't use Mozilla as that seems to have
another bug that limits the maximum data transfer rate.
--
Philip Gladstone 978-ZEN-TOAD (978-936-8623)
Cisco Systems, Inc Boxboro, MA
Re: Performance of TransmitFile on Windows Servers with 2.0.49
Posted by Bill Stoddard <bi...@wstoddard.com>.
Bill Stoddard wrote:
> Philip Gladstone wrote:
>
>> I noticed that the performance of TransmitFile (used when EnableSendFile
>> On on Windows platforms) was significantly worse than EnableSendFile Off.
>>
>> It turns out that the way that TransmitFile is called is *without* the
>> TF_WRITE_BEHIND flag. This means that TransmitFile does not complete (or
>> rather that the socket is not signalled) until *all the data* has been
>> ack'ed by the client. Windows clients send acks (roughly) every other
>> data packet, or after a 200ms timeout. Thus, about half the time, the
>> TransmitFile does not complete until this 200ms timeout has triggered.
>>
>> This reduces the throughput on high speed networks significantly. On a
>> 100Mbit LAN, the throughput drops from 11MBytes/sec to around
>> 2.5MBytes/sec.
>>
>> My question is: is there a good reason that the TF_WRITE_BEHIND flag is
>> not being used?
Try applying this patch to apr and let me know how it works.
Bill
$ cvs diff -u sendrecv.c
Index: sendrecv.c
===================================================================
RCS file: /home/cvs/apr/network_io/win32/sendrecv.c,v
retrieving revision 1.64.2.1
diff -u -r1.64.2.1 sendrecv.c
--- sendrecv.c 13 Feb 2004 09:33:51 -0000 1.64.2.1
+++ sendrecv.c 12 Apr 2004 16:14:17 -0000
@@ -237,7 +237,7 @@
apr_status_t status = APR_SUCCESS;
apr_ssize_t rv;
apr_off_t curoff = *offset;
- DWORD dwFlags = 0;
+ DWORD dwFlags = TF_WRITE_BEHIND;
DWORD nbytes;
OVERLAPPED overlapped;
TRANSMIT_FILE_BUFFERS tfb, *ptfb = NULL;
@@ -323,6 +323,7 @@
/* Disconnect the socket after last send */
if ((flags & APR_SENDFILE_DISCONNECT_SOCKET)
&& !sendv_trailers) {
+ dwFlags = 0;
dwFlags |= TF_REUSE_SOCKET;
dwFlags |= TF_DISCONNECT;
disconnected = 1;
@@ -333,7 +334,6 @@
#if APR_HAS_LARGE_FILES
overlapped.OffsetHigh = (DWORD)(curoff >> 32);
#endif
- /* XXX BoundsChecker claims dwFlags must not be zero. */
rv = TransmitFile(sock->socketdes, /* socket */
file->filehand, /* open file descriptor of the file to be sent */
nbytes, /* number of bytes to send. 0=send all */
Re: Performance of TransmitFile on Windows Servers with 2.0.49
Posted by Bill Stoddard <bi...@wstoddard.com>.
Bill Stoddard wrote:
> Philip Gladstone wrote:
>
>> I noticed that the performance of TransmitFile (used when EnableSendFile
>> On on Windows platforms) was significantly worse than EnableSendFile Off.
>>
>> It turns out that the way that TransmitFile is called is *without* the
>> TF_WRITE_BEHIND flag. This means that TransmitFile does not complete (or
>> rather that the socket is not signalled) until *all the data* has been
>> ack'ed by the client. Windows clients send acks (roughly) every other
>> data packet, or after a 200ms timeout. Thus, about half the time, the
>> TransmitFile does not complete until this 200ms timeout has triggered.
>>
>> This reduces the throughput on high speed networks significantly. On a
>> 100Mbit LAN, the throughput drops from 11MBytes/sec to around
>> 2.5MBytes/sec.
>>
>> My question is: is there a good reason that the TF_WRITE_BEHIND flag is
>> not being used?
Try applying this patch to apr and let me know how it works.
Bill
$ cvs diff -u sendrecv.c
Index: sendrecv.c
===================================================================
RCS file: /home/cvs/apr/network_io/win32/sendrecv.c,v
retrieving revision 1.64.2.1
diff -u -r1.64.2.1 sendrecv.c
--- sendrecv.c 13 Feb 2004 09:33:51 -0000 1.64.2.1
+++ sendrecv.c 12 Apr 2004 16:14:17 -0000
@@ -237,7 +237,7 @@
apr_status_t status = APR_SUCCESS;
apr_ssize_t rv;
apr_off_t curoff = *offset;
- DWORD dwFlags = 0;
+ DWORD dwFlags = TF_WRITE_BEHIND;
DWORD nbytes;
OVERLAPPED overlapped;
TRANSMIT_FILE_BUFFERS tfb, *ptfb = NULL;
@@ -323,6 +323,7 @@
/* Disconnect the socket after last send */
if ((flags & APR_SENDFILE_DISCONNECT_SOCKET)
&& !sendv_trailers) {
+ dwFlags = 0;
dwFlags |= TF_REUSE_SOCKET;
dwFlags |= TF_DISCONNECT;
disconnected = 1;
@@ -333,7 +334,6 @@
#if APR_HAS_LARGE_FILES
overlapped.OffsetHigh = (DWORD)(curoff >> 32);
#endif
- /* XXX BoundsChecker claims dwFlags must not be zero. */
rv = TransmitFile(sock->socketdes, /* socket */
file->filehand, /* open file descriptor of the file to be sent */
nbytes, /* number of bytes to send. 0=send all */
Re: Performance of TransmitFile on Windows Servers with 2.0.49
Posted by Bill Stoddard <bi...@wstoddard.com>.
Philip Gladstone wrote:
> I noticed that the performance of TransmitFile (used when EnableSendFile
> On on Windows platforms) was significantly worse than EnableSendFile Off.
>
> It turns out that the way that TransmitFile is called is *without* the
> TF_WRITE_BEHIND flag. This means that TransmitFile does not complete (or
> rather that the socket is not signalled) until *all the data* has been
> ack'ed by the client. Windows clients send acks (roughly) every other
> data packet, or after a 200ms timeout. Thus, about half the time, the
> TransmitFile does not complete until this 200ms timeout has triggered.
>
> This reduces the throughput on high speed networks significantly. On a
> 100Mbit LAN, the throughput drops from 11MBytes/sec to around
> 2.5MBytes/sec.
>
> My question is: is there a good reason that the TF_WRITE_BEHIND flag is
> not being used?
>
> Philip
>
> p.s. If you want to try this, don't use Mozilla as that seems to have
> another bug that limits the maximum data transfer rate.
Philip,
Submit a patch to dev@apr.apache.org and I'll take a look. Sounds like we need to set TF_WRITE_BEHIND.
Bill