You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Wilfredo Sánchez Vega <ws...@apple.com> on 2007/11/13 22:06:42 UTC
sendfile(2) misbehaves when header iovecs are specified
Davi-
Regarding this issue:
http://people.apache.org/~davi/leopard-sendfile.c
Our kernel folks say this isn't a bug:
No, this is not a bug. Our sendfile(2) implementation is
similar to FreeBSD 4.x, where the 4th argument to the system
call (the value-result length parameter) includes the header
length. In FreeBSD 5.x, this length does not include the
header, but that is not the semantics followed by Mac OS X.
Is this sufficient information? I've still got the bug open until
I hear from you.
-wsv
Re: sendfile(2) misbehaves when header iovecs are specified
Posted by Wilfredo Sánchez Vega <ws...@apple.com>.
OK, I've sent the bug back with this comment/request.
-wsv
On Nov 13, 2007, at 2:22 PM, Aaron Bannert wrote:
> Hmm, that's not the behavior I am seeing with Leopard's sendfile. I
> am seeing the value-result parameter come back with the total byes
> in the file plus the total size of the trailers, excluding the
> headers.
>
> For example, with a header of 80,000 bytes, a file of 200,000 bytes,
> and a trailer of 90,029 bytes (from APR's test/sendfile.c test). The
> input to the *len param starts as 200,000 (the size of the file) and
> after the call it comes back set as 290,029. If you're saying that
> the *len return result should be the total bytes sent (headers +
> file + trailers) then the result should have been 370,049. Note that
> in this example, the other end of the connection appears to have
> received at least 119980 bytes.
>
>
> Could we get a detailed explanation of what all the expected inputs
> and outputs are for sendfile in Leopard() (and if it's different
> that previous versions, the same for those would help too).
>
> -aaron
>
>
> On Nov 13, 2007, at 1:06 PM, Wilfredo Sánchez Vega wrote:
>
>> Davi-
>>
>> Regarding this issue:
>>
>> http://people.apache.org/~davi/leopard-sendfile.c
>>
>> Our kernel folks say this isn't a bug:
>>
>> No, this is not a bug. Our sendfile(2) implementation is
>> similar to FreeBSD 4.x, where the 4th argument to the system
>> call (the value-result length parameter) includes the header
>> length. In FreeBSD 5.x, this length does not include the
>> header, but that is not the semantics followed by Mac OS X.
>>
>> Is this sufficient information? I've still got the bug open until
>> I hear from you.
>>
>> -wsv
>>
>
Re: sendfile(2) misbehaves when header iovecs are specified
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Since the hd^H^Htr iovecs work as expected...
I believe we want to implement the header vecs in terms of a writev call,
followed by a sendfile of the file (with file-length) and trailer vecs.
This will spare us from walking the header vector list for sizes and making
all sorts of asinine adjustments, while reducing it still to two kernel
calls.
Comments?
Bill
Davi Arnaut wrote:
> Aaron Bannert wrote:
>> Hmm, that's not the behavior I am seeing with Leopard's sendfile. I am
>> seeing the value-result parameter come back with the total byes in the
>> file plus the total size of the trailers, excluding the headers.
>>
>> For example, with a header of 80,000 bytes, a file of 200,000 bytes,
>> and a trailer of 90,029 bytes (from APR's test/sendfile.c test). The
>> input to the *len param starts as 200,000 (the size of the file) and
>> after the call it comes back set as 290,029. If you're saying that the
>> *len return result should be the total bytes sent (headers + file +
>> trailers) then the result should have been 370,049. Note that in this
>> example, the other end of the connection appears to have received at
>> least 119980 bytes.
>>
>>
>> Could we get a detailed explanation of what all the expected inputs
>> and outputs are for sendfile in Leopard() (and if it's different that
>> previous versions, the same for those would help too).
>>
>
>>>From what I could figure out, it works like this:
>
> o If length <= headers_length: Length output is headers length +
> trailers length. No bytes from the file are sent, headers and trailers
> are sent.
>
> o If length > headers_length, Length output is headers length + trailers
> length + file bytes sent. (length - headers length) bytes from the file
> are sent, headers and trailers are sent.
>
> o If length == 0, Length is headers_length + trailers_length + file
> size. Everything is sent.
>
> Strange...
>
> Regards,
>
> Davi Arnaut
>
>
Re: sendfile(2) misbehaves when header iovecs are specified
Posted by Davi Arnaut <da...@apache.org>.
Wilfredo Sánchez Vega wrote:
> More from the networking team:
>
> """
> With regard's to Aaron's comments above, the return value is as
> expected. The "len" parameter specifies the number of bytes for the
> header (if any) and the (possibly partial) contents of the file. In
> this case, it is 200,000, meaning the entire 80,000 bytes header plus
> 120,000 of the file contents, followed by the trailer, which is 90,029
> bytes, will be sent. The total bytes sent is therefore 290,029 bytes
> (80,000 + 120,000 + 90,029). It implies that 80,000 bytes of the file
> contents did not get sent; only the first 120,000 bytes did.
>
> So, to elaborate this, on input "len" implies the maximum number of
> bytes in the header and/or file to be sent. It does not control the
> trailer whatsoever; if a trailer exists, all of it will be sent. If
> "len" is 0, all of the header and/or file will be sent before the
> trailer (all of it, always) finally gets sent. On output, "len"
> specifies the total number of bytes sent.
>
> I agree the man page is not clear about this and should be modified.
> """
>
> This looks to agree with what you are seeing. Correct?
>
Yes. Thanks.
--
Davi Arnaut
Re: sendfile(2) misbehaves when header iovecs are specified
Posted by Wilfredo Sánchez Vega <ws...@apple.com>.
More from the networking team:
"""
With regard's to Aaron's comments above, the return value is as
expected. The "len" parameter specifies the number of bytes for the
header (if any) and the (possibly partial) contents of the file. In
this case, it is 200,000, meaning the entire 80,000 bytes header plus
120,000 of the file contents, followed by the trailer, which is 90,029
bytes, will be sent. The total bytes sent is therefore 290,029 bytes
(80,000 + 120,000 + 90,029). It implies that 80,000 bytes of the file
contents did not get sent; only the first 120,000 bytes did.
So, to elaborate this, on input "len" implies the maximum number of
bytes in the header and/or file to be sent. It does not control the
trailer whatsoever; if a trailer exists, all of it will be sent. If
"len" is 0, all of the header and/or file will be sent before the
trailer (all of it, always) finally gets sent. On output, "len"
specifies the total number of bytes sent.
I agree the man page is not clear about this and should be modified.
"""
This looks to agree with what you are seeing. Correct?
-wsv
On Nov 13, 2007, at 5:27 PM, Davi Arnaut wrote:
> Aaron Bannert wrote:
>> Hmm, that's not the behavior I am seeing with Leopard's sendfile. I
>> am
>> seeing the value-result parameter come back with the total byes in
>> the
>> file plus the total size of the trailers, excluding the headers.
>>
>> For example, with a header of 80,000 bytes, a file of 200,000 bytes,
>> and a trailer of 90,029 bytes (from APR's test/sendfile.c test). The
>> input to the *len param starts as 200,000 (the size of the file) and
>> after the call it comes back set as 290,029. If you're saying that
>> the
>> *len return result should be the total bytes sent (headers + file +
>> trailers) then the result should have been 370,049. Note that in this
>> example, the other end of the connection appears to have received at
>> least 119980 bytes.
>>
>>
>> Could we get a detailed explanation of what all the expected inputs
>> and outputs are for sendfile in Leopard() (and if it's different that
>> previous versions, the same for those would help too).
>>
>
> From what I could figure out, it works like this:
>
> o If length <= headers_length: Length output is headers length +
> trailers length. No bytes from the file are sent, headers and trailers
> are sent.
>
> o If length > headers_length, Length output is headers length +
> trailers
> length + file bytes sent. (length - headers length) bytes from the
> file
> are sent, headers and trailers are sent.
>
> o If length == 0, Length is headers_length + trailers_length + file
> size. Everything is sent.
>
> Strange...
>
> Regards,
>
> Davi Arnaut
Re: sendfile(2) misbehaves when header iovecs are specified
Posted by Davi Arnaut <da...@apache.org>.
Aaron Bannert wrote:
> Hmm, that's not the behavior I am seeing with Leopard's sendfile. I am
> seeing the value-result parameter come back with the total byes in the
> file plus the total size of the trailers, excluding the headers.
>
> For example, with a header of 80,000 bytes, a file of 200,000 bytes,
> and a trailer of 90,029 bytes (from APR's test/sendfile.c test). The
> input to the *len param starts as 200,000 (the size of the file) and
> after the call it comes back set as 290,029. If you're saying that the
> *len return result should be the total bytes sent (headers + file +
> trailers) then the result should have been 370,049. Note that in this
> example, the other end of the connection appears to have received at
> least 119980 bytes.
>
>
> Could we get a detailed explanation of what all the expected inputs
> and outputs are for sendfile in Leopard() (and if it's different that
> previous versions, the same for those would help too).
>
>From what I could figure out, it works like this:
o If length <= headers_length: Length output is headers length +
trailers length. No bytes from the file are sent, headers and trailers
are sent.
o If length > headers_length, Length output is headers length + trailers
length + file bytes sent. (length - headers length) bytes from the file
are sent, headers and trailers are sent.
o If length == 0, Length is headers_length + trailers_length + file
size. Everything is sent.
Strange...
Regards,
Davi Arnaut
Re: sendfile(2) misbehaves when header iovecs are specified
Posted by Aaron Bannert <aa...@codemass.com>.
Hmm, that's not the behavior I am seeing with Leopard's sendfile. I am
seeing the value-result parameter come back with the total byes in the
file plus the total size of the trailers, excluding the headers.
For example, with a header of 80,000 bytes, a file of 200,000 bytes,
and a trailer of 90,029 bytes (from APR's test/sendfile.c test). The
input to the *len param starts as 200,000 (the size of the file) and
after the call it comes back set as 290,029. If you're saying that the
*len return result should be the total bytes sent (headers + file +
trailers) then the result should have been 370,049. Note that in this
example, the other end of the connection appears to have received at
least 119980 bytes.
Could we get a detailed explanation of what all the expected inputs
and outputs are for sendfile in Leopard() (and if it's different that
previous versions, the same for those would help too).
-aaron
On Nov 13, 2007, at 1:06 PM, Wilfredo Sánchez Vega wrote:
> Davi-
>
> Regarding this issue:
>
> http://people.apache.org/~davi/leopard-sendfile.c
>
> Our kernel folks say this isn't a bug:
>
> No, this is not a bug. Our sendfile(2) implementation is
> similar to FreeBSD 4.x, where the 4th argument to the system
> call (the value-result length parameter) includes the header
> length. In FreeBSD 5.x, this length does not include the
> header, but that is not the semantics followed by Mac OS X.
>
> Is this sufficient information? I've still got the bug open until
> I hear from you.
>
> -wsv
>
Re: sendfile(2) misbehaves when header iovecs are specified
Posted by Davi Arnaut <da...@apache.org>.
Wilfredo Sánchez Vega wrote:
> Davi-
>
> Regarding this issue:
>
> http://people.apache.org/~davi/leopard-sendfile.c
>
> Our kernel folks say this isn't a bug:
>
> No, this is not a bug. Our sendfile(2) implementation is
> similar to FreeBSD 4.x, where the 4th argument to the system
> call (the value-result length parameter) includes the header
> length. In FreeBSD 5.x, this length does not include the
> header, but that is not the semantics followed by Mac OS X.
>
> Is this sufficient information? I've still got the bug open until
> I hear from you.
Yes, makes more sense now. I just think the man page should be corrected:
"The len argument is a value-result parameter, that specifies how many
bytes of the file should be sent and/or"
It should remove the part "bytes of the file".
Thanks.
Regards,
--
Davi Arnaut