You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by "William A. Rowe, Jr." <wr...@rowe-clan.net> on 2007/10/30 00:11:13 UTC
Clean break for DYLD now?
Is OS/X 10.5 - Darwin 9 our opportunity to build the usual dl API?
Since 10.5 has never built APR because of an instant sendfile gotcha,
I'd like to roll apr finally, tomorrow night, start the 3 day clock
and release at the end of this week so httpd can pick it up.
But it seems like this is the one-and-only opportunity for us to fix
this particular implementation detail before Darwin 10 comes around.
We can simply force dl instead of dyld for darwin >= 9...
Thoughts?
Bill
Re: Clean break for DYLD now?
Posted by Aaron Bannert <aa...@codemass.com>.
On Nov 1, 2007, at 6:08 PM, William A. Rowe, Jr. wrote:
> William A. Rowe, Jr. wrote:
>>> We don't test sendfile by default. Within APR (trunk), simply...
>>>
>>> $ cd test
>>> $ make
>>> $ ./sendfile server &
>>> $ ./sendfile client blocking
>>>
>>> and you'll see how either the kernel library or the implementation
>>> (or maybe even the test) is broken.
>> Or (and I should have seen the possibility earlier) an interop flaw
>> based
>> on CPU or filesystem. So for the record, the failure I quoted
>> before about
>> 119880 bytes into the stream was on a journaled, hfs filesystem
>> running on
>> the G4, retail not server.
>
> Correction, faults at 119980 bytes into the stream, and I just
> reproduced the
> flaw on journaled, ufs.
I didn't understand what you meant before, but now I'm seeing the
problem.
It appears that apr_socket_sendfile() (and probably the underlying lib
or syscall) is updating the &len value-return parameter with the size
of the file PLUS the size of the trailers. I added some debugging
output to see what's going on and here's what I got on the client side:
Creating a test file...
Sending the file...
headers[0] size 15
headers[1] size 5
headers[2] size 80000
total headers length 80020
trailers[0] size 19
trailers[1] size 10
trailers[2] size 90000
total trailers length 90029
total iov length 170049
calling apr_socket_sendfile() with len 200000
apr_socket_sendfile() updated offset with 0
apr_socket_sendfile() updated len with 290029
bytes really sent: 370049
apr_socket_sendfile() didn't report the correct number of bytes sent!
As you can see, the returned len == 290029, which is the same as the
file size (200000) + the trailers size (90029). If the syscall is
returning this then it's a kernel bug since it's a change in the
Darwin sendfile semantics.
Now I'll see if I can get this fancy new dtrace stuff to report the
syscall results...
-aaron
Re: Clean break for DYLD now?
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
William A. Rowe, Jr. wrote:
>> We don't test sendfile by default. Within APR (trunk), simply...
>>
>> $ cd test
>> $ make
>> $ ./sendfile server &
>> $ ./sendfile client blocking
>>
>> and you'll see how either the kernel library or the implementation (or
>> maybe even the test) is broken.
>
> Or (and I should have seen the possibility earlier) an interop flaw based
> on CPU or filesystem. So for the record, the failure I quoted before about
> 119880 bytes into the stream was on a journaled, hfs filesystem running on
> the G4, retail not server.
Correction, faults at 119980 bytes into the stream, and I just reproduced the
flaw on journaled, ufs.
Re: Clean break for DYLD now?
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
William A. Rowe, Jr. wrote:
>>
>> And, APR trunk seems to work fine on 10.5 AFAICT - matching what Aaron
>> reported. So, I'm at a loss to even comment on the sendfile issue you
>> obliquely referred to. -- justin
>
> Did you try 1.2.x? Nope. It's fine because I added an implementation of
> sendfile, do this on 1.2.x and it breaks (sendfile() resolved, no impl.)
>
> We don't test sendfile by default. Within APR (trunk), simply...
>
> $ cd test
> $ make
> $ ./sendfile server &
> $ ./sendfile client blocking
>
> and you'll see how either the kernel library or the implementation (or
> maybe even the test) is broken.
Or (and I should have seen the possibility earlier) an interop flaw based
on CPU or filesystem. So for the record, the failure I quoted before about
119880 bytes into the stream was on a journaled, hfs filesystem running on
the G4, retail not server.
Bill
Re: Clean break for DYLD now?
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Justin Erenkrantz wrote:
> On Nov 1, 2007 3:55 PM, William A. Rowe, Jr. <wr...@rowe-clan.net> wrote:
>> FWIW - this is how apple built to 10.5's /usr/lib ...
>
> Huh?
>
> % uname -mrsv
> Darwin 9.0.0 Darwin Kernel Version 9.0.0: Tue Oct 9 21:37:58 PDT
> 2007; root:xnu-1228~1/RELEASE_PPC Power Macintosh
> % ls /usr/lib/*.dylib | wc -l
> 314
nm /usr/lib/libapr-1.dyld | grep dl
U _dlclose
U _dlerror
U _dlopen
U _dlsym
...
similarly look for dyld (NS) and none of those api's are used.
But by default we will compile to the dyld API (and see a bunch of
warnings that the API is deprecated). There's definitely a difference.
> I have *no* idea what this thread is even about. Can you please give
> us some context or something?
>
> And, APR trunk seems to work fine on 10.5 AFAICT - matching what Aaron
> reported. So, I'm at a loss to even comment on the sendfile issue you
> obliquely referred to. -- justin
Did you try 1.2.x? Nope. It's fine because I added an implementation of
sendfile, do this on 1.2.x and it breaks (sendfile() resolved, no impl.)
We don't test sendfile by default. Within APR (trunk), simply...
$ cd test
$ make
$ ./sendfile server &
$ ./sendfile client blocking
and you'll see how either the kernel library or the implementation (or maybe
even the test) is broken.
Bill
Re: Clean break for DYLD now?
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
On Nov 1, 2007 3:55 PM, William A. Rowe, Jr. <wr...@rowe-clan.net> wrote:
> FWIW - this is how apple built to 10.5's /usr/lib ...
Huh?
% uname -mrsv
Darwin 9.0.0 Darwin Kernel Version 9.0.0: Tue Oct 9 21:37:58 PDT
2007; root:xnu-1228~1/RELEASE_PPC Power Macintosh
% ls /usr/lib/*.dylib | wc -l
314
I have *no* idea what this thread is even about. Can you please give
us some context or something?
And, APR trunk seems to work fine on 10.5 AFAICT - matching what Aaron
reported. So, I'm at a loss to even comment on the sendfile issue you
obliquely referred to. -- justin
Re: Clean break for DYLD now?
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
FWIW - this is how apple built to 10.5's /usr/lib ...
... not that it's apropos of anything, we could care less if they don't
bother to submit patches.
;-)
Bill
William A. Rowe, Jr. wrote:
> I'll be rolling without a change (which means our next chance to
> touch this for apr 1.x will be Darwin 10)...
>
> ...unless you folk speak up quick.
>
> Bill
>
> William A. Rowe, Jr. wrote:
>> Is OS/X 10.5 - Darwin 9 our opportunity to build the usual dl API?
>>
>> Since 10.5 has never built APR because of an instant sendfile gotcha,
>> I'd like to roll apr finally, tomorrow night, start the 3 day clock
>> and release at the end of this week so httpd can pick it up.
>>
>> But it seems like this is the one-and-only opportunity for us to fix
>> this particular implementation detail before Darwin 10 comes around.
>> We can simply force dl instead of dyld for darwin >= 9...
>>
>> Thoughts?
>>
>> Bill
>>
>>
>
>
>
Re: Clean break for DYLD now?
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
I'll be rolling without a change (which means our next chance to
touch this for apr 1.x will be Darwin 10)...
...unless you folk speak up quick.
Bill
William A. Rowe, Jr. wrote:
> Is OS/X 10.5 - Darwin 9 our opportunity to build the usual dl API?
>
> Since 10.5 has never built APR because of an instant sendfile gotcha,
> I'd like to roll apr finally, tomorrow night, start the 3 day clock
> and release at the end of this week so httpd can pick it up.
>
> But it seems like this is the one-and-only opportunity for us to fix
> this particular implementation detail before Darwin 10 comes around.
> We can simply force dl instead of dyld for darwin >= 9...
>
> Thoughts?
>
> Bill
>
>