You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Colm MacCarthaigh <co...@stdlib.net> on 2005/11/02 12:31:26 UTC

Re: [vote] 2.1.9 as beta

On Wed, Nov 02, 2005 at 11:21:18AM +0000, Joe Orton wrote:
> On Wed, Nov 02, 2005 at 10:44:02AM +0000, Colm MacCarthaigh wrote:
> > On Wed, Nov 02, 2005 at 10:33:50AM +0000, Joe Orton wrote:
> > > This sounds very confused.  On 64-bit platforms there are never any 
> > > magic CFLAGS needed, no sendfile64() needed, and should be no problems 
> > > handling >2Gb files in the first place.
> > 
> > Doesn't seem to be that way on IA64, my sys/sendfile.h has:
> 
> off_t is 64-bit on IA64, the LFS stuff is probably just there for -m32 
> builds or compatibility.  I just tested a 3Gb download with 2.1.9 on my 
> x86_64 box and it worked fine.  (as it should with 2.0.x also; nothing 
> really has changed on this front in 2.1.x for 64-bit builds)

I think you're right, and I think I've found the source of the problem:
apr's network_io/unix/sendrecv.c has:

    #if APR_HAS_LARGE_FILES && defined(HAVE_SENDFILE64)
        apr_off_t off = *offset;
    #define sendfile sendfile64
    
    #elif APR_HAS_LARGE_FILES && SIZEOF_OFF_T == 4
        /* 64-bit apr_off_t but no sendfile64(): fail if trying to send
         * past the 2Gb limit. */
        off_t off;
    
        if ((apr_int64_t)*offset + *len > INT_MAX) {
            return EINVAL;
        }
    
and it's the latter branch that gets trigged on IA64, because
APR_HAS_LARGE_FILES is defined but HAVE_SENDFILE64 isn't. The trouble is
that INT_MAX is;

	#  define INT_MAX       2147483647

so, apr_socket_sendfile is returning EINVAL when it shouldn't be.

-- 
Colm MacCárthaigh                        Public Key: colm+pgp@stdlib.net

Re: [vote] 2.1.9 as beta

Posted by Colm MacCarthaigh <co...@stdlib.net>.
On Wed, Nov 02, 2005 at 02:50:55PM +0000, Joe Orton wrote:
> Really I don't think it's right to change the code at all to try to
> cope with the Nth latest "sendfile is broken if..." issue.  Just
> "EnableSendfile off" as should be default.

Definitely not the preferred option in my case, don't know how many
people would be likewise affected though (I'm guessing a handful). But
fair enough, I'll add it to list of reasons to disable sendfile, and
just run the patch myself.

-- 
Colm MacCárthaigh                        Public Key: colm+pgp@stdlib.net

Re: [vote] 2.1.9 as beta

Posted by Joe Orton <jo...@redhat.com>.
On Wed, Nov 02, 2005 at 02:39:12PM +0000, Colm MacCarthaigh wrote:
> On Wed, Nov 02, 2005 at 01:34:56PM +0000, Joe Orton wrote:
> > Seems to work OK for me with RHEL4/IA64 (2.6.9-22.0.1.EL) with my normal 
> > sendfile test app over loopback.
> > 
> > open("6G.sparse", O_RDONLY)             = 3
> > fstat(3, {st_mode=S_IFREG|0664, st_size=6442450945, ...}) = 0
> > sendfile(1, 3, [0], 6442450945)         = 6442450945
> 
> Thanks, I'll try a new kernel/glibc at some point. In the meantime, I think
> doing;

Really I don't think it's right to change the code at all to try to cope 
with the Nth latest "sendfile is broken if..." issue.  Just 
"EnableSendfile off" as should be default.

joe

Re: [vote] 2.1.9 as beta

Posted by Colm MacCarthaigh <co...@stdlib.net>.
On Wed, Nov 02, 2005 at 01:34:56PM +0000, Joe Orton wrote:
> Seems to work OK for me with RHEL4/IA64 (2.6.9-22.0.1.EL) with my normal 
> sendfile test app over loopback.
> 
> open("6G.sparse", O_RDONLY)             = 3
> fstat(3, {st_mode=S_IFREG|0664, st_size=6442450945, ...}) = 0
> sendfile(1, 3, [0], 6442450945)         = 6442450945

Thanks, I'll try a new kernel/glibc at some point. In the meantime, I think
doing;

Index: core_filters.c
===================================================================
--- core_filters.c      (revision 330237)
+++ core_filters.c      (working copy)
@@ -561,9 +561,16 @@
                     (void)apr_socket_opt_set(s, APR_TCP_NOPUSH, 0);
                 }
                 if (rv != APR_SUCCESS) {
-                    return rv;
+                    /* There are cases in which a buggy sendfile can fail, but 
+                     * an ordinary write may succeed. Let the bucket pass through 
+                     * to the non-sendfile write. If it's really a problem with
+                     * the socket, we'll find out quickly. */
+                    did_sendfile = 0;
+                }
+                else {
+                    break;
                 }
-                break;
             }
         }
 #endif /* APR_HAS_SENDFILE */
 
is reasonable. 

Basically is sendfile() returns an error, rather than give up trying to write
the file, move on to using ordinary write/writev which may or may not work.  If
it's a real problem (ie a dead socket or something), we'll find out anyway, and
if it works we get one less annoyance for our users.

-- 
Colm MacCárthaigh                        Public Key: colm+pgp@stdlib.net

Re: [vote] 2.1.9 as beta

Posted by Joe Orton <jo...@redhat.com>.
On Wed, Nov 02, 2005 at 01:13:41PM +0000, Colm MacCarthaigh wrote:
> On Wed, Nov 02, 2005 at 11:49:15AM +0000, Colm MacCarthaigh wrote:
> > On Wed, Nov 02, 2005 at 11:38:17AM +0000, Joe Orton wrote:
> > > > and it's the latter branch that gets trigged on IA64
> > > 
> > > How have you managed to get SIZEOF_OFF_T == 4 as true on IA64?  
> > 
> > Hmmm, no, it's 8. As is size_t. I'm going back to scratch at looking at
> > what's up with gdb.
> 
> O.k., it seems sendfile() is buggy and really doesn't support files >2Gb
> on Linux on IA64, at least with my kernel (2.6.12.1). 
> 
>    sendfile(10, 11, [0], 4686706688)     = -1 EINVAL (Invalid argument)
> 
> Messing with defines sufficiently that APR uses sendfile64() is no help,
> and ends up back at the same system call anyway. 
>
> Anyone else got an IA64 Linux box they can confirm this on?

Seems to work OK for me with RHEL4/IA64 (2.6.9-22.0.1.EL) with my normal 
sendfile test app over loopback.

open("6G.sparse", O_RDONLY)             = 3
fstat(3, {st_mode=S_IFREG|0664, st_size=6442450945, ...}) = 0
sendfile(1, 3, [0], 6442450945)         = 6442450945

joe

Re: [vote] 2.1.9 as beta

Posted by Joe Orton <jo...@redhat.com>.
On Wed, Nov 02, 2005 at 01:13:41PM +0000, Colm MacCarthaigh wrote:
> On Wed, Nov 02, 2005 at 11:49:15AM +0000, Colm MacCarthaigh wrote:
> > On Wed, Nov 02, 2005 at 11:38:17AM +0000, Joe Orton wrote:
> > > > and it's the latter branch that gets trigged on IA64
> > > 
> > > How have you managed to get SIZEOF_OFF_T == 4 as true on IA64?  
> > 
> > Hmmm, no, it's 8. As is size_t. I'm going back to scratch at looking at
> > what's up with gdb.
> 
> O.k., it seems sendfile() is buggy and really doesn't support files >2Gb
> on Linux on IA64, at least with my kernel (2.6.12.1). 
> 
>    sendfile(10, 11, [0], 4686706688)     = -1 EINVAL (Invalid argument)
> 
> Messing with defines sufficiently that APR uses sendfile64() is no help,
> and ends up back at the same system call anyway. 
>
> Anyone else got an IA64 Linux box they can confirm this on?

Seems to work OK for me with RHEL4/IA64 (2.6.9-22.0.1.EL) with my normal 
sendfile test app over loopback.

open("6G.sparse", O_RDONLY)             = 3
fstat(3, {st_mode=S_IFREG|0664, st_size=6442450945, ...}) = 0
sendfile(1, 3, [0], 6442450945)         = 6442450945

joe

Re: [vote] 2.1.9 as beta

Posted by Colm MacCarthaigh <co...@stdlib.net>.
On Wed, Nov 02, 2005 at 11:49:15AM +0000, Colm MacCarthaigh wrote:
> On Wed, Nov 02, 2005 at 11:38:17AM +0000, Joe Orton wrote:
> > > and it's the latter branch that gets trigged on IA64
> > 
> > How have you managed to get SIZEOF_OFF_T == 4 as true on IA64?  
> 
> Hmmm, no, it's 8. As is size_t. I'm going back to scratch at looking at
> what's up with gdb.

O.k., it seems sendfile() is buggy and really doesn't support files >2Gb
on Linux on IA64, at least with my kernel (2.6.12.1). 

   sendfile(10, 11, [0], 4686706688)     = -1 EINVAL (Invalid argument)

Messing with defines sufficiently that APR uses sendfile64() is no help,
and ends up back at the same system call anyway. 

Anyone else got an IA64 Linux box they can confirm this on?

-- 
Colm MacCárthaigh                        Public Key: colm+pgp@stdlib.net

Re: [vote] 2.1.9 as beta

Posted by Colm MacCarthaigh <co...@stdlib.net>.
On Wed, Nov 02, 2005 at 11:49:15AM +0000, Colm MacCarthaigh wrote:
> On Wed, Nov 02, 2005 at 11:38:17AM +0000, Joe Orton wrote:
> > > and it's the latter branch that gets trigged on IA64
> > 
> > How have you managed to get SIZEOF_OFF_T == 4 as true on IA64?  
> 
> Hmmm, no, it's 8. As is size_t. I'm going back to scratch at looking at
> what's up with gdb.

O.k., it seems sendfile() is buggy and really doesn't support files >2Gb
on Linux on IA64, at least with my kernel (2.6.12.1). 

   sendfile(10, 11, [0], 4686706688)     = -1 EINVAL (Invalid argument)

Messing with defines sufficiently that APR uses sendfile64() is no help,
and ends up back at the same system call anyway. 

Anyone else got an IA64 Linux box they can confirm this on?

-- 
Colm MacCárthaigh                        Public Key: colm+pgp@stdlib.net

Re: [vote] 2.1.9 as beta

Posted by Colm MacCarthaigh <co...@stdlib.net>.
On Wed, Nov 02, 2005 at 11:38:17AM +0000, Joe Orton wrote:
> > and it's the latter branch that gets trigged on IA64
> 
> How have you managed to get SIZEOF_OFF_T == 4 as true on IA64?  

Hmmm, no, it's 8. As is size_t. I'm going back to scratch at looking at
what's up with gdb.

-- 
Colm MacCárthaigh                        Public Key: colm+pgp@stdlib.net

Re: [vote] 2.1.9 as beta

Posted by Colm MacCarthaigh <co...@stdlib.net>.
On Wed, Nov 02, 2005 at 11:38:17AM +0000, Joe Orton wrote:
> > and it's the latter branch that gets trigged on IA64
> 
> How have you managed to get SIZEOF_OFF_T == 4 as true on IA64?  

Hmmm, no, it's 8. As is size_t. I'm going back to scratch at looking at
what's up with gdb.

-- 
Colm MacCárthaigh                        Public Key: colm+pgp@stdlib.net

Re: [vote] 2.1.9 as beta

Posted by Joe Orton <jo...@redhat.com>.
On Wed, Nov 02, 2005 at 11:31:26AM +0000, Colm MacCarthaigh wrote:
> I think you're right, and I think I've found the source of the problem:
> apr's network_io/unix/sendrecv.c has:
> 
>     #if APR_HAS_LARGE_FILES && defined(HAVE_SENDFILE64)
>         apr_off_t off = *offset;
>     #define sendfile sendfile64
>     
>     #elif APR_HAS_LARGE_FILES && SIZEOF_OFF_T == 4
>         /* 64-bit apr_off_t but no sendfile64(): fail if trying to send
>          * past the 2Gb limit. */
...
> and it's the latter branch that gets trigged on IA64

How have you managed to get SIZEOF_OFF_T == 4 as true on IA64?  Can you 
upload the APR config.log somewhere?

joe

Re: [vote] 2.1.9 as beta

Posted by Joe Orton <jo...@redhat.com>.
On Wed, Nov 02, 2005 at 11:31:26AM +0000, Colm MacCarthaigh wrote:
> I think you're right, and I think I've found the source of the problem:
> apr's network_io/unix/sendrecv.c has:
> 
>     #if APR_HAS_LARGE_FILES && defined(HAVE_SENDFILE64)
>         apr_off_t off = *offset;
>     #define sendfile sendfile64
>     
>     #elif APR_HAS_LARGE_FILES && SIZEOF_OFF_T == 4
>         /* 64-bit apr_off_t but no sendfile64(): fail if trying to send
>          * past the 2Gb limit. */
...
> and it's the latter branch that gets trigged on IA64

How have you managed to get SIZEOF_OFF_T == 4 as true on IA64?  Can you 
upload the APR config.log somewhere?

joe