[PATCH]: lengths - brigades v.s. buckets.

["William A. Rowe, Jr." <wr...@rowe-clan.net>]

The upshot of this patch:  Brigades deal with apr_off_t bytes, but a single bucket never
deals with more than apr_size_t bytes :)


From: "Roy T. Fielding" <fi...@ebuilt.com>
Sent: Thursday, July 12, 2001 11:54 PM

> Bill wrote some time ago;
>
> > This means a huge file would need to be split by the caller into multiple file
> > buckets, no longer than ssize_t [actually, apr_size_t].  Is this reasonable?
> 
> Wouldn't that make it difficult to call sendfile on a file bucket that
> points to a huge file?

Good question, let's see, from FreeBSD...

int sendfile(int fd, int s, off_t offset, size_t nbytes,
             struct sf_hdtr *hdtr, off_t *sbytes, int flags)

the offset is an off_t (that's healthy), but the bytes to send is a size_t.  Hmmm,
so we can't request more bytes, unless we leave it at 0.  We are told that we sent
up to off_t, a healthy big number.

It returns an off_t for bytes sent.  Interesting mishmash.

>From Linux glibc2...

int sendfile(int out_fd, int in_fd, off_t *offset, size_t count)

Ok, same story here.  Rather broken since we are only told of sending int bytes,
which is most definately a discrepancy (between 32 and 64 bit builds as well.)  
Nothing in the man page about using count==0 to send the remainder of the file.

I'd argue that sendfile against a huge file bucket is inherantly non-portable.  Lo and
behold, we defined apr_sendfile in terms of a size_t.

At the same time, apr_brigade_consume and apr_brigade_length need to be playing with apr_off_t,
since the aggregated size of all buckets (file+memory) can certainly exceed the memory space!!!  

Do we push these discrepancies on the apr implementor, or the user?  In today's patch,
I propose I push this back at the user.  I'd move the definition of 'size indeterminate' to
start of -1 (from the header, "If length == -1, start == -1") and length of (apr_size_t)(-1)
so we can have a full apr_size_t of data in a bucket.  Passing 0xffffffff for a read isn't
necessarily bad, since the actual length read should be returned.

If we want to pull these off the user, then we need to first modify apr_sendfile(),
and this patch can be used as the basis for changing the entire behavior.

Bill