You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Joe Orton <jo...@manyfish.co.uk> on 2004/03/20 12:04:26 UTC
RFC: APR_LARGEFILE flag for APR 0.9
APR 0.9.x can't change the size of apr_off_t, but it's still useful to
be able to use O_LARGEFILE in a few specific cases on LFS platforms when
the application knows it is safe: to allow writing >2Gb log files in
particular. It's not safe to use in general, since {f,l,}stat() will
fail on a >2gb file with a 32-bit off_t, so needs to come with a big
warning sign.
(apr_file_transfer_contents() could make use of O_LARGEFILE, but
annoyingly it won't work for >2Gb files in the common case where
to_perms == APR_FILE_SOURCE_PERMS since that needs to stat() the source
file)
Any comments?
--- include/apr_file_io.h 13 Feb 2004 09:33:44 -0000 1.144.2.1
+++ include/apr_file_io.h 20 Mar 2004 10:59:11 -0000
@@ -69,6 +69,20 @@
is opened */
#define APR_SENDFILE_ENABLED 0x01000 /**< Advisory flag that this file should
support apr_sendfile operation */
+#define APR_LARGEFILE 0x04000 /**< Platform dependent flag to enable large file
+ support; WARNING see below. */
+
+/** @warning The APR_LARGEFILE flag only has effect on some platforms
+ * where sizeof(apr_off_t) == 4. When implemented, it allows opening
+ * and writing to a file which exceeds the size which can be
+ * represented by apr_off_t (2 gigabytes). When a file's size does
+ * exceed 2Gb, apr_file_info_get() will fail with an error on the
+ * descriptor, likewise apr_stat()/apr_lstat() will fail on the
+ * filename. apr_dir_read() will fail with APR_INCOMPLETE on a
+ * directory entry for a large file depending on the particular
+ * APR_FINFO_* flags. Generally, it is not recommended to use this
+ * flag. */
+
/** @} */
/**
--- file_io/unix/open.c 13 Feb 2004 09:33:43 -0000 1.111.2.1
+++ file_io/unix/open.c 20 Mar 2004 10:59:11 -0000
@@ -98,6 +98,11 @@
oflags |= O_BINARY;
}
#endif
+#ifdef O_LARGEFILE
+ if (flag & APR_LARGEFILE) {
+ oflags |= O_LARGEFILE;
+ }
+#endif
#if APR_HAS_THREADS
if ((flag & APR_BUFFERED) && (flag & APR_XTHREAD)) {
Re: RFC: APR_LARGEFILE flag for APR 0.9
Posted by Joe Orton <jo...@manyfish.co.uk>.
On Sat, Mar 20, 2004 at 07:53:48AM -0500, Jeff Trawick wrote:
> Joe Orton wrote:
> >APR 0.9.x can't change the size of apr_off_t, but it's still useful to
> >be able to use O_LARGEFILE in a few specific cases on LFS platforms when
> >the application knows it is safe: to allow writing >2Gb log files in
> >particular. It's not safe to use in general, since {f,l,}stat() will
> >fail on a >2gb file with a 32-bit off_t, so needs to come with a big
> >warning sign.
>
> So for any file where the app will only open it or append to it, it can
> safely specify APR_LARGEFILE and the right thing will happen, with big
> "log" files where possible.
Yup, read()s from it will work too... even sendfile() might...
> [In the hope that this isn't really needed for 1.0]
> With APR 1.0, what is the reason for not having the same flag? Isn't there
> still the same situation where APR 1.0 will usually be built without large
> file support, and a random user can't rebuild APR with LFS without breaking
> existing apps, but may still want to support large "log" files?
I'm working on 1.0 changes to enable LFS support by defining apr_off_t
as off64_t and hence making O_LARGEFILE the default, so I was thinking
APR_LARGEFILE could be either undefined or a nullop in 1.0, either way
works.
The tricky things for 1.0 are when/if to enable apr_off_t==off64_t and
the ABI impact of that decision, and dealing with sendfile support.
> (Related trivia: Maybe README.dev needs notes on large file support,
> starting with the basic Unix requirement you outlined in a PR yesterday
>
> export CPPFLAGS="-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"
> ./configure
>
> and then having folks add comments on platforms where it has been tested
> and/or where there are special considerations.)
Good idea, thanks.
joe
Re: RFC: APR_LARGEFILE flag for APR 0.9
Posted by Joe Orton <jo...@manyfish.co.uk>.
On Thu, Mar 25, 2004 at 11:34:59PM -0600, William Rowe wrote:
> At 06:03 PM 3/25/2004, Joe Orton wrote:
> >On Thu, Mar 25, 2004 at 03:49:09PM -0800, Stas Bekman wrote:
> >>
> >> That's a bad idea, IMHO, because ARP_HAS_LARGE_FILES will still report
> >> false. You suggest to create a mess, where the library will report one
> >> thing but behave as another.
> >
> >I was thinking about this today as well... what do you expect
> >APR_HAS_LARGE_FILES to actually mean? "sizeof(apr_off_t) >
> >sizeof(apr_size_t)"? or "sizeof(apr_off_t) > off_t"? or
> >sizeof(apr_off_t)==8? Any of these are trivial to implement in the
> >configure script.
>
> .02 euro (worth more than US 2c) from the guy who invented the flag?
>
> APR_HAS_LARGE_FILES means that an apr_off_t is bigger than
> all addressable memory (size_t). You can't fit the apr_off_t into any
> [apr_][s]size_t placeholder.
Well, you may have invented it, but that's not what it does :)
APR_HAS_LARGE_FILES is always zero on Unix in 0.9.x currently, and yet
sizeof(apr_off_t) will be greater than sizeof(size_t) on the 32-bit
*BSDs with a 64-bit native off_t, similarly if built with
-D_FILE_OFFSET_BITS=64 on LFS platforms.
> Honestly I dont care if there is any relationship between óff_t and
> apr_off_t, otherwise why did we define our own type? It's (possibly)
> an implementation dependent detail but otherwise irrelevant.
>
> Bill
>
Re: RFC: APR_LARGEFILE flag for APR 0.9
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 06:03 PM 3/25/2004, Joe Orton wrote:
>On Thu, Mar 25, 2004 at 03:49:09PM -0800, Stas Bekman wrote:
>>
>> That's a bad idea, IMHO, because ARP_HAS_LARGE_FILES will still report
>> false. You suggest to create a mess, where the library will report one
>> thing but behave as another.
>
>I was thinking about this today as well... what do you expect
>APR_HAS_LARGE_FILES to actually mean? "sizeof(apr_off_t) >
>sizeof(apr_size_t)"? or "sizeof(apr_off_t) > off_t"? or
>sizeof(apr_off_t)==8? Any of these are trivial to implement in the
>configure script.
.02 euro (worth more than US 2c) from the guy who invented the flag?
APR_HAS_LARGE_FILES means that an apr_off_t is bigger than
all addressable memory (size_t). You can't fit the apr_off_t into any
[apr_][s]size_t placeholder.
Honestly I dont care if there is any relationship between óff_t and
apr_off_t, otherwise why did we define our own type? It's (possibly)
an implementation dependent detail but otherwise irrelevant.
Bill
Re: RFC: APR_LARGEFILE flag for APR 0.9
Posted by Stas Bekman <st...@stason.org>.
Joe Orton wrote:
[...]
>>I expect a true value of APR_HAS_LARGE_FILES to mean "-D_LARGEFILE_SOURCE
>>-D_FILE_OFFSET_BITS=64" on linux, and whatever are the equivalents are on
>>other platforms (are they different or the same?) and whatever that implies
>>for the type sizes.
>
>
> Hmmm. I think it comes down to this:
>
> 1. APR_HAS_LARGE_FILES is useless in APR 0.9.4 and earlier on Unix
> because it is never defined to 1 regardless of whether APR was built
> with _FILE_OFFSET_BITS=64.
>
> 2. APR_HAS_LARGE_FILES is not necessary in APR 0.9.5 and later on Unix
> because the size of apr_off_t as defined in apr.h does not change with
> _FILE_OFFSET_BITS.
>
> so I think you're right, the only useful definition for
> APR_HAS_LARGE_FILES is that it is 1 iff sizeof(apr_off_t) !=
> sizeof(native off_t). I'd quite like to get input from the SVN crowd
> about this too.
Do you suggest that off_t is the only type that changes when LFS comes in?
>>Since I have to glue perl and apr, I want to be able to say:
>>
>> seek file, pos, whence
>>
>>and non-zero pos and whence passed correctly from/to the C(Perl)/C(APR)
>>side. If we have a disagreement in type lengths things go randomly broken
>>and lots of hair gets lost.
>
>
> The problem which causes serious alopecia is the disagreement between
> the apr_off_t exposed by apr.h when mod_perl is built with
> -D_FILE_OFFSET_BITS=64, and the apr_off_t *when APR was built*.
>
> Since 0.9.5 works round this problem, it's now simple to cope with a
> sizeof(Off_t) != sizeof(apr_off_t) in mod_perl; you just need to do
> thunk between the two any place they are used together.
>
> if (sizeof(apr_off_t) == 4 && sizeof(Off_t) == 8)
> croak if offset > LONG_MAX
> etc
> }
That's actually a great idea, Joe! instead of relying on flags which may or
may be wrong, compare the actual data types. The only issue I have is that
apr_off_t is probably not the only data type that I need to check for possible
collisions. Is that right? If so, that's why it's so much easier to have one
define to do the check against and not all possible types, which may spring
into existance post certain release.
So far I understood that relying on APR_HAS_LARGE_FILES is a bad idea, at
least because you may be able to get more out of apr, but since the define may
be unset, you won't be able to. Did I get it right?
>>>>>>From below I take it you really want "sizeof(apr_off_t) > off_t"?
>>>
>>>That is consistent with what APR implements for Netware and Win32.
>>>
>>>It's inconsistent in that for e.g. FreeBSD or any 64-bit Unix, you get
>>>"large files" by default courtesy of a 64-bit off_t, yet
>>>APR_HAS_LARGE_FILES would be 0.
>>
>>I think that's fine. Since in that case USE_LARGE_FILES will be false, no?
>>and perl and apr will agree on the type lengths. Am I correct?
>
>
> If Perl only defines USE_LARGE_FILES when it defines
> _FILE_OFFSET_BITS=64, then yes.
Yes, it defines both: -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: RFC: APR_LARGEFILE flag for APR 0.9
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 05:28 AM 3/26/2004, Joe Orton wrote:
>On Thu, Mar 25, 2004 at 04:44:02PM -0800, Stas Bekman wrote:
>> I expect a true value of APR_HAS_LARGE_FILES to mean "-D_LARGEFILE_SOURCE
>> -D_FILE_OFFSET_BITS=64" on linux, and whatever are the equivalents are on
>> other platforms (are they different or the same?) and whatever that implies
>> for the type sizes.
>
>Hmmm. I think it comes down to this:
>
>1. APR_HAS_LARGE_FILES is useless in APR 0.9.4 and earlier on Unix
>because it is never defined to 1 regardless of whether APR was built
>with _FILE_OFFSET_BITS=64.
No argument there, but it is useful on platforms which did implement it.
>2. APR_HAS_LARGE_FILES is not necessary in APR 0.9.5 and later on Unix
>because the size of apr_off_t as defined in apr.h does not change with
>_FILE_OFFSET_BITS.
>
>so I think you're right, the only useful definition for
>APR_HAS_LARGE_FILES is that it is 1 iff sizeof(apr_off_t) !=
>sizeof(native off_t). I'd quite like to get input from the SVN crowd
>about this too.
Hmmm. The useful definition is "ya, this build of APR knows how to play
with files >2GB." This becomes especially critical in 1.0, iff we go with some
constant 64 bit file addressing across platforms. Though in that case, some
dynamic test would be far more useful than a static, since both become
binary compatible.
Bill
Re: RFC: APR_LARGEFILE flag for APR 0.9
Posted by Joe Orton <jo...@redhat.com>.
On Thu, Mar 25, 2004 at 04:44:02PM -0800, Stas Bekman wrote:
> Joe Orton wrote:
> >On Thu, Mar 25, 2004 at 03:49:09PM -0800, Stas Bekman wrote:
> >
> >>Jeff Trawick wrote:
> >>
> >>>(Related trivia: Maybe README.dev needs notes on large file support,
> >>>starting with the basic Unix requirement you outlined in a PR yesterday
> >>>
> >>>export CPPFLAGS="-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"
> >>>./configure
> >>>
> >>>and then having folks add comments on platforms where it has been tested
> >>>and/or where there are special considerations.)
> >>
> >>That's a bad idea, IMHO, because ARP_HAS_LARGE_FILES will still report
> >>false. You suggest to create a mess, where the library will report one
> >>thing but behave as another.
> >
> >
> >I was thinking about this today as well... what do you expect
> >APR_HAS_LARGE_FILES to actually mean? "sizeof(apr_off_t) >
> >sizeof(apr_size_t)"? or "sizeof(apr_off_t) > off_t"? or
> >sizeof(apr_off_t)==8? Any of these are trivial to implement in the
> >configure script.
>
> I expect a true value of APR_HAS_LARGE_FILES to mean "-D_LARGEFILE_SOURCE
> -D_FILE_OFFSET_BITS=64" on linux, and whatever are the equivalents are on
> other platforms (are they different or the same?) and whatever that implies
> for the type sizes.
Hmmm. I think it comes down to this:
1. APR_HAS_LARGE_FILES is useless in APR 0.9.4 and earlier on Unix
because it is never defined to 1 regardless of whether APR was built
with _FILE_OFFSET_BITS=64.
2. APR_HAS_LARGE_FILES is not necessary in APR 0.9.5 and later on Unix
because the size of apr_off_t as defined in apr.h does not change with
_FILE_OFFSET_BITS.
so I think you're right, the only useful definition for
APR_HAS_LARGE_FILES is that it is 1 iff sizeof(apr_off_t) !=
sizeof(native off_t). I'd quite like to get input from the SVN crowd
about this too.
> Since I have to glue perl and apr, I want to be able to say:
>
> seek file, pos, whence
>
> and non-zero pos and whence passed correctly from/to the C(Perl)/C(APR)
> side. If we have a disagreement in type lengths things go randomly broken
> and lots of hair gets lost.
The problem which causes serious alopecia is the disagreement between
the apr_off_t exposed by apr.h when mod_perl is built with
-D_FILE_OFFSET_BITS=64, and the apr_off_t *when APR was built*.
Since 0.9.5 works round this problem, it's now simple to cope with a
sizeof(Off_t) != sizeof(apr_off_t) in mod_perl; you just need to do
thunk between the two any place they are used together.
if (sizeof(apr_off_t) == 4 && sizeof(Off_t) == 8)
croak if offset > LONG_MAX
etc
}
> >>>From below I take it you really want "sizeof(apr_off_t) > off_t"?
> >That is consistent with what APR implements for Netware and Win32.
> >
> >It's inconsistent in that for e.g. FreeBSD or any 64-bit Unix, you get
> >"large files" by default courtesy of a 64-bit off_t, yet
> >APR_HAS_LARGE_FILES would be 0.
>
> I think that's fine. Since in that case USE_LARGE_FILES will be false, no?
> and perl and apr will agree on the type lengths. Am I correct?
If Perl only defines USE_LARGE_FILES when it defines
_FILE_OFFSET_BITS=64, then yes.
joe
Re: RFC: APR_LARGEFILE flag for APR 0.9
Posted by Stas Bekman <st...@stason.org>.
Joe Orton wrote:
> On Thu, Mar 25, 2004 at 03:49:09PM -0800, Stas Bekman wrote:
>
>>Jeff Trawick wrote:
>>
>>>(Related trivia: Maybe README.dev needs notes on large file support,
>>>starting with the basic Unix requirement you outlined in a PR yesterday
>>>
>>>export CPPFLAGS="-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"
>>>./configure
>>>
>>>and then having folks add comments on platforms where it has been tested
>>>and/or where there are special considerations.)
>>
>>That's a bad idea, IMHO, because ARP_HAS_LARGE_FILES will still report
>>false. You suggest to create a mess, where the library will report one
>>thing but behave as another.
>
>
> I was thinking about this today as well... what do you expect
> APR_HAS_LARGE_FILES to actually mean? "sizeof(apr_off_t) >
> sizeof(apr_size_t)"? or "sizeof(apr_off_t) > off_t"? or
> sizeof(apr_off_t)==8? Any of these are trivial to implement in the
> configure script.
I expect a true value of APR_HAS_LARGE_FILES to mean "-D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64" on linux, and whatever are the equivalents are on
other platforms (are they different or the same?) and whatever that implies
for the type sizes.
Since I have to glue perl and apr, I want to be able to say:
seek file, pos, whence
and non-zero pos and whence passed correctly from/to the C(Perl)/C(APR) side.
If we have a disagreement in type lengths things go randomly broken and lots
of hair gets lost.
>>>From below I take it you really want "sizeof(apr_off_t) > off_t"?
> That is consistent with what APR implements for Netware and Win32.
>
> It's inconsistent in that for e.g. FreeBSD or any 64-bit Unix, you get
> "large files" by default courtesy of a 64-bit off_t, yet
> APR_HAS_LARGE_FILES would be 0.
I think that's fine. Since in that case USE_LARGE_FILES will be false, no? and
perl and apr will agree on the type lengths. Am I correct?
>>/* both perl and apr have largefile support enabled */
>>#if defined(USE_LARGE_FILES) && APR_HAS_LARGE_FILES
>>#define MP_LARGE_FILES_ENABLED
>>#endif
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: RFC: APR_LARGEFILE flag for APR 0.9
Posted by Joe Orton <jo...@manyfish.co.uk>.
On Thu, Mar 25, 2004 at 03:49:09PM -0800, Stas Bekman wrote:
> Jeff Trawick wrote:
> >(Related trivia: Maybe README.dev needs notes on large file support,
> >starting with the basic Unix requirement you outlined in a PR yesterday
> >
> >export CPPFLAGS="-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"
> >./configure
> >
> >and then having folks add comments on platforms where it has been tested
> >and/or where there are special considerations.)
>
> That's a bad idea, IMHO, because ARP_HAS_LARGE_FILES will still report
> false. You suggest to create a mess, where the library will report one
> thing but behave as another.
I was thinking about this today as well... what do you expect
APR_HAS_LARGE_FILES to actually mean? "sizeof(apr_off_t) >
sizeof(apr_size_t)"? or "sizeof(apr_off_t) > off_t"? or
sizeof(apr_off_t)==8? Any of these are trivial to implement in the
configure script.
>From below I take it you really want "sizeof(apr_off_t) > off_t"?
That is consistent with what APR implements for Netware and Win32.
It's inconsistent in that for e.g. FreeBSD or any 64-bit Unix, you get
"large files" by default courtesy of a 64-bit off_t, yet
APR_HAS_LARGE_FILES would be 0.
> e.g. in mod_perl we use the following logic to decide whether mp can
> support large files or not.
>
> I'd rather see:
>
> export CPPFLAGS="-ARP_HAS_LARGE_FILES=1"
>
> or something like that
>
> /* both perl and apr have largefile support enabled */
> #if defined(USE_LARGE_FILES) && APR_HAS_LARGE_FILES
> #define MP_LARGE_FILES_ENABLED
> #endif
>
> /* both perl and apr have largefile support disabled */
> #if (!defined(USE_LARGE_FILES)) && !APR_HAS_LARGE_FILES
> #define MP_LARGE_FILES_DISABLED
> #endif
>
> /* perl largefile support is enabled, apr support is disabled */
> #if defined(USE_LARGE_FILES) && !APR_HAS_LARGE_FILES
> #define MP_LARGE_FILES_PERL_ONLY
> #endif
>
> /* apr largefile support is enabled, perl support is disabled */
> #if (!defined(USE_LARGE_FILES)) && APR_HAS_LARGE_FILES
> #define MP_LARGE_FILES_APR_ONLY
> #endif
>
> /* conflict due to not have either both perl and apr
> * support enabled or both disabled
> */
> #if defined(MP_LARGE_FILES_APR_ONLY) || defined(MP_LARGE_FILES_PERL_ONLY)
> #define MP_LARGE_FILES_CONFLICT
> #endif
>
> --
> __________________________________________________________________
> Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
> http://stason.org/ mod_perl Guide ---> http://perl.apache.org
> mailto:stas@stason.org http://use.perl.org http://apacheweek.com
> http://modperlbook.org http://apache.org http://ticketmaster.com
Re: RFC: APR_LARGEFILE flag for APR 0.9
Posted by Stas Bekman <st...@stason.org>.
Jeff Trawick wrote:
> Joe Orton wrote:
>
>> APR 0.9.x can't change the size of apr_off_t, but it's still useful to
>> be able to use O_LARGEFILE in a few specific cases on LFS platforms when
>> the application knows it is safe: to allow writing >2Gb log files in
>> particular. It's not safe to use in general, since {f,l,}stat() will
>> fail on a >2gb file with a 32-bit off_t, so needs to come with a big
>> warning sign.
>
>
> So for any file where the app will only open it or append to it, it can
> safely specify APR_LARGEFILE and the right thing will happen, with big
> "log" files where possible.
>
> +1
>
> [In the hope that this isn't really needed for 1.0]
> With APR 1.0, what is the reason for not having the same flag? Isn't
> there still the same situation where APR 1.0 will usually be built
> without large file support, and a random user can't rebuild APR with LFS
> without breaking existing apps, but may still want to support large
> "log" files?
>
> Having APR always use long file offsets when interacting with the app
> and for non-LFS builds doing the ugly conversion (and possibly
> generating the EFBIG) when interacting with the OS seems to be a way to
> make everything work without APR binary compatibility issues. I'm not
> suggesting that is the way to go, but unless there is some sort of
> solution it seems that we have an equivalent need for APR_LARGEFILE flag
> with APR 1.0.
>
> (Related trivia: Maybe README.dev needs notes on large file support,
> starting with the basic Unix requirement you outlined in a PR yesterday
>
> export CPPFLAGS="-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"
> ./configure
>
> and then having folks add comments on platforms where it has been tested
> and/or where there are special considerations.)
That's a bad idea, IMHO, because ARP_HAS_LARGE_FILES will still report false.
You suggest to create a mess, where the library will report one thing but
behave as another.
e.g. in mod_perl we use the following logic to decide whether mp can support
large files or not.
I'd rather see:
export CPPFLAGS="-ARP_HAS_LARGE_FILES=1"
or something like that
/* both perl and apr have largefile support enabled */
#if defined(USE_LARGE_FILES) && APR_HAS_LARGE_FILES
#define MP_LARGE_FILES_ENABLED
#endif
/* both perl and apr have largefile support disabled */
#if (!defined(USE_LARGE_FILES)) && !APR_HAS_LARGE_FILES
#define MP_LARGE_FILES_DISABLED
#endif
/* perl largefile support is enabled, apr support is disabled */
#if defined(USE_LARGE_FILES) && !APR_HAS_LARGE_FILES
#define MP_LARGE_FILES_PERL_ONLY
#endif
/* apr largefile support is enabled, perl support is disabled */
#if (!defined(USE_LARGE_FILES)) && APR_HAS_LARGE_FILES
#define MP_LARGE_FILES_APR_ONLY
#endif
/* conflict due to not have either both perl and apr
* support enabled or both disabled
*/
#if defined(MP_LARGE_FILES_APR_ONLY) || defined(MP_LARGE_FILES_PERL_ONLY)
#define MP_LARGE_FILES_CONFLICT
#endif
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: RFC: APR_LARGEFILE flag for APR 0.9
Posted by Jeff Trawick <tr...@attglobal.net>.
Joe Orton wrote:
> APR 0.9.x can't change the size of apr_off_t, but it's still useful to
> be able to use O_LARGEFILE in a few specific cases on LFS platforms when
> the application knows it is safe: to allow writing >2Gb log files in
> particular. It's not safe to use in general, since {f,l,}stat() will
> fail on a >2gb file with a 32-bit off_t, so needs to come with a big
> warning sign.
So for any file where the app will only open it or append to it, it can safely
specify APR_LARGEFILE and the right thing will happen, with big "log" files
where possible.
+1
[In the hope that this isn't really needed for 1.0]
With APR 1.0, what is the reason for not having the same flag? Isn't there
still the same situation where APR 1.0 will usually be built without large file
support, and a random user can't rebuild APR with LFS without breaking existing
apps, but may still want to support large "log" files?
Having APR always use long file offsets when interacting with the app and for
non-LFS builds doing the ugly conversion (and possibly generating the EFBIG)
when interacting with the OS seems to be a way to make everything work without
APR binary compatibility issues. I'm not suggesting that is the way to go, but
unless there is some sort of solution it seems that we have an equivalent need
for APR_LARGEFILE flag with APR 1.0.
(Related trivia: Maybe README.dev needs notes on large file support, starting
with the basic Unix requirement you outlined in a PR yesterday
export CPPFLAGS="-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"
./configure
and then having folks add comments on platforms where it has been tested and/or
where there are special considerations.)