You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Eric Covener <co...@gmail.com> on 2013/12/02 01:29:37 UTC
buffered files read after apr_file_close()'ed
I am looking at a httpd bug that causes a hang on windows but succeeds on unix.
It seems that (short) files are opened w/ buffering, read,
apr_file_closed, and read again [succesfully on unix]
On Unix, they sare satisfied out of the buffer. file->fileset is -1.
On Windows, the destroyed apr_thread_mutex causes a hang.
Is reading from the closed file on the extreme bogus end of the
spectrum as I suspect and just getting lucky on the unix case?
Should they blow up in 2.0 during a read if they've been closed?
--
Eric Covener
covener@gmail.com
Re: buffered files read after apr_file_close()'ed
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On Tue, 03 Dec 2013 15:55:25 +0100
Stefan Ruppert <sr...@myarm.com> wrote:
> Am 03.12.2013 15:15, schrieb Daniel Lescohier:
>
> > However, the functions still need to change to return errors for the
> > cases when they are unbuffered files. The buffer won't be
> > referenced (and cause a crash) in those cases.
>
> But in this case the OS will return an EBADF if read() or write() is
> called.
Correct, provided we've set fd = -1 (in case a new file has been opened
with the prior fd assignment). On windows, INVALID_HANDLE value would
accomplish the same thing, while NULL'ing out the mutex will avoid the
reported issue, and we can simply toggle a buffered file to false to
cause i/o to occur (and fail).
This adds three assignment ops to the close operation, and no cpu ops
to all the common read/write/test cases, which is the best all-around
solution IMHO.
The reason to solve, following Brane's initial comments, is that as our
devs live on Windows, or Linux, or OS/X or wherever, that things should
try and fail in a somewhat consistent manner, to let those persistent
devs track down their bugs.
Re: buffered files read after apr_file_close()'ed
Posted by Stefan Ruppert <sr...@myarm.com>.
Am 03.12.2013 15:15, schrieb Daniel Lescohier:
> A null pointer will cause a segmentation fault. Is it guaranteed that
> the memory page that the pointer 0xdeadbeef points to is one that is not
> marked readable & writable, so that one would get a segmentation fault
> on reading/writing to it?
A null pointer will not cause a segmantation fault on all systems. For
example AIX will silently ignore a dereferencing of a null pointer
resulting in unexpected behaviour. But your question about 0xdeadbeef is
still valid. Its a common practice in memory debugging libraries to mark
free'd memory with a pattern which is unlikely a valid pointer. However
it depends on the OS and its memory management if 0xdeadbeef is invalid
or not. But from my knowledge most OSes uses only the lower half of the
address space for user processes.
> However, the functions still need to change to return errors for the
> cases when they are unbuffered files. The buffer won't be referenced
> (and cause a crash) in those cases.
But in this case the OS will return an EBADF if read() or write() is called.
> I think the read/write functions should also do *nbytes=0 when filedes<0
> so caller's loops will stop looping.
>
> In addition, apr_file_eof should return an error APR_EBADF:
Some sanity checks can help but does not have to...
Stefan
Re: buffered files read after apr_file_close()'ed
Posted by Jeff Trawick <tr...@gmail.com>.
On Tue, Dec 3, 2013 at 9:15 AM, Daniel Lescohier
<da...@cbsi.com>wrote:
> A null pointer will cause a segmentation fault. Is it guaranteed that the
> memory page that the pointer 0xdeadbeef points to is one that is not marked
> readable & writable, so that one would get a segmentation fault on
> reading/writing to it?
>
> However, the functions still need to change to return errors for the cases
> when they are unbuffered files. The buffer won't be referenced (and cause
> a crash) in those cases.
>
> I think the read/write functions should also do *nbytes=0 when filedes<0
> so caller's loops will stop looping.
>
For unbuffered files they'll generally get rv == EBADF for free, right?
And apr_file_read() for Unix seems to set *nbytes = 0 if an error occurs??
>
> In addition, apr_file_eof should return an error APR_EBADF:
>
> APR_DECLARE(apr_status_t) apr_file_eof(apr_file_t *fptr)
> {
> + if (fptr->filedes < 0) {
> + return APR_EBADF;
> + }
> if (fptr->eof_hit == 1) {
> return APR_EOF;
> }
> return APR_SUCCESS;
> }
>
>
(shrug)
* Cheap mechanisms in apr_FOO_close to help catch application bugs later
(e.g., invalidating pointers or file descriptors): awesome
* Adding sanity checking like this for application bugs: never-ending work
* Crashes due to application bugs: fine with me
* Negative feedback due to application bugs as long as we don't have to
check for the condition: awesome
* Hangs or incorrect/invalid feedback due to application bugs when
otherwise APR would require explicit code to check for it: fine with me
But then this particular mod_cache_disk bug doesn't seem so interesting to
me personally :) (My own Mileage May Vary)
Maybe the extra sanity check is not in an API called over and over, so it
doesn't clutter the main path. (not so bad)
>
> On Tue, Dec 3, 2013 at 8:41 AM, Stefan Ruppert <sr...@myarm.com> wrote:
>
>> Am 03.12.2013 14:18, schrieb Jeff Trawick:
>>
>>> On Tue, Dec 3, 2013 at 3:16 AM, Stefan Ruppert <sr@myarm.com
>>> <ma...@myarm.com>> wrote:
>>>
>>> Am 03.12.2013 00:37, schrieb William A. Rowe Jr.:
>>>
>>> On Mon, 02 Dec 2013 01:34:58 +0100
>>> Branko Čibej <brane@apache.org <ma...@apache.org>> wrote:
>>>
>>> On 02.12.2013 01:29, Eric Covener wrote:
>>>
>>> I am looking at a httpd bug that causes a hang on
>>> windows but
>>> succeeds on unix.
>>>
>>> It seems that (short) files are opened w/ buffering,
>>> read,
>>> apr_file_closed, and read again [succesfully on unix]
>>>
>>> On Unix, they sare satisfied out of the buffer.
>>> file->fileset is
>>> -1. On Windows, the destroyed apr_thread_mutex causes a
>>> hang.
>>>
>>> Is reading from the closed file on the extreme bogus end
>>> of the
>>> spectrum as I suspect and just getting lucky on the unix
>>> case?
>>>
>>>
>>> I'd certainly call a successful read from a closed file a
>>> bug.
>>>
>>> Should they blow up in 2.0 during a read if they've been
>>> closed?
>>>
>>>
>>> Dunno ... my gut feeling in cases like this is to just leave
>>> it be.
>>> Developers should have some fun finding heisenbugs, too. :)
>>>
>>>
>>> If we fix this, it needs to be minimal impact. Zero'ing out the
>>> buffer on close seems like the lowest cpu impact.
>>>
>>>
>>> A minimal impact and IMHO the correct fix is to return an error in
>>> buffered I/O if the file was closed.
>>>
>>> An application should not call a read or write function on a closed
>>> file. If it does its a bug and it should be notified about that fact.
>>>
>>>
>>> I keep telling myself over the years that APR's propensity to crash when
>>> called in situations as wrong as this (quite different from proprietary
>>> libraries with other considerations) is actually very good for code
>>> quality in the long run.
>>>
>>> What's the least we can do for this case that avoids having to check
>>> good calls for validity and yet give unmistakable feedback? Clear the
>>> buffer pointer during close?
>>>
>>
>> You are absolutly right, after a close the file structure isn't valid
>> anymore thus checking the file descriptor is a shot in the dark...
>>
>> Maybe not clearing the buffer pointer but assigning 0xdeadbeef to force a
>> crash?
>>
>> Regards,
>> Stefan
>>
>>
>
--
Born in Roswell... married an alien...
http://emptyhammock.com/
Re: buffered files read after apr_file_close()'ed
Posted by Daniel Lescohier <da...@cbsi.com>.
A null pointer will cause a segmentation fault. Is it guaranteed that the
memory page that the pointer 0xdeadbeef points to is one that is not marked
readable & writable, so that one would get a segmentation fault on
reading/writing to it?
However, the functions still need to change to return errors for the cases
when they are unbuffered files. The buffer won't be referenced (and cause
a crash) in those cases.
I think the read/write functions should also do *nbytes=0 when filedes<0 so
caller's loops will stop looping.
In addition, apr_file_eof should return an error APR_EBADF:
APR_DECLARE(apr_status_t) apr_file_eof(apr_file_t *fptr)
{
+ if (fptr->filedes < 0) {
+ return APR_EBADF;
+ }
if (fptr->eof_hit == 1) {
return APR_EOF;
}
return APR_SUCCESS;
}
On Tue, Dec 3, 2013 at 8:41 AM, Stefan Ruppert <sr...@myarm.com> wrote:
> Am 03.12.2013 14:18, schrieb Jeff Trawick:
>
>> On Tue, Dec 3, 2013 at 3:16 AM, Stefan Ruppert <sr@myarm.com
>> <ma...@myarm.com>> wrote:
>>
>> Am 03.12.2013 00:37, schrieb William A. Rowe Jr.:
>>
>> On Mon, 02 Dec 2013 01:34:58 +0100
>> Branko Čibej <brane@apache.org <ma...@apache.org>> wrote:
>>
>> On 02.12.2013 01:29, Eric Covener wrote:
>>
>> I am looking at a httpd bug that causes a hang on
>> windows but
>> succeeds on unix.
>>
>> It seems that (short) files are opened w/ buffering, read,
>> apr_file_closed, and read again [succesfully on unix]
>>
>> On Unix, they sare satisfied out of the buffer.
>> file->fileset is
>> -1. On Windows, the destroyed apr_thread_mutex causes a
>> hang.
>>
>> Is reading from the closed file on the extreme bogus end
>> of the
>> spectrum as I suspect and just getting lucky on the unix
>> case?
>>
>>
>> I'd certainly call a successful read from a closed file a bug.
>>
>> Should they blow up in 2.0 during a read if they've been
>> closed?
>>
>>
>> Dunno ... my gut feeling in cases like this is to just leave
>> it be.
>> Developers should have some fun finding heisenbugs, too. :)
>>
>>
>> If we fix this, it needs to be minimal impact. Zero'ing out the
>> buffer on close seems like the lowest cpu impact.
>>
>>
>> A minimal impact and IMHO the correct fix is to return an error in
>> buffered I/O if the file was closed.
>>
>> An application should not call a read or write function on a closed
>> file. If it does its a bug and it should be notified about that fact.
>>
>>
>> I keep telling myself over the years that APR's propensity to crash when
>> called in situations as wrong as this (quite different from proprietary
>> libraries with other considerations) is actually very good for code
>> quality in the long run.
>>
>> What's the least we can do for this case that avoids having to check
>> good calls for validity and yet give unmistakable feedback? Clear the
>> buffer pointer during close?
>>
>
> You are absolutly right, after a close the file structure isn't valid
> anymore thus checking the file descriptor is a shot in the dark...
>
> Maybe not clearing the buffer pointer but assigning 0xdeadbeef to force a
> crash?
>
> Regards,
> Stefan
>
>
Re: buffered files read after apr_file_close()'ed
Posted by Stefan Ruppert <sr...@myarm.com>.
Am 03.12.2013 14:18, schrieb Jeff Trawick:
> On Tue, Dec 3, 2013 at 3:16 AM, Stefan Ruppert <sr@myarm.com
> <ma...@myarm.com>> wrote:
>
> Am 03.12.2013 00:37, schrieb William A. Rowe Jr.:
>
> On Mon, 02 Dec 2013 01:34:58 +0100
> Branko Čibej <brane@apache.org <ma...@apache.org>> wrote:
>
> On 02.12.2013 01:29, Eric Covener wrote:
>
> I am looking at a httpd bug that causes a hang on
> windows but
> succeeds on unix.
>
> It seems that (short) files are opened w/ buffering, read,
> apr_file_closed, and read again [succesfully on unix]
>
> On Unix, they sare satisfied out of the buffer.
> file->fileset is
> -1. On Windows, the destroyed apr_thread_mutex causes a
> hang.
>
> Is reading from the closed file on the extreme bogus end
> of the
> spectrum as I suspect and just getting lucky on the unix
> case?
>
>
> I'd certainly call a successful read from a closed file a bug.
>
> Should they blow up in 2.0 during a read if they've been
> closed?
>
>
> Dunno ... my gut feeling in cases like this is to just leave
> it be.
> Developers should have some fun finding heisenbugs, too. :)
>
>
> If we fix this, it needs to be minimal impact. Zero'ing out the
> buffer on close seems like the lowest cpu impact.
>
>
> A minimal impact and IMHO the correct fix is to return an error in
> buffered I/O if the file was closed.
>
> An application should not call a read or write function on a closed
> file. If it does its a bug and it should be notified about that fact.
>
>
> I keep telling myself over the years that APR's propensity to crash when
> called in situations as wrong as this (quite different from proprietary
> libraries with other considerations) is actually very good for code
> quality in the long run.
>
> What's the least we can do for this case that avoids having to check
> good calls for validity and yet give unmistakable feedback? Clear the
> buffer pointer during close?
You are absolutly right, after a close the file structure isn't valid
anymore thus checking the file descriptor is a shot in the dark...
Maybe not clearing the buffer pointer but assigning 0xdeadbeef to force
a crash?
Regards,
Stefan
Re: buffered files read after apr_file_close()'ed
Posted by Eric Covener <co...@gmail.com>.
> What's the least we can do for this case that avoids having to check good
> calls for validity and yet give unmistakable feedback? Clear the buffer
> pointer during close?
My first thought was the same as Steffan's. To short-circuit Windows,
where it hangs due to the destroyed mutex, I think we'd need a 2nd
tweak beyond clearing the buffer:
if (thefile->buffered) {
char *pos = (char *)buf;
apr_size_t blocksize;
apr_size_t size = *len;
apr_thread_mutex_lock(thefile->mutex);
fd08ddfc 6eec5ca5 libapr_1!apr_thread_mutex_lock+0x29
[C:\cygwin\home\covener\SRC\2.4.x\srclib\apr\locks\win32\thread_mutex.c
@ 89]
fd08de20 6eec64ec libapr_1!apr_file_read+0xe5
[C:\cygwin\home\covener\SRC\2.4.x\srclib\apr\file_io\win32\readwrite.c
@ 186]
fd08de40 6fbd1f7c libapr_1!apr_file_gets+0x2c
[C:\cygwin\home\covener\SRC\2.4.x\srclib\apr\file_io\win32\readwrite.c
@ 455]
fd08fe60 6fbd1eee mod_cache_disk!read_table+0x2c
[C:\cygwin\home\covener\SRC\2.4.x\modules\cache\mod_cache_disk.c @
790]
fd08fe84 6fbe13bf mod_cache_disk!recall_headers+0x7e
[C:\cygwin\home\covener\SRC\2.4.x\modules\cache\mod_cache_disk.c @
884]
fd08feac 6fbe5311 mod_cache!cache_select+0xcf
[C:\cygwin\home\covener\SRC\2.4.x\modules\cache\cache_storage.c @ 254]
fd08fecc 6ff027b9 mod_cache!cache_quick_handler+0x171
[C:\cygwin\home\covener\SRC\2.4.x\modules\cache\mod_cache.c @ 163]
fd08fee8 6ff11f76 libhttpd!ap_run_quick_handler+0x29
[C:\cygwin\home\covener\SRC\2.4.x\server\config.c @ 172]
(line numbers probably don't match any revision exactly, but
read_table is always reading from a closed apr_file_t)
Re: buffered files read after apr_file_close()'ed
Posted by Branko Čibej <br...@apache.org>.
On 03.12.2013 14:18, Jeff Trawick wrote:
>
> I keep telling myself over the years that APR's propensity to crash
> when called in situations as wrong as this (quite different from
> proprietary libraries with other considerations) is actually very good
> for code quality in the long run.
Eh. I was in that camp until I started seeing tons of complaints from
users about Subversion crashing the Windows shell because of this kind
of thinking.
It's OK-ish to crash an HTTPd worker process. It's not at all OK to
crash a long-lived UI application. APR is used in both situations.
-- Brane
Re: buffered files read after apr_file_close()'ed
Posted by Jeff Trawick <tr...@gmail.com>.
On Tue, Dec 3, 2013 at 3:16 AM, Stefan Ruppert <sr...@myarm.com> wrote:
> Am 03.12.2013 00:37, schrieb William A. Rowe Jr.:
>
> On Mon, 02 Dec 2013 01:34:58 +0100
>> Branko Čibej <br...@apache.org> wrote:
>>
>> On 02.12.2013 01:29, Eric Covener wrote:
>>>
>>>> I am looking at a httpd bug that causes a hang on windows but
>>>> succeeds on unix.
>>>>
>>>> It seems that (short) files are opened w/ buffering, read,
>>>> apr_file_closed, and read again [succesfully on unix]
>>>>
>>>> On Unix, they sare satisfied out of the buffer. file->fileset is
>>>> -1. On Windows, the destroyed apr_thread_mutex causes a hang.
>>>>
>>>> Is reading from the closed file on the extreme bogus end of the
>>>> spectrum as I suspect and just getting lucky on the unix case?
>>>>
>>>
>>> I'd certainly call a successful read from a closed file a bug.
>>>
>>> Should they blow up in 2.0 during a read if they've been closed?
>>>>
>>>
>>> Dunno ... my gut feeling in cases like this is to just leave it be.
>>> Developers should have some fun finding heisenbugs, too. :)
>>>
>>
>> If we fix this, it needs to be minimal impact. Zero'ing out the
>> buffer on close seems like the lowest cpu impact.
>>
>>
> A minimal impact and IMHO the correct fix is to return an error in
> buffered I/O if the file was closed.
>
> An application should not call a read or write function on a closed file.
> If it does its a bug and it should be notified about that fact.
>
I keep telling myself over the years that APR's propensity to crash when
called in situations as wrong as this (quite different from proprietary
libraries with other considerations) is actually very good for code quality
in the long run.
What's the least we can do for this case that avoids having to check good
calls for validity and yet give unmistakable feedback? Clear the buffer
pointer during close?
> Thus I would propose the following fix in apr_file_read() and any other
> read/write function which uses buffered I/O:
>
> APR_DECLARE(apr_status_t) apr_file_read(apr_file_t *thefile, void *buf,
> apr_size_t *nbytes)
> {
> apr_ssize_t rv;
> apr_size_t bytes_read;
>
> + if (thefile->filedes < 0) {
> + return APR_EBADF;
> + }
>
> if (*nbytes <= 0) {
> *nbytes = 0;
> return APR_SUCCESS;
> }
>
> Regards,
> Stefan
>
>
--
Born in Roswell... married an alien...
http://emptyhammock.com/
Re: buffered files read after apr_file_close()'ed
Posted by Stefan Ruppert <sr...@myarm.com>.
Am 03.12.2013 00:37, schrieb William A. Rowe Jr.:
> On Mon, 02 Dec 2013 01:34:58 +0100
> Branko Čibej <br...@apache.org> wrote:
>
>> On 02.12.2013 01:29, Eric Covener wrote:
>>> I am looking at a httpd bug that causes a hang on windows but
>>> succeeds on unix.
>>>
>>> It seems that (short) files are opened w/ buffering, read,
>>> apr_file_closed, and read again [succesfully on unix]
>>>
>>> On Unix, they sare satisfied out of the buffer. file->fileset is
>>> -1. On Windows, the destroyed apr_thread_mutex causes a hang.
>>>
>>> Is reading from the closed file on the extreme bogus end of the
>>> spectrum as I suspect and just getting lucky on the unix case?
>>
>> I'd certainly call a successful read from a closed file a bug.
>>
>>> Should they blow up in 2.0 during a read if they've been closed?
>>
>> Dunno ... my gut feeling in cases like this is to just leave it be.
>> Developers should have some fun finding heisenbugs, too. :)
>
> If we fix this, it needs to be minimal impact. Zero'ing out the
> buffer on close seems like the lowest cpu impact.
>
A minimal impact and IMHO the correct fix is to return an error in
buffered I/O if the file was closed.
An application should not call a read or write function on a closed
file. If it does its a bug and it should be notified about that fact.
Thus I would propose the following fix in apr_file_read() and any other
read/write function which uses buffered I/O:
APR_DECLARE(apr_status_t) apr_file_read(apr_file_t *thefile, void *buf,
apr_size_t *nbytes)
{
apr_ssize_t rv;
apr_size_t bytes_read;
+ if (thefile->filedes < 0) {
+ return APR_EBADF;
+ }
if (*nbytes <= 0) {
*nbytes = 0;
return APR_SUCCESS;
}
Regards,
Stefan
Re: buffered files read after apr_file_close()'ed
Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On Mon, 02 Dec 2013 01:34:58 +0100
Branko Čibej <br...@apache.org> wrote:
> On 02.12.2013 01:29, Eric Covener wrote:
> > I am looking at a httpd bug that causes a hang on windows but
> > succeeds on unix.
> >
> > It seems that (short) files are opened w/ buffering, read,
> > apr_file_closed, and read again [succesfully on unix]
> >
> > On Unix, they sare satisfied out of the buffer. file->fileset is
> > -1. On Windows, the destroyed apr_thread_mutex causes a hang.
> >
> > Is reading from the closed file on the extreme bogus end of the
> > spectrum as I suspect and just getting lucky on the unix case?
>
> I'd certainly call a successful read from a closed file a bug.
>
> > Should they blow up in 2.0 during a read if they've been closed?
>
> Dunno ... my gut feeling in cases like this is to just leave it be.
> Developers should have some fun finding heisenbugs, too. :)
If we fix this, it needs to be minimal impact. Zero'ing out the
buffer on close seems like the lowest cpu impact.
Re: buffered files read after apr_file_close()'ed
Posted by Branko Čibej <br...@apache.org>.
On 02.12.2013 01:29, Eric Covener wrote:
> I am looking at a httpd bug that causes a hang on windows but succeeds on unix.
>
> It seems that (short) files are opened w/ buffering, read,
> apr_file_closed, and read again [succesfully on unix]
>
> On Unix, they sare satisfied out of the buffer. file->fileset is -1.
> On Windows, the destroyed apr_thread_mutex causes a hang.
>
> Is reading from the closed file on the extreme bogus end of the
> spectrum as I suspect and just getting lucky on the unix case?
I'd certainly call a successful read from a closed file a bug.
> Should they blow up in 2.0 during a read if they've been closed?
Dunno ... my gut feeling in cases like this is to just leave it be.
Developers should have some fun finding heisenbugs, too. :)
-- Brane