file bucket, mmaps and SIGBUS

[Stefan Eissing <st...@greenbytes.de>]

The reported problem of segmentation faults in mod_http2 during a graceful restart seems to boil down to the fact that the document root is updated at the same time. On my OS X I can reproduce the SIGBUS when GETting the same 10 MB file again and again and copying over it.

When I disable mmap for the file buckets, the crash does not occur and the request fails as expected.

So, my understanding is that apr_bucket_read() hands out a pointer to the mmap'ed memory and when I copy that data somewhere else, and the underlying file is gone, a SIGBUS occurs as the OS has no other means of telling me about the problem.

Interestingly, when I send the same buckets down to the core output filters, the problem does not happen. Since this seems to involve writes using iovec's the kernel might handle this properly with a return code? I am not sure. Or maybe it's the sendfile() that takes care of it...

Disabling mmap in general seems not really a good solution, but it is not as bad as http2 sends file out in <= 16KB chunks anyway. Since for TLS, I need to assemble frame data anyway, reading directly from file into a properly allocated frame buffer might be better anyway.

However I'd like to understand the problem and my options here: what would I need to do to use mmap'ed files properly? Catch SIGBUS during the copy? Advice appreciated.

Cheers,

-Stefan

Re: file bucket, mmaps and SIGBUS

[Stefan Eissing <st...@greenbytes.de>]

> Am 19.04.2016 um 13:50 schrieb Yann Ylavic <yl...@gmail.com>:
> 
> Hi Stefan,
> 
> On Tue, Apr 19, 2016 at 12:02 PM, Stefan Eissing
> <st...@greenbytes.de> wrote:
>> The reported problem of segmentation faults in mod_http2 during a graceful restart seems to boil down to the fact that the document root is updated at the same time. On my OS X I can reproduce the SIGBUS when GETting the same 10 MB file again and again and copying over it.
> 
> Well known issue of mmap() with truncated files :/
> 
>> 
>> However I'd like to understand the problem and my options here: what would I need to do to use mmap'ed files properly? Catch SIGBUS during the copy? Advice appreciated.
> 
> Playing with SIGBUS is not wise IMHO, you could possibly disable
> mmap() on a per bucket basis by using apr_bucket_file_enable_mmap()...

Thanks Yann, I tested that already and it works nicely. I see no negative impacts on my OS X tests, even some perf improvements for large files.

Off it is then.

- Stefan

Re: file bucket, mmaps and SIGBUS

[Yann Ylavic <yl...@gmail.com>]

Hi Stefan,

On Tue, Apr 19, 2016 at 12:02 PM, Stefan Eissing
<st...@greenbytes.de> wrote:
> The reported problem of segmentation faults in mod_http2 during a graceful restart seems to boil down to the fact that the document root is updated at the same time. On my OS X I can reproduce the SIGBUS when GETting the same 10 MB file again and again and copying over it.

Well known issue of mmap() with truncated files :/

>
> However I'd like to understand the problem and my options here: what would I need to do to use mmap'ed files properly? Catch SIGBUS during the copy? Advice appreciated.

Playing with SIGBUS is not wise IMHO, you could possibly disable
mmap() on a per bucket basis by using apr_bucket_file_enable_mmap()...

Regards,
Yann.