You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by Zhe Zhang <zh...@us.ibm.com> on 2013/01/11 16:28:37 UTC
Question about httpd memory buffer alignment
Hello,
I would like httpd to use 4K-aligned buffers so that I can do direct I/O on
the html files being served. Could anyone tell me which file I need to
modify? I've already changed APR_BUCKET_BUFF_SIZE to 8192 so at least the
buffer size is multiple of 4K.
Thanks!
---
Zhe Zhang, Research Staff Member
Cloud Application Management
IBM T. J. Watson Research Center
http://researcher.ibm.com/person/us-zhezhang
Re: Question about httpd memory buffer alignment
Posted by Zhe Zhang <zh...@us.ibm.com>.
Ben Reser <be...@reser.org> wrote on 01/11/2013 02:48:00 PM:
> From: Ben Reser <be...@reser.org>
> To: Zhe Zhang/Watson/IBM@IBMUS,
> Cc: dev@apr.apache.org
> Date: 01/11/2013 02:48 PM
> Subject: Re: Question about httpd memory buffer alignment
>
> On Fri, Jan 11, 2013 at 11:21 AM, Ben Reser <be...@reser.org> wrote:
> > If you're trying to align the actual memory that's going to mean
> > messing with the memory pooling system for APR. Probably writing your
> > own allocator. And using that allocator with
> > apr_brigade_insert_file() call in default_handler() in server/core.c.
>
> If this is what you want you might be able to mess with
> APR_ALIGN_DEFAULT in include/apr_general.h, looks like it's aligning
> to 8 bytes.
>
Changed to 4096 but I'm still getting 8 bytes aligned buffers.
Thanks,
Zhe
Re: Question about httpd memory buffer alignment
Posted by Ben Reser <be...@reser.org>.
On Fri, Jan 11, 2013 at 11:21 AM, Ben Reser <be...@reser.org> wrote:
> If you're trying to align the actual memory that's going to mean
> messing with the memory pooling system for APR. Probably writing your
> own allocator. And using that allocator with
> apr_brigade_insert_file() call in default_handler() in server/core.c.
If this is what you want you might be able to mess with
APR_ALIGN_DEFAULT in include/apr_general.h, looks like it's aligning
to 8 bytes.
Re: Question about httpd memory buffer alignment
Posted by Ben Reser <be...@reser.org>.
On Fri, Jan 11, 2013 at 10:54 AM, Zhe Zhang <zh...@us.ibm.com> wrote:
> Hi Ben, thank you for the reply. I have disabled mmap and sendfile options.
> I'm getting 8192 bytes buffers; so the size is fine, but I want the starting
> address of the buffer to be 4K-aligned too. Now it seems aligned at 8 bytes
> boundary (xxxxx8 instead of xxxx000).
I'm assuming you're trying to align the access of data on the disk to
the actual blocks on the disk so that you're avoiding reading a block
multiple times across reads. If you're seeing the access is off by 8
bytes, it sounds like there's someplace that apr_file_ungetc() is
getting used.
If you're trying to align the actual memory that's going to mean
messing with the memory pooling system for APR. Probably writing your
own allocator. And using that allocator with
apr_brigade_insert_file() call in default_handler() in server/core.c.
Re: Question about httpd memory buffer alignment
Posted by Zhe Zhang <zh...@us.ibm.com>.
Ben Reser <be...@reser.org> wrote on 01/11/2013 01:45:43 PM:
> From: Ben Reser <be...@reser.org>
> To: Zhe Zhang/Watson/IBM@IBMUS,
> Cc: dev@apr.apache.org
> Date: 01/11/2013 01:46 PM
> Subject: Re: Question about httpd memory buffer alignment
>
> On Fri, Jan 11, 2013 at 7:28 AM, Zhe Zhang <zh...@us.ibm.com> wrote:
> > I would like httpd to use 4K-aligned buffers so that I can do direct
I/O on
> > the html files being served. Could anyone tell me which file I need to
> > modify? I've already changed APR_BUCKET_BUFF_SIZE to 8192 so at least
the
> > buffer size is multiple of 4K.
>
> If that isn't doing what you want then there are two possible places
> that the read is happening.
Hi Ben, thank you for the reply. I have disabled mmap and sendfile options.
I'm getting 8192 bytes buffers; so the size is fine, but I want the
starting address of the buffer to be 4K-aligned too. Now it seems aligned
at 8 bytes boundary (xxxxx8 instead of xxxx000).
>
> 1) Memory mapping is enabled and then the file is read by the kernel
> via the mmap system call. See file_make_mmap() in APR's
> buckets/apr_buckets_file.c.
>
> You can disable this with EnableMMAP off configuration directive to
httpd.
>
> 2) Sendfile support is enabled and then the file is read by the kernel
> via the sendfile system call. See send_brigade_nonblocking() in
> httpd's server/core_filters.c.
>
> You can disable this with EnableSendfile off configuration directiveto
httpd.
>
Re: Question about httpd memory buffer alignment
Posted by Ben Reser <be...@reser.org>.
On Fri, Jan 11, 2013 at 7:28 AM, Zhe Zhang <zh...@us.ibm.com> wrote:
> I would like httpd to use 4K-aligned buffers so that I can do direct I/O on
> the html files being served. Could anyone tell me which file I need to
> modify? I've already changed APR_BUCKET_BUFF_SIZE to 8192 so at least the
> buffer size is multiple of 4K.
If that isn't doing what you want then there are two possible places
that the read is happening.
1) Memory mapping is enabled and then the file is read by the kernel
via the mmap system call. See file_make_mmap() in APR's
buckets/apr_buckets_file.c.
You can disable this with EnableMMAP off configuration directive to httpd.
2) Sendfile support is enabled and then the file is read by the kernel
via the sendfile system call. See send_brigade_nonblocking() in
httpd's server/core_filters.c.
You can disable this with EnableSendfile off configuration directive to httpd.
Re: Question about httpd memory buffer alignment
Posted by Stefan Fritsch <sf...@sfritsch.de>.
On Friday 11 January 2013, Zhe Zhang wrote:
> I would like httpd to use 4K-aligned buffers so that I can do
> direct I/O on the html files being served. Could anyone tell me
> which file I need to modify? I've already changed
> APR_BUCKET_BUFF_SIZE to 8192 so at least the buffer size is
> multiple of 4K.
There is "--enable-allocator-uses-mmap" when compiling apr. It will
cause the apr allocator to do page aligned allocations.
But I think that would only be the first step because apr will always
put some management struct (apr_memnode_t and/or node_header_t) at the
start of the allocated space. This would somehow have to be changed so
that the management stuff is placed somewhere else.