You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Christophe JAILLET <ch...@wanadoo.fr> on 2016/09/01 06:27:03 UTC
Re: Unbounded memory usage in mod_dav + mod_headers/mod_deflate/...
Le 29/08/2016 � 18:12, Evgeny Kotkov a �crit :
> Evgeny Kotkov <ev...@visualsvn.com> writes:
>
>> It might be possible to rework mod_dav_svn, although it's going to take
>> some time. Currently, every top-level handler receives an `ap_filter_t *`
>> and passes it further, and all these places would have to be updated so
>> that the actual writes would target r->output_filters.
>>
>> There also is the function in the core part of mod_dav that shares the same
>> kind of problem and requires a separate fix. Please see mod_dav.h:554:
>>
>> DAV_DECLARE(void) dav_send_one_response(dav_response *response,
>> apr_bucket_brigade *bb,
>> ap_filter_t *output,
>> apr_pool_t *pool);
>>
>> For a start, I prepared two patches for mod_dav:
>>
>> - The first patch fixes the problem in dav_send_one_response(). Please note
>> that while dav_send_one_response() is public API, it's not yet released as
>> of 2.4.23. So, this patch changes it directly, without introducing a new
>> dav_send_one_response2() function for compatibility.
>>
>> - The second patch notes the API issue in the docs for the mentioned DAV
>> hooks, deliver(), deliver_report() and merge() \u2014 for the sake of anyone
>> who might be implementing a DAV provider.
>>
>> The patches are attached (log messages are in the beginning of each file).
> For the record, I completed a rework of mod_dav_svn that fixes a part
> of the issue. Now, all writes within mod_dav_svn target the actual filter
> list in r->output_filters, and this fixes one cause of the unbounded memory
> usage:
>
> https://svn.apache.org/r1758224
> https://svn.apache.org/r1758207
> https://svn.apache.org/r1758204
>
> However, the second cause of the unbounded memory usage is still present
> in mod_dav. This part of the issue is addressed by my patches from the
> previous e-mail.
>
> Does someone have the time and energy to review these two patches? Or
> maybe there's something in these patches that requires improving?
>
> Just in case, the patches are also available here:
>
> https://mail-archives.apache.org/mod_mbox/httpd-dev/201608.mbox/raw/%3CCAP_GPNhB92KN1KF7TO%3DGs%3DkDpO6weA5%2Bf3nxH3ni0CL0wvYj%2Bg%40mail.gmail.com%3E/2
> https://mail-archives.apache.org/mod_mbox/httpd-dev/201608.mbox/raw/%3CCAP_GPNhB92KN1KF7TO%3DGs%3DkDpO6weA5%2Bf3nxH3ni0CL0wvYj%2Bg%40mail.gmail.com%3E/3
>
> Thanks in advance!
>
>
> Regards,
> Evgeny Kotkov
>
Hi,
while looking at some memory consumption issue in mod_dav, you can also
have a look at the proposed patches in :
https://bz.apache.org/bugzilla/show_bug.cgi?id=48130
Definitively not the same approach, but the same goal.
CJ
Re: Unbounded memory usage in mod_dav + mod_headers/mod_deflate/...
Posted by Evgeny Kotkov <ev...@visualsvn.com>.
Evgeny Kotkov <ev...@visualsvn.com> writes:
> However, the problem I described in this thread is different, and it just
> happens to have the same visible consequence.
>
> What is probably more important, the dav_send_one_response() function
> that has the flaw (see the proposed patch) *just* got promoted into a public
> API, and will become the part of the next 2.4.x release [2]. So I prepared
> the patch, assuming that it would be better to fix this before it becomes
> public.
I committed the fix for dav_send_one_response() function in r1764040 [1],
and proposed it for backport to 2.4.x. Apart from this, I committed the
second patch that adds the note about the API issue in r1764043 [2].
[1] https://svn.apache.org/r1764040
[2] https://svn.apache.org/r1764043
Regards,
Evgeny Kotkov
Re: Unbounded memory usage in mod_dav + mod_headers/mod_deflate/...
Posted by Evgeny Kotkov <ev...@visualsvn.com>.
Evgeny Kotkov <ev...@visualsvn.com> writes:
> However, the problem I described in this thread is different, and it just
> happens to have the same visible consequence.
>
> What is probably more important, the dav_send_one_response() function
> that has the flaw (see the proposed patch) *just* got promoted into a public
> API, and will become the part of the next 2.4.x release [2]. So I prepared
> the patch, assuming that it would be better to fix this before it becomes
> public.
I committed the fix for dav_send_one_response() function in r1764040 [1],
and proposed it for backport to 2.4.x. Apart from this, I committed the
second patch that adds the note about the API issue in r1764043 [2].
[1] https://svn.apache.org/r1764040
[2] https://svn.apache.org/r1764043
Regards,
Evgeny Kotkov
Re: Unbounded memory usage in mod_dav + mod_headers/mod_deflate/...
Posted by Evgeny Kotkov <ev...@visualsvn.com>.
Christophe JAILLET <ch...@wanadoo.fr> writes:
> Hi,
>
> while looking at some memory consumption issue in mod_dav, you can also
> have a look at the proposed patches in :
> https://bz.apache.org/bugzilla/show_bug.cgi?id=48130
>
> Definitively not the same approach, but the same goal.
Thanks for the link. We had a lengthy discussion about the mod_dav's
memory usage during PROPFIND requests in svn-dev@ [1].
However, the problem I described in this thread is different, and it just
happens to have the same visible consequence.
What is probably more important, the dav_send_one_response() function
that has the flaw (see the proposed patch) *just* got promoted into a public
API, and will become the part of the next 2.4.x release [2]. So I prepared
the patch, assuming that it would be better to fix this before it becomes
public.
[1] https://mail-archives.apache.org/mod_mbox/subversion-dev/201512.mbox/%3CCAP_GPNhA4hBFdOC7Z1D-K9h_NHm8d7WjyfSF4oUOteUepkjaaA@mail.gmail.com%3E
[2] https://svn.apache.org/r1756561
Regards,
Evgeny Kotkov
Re: Unbounded memory usage in mod_dav + mod_headers/mod_deflate/...
Posted by Evgeny Kotkov <ev...@visualsvn.com>.
Christophe JAILLET <ch...@wanadoo.fr> writes:
> Hi,
>
> while looking at some memory consumption issue in mod_dav, you can also
> have a look at the proposed patches in :
> https://bz.apache.org/bugzilla/show_bug.cgi?id=48130
>
> Definitively not the same approach, but the same goal.
Thanks for the link. We had a lengthy discussion about the mod_dav's
memory usage during PROPFIND requests in svn-dev@ [1].
However, the problem I described in this thread is different, and it just
happens to have the same visible consequence.
What is probably more important, the dav_send_one_response() function
that has the flaw (see the proposed patch) *just* got promoted into a public
API, and will become the part of the next 2.4.x release [2]. So I prepared
the patch, assuming that it would be better to fix this before it becomes
public.
[1] https://mail-archives.apache.org/mod_mbox/subversion-dev/201512.mbox/%3CCAP_GPNhA4hBFdOC7Z1D-K9h_NHm8d7WjyfSF4oUOteUepkjaaA@mail.gmail.com%3E
[2] https://svn.apache.org/r1756561
Regards,
Evgeny Kotkov