[jira] Updated: (MODPYTHON-222) Support for chunked transfer encoding on request content.

["Alex Cichowski (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MODPYTHON-222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alex Cichowski updated MODPYTHON-222:
-------------------------------------

    Attachment: requestobject.c
                mod_python-211-212-222-fix-revised.patch

The attached patch should fix up this issue, as well as MODPYTHON-211 (readlines() leak), MODPYTHON-212 (read() with no arguments bug), and possibly MODPYTHON-234 (read() SystemError).

Changes included:

(1) Make read() and readline() ignore the content length in the case where no size argument is specified, and instead call ap_get_client_block() in a loop until it returns zero. The result buffer is dynamically expanded as necessary with _PyString_Resize().

(2) Add a request.set_read_policy() call to specify whether REQUEST_NO_BODY, REQUEST_CHUNKED_ERROR or REQUEST_CHUNKED_DECHUNK is passed to ap_setup_client_block() on the first read()/readline() call. The default is REQUEST_CHUNKED_DECHUNK, but could be set to REQUEST_CHUNKED_ERROR for compatibility with the current behaviour.

(3) Rewrite readlines() to be more rigorous in error handling and memory management.

(4) Add a new variation on read() called read_partial(), which attempts to ensure that it has returned all buffered data before blocking, by returning less than the requested amount where necessary. This is helpful to avoid deadlock when you would like to have the server read and respond to part of the client's request before the client sends the next part. I don't know whether it is strictly valid to use HTTP in this way or not, but I think it's nice to have this feature available anyway.

This patch is against the mod_python-3.3.1 release.

I have also attached an updated copy of requestobject.c for easy viewing without applying the patch.

> Support for chunked transfer encoding on request content.
> ---------------------------------------------------------
>
>                 Key: MODPYTHON-222
>                 URL: https://issues.apache.org/jira/browse/MODPYTHON-222
>             Project: mod_python
>          Issue Type: New Feature
>          Components: core
>    Affects Versions: 3.3.1
>            Reporter: Graham Dumpleton
>         Attachments: mod_python-211-212-222-fix-revised.patch, requestobject.c
>
>
> It is currently not possible to use chunked transfer encoding on request content delivered to a mod_python request handler.
> The use of chunked transfer encoding is explicitly blocked in C code by:
>         rc = ap_setup_client_block(self->request_rec, REQUEST_CHUNKED_ERROR);
> To allow chunked transfer encoding instead of REQUEST_CHUNKED_ERROR it would be necessary to supply REQUEST_CHUNKED_DECHUNK.
> Problem is that it isn't that simple.
> First off, the problems associated with MODPYTHON-212 have to be fixed with code being able to cope with there being no content length.
> The next issue is that req.read() method is currently documented as behaving as:
>   If the len argument is negative or omitted, reads all data given by the client.
> This means that can't have req.read() with no arguments mean give me everything that is currently available in input buffers as everyone currently expects it to return everything sent by client. Thus, to be able to process streaming data one would have to supply an amount of data that one wants to read. The code for that though will always try to ensure that that exact amount of data is read and will block if not enough and not end of input. A handler though may not want it to block and be happy with just getting what is read and only expect it to block if nothing currently available.
> In other words, the current specification for how req.read() behaves is incompatible with what would be required to support chunked transfer encoding on request content.
> Not sure how this conflict can be resolved.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.