[jira] Created: (MODPYTHON-215) Using compression on request content and FieldStorage can cause problems.

["Graham Dumpleton (JIRA)" <ji...@apache.org>]

Using compression on request content and FieldStorage can cause problems.
-------------------------------------------------------------------------

                 Key: MODPYTHON-215
                 URL: https://issues.apache.org/jira/browse/MODPYTHON-215
             Project: mod_python
          Issue Type: Bug
          Components: core
    Affects Versions: 3.2.10, 3.3
            Reporter: Graham Dumpleton


The problems occur when a client uses 'Content-Encoding' of 'gzip', ie., it compresses the content of a POST request and Apache is configured to decompress it.

The first issue is that the original content length header will be the length of data in the body before it is decompressed. If the content type is "application/x-www-form-urlencoded" this will result in not all the decompressed data (which would normally be greater in size) being read in and used as it uses the original value of the content length header explicitly when doing the read.

        if ctype.startswith("application/x-www-form-urlencoded"):
            pairs = parse_qsl(req.read(clen), keep_blank_values)

Note that the change in content size is not a problem with "multipart/*" POST requests as it reads data in blocks until req.read() returns an empty string to indicate end of data and doesn't consult content length at all.

The second issue is if Apache 2.2 is being used and the mod_filter FilterProtocol directive is used to correctly indicate that input filters are being used which will change the size of the content, then it will actually zap the content length header, removing it from the set of input headers. If this is done it will cause FieldStorage to wrongly flag the request as not having a content length specified in the first place.

        try:
            clen = int(req.headers_in["content-length"])
        except (KeyError, ValueError):
            # absent content-length is not acceptable
            raise apache.SERVER_RETURN, apache.HTTP_LENGTH_REQUIRED

What should be done to fix this later issue is for FieldStorage to rely on the fact that req.read() internally performs a check on first read to ensure that the content length header was set. It would just need to be checked that this check is before mod_filter zaps the content length header on determining that a input filter which actually change the length.

As to the attempt to use the content length header when it is "application/x-www-form-urlencoded", it perhaps can do blocked reads as well until it accumulates all the content, join it back together and then use it. Note that it can't just use req.read() with no arguments because of MODPYTHON-212.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira