[jira] [Created] (KAFKA-275) max.message.size is not enforced for compressed messages

["Neha Narkhede (Created) (JIRA)" <ji...@apache.org>]

max.message.size is not enforced for compressed messages
--------------------------------------------------------

                 Key: KAFKA-275
                 URL: https://issues.apache.org/jira/browse/KAFKA-275
             Project: Kafka
          Issue Type: Bug
          Components: core
    Affects Versions: 0.7
            Reporter: Neha Narkhede


The max.message.size check is not performed for compressed messages, but only for each message that forms a compressed message. Due to this, even if the max.message.size is set to 1MB, the producer can technically send n 1MB messages as one compressed message. This can cause memory issues on the server as well as deserialization issues on the consumer. The consumer's fetch size has to be > max.message.size in order to be able to read data. If one message is larger than the fetch.size, the consumer will throw an exception and cannot proceed until the fetch.size is increased. 

Due to this bug, even if the fetch.size > max.message.size, the consumer can still get stuck on a message that is larger than max.message.size.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira