You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "Michael Coon (JIRA)" <ji...@apache.org> on 2016/04/08 21:31:25 UTC

[jira] [Created] (KAFKA-3534) Deserialize on demand when default time extractor used

Michael Coon created KAFKA-3534:
-----------------------------------

             Summary: Deserialize on demand when default time extractor used
                 Key: KAFKA-3534
                 URL: https://issues.apache.org/jira/browse/KAFKA-3534
             Project: Kafka
          Issue Type: Improvement
          Components: streams
    Affects Versions: 0.9.0.1
            Reporter: Michael Coon
            Assignee: Guozhang Wang
            Priority: Minor


When records are added to the RecordQueue, they are deserialized at that time in order to extract the timestamp. But for some data flows where large messages are consumed (particularly compressed messages), this can result in large spikes in memory as all messages must be deserialized prior to processing (and getting out of memory). An optimization might be to only require deserialization at this stage if a non-default timestamp extractor is being used.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)