[jira] [Commented] (KAFKA-1253) Implement compression in new producer

["Guozhang Wang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13906339#comment-13906339 ] 

Guozhang Wang commented on KAFKA-1253:
--------------------------------------

It seems incremental compression would not be possible with nio.ByteBuffer, so we cannot really do that in RecordAccumulator's append call. So Instead I am proposing the following:

1. Make compression type global to the producer client by adding COMPRESSION_TYPE_CONFIG to ProducerConfig. Then all the messages ever sent by this client would use the same compression type.

2. In Sender.collate function, compress the messages in buffer when creating the InFlightRequest. By doing this the buffer pool would always contain non-compressed data.

Thoughts?

> Implement compression in new producer
> -------------------------------------
>
>                 Key: KAFKA-1253
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1253
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: producer 
>            Reporter: Jay Kreps
>




--
This message was sent by Atlassian JIRA
(v6.1.5#6160)