[jira] [Closed] (KAFKA-512) Remove checksum from ByteBufferMessageSet.iterator

["Jun Rao (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jun Rao closed KAFKA-512.
-------------------------

    Assignee: Jay Kreps
    
> Remove checksum from ByteBufferMessageSet.iterator
> --------------------------------------------------
>
>                 Key: KAFKA-512
>                 URL: https://issues.apache.org/jira/browse/KAFKA-512
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Jay Kreps
>            Assignee: Jay Kreps
>            Priority: Blocker
>              Labels: bugs
>             Fix For: 0.8
>
>
> Messages are explicitly checksumed in Log.append. But there is also a checksum computed and checked automatically in ByteBufferMessageSet.iterator as we iterate. This iterator is used quite a lot and as a result we compute this checksum 39 times on a single message produce. It turns out the default crc32 implementation in java is quite expensive so this is not good.
> The proposed fix is to remove the automatic checksum from the iterator and add explicit isValid() checks in the consumer as well as retaining the existing check in Log.append().
> If folks are in agreement I will probably include this in the KAFKA-506 patch as that already contains a lot of ByteBufferMessageSet changes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira