You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Jun Rao (JIRA)" <ji...@apache.org> on 2012/10/09 17:44:04 UTC
[jira] [Resolved] (KAFKA-512) Remove checksum from
ByteBufferMessageSet.iterator
[ https://issues.apache.org/jira/browse/KAFKA-512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jun Rao resolved KAFKA-512.
---------------------------
Resolution: Fixed
Fixed in KAFKA-506.
> Remove checksum from ByteBufferMessageSet.iterator
> --------------------------------------------------
>
> Key: KAFKA-512
> URL: https://issues.apache.org/jira/browse/KAFKA-512
> Project: Kafka
> Issue Type: Bug
> Reporter: Jay Kreps
> Priority: Blocker
> Labels: bugs
> Fix For: 0.8
>
>
> Messages are explicitly checksumed in Log.append. But there is also a checksum computed and checked automatically in ByteBufferMessageSet.iterator as we iterate. This iterator is used quite a lot and as a result we compute this checksum 39 times on a single message produce. It turns out the default crc32 implementation in java is quite expensive so this is not good.
> The proposed fix is to remove the automatic checksum from the iterator and add explicit isValid() checks in the consumer as well as retaining the existing check in Log.append().
> If folks are in agreement I will probably include this in the KAFKA-506 patch as that already contains a lot of ByteBufferMessageSet changes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira