You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "Dana Powers (JIRA)" <ji...@apache.org> on 2016/01/27 20:43:39 UTC

[jira] [Created] (KAFKA-3160) Kafka LZ4 framing code miscalculates header checksum

Dana Powers created KAFKA-3160:
----------------------------------

             Summary: Kafka LZ4 framing code miscalculates header checksum
                 Key: KAFKA-3160
                 URL: https://issues.apache.org/jira/browse/KAFKA-3160
             Project: Kafka
          Issue Type: Bug
          Components: compression
    Affects Versions: 0.9.0.0, 0.8.2.1, 0.8.2.0, 0.8.2.2
            Reporter: Dana Powers


KAFKA-1493 implements the LZ4 framing specification, but it incorrectly calculates the header checksum. Specifically, the current implementation includes the 4-byte MagicNumber in the checksum, which is incorrect.
http://cyan4973.github.io/lz4/lz4_Frame_format.html

Third-party clients that attempt to use off-the-shelf lz4 framing find that brokers reject messages as having a corrupt checksum. So currently non-java clients must 'fixup' lz4 packets to deal with the broken checksum.

Magnus first identified this issue in librdkafka; kafka-python has the same problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)