You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by "tanmay (Jira)" <ji...@apache.org> on 2026/04/25 16:05:00 UTC

[jira] [Created] (KAFKA-20528) Implement missing LZ4 Content Checksum in Lz4BlockOutputStream and Lz4BlockInputStream

tanmay created KAFKA-20528:
------------------------------

             Summary: Implement missing LZ4 Content Checksum in Lz4BlockOutputStream and Lz4BlockInputStream
                 Key: KAFKA-20528
                 URL: https://issues.apache.org/jira/browse/KAFKA-20528
             Project: Kafka
          Issue Type: Improvement
          Components: clients, compression
            Reporter: tanmay


While reviewing the LZ4 compression implementation in the Kafka clients module, I noticed that we are not implementing the Content Checksum feature defined in the LZ4 Frame Format specification (v1.5.1).

Currently, Lz4BlockOutputStream correctly handles block-level checksums (if configured), but it leaves three explicit TODO comments regarding the final content checksum. Specifically:

1. Lz4BlockOutputStream.java does not write the contentSize in the header or validate the flg for it.
2. Lz4BlockOutputStream.java writes the endmark  but fails to compute and append the   final 4-byte XXHash32 content checksum across the entire uncompressed payload.
3. L3.z4BlockInputStream.java reads the 4-byte checksum if the ContentChecksumSet flag is true, but merely discards it without actually verifying it against the decompressed stream.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)