[jira] [Created] (KAFKA-9716) Values of compression-rate and compression-rate-avg are misleading

["Christian Kosmowski (Jira)" <ji...@apache.org>]

Christian Kosmowski created KAFKA-9716:
------------------------------------------

             Summary: Values of compression-rate and compression-rate-avg are misleading
                 Key: KAFKA-9716
                 URL: https://issues.apache.org/jira/browse/KAFKA-9716
             Project: Kafka
          Issue Type: Bug
          Components: clients, compression
    Affects Versions: 2.4.1
            Reporter: Christian Kosmowski


The values of the following metrics:

compression-rate and compression-rate-avg and basically every other compression-rate (i.e.) topic compression rate

are confusing.

They are calculated as follows:
{code:java}
if (numRecords == 0L) {
    buffer().position(initialPosition);
    builtRecords = MemoryRecords.EMPTY;
} else {
    if (magic > RecordBatch.MAGIC_VALUE_V1)
        this.actualCompressionRatio = (float) writeDefaultBatchHeader() / this.uncompressedRecordsSizeInBytes;
    else if (compressionType != CompressionType.NONE)
        this.actualCompressionRatio = (float) writeLegacyCompressedWrapperHeader() / this.uncompressedRecordsSizeInBytes;

    ByteBuffer buffer = buffer().duplicate();
    buffer.flip();
    buffer.position(initialPosition);
    builtRecords = MemoryRecords.readableRecords(buffer.slice());
}
{code}
basically the compressed size is divided by the uncompressed size which leads to a value < 1 for high compression (good if you want compression) or > 1 for poor compression (bad if you want compression).

From the name "compression rate" i would expect the exact opposite. Apart from the fact that the word "rate" usually refers to comparisons based on values of different units (miles per hour) the correct word "ratio" would refer to the uncompressed size divided by the compressed size.

So if the compressed data takes half the space of the uncompressed data the correct value for compression ratio (or rate) would be 2 and not 0.5 as kafka reports it. That is really confusing and i would AT LEAST expect that this behaviour would be documented somewhere, but it's not all documentation sources just say "the compression rate".



--
This message was sent by Atlassian Jira
(v8.3.4#803005)