You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Christopher Beard <cb...@bloomberg.net> on 2020/11/27 18:07:56 UTC
Re: [DISCUSS] KIP-640 Add log compression analysis tool
Bump. I'd like to gather more feedback on this!
Chris
On 2020/08/17 20:23:51, "Christopher Beard (BLOOMBERG/ 919 3RD A)" <cb...@bloomberg.net> wrote:
> Hi everyone,
>
> I would like to start a discussion on KIP-640:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-640%3A+Add+log+compression+analysis+tool
>
> This KIP outlines a new CLI tool which helps compare how the various compression types supported by Kafka reduce the size of a log (and therefore more broadly, of a topic).
>
> I've put together a PR that might help serve as a starting point for comments and suggestions.
> [WIP] PR: https://github.com/apache/kafka/pull/9193
>
> Thanks,
> Chris Beard
Re: [DISCUSS] KIP-640 Add log compression analysis tool
Posted by James Cheng <wu...@gmail.com>.
Chris,
This (understandably) requires access to the log segment files on disk. Managed Kafka services are becoming more popular (Confluent Cloud, Amazon MSK) and they do not expose the log segment files on disk. It’d be great to have an equivalent functionality that would work on managed services.
But, that would be a completely different implementation, so maybe it’s not something you want to handle right now.
-James
Sent from my iPhone
> On Nov 27, 2020, at 10:14 AM, Christopher Beard <cb...@bloomberg.net> wrote:
>
> Bump. I'd like to gather more feedback on this!
>
> Chris
>
>> On 2020/08/17 20:23:51, "Christopher Beard (BLOOMBERG/ 919 3RD A)" <cb...@bloomberg.net> wrote:
>> Hi everyone,
>>
>> I would like to start a discussion on KIP-640:
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-640%3A+Add+log+compression+analysis+tool
>>
>> This KIP outlines a new CLI tool which helps compare how the various compression types supported by Kafka reduce the size of a log (and therefore more broadly, of a topic).
>>
>> I've put together a PR that might help serve as a starting point for comments and suggestions.
>> [WIP] PR: https://github.com/apache/kafka/pull/9193
>>
>> Thanks,
>> Chris Beard