Re: [DISCUSS] KIP-390: Add producer option to adjust compression level

[smiklos <sz...@gmail.com>]

Hi,

Is there any update on this? I've done performance test with Avro data 
and Snappy compression.

Setting the buffer from 32kb to 128kb brings a rough 10% decrease in 
storage which is a big deal.

I could offer working on this as well.

Best regards,

Miklos

Re: [DISCUSS] KIP-390: Add producer option to adjust compression level

[Dongjin Lee <do...@apache.org>]

Hi Smiklos,


Thanks for your interest in this issue. I am the author of this PR and now
rebasing the code to the latest trunk.


I have some questions:



   1. Could you share how you conducted the benchmark? I want to run the
   full validation with all cases.
   2. As you can see in the PR, it proposes three configuration
   alternatives. Which one do you prefer?


Thanks,

Dongjin


+1. Sorry for the late reply. I was working on another issue.

On Wed, Jan 15, 2020 at 2:01 AM smiklos <sz...@gmail.com> wrote:

> Hi,
>
> Is there any update on this? I've done performance test with Avro data
> and Snappy compression.
>
> Setting the buffer from 32kb to 128kb brings a rough 10% decrease in
> storage which is a big deal.
>
> I could offer working on this as well.
>
> Best regards,
>
> Miklos
>
>
> --
*Dongjin Lee*

*A hitchhiker in the mathematical world.*
*github:  <http://goog_969573159/>github.com/dongjinleekr
<https://github.com/dongjinleekr>linkedin: kr.linkedin.com/in/dongjinleekr
<https://kr.linkedin.com/in/dongjinleekr>speakerdeck: speakerdeck.com/dongjin
<https://speakerdeck.com/dongjin>*