You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Weijie Guo (Jira)" <ji...@apache.org> on 2022/07/05 14:37:00 UTC

[jira] [Commented] (FLINK-28382) Introduce new compression algorithms of higher compression ratio

    [ https://issues.apache.org/jira/browse/FLINK-28382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17562696#comment-17562696 ] 

Weijie Guo commented on FLINK-28382:
------------------------------------

[~kevin.cyj] Maybe i can help introduce some compression algorithms such as z-std, and the default lz4 compressor should support configurable compression levels.

> Introduce new compression algorithms of higher compression ratio
> ----------------------------------------------------------------
>
>                 Key: FLINK-28382
>                 URL: https://issues.apache.org/jira/browse/FLINK-28382
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Runtime / Network
>            Reporter: Yingjie Cao
>            Priority: Major
>             Fix For: 1.16.0
>
>
> Currently, we use lz4 for shuffle data compression which is a good balance between IO optimization and CPU consumption. But for some scenarios, the IO becomes bottleneck and the storage space is limited (especially for k8s environment). For these cases, we need compression algorithms of higher compression ratio to further reduce IO.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)