[jira] [Resolved] (KUDU-1600) Disallow enabling compression on already-compressed encodings

["Todd Lipcon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KUDU-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Todd Lipcon resolved KUDU-1600.
-------------------------------
       Resolution: Fixed
         Assignee: Todd Lipcon
    Fix Version/s: 1.3.0

Rather than disallow such encodings, I added some code in d9bd5b8752efd4ae407043e491b7fd1e0b885a92 which automatically disables compression on a per-block basis when the codec is found to not reach a target compression ratio.

> Disallow enabling compression on already-compressed encodings
> -------------------------------------------------------------
>
>                 Key: KUDU-1600
>                 URL: https://issues.apache.org/jira/browse/KUDU-1600
>             Project: Kudu
>          Issue Type: Bug
>          Components: cfile, master, perf
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 1.3.0
>
>
> Some encodings (eg bitshuffle) are already compressed with LZ4, and enabling another pass of LZ4 compression on top of it just burns CPU. We should disallow such pairs of encodings.
> This is slightly trickier in the case of something like dictionary encoding, since it has the fallback path. For blocks that have fallen back to plain encoding, LZ4 is a good idea, whereas for blocks that are dict-encoded, the code words are already compressed and LZ4 is a waste.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)