[jira] [Commented] (PHOENIX-5595) Use ROW_INDEX_V1 block encoding and zSTD compression by default

["Andrew Kyle Purtell (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-5595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16983788#comment-16983788 ] 

Andrew Kyle Purtell commented on PHOENIX-5595:
----------------------------------------------

I think it would be fine to try for ZSTD as default if available, and fall back to FAST_DIFF and no compression if not. Or perhaps ROW_INDEX and GZ would still be less bad than other options as fallback, given GZ has a pure Java implementation in every JRE. We only take the decompression hit once at read, while with FAST_DIFF, at every scan. 

> Use ROW_INDEX_V1 block encoding and zSTD compression by default
> ---------------------------------------------------------------
>
>                 Key: PHOENIX-5595
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5595
>             Project: Phoenix
>          Issue Type: Wish
>            Reporter: Lars Hofhansl
>            Priority: Major
>
> Phoenix defaults to FAST_DIFF block encoding and no compression (not needed with FAST_DIFF).
> I blogged about this extensively here: http://hadoop-hbase.blogspot.com/2018/10/apache-hbase-and-apache-phoenix-more-on.html
> We should switch the default to block encoding ROW_INDEX_V1 and compression zSTD for all newly created tables (including global indexes). Local indexes can stay with FAST_DIFF, but perhaps for completeness we should just switch everything.
> The only wrinkle is that FAST_DIFF also does compression (i.e. the diff encoding), and ROW_INDEX_V1 actually increases the block size a little bit since it keeps in a index of row keys so that it can do binary search inside of an HFile block. Hence it needs to be paired with compression. Every test I did suggests that zSTD is the best.
> The main wrinkle is that zSTD needs a Hadoop/HBase build with native zSTD support compiled.
> I marked this as a Wish... Perhaps we can discuss here.
> What I do know is that FAST_DIFF has outgrown its usefulness, seeking into FAST_DIFF is (naturally) slow since it would need to seek to that last know fully stored key and then play all the diffs forward from there to the actual row we want to seek to. This impacts GETs.
> zSTD also offers better compression and thus reduced IO even when paired with ROW_INDEX_V1.
> [~apurtell] What we discussed a while back.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)