[jira] Commented: (HIVE-819) Add lazy decompress ability to RCFile

["He Yongqiang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754659#action_12754659 ] 

He Yongqiang commented on HIVE-819:
-----------------------------------

It turns out that changing the filter condition from 9 to 8, the number of decompressions is not reduced at all compared to eager decompression.  That means it needs to decompress all bytes just not applying lazy decompression. It needs to decompress all block data of other needed columns because there is always one row with duration >9  in every block...

> Add lazy decompress ability to RCFile
> -------------------------------------
>
>                 Key: HIVE-819
>                 URL: https://issues.apache.org/jira/browse/HIVE-819
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Query Processor, Serializers/Deserializers
>            Reporter: He Yongqiang
>             Fix For: 0.5.0
>
>         Attachments: hive-819-2009-9-12.patch
>
>
> This is especially useful for a filter scanning. 
> For example, for query 'select a, b, c from table_rc_lazydecompress where a>1;' we only need to decompress the block data of b,c columns when one row's column 'a' in that block satisfies the filter condition.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.