You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hive.apache.org by "Prasanth Jayachandran (JIRA)" <ji...@apache.org> on 2015/11/18 03:50:10 UTC

[jira] [Commented] (HIVE-12450) OrcFileMergeOperator does not use correct compression buffer size

    [ https://issues.apache.org/jira/browse/HIVE-12450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15010089#comment-15010089 ] 

Prasanth Jayachandran commented on HIVE-12450:
----------------------------------------------

[~sershe] Can you take a look at this patch? Most changes are test + golden file out.

> OrcFileMergeOperator does not use correct compression buffer size
> -----------------------------------------------------------------
>
>                 Key: HIVE-12450
>                 URL: https://issues.apache.org/jira/browse/HIVE-12450
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 1.2.0, 1.3.0, 1.2.1, 2.0.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>         Attachments: HIVE-12450.1.patch
>
>
> OrcFileMergeOperator checks for compatibility before merging orc files. This compatibility check include checking compression buffer size. But the output file that is created does not honor the compression buffer size and always defaults to 256KB. This will not be a problem when reading the orc file but can create unwanted memory pressure because of wasted space within compression buffer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)