You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hive.apache.org by "Sergey Shelukhin (JIRA)" <ji...@apache.org> on 2017/02/11 03:45:42 UTC

[jira] [Updated] (HIVE-15665) LLAP: OrcFileMetadata objects in cache can impact heap usage

     [ https://issues.apache.org/jira/browse/HIVE-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sergey Shelukhin updated HIVE-15665:
------------------------------------
    Attachment: HIVE-15665.WIP.patch

WIP patch that refactors buffers for multiple type support, and changes file metadata to a buffer. Need to work on stripe metadata, which doesn't have a convenient method to get a buffer, and will also probably need separate caching for row indexes.

> LLAP: OrcFileMetadata objects in cache can impact heap usage
> ------------------------------------------------------------
>
>                 Key: HIVE-15665
>                 URL: https://issues.apache.org/jira/browse/HIVE-15665
>             Project: Hive
>          Issue Type: Improvement
>          Components: llap
>            Reporter: Rajesh Balamohan
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-15665.WIP.patch
>
>
> OrcFileMetadata internally has filestats, stripestats etc which are allocated in heap. On large data sets, this could have an impact on the heap usage and the memory usage by different executors in LLAP.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)