You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hive.apache.org by "Sergey Shelukhin (JIRA)" <ji...@apache.org> on 2017/03/12 00:24:04 UTC

[jira] [Commented] (HIVE-15665) LLAP: OrcFileMetadata objects in cache can impact heap usage

    [ https://issues.apache.org/jira/browse/HIVE-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15906367#comment-15906367 ] 

Sergey Shelukhin commented on HIVE-15665:
-----------------------------------------

Indexes are going to be read like regular streams... I was able to work on this a little bit more, I now have unfinished code for all 3 metadata cache constituents :)

> LLAP: OrcFileMetadata objects in cache can impact heap usage
> ------------------------------------------------------------
>
>                 Key: HIVE-15665
>                 URL: https://issues.apache.org/jira/browse/HIVE-15665
>             Project: Hive
>          Issue Type: Improvement
>          Components: llap
>            Reporter: Rajesh Balamohan
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-15665.WIP.patch
>
>
> OrcFileMetadata internally has filestats, stripestats etc which are allocated in heap. On large data sets, this could have an impact on the heap usage and the memory usage by different executors in LLAP.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)