[jira] [Comment Edited] (ORC-21) Add functionality to estimate memory footprint

["Aliaksei Sandryhaila (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ORC-21?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639347#comment-14639347 ] 

Aliaksei Sandryhaila edited comment on ORC-21 at 7/23/15 6:46 PM:
------------------------------------------------------------------

We use two components for reading ORC files: a reader and a reusable batch that we fill up with data. Since the end user should not know or worry about the internal workings of these components, both should be able to report how much memory that need.

Proposed solution: add methods uint64_t Reader::memoryUse() and uint64_t ColumnVectorBatch::memoryUse() that return an exact value (or at least an upper bound estimate) of the memory footprint of the respective classes/subclasses.


was (Author: asandryh):
We use two components for reading ORC files: a reader and a reusable batch that we fill up with data. Since the end user should not know or worry about the internal workings of these components, both should be able to report how much memory that need.

Proposed solution: add methods uint64_t Reader::memoryUse() and uint64_t ColumnVectorBatch::memoryUse() that return an exact, or at least an upper bound, on the memory footprint of the respective classes/subclasses.

> Add functionality to estimate memory footprint
> ----------------------------------------------
>
>                 Key: ORC-21
>                 URL: https://issues.apache.org/jira/browse/ORC-21
>             Project: Orc
>          Issue Type: Task
>            Reporter: Aliaksei Sandryhaila
>            Assignee: Aliaksei Sandryhaila
>
> ORC library allocates multiple large buffers to read and materialize ORC files. For stability of applications that use the library, it may be desirable to have an estimate (preferably, a tight upper bound) of a memory footprint.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)