You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hive.apache.org by "Sergey Shelukhin (JIRA)" <ji...@apache.org> on 2016/06/02 23:29:59 UTC

[jira] [Comment Edited] (HIVE-13913) LLAP: introduce backpressure to recordreader

    [ https://issues.apache.org/jira/browse/HIVE-13913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313289#comment-15313289 ] 

Sergey Shelukhin edited comment on HIVE-13913 at 6/2/16 11:29 PM:
------------------------------------------------------------------

Simple patch. I am removing the pause mechanism since it's hard to propagate to reader. Better backpressure is possible, but for that a thread boundary is needed between reading and decompression (too much memory usage potential after decompression if we use the existing one before decoding to avoid small allocations in decoding) and probably an additional check so we don't read to much.


was (Author: sershe):
Simple patch. I am removing the pause mechanism since it's hard to propagate to recordreader. 
Better backpressure is possible, but for that a thread boundary is needed between reading and decompression (too much memory usage potential after decompression if we use the existing one before decoding to avoid small allocations in decoding) and probably an additional check so we don't read to much.

> LLAP: introduce backpressure to recordreader
> --------------------------------------------
>
>                 Key: HIVE-13913
>                 URL: https://issues.apache.org/jira/browse/HIVE-13913
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-13913.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)