You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Qiang.Kang (Jira)" <ji...@apache.org> on 2020/10/24 07:40:00 UTC
[jira] [Commented] (HIVE-24311) Rowcontainer should reset
readBlocks when we clear rows to prevent OOM.
[ https://issues.apache.org/jira/browse/HIVE-24311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17220022#comment-17220022 ]
Qiang.Kang commented on HIVE-24311:
-----------------------------------
Just added a patch for this issue. Anybody help to review the code? Thanks!
> Rowcontainer should reset readBlocks when we clear rows to prevent OOM.
> -----------------------------------------------------------------------
>
> Key: HIVE-24311
> URL: https://issues.apache.org/jira/browse/HIVE-24311
> Project: Hive
> Issue Type: Bug
> Affects Versions: All Versions
> Reporter: Qiang.Kang
> Assignee: Qiang.Kang
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hi, We found that `Rowcontainer.clearRows()` only reset some index, such as: `addCursor`, `itrCursor`, etc, without resetting read blocks.
> As we all know, `currentReadBlock` and `currentWriteBlock` is the main memory usage of a `Rowcontainer` and might be very large, depending on the data pattern.
> `currentReadBlock` and `currentWriteBlock` won't be the same object after rowcontainer flushed data to disk.
> Resetting `currentReadBlock` and `currentWriteBlock` while clearing rows will prevent OOM.
>
> Therefore, I submit a patch to reset read blocks for `Rowcontainer`, just like `PTFRowcontainer` does.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)