You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "mahesh kumar behera (Jira)" <ji...@apache.org> on 2020/01/21 06:39:00 UTC
[jira] [Comment Edited] (HIVE-22753) Fix gradual mem leak:
Operationlog related appenders should be cleared up on errors
[ https://issues.apache.org/jira/browse/HIVE-22753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17019885#comment-17019885 ]
mahesh kumar behera edited comment on HIVE-22753 at 1/21/20 6:38 AM:
---------------------------------------------------------------------
We should not clean up the operation log before close operation. User may try to query operation log using the operation handle. Anyways for failed operation, operation close is called and that should clear up the operation log. Do you think this could be the reason for the leak ? https://issues.apache.org/jira/browse/HIVE-22733
was (Author: maheshk114):
We should not clean up the operation log before close operation. User may try to query operation log using the operation handle. Anyways for failed operation, operation close is called and that should clear up the operation log.
> Fix gradual mem leak: Operationlog related appenders should be cleared up on errors
> ------------------------------------------------------------------------------------
>
> Key: HIVE-22753
> URL: https://issues.apache.org/jira/browse/HIVE-22753
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Priority: Minor
> Attachments: HIVE-22753.1.patch, image-2020-01-21-11-14-37-911.png, image-2020-01-21-11-17-59-279.png, image-2020-01-21-11-18-37-294.png
>
>
> In case of exception in SQLOperation, operational log does not get cleared up. This causes gradual build up of HushableRandomAccessFileAppender causing HS2 to OOM after some time.
> !image-2020-01-21-11-14-37-911.png|width=431,height=267!
>
> Allocation tree
> !image-2020-01-21-11-18-37-294.png|width=425,height=178!
>
> Prod instance mem
> !image-2020-01-21-11-17-59-279.png|width=698,height=209!
>
> Each HushableRandomAccessFileAppender holds internal ref to RandomAccessFileAppender which holds a 256 KB bytebuffer, causing the mem leak.
> Related ticket: HIVE-18820
--
This message was sent by Atlassian Jira
(v8.3.4#803005)