You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "chenjianxun (JIRA)" <ji...@apache.org> on 2019/03/12 10:47:00 UTC
[jira] [Comment Edited] (HIVE-16220) Memory leak when creating a
table using location and NameNode in HA
[ https://issues.apache.org/jira/browse/HIVE-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790264#comment-16790264 ]
chenjianxun edited comment on HIVE-16220 at 3/12/19 10:46 AM:
--------------------------------------------------------------
is there any progess for this issue?
i have got the same problem, the instances of "org.apache.hadoop.hive.conf.HiveConf" keep growing.
and the dump file of hiveserver2 shows that there is a memory leak problem
i am using hive 2.3.4 and hadoop-2.6.0-cdh5.16.1
Leak Suspects
1,041 instances of *"org.apache.hadoop.hive.conf.HiveConf"*, loaded by *"sun.misc.Launcher$AppClassLoader @ 0xeaaa0058"* occupy *270,268,056 (52.70%)* bytes. These instances are referenced from one instance of *"java.util.HashMap$Node[]"*, loaded by *"<system class loader>"*
was (Author: chenjianxun):
is there any progess for this issue?
i have got the same problem, the instances of "org.apache.hadoop.hive.conf.HiveConf" keep growing.
and the dump file of hiveserver2 shows that there is a memory leak problem.
Leak Suspects
1,041 instances of *"org.apache.hadoop.hive.conf.HiveConf"*, loaded by *"sun.misc.Launcher$AppClassLoader @ 0xeaaa0058"* occupy *270,268,056 (52.70%)* bytes. These instances are referenced from one instance of *"java.util.HashMap$Node[]"*, loaded by *"<system class loader>"*
> Memory leak when creating a table using location and NameNode in HA
> -------------------------------------------------------------------
>
> Key: HIVE-16220
> URL: https://issues.apache.org/jira/browse/HIVE-16220
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
> Affects Versions: 1.2.1
> Environment: HDP-2.4.0.0
> Reporter: Angel Alvarez Pascua
> Priority: Major
>
> The following simple DDL
> CREATE TABLE `test`(`field` varchar(1)) LOCATION 'hdfs://benderHA/apps/hive/warehouse/test'
> ends up generating a huge memory leak in the HiveServer2 service.
> After two weeks without a restart, the service stops suddenly because of OutOfMemory errors.
> This only happens when we're in an environment in which the NameNode is in HA, otherwise, nothing (so weird) happens. If the location clause is not present, everything is also fine.
> It seems, multiples instances of Hadoop configuration are created when we're in an HA environment:
> <AFTER ONE EXECUTIONS OF CREATE TABLE WITH LOCATION>
> 2.618 instances of "org.apache.hadoop.conf.Configuration", loaded by "sun.misc.Launcher$AppClassLoader @ 0x4d260de88"
> occupy 350.263.816 (81,66%) bytes. These instances are referenced from one instance of "java.util.HashMap$Node[]",
> loaded by "<system class loader>"
> <AFTER TWO EXECUTIONS OF CREATE TABLE WITH LOCATION>
> 5.216 instances of "org.apache.hadoop.conf.Configuration", loaded by "sun.misc.Launcher$AppClassLoader @ 0x4d260de88"
> occupy 699.901.416 (87,32%) bytes. These instances are referenced from one instance of "java.util.HashMap$Node[]",
> loaded by "<system class loader>"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)