You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Carl Steinbach (JIRA)" <ji...@apache.org> on 2017/02/13 03:37:41 UTC
[jira] [Updated] (HIVE-14764) Enabling
"hive.metastore.metrics.enabled" throws OOM in HiveMetastore
[ https://issues.apache.org/jira/browse/HIVE-14764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-14764:
----------------------------------
Labels: JMX Metrics Monitoring (was: )
> Enabling "hive.metastore.metrics.enabled" throws OOM in HiveMetastore
> ---------------------------------------------------------------------
>
> Key: HIVE-14764
> URL: https://issues.apache.org/jira/browse/HIVE-14764
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Reporter: Rajesh Balamohan
> Assignee: Carl Steinbach
> Priority: Minor
> Labels: JMX, Metrics, Monitoring
> Fix For: 2.2.0, 2.1.1
>
> Attachments: hd_1.png, hd_2.png, HIVE-14764.1.patch
>
>
> After running some queries with metrics enabled, metastore starts throwing the following messages.
> {noformat}
> Caused by: java.sql.SQLException: java.lang.OutOfMemoryError: GC overhead limit exceeded
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1075)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:984)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:929)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:433)
> at com.mysql.jdbc.PreparedStatement.getInstance(PreparedStatement.java:877)
> at com.mysql.jdbc.ConnectionImpl.clientPrepareStatement(ConnectionImpl.java:1489)
> at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4343)
> at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4242)
> at com.jolbox.bonecp.ConnectionHandle.prepareStatement(ConnectionHandle.java:1024)
> at org.datanucleus.store.rdbms.SQLController.getStatementForQuery(SQLController.java:350)
> at org.datanucleus.store.rdbms.SQLController.getStatementForQuery(SQLController.java:295)
> at org.datanucleus.store.rdbms.scostore.JoinListStore.listIterator(JoinListStore.java:761)
> ... 36 more
> Nested Throwables StackTrace:
> java.sql.SQLException: java.lang.OutOfMemoryError: GC overhead limit exceeded
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1075)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:984)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:929)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:433)
> at com.mysql.jdbc.PreparedStatement.getInstance(PreparedStatement.java:877)
> at com.mysql.jdbc.ConnectionImpl.clientPrepareStatement(ConnectionImpl.java:1489)
> at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4343)
> at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4242)
> at com.jolbox.bonecp.ConnectionHandle.prepareStatement(ConnectionHandle.java:1024)
> at org.datanucleus.store.rdbms.SQLController.getStatementForQuery(SQLController.java:350)
> at org.datanucleus.store.rdbms.SQLController.getStatementForQuery(SQLController.java:295)
> at org.datanucleus.store.rdbms.scostore.JoinListStore.listIterator(JoinListStore.java:761)
> at org.datanucleus.store.rdbms.scostore.AbstractListStore.listIterator(AbstractListStore.java:93)
> at org.datanucleus.store.rdbms.scostore.AbstractListStore.iterator(AbstractListStore.java:83)
> at org.datanucleus.store.types.wrappers.backed.List.loadFromStore(List.java:264)
> at org.datanucleus.store.types.wrappers.backed.List.iterator(List.java:492)
> at org.apache.hadoop.hive.metastore.ObjectStore.convertToFieldSchemas(ObjectStore.java:1199)
> at org.apache.hadoop.hive.metastore.ObjectStore.convertToStorageDescriptor(ObjectStore.java:1266)
> at org.apache.hadoop.hive.metastore.ObjectStore.convertToStorageDescriptor(ObjectStore.java:1281)
> at org.apache.hadoop.hive.metastore.ObjectStore.convertToTable(ObjectStore.java:1138)
> at org.apache.hadoop.hive.metastore.ObjectStore.ensureGetTable(ObjectStore.java:2651)
> at org.apache.hadoop.hive.metastore.ObjectStore.updatePartitionColumnStatistics(ObjectStore.java:6141)
> {noformat}
> HiveMetastore uses start/end functions for starting/ending the scope in MetricsFactory. In some places in HiveMetastore the function names are not matching causing gradual memory leak in metastore with metrics enabled.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)