You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Dagang Wei (Jira)" <ji...@apache.org> on 2020/07/07 20:43:00 UTC
[jira] [Comment Edited] (HIVE-20600) Metastore connection leak
[ https://issues.apache.org/jira/browse/HIVE-20600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153084#comment-17153084 ]
Dagang Wei edited comment on HIVE-20600 at 7/7/20, 8:42 PM:
------------------------------------------------------------
[~nemonlou], I'm looking for the patch for 2.3.3, do you know if the patch attached in this issue solves the problem? [^HIVE-20600.patch]
was (Author: functicons):
nemonlou, I'm looking for the patch for 2.3.3, do you know if the patch attached in this issue solves the problem? https://issues.apache.org/jira/secure/attachment/12940453/HIVE-20600.patch
> Metastore connection leak
> -------------------------
>
> Key: HIVE-20600
> URL: https://issues.apache.org/jira/browse/HIVE-20600
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
> Affects Versions: 2.3.3
> Reporter: Damon Cortesi
> Priority: Major
> Attachments: HIVE-20600.patch, consume_threads.py
>
>
> Within the execute method of HiveServer2, there appears to be a connection leak. With fairly straightforward series of INSERT statements, the connection count in the logs continues to increase over time. Under certain loads, this can also consume all underlying threads of the Hive metastore and result in HS2 becoming unresponsive to new connections.
> The log below is the result of some python code executing a single insert statement, and then looping through a series of 10 more insert statements. We can see there's one dangling connection left open after each execution leaving us with 12 open connections (11 from the execute statements + 1 from HS2 startup).
> {code}
> 2018-09-19T17:14:32,108 INFO [main([])]: hive.metastore (HiveMetaStoreClient.java:open(481)) - Opened a connection to metastore, current connections: 1
> 2018-09-19T17:14:48,175 INFO [29049f74-73c4-4f48-9cf7-b4bfe524a85b HiveServer2-Handler-Pool: Thread-31([])]: hive.metastore (HiveMetaStoreClient.java:open(481)) - Opened a connection to metastore, current connections: 2
> 2018-09-19T17:15:05,543 INFO [HiveServer2-Background-Pool: Thread-36([])]: hive.metastore (HiveMetaStoreClient.java:close(564)) - Closed a connection to metastore, current connections: 1
> 2018-09-19T17:15:05,548 INFO [HiveServer2-Background-Pool: Thread-36([])]: hive.metastore (HiveMetaStoreClient.java:open(481)) - Opened a connection to metastore, current connections: 2
> 2018-09-19T17:15:05,932 INFO [HiveServer2-Background-Pool: Thread-36([])]: hive.metastore (HiveMetaStoreClient.java:close(564)) - Closed a connection to metastore, current connections: 1
> 2018-09-19T17:15:05,935 INFO [HiveServer2-Background-Pool: Thread-36([])]: hive.metastore (HiveMetaStoreClient.java:open(481)) - Opened a connection to metastore, current connections: 2
> 2018-09-19T17:15:06,123 INFO [HiveServer2-Background-Pool: Thread-36([])]: hive.metastore (HiveMetaStoreClient.java:close(564)) - Closed a connection to metastore, current connections: 1
> 2018-09-19T17:15:06,126 INFO [HiveServer2-Background-Pool: Thread-36([])]: hive.metastore (HiveMetaStoreClient.java:open(481)) - Opened a connection to metastore, current connections: 2
> ...
> 2018-09-19T17:15:20,626 INFO [29049f74-73c4-4f48-9cf7-b4bfe524a85b HiveServer2-Handler-Pool: Thread-31([])]: hive.metastore (HiveMetaStoreClient.java:open(481)) - Opened a connection to metastore, current connections: 12
> 2018-09-19T17:15:21,153 INFO [HiveServer2-Background-Pool: Thread-162([])]: hive.metastore (HiveMetaStoreClient.java:close(564)) - Closed a connection to metastore, current connections: 11
> 2018-09-19T17:15:21,155 INFO [HiveServer2-Background-Pool: Thread-162([])]: hive.metastore (HiveMetaStoreClient.java:open(481)) - Opened a connection to metastore, current connections: 12
> 2018-09-19T17:15:21,306 INFO [HiveServer2-Background-Pool: Thread-162([])]: hive.metastore (HiveMetaStoreClient.java:close(564)) - Closed a connection to metastore, current connections: 11
> 2018-09-19T17:15:21,308 INFO [HiveServer2-Background-Pool: Thread-162([])]: hive.metastore (HiveMetaStoreClient.java:open(481)) - Opened a connection to metastore, current connections: 12
> 2018-09-19T17:15:21,385 INFO [HiveServer2-Background-Pool: Thread-162([])]: hive.metastore (HiveMetaStoreClient.java:close(564)) - Closed a connection to metastore, current connections: 11
> 2018-09-19T17:15:21,387 INFO [HiveServer2-Background-Pool: Thread-162([])]: hive.metastore (HiveMetaStoreClient.java:open(481)) - Opened a connection to metastore, current connections: 12
> 2018-09-19T17:15:21,541 INFO [HiveServer2-Handler-Pool: Thread-31([])]: hive.metastore (HiveMetaStoreClient.java:open(481)) - Opened a connection to metastore, current connections: 13
> 2018-09-19T17:15:21,542 INFO [HiveServer2-Handler-Pool: Thread-31([])]: hive.metastore (HiveMetaStoreClient.java:close(564)) - Closed a connection to metastore, current connections: 12
> {code}
> Attached is a simple [impyla|https://github.com/cloudera/impyla] script that triggers the condition.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)