[jira] [Commented] (HIVE-16844) Fix Connection leak in ObjectStore when new Conf object is used

["Mithun Radhakrishnan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-16844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16075583#comment-16075583 ] 

Mithun Radhakrishnan commented on HIVE-16844:
---------------------------------------------

Sorry to resurrect this discussion. I was pondering over the solution on HIVE-16908, and wondered whether the solution here is complete. Here's the code to {{ObjectStore::setConf()}}:

{code:java|title=ObjectStore.java}
  @Override
  @SuppressWarnings("nls")
  public void setConf(Configuration conf) {
    // Although an instance of ObjectStore is accessed by one thread, there may
    // be many threads with ObjectStore instances. So the static variables
    // pmf and prop need to be protected with locks.
    pmfPropLock.lock();
    try {
      isInitialized = false;
      hiveConf = conf;
      configureSSL(conf);
      Properties propsFromConf = getDataSourceProps(conf);
      boolean propsChanged = !propsFromConf.equals(prop);

      if (propsChanged) {
        if (pmf != null){
          clearOutPmfClassLoaderCache(pmf);
          // close the underlying connection pool to avoid leaks
          pmf.close();
        }
        pmf = null;
        prop = null;
      }
    ...
  }
{code}

Note that {{pmfPropLock}} is locked before {{pmf.close()}} is called. But this is also the only place where {{pmfPropLock}} is used. So, if another thread is in the middle of accessing {{pmf}}, it is possible that the instance is messed up for that thread.
Before this code change, resetting {{pmf}} would not affect any threads with an outstanding reference.

> Fix Connection leak in ObjectStore when new Conf object is used
> ---------------------------------------------------------------
>
>                 Key: HIVE-16844
>                 URL: https://issues.apache.org/jira/browse/HIVE-16844
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>            Reporter: Sunitha Beeram
>            Assignee: Sunitha Beeram
>             Fix For: 3.0.0
>
>         Attachments: HIVE-16844.1.patch
>
>
> The code path in ObjectStore.java currently leaks BoneCP (or Hikari) connection pools when a new configuration object is passed in. The code needs to ensure that the persistence-factory is closed before it is nullified.
> The relevant code is [here|https://github.com/apache/hive/blob/master/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L290]. Note that pmf is set to null, but the underlying connection pool is not closed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)