[jira] [Created] (SPARK-23275) hive/tests have been failing when run locally on the laptop (Mac) with OOM

["Dilip Biswal (JIRA)" <ji...@apache.org>]

Dilip Biswal created SPARK-23275:
------------------------------------

             Summary: hive/tests have been failing when run locally on the laptop (Mac) with OOM 
                 Key: SPARK-23275
                 URL: https://issues.apache.org/jira/browse/SPARK-23275
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.3.1
            Reporter: Dilip Biswal


hive tests have been failing when they are run locally (Mac Os)  after a recent change in the trunk. After running the tests for some time, the test fails with OOM with  Error: unable to create new native thread. 

I noticed the thread count goes all the way up to 2000+ after which we start getting these OOM errors. Most of the threads seem to be related to the connection pool in hive metastore (BoneCP-xxxxx-xxxx ). This behaviour change is happening after we made the following change to HiveClientImpl.reset()

{code}
 def reset(): Unit = withHiveState {
    try {
      // code
    } finally {
      runSqlHive("USE default")  ===> this is causing the issue
    }
{code}


I am proposing to temporarily back-out part of a fix made to address SPARK-23000 to resolve this issue while we work-out the exact reason for this sudden increase in thread counts.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org