[jira] [Assigned] (HIVE-17586) Make HS2 BackgroundOperationPool not fixed

["Xuefu Zhang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-17586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xuefu Zhang reassigned HIVE-17586:
----------------------------------


> Make HS2 BackgroundOperationPool not fixed
> ------------------------------------------
>
>                 Key: HIVE-17586
>                 URL: https://issues.apache.org/jira/browse/HIVE-17586
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>    Affects Versions: 1.1.0
>            Reporter: Xuefu Zhang
>            Assignee: Xuefu Zhang
>
> Currently the threadpool for background asynchronous operatons has a fixed size controled by {{hive.server2.async.exec.threads}}. However, the thread factory supplied for this threadpool is {{ThreadFactoryWithGarbageCleanup}} which creates ThreadWithGarbageCleanup. Since this is a fixed threadpool, the thread is actually never killed, defecting the purpose of garbage cleanup as noted in the thread class name. On the other hand, since these threads never go away, significant resources such as threadlocal variables (classloaders, hiveconfs, etc) are holding up even if there is no operation running. This can lead to escalated HS2 memory usage.
> Ideally, the threadpool should not be fixed, allowing thread to die out so resources can be reclaimed. The existing config {{hive.server2.async.exec.threads}} is treated as the max, and we can add a min for the threadpool {{hive.server2.async.exec.min.threads}}. Default value for this configure is -1, which keeps the existing behavior.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)