You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Shohei Okumiya (Jira)" <ji...@apache.org> on 2025/04/14 07:09:00 UTC

[jira] [Resolved] (HIVE-27414) HiveServer2 is not shut down properly in OOM situations when HoS is used

     [ https://issues.apache.org/jira/browse/HIVE-27414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shohei Okumiya resolved HIVE-27414.
-----------------------------------
    Fix Version/s: NA
       Resolution: Won't Fix

We have discontinued Hive on Spark and EoLed Hive 3. HIVE-26134

> HiveServer2 is not shut down properly in OOM situations when HoS is used
> ------------------------------------------------------------------------
>
>                 Key: HIVE-27414
>                 URL: https://issues.apache.org/jira/browse/HIVE-27414
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>    Affects Versions: 1.2.0, 2.0.0, 2.1.1, 2.2.0, 2.3.0, 3.1.0, 2.4.0, 3.0.0
>            Reporter: Miklos Szurap
>            Priority: Minor
>              Labels: OOM, hiveonspark, oom
>             Fix For: NA
>
>
> When an OOM happens, the HiveServer2.stop() is called from the oomHook. That shuts down the HS2, however if the default execution engine is not "spark" but the HiveOnSpark execution engine has been used by any session before, then the check in the stop() is not properly evaluated and hence it does not shut down the SparkSessionManagerImpl properly:
> {code}
> if (hiveConf != null && hiveConf.getVar(ConfVars.HIVE_EXECUTION_ENGINE).equals("spark")) {
>   try {
>     SparkSessionManagerImpl.getInstance().shutdown();
>   } catch(Exception ex) {
>     LOG.error("Spark session pool manager failed to stop during HiveServer2 shutdown.", ex);
>   }
> }
> {code}
> This leaves back a "user" (non-daemon) thread related to the SparkSessionManagerImpl, so the JVM does not exit, the HS2 is unresponsive to user requests as the CLI service and thrift service has been stopped before.
> Due to HIVE-24411 the OOM is not handled well, so even the {{-XX:OnOutOfMemoryError}} flag is useless and the JVM is not killed.
> The [HiveServer2.stop()|https://github.com/apache/hive/blob/37a4fd3515fd4c54025b2f1e8c443914f33b5652/service/src/java/org/apache/hive/service/server/HiveServer2.java#L939-L945] method should use a check like SparkSessionManagerImpl.isInited(). 
> (this method does not exist - just gave as an example)
> The master branch does not have SparkSessionManagerImpl anymore, this is applicable only to the earlier versions.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)