You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Paulo Motta (Jira)" <ji...@apache.org> on 2022/01/05 15:40:00 UTC
[jira] [Updated] (CASSANDRA-16939) Shutdown hook is removed after non-fatal OutOfMemoryError
[ https://issues.apache.org/jira/browse/CASSANDRA-16939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paulo Motta updated CASSANDRA-16939:
------------------------------------
Labels: lhf (was: )
> Shutdown hook is removed after non-fatal OutOfMemoryError
> ---------------------------------------------------------
>
> Key: CASSANDRA-16939
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16939
> Project: Cassandra
> Issue Type: Bug
> Reporter: Paulo Motta
> Priority: Normal
> Labels: lhf
>
> On CASSANDRA-7507 the drain shutdown hook is removed when the process hits an OutOfMemoryError, to avoid trying to do a clean shutdown when the node runs out of heap space.
> On CASSANDRA-13006 cassandra started relying on JVM flags (ExitOnOutOfMemoryError and CrashOnOutOfMemoryError) to stop the node when hitting and OutOfMemoryError.
> However, there are non-fatal {{OutOfMemoryErrors}} such as {{OutOfMemory: unable to create new native thread}} or {{OutOfMemory: map failed}} which [do not cause the process to crash even with the ExitOnOutOfMemoryError or CrashOnOutOfMemoryError flags|https://bugs.openjdk.java.net/browse/JDK-8155004].
> Since the shutdown hook is removed after non-fatal OutOfMemory errors, it's no longer possible to do a clean shutdown (via {{SIGTERM kill}} or {{nodetool stopdaemon}}).
> I believe the intent of CASSANDRA-7507 was to remove the shutdown hook only on fatal OutOfMemoryErrors (such as Heap Space Exhausted), those causing the node to crash. If a node is kept running after an OutOfMemoryError, this should not prevent the node from being cleanly shutdown afterwards.
> We should either make the JVM exit on any OutOfMemory error, or remove the drain shutdown hook only on fatal OutOfMemoryErrors, those that will cause the JVM to crash straight away.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org