[jira] [Created] (IGNITE-20846) Expose cancel flag in conventional APIs for shutting down nodes

["Tiago Marques Godinho (Jira)" <ji...@apache.org>]

Tiago Marques Godinho created IGNITE-20846:
----------------------------------------------

             Summary: Expose cancel flag in conventional APIs for shutting down nodes
                 Key: IGNITE-20846
                 URL: https://issues.apache.org/jira/browse/IGNITE-20846
             Project: Ignite
          Issue Type: Wish
          Components: general
    Affects Versions: 2.15
            Reporter: Tiago Marques Godinho


*Context:*

My ultimate goal is to shutdown a node gracefully. I would like that any necessary maintenance task is performed when during node shutdown as opposed to when the node starts. I'm particularly interested executing the checkpoint operation.

Currently, the GracefullShutdown Policy will not trigger a checkpoint in the node that's about to be shutdown.
The shutdown process is controlled by the [IgnitionEx.IgniteNamedInstance#stop0|https://github.com/apache/ignite/blob/2380ff35d69dc09000886c9df64dc76352022e70/modules/core/src/main/java/org/apache/ignite/internal/IgnitionEx.java#L2152] method. if we are using the GracefulShutdown policy, the shutting down node might force a checkpoint on other nodes, using the [CheckCpHistTask,|https://github.com/apache/ignite/blob/2380ff35d69dc09000886c9df64dc76352022e70/modules/core/src/main/java/org/apache/ignite/internal/IgnitionEx.java#L2265-L2285] among other tasks. However, it does not force a checkpoint in the current node.

Therefore, even with the GracefulShutdown policy selected, the node will do a checkpoint based on the cancel flag.

The cancel flag trickles down to the [Checkpointer#shutdownCheckpointer|https://github.com/apache/ignite/blob/2380ff35d69dc09000886c9df64dc76352022e70/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/checkpoint/Checkpointer.java#L908] method.
When the cancel flag is:
 * enabled - the checkpointer using the shutdowNow path which will skip any previously scheduled checkpoint that's about to run. No checkpoint will run before the node shutdown.
 * disabled - the checkpointer will run at least one more time before the shutdown, provided some other process does not interrupt the thread. The shutdown process will also wait for the checkpointer to be fully stopped.

It's important to mention that the default shutdown hook will set the cancel flag to true on {_}SIGINT{_}/{_}SIGTERM{_} and must be overriden programatically.
Moreover, The [IgniteKillTask|https://github.com/apache/ignite/blob/2380ff35d69dc09000886c9df64dc76352022e70/modules/core/src/main/java/org/apache/ignite/internal/cluster/IgniteKillTask.java#L37], used by the Java client, will also set cancel to true with no possibility of overriding.

*Request:*

Could we expose the cancel flag in the Ignite Java API and/or make it configurable?

*Notes:*
The GracefulShutdown process seems to wait:
 * Until there's at least one additional copy of the cache group in the cluster.
 * Until a re-balance process finishes.
 * There's an update in the topology.

My understanding is that this is not enough to ensure that the node being shutdown finishes all the necessary processes, including any due checkpoint.

Moreover, when shutting down a whole cluster simultaneously, the graceful shutdown often blocks indefinitely because there [are no available copies of some of the partitions|https://github.com/apache/ignite/blob/2380ff35d69dc09000886c9df64dc76352022e70/modules/core/src/main/java/org/apache/ignite/internal/IgnitionEx.java#L2351]. It should probably have some sort of timeout. Probably this should be reported separately.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)