[jira] [Updated] (FLINK-41005) MiniCluster should clean up partially started services when start() fails

["Dale Lane (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-41005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dale Lane updated FLINK-41005:
------------------------------
    Description: 
If MiniCluster start() fails part way, the catch block calls close() to cleanup, but {{running}} is only set at the end of a successful start, and closeInternal doesn't do anything unless {{running}} is true. Anything started before the failure will be left running.

{*}Impact:{*}{*}{*}

prod: processes that exit after a failed start (e.g. the MiniCluster application entrypoint from FLINK-40860) leave behind files in java.io.tmpdir

tests: non-daemon threads for RPC and metrics Pekko actor systems, daemon threads for blob server listener, IOManager, mini-cluster-io executor and timers - all of which stay in the surefire fork, plus the REST port stays bound

*Proposed fix:*

In the catch block of {{{}start(){}}}, run the shutdown sequence regardless of {{running}} (e.g. extract the body of {{closeInternal}} into a method both use), making each step tolerate services that were never created (null fields, empty TaskManager list), and wait for it before rethrowing.

*Testing:*

fail {{start()}} at a few points (before and after TaskManagers / dispatcher components are started) and assert that no services are left running

  was:If MiniCluster start() fails part way, the catch block calls close() to cleanup, but {{running}} is only set at the end of a successful start, and closeInternal doesn't do anything unless {{running}} is true. Anything started before the failure will be left running. 


> MiniCluster should clean up partially started services when start() fails
> -------------------------------------------------------------------------
>
>                 Key: FLINK-41005
>                 URL: https://issues.apache.org/jira/browse/FLINK-41005
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Coordination
>            Reporter: Dale Lane
>            Priority: Minor
>
> If MiniCluster start() fails part way, the catch block calls close() to cleanup, but {{running}} is only set at the end of a successful start, and closeInternal doesn't do anything unless {{running}} is true. Anything started before the failure will be left running.
> {*}Impact:{*}{*}{*}
> prod: processes that exit after a failed start (e.g. the MiniCluster application entrypoint from FLINK-40860) leave behind files in java.io.tmpdir
> tests: non-daemon threads for RPC and metrics Pekko actor systems, daemon threads for blob server listener, IOManager, mini-cluster-io executor and timers - all of which stay in the surefire fork, plus the REST port stays bound
> *Proposed fix:*
> In the catch block of {{{}start(){}}}, run the shutdown sequence regardless of {{running}} (e.g. extract the body of {{closeInternal}} into a method both use), making each step tolerate services that were never created (null fields, empty TaskManager list), and wait for it before rethrowing.
> *Testing:*
> fail {{start()}} at a few points (before and after TaskManagers / dispatcher components are started) and assert that no services are left running



--
This message was sent by Atlassian Jira
(v8.20.10#820010)