[jira] [Created] (FLINK-16279) Per job Yarn application leak in normal execution mode.

["Wenlong Lyu (Jira)" <ji...@apache.org>]

Wenlong Lyu created FLINK-16279:
-----------------------------------

             Summary: Per job Yarn application leak in normal execution mode.
                 Key: FLINK-16279
                 URL: https://issues.apache.org/jira/browse/FLINK-16279
             Project: Flink
          Issue Type: Bug
          Components: Runtime / Coordination
    Affects Versions: 1.10.0
            Reporter: Wenlong Lyu


I run a job in yarn per job mode using {{env.executeAsync}}, the job failed but the yarn cluster didn't be destroyed.

After some research on the code, I found that:

when running in attached mode, MiniDispatcher will neve set {{shutDownfuture}} before received a request from job client. 
{code}
		if (executionMode == ClusterEntrypoint.ExecutionMode.NORMAL) {
			// terminate the MiniDispatcher once we served the first JobResult successfully
			jobResultFuture.thenAccept((JobResult result) -> {
				ApplicationStatus status = result.getSerializedThrowable().isPresent() ?
						ApplicationStatus.FAILED : ApplicationStatus.SUCCEEDED;

				LOG.debug("Shutting down per-job cluster because someone retrieved the job result.");
				shutDownFuture.complete(status);
			});
		} 
{code}

However, when running in async mode(submit job by env.executeAsync), there may be no request from job client because when a user find that the job is failed from job client, he may never request the result again.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)