[jira] [Commented] (FLINK-24294) Resources leak in the StreamTask constructor

["Anton Kalashnikov (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-24294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429903#comment-17429903 ] 

Anton Kalashnikov commented on FLINK-24294:
-------------------------------------------

[~pnowojski] , Not exactly. We had two problems:
 * Possible resource leak if the exception happens after the StreamTask was created.
 * Possible resource leak if the exception happens inside of the constructor of the StreamTask after some resources were allocated

The first one is the more possible problem which has been reported a couple of times already and the fix for it was kind of straightforward. It is exactly what I fixed [here|[https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/taskmanager/Task.java#L914:L946].]

The second one is more tricky to resolve. I wanted to fix it at the same PR but after discussion, we decided to split that task into two because this problem requires a more complex solution and it is not so urgent as the first one. So this is ticket exactly about the problem inside of the constructor.

> Resources leak in the StreamTask constructor
> --------------------------------------------
>
>                 Key: FLINK-24294
>                 URL: https://issues.apache.org/jira/browse/FLINK-24294
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Task
>    Affects Versions: 1.14.0
>            Reporter: Anton Kalashnikov
>            Priority: Major
>
> Since we are initializing a lot of resources in the StreamTask constructor like RecordWriter, timerServices, etc. it is possible that some of these resources remain open if the exception happens below the initialization in the same constructor.
> So in my opinion, we have two choices here: 
> * Avoiding allocation of resources in the constructor which allows us to do something like:
> {noformat}
> StreamTask task = new StreamTask(); //no leaks if it fails
> try { 
>   task.init();
>   ....
> } finally {
>   task.cleanUp();
> }
> {noformat}
> *  or we can rewrite a code in such a way that exception in any constructor(ex. StreamTask) guarantee releasing the earlier allocated resources in this constructor. But it is not so easy to implement(see. initialization of recordWriter in StreamTask constructor)
> So perhaps it makes sense to separate creating object from initialization(allocation resources)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)