[jira] [Updated] (FLINK-21578) Closeable Sink Committer/GlobalCommitter were created to function in onestep during job graph composition

["Yun Gao (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-21578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yun Gao updated FLINK-21578:
----------------------------
    Fix Version/s: 1.16.0

> Closeable Sink Committer/GlobalCommitter were created to function in onestep during job graph composition
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-21578
>                 URL: https://issues.apache.org/jira/browse/FLINK-21578
>             Project: Flink
>          Issue Type: Improvement
>          Components: API / DataStream
>    Affects Versions: 1.13.0
>            Reporter: Kezhu Wang
>            Priority: Minor
>              Labels: auto-deprioritized-major
>             Fix For: 1.15.0, 1.16.0
>
>
> Normally, functions/operators are created in job graph composition phase for serialization and transmission. Them are "opened" in flink cluster to function. This two steps procedure succeed in that there will be no resource-cleanup requirement in job graph composition phase.
> While {{Committer}} and {{GlobalCommitter}} has no such "open" operatin but they were created in job graph composition phase.
> Following are fixes I could image if we converge to "this is problematic".
>  # Add {{open}} or similar method for these two classes.
>  # Add {{hasCommitter}}, {{hasGlobalCommitter}} to {{Sink}} and make {{createCommitter}} and others not optional(enforce this in runtime).
> Personally, I am a bit preferring second approach for possible less code path touching in job graph composition phase. But first approach has advantage that it could be an no breaking change.
> There might be other approaches though.
> cc [~guoweima] [~gaoyunhaii]  [~aljoscha]  [~kkl0u]



--
This message was sent by Atlassian Jira
(v8.20.1#820001)