[jira] [Commented] (FLINK-21467) Document possible recommended usage of Bounded{One/Multi}Input.endInput and emphasize that they could be called multiple times

["Piotr Nowojski (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-21467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17289946#comment-17289946 ] 

Piotr Nowojski commented on FLINK-21467:
----------------------------------------

Yes, I agree {{Bounded{One/Multi}Input}} interfaces should be documented. Note, that it's not only that {{endOfInput}} can be called multiple times. There is a very remote possibility that the following scenario will happen:
1. checkpoint is taken (successfully)
2. sources are finishing
3. endOfInput is issued
4. job fails
5. job restarts to checkpoint 1.
6. after failover, because of some non deterministic logic in the source, sources are not finishing

So the {{endOfInput()}} from 3., will not be repeated, and it should have been discarded. Two phase commit support from FLIP-147 should provide a way how to deal with this kind of issues.

However there is a disagreement whether the operators should be our official API or not in the first place. (I think they should and currently operator API is `@PublicEvolving`)

> Document possible recommended usage of Bounded{One/Multi}Input.endInput and emphasize that they could be called multiple times
> ------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-21467
>                 URL: https://issues.apache.org/jira/browse/FLINK-21467
>             Project: Flink
>          Issue Type: Improvement
>          Components: API / DataStream
>    Affects Versions: 1.13.0
>            Reporter: Kezhu Wang
>            Priority: Major
>
> It is too tempting to use these api, especially {{BoundedOneInput.endInput}}, to commit final result before FLIP-147 delivered. And this will cause re-commit after failover as [~gaoyunhaii] has pointed out in FLINK-21132.
> I have [pointed|https://github.com/apache/iceberg/issues/2033#issuecomment-784153620] this out in [apache/iceberg#2033|https://github.com/apache/iceberg/issues/2033], please correct me if I was wrong.
> cc [~aljoscha] [~pnowojski] [~roman_khachatryan]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)