You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Jiayi Liao (Jira)" <ji...@apache.org> on 2019/11/20 03:13:00 UTC
[jira] [Issue Comment Deleted] (FLINK-14862) Fuse initalizeState
and open passes in StreamTask#invoke
[ https://issues.apache.org/jira/browse/FLINK-14862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jiayi Liao updated FLINK-14862:
-------------------------------
Comment: was deleted
(was: Should we send records before the operators are open?
The question occurs to me because I've met the same problem when I was doing FLINK-5601 in our inner version of flink. And I choose to send the wartermark in open function at last.
)
> Fuse initalizeState and open passes in StreamTask#invoke
> --------------------------------------------------------
>
> Key: FLINK-14862
> URL: https://issues.apache.org/jira/browse/FLINK-14862
> Project: Flink
> Issue Type: New Feature
> Components: Runtime / Task
> Reporter: Igal Shilman
> Priority: Minor
>
> Currently chained operators lack the ability to send records downstream during initializeState (AbstractStreamOperator#initalizeState), because downstream operators in the chain didn't open yet (AbstractStreamOperator#open).
> The main problem with that is that the rawOperatorState and the rawKeyedStateInputs are only valid during the execution of initializeState, and with large raw states materializing to hand them over to be used at open is not feasible (at least without spilling to disk)
> So, I would propose to fuse the loops here:
> https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java#L435,L436
> Into a single loop that calls initalizeState and immediately open.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)