[jira] [Updated] (FLINK-14709) Allow outputting elements in close method of chained drivers.

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-14709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated FLINK-14709:
-----------------------------------
    Labels: pull-request-available  (was: )

> Allow outputting elements in close method of chained drivers.
> -------------------------------------------------------------
>
>                 Key: FLINK-14709
>                 URL: https://issues.apache.org/jira/browse/FLINK-14709
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Task
>    Affects Versions: 1.7.2, 1.8.1, 1.9.1
>            Reporter: David Moravek
>            Priority: Major
>              Labels: pull-request-available
>
> Currently, BatchTask and DataSourceTask only allow outputting elements in close method of "rich" operators, that they directly execute.
> Task workflow is as follows:
> 1) open "head" driver  (calls "open" method on udf)
> 2) open chained drivers
> 3) run "head" driver
> 4) close "head" driver (calls "close" method on udf)
> 5) close output collector (no elements can be collected after this point)
> 6) close chained drivers
> In order to properly support outputs from close method, we want to switch 6) and 5). We also need to tweak implementation of Reduce / Combine chained drivers, because they dispose sorters in closeTask method (this should be done in the close method).
> This would bring huge performance improvement for Beam users, because we could properly implement bundling on batch (whole partition = single bundle).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)