You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Steven Zhen Wu (Jira)" <ji...@apache.org> on 2021/02/11 16:45:00 UTC

[jira] [Updated] (FLINK-21364) piggyback finishedSplitIds in RequestSplitEvent

     [ https://issues.apache.org/jira/browse/FLINK-21364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steven Zhen Wu updated FLINK-21364:
-----------------------------------
    Description: 
For some split assignment strategy, the enumerator/assigner needs to track the completed splits to advance watermark for event time alignment or rough ordering. Right now, `RequestSplitEvent` for FLIP-27 source doesn't support pass-along of the `finishedSplitIds` info and hence we have to create our own custom source event type for Iceberg source. 

Here is the proposal of add such optional info to `RequestSplitEvent`.
{code}
public RequestSplitEvent(
    @Nullable String hostName, 
    @Nullable Collection<String> finishedSplitIds)
{code}

  was:
For some split assignment strategy, the enumerator/assigner needs to track the completed splits to advance watermark for event time alignment or rough ordering. Right now, `RequestSplitEvent` for FLIP-27 source doesn't support pass-along of the `finishedSplitIds` info and hence we have to create our own custom source event type for Iceberg source. 

Here is the proposal of add such optional info to `RequestSplitEvent`.
```
public RequestSplitEvent(
    @Nullable String hostName, 
    @Nullable Collection<String> finishedSplitIds)
```


> piggyback finishedSplitIds in RequestSplitEvent
> -----------------------------------------------
>
>                 Key: FLINK-21364
>                 URL: https://issues.apache.org/jira/browse/FLINK-21364
>             Project: Flink
>          Issue Type: Improvement
>          Components: Connectors / Common
>    Affects Versions: 1.12.1
>            Reporter: Steven Zhen Wu
>            Priority: Major
>
> For some split assignment strategy, the enumerator/assigner needs to track the completed splits to advance watermark for event time alignment or rough ordering. Right now, `RequestSplitEvent` for FLIP-27 source doesn't support pass-along of the `finishedSplitIds` info and hence we have to create our own custom source event type for Iceberg source. 
> Here is the proposal of add such optional info to `RequestSplitEvent`.
> {code}
> public RequestSplitEvent(
>     @Nullable String hostName, 
>     @Nullable Collection<String> finishedSplitIds)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)