You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Andrey Zagrebin (Jira)" <ji...@apache.org> on 2020/11/16 09:32:00 UTC

[jira] [Commented] (FLINK-20118) Test New File Source API - Continuous Streaming Execution

    [ https://issues.apache.org/jira/browse/FLINK-20118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17232648#comment-17232648 ] 

Andrey Zagrebin commented on FLINK-20118:
-----------------------------------------

I suggest to write an e2e test which generates random files and let file stream source to consume them.

The output can be saved with e.g. a streaming sink w/o any transformation into output files. The output can be verified against the input checking that the order of lines for each input file is preserved in the output files, e.g. after each new input file is generated and consumed. TM failures can be introduced in between.

[~sewen] wdyt?

> Test New File Source API - Continuous Streaming Execution
> ---------------------------------------------------------
>
>                 Key: FLINK-20118
>                 URL: https://issues.apache.org/jira/browse/FLINK-20118
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Connectors / Common
>    Affects Versions: 1.12.0
>            Reporter: Robert Metzger
>            Assignee: Andrey Zagrebin
>            Priority: Critical
>             Fix For: 1.12.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)