You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "jackylau (Jira)" <ji...@apache.org> on 2023/02/10 05:25:00 UTC

[jira] [Created] (FLINK-31006) job is not finished when using pipeline mode to run bounded source like kafka/pulsar

jackylau created FLINK-31006:
--------------------------------

             Summary: job is not finished when using pipeline mode to run bounded source like kafka/pulsar
                 Key: FLINK-31006
                 URL: https://issues.apache.org/jira/browse/FLINK-31006
             Project: Flink
          Issue Type: Bug
          Components: Connectors / Kafka, Connectors / Pulsar
    Affects Versions: 1.17.0
            Reporter: jackylau
             Fix For: 1.17.0
         Attachments: image-2023-02-10-13-20-52-890.png, image-2023-02-10-13-23-38-430.png, image-2023-02-10-13-24-46-929.png

when i do failover works like kill jm/tm when using  pipeline mode to run bounded source like kafka, i found job is not finished, when every partition data has consumed.

 

After dig into code, i found this logical not run when JM recover. the partition infos are not changed. so noMoreNewPartitionSplits is not set to true. then this will not run 

 

!image-2023-02-10-13-23-38-430.png!

 

!image-2023-02-10-13-24-46-929.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)