You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Aljoscha Krettek (Jira)" <ji...@apache.org> on 2020/09/01 12:32:00 UTC

[jira] [Commented] (FLINK-19099) consume kafka message repeat

    [ https://issues.apache.org/jira/browse/FLINK-19099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17188410#comment-17188410 ] 

Aljoscha Krettek commented on FLINK-19099:
------------------------------------------

Did you completely cancel your job and start it fresh? Without starting it from a savepoint or retained checkpoint?

If you want the Flink job to keep the read offset you need to take a savepoint before stopping the job or stop it with a savepoint and then restart the job later from that savepoint.

> consume kafka message repeat
> ----------------------------
>
>                 Key: FLINK-19099
>                 URL: https://issues.apache.org/jira/browse/FLINK-19099
>             Project: Flink
>          Issue Type: Bug
>          Components: API / DataStream, Connectors / Kafka
>    Affects Versions: 1.11.0
>            Reporter: zouwenlong
>            Priority: Major
>         Attachments: image-2020-08-31-19-23-59-038.png, image-2020-08-31-19-25-09-548.png
>
>
> when taksmanager be killed ,my job consume some message , but the offset in not commit,
> then restart it ,my job consume kafka message repeat,  I used checkpoint  and set 5 seconds ,
> I think this is a very common problem，how to solve this problem？



--
This message was sent by Atlassian Jira
(v8.3.4#803005)