You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Qingsheng Ren (Jira)" <ji...@apache.org> on 2020/10/30 08:49:00 UTC

[jira] [Commented] (FLINK-19893) stop consuming in specific offset or timestamp

    [ https://issues.apache.org/jira/browse/FLINK-19893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223497#comment-17223497 ] 

Qingsheng Ren commented on FLINK-19893:
---------------------------------------

I think this feature will be added in the new Kafka source based on FLIP-27. There will be a boundedness option in the new source, which you can specify a stopping offset if the boundedness of the Kafka source is set to BOUNDED. You can find more detail in FLINK-18323.

> stop consuming in specific offset or timestamp
> ----------------------------------------------
>
>                 Key: FLINK-19893
>                 URL: https://issues.apache.org/jira/browse/FLINK-19893
>             Project: Flink
>          Issue Type: Improvement
>          Components: Connectors / Kafka
>            Reporter: zoucao
>            Priority: Major
>
> Hi all, I think something new  for kafka connector should be added.
> When using flink to consume message from MQ, there exists a large number of message in topic, not been consumed, and the production rate is equal to consumption rate，such that we have a high latency. In that condition, two task need to be started, one consumes the latest message, the other consumes the earliest message and stop consuming in the current offset or timestamp. May be we should support kafak connector in stopping consuming in specific offset or timestamp.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)