You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Renjie Liu (JIRA)" <ji...@apache.org> on 2017/01/11 11:12:58 UTC

[jira] [Created] (FLINK-5445) flink kafka consumer should support rate control

Renjie Liu created FLINK-5445:
---------------------------------

             Summary: flink kafka consumer should support rate control
                 Key: FLINK-5445
                 URL: https://issues.apache.org/jira/browse/FLINK-5445
             Project: Flink
          Issue Type: Bug
          Components: Kafka Connector
    Affects Versions: 1.1.3
            Reporter: Renjie Liu
            Priority: Critical


Now kafka connector does not support rate control at all, this may lead checkpoints to fail and expire if downstream processor is slow. Though this will not compromise the guarantee of flink's checkpoint, this will stop kafka consumer from comitting offsets since kafka consumer commits offsets in the callback of checkpoint complete. The committed offsets is important for monitoring purpose. So I think we need to support rate limit control in kafka consumer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)