You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "buptljy (JIRA)" <ji...@apache.org> on 2018/04/18 17:37:00 UTC

[jira] [Updated] (FLINK-9178) Add rate control for kafka source

     [ https://issues.apache.org/jira/browse/FLINK-9178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

buptljy updated FLINK-9178:
---------------------------
    Description: 
When I want to run the flink program from the earliest offset in Kafka, it'll be very easy to cause OOM if there are too much data, because of too many HeapMemorySegment in NetworkBufferPool.

Maybe we should have some settings to control the rate of the receiving data?

  was:
When I want to run the flink program from the earliest offset in Kafka, it'll be very easy to cause OOM because of too many HeapMemorySegment in NetworkBufferPool.

I think we should support this "rerun" situation, which is very common for most businesses.


> Add rate control for kafka source
> ---------------------------------
>
>                 Key: FLINK-9178
>                 URL: https://issues.apache.org/jira/browse/FLINK-9178
>             Project: Flink
>          Issue Type: Improvement
>          Components: Kafka Connector
>            Reporter: buptljy
>            Priority: Major
>
> When I want to run the flink program from the earliest offset in Kafka, it'll be very easy to cause OOM if there are too much data, because of too many HeapMemorySegment in NetworkBufferPool.
> Maybe we should have some settings to control the rate of the receiving data?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)