You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by fmthoma <gi...@git.apache.org> on 2018/06/04 08:20:40 UTC

[GitHub] flink issue #6021: [FLINK-9374] [kinesis] Enable FlinkKinesisProducer Backpr...

Github user fmthoma commented on the issue:

    https://github.com/apache/flink/pull/6021
  
    @tzulitai @bowenli86 I've made some more changes while investigating awslabs/amazon-kinesis-producer#183:
    
    * I've followed your suggestion and used `wait()` instead of `Thread.sleep()`, see `TimeoutLatch`. This allows much smaller queue sizes.
    * The timeout of 500ms is too high, i've lowered it to 100ms.
    * I've added two metrics: A `Gauge` for the `outstandingRecordsCount`, and a `Counter` for the backpressure cycles (i.e. the number of times the check `outstandingRecordsCount <= queueLimit` fails).
    
    I updated the documentation, the recommended queue limit is now much lower.


---