[jira] [Reopened] (FLINK-6365) Adapt default values of the Kinesis connector

["Tzu-Li (Gordon) Tai (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-6365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tzu-Li (Gordon) Tai reopened FLINK-6365:
----------------------------------------

> Adapt default values of the Kinesis connector
> ---------------------------------------------
>
>                 Key: FLINK-6365
>                 URL: https://issues.apache.org/jira/browse/FLINK-6365
>             Project: Flink
>          Issue Type: Improvement
>          Components: Kinesis Connector
>    Affects Versions: 1.2.0
>            Reporter: Steffen Hausmann
>            Assignee: Bowen Li
>            Priority: Minor
>             Fix For: 1.4.0
>
>
> As discussed in http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kinesis-connector-SHARD-GETRECORDS-MAX-default-value-td12332.html, it seems reasonable to change the default values of the Kinesis connector to follow KCL’s default settings. I suggest to adapt at least the values for SHARD_GETRECORDS_MAX and SHARD_GETRECORDS_INTERVAL_MILLIS. 
> As a Kinesis shard is currently limited to 5 get operations per second, you can observe high ReadProvisionedThroughputExceeded rates with the current default value for SHARD_GETRECORDS_INTERVAL_MILLIS of 0; it seem reasonable to increase it to 200. As it's described in the email thread, it seems furthermore desirable to increase the default value for SHARD_GETRECORDS_MAX to 10000.
> The values that are used by the KCL can be found here: https://github.com/awslabs/amazon-kinesis-client/blob/master/src/main/java/com/amazonaws/services/kinesis/clientlibrary/lib/worker/KinesisClientLibConfiguration.java
> Thanks for looking into this!
> Steffen



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)