[jira] [Updated] (FLINK-34023) Expose Kinesis client retry config in sink

["Hong Liang Teoh (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-34023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hong Liang Teoh updated FLINK-34023:
------------------------------------
    Fix Version/s: aws-connector-5.1.0

> Expose Kinesis client retry config in sink
> ------------------------------------------
>
>                 Key: FLINK-34023
>                 URL: https://issues.apache.org/jira/browse/FLINK-34023
>             Project: Flink
>          Issue Type: Improvement
>          Components: Connectors / Kinesis
>            Reporter: Brad Atcheson
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: aws-connector-5.1.0
>
>
> The consumer side exposes client retry configuration like [flink.shard.getrecords.maxretries|https://nightlies.apache.org/flink/flink-docs-stable/api/java/org/apache/flink/streaming/connectors/kinesis/config/ConsumerConfigConstants.html#SHARD_GETRECORDS_RETRIES] but the producer side lacks similar config for PutRecords.
> The KinesisStreamsSinkWriter constructor calls 
> {code}
> this.httpClient = AWSGeneralUtil.createAsyncHttpClient(kinesisClientProperties);
> this.kinesisClient = buildClient(kinesisClientProperties, this.httpClient);
> {code}
> But those methods only refer to these values (aside from endpoint/region/creds) in the kinesisClientProperties:
> * aws.http-client.max-concurrency
> * aws.http-client.read-timeout
> * aws.trust.all.certificates
> * aws.http.protocol.version
> Without control over retry, users can observe exceptions like {code}Request attempt 2 failure: Unable to execute HTTP request: connection timed out after 2000 ms: kinesis.us-west-2.amazonaws.com{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)