You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Hong Liang Teoh (Jira)" <ji...@apache.org> on 2024/06/06 10:48:00 UTC
[jira] [Updated] (FLINK-35541) Introduce retry limiting for AWS connector sinks
[ https://issues.apache.org/jira/browse/FLINK-35541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Liang Teoh updated FLINK-35541:
------------------------------------
Description:
Currently if the record write operation in the sink consistently fails with retriable error, sinks will retry indefinitely. In case when cause of the error is not resolved this may lead to poison pill.
Proposal here is to add a configurable retry limit for each record. Users can specify a maximum retry per record, and the sink will fail once the retry limit is reached.
We will implement this for all AWS connectors:
* DDBSink
* Firehose Sink
* Kinesis Sink
was:Currently if the record write operation in the sink consistently fails with retriable error, sinks will retry indefinitely. In case when cause of the error is not resolved this may lead to stuck operator.
> Introduce retry limiting for AWS connector sinks
> ------------------------------------------------
>
> Key: FLINK-35541
> URL: https://issues.apache.org/jira/browse/FLINK-35541
> Project: Flink
> Issue Type: Technical Debt
> Components: Connectors / AWS, Connectors / DynamoDB, Connectors / Firehose, Connectors / Kinesis
> Affects Versions: aws-connector-4.2.0
> Reporter: Aleksandr Pilipenko
> Priority: Major
>
> Currently if the record write operation in the sink consistently fails with retriable error, sinks will retry indefinitely. In case when cause of the error is not resolved this may lead to poison pill.
>
> Proposal here is to add a configurable retry limit for each record. Users can specify a maximum retry per record, and the sink will fail once the retry limit is reached.
>
> We will implement this for all AWS connectors:
> * DDBSink
> * Firehose Sink
> * Kinesis Sink
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)