[jira] [Created] (FLINK-19898) [Kinesis][EFO] Ignore ReadTimeoutException from SubcribeToShard retry policy

["Danny Cranmer (Jira)" <ji...@apache.org>]

Danny Cranmer created FLINK-19898:
-------------------------------------

             Summary: [Kinesis][EFO] Ignore ReadTimeoutException from SubcribeToShard retry policy
                 Key: FLINK-19898
                 URL: https://issues.apache.org/jira/browse/FLINK-19898
             Project: Flink
          Issue Type: Improvement
          Components: Connectors / Kinesis
            Reporter: Danny Cranmer
             Fix For: 1.12.0


*Background* 

The Flink Kinesis EFO consumer has a {{SubscribeToShard}} retry policy which will terminate the job after a given number of subsequent attempt failures. In high backpressure scenarios the Netty HTTP Client throws a {{ReadTimeoutException}} when the consumer takes longer than 30s to process a batch. If this happens (by default) 10 times in a row, the job will terminate. There is no need to terminate in this condition, and the restart results in the job falling further behind.

*Scope*

Exclude the {{ReadTimeoutException}} from the {{SubscribeToShard}} retry policy, such that that connector will gracefully reconnect once the consumer has processed the queued records.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)