You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Alexey Romanenko (Jira)" <ji...@apache.org> on 2020/04/03 09:34:00 UTC

[jira] [Updated] (BEAM-9476) KinesisIO DescribeStream transient errors are not retried

     [ https://issues.apache.org/jira/browse/BEAM-9476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alexey Romanenko updated BEAM-9476:
-----------------------------------
    Status: Open  (was: Triage Needed)

> KinesisIO DescribeStream transient errors are not retried
> ---------------------------------------------------------
>
>                 Key: BEAM-9476
>                 URL: https://issues.apache.org/jira/browse/BEAM-9476
>             Project: Beam
>          Issue Type: Bug
>          Components: io-java-kinesis
>            Reporter: Aaron Meihm
>            Priority: Major
>          Time Spent: 5h
>  Remaining Estimate: 0h
>
> During Kinesis stream setup {{DescribeStream}} is used in {{listShards}}. This API call has quota limits that can become problematic when attempting to configure multiple Kinesis streams in the same AWS account. AWS currently limits this call by default to 10 times per second for a given account. With multiple streams, this can be hit and results in a {{RuntimeException}} being thrown immediately upon the first error.
> Ideally the rate limited call can be retried a number of times before giving up instead of failing immediately.
> I have a work in progress PR that resolves this issue at https://github.com/apache/beam/pull/10973.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)