[jira] [Closed] (FLINK-20629) [Kinesis][EFO] Migrate from DescribeStream to DescribeStreamSummary

["Tzu-Li (Gordon) Tai (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-20629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tzu-Li (Gordon) Tai closed FLINK-20629.
---------------------------------------
    Resolution: Fixed

Merged.

flink/master: d900d90d8d8214285bb436607d91275069236c90
flink/release-1.12: e99c613eac6ef2ce7c6e8776e94f961b82e4ab14

> [Kinesis][EFO] Migrate from DescribeStream to DescribeStreamSummary
> -------------------------------------------------------------------
>
>                 Key: FLINK-20629
>                 URL: https://issues.apache.org/jira/browse/FLINK-20629
>             Project: Flink
>          Issue Type: Improvement
>          Components: Connectors / Kinesis
>    Affects Versions: 1.12.0
>            Reporter: Danny Cranmer
>            Assignee: Danny Cranmer
>            Priority: Minor
>              Labels: pull-request-available
>             Fix For: 1.13.0, 1.12.1
>
>
> *Background*
> The Kinesis EFO connector invokes {{DescribeStream}} during startup to acquire the stream ARN. This call also includes the shard information and has a TPS of 10. A similar service exists, {{DescribeStreamSummary}} that has a TPS of 20 and a lighter response payload size.
> During startup sources with high parallelism compete to call this service (in {{LAZY}} mode), resulting in backoff and retry. Essentially the startup time can grow by 1s for every 10 parallelism, due to the 10 TPS. Migrating to {{DescribeStreamSummary}} will improve startup time.
> *Scope*
> Migrate call to {{DescribeStream}} to use {{DescribeStreamSummary}} instead.
> *Note*
> I have targeted {{1.12.1}}, let me know if we should instead target {{1.13}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)