You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Danny Cranmer (Jira)" <ji...@apache.org> on 2023/01/17 11:58:00 UTC

[jira] [Commented] (FLINK-30418) Implement synchronous KinesisClient in EFO

    [ https://issues.apache.org/jira/browse/FLINK-30418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17677723#comment-17677723 ] 

Danny Cranmer commented on FLINK-30418:
---------------------------------------

Merged commit [{{70c84cb}}|https://github.com/apache/flink-connector-aws/commit/70c84cb7696151f502d9e6f546ee0f3bb6b327cf] into apache:main 

> Implement synchronous KinesisClient in EFO
> ------------------------------------------
>
>                 Key: FLINK-30418
>                 URL: https://issues.apache.org/jira/browse/FLINK-30418
>             Project: Flink
>          Issue Type: Improvement
>          Components: Connectors / AWS
>            Reporter: Daren Wong
>            Assignee: Daren Wong
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: aws-connector-4.1.0
>
>
> h2. What?
> Use synchronous KinesisClient for consumer registration/de-registration/describe.
> h2.  
> h2. Why?
> KinesisProxyV2 supports the following operations:
>  * subscribeToShard
>  * describeStreamSummary
>  * describeStreamConsumer
>  * registerStreamConsumer
>  * deregisterStreamConsumer
>  
> All the above operations are currently performed via an AsyncHTTPClient. However, only subsribeToShard needs to be asynchronous, the rest are called synchronously.
>  
> In addition, there are known issue with async client as follows:
>  * Async client can cause the following RuntimeException when app parallelism is high.
> {quote}Unable to execute HTTP request: The channel was closed before the protocol could be determined.
> {quote}
>  * It can also cause deadlock situation as described in https://issues.apache.org/jira/browse/FLINK-30304



--
This message was sent by Atlassian Jira
(v8.20.10#820010)