[jira] [Updated] (FLINK-29311) Unable to use http for Kinesis consumer when testing locally

["Fabian Paul (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-29311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Fabian Paul updated FLINK-29311:
--------------------------------
    Affects Version/s: 1.15.3
                           (was: 1.15.2)

> Unable to use http for Kinesis consumer when testing locally
> ------------------------------------------------------------
>
>                 Key: FLINK-29311
>                 URL: https://issues.apache.org/jira/browse/FLINK-29311
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors / Kinesis, Stateful Functions
>    Affects Versions: 1.15.3
>         Environment: Here is my Kinesis module code
>  
> kind: io.statefun.kinesis.v1/ingress
> spec:
>   id: com.sentiment
>   awsRegion:
>     id: us-east-1
>     type: custom-endpoint
>     customEndpoint: [https://localstack:4566|https://localstack:4566/]
>   awsCredentials:
>     type: basic
>     id: us-east-1
>     accessKeyId: key
>     secretAccessKey: secret
>   startupPosition:
>     type: latest
>   streams:
>     - stream: customer-details
>       valueType: SentimentEvent
>       targets:
>         - sentiment
>   clientConfigProperties:
>     - SocketTimeout: 9999
>     - MaxConnections: 15
> And I am using  the latest apache/statefun Docker image within docker compose
>            Reporter: Nicolas Pocock
>            Priority: Major
>
> I am attempting to run the StateFun docker image locally in a docker-compose stack. I have Kinesis running using Localstack and want to use it as a consumer. 
> The documentation says to use a custom endpoint you can use the following config with the _io.statefun.kinesis.v1/ingress_ module:
> {code:java}
> awsCredentials:
>   type: custom-endpoint
>   endpoint: https://localhost:4567
>   id: us-west-1{code}
> The above errors as the custom endpoint property is on the AWS region instead so it needs to be:
> {code:java}
> awsRegion:
>   id: us-east-1
>   type: custom-endpoint
>   endpoint: http://localstack:4566{code}
> However, I am unable to use HTTP as it complains that:
> {noformat}
> Caused by: java.lang.IllegalArgumentException: Invalid service endpoint url: http://localstack:4566; Only custom service endpoints using HTTPS are supported{noformat}
> I tried using HTTPS but it errors with a certificate issue, I think this is an issue with Localstack.
> Some of the docs for the Kinesis connectors show HTTP being used with a custom endpoint
> [https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/connectors/datastream/kinesis/#using-custom-kinesis-endpoints]
> It would be nice if we can use HTTP when testing locally
> I see from this issue [https://github.com/localstack/localstack/issues/893]  there is a configuration option TRUST_ALL_CERTIFICATES that can be provided to the NettyNioAsyncHttpClient that allows the above to work for local dev.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)