[jira] [Commented] (FLINK-7673) Flink Kinesis connector - remove dependency on the ASL code

["Robert Metzger (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-7673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16177629#comment-16177629 ] 

Robert Metzger commented on FLINK-7673:
---------------------------------------

Thank you for looking into this! It is indeed annoying that we can not provide the dependency on maven central.

This page explains how to contrib: http://flink.apache.org/how-to-contribute.html

Minimum functionality: I fear that we would expect the same functionality as the current implementation. I could also imagine having the ASL-free implementation available next to the current one (maybe in Apache Bahir in the beginning).

[~tzulitai] has done most of the Kinesis connector implementation. He has also looked into providing a ASL-free implementation. IIRC, the biggest problem was the implementation of how Kinesis is aggregating small messages into one bigger message. We would probably need to reverse-engineer this implementation.


> Flink Kinesis connector - remove dependency on the ASL code
> -----------------------------------------------------------
>
>                 Key: FLINK-7673
>                 URL: https://issues.apache.org/jira/browse/FLINK-7673
>             Project: Flink
>          Issue Type: Improvement
>          Components: Kinesis Connector
>            Reporter: Radoslaw Gruchalski
>
> Currently the Flink Kinesis connector depends on two artifacts which are available under Amazon Software License code. The artifacts are:
> {noformat}
>                <dependency>
>                        <groupId>com.amazonaws</groupId>
>                        <artifactId>amazon-kinesis-producer</artifactId>
>                        <version>${aws.kinesis-kpl.version}</version>
>                </dependency>
>                <dependency>
>                        <groupId>com.amazonaws</groupId>
>                        <artifactId>amazon-kinesis-client</artifactId>
>                        <version>${aws.kinesis-kcl.version}</version>
>                        <!--
>                                We're excluding the below from the KCL since we'll only be using the
>                                com.amazonaws.services.kinesis.clientlibrary.types.UserRecord class, which will not need these dependencies.
>                        -->
>                        <exclusions>
>                                <exclusion>
>                                        <groupId>com.amazonaws</groupId>
>                                        <artifactId>aws-java-sdk-dynamodb</artifactId>
>                                </exclusion>
>                                <exclusion>
>                                        <groupId>com.amazonaws</groupId>
>                                        <artifactId>aws-java-sdk-cloudwatch</artifactId>
>                                </exclusion>
>                        </exclusions>
>                </dependency>
> {noformat}
> This prevents the connector being published to Maven Central. I would like to contribute to Flink Kinesis connector by replacing the ASL licensed code with an implementation using AWS SDK for Java. For that, I'd like to understand two things:
> - what's the correct process for making the contribution (contributor agreement, etc.)
> - what's the minimum functionality the alternative implementation would have to provide in order to get accepted
> There are three classes in the connector's source requiring modification:
> - org.apache.flink.streaming.connectors.kinesis.FlinkKinesisProducer
> - org.apache.flink.streaming.connectors.kinesis.util.KinesisConfigUtil
> - org.apache.flink.streaming.connectors.kinesis.internals.ShardConsumer
> Thank you.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)