You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "vinoyang (JIRA)" <ji...@apache.org> on 2018/11/23 08:33:00 UTC
[jira] [Commented] (FLINK-9461) Disentangle flink-connector-kafka
from flink-table and flink-json
[ https://issues.apache.org/jira/browse/FLINK-9461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16696503#comment-16696503 ]
vinoyang commented on FLINK-9461:
---------------------------------
Hi all,
The discussion of this issue does not seem to have a conclusion. Not long ago, I just participated in the features of modern Kafka 2.0 and created a new module: flink-connector-kafka. From the history of the discussion, [~fhueske] 's point of view seems to think that the flink table API should be treated peer-to-peer with the flink datastream/dataset API. I prefer the table API to be an upper layer lib. When the user only uses the DataStream API and the Kafka connector, it does not have to depend on the flink-table.
I suggest that for this case, we introduce a parent module and a submodule, such as "flink-table-source-sink/flink-table-source-sink-kafka", but there seems to be a problem here, "flink-table-source-sink" seems to be a sub-module of "flink-libraries". Once this is done, a three-tier structure will be introduced, which will break the convention of the current flink project.
I hope to hear your thoughts. cc [~till.rohrmann] [~Zentol] [~twalthr] and [~pnowojski]
> Disentangle flink-connector-kafka from flink-table and flink-json
> -----------------------------------------------------------------
>
> Key: FLINK-9461
> URL: https://issues.apache.org/jira/browse/FLINK-9461
> Project: Flink
> Issue Type: Sub-task
> Components: Build System
> Affects Versions: 1.5.0
> Reporter: Till Rohrmann
> Assignee: vinoyang
> Priority: Major
> Fix For: 1.8.0
>
>
> Currently, the {{flink-connector-kafka}} module has a dependency on {{flink-table}} and {{flink-json}}. The reason seems to be that the module contains the {{KafkaJsonTableSource}} and {{KafkaJsonTableSink}}. Even though the {{flink-table}} and {{flink-json}} dependency are marked as optional, the {{flink-connector-kafka}} will still contain the table sources and sinks. I think this is not a clean design.
> I would propose to move the table sources and sinks into a dedicated module which depends on {{flink-connector-kafka}}. That way we would better separate dependencies and could remove {{flink-table}} and {{flink-json}} from {{flink-connector-kafka}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)