You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Shivam Sharma <28...@gmail.com> on 2018/07/10 10:28:17 UTC
Want to write Kafka Sink to SQL Client by Flink-1.5
Hi All,
We want to write Kafka Sink functionality for Flink(1.5) SQL Client. We
have read the code and chalk out a rough plan for implementation.
Any guidance for this implementation will be very helpful.
Thanks
--
Shivam Sharma
Data Engineer @ Goibibo
Indian Institute Of Information Technology, Design and Manufacturing
Jabalpur
Mobile No- (+91) 8882114744
Email:- 28shivamsharma@gmail.com
LinkedIn:-*https://www.linkedin.com/in/28shivamsharma
<https://www.linkedin.com/in/28shivamsharma>*
Re: Want to write Kafka Sink to SQL Client by Flink-1.5
Posted by Shivam Sharma <28...@gmail.com>.
Awesome!!! It will be helpful if you share that PR.
Thanks
Re: Want to write Kafka Sink to SQL Client by Flink-1.5
Posted by Timo Walther <tw...@apache.org>.
Hi Shivam,
a Kafka sink for the SQL Client will be part of Flink 1.6. For this we
need to do provide basic interfaces that sinks can extends as Rong
mentioned (FLINK-8866). In order to support all formats that also
sources support we also working on separating the connector from the
formats [1]. PR for these features are ready and I'm working on
integrating them right now. Once this is done and we have support for
INSERT INTO in SQL Client a Kafka sink implementation is straightforward.
Regards,
Timo
[1] https://issues.apache.org/jira/browse/FLINK-8558
Am 12.07.18 um 02:45 schrieb Rong Rong:
> Hi Shivam,
> Thank you for interested in contributing to Kafka Sink for SQL client.
> Could you share your plan for implementation. I have some questions as
> there might have been some overlap with current implementations.
>
> On a higher level,
> 1. Are you using some type of metadata store to host topic schemas
> (Kafka can essentially be schema-less), it might be great to take a
> look at the TableSource/SinkFactory [1][2]
> 2. There's already a KafkaTableSource and KafkaTableSink available, I
> am assuming you are trying to contribute to the configuration in SQL
> Client to make it easier to interact with a Kafka table?
>
> Thanks,
> Rong
>
> [1]: https://issues.apache.org/jira/browse/FLINK-8839
> [2]: https://issues.apache.org/jira/browse/FLINK-8866
>
> On Tue, Jul 10, 2018 at 3:28 AM Shivam Sharma
> <28shivamsharma@gmail.com <ma...@gmail.com>> wrote:
>
> Hi All,
>
> We want to write Kafka Sink functionality for Flink(1.5) SQL
> Client. We have read the code and chalk out a rough plan for
> implementation.
>
> Any guidance for this implementation will be very helpful.
>
> Thanks
> --
> Shivam Sharma
> Data Engineer @ Goibibo
> Indian Institute Of Information Technology, Design and
> Manufacturing Jabalpur
> Mobile No- (+91) 8882114744
> Email:- 28shivamsharma@gmail.com <ma...@gmail.com>
> LinkedIn:-_https://www.linkedin.com/in/28shivamsharma_
>
Re: Want to write Kafka Sink to SQL Client by Flink-1.5
Posted by Rong Rong <wa...@gmail.com>.
Hi Shivam,
Thank you for interested in contributing to Kafka Sink for SQL client.
Could you share your plan for implementation. I have some questions as
there might have been some overlap with current implementations.
On a higher level,
1. Are you using some type of metadata store to host topic schemas (Kafka
can essentially be schema-less), it might be great to take a look at the
TableSource/SinkFactory [1][2]
2. There's already a KafkaTableSource and KafkaTableSink available, I am
assuming you are trying to contribute to the configuration in SQL Client to
make it easier to interact with a Kafka table?
Thanks,
Rong
[1]: https://issues.apache.org/jira/browse/FLINK-8839
[2]: https://issues.apache.org/jira/browse/FLINK-8866
On Tue, Jul 10, 2018 at 3:28 AM Shivam Sharma <28...@gmail.com>
wrote:
> Hi All,
>
> We want to write Kafka Sink functionality for Flink(1.5) SQL Client. We
> have read the code and chalk out a rough plan for implementation.
>
> Any guidance for this implementation will be very helpful.
>
> Thanks
> --
> Shivam Sharma
> Data Engineer @ Goibibo
> Indian Institute Of Information Technology, Design and Manufacturing
> Jabalpur
> Mobile No- (+91) 8882114744
> Email:- 28shivamsharma@gmail.com
> LinkedIn:-*https://www.linkedin.com/in/28shivamsharma
> <https://www.linkedin.com/in/28shivamsharma>*
>