You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Georgios Kaklamanos <ge...@gwdg.de> on 2017/11/26 01:31:14 UTC
Status of Kafka011JsonTableSink for 1.4.0 release?
Hello,
I'm new to Flink and I'd like to use it along with Kafka 0.11, to do
some processing on a stream of JSON data.
I've been looking the docs of the Table API (and the relevant GitHub
code), and I see that recently there was a Kafka011JsonTableSource
submitted, which will be available in the Flink 1.4 release.
Would there also be a Kafka011JsonTableSink in the near future (1.4
release), or I'd have to fall back to Kafka 0.10 for now?
Thank you for your time.
Best Regards,
George
--
------------------------------------------------------------------
Georgios Kaklamanos
Research Assistant, e-Science Group, GWDG
mailto: georgios.kaklamanos@gwdg.de
Telefon: 0551 201-26803
------------------------------------------------------------------
GWDG - Gesellschaft für wissenschaftliche
Datenverarbeitung mbH Göttingen
Am Faßberg 11, 37077 Göttingen, Germany
WWW: www.gwdg.de mailto: gwdg@gwdg.de
Phone: +49 (0) 551 201-1510
Fax: +49 (0) 551 201-2150
------------------------------------------------------------------
Geschäftsführer: Prof. Dr. Ramin Yahyapour
Aufsichtsratsvorsitzender: Prof. Dr. Christian Griesinger
Sitz der Gesellschaft: Göttingen
Registergericht: Göttingen
Handelsregister-Nr. B 598
------------------------------------------------------------------
Zertifiziert nach ISO 9001
------------------------------------------------------------------
Re: Status of Kafka011JsonTableSink for 1.4.0 release?
Posted by Georgios Kaklamanos <ge...@gwdg.de>.
Hi Fabian,
Thanks for the answer.
I had seen the Kafka Producer but, from a quick look, I didn't seem to
find something like a JSON Serialization Schema, which I need since the
next app in my pipeline, expects to read the data in JSON.
So hoping for a TableJSONSink, I didn't look more into it. I'll check it
now.
Best,
George
On 11/27/2017 10:53 AM, Fabian Hueske wrote:
> Hi George,
>
> Flink 1.4 will not include a KafkaTableSink for Kafka 0.11 but a
> DataStream API SinkFunction (KafkaProducer).
> As an alternative to usingthe Kafka010TableSink, you can also convert
> the result Table into a DataStream and use the KafkaProducer for Kafka
> 0.11 to emit the DataStream.
>
> We will hopefully add Kafka 0.11 TableSink with Flink 1.5 which is
> scheduled for early next year.
>
> Best, Fabian
>
> 2017-11-26 2:31 GMT+01:00 Georgios Kaklamanos
> <georgios.kaklamanos@gwdg.de <ma...@gwdg.de>>:
>
> Hello,
>
> I'm new to Flink and I'd like to use it along with Kafka 0.11, to do
> some processing on a stream of JSON data.
>
> I've been looking the docs of the Table API (and the relevant GitHub
> code), and I see that recently there was a Kafka011JsonTableSource
> submitted, which will be available in the Flink 1.4 release.
>
> Would there also be a Kafka011JsonTableSink in the near future (1.4
> release), or I'd have to fall back to Kafka 0.10 for now?
>
> Thank you for your time.
>
> Best Regards,
> George
>
>
>
> --
> ------------------------------------------------------------------
> Georgios Kaklamanos
> Research Assistant, e-Science Group, GWDG
> mailto: georgios.kaklamanos@gwdg.de <ma...@gwdg.de>
> Telefon: 0551 201-26803
> ------------------------------------------------------------------
> GWDG - Gesellschaft für wissenschaftliche
> Datenverarbeitung mbH Göttingen
> Am Faßberg 11, 37077 Göttingen, Germany
>
> WWW: www.gwdg.de <http://www.gwdg.de> mailto: gwdg@gwdg.de
> <ma...@gwdg.de>
> Phone: +49 (0) 551 201-1510 <tel:%2B49%20%280%29%20551%20201-1510>
> Fax: +49 (0) 551 201-2150 <tel:%2B49%20%280%29%20551%20201-2150>
> ------------------------------------------------------------------
> Geschäftsführer: Prof. Dr. Ramin Yahyapour
> Aufsichtsratsvorsitzender: Prof. Dr. Christian Griesinger
> Sitz der Gesellschaft: Göttingen
> Registergericht: Göttingen
> Handelsregister-Nr. B 598
> ------------------------------------------------------------------
> Zertifiziert nach ISO 9001
> ------------------------------------------------------------------
>
>
>
--
------------------------------------------------------------------
Georgios Kaklamanos
Research Assistant, e-Science Group, GWDG
mailto: georgios.kaklamanos@gwdg.de
Telefon: 0551 201-26803
------------------------------------------------------------------
GWDG - Gesellschaft für wissenschaftliche
Datenverarbeitung mbH Göttingen
Am Faßberg 11, 37077 Göttingen, Germany
WWW: www.gwdg.de mailto: gwdg@gwdg.de
Phone: +49 (0) 551 201-1510
Fax: +49 (0) 551 201-2150
------------------------------------------------------------------
Geschäftsführer: Prof. Dr. Ramin Yahyapour
Aufsichtsratsvorsitzender: Prof. Dr. Christian Griesinger
Sitz der Gesellschaft: Göttingen
Registergericht: Göttingen
Handelsregister-Nr. B 598
------------------------------------------------------------------
Zertifiziert nach ISO 9001
------------------------------------------------------------------
Re: Status of Kafka011JsonTableSink for 1.4.0 release?
Posted by Fabian Hueske <fh...@gmail.com>.
Hi George,
Flink 1.4 will not include a KafkaTableSink for Kafka 0.11 but a DataStream
API SinkFunction (KafkaProducer).
As an alternative to usingthe Kafka010TableSink, you can also convert the
result Table into a DataStream and use the KafkaProducer for Kafka 0.11 to
emit the DataStream.
We will hopefully add Kafka 0.11 TableSink with Flink 1.5 which is
scheduled for early next year.
Best, Fabian
2017-11-26 2:31 GMT+01:00 Georgios Kaklamanos <ge...@gwdg.de>:
> Hello,
>
> I'm new to Flink and I'd like to use it along with Kafka 0.11, to do
> some processing on a stream of JSON data.
>
> I've been looking the docs of the Table API (and the relevant GitHub
> code), and I see that recently there was a Kafka011JsonTableSource
> submitted, which will be available in the Flink 1.4 release.
>
> Would there also be a Kafka011JsonTableSink in the near future (1.4
> release), or I'd have to fall back to Kafka 0.10 for now?
>
> Thank you for your time.
>
> Best Regards,
> George
>
>
>
> --
> ------------------------------------------------------------------
> Georgios Kaklamanos
> Research Assistant, e-Science Group, GWDG
> mailto: georgios.kaklamanos@gwdg.de
> Telefon: 0551 201-26803
> ------------------------------------------------------------------
> GWDG - Gesellschaft für wissenschaftliche
> Datenverarbeitung mbH Göttingen
> Am Faßberg 11, 37077 Göttingen, Germany
>
> WWW: www.gwdg.de mailto: gwdg@gwdg.de
> Phone: +49 (0) 551 201-1510
> Fax: +49 (0) 551 201-2150
> ------------------------------------------------------------------
> Geschäftsführer: Prof. Dr. Ramin Yahyapour
> Aufsichtsratsvorsitzender: Prof. Dr. Christian Griesinger
> Sitz der Gesellschaft: Göttingen
> Registergericht: Göttingen
> Handelsregister-Nr. B 598
> ------------------------------------------------------------------
> Zertifiziert nach ISO 9001
> ------------------------------------------------------------------
>
>
>