You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Elliot West <te...@gmail.com> on 2019/01/09 21:03:33 UTC

Producing binary Avro to Kafka

Hello,

What is the recommended flink streaming approach for serialising a POJO to
Avro according to a schema, and pushing the subsequent byte array into a
Kafka sink? Also, is there any existing approach for prepending the schema
id to the payload (following the Confluent pattern)?

Thanks,

Elliot.

Re: Producing binary Avro to Kafka

Posted by Suneel Marthi <sm...@apache.org>.
This was presented at Flink Forward Ber;lin 2017 - see the slide deck here
https://smarthi.github.io/flink-forward-berlin-2017-moving-beyond-moving-bytes/#/19

You should be able to leverage Confluent/Horton schema registries from
flink pipelines.

On Wed, Jan 9, 2019 at 4:14 PM Elliot West <te...@gmail.com> wrote:

> Hello,
>
> What is the recommended flink streaming approach for serialising a POJO to
> Avro according to a schema, and pushing the subsequent byte array into a
> Kafka sink? Also, is there any existing approach for prepending the schema
> id to the payload (following the Confluent pattern)?
>
> Thanks,
>
> Elliot.
>