Read Avro Data using Spark Streaming

[Divya Narayan <na...@gmail.com>]

Hi,

I produced avro data to kafka topic using schema registry and now I want to
use spark streaming to read that data and do some computation in real time.
Can some one please give a sample code for doing that . I couldn't find any
working code online. I am using spark version 2.2.0 and
spark-streaming-kafka-0-10_2.11.

Thanks
Divya

Re: Read Avro Data using Spark Streaming

[chandan prakash <ch...@gmail.com>]

In case you are using old Spark Streaming and processing Avro data from
kafka, this might be useful:
https://www.linkedin.com/pulse/avro-data-processing-spark-streaming-kafka-chandan-prakash/

Regards,
Chandan

On Sat, Nov 3, 2018 at 9:04 AM Divya Narayan <na...@gmail.com>
wrote:

> Hi,
>
> I produced avro data to kafka topic using schema registry and now I want
> to use spark streaming to read that data and do some computation in real
> time. Can some one please give a sample code for doing that . I couldn't
> find any working code online. I am using spark version 2.2.0 and
> spark-streaming-kafka-0-10_2.11.
>
> Thanks
> Divya
>


-- 
Chandan Prakash

Re: Read Avro Data using Spark Streaming

[Michael Shtelma <ms...@gmail.com>]

Hi,

you can use this project in order to read Avro using Spark Structured
Streaming:
https://github.com/AbsaOSS/ABRiS

Spark 2.4 has also built in support for Avro, so you can use from_avro
function in Spark 2.4.

Best,
Michael


On Sat, Nov 3, 2018 at 4:34 AM Divya Narayan <na...@gmail.com>
wrote:

> Hi,
>
> I produced avro data to kafka topic using schema registry and now I want
> to use spark streaming to read that data and do some computation in real
> time. Can some one please give a sample code for doing that . I couldn't
> find any working code online. I am using spark version 2.2.0 and
> spark-streaming-kafka-0-10_2.11.
>
> Thanks
> Divya
>