[GitHub] [pulsar] sijie commented on issue #4585: [spark streaming] support spark structured streaming

[GitBox <gi...@apache.org>]

sijie commented on issue #4585: [spark streaming] support spark structured streaming 
URL: https://github.com/apache/pulsar/issues/4585#issuecomment-512911758
 
 
   for people who are interested in using spark structure streaming with Pulsar, @yjshen has implemented a decent spark connector here: https://github.com/streamnative/pulsar-spark
   
   The spark connector supports both streaming and batch jobs and write data back to pulsar as well. the implementation is fully integrated with Pulsar schema. You can also use Spark SQL to query the data in pulsar.
   
   He also wrote a blog post about it : https://medium.com/streamnative/apache-pulsar-as-one-storage-455222c59017
   
   We are looking forward to contributing this back to upstream pulsar or spark. thoughts and feedbacks are welcome.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services