You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Luciano Resende <lu...@gmail.com> on 2016/06/27 23:14:12 UTC
Spark streaming connectors available for testing
The Apache Bahir project is voting a release based on Spark 2.0.0-preview.
https://www.mail-archive.com/dev@bahir.apache.org/msg00085.html
It currently provides the following Apache Spark Streaming connectors:
streaming-akka
streaming-mqtt
streaming-twitter
streaming-zeromq
While we are continuing to work towards a release to support Spark 2.0.0,
we appreciate your help around testing the release and the current Spark
Streaming connectors.
To add the connectors to your scala application, the best way is to build
the source of Bahir with 'mvn clean install' which will make the necessary
dependencies available in your local maven repository and will enable you
to reference the connectors in your application and also submit your
application to a local Spark test environment utilizing --packages.
Build:
mvn clean install
Add repository to your scala application (build.sbt):
resolvers += "Local Maven Repository" at "file://" +
Path.userHome.absolutePath + "/.m2/repository"
Submit your application to a local Spark test environment:
bin/spark-submit --master spark://127.0.0.1:7077 --packages
org.apache.bahir:spark-streaming-akka_2.11:2.0.0-preview --class
org.apache.spark.examples.streaming.akka.ActorWordCount
~/opensource/apache/bahir/streaming-akka-examples/target/scala-2.11/streaming-akka-examples_2.11-1.0.jar
localhost 9999
The Bahir community welcomes questions, comments, bug reports and all your
feedback.
http://bahir.apache.org/community/
Thanks