You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Luciano Resende <lu...@gmail.com> on 2016/06/27 23:14:12 UTC

Spark streaming connectors available for testing

The Apache Bahir project is voting a release based on Spark 2.0.0-preview.
https://www.mail-archive.com/dev@bahir.apache.org/msg00085.html

It currently provides the following Apache Spark Streaming connectors:

    streaming-akka
    streaming-mqtt
    streaming-twitter
    streaming-zeromq

While we are continuing to work towards a release to support Spark 2.0.0,
we appreciate your help around testing the release and the current Spark
Streaming connectors.

To add the connectors to your scala application, the best way is to build
the source of Bahir with 'mvn clean install' which will make the necessary
dependencies available in your local maven repository and will enable you
to reference the connectors in your application and also submit  your
application to a local Spark test environment utilizing --packages.

Build:
mvn clean install

Add repository to your scala application (build.sbt):
resolvers += "Local Maven Repository" at "file://" +
Path.userHome.absolutePath + "/.m2/repository"

Submit your application to a local Spark test environment:
bin/spark-submit --master spark://127.0.0.1:7077 --packages
org.apache.bahir:spark-streaming-akka_2.11:2.0.0-preview --class
org.apache.spark.examples.streaming.akka.ActorWordCount
~/opensource/apache/bahir/streaming-akka-examples/target/scala-2.11/streaming-akka-examples_2.11-1.0.jar
localhost 9999


The Bahir community welcomes questions, comments, bug reports and all your
feedback.

http://bahir.apache.org/community/

Thanks