You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by SK <sk...@gmail.com> on 2014/09/20 02:52:56 UTC
Spark Streaming compilation error: algebird not a member of package
com.twitter
Hi,
I am using the latest release Spark 1.1.0. I am trying to build the
streaming examples (under examples/streaming) as a standalone project with
the following streaming.sbt file. When I run sbt assembly, I get an error
stating that object algebird is not a member of package com.twitter. I
tried adding the dependency spark-streaming-algebird, but that was not
recognized. What dependency should I be including for algebird?
import AssemblyKeys._
assemblySettings
name := "spark_stream_examples"
version := "1.0"
scalaVersion := "2.10.4"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "1.1.0" %
"provided"
libraryDependencies += "org.apache.spark" %% "spark-streaming" % "1.1.0" %
"provided"
libraryDependencies += "org.apache.spark" %% "spark-streaming-twitter" %
"1.1.0"
libraryDependencies += "org.apache.spark" %% "spark-streaming-flume" %
"1.1.0"
libraryDependencies += "org.apache.spark" %% "spark-streaming-zeromq" %
"1.1.0"
libraryDependencies += "org.apache.spark" %% "spark-streaming-mqtt" %
"1.1.0"
libraryDependencies += "org.apache.spark" %% "spark-streaming-kafka" %
"1.1.0"
//libraryDependencies += "org.apache.spark" %% "spark-streaming-algebird" %
"1.1.0"
resolvers += "Akka Repository" at "http://repo.akka.io/releases/"
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-compilation-error-algebird-not-a-member-of-package-com-twitter-tp14709.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Spark Streaming compilation error: algebird not a member of
package com.twitter
Posted by Tathagata Das <ta...@gmail.com>.
There is not artifact call spark-streaming-algebird . To use the algebird,
you will have add the following dependency (in maven format)
<dependency>
<groupId>com.twitter</groupId>
<artifactId>algebird-core_${scala.binary.version}</artifactId>
<version>0.1.11</version>
</dependency>
This is what is used in spark/examples/pom.xml
TD
On Sat, Sep 20, 2014 at 6:22 AM, SK <sk...@gmail.com> wrote:
> Hi,
>
> I am using the latest release Spark 1.1.0. I am trying to build the
> streaming examples (under examples/streaming) as a standalone project with
> the following streaming.sbt file. When I run sbt assembly, I get an error
> stating that object algebird is not a member of package com.twitter. I
> tried adding the dependency spark-streaming-algebird, but that was not
> recognized. What dependency should I be including for algebird?
>
> import AssemblyKeys._
>
> assemblySettings
>
> name := "spark_stream_examples"
>
> version := "1.0"
>
> scalaVersion := "2.10.4"
>
>
>
> libraryDependencies += "org.apache.spark" %% "spark-sql" % "1.1.0" %
> "provided"
>
> libraryDependencies += "org.apache.spark" %% "spark-streaming" % "1.1.0" %
> "provided"
>
> libraryDependencies += "org.apache.spark" %% "spark-streaming-twitter" %
> "1.1.0"
>
> libraryDependencies += "org.apache.spark" %% "spark-streaming-flume" %
> "1.1.0"
>
> libraryDependencies += "org.apache.spark" %% "spark-streaming-zeromq" %
> "1.1.0"
>
> libraryDependencies += "org.apache.spark" %% "spark-streaming-mqtt" %
> "1.1.0"
>
> libraryDependencies += "org.apache.spark" %% "spark-streaming-kafka" %
> "1.1.0"
>
> //libraryDependencies += "org.apache.spark" %% "spark-streaming-algebird" %
> "1.1.0"
>
>
>
> resolvers += "Akka Repository" at "http://repo.akka.io/releases/"
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-compilation-error-algebird-not-a-member-of-package-com-twitter-tp14709.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>