You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by sivarani <wh...@gmail.com> on 2014/12/09 10:46:01 UTC
spark 1.1.1 Maven dependency
Dear All,
I am using spark streaming, It was working fine when i was using spark1.0.2,
now i repeatedly getting few issue
Like class not found, i am using the same pom.xml with the updated version
for all spark modules
i am using spark-core,streaming, streaming with kafka modules..
Its constantly keeps throwing errors for no commons-configuation,
commons-langs, logging
How to get all the dependencies for running spark streaming.. Is there any
way or we just have to find by trial and error methord?
my pom dependencies
<dependencies>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.0.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.10</artifactId>
<version>1.0.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka_2.10</artifactId>
<version>1.0.2</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.5</version>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.6</version>
</dependency>
Am i missing something here?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-1-1-Maven-dependency-tp20590.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: spark 1.1.1 Maven dependency
Posted by Sean Owen <so...@cloudera.com>.
Are you using the Commons libs in your app? then you need to depend on
them directly, and not rely on them accidentally being provided by
Spark. There is no trial and error; you must declare all the
dependencies you use in your own code.
Otherwise perhaps you are not actually running with the Spark assembly
JAR at runtime but somehow only trying to run against the Spark jar
itself. This has none of the dependencies that Spark needs.
On Tue, Dec 9, 2014 at 3:46 AM, sivarani <wh...@gmail.com> wrote:
> Dear All,
>
> I am using spark streaming, It was working fine when i was using spark1.0.2,
> now i repeatedly getting few issue
>
> Like class not found, i am using the same pom.xml with the updated version
> for all spark modules
> i am using spark-core,streaming, streaming with kafka modules..
>
> Its constantly keeps throwing errors for no commons-configuation,
> commons-langs, logging
>
> How to get all the dependencies for running spark streaming.. Is there any
> way or we just have to find by trial and error methord?
>
> my pom dependencies
>
> <dependencies>
> <dependency>
> <groupId>javax.servlet</groupId>
> <artifactId>servlet-api</artifactId>
> <version>2.5</version>
> </dependency>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-core_2.10</artifactId>
> <version>1.0.2</version>
> </dependency>
>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-streaming_2.10</artifactId>
> <version>1.0.2</version>
> </dependency>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-streaming-kafka_2.10</artifactId>
> <version>1.0.2</version>
> </dependency>
>
> <dependency>
> <groupId>org.slf4j</groupId>
> <artifactId>slf4j-log4j12</artifactId>
> <version>1.7.5</version>
> </dependency>
> <dependency>
> <groupId>commons-logging</groupId>
> <artifactId>commons-logging</artifactId>
> <version>1.1.1</version>
> </dependency>
> <dependency>
> <groupId>commons-configuration</groupId>
> <artifactId>commons-configuration</artifactId>
> <version>1.6</version>
> </dependency>
>
> Am i missing something here?
>
>
>
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-1-1-1-Maven-dependency-tp20590.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org