You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by VG <vl...@gmail.com> on 2016/07/22 17:31:20 UTC
Error in running JavaALSExample example from spark examples
I am getting the following error
Exception in thread "main" java.lang.NoSuchMethodError:
scala.reflect.api.JavaUniverse.runtimeMirror(Ljava/lang/ClassLoader;)Lscala/reflect/api/JavaMirrors$JavaMirror;
at org.apache.spark.ml.recommendation.ALS.fit(ALS.scala:452)
Any suggestions to resolve this
VG
Re: Error in running JavaALSExample example from spark examples
Posted by Aaron Ilovici <ai...@wayfair.com>.
Your error stems from spark.ml, and in your pom mllib is the only dependency that is 2.10. Is there a reason for this? IE, you tell maven mllib 2.10 is provided at runtime. Is 2.10 on the machine, or is 2.11?
-Aaron
From: VG <vl...@gmail.com>
Date: Friday, July 22, 2016 at 1:49 PM
To: Sean Owen <so...@cloudera.com>
Cc: User <us...@spark.apache.org>
Subject: Re: Error in running JavaALSExample example from spark examples
Sean,
I am only using the maven dependencies for spark in my pom file.
I don't have anything else. I guess maven dependency should resolve to the correct scala version .. isn;t it ? Any ideas.
<dependency><!-- Spark -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.0.0-preview</version>
<scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql_2.11 -->
<dependency><!-- Spark SQL -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.0.0-preview</version>
<scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming_2.11 -->
<dependency><!-- Spark Streaming -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.11</artifactId>
<version>2.0.0-preview</version>
<scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-mllib_2.10 -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.10</artifactId>
<version>2.0.0-preview</version>
<scope>provided</scope>
</dependency>
On Fri, Jul 22, 2016 at 11:16 PM, Sean Owen <so...@cloudera.com>> wrote:
-dev
Looks like you are mismatching the version of Spark you deploy on at
runtime then. Sounds like it was built for Scala 2.10
On Fri, Jul 22, 2016 at 6:43 PM, VG <vl...@gmail.com>> wrote:
> Using 2.0.0-preview using maven
> So all dependencies should be correct I guess
>
> <dependency><!-- Spark -->
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-core_2.11</artifactId>
> <version>2.0.0-preview</version>
> <scope>provided</scope>
> </dependency>
>
> I see in maven dependencies that this brings in
> scala-reflect-2.11.4
> scala-compiler-2.11.0
>
> and so on
>
>
>
> On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici <ai...@wayfair.com>>
> wrote:
>>
>> What version of Spark/Scala are you running?
>>
>>
>>
>> -Aaron
>
>
Re: Error in running JavaALSExample example from spark examples
Posted by VG <vl...@gmail.com>.
Great. thanks a ton for helping out on this Sean.
I somehow messed this up (and was running in loops for last 2 hours )
thanks again
-VG
On Fri, Jul 22, 2016 at 11:28 PM, Sean Owen <so...@cloudera.com> wrote:
> You mark these provided, which is correct. If the version of Scala
> provided at runtime differs, you'll have a problem.
>
> In fact you can also see you mixed Scala versions in your dependencies
> here. MLlib is on 2.10.
>
> On Fri, Jul 22, 2016 at 6:49 PM, VG <vl...@gmail.com> wrote:
> > Sean,
> >
> > I am only using the maven dependencies for spark in my pom file.
> > I don't have anything else. I guess maven dependency should resolve to
> the
> > correct scala version .. isn;t it ? Any ideas.
> >
> > <dependency><!-- Spark -->
> > <groupId>org.apache.spark</groupId>
> > <artifactId>spark-core_2.11</artifactId>
> > <version>2.0.0-preview</version>
> > <scope>provided</scope>
> > </dependency>
> > <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql_2.11
> -->
> > <dependency><!-- Spark SQL -->
> > <groupId>org.apache.spark</groupId>
> > <artifactId>spark-sql_2.11</artifactId>
> > <version>2.0.0-preview</version>
> > <scope>provided</scope>
> > </dependency>
> > <!--
> > https://mvnrepository.com/artifact/org.apache.spark/spark-streaming_2.11
> -->
> > <dependency><!-- Spark Streaming -->
> > <groupId>org.apache.spark</groupId>
> > <artifactId>spark-streaming_2.11</artifactId>
> > <version>2.0.0-preview</version>
> > <scope>provided</scope>
> > </dependency>
> > <!--
> https://mvnrepository.com/artifact/org.apache.spark/spark-mllib_2.10
> > -->
> > <dependency>
> > <groupId>org.apache.spark</groupId>
> > <artifactId>spark-mllib_2.10</artifactId>
> > <version>2.0.0-preview</version>
> > <scope>provided</scope>
> > </dependency>
> >
> >
> >
> > On Fri, Jul 22, 2016 at 11:16 PM, Sean Owen <so...@cloudera.com> wrote:
> >>
> >> -dev
> >> Looks like you are mismatching the version of Spark you deploy on at
> >> runtime then. Sounds like it was built for Scala 2.10
> >>
> >> On Fri, Jul 22, 2016 at 6:43 PM, VG <vl...@gmail.com> wrote:
> >> > Using 2.0.0-preview using maven
> >> > So all dependencies should be correct I guess
> >> >
> >> > <dependency><!-- Spark -->
> >> > <groupId>org.apache.spark</groupId>
> >> > <artifactId>spark-core_2.11</artifactId>
> >> > <version>2.0.0-preview</version>
> >> > <scope>provided</scope>
> >> > </dependency>
> >> >
> >> > I see in maven dependencies that this brings in
> >> > scala-reflect-2.11.4
> >> > scala-compiler-2.11.0
> >> >
> >> > and so on
> >> >
> >> >
> >> >
> >> > On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici <ailovici@wayfair.com
> >
> >> > wrote:
> >> >>
> >> >> What version of Spark/Scala are you running?
> >> >>
> >> >>
> >> >>
> >> >> -Aaron
> >> >
> >> >
> >
> >
>
Re: Error in running JavaALSExample example from spark examples
Posted by Sean Owen <so...@cloudera.com>.
You mark these provided, which is correct. If the version of Scala
provided at runtime differs, you'll have a problem.
In fact you can also see you mixed Scala versions in your dependencies
here. MLlib is on 2.10.
On Fri, Jul 22, 2016 at 6:49 PM, VG <vl...@gmail.com> wrote:
> Sean,
>
> I am only using the maven dependencies for spark in my pom file.
> I don't have anything else. I guess maven dependency should resolve to the
> correct scala version .. isn;t it ? Any ideas.
>
> <dependency><!-- Spark -->
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-core_2.11</artifactId>
> <version>2.0.0-preview</version>
> <scope>provided</scope>
> </dependency>
> <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql_2.11 -->
> <dependency><!-- Spark SQL -->
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-sql_2.11</artifactId>
> <version>2.0.0-preview</version>
> <scope>provided</scope>
> </dependency>
> <!--
> https://mvnrepository.com/artifact/org.apache.spark/spark-streaming_2.11 -->
> <dependency><!-- Spark Streaming -->
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-streaming_2.11</artifactId>
> <version>2.0.0-preview</version>
> <scope>provided</scope>
> </dependency>
> <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-mllib_2.10
> -->
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-mllib_2.10</artifactId>
> <version>2.0.0-preview</version>
> <scope>provided</scope>
> </dependency>
>
>
>
> On Fri, Jul 22, 2016 at 11:16 PM, Sean Owen <so...@cloudera.com> wrote:
>>
>> -dev
>> Looks like you are mismatching the version of Spark you deploy on at
>> runtime then. Sounds like it was built for Scala 2.10
>>
>> On Fri, Jul 22, 2016 at 6:43 PM, VG <vl...@gmail.com> wrote:
>> > Using 2.0.0-preview using maven
>> > So all dependencies should be correct I guess
>> >
>> > <dependency><!-- Spark -->
>> > <groupId>org.apache.spark</groupId>
>> > <artifactId>spark-core_2.11</artifactId>
>> > <version>2.0.0-preview</version>
>> > <scope>provided</scope>
>> > </dependency>
>> >
>> > I see in maven dependencies that this brings in
>> > scala-reflect-2.11.4
>> > scala-compiler-2.11.0
>> >
>> > and so on
>> >
>> >
>> >
>> > On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici <ai...@wayfair.com>
>> > wrote:
>> >>
>> >> What version of Spark/Scala are you running?
>> >>
>> >>
>> >>
>> >> -Aaron
>> >
>> >
>
>
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: Error in running JavaALSExample example from spark examples
Posted by VG <vl...@gmail.com>.
Sean,
I am only using the maven dependencies for spark in my pom file.
I don't have anything else. I guess maven dependency should resolve to the
correct scala version .. isn;t it ? Any ideas.
<dependency><!-- Spark -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.0.0-preview</version>
<scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql_2.11 -->
<dependency><!-- Spark SQL -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.0.0-preview</version>
<scope>provided</scope>
</dependency>
<!--
https://mvnrepository.com/artifact/org.apache.spark/spark-streaming_2.11 -->
<dependency><!-- Spark Streaming -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.11</artifactId>
<version>2.0.0-preview</version>
<scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-mllib_2.10
-->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.10</artifactId>
<version>2.0.0-preview</version>
<scope>provided</scope>
</dependency>
On Fri, Jul 22, 2016 at 11:16 PM, Sean Owen <so...@cloudera.com> wrote:
> -dev
> Looks like you are mismatching the version of Spark you deploy on at
> runtime then. Sounds like it was built for Scala 2.10
>
> On Fri, Jul 22, 2016 at 6:43 PM, VG <vl...@gmail.com> wrote:
> > Using 2.0.0-preview using maven
> > So all dependencies should be correct I guess
> >
> > <dependency><!-- Spark -->
> > <groupId>org.apache.spark</groupId>
> > <artifactId>spark-core_2.11</artifactId>
> > <version>2.0.0-preview</version>
> > <scope>provided</scope>
> > </dependency>
> >
> > I see in maven dependencies that this brings in
> > scala-reflect-2.11.4
> > scala-compiler-2.11.0
> >
> > and so on
> >
> >
> >
> > On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici <ai...@wayfair.com>
> > wrote:
> >>
> >> What version of Spark/Scala are you running?
> >>
> >>
> >>
> >> -Aaron
> >
> >
>
Re: Error in running JavaALSExample example from spark examples
Posted by Sean Owen <so...@cloudera.com>.
-dev
Looks like you are mismatching the version of Spark you deploy on at
runtime then. Sounds like it was built for Scala 2.10
On Fri, Jul 22, 2016 at 6:43 PM, VG <vl...@gmail.com> wrote:
> Using 2.0.0-preview using maven
> So all dependencies should be correct I guess
>
> <dependency><!-- Spark -->
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-core_2.11</artifactId>
> <version>2.0.0-preview</version>
> <scope>provided</scope>
> </dependency>
>
> I see in maven dependencies that this brings in
> scala-reflect-2.11.4
> scala-compiler-2.11.0
>
> and so on
>
>
>
> On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici <ai...@wayfair.com>
> wrote:
>>
>> What version of Spark/Scala are you running?
>>
>>
>>
>> -Aaron
>
>
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: Error in running JavaALSExample example from spark examples
Posted by Inam Ur Rehman <in...@gmail.com>.
Hello guys..i know its irrelevant to this topic but i've been looking
desperately for the solution. I am facing en exception
http://apache-spark-user-list.1001560.n3.nabble.com/how-to-resolve-you-must-build-spark-with-hive-exception-td27390.html
plz help me.. I couldn't find any solution..
On Fri, Jul 22, 2016 at 10:43 PM, VG <vl...@gmail.com> wrote:
> Using 2.0.0-preview using maven
> So all dependencies should be correct I guess
>
> <dependency><!-- Spark -->
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-core_2.11</artifactId>
> <version>2.0.0-preview</version>
> <scope>provided</scope>
> </dependency>
>
> I see in maven dependencies that this brings in
> scala-reflect-2.11.4
> scala-compiler-2.11.0
>
> and so on
>
>
>
> On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici <ai...@wayfair.com>
> wrote:
>
>> What version of Spark/Scala are you running?
>>
>>
>>
>> -Aaron
>>
>
>
Re: Error in running JavaALSExample example from spark examples
Posted by VG <vl...@gmail.com>.
Using 2.0.0-preview using maven
So all dependencies should be correct I guess
<dependency><!-- Spark -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.0.0-preview</version>
<scope>provided</scope>
</dependency>
I see in maven dependencies that this brings in
scala-reflect-2.11.4
scala-compiler-2.11.0
and so on
On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici <ai...@wayfair.com>
wrote:
> What version of Spark/Scala are you running?
>
>
>
> -Aaron
>
Re: Error in running JavaALSExample example from spark examples
Posted by VG <vl...@gmail.com>.
Using 2.0.0-preview using maven
So all dependencies should be correct I guess
<dependency><!-- Spark -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.0.0-preview</version>
<scope>provided</scope>
</dependency>
I see in maven dependencies that this brings in
scala-reflect-2.11.4
scala-compiler-2.11.0
and so on
On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici <ai...@wayfair.com>
wrote:
> What version of Spark/Scala are you running?
>
>
>
> -Aaron
>
Re: Error in running JavaALSExample example from spark examples
Posted by Aaron Ilovici <ai...@wayfair.com>.
What version of Spark/Scala are you running?
-Aaron