You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Dan Dong <do...@gmail.com> on 2015/07/23 22:45:27 UTC
java.lang.NoSuchMethodError for "list.toMap".
Hi,
When I ran with spark-submit the following simple Spark program of:
import org.apache.spark.SparkContext._
import org.apache.spark.SparkConf
import org.apache.spark.rdd.RDD
import org.apache.spark.SparkContext
import org.apache.spark._
import SparkContext._
object TEST2{
def main(args:Array[String])
{
val conf = new SparkConf().setAppName("TEST")
val sc=new SparkContext(conf)
val list=List(("aa",1),("bb",2),("cc",3))
val maps=list.toMap
}
}
I got java.lang.NoSuchMethodError for the line of "val maps=list.toMap".
But in a spark-shell or simply scala, it has no problem:
scala> val list=List(("aa",1),("bb",2),("cc",3))
list: List[(String, Int)] = List((aa,1), (bb,2), (cc,3))
scala> val maps=list.toMap
maps: scala.collection.immutable.Map[String,Int] = Map(aa -> 1, bb -> 2, cc
-> 3)
So to use "toMap" method, what am I missing in spark-submit? I use "sbt
package" to compile the program and without problem. Thanks!
Cheers,
Dan
Re: java.lang.NoSuchMethodError for "list.toMap".
Posted by Dan Dong <do...@gmail.com>.
Hi, Akhil,
Yes, in the build.sbt I wrongly set it to the installed scala version of
2.11.6 on the cluster, fixed now. Thanks!
Cheers,
Dan
2015-07-27 2:29 GMT-05:00 Akhil Das <ak...@sigmoidanalytics.com>:
> Whats in your build.sbt? You could be messing with the scala version it
> seems.
>
> Thanks
> Best Regards
>
> On Fri, Jul 24, 2015 at 2:15 AM, Dan Dong <do...@gmail.com> wrote:
>
>> Hi,
>> When I ran with spark-submit the following simple Spark program of:
>> import org.apache.spark.SparkContext._
>> import org.apache.spark.SparkConf
>> import org.apache.spark.rdd.RDD
>> import org.apache.spark.SparkContext
>> import org.apache.spark._
>> import SparkContext._
>>
>> object TEST2{
>> def main(args:Array[String])
>> {
>> val conf = new SparkConf().setAppName("TEST")
>> val sc=new SparkContext(conf)
>>
>> val list=List(("aa",1),("bb",2),("cc",3))
>> val maps=list.toMap
>> }
>>
>> }
>>
>> I got java.lang.NoSuchMethodError for the line of "val maps=list.toMap".
>> But in a spark-shell or simply scala, it has no problem:
>>
>> scala> val list=List(("aa",1),("bb",2),("cc",3))
>> list: List[(String, Int)] = List((aa,1), (bb,2), (cc,3))
>>
>> scala> val maps=list.toMap
>> maps: scala.collection.immutable.Map[String,Int] = Map(aa -> 1, bb -> 2,
>> cc -> 3)
>>
>> So to use "toMap" method, what am I missing in spark-submit? I use "sbt
>> package" to compile the program and without problem. Thanks!
>>
>> Cheers,
>> Dan
>>
>>
>
Re: java.lang.NoSuchMethodError for "list.toMap".
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Whats in your build.sbt? You could be messing with the scala version it
seems.
Thanks
Best Regards
On Fri, Jul 24, 2015 at 2:15 AM, Dan Dong <do...@gmail.com> wrote:
> Hi,
> When I ran with spark-submit the following simple Spark program of:
> import org.apache.spark.SparkContext._
> import org.apache.spark.SparkConf
> import org.apache.spark.rdd.RDD
> import org.apache.spark.SparkContext
> import org.apache.spark._
> import SparkContext._
>
> object TEST2{
> def main(args:Array[String])
> {
> val conf = new SparkConf().setAppName("TEST")
> val sc=new SparkContext(conf)
>
> val list=List(("aa",1),("bb",2),("cc",3))
> val maps=list.toMap
> }
>
> }
>
> I got java.lang.NoSuchMethodError for the line of "val maps=list.toMap".
> But in a spark-shell or simply scala, it has no problem:
>
> scala> val list=List(("aa",1),("bb",2),("cc",3))
> list: List[(String, Int)] = List((aa,1), (bb,2), (cc,3))
>
> scala> val maps=list.toMap
> maps: scala.collection.immutable.Map[String,Int] = Map(aa -> 1, bb -> 2,
> cc -> 3)
>
> So to use "toMap" method, what am I missing in spark-submit? I use "sbt
> package" to compile the program and without problem. Thanks!
>
> Cheers,
> Dan
>
>