You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by boci <bo...@gmail.com> on 2015/05/24 02:05:08 UTC
Strange ClassNotFound exeption
Hi guys!
I have a small spark application. It's query some data from postgres,
enrich it and write to elasticsearch. When I deployed into spark container
I got a very fustrating error:
https://gist.github.com/b0c1/66527e00bada1e4c0dc3
Spark version: 1.3.1
Hadoop version: 2.6.0
Additional info:
serialization: kryo
rdd: custom rdd to query
I not understand
1. akka.actor.SelectionPath doesn't exists in 1.3.1
2. I checked all dependencies in my project, I only have
"org.spark-project.akka:akka-*_2.10:2.3.4-spark:jar" doesn't have
3. I not found any reference for this...
4. I created own RDD, it's work, but I need to register to kryo? (mapRow
using ResultSet, I need to create
5. I used some months ago and it's already worked with spark 1.2... I
recompiled with 1.3.1 but I got this strange error
Any idea?
----------------------------------------------------------------------------------------------------------------------------------
Skype: boci13, Hangout: boci.boci@gmail.com
Re: Strange ClassNotFound exeption
Posted by Ted Yu <yu...@gmail.com>.
Can you pastebin the class path ?
Thanks
> On May 24, 2015, at 5:02 AM, boci <bo...@gmail.com> wrote:
>
> Yeah, I have same jar with same result, I run in docker container and I using same docker container with my another project... the only difference is the postgresql jdbc driver and the custom RDD... no additional dependencies (both single jar generated with same assembly configuration with same dependencies) and the second is work like a charm
>
> Another idea?
>
> 2015. máj. 24. 2:41 ezt írta ("Ted Yu" <yu...@gmail.com>):
>> In my local maven repo, I found:
>>
>> $ jar tvf /Users/tyu/.m2/repository//org/spark-project/akka/akka-actor_2.10/2.3.4-spark/akka-actor_2.10-2.3.4-spark.jar | grep SelectionPath
>> 521 Mon Sep 29 12:05:36 PDT 2014 akka/actor/SelectionPathElement.class
>>
>> Is the above jar in your classpath ?
>>
>>> On Sat, May 23, 2015 at 5:05 PM, boci <bo...@gmail.com> wrote:
>>> Hi guys!
>>>
>>> I have a small spark application. It's query some data from postgres, enrich it and write to elasticsearch. When I deployed into spark container I got a very fustrating error:
>>> https://gist.github.com/b0c1/66527e00bada1e4c0dc3
>>>
>>> Spark version: 1.3.1
>>> Hadoop version: 2.6.0
>>> Additional info:
>>> serialization: kryo
>>> rdd: custom rdd to query
>>>
>>> I not understand
>>> 1. akka.actor.SelectionPath doesn't exists in 1.3.1
>>> 2. I checked all dependencies in my project, I only have "org.spark-project.akka:akka-*_2.10:2.3.4-spark:jar" doesn't have
>>> 3. I not found any reference for this...
>>> 4. I created own RDD, it's work, but I need to register to kryo? (mapRow using ResultSet, I need to create
>>> 5. I used some months ago and it's already worked with spark 1.2... I recompiled with 1.3.1 but I got this strange error
>>>
>>> Any idea?
>>>
>>> ----------------------------------------------------------------------------------------------------------------------------------
>>> Skype: boci13, Hangout: boci.boci@gmail.com
Re: Strange ClassNotFound exeption
Posted by boci <bo...@gmail.com>.
Yeah, I have same jar with same result, I run in docker container and I
using same docker container with my another project... the only difference
is the postgresql jdbc driver and the custom RDD... no additional
dependencies (both single jar generated with same assembly configuration
with same dependencies) and the second is work like a charm
Another idea?
2015. máj. 24. 2:41 ezt írta ("Ted Yu" <yu...@gmail.com>):
> In my local maven repo, I found:
>
> $ jar tvf
> /Users/tyu/.m2/repository//org/spark-project/akka/akka-actor_2.10/2.3.4-spark/akka-actor_2.10-2.3.4-spark.jar
> | grep SelectionPath
> 521 Mon Sep 29 12:05:36 PDT 2014 akka/actor/SelectionPathElement.class
>
> Is the above jar in your classpath ?
>
> On Sat, May 23, 2015 at 5:05 PM, boci <bo...@gmail.com> wrote:
>
>> Hi guys!
>>
>> I have a small spark application. It's query some data from postgres,
>> enrich it and write to elasticsearch. When I deployed into spark container
>> I got a very fustrating error:
>> https://gist.github.com/b0c1/66527e00bada1e4c0dc3
>>
>> Spark version: 1.3.1
>> Hadoop version: 2.6.0
>> Additional info:
>> serialization: kryo
>> rdd: custom rdd to query
>>
>> I not understand
>> 1. akka.actor.SelectionPath doesn't exists in 1.3.1
>> 2. I checked all dependencies in my project, I only have
>> "org.spark-project.akka:akka-*_2.10:2.3.4-spark:jar" doesn't have
>> 3. I not found any reference for this...
>> 4. I created own RDD, it's work, but I need to register to kryo? (mapRow
>> using ResultSet, I need to create
>> 5. I used some months ago and it's already worked with spark 1.2... I
>> recompiled with 1.3.1 but I got this strange error
>>
>> Any idea?
>>
>>
>> ----------------------------------------------------------------------------------------------------------------------------------
>> Skype: boci13, Hangout: boci.boci@gmail.com
>>
>
>
Re: Strange ClassNotFound exeption
Posted by Ted Yu <yu...@gmail.com>.
In my local maven repo, I found:
$ jar tvf
/Users/tyu/.m2/repository//org/spark-project/akka/akka-actor_2.10/2.3.4-spark/akka-actor_2.10-2.3.4-spark.jar
| grep SelectionPath
521 Mon Sep 29 12:05:36 PDT 2014 akka/actor/SelectionPathElement.class
Is the above jar in your classpath ?
On Sat, May 23, 2015 at 5:05 PM, boci <bo...@gmail.com> wrote:
> Hi guys!
>
> I have a small spark application. It's query some data from postgres,
> enrich it and write to elasticsearch. When I deployed into spark container
> I got a very fustrating error:
> https://gist.github.com/b0c1/66527e00bada1e4c0dc3
>
> Spark version: 1.3.1
> Hadoop version: 2.6.0
> Additional info:
> serialization: kryo
> rdd: custom rdd to query
>
> I not understand
> 1. akka.actor.SelectionPath doesn't exists in 1.3.1
> 2. I checked all dependencies in my project, I only have
> "org.spark-project.akka:akka-*_2.10:2.3.4-spark:jar" doesn't have
> 3. I not found any reference for this...
> 4. I created own RDD, it's work, but I need to register to kryo? (mapRow
> using ResultSet, I need to create
> 5. I used some months ago and it's already worked with spark 1.2... I
> recompiled with 1.3.1 but I got this strange error
>
> Any idea?
>
>
> ----------------------------------------------------------------------------------------------------------------------------------
> Skype: boci13, Hangout: boci.boci@gmail.com
>