You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by dash <bs...@nd.edu> on 2014/07/02 19:03:40 UTC

Worker can not find custom KryoRegistrator

Hi, I'm using Spark 1.1.0 standalone with 5 workers and 1 driver, and Kryo
settings are 



When I submit this job, the driver works fine but workers will throw
ClassNotFoundException saying they can not found HDTMKryoRegistrator.

Any idea about this problem? I googled this but there is only one post looks
similar to my problem and seems it is fixed in 0.6

https://groups.google.com/forum/#!topic/spark-users/YHHkg7ELwRs



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Worker-can-not-find-custom-KryoRegistrator-tp8659.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Worker can not find custom KryoRegistrator

Posted by "Baoxu Shi(Dash)" <bs...@nd.edu>.
Don’t know why the setting does not appear in the last mail:

      .set("spark.serializer", "org.apache.spark.serializer.KryoSerializer")
      .set("spark.kryo.registrator", new HDTMKryoRegistrator().getClass.getName)


On Jul 2, 2014, at 1:03 PM, dash <bs...@nd.edu> wrote:

> Hi, I'm using Spark 1.1.0 standalone with 5 workers and 1 driver, and Kryo
> settings are 
> 
> 
> 
> When I submit this job, the driver works fine but workers will throw
> ClassNotFoundException saying they can not found HDTMKryoRegistrator.
> 
> Any idea about this problem? I googled this but there is only one post looks
> similar to my problem and seems it is fixed in 0.6
> 
> https://groups.google.com/forum/#!topic/spark-users/YHHkg7ELwRs
> 
> 
> 
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Worker-can-not-find-custom-KryoRegistrator-tp8659.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.