Spark hanging forever when doing decision tree training

[Loic Quertenmont <lo...@gmail.com>]

Hello,

I am new to spark and I am currently learning how to use classification
algorithm with it.
For now on I am playing with a rather small dataset and training a decision
tree on my laptop  (running with --master local[1]).
However, systematically I see that my jobs are hanging forever at the
training stage.
Looking at the application UI, I see that my ongoing stage as still 15 out
of 16 task to do.
But none are scheduled and looking at the executors tab I see that my single
executor is indeed not processing anything (even if executor memory is still
fine).
<http://apache-spark-user-list.1001560.n3.nabble.com/file/n26922/Stage.png> 
<http://apache-spark-user-list.1001560.n3.nabble.com/file/n26922/Executor.png> 

If I ask the details of the hanging job I see:
org.apache.spark.rdd.RDD.count(RDD.scala:1125)
org.apache.spark.mllib.tree.impl.DecisionTreeMetadata$.buildMetadata(DecisionTreeMetadata.scala:114)
org.apache.spark.ml.tree.impl.RandomForest$.run(RandomForest.scala:65)
org.apache.spark.ml.classification.DecisionTreeClassifier.train(DecisionTreeClassifier.scala:77)
org.apache.spark.ml.classification.DecisionTreeClassifier.train(DecisionTreeClassifier.scala:40)

Any idea of what my problem could be?
My code is compiled with spark 1.6.1

Thanks in advance,
Loic



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-hanging-forever-when-doing-decision-tree-training-tp26922.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org