Running spark 0.9 on mesos 0.15

[elyast <lu...@gmail.com>]

Hi,

Quick question do I need to compile spark against exactly same version of
mesos library, currently spark depends on 0.13.

The problem I am facing is following I am running MLib example with SVM and
it works nicely when I use coarse grained mode, however when running fine
grained mode on mesos the tasks scheduling gots stuck. 
(I know about ActorNotFound exception I applied latest git pull request on
branch-0.9)

Strange thing is for simple things fine grained mode works ok, however when
doing graph examples or MLib ones its just hangs (standalone job) looks like
waiting for something and spark ui shows one task in active stage but not
scheduled yet.

Happy to do more testing if needed
Thanks for information

Best regards
Lukasz Jastrzebski




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Running-spark-0-9-on-mesos-0-15-tp2326.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: spark job stuck when running on mesos fine grained mode

[elyast <lu...@gmail.com>]

Hi Prabeesh,

I've recently moved to mesos 0.18.2 and spark 1.0, so far no problems in
fine grained mode, even for grapx or mllib workflows. If u have specific
code snippets I can try it out.

Best regards
Lukasz



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-job-stuck-when-running-on-mesos-fine-grained-mode-tp2326p8552.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: spark job stuck when running on mesos fine grained mode

[prabeesh <pr...@gmail.com>]

Hi Lukasz Jastrzebski ,

I have a question regarding Shark execution on mesos.

I am querying a file which is in hdfs and write results back to hdfs. The
problem I am facing in this is , unable to write output to hdfs. ie, when I
use the method SaveAsTextfile() , then the job is getting resubmitted. have
you ever faced such a scenario while working with shark. if so and if u
solved it can you please help us in solving the same.

14/05/30 10:16:35 INFO DAGScheduler: The failed fetch was from Stage 4
(mapPartitionsWithIndex at Operator.scala:333); marking it for resubmission
14/05/30 10:16:35 INFO DAGScheduler: Executor lost:
201405291518-3644595722-5050-17933-1 (epoch 148)






--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-job-stuck-when-running-on-mesos-fine-grained-mode-tp2326p6556.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.