You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by DanielH <dh...@ebay.com> on 2014/06/10 09:36:59 UTC

Re: debugger

Hi Josh,

I came across this post when looking for a debugger or RDD visualization
tool for Spark. I am using Spark 0.9.1 and upgrading soon to Spark 1.0. The
links you posted are dead. Can you please direct me to how I can debug my
existing Spark job.

Will I need to edit my existing job's code in addition to setting any
environment variables/parameters.

The problem: I am running Bagel on a very large graph and when the job gets
to the final step (saveAsTextFile) it will hang for up to many days until I
kill it. Oftentimes if I simply rerun the job, it will finish in an hour
which is the expected amount of time it should take.

Thanks!



--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/debugger-tp284p6982.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.