DAG visualization: no visualization information available with history server

[Raghava <m....@gmail.com>]

Hello All,

I am running the history server for a completed application. This
application was run with the following parameters

bin/spark-submit --class <class_name> --master local[2] --conf
spark.local.dir=/mnt/ --conf spark.eventLog.dir=/mnt/sparklog/ --conf
spark.eventLog.enabled=true --conf spark.ui.retainedJobs=10000 --conf
spark.ui.retainedStages=10000 <jar_name>

In the Spark Web UI (http://localhost:18080/), the DAG visualization of only
the most recent job is available. For rest of the jobs, I get the following
message 

No visualization information available for this job!
If this is an old job, its visualization metadata may have been cleaned up
over time.
You may consider increasing the value of spark.ui.retainedJobs and
spark.ui.retainedStages.

I did increase the retainedJobs and retainedStages value to 10,000. What
else should be done to retain the visualization information of all the
jobs/stages?

Thanks in advance.

Raghava.




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/DAG-visualization-no-visualization-information-available-with-history-server-tp26117.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org