You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Kevin Cox (JIRA)" <ji...@apache.org> on 2015/07/31 21:29:04 UTC
[jira] [Commented] (SPARK-5925) YARN - Spark progress bar stucks at
10% but after finishing shows 100%
[ https://issues.apache.org/jira/browse/SPARK-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14649688#comment-14649688 ]
Kevin Cox commented on SPARK-5925:
----------------------------------
Could a simple `stage / number of stages` be a good enough estimate? It is nice to just get a rough idea.
> YARN - Spark progress bar stucks at 10% but after finishing shows 100%
> ----------------------------------------------------------------------
>
> Key: SPARK-5925
> URL: https://issues.apache.org/jira/browse/SPARK-5925
> Project: Spark
> Issue Type: Improvement
> Components: YARN
> Affects Versions: 1.2.1
> Reporter: Laszlo Fesus
> Priority: Minor
>
> I did set up a yarn cluster (CDH5) and spark (1.2.1), and also started Spark History Server. Now I am able to click on more details on yarn's web interface and get redirected to the appropriate spark logs during both job execution and also after the job has finished.
> My only concern is that while a spark job is being executed (either yarn-client or yarn-cluster), the progress bar stucks at 10% and doesn't increase as for MapReduce jobs. After finishing, it shows 100% properly, but we are loosing the real-time tracking capability of the status bar.
> Also tested yarn restful web interface, and it retrieves again 10% during (yarn) spark job execution, and works well again after finishing. (I suppose for the while being I should have a look on Spark Job Server and see if it's possible to track the job via its restful web interface.)
> Did anyone else experience this behaviour? Thanks in advance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org