[jira] [Updated] (SPARK-2035) Make a stage's call stack available on the UI

["Daniel Darabos (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-2035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Darabos updated SPARK-2035:
----------------------------------

    Attachment: example-html.tgz

I've sent a pull request (https://github.com/apache/spark/pull/981), and here is an example of the resulting HTML. It is the worst possible example, because I used `spark-shell`, but it's hopefully enough to demo the idea.

> Make a stage's call stack available on the UI
> ---------------------------------------------
>
>                 Key: SPARK-2035
>                 URL: https://issues.apache.org/jira/browse/SPARK-2035
>             Project: Spark
>          Issue Type: Improvement
>          Components: Web UI
>            Reporter: Daniel Darabos
>            Priority: Minor
>         Attachments: example-html.tgz
>
>
> Currently the stage table displays the file name and line number that is the call site that triggered the given stage. This is enormously useful for understanding the execution. But once a project adds utility classes and other indirections, the call site can become less meaningful, because the interesting line is further up the stack.
> An idea to fix this is to display the entire call stack that triggered the stage. It would be collapsed by default and could be revealed with a click.
> I have started working on this. It is a good way to learn about how the RDD interface ties into the UI.



--
This message was sent by Atlassian JIRA
(v6.2#6252)