You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2015/10/01 19:37:26 UTC

[jira] [Updated] (SPARK-10897) Custom job/stage names

     [ https://issues.apache.org/jira/browse/SPARK-10897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen updated SPARK-10897:
------------------------------
    Component/s: Web UI

The problem is that I'm not sure stages are really user-visible as they're part of how Spark implements methods. For example if groupBy results in 3 stages, which one gets the name? if 3 method calls result in 1 stage?I don't think it's impossible but not sure about the details of the semantics. 

I'm aware of the discussion, but is the motivation really to just display something farther up the call stack? e.g. if you run a Crunch job via Spark you're more interested in what Crunch method invoked a Spark method that invoked a stage, rather than just the Spark method. The Crunch method is still in the trace.

> Custom job/stage names
> ----------------------
>
>                 Key: SPARK-10897
>                 URL: https://issues.apache.org/jira/browse/SPARK-10897
>             Project: Spark
>          Issue Type: Wish
>          Components: Web UI
>            Reporter: Nithin Asokan
>            Priority: Minor
>
> Logging this jira to get some opinion about discussion I started on [user-list|http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Job-Stage-names-tt24867.html]
> I would like to get some thoughts about having custom stage/job names. Currently I believe the stage names cannot be controlled by user, but if allowed we can have libraries like Apache [Crunch|https://crunch.apache.org/] to dynamically set stage names based on the type of processing(action/transformation) it is performing. 
> Is it possible for Spark to support custom names? Will it make sense to allow users set stage names?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org