You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2015/02/27 10:53:04 UTC
[jira] [Resolved] (SPARK-1015) Visualize the DAG of RDD
[ https://issues.apache.org/jira/browse/SPARK-1015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-1015.
------------------------------
Resolution: Won't Fix
That sounds great, but also sounds like something non-core enough that it can be a separate package that users could bring in and which could be advertised at http://spark-packages.org/
> Visualize the DAG of RDD
> -------------------------
>
> Key: SPARK-1015
> URL: https://issues.apache.org/jira/browse/SPARK-1015
> Project: Spark
> Issue Type: New Feature
> Components: Spark Core
> Affects Versions: 0.9.0
> Reporter: Jeff Zhang
>
> The DAG of RDD can help user understand the data flow and how spark get the final RDD executed. It could help user to find chances to optimize the execution of some complex RDD. I will leverage graphviz to visualize the DAG.
> For this task, I plan to split it into 2 steps.
> Step 1. Just visualize the simple DAG graph. Each RDD is one node, and there will be one edge between the parent RDD and child RDD. ( I attach one simple graph in the attachments )
> Step 2. Put RDD in the same stage into one sub graph. This may need to extract the splitting staging related code in DAGSchduler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org