You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2019/10/08 05:44:17 UTC
[jira] [Resolved] (SPARK-23777) Missing DAG arrows between stages
[ https://issues.apache.org/jira/browse/SPARK-23777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-23777.
----------------------------------
Resolution: Incomplete
> Missing DAG arrows between stages
> ---------------------------------
>
> Key: SPARK-23777
> URL: https://issues.apache.org/jira/browse/SPARK-23777
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 1.3.0, 2.3.0
> Reporter: Stefano Pettini
> Priority: Trivial
> Labels: bulk-closed
> Attachments: Screenshot-2018-3-23 RDDTestApp - Details for Job 0.png
>
>
> In the Spark UI DAGs, sometimes there are missing arrows between stages. It seems to happen when the same RDD is shuffled twice.
> For example in this case:
> {{val a = context.parallelize(List(10, 20, 30, 40, 50)).keyBy(_ / 10)}}
> {{val b = a join a}}
> {{b.collect()}}
> There's a missing arrow from stage 1 to 2.
> _This is an old one, since 1.3.0 at least, still reproducible in 2.3.0._
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org