You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Reynold Xin (JIRA)" <ji...@apache.org> on 2018/12/10 21:59:00 UTC

[jira] [Resolved] (SPARK-26139) Support passing shuffle metrics to exchange operator

     [ https://issues.apache.org/jira/browse/SPARK-26139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Reynold Xin resolved SPARK-26139.
---------------------------------
       Resolution: Fixed
    Fix Version/s: 3.0.0

> Support passing shuffle metrics to exchange operator
> ----------------------------------------------------
>
>                 Key: SPARK-26139
>                 URL: https://issues.apache.org/jira/browse/SPARK-26139
>             Project: Spark
>          Issue Type: New Feature
>          Components: SQL
>    Affects Versions: 2.4.0
>            Reporter: Reynold Xin
>            Assignee: Reynold Xin
>            Priority: Major
>             Fix For: 3.0.0
>
>
> Due to the way Spark's architected (SQL is defined on top of the RDD API), there are two separate metrics system used in core vs SQL. Ideally, we'd want to be able to get the shuffle metrics for each of the exchange operator independently, e.g. blocks read, number of records.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org