[jira] [Resolved] (SPARK-29072) Properly track shuffle write time with refactor

["Imran Rashid (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-29072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Imran Rashid resolved SPARK-29072.
----------------------------------
    Fix Version/s: 3.0.0
       Resolution: Fixed

Issue resolved by pull request 25780
[https://github.com/apache/spark/pull/25780]

> Properly track shuffle write time with refactor
> -----------------------------------------------
>
>                 Key: SPARK-29072
>                 URL: https://issues.apache.org/jira/browse/SPARK-29072
>             Project: Spark
>          Issue Type: Bug
>          Components: Shuffle
>    Affects Versions: 3.0.0
>            Reporter: Matt Cheah
>            Assignee: Matt Cheah
>            Priority: Major
>             Fix For: 3.0.0
>
>
> From SPARK-28209, SPARK-28570, and SPARK-28571, we used the new shuffle writer plugin API across all the shuffle writers. However, we accidentally lost time tracking metrics for shuffle writes in the process, particularly for UnsafeShuffleWriter when writing with streams (without transferTo), as well as the SortShuffleWriter.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org