You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Steve Vaughan (Jira)" <ji...@apache.org> on 2024/02/14 19:54:00 UTC

[jira] [Created] (SPARK-47050) Collect and publish partition level metrics

Steve Vaughan created SPARK-47050:
-------------------------------------

             Summary: Collect and publish partition level metrics
                 Key: SPARK-47050
                 URL: https://issues.apache.org/jira/browse/SPARK-47050
             Project: Spark
          Issue Type: New Feature
          Components: Spark Core
    Affects Versions: 3.5.0
            Reporter: Steve Vaughan


`WriteJobStatsTracker` currently collects the number of file, bytes, and rows written across all partitions.  This would update the metrics to also collect this same information for each committed partition.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org