You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2024/03/19 23:32:00 UTC
[jira] [Closed] (SPARK-47290) Extend CustomTaskMetric to allow metric values from multiple sources
[ https://issues.apache.org/jira/browse/SPARK-47290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun closed SPARK-47290.
---------------------------------
Sure. I closed this.
> Extend CustomTaskMetric to allow metric values from multiple sources
> --------------------------------------------------------------------
>
> Key: SPARK-47290
> URL: https://issues.apache.org/jira/browse/SPARK-47290
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Parth Chandra
> Priority: Major
> Labels: pull-request-available
>
> Custom task metrics allow a DSV2 to add metrics that can be displayed in the UI. However, for DSV2 file sources, the FilePartitionReader may have multiple file readers, and each of these may report their metrics which need to be aggregated and bubbled up to the Scan. There is no way currently to update a metric value.
> A new interface that extends CustomTaskMetric and defines a way to allow updates would allow a DSV2 file scan implementation to over come this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org