[jira] [Commented] (SPARK-29654) Add configuration to allow disabling registration of static sources to the metrics system

["Pavol Knapek (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-29654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17181827#comment-17181827 ] 

Pavol Knapek commented on SPARK-29654:
--------------------------------------

Is there any chance to have this backported to a minor version of Spark 2.x.x (2.4.7) ?

> Add configuration to allow disabling registration of static sources to the metrics system
> -----------------------------------------------------------------------------------------
>
>                 Key: SPARK-29654
>                 URL: https://issues.apache.org/jira/browse/SPARK-29654
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 3.0.0
>            Reporter: Luca Canali
>            Assignee: Luca Canali
>            Priority: Minor
>             Fix For: 3.0.0
>
>
> The Spark metrics system produces many different metrics and not all of them are used at the same time. This proposes to introduce a configuration parameter to allow disabling the registration of metrics in the "static sources" category, in other to reduce the load and clutter on the sink, in the cases when the metrics in question are not needed. The metrics registerd as "static sources" are under the namespaces CodeGenerator and HiveExternalCatalog and can produce a significant amount of data, as they are registered for the driver and executors.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org