[jira] [Created] (SPARK-27641) Unregistering a single Metrics Source with no metrics leads to removing all the from other sources with the same name

["Sergey Zhemzhitsky (JIRA)" <ji...@apache.org>]

Sergey Zhemzhitsky created SPARK-27641:
------------------------------------------

             Summary: Unregistering a single Metrics Source with no metrics leads to removing all the from other sources with the same name
                 Key: SPARK-27641
                 URL: https://issues.apache.org/jira/browse/SPARK-27641
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 2.4.2, 2.3.3, 2.2.3
            Reporter: Sergey Zhemzhitsky


Currently Spark allows registering multiple Metric Sources with the same source name like the following

{code:scala}
val acc1 = sc.longAccumulator
LongAccumulatorSource.register(sc, {"acc1" -> acc1})

val acc2 = sc.longAccumulator
LongAccumulatorSource.register(sc, {"acc2" -> acc2})
{code}

In that case there are two metric sources registered and both of these sources have the same name - [AccumulatorSource|https://github.com/apache/spark/blob/6ef45301a46c47c12fbc74bb9ceaffea685ed944/core/src/main/scala/org/apache/spark/metrics/source/AccumulatorSource.scala#L47]

If you try to unregister the source with no accumulators and metrics registered like the following
{code:scala}
SparkEnv.get.metricsSystem.removeSource(new LongAccumulatorSource)
{code}
... then all the metrics for all the sources with the same name will be unregistered because of the [following|https://github.com/apache/spark/blob/6ef45301a46c47c12fbc74bb9ceaffea685ed944/core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala#L171] snippet which removes all matching records which start with the corresponding prefix which includes the source name, but does not include metric name to be removed.
{code:scala}
def removeSource(source: Source) {
  sources -= source
  val regName = buildRegistryName(source)
  registry.removeMatching((name: String, _: Metric) => name.startsWith(regName))
}
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org