[jira] [Commented] (BEAM-12032) Spark warns on SparkBeamMetric already registered

["Beam JIRA Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-12032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17380059#comment-17380059 ] 

Beam JIRA Bot commented on BEAM-12032:
--------------------------------------

This issue was marked "stale-P2" and has not received a public comment in 14 days. It is now automatically moved to P3. If you are still affected by it, you can comment and move it back to P2.

> Spark warns on SparkBeamMetric already registered
> -------------------------------------------------
>
>                 Key: BEAM-12032
>                 URL: https://issues.apache.org/jira/browse/BEAM-12032
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-spark
>            Reporter: Ismaël Mejía
>            Priority: P3
>
> Spark runner in some environments (Spark 3 and Databricks) reports a warning on Unknown metric type. The pipeline works well and the metrics appear correct but the warning seems to be a problem of the runner.
> {quote}
>  [2021-03-23T15:10:02,222] [INFO ] o.a.s.m.MetricsSystem - Metrics already registered
>  java.lang.IllegalArgumentException: Unknown metric type: class org.apache.beam.runners.spark.metrics.SparkBeamMetric
>  at com.codahale.metrics.MetricRegistry.notifyListenerOfAddedMetric(MetricRegistry.java:532) ~[spark-3.0.1-yarn-shuffle.jar:?]
>  at com.codahale.metrics.MetricRegistry.addListener(MetricRegistry.java:363) ~[spark-3.0.1-yarn-shuffle.jar:?]
>  at com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:93) ~[spark-3.0.1-yarn-shuffle.jar:?]
>  at org.apache.spark.metrics.MetricsSystem.registerSource(MetricsSystem.scala:165) [spark-core_2.12-3.0.1.jar:3.0.1]
>  at org.apache.beam.runners.spark.SparkRunner.registerMetricsSource(SparkRunner.java:275) [beam-runners-spark-2.27.0.jar:?]
>  at org.apache.beam.runners.spark.SparkRunner.run(SparkRunner.java:250) [beam-runners-spark-2.27.0.jar:?]
>  at org.apache.beam.runners.spark.SparkRunner.run(SparkRunner.java:92) [beam-runners-spark-2.27.0.jar:?]
>  at org.apache.beam.sdk.Pipeline.run(Pipeline.java:322) [beam-sdks-java-harness-2.27.0.jar:?]{quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)