[GitHub] [beam] mosche opened a new issue, #22384: [Feature Request]: Use Spark 3 plugin framework to support metrics for Spark runner

[GitBox <gi...@apache.org>]

mosche opened a new issue, #22384:
URL: https://github.com/apache/beam/issues/22384

   ### What would you like to happen?
   
   Currently the metrics integration requires users to configure a custom, Beam-specific Spark metrics sink which is hard to understand. 
   
   At least for Spark 3, this should be based on the Spark plugin framework to make the integration more seamless.
   This also requires BeamMetricSet to become an actual MetricRegistry (instead of a Gauge).
   
   ### Issue Priority
   
   Priority: 2
   
   ### Issue Component
   
   Component: runner-spark


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] echauchot commented on issue #22384: [Feature Request]: Use Spark 3 plugin framework to support metrics for Spark runner

[GitBox <gi...@apache.org>]

echauchot commented on issue #22384:
URL: https://github.com/apache/beam/issues/22384#issuecomment-1191245833

   :+1:


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org