You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Ismaël Mejía (Jira)" <ji...@apache.org> on 2020/03/05 11:42:00 UTC

[jira] [Assigned] (BEAM-5212) Update native spark metrics

     [ https://issues.apache.org/jira/browse/BEAM-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ismaël Mejía reassigned BEAM-5212:
----------------------------------

    Assignee:     (was: Amit Sela)

> Update native spark metrics
> ---------------------------
>
>                 Key: BEAM-5212
>                 URL: https://issues.apache.org/jira/browse/BEAM-5212
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-spark
>            Reporter: Etienne Chauchot
>            Priority: Major
>
> Native engine metrics are updated on {{finishBundle}} by DoFnRunners. Spark uses {{SimpleDoFnRunner}} that is in runner-core and thus contains nothing spark native. It might be good to extend {{SimpleDoFnRunner}} to add native metrics support just like it was done in Flink with {{DoFnRunnerWithMetrics}} class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)