You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Etienne Chauchot (JIRA)" <ji...@apache.org> on 2018/05/16 08:26:00 UTC

[jira] [Commented] (BEAM-3967) Create a langage agnostic definition of MetricsPusher

    [ https://issues.apache.org/jira/browse/BEAM-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16477023#comment-16477023 ] 

Etienne Chauchot commented on BEAM-3967:
----------------------------------------

MetricsPusher is quite a simple piece of software hosted in the runner side what it does is this:
 * reads options provided by the user (what sink to use, what frequency to push, additional sink related parameters)
 *  schedules a thread which regularly merges the metrics of a given pipeline and pushes them to the configured metrics sink.
 * at each thread tick, it checks the status of the pipeline and if it is in terminal state, then it terminates the pushing thread.

So, for the communication between SDKs and runners, the only thing needed is the pipelineOptions all the other things are runner dependent and runner side.

> Create a langage agnostic definition of MetricsPusher
> -----------------------------------------------------
>
>                 Key: BEAM-3967
>                 URL: https://issues.apache.org/jira/browse/BEAM-3967
>             Project: Beam
>          Issue Type: Sub-task
>          Components: runner-core
>            Reporter: Etienne Chauchot
>            Assignee: Etienne Chauchot
>            Priority: Major
>
> Part of portability effort, we need to define a langage agnostic definition of Metrics pusher. Java runners are free to use default java implementation of MetricsPusher. But runners or SDKs that are not in java need this. Especially Dataflow engine will use it to instantiate the MetricsPusher (see https://issues.apache.org/jira/browse/BEAM-3926)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)