[jira] [Resolved] (BEAM-4775) JobService should support returning metrics

["Lukasz Gajowy (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-4775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Lukasz Gajowy resolved BEAM-4775.
---------------------------------
    Fix Version/s: Not applicable
       Resolution: Fixed

the JobServer part is done - all kinds of MonitroingInfos are forwarded through grpc to PortableRunners (sdk side). The PortableRunners can decide how to digest them (e.g. create MetricsResult from monitoring infos where possible). 

> JobService should support returning metrics
> -------------------------------------------
>
>                 Key: BEAM-4775
>                 URL: https://issues.apache.org/jira/browse/BEAM-4775
>             Project: Beam
>          Issue Type: Bug
>          Components: beam-model
>            Reporter: Eugene Kirpichov
>            Assignee: Lukasz Gajowy
>            Priority: Major
>             Fix For: Not applicable
>
>          Time Spent: 55h
>  Remaining Estimate: 0h
>
> Design doc: [https://s.apache.org/get-metrics-api].
> Further discussion is ongoing on [this doc|https://docs.google.com/document/d/1m83TsFvJbOlcLfXVXprQm1B7vUakhbLZMzuRrOHWnTg/edit?ts=5c826bb4#heading=h.faqan9rjc6dm].
> We want to report job metrics back to the portability harness from the runner harness, for displaying to users.
> h1. Relevant PRs in flight:
> h2. Ready for Review:
>  * [#8022|https://github.com/apache/beam/pull/8022]: correct the Job RPC protos from [#8018|https://github.com/apache/beam/pull/8018].
> h2. Iterating / Discussing:
>  * [#7971|https://github.com/apache/beam/pull/7971]: Flink portable metrics: get ptransform from MonitoringInfo, not stage name
>  ** this is a simpler, Flink-specific PR that is basically duplicated inside each of the following two, so may be worth trying to merge in first
>  * #[7915|https://github.com/apache/beam/pull/7915]: use MonitoringInfo data model in Java SDK metrics
>  * [#7868|https://github.com/apache/beam/pull/7868]: MonitoringInfo URN tweaks
> h2. Merged
>  * [#8018|https://github.com/apache/beam/pull/8018]: add job metrics RPC protos
>  * [#7867|https://github.com/apache/beam/pull/7867]: key MetricResult by a MetricKey
>  * [#7938|https://github.com/apache/beam/pull/7938]: move MonitoringInfo protos to model/pipeline module
>  * [#7883|https://github.com/apache/beam/pull/7883]: Add MetricQueryResults.allMetrics() helper
>  * [#7866|https://github.com/apache/beam/pull/7866]: move function helpers from fn-harness to sdks/java/core
>  * [#7890|https://github.com/apache/beam/pull/7890]: consolidate MetricResult implementations
> h2. Closed
>  * [#7934|https://github.com/apache/beam/pull/7934]: job metrics RPC + SDK support
>  * [#7876|https://github.com/apache/beam/pull/7876]: Clean up metric protos; support integer distributions, gauges



--
This message was sent by Atlassian Jira
(v8.3.4#803005)