You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/03/06 01:05:00 UTC

[jira] [Work logged] (BEAM-11033) Update Dataflow metrics processor to handle portable jobs

     [ https://issues.apache.org/jira/browse/BEAM-11033?focusedWorklogId=561661&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-561661 ]

ASF GitHub Bot logged work on BEAM-11033:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 06/Mar/21 01:04
            Start Date: 06/Mar/21 01:04
    Worklog Time Spent: 10m 
      Work Description: chamikaramj commented on pull request #14158:
URL: https://github.com/apache/beam/pull/14158#issuecomment-791822438


   R: @kennknowles 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 561661)
    Time Spent: 5h  (was: 4h 50m)

> Update Dataflow metrics processor to handle portable jobs
> ---------------------------------------------------------
>
>                 Key: BEAM-11033
>                 URL: https://issues.apache.org/jira/browse/BEAM-11033
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-py-core
>            Reporter: Chamikara Madhusanka Jayalath
>            Assignee: Chamikara Madhusanka Jayalath
>            Priority: P1
>             Fix For: 2.29.0
>
>          Time Spent: 5h
>  Remaining Estimate: 0h
>
> Currently, Dataflow metrics processor expects Dataflow internal step names generated for v1beta3 job description in metrics returned by Dataflow service: [https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/dataflow/dataflow_metrics.py#L97]
>  
> But with portable job submission, Dataflow uses PTransform ID (in proto pipeline) as the internal step name. Hence metrics processor should be updated to handle this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)