[GitHub] [beam] damccorm opened a new issue, #20380: Unskip test_metrics (py) in Spark runner

[GitBox <gi...@apache.org>]

damccorm opened a new issue, #20380:
URL: https://github.com/apache/beam/issues/20380

   For the test_metrics failure, I found that metrics are being passed to Python. The test breaks because no metrics match the filter [1], because the step names are transformed somehow such that the filter logic is too strict to recognize [2]:
   
   Spark Runner:
   MetricKey(step=ref_AppliedPTransform_count1_17, metric=MetricName(namespace=ns, name=counter), labels={}): 2
   MetricKey(step=ref_AppliedPTransform_count2_18, metric=MetricName(namespace=ns, name=counter), labels={}): 4
   ...
   
   Fn API Runner:
   MetricKey(step=count1, metric=MetricName(namespace=ns, name=counter), labels={}): 2,
   MetricKey(step=count2, metric=MetricName(namespace=ns, name=counter), labels={}): 4
   
   Also, note that Flink has its own, completely different implementation of test_metrics [3].
   
   [1] https://github.com/apache/beam/blob/2ef7b9db8af015dcba544b93df00a4e54cd8caf2/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner_test.py#L744
   [2] https://github.com/apache/beam/blob/2ef7b9db8af015dcba544b93df00a4e54cd8caf2/sdks/python/apache_beam/metrics/metric.py#L151-L155
   [3] https://github.com/apache/beam/blob/2ef7b9db8af015dcba544b93df00a4e54cd8caf2/sdks/python/apache_beam/runners/portability/flink_runner_test.py#L251
   
   
   
   Imported from Jira [BEAM-10689](https://issues.apache.org/jira/browse/BEAM-10689). Original Jira may contain additional context.
   Reported by: ibzib.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org