[GitHub] [beam] kennknowles opened a new issue, #19433: Antipattern will break legacy dataflow python pipelines if a new cy_combiner is added and used in the python counter_factory

[GitBox <gi...@apache.org>]

kennknowles opened a new issue, #19433:
URL: https://github.com/apache/beam/issues/19433

   There are two maps keyed by the cy_combiner object type in dataflow/internal/apiclient.py
   structured_counter_translations and counter_translations 
   
   The only way to make changes today without breaking the legacy python dataflow runner harness.  
   
   (1) Introduce the new cy_combiner and modify the apiclient.py's maps which are keyed by the new types in a first PR
   (2) Import and release the dataflow containers
   (3) Add your code to use the new cy_combiner in the beam SDK
   
   Prior to step 2, you cannot even run dataflow pipelines using your PR. As you will hit a KeyError when the cy_combiner type is looked up in the api_client maps.
   
   Naturally a non dataflow contributor will just try to use the cy_combiner and unexpectedly break the legacy python dataflow runner harness.  
   
   One solution is to catch the key error and log a warning that the metric is dropped, rather than failing the pipeline.
   
   Imported from Jira [BEAM-7050](https://issues.apache.org/jira/browse/BEAM-7050). Original Jira may contain additional context.
   Reported by: ajamato@google.com.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org