You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/09/03 20:56:15 UTC

[GitHub] [beam] pabloem commented on a change in pull request #12762: Ensuring that BigQuery jobs are tagged with the Dataflow step that launches them

pabloem commented on a change in pull request #12762:
URL: https://github.com/apache/beam/pull/12762#discussion_r483249377



##########
File path: sdks/python/apache_beam/io/gcp/bigquery_io_metadata.py
##########
@@ -64,6 +64,8 @@ def create_bigquery_io_metadata():
     # As we do not want a bad label to fail the BQ job.
     if _is_valid_cloud_label_value(dataflow_job_id):
       kwargs['beam_job_id'] = dataflow_job_id
+  if step_name:
+    kwargs['step_name'] = step_name

Review comment:
       You're right that we need to sanitize. The step name would be something like `CustomerTransform/ReadFromBigquery/Read/_CustomBigQuertSouce`. I can add sanitization to it.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org