You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@beam.apache.org by Sheldon Teerlink <sh...@mx.com> on 2023/07/24 16:48:33 UTC

[Question] Is there an idiom for executing BQ SQL MERGE?

Is there a way to run a BigQuery SQL MERGE command in a Python pipeline?
The examples I've seen write the PCollection straight to a BQ table.

I'd like to read messages from Pub/Sub, perform a transform, write results
to a BQ staging table, run a SQL MERGE between the staging table and a
master table and finally drop the staging table.,

If there is a better pattern to follow please advise.

Could someone please invite me to the ASF #beam channel in Slack?