[jira] [Commented] (AIRFLOW-5542) Export a BigQuery query result to GCS

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942823#comment-16942823 ] 

ASF GitHub Bot commented on AIRFLOW-5542:
-----------------------------------------

meftasadat commented on pull request #6173: [AIRFLOW-5542] Added BQ query results export to GCS  operator
URL: https://github.com/apache/airflow/pull/6173
 
 
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Export a BigQuery query result to GCS
> -------------------------------------
>
>                 Key: AIRFLOW-5542
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5542
>             Project: Apache Airflow
>          Issue Type: New Feature
>          Components: operators
>    Affects Versions: 1.10.5
>            Reporter: Mefta Sadat
>            Priority: Minor
>
> h2. Problem
> There is currently no direct operator to export a BQ query result dataset to GCS.
> h2. Proposed implementation
> This problem can be solved by creating a temporary table, storing the query results in the temp table, exporting the table to GCS and then finally deleting the table
> The following methods in airflow.gcp.hooks.bigquery_hook can be leveraged to achieve this solution:
> create_empty_dataset -> run_query -> run_extract -> run_table_delete -> delete_dataset
> The proposer operator is just a wrapper around these methods. It executes the above-mentioned methods in the necessary order and handles any exceptions that may occur.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)