[jira] [Updated] (AIRFLOW-3538) BigQueryToCloudStorageOperator to support bucket

["Aizhamal Nurmamat kyzy (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-3538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aizhamal Nurmamat kyzy updated AIRFLOW-3538:
--------------------------------------------
    Labels: bigquery  (was: )

> BigQueryToCloudStorageOperator to support bucket
> ------------------------------------------------
>
>                 Key: AIRFLOW-3538
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3538
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: gcp, operators
>    Affects Versions: 2.0.0
>            Reporter: jack
>            Priority: Major
>              Labels: bigquery
>
> Currently
> {code:java}
> BigQueryToCloudStorageOperator  {code}
> does not support bucket property.
>  
> This means that
> {code:java}
> destination_cloud_storage_uris {code}
> needs the full path e.g
> {code:java}
> gs://bucket....{code}
>  
> This behavior isn't consistent with other operators like:
> GoogleCloudStorageDownloadOperator
> MySqlToGoogleCloudStorageOperator
> GoogleCloudStorageToBigQueryOperator
>  
> and many more... all support bucket and don't require the user to define the full path but only relative path after the bucket name.
>  
> Consistency is important because when using many different operators if they have similar templates it reduce the need to define specific variables.
> For example in my case I must have two variables
> a - gs://bucket_name/folder
> b - folder   (bucket_name already defined in the bucket property).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)