You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2019/08/14 09:20:00 UTC
[jira] [Commented] (AIRFLOW-4908) Implement BigQuery
Hooks/Operators for update_dataset, patch_dataset and get_dataset
[ https://issues.apache.org/jira/browse/AIRFLOW-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907067#comment-16907067 ]
ASF GitHub Bot commented on AIRFLOW-4908:
-----------------------------------------
mik-laj commented on pull request #5546: [AIRFLOW-4908] BigQuery Hooks/Operators for update_dataset, patch_dataset, get_dataset
URL: https://github.com/apache/airflow/pull/5546
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Implement BigQuery Hooks/Operators for update_dataset, patch_dataset and get_dataset
> ------------------------------------------------------------------------------------
>
> Key: AIRFLOW-4908
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4908
> Project: Apache Airflow
> Issue Type: New Feature
> Components: gcp
> Affects Versions: 2.0.0
> Reporter: Ryan Yuan
> Assignee: Ryan Yuan
> Priority: Critical
>
> To create a BigQuery sink for GCP Stackdriver Logging, I have to assign `WRITER` access to group `cloud-logs@google.com` to access BQ dataset. However, current BigQueryHook doesn't support updating/patching dataset.
> Reference: [https://googleapis.github.io/google-cloud-python/latest/logging/usage.html#export-to-bigquery]
> Implement GCP Stackdriver Logging: https://issues.apache.org/jira/browse/AIRFLOW-4779
> While it is missing update_dataset and patch_dataset, BigQueryHook has get_dataset but it doesn't have operator for it.
>
> Features to be implemented:
> BigQueryBaseCursor.patch_dataset
> BigQueryBaseCursor.update_dataset
> BigQueryPatchDatasetOperator
> BigQueryUpdateDatasetOperator
> BigQueryGetDatasetOperator
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)