You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Kaxil Naik (JIRA)" <ji...@apache.org> on 2019/01/12 13:32:00 UTC
[jira] [Reopened] (AIRFLOW-732) Unable to upload AVRO file from GCS
to BigQuery with no schema
[ https://issues.apache.org/jira/browse/AIRFLOW-732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kaxil Naik reopened AIRFLOW-732:
--------------------------------
> Unable to upload AVRO file from GCS to BigQuery with no schema
> --------------------------------------------------------------
>
> Key: AIRFLOW-732
> URL: https://issues.apache.org/jira/browse/AIRFLOW-732
> Project: Apache Airflow
> Issue Type: Bug
> Components: operators
> Affects Versions: 1.7.1.3, 1.8.0
> Reporter: Tahir Fayyaz
> Priority: Minor
> Labels: google-cloud-bigquery, google-cloud-storage
> Fix For: 1.7.1.3, 1.8.0
>
>
> For the gcs_to_bq the operator you have to provide a schema and the fails if you do not provide a schema.
> However for AVRO uploads you do not need to provide a schema as this optional.
> Currently there is no schema is provided None is actually passed through which causes the job to fail for AVRO uploads.
> You can also actually tell the able to autodetect for CSV and JSON also using (configuration.load.autodetect) - more details https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)