You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Michael Ghen (JIRA)" <ji...@apache.org> on 2018/10/26 21:23:00 UTC

[jira] [Created] (AIRFLOW-3261) Add All BigQuery Job Errors to Logs

Michael Ghen created AIRFLOW-3261:
-------------------------------------

             Summary: Add All BigQuery Job Errors to Logs
                 Key: AIRFLOW-3261
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3261
             Project: Apache Airflow
          Issue Type: Improvement
            Reporter: Michael Ghen


When running a BigQuery load job, errors reported only contain the final error. This hides information about the real error. The only workaround is to log into the GCP console and find the logs in Logging using the job's ID.

The `error_result` is selected for the Exception. All errors are stored in `errors` in the job's status.

[https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/hooks/bigquery_hook.py#L1080]

In the google-cloud-python package, you can see they implement methods for `error_result` and `errors` ([https://github.com/googleapis/google-cloud-python/blob/333c52a6c4d825d8c739833e24ef7d021bb5d1c4/bigquery/google/cloud/bigquery/job.py#L445)]

 

This enhancement would change the output to include all errors and not just the final result.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)