You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Miłosz Szymczak (JIRA)" <ji...@apache.org> on 2018/08/31 13:06:00 UTC
[jira] [Updated] (AIRFLOW-2991) Log path to driver output after
Dataproc job is completed
[ https://issues.apache.org/jira/browse/AIRFLOW-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Miłosz Szymczak updated AIRFLOW-2991:
-------------------------------------
Description:
It's difficult to keep track of logs from Google Cloud Dataproc jobs invoked by Airflow. Dataproc API provides an output property [driverOutputResourceUri|https://cloud.google.com/dataproc/docs/reference/rest/v1/projects.regions.jobs#Job] containing path to stdout of job's driver. It's very convenient to be able to quickly check the output log in case of errors without opening Google Cloud Console.
The change involves printing INFO-level log with value of *driverOutputResourceUri* after Dataproc job is finished.
was:
It's difficult to keep track of logs from Google Cloud Dataproc jobs invoked by Airflow. Dataproc API provides an output property **[driverOutputResourceUri|https://cloud.google.com/dataproc/docs/reference/rest/v1/projects.regions.jobs#Job] containing path to stdout of job's driver. It's very convenient to be able to quickly check the output log in case of errors without opening Google Cloud Console.
The change involves printing INFO-level log with value of *driverOutputResourceUri* after Dataproc job is finished.
> Log path to driver output after Dataproc job is completed
> ---------------------------------------------------------
>
> Key: AIRFLOW-2991
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2991
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib
> Affects Versions: 1.10.1
> Reporter: Miłosz Szymczak
> Assignee: Miłosz Szymczak
> Priority: Minor
> Labels: pull-request-available
>
> It's difficult to keep track of logs from Google Cloud Dataproc jobs invoked by Airflow. Dataproc API provides an output property [driverOutputResourceUri|https://cloud.google.com/dataproc/docs/reference/rest/v1/projects.regions.jobs#Job] containing path to stdout of job's driver. It's very convenient to be able to quickly check the output log in case of errors without opening Google Cloud Console.
> The change involves printing INFO-level log with value of *driverOutputResourceUri* after Dataproc job is finished.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)