You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Miłosz Szymczak (JIRA)" <ji...@apache.org> on 2018/08/31 13:13:00 UTC
[jira] [Work started] (AIRFLOW-2991) Log path to driver output
after Dataproc job is completed
[ https://issues.apache.org/jira/browse/AIRFLOW-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on AIRFLOW-2991 started by Miłosz Szymczak.
------------------------------------------------
> Log path to driver output after Dataproc job is completed
> ---------------------------------------------------------
>
> Key: AIRFLOW-2991
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2991
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib
> Affects Versions: 1.10.1
> Reporter: Miłosz Szymczak
> Assignee: Miłosz Szymczak
> Priority: Minor
> Labels: pull-request-available
>
> It's difficult to keep track of logs from Google Cloud Dataproc jobs invoked by Airflow. Dataproc API provides an output property [driverOutputResourceUri|https://cloud.google.com/dataproc/docs/reference/rest/v1/projects.regions.jobs#Job] containing path to stdout of job's driver. It's very convenient to be able to quickly check the output log in case of errors without opening Google Cloud Console.
> The change involves printing INFO-level log with value of *driverOutputResourceUri* after Dataproc job is finished.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)