[jira] [Resolved] (AIRFLOW-1619) Add sleep time parameter for GCP dataflow hook

["Chris Riccomini (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-1619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Riccomini resolved AIRFLOW-1619.
--------------------------------------
       Resolution: Fixed
    Fix Version/s: 1.10.0

> Add sleep time parameter for GCP dataflow hook
> ----------------------------------------------
>
>                 Key: AIRFLOW-1619
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1619
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: contrib, Dataflow, gcp, hooks
>            Reporter: Thomas Buida
>            Assignee: Thomas Buida
>            Priority: Minor
>             Fix For: 1.10.0
>
>
> In the GCP dataflow hook, the sleep time while waiting for the dataflow job to be done is hard coded to 10 seconds. For long running jobs (hours or even several days), this rate is overkill when a poll time of several minutes or even an hour may be sufficient.
> Adding a poll_sleep parameter to the GCP dataflow operator that is used in the hook would allow users to set a more sensible polling interval for long running jobs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)