[jira] [Commented] (AIRFLOW-1619) Add sleep time parameter for GCP dataflow hook

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-1619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170440#comment-16170440 ] 

ASF subversion and git services commented on AIRFLOW-1619:
----------------------------------------------------------

Commit 1c5b0f5a39b3c855da0ff6cd32a6e0b4dbffa58e in incubator-airflow's branch refs/heads/master from [~tjbiii]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=1c5b0f5 ]

[AIRFLOW-1619] Add poll_sleep parameter to GCP dataflow operator

Closes #2612 from TJBIII/gcp_dataflow_poll_sleep


> Add sleep time parameter for GCP dataflow hook
> ----------------------------------------------
>
>                 Key: AIRFLOW-1619
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1619
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: contrib, Dataflow, gcp, hooks
>            Reporter: Thomas Buida
>            Assignee: Thomas Buida
>            Priority: Minor
>
> In the GCP dataflow hook, the sleep time while waiting for the dataflow job to be done is hard coded to 10 seconds. For long running jobs (hours or even several days), this rate is overkill when a poll time of several minutes or even an hour may be sufficient.
> Adding a poll_sleep parameter to the GCP dataflow operator that is used in the hook would allow users to set a more sensible polling interval for long running jobs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)