You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Evgeny Podlepaev (JIRA)" <ji...@apache.org> on 2018/07/24 01:51:00 UTC
[jira] [Updated] (AIRFLOW-2774) DataFlowPythonOperator needs to
support DirectRunner to facilitate end-to-end testing of Airflow dag
[ https://issues.apache.org/jira/browse/AIRFLOW-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Evgeny Podlepaev updated AIRFLOW-2774:
--------------------------------------
Summary: DataFlowPythonOperator needs to support DirectRunner to facilitate end-to-end testing of Airflow dag (was: DataFlowPythonOperator needs to support DirectRunner to facilitate local testing)
> DataFlowPythonOperator needs to support DirectRunner to facilitate end-to-end testing of Airflow dag
> ----------------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-2774
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2774
> Project: Apache Airflow
> Issue Type: Improvement
> Components: Dataflow
> Affects Versions: 1.9.0
> Reporter: Evgeny Podlepaev
> Priority: Minor
>
> **DataFlowPythonOperator needs to support DirectRunner as a runner option to facilitate local testing of the entire pipeline. Right now if DirectRunner is set via job options, the DataFlowHook will wait infinitely trying to get status of the remote job which does not exist:
> _DataflowJob(self.get_conn(), variables['project'], name,
> variables['region'], self.poll_sleep).wait_for_done()
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)