You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Bjorn Olsen (Jira)" <ji...@apache.org> on 2019/12/04 07:56:00 UTC
[jira] [Updated] (AIRFLOW-6170) BranchPythonOperator does not do
XCom push of returned value
[ https://issues.apache.org/jira/browse/AIRFLOW-6170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bjorn Olsen updated AIRFLOW-6170:
---------------------------------
Description:
BranchPythonOperator subclasses PythonOperator and this means that after it has selected a branch we get a message like this:
[2019-12-04 08:39:59,960] \{python_operator.py:114} INFO - Done. Returned value was: chosen_task
This returned value is from the execute() method. A user would expect the returned value from this to be pushed to an XCOM but this does not happen due to no return value from the BranchPythonOperator's execute method.
If we do an XCOM push of the result then we can use the decision made by BranchPythonOperator in downstream tasks.
Eg consider this dependency chain:
get >> branch >> [ create, update ]>> join >> execute
The 'execute' task might need to know whether the 'branch' decided to create a new thing to run, or whether to use the existing one from the 'get'. Without an XCOM push from the branch return value, it is difficult to pick up the correct value later on.
was:
BranchPythonOperator subclasses PythonOperator and this means that after it has selected a branch we get a message like this:
[2019-12-04 08:39:59,960] \{python_operator.py:114} INFO - Done. Returned value was: chosen_task
This returned value is from the execute() method. A user would expect the returned value from this to be pushed to an XCOM but this does not happen due to no return value from the BranchPythonOperator.
If we do an XCOM push of the result then we can use the decision made by BranchPythonOperator in downstream tasks.
Eg consider this dependency chain:
get >> branch >> [ create, update ]>> join >> execute
The 'execute' task might need to know whether the 'branch' decided to create a new thing to run, or whether to use the existing one from the 'get'. Without an XCOM push from the branch return value, it is difficult to pick up the correct value later on.
> BranchPythonOperator does not do XCom push of returned value
> ------------------------------------------------------------
>
> Key: AIRFLOW-6170
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6170
> Project: Apache Airflow
> Issue Type: Improvement
> Components: operators
> Affects Versions: 1.10.6
> Reporter: Bjorn Olsen
> Assignee: Bjorn Olsen
> Priority: Major
>
> BranchPythonOperator subclasses PythonOperator and this means that after it has selected a branch we get a message like this:
> [2019-12-04 08:39:59,960] \{python_operator.py:114} INFO - Done. Returned value was: chosen_task
>
> This returned value is from the execute() method. A user would expect the returned value from this to be pushed to an XCOM but this does not happen due to no return value from the BranchPythonOperator's execute method.
>
>
>
> If we do an XCOM push of the result then we can use the decision made by BranchPythonOperator in downstream tasks.
> Eg consider this dependency chain:
> get >> branch >> [ create, update ]>> join >> execute
> The 'execute' task might need to know whether the 'branch' decided to create a new thing to run, or whether to use the existing one from the 'get'. Without an XCOM push from the branch return value, it is difficult to pick up the correct value later on.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)