Unexpected behavior change for branching

[Michel Goldstein <mi...@gmail.com>]

Hi,

Recently I've upgraded an Airflow installation from 1.8.0 to 1.10.0 (yes, I
know, it took some time to get there) and, while the upgrade was mostly
painless, there was one behavior change that I wasn't quite able to
understand why it happened and what the best approach is to get back to the
same state. It's a pretty simple case in branching, using the
BranchingPythonOperator, in the "classic" diamond path:

Choice >> [ A, B ] >> Common

Where Choice is the BranchingPythonOperator which would choose between A or
B depending on some external state and then "Common" should run independent
of whether path A or B were chosen.

In 1.8.0 that was the default behavior. However, after the upgrade,
"Common" is marked as skipped.

I tried then to switch Common's trigger_rule to "one_success", but, while
in the success case it worked, if the chosen path failed, it causes again
for Common to instead of failing to be skipped and then the whole DAG is
marked as successful, which is also unexpected.

Is there something that I'm missing here? Even the documentation on
branching [1] gives an example where the "join" task runs and is not
skipped (unless it's a direct dependency of the branching operator), so
it's odd that it doesn't seem to be the default behavior anymore.

Thanks,
Michel Goldstein

[1] https://airflow.apache.org/concepts.html#branching