SubDag status shows running when all task instances are complete

[David Klosowski <da...@thinknear.com>]

Hi All,

In 1.7.1.3 I had experienced the exact issue described here:

https://issues.apache.org/jira/browse/AIRFLOW-736

The underlying task instances of a SubDag are complete; however, the SubDag
itself shows as running and is not marked as successful (until manually
done so).  It was an intermittent issue and I suspected that there was an
issue in the scheduler.  This issue only shows itself with pools in use and
I've since not been using them; but now would really like to use pools to
help mitigate the case of having many ExternalTaskSensors starving
resources and deadlocking b/c there are no tasks that can actually do work
(this is using celery).

Looking at the above issue it appears that it was resolved with a
resolution of "Cannot Reproduce". Which version was this validated against
(1.7.x or 1.8.x)?  I've been curious if 1.8.x would fix this issue but have
been hesitant to migrate to it given the concern around testing it against
all our use cases and against all of our custom subdags, operators, and
sensors.

Would someone provide more insight into the above ticket and if they have
seen this issue in 1.8.x?

Thanks.

Regards,
David