external dag dependency: clear downstream task in another dag

[Michael Gong <go...@hotmail.com>]

Hi,

When I clear the downstream tasks of 1 task from 1 dag, can it also clear its downstream in another dag ?


For example:


dag1: task1

dag2: task2 (which is an ExternalTaskSensor, and depends on task1)


When I clear task1's downstream, is it possible that task2 is also cleared ?



Any comments are welcomed.


Thanks.

Michael

Re: external dag dependency: clear downstream task in another dag

[Laura Lorenz <ll...@industrydive.com>]

I don't think so.

As I understand it the way the ExternalTaskSensor works is a simple,
pull-based dependency; its polls the metadata database until its associated
'external' TaskInstance has a state it expects (e.g. "SUCCESS"), so from
the ExternalTaskSensor's point of view a cleared 'external' TaskInstance,
not having the approved state yet, means it could just keep waiting forever
(until the timeout of course) and has no other connection. They don't have
any explicit `set_upstream()` or `set_downstream()` kind of relationship so
the upstream/downstream 'clear' or 'mark success' type of CLI traversal
won't find them.

On Wed, Oct 26, 2016 at 1:55 PM, Michael Gong <go...@hotmail.com> wrote:

> Hi,
>
> When I clear the downstream tasks of 1 task from 1 dag, can it also clear
> its downstream in another dag ?
>
>
> For example:
>
>
> dag1: task1
>
> dag2: task2 (which is an ExternalTaskSensor, and depends on task1)
>
>
> When I clear task1's downstream, is it possible that task2 is also cleared
> ?
>
>
>
> Any comments are welcomed.
>
>
> Thanks.
>
> Michael
>
>
>