You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Abhishek Sinha <ab...@infoworks.io> on 2018/11/16 20:41:34 UTC
Task got executed even when it's only immediate parent was in
upstream failed state
Airflow Version 1.8.2, Celery and Postgres
Task (and its downstream nodes) got executed even when it's only immediate
parent was in upstream failed state. The trigger rule for all nodes was set
to "all_success".
Looks like a bug in version 1.8.2. Can someone help me on this?
Regards,
Abhishek
Re: Task got executed even when it's only immediate parent was in
upstream failed state
Posted by "Driesprong, Fokko" <fo...@driesprong.frl>.
Hi Abhishek,
That sounds very specific. There are a lot of improvements since 1.10. For
now, the only advice I can give, please save the logs when this happens
again.
Cheers, Fokko
Op zo 18 nov. 2018 om 18:36 schreef Abhishek Sinha <ab...@infoworks.io>:
> Hi Fokko,
>
> This would be difficult to test. I am not sure of the condition when it
> occurred. Our DAG runs nightly and this has happened once in the past three
> months.
>
> If this is something that has been identified and fixed in 1.10, I can
> make an attempt to upgrade it. But I did not see anything related to this
> in the changelog.
>
>
>
>
>
>
> Regards,
>
> Abhishek
>
> On 18 November 2018 at 8:54:43 AM, Driesprong, Fokko (fokko@driesprong.frl)
> wrote:
>
> Hi Abhishek,
>
> Do you think it would be difficult to verify if this behavior still exists
> in 1.10? Maybe write a dummy dag, and run it using Docker.
>
> Cheers, Fokko
>
> Op vr 16 nov. 2018 om 21:45 schreef Abhishek Sinha <ab...@infoworks.io>:
>
>
> > Airflow Version 1.8.2, Celery and Postgres
> >
> > Task (and its downstream nodes) got executed even when it's only
> immediate
> > parent was in upstream failed state. The trigger rule for all nodes was
> set
> > to "all_success".
> >
> > Looks like a bug in version 1.8.2. Can someone help me on this?
> >
> >
> >
> >
> >
> >
> >
> >
> > Regards,
> >
> > Abhishek
> >
>
>
Re: Task got executed even when it's only immediate parent was in
upstream failed state
Posted by Abhishek Sinha <ab...@infoworks.io>.
Hi Fokko,
This would be difficult to test. I am not sure of the condition when it
occurred. Our DAG runs nightly and this has happened once in the past three
months.
If this is something that has been identified and fixed in 1.10, I can make
an attempt to upgrade it. But I did not see anything related to this in the
changelog.
Regards,
Abhishek
On 18 November 2018 at 8:54:43 AM, Driesprong, Fokko (fokko@driesprong.frl)
wrote:
Hi Abhishek,
Do you think it would be difficult to verify if this behavior still exists
in 1.10? Maybe write a dummy dag, and run it using Docker.
Cheers, Fokko
Op vr 16 nov. 2018 om 21:45 schreef Abhishek Sinha <ab...@infoworks.io>:
> Airflow Version 1.8.2, Celery and Postgres
>
> Task (and its downstream nodes) got executed even when it's only
immediate
> parent was in upstream failed state. The trigger rule for all nodes was
set
> to "all_success".
>
> Looks like a bug in version 1.8.2. Can someone help me on this?
>
>
>
>
>
>
>
>
> Regards,
>
> Abhishek
>
Re: Task got executed even when it's only immediate parent was in
upstream failed state
Posted by "Driesprong, Fokko" <fo...@driesprong.frl>.
Hi Abhishek,
Do you think it would be difficult to verify if this behavior still exists
in 1.10? Maybe write a dummy dag, and run it using Docker.
Cheers, Fokko
Op vr 16 nov. 2018 om 21:45 schreef Abhishek Sinha <ab...@infoworks.io>:
> Airflow Version 1.8.2, Celery and Postgres
>
> Task (and its downstream nodes) got executed even when it's only immediate
> parent was in upstream failed state. The trigger rule for all nodes was set
> to "all_success".
>
> Looks like a bug in version 1.8.2. Can someone help me on this?
>
>
>
>
>
>
>
>
> Regards,
>
> Abhishek
>