[jira] [Commented] (AIRFLOW-1786) Upstream Sensor skipped but whole DAG fails

["Apache Spark (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16601401#comment-16601401 ] 

Apache Spark commented on AIRFLOW-1786:
---------------------------------------

User 'artem-kirillov' has created a pull request for this issue:
https://github.com/apache/incubator-airflow/pull/3509

> Upstream Sensor skipped but whole DAG fails
> -------------------------------------------
>
>                 Key: AIRFLOW-1786
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1786
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: DAG
>    Affects Versions: Airflow 1.8
>            Reporter: Felix Yuan
>            Assignee: Artem Kirillov
>            Priority: Minor
>             Fix For: 1.10.0, 2.0.0
>
>
> I have created a daily DAG, which first monitor whether a file is ready on FTP server, then execute an ETL operation. If someday the file is missing, this is expected, so I set the "soft_fail" as True to simply skipped the FTP sensor.
> During execution, I see the task of "FTP sensor" is surely be skipped, but the whole DAG is marked as failed.
> My expectation is the upstream task (FTP Sensor) is skipped, then all downstream tasks are skipped as well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)