You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "jack (JIRA)" <ji...@apache.org> on 2019/08/04 11:20:00 UTC

[jira] [Commented] (AIRFLOW-4718) FileSensor does not distinguish non-existent file from other IO failures

    [ https://issues.apache.org/jira/browse/AIRFLOW-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16899611#comment-16899611 ] 

jack commented on AIRFLOW-4718:
-------------------------------

Can you raise PR for this?

> FileSensor does not distinguish non-existent file from other IO failures
> ------------------------------------------------------------------------
>
>                 Key: AIRFLOW-4718
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-4718
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: contrib
>    Affects Versions: 1.10.3
>            Reporter: Kirill Zhuravlev
>            Priority: Minor
>
> airflow.contrib.sensors.file_sensor.FileSensor does not distinguish non-existent file from other IO failures (e.g. wrong permissions, etc.)
> This limits its uses in scenarios with `soft_fail=True`, where it can mistake a different IOError for missing file and skip the rest of the workflow.
> Example case:
>  * file is present in the fs
>  * it (its parent folder) doesn't have listing permissions for airflow user
>  * `soft_fail=True` for the sensor
>  * sensor skips the workflow despite the file being there
> Expected behavior:
>  * failed task in case of any exception other than IOError with errno=={{errno.}}{{ENOENT}} (==2)



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)