[jira] [Resolved] (AIRFLOW-5466) Custom library setup.py fails airflow

["Ash Berlin-Taylor (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-5466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ash Berlin-Taylor resolved AIRFLOW-5466.
----------------------------------------
    Resolution: Information Provided

Airflow can't know without being told that it shouldn't process files - by default it expects all python files under the dags/ folder to be dags.

If your project doesn't follow that pattern than that's what {{.airflowignore}} https://airflow.apache.org/concepts.html#airflowignore is for.

> Custom library setup.py fails airflow
> -------------------------------------
>
>                 Key: AIRFLOW-5466
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5466
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: cli
>    Affects Versions: 1.10.3, 1.10.4, 1.10.5
>            Reporter: Yegor Andreenko
>            Priority: Major
>
> I have created custom library to manage dags description. It's common used functions like get configuration, slack_hook definition, etc. 
> For that library I created `setup.py` file inside my project(seems legit). This project is supposed to be used like 
> ```
> <project>/
>                 dags/
>                         setup.py
>                         dags_1.py
>                         dags_N.py
>                         config/.....
>                         __init__.py 
>                 test/
> ```
> And we usually deploy dags folder to airflow.
> Airflow will crash `initdb`, `scheduler`, `webserver`, `list_dags` commands(all cli commands) if it encounters `setup.py` file in `dags` folder with an error message:
> ```
> usage: airflow [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
>  or: airflow --help [cmd1 cmd2 ...]
>  or: airflow --help-commands
>  or: airflow cmd --help
>  error: invalid command 'webserver'
> ```
> Expected behaviour:
>  - airflow shouldn't fail on `setup.py` file in `dags` folder
>  - or have a more clear message what went wrong



--
This message was sent by Atlassian Jira
(v8.3.4#803005)