[jira] [Commented] (AIRFLOW-6011) Airflow treats local dependencies as DAG files if they contain the string "DAG"

["Ash Berlin-Taylor (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-6011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16976713#comment-16976713 ] 

Ash Berlin-Taylor commented on AIRFLOW-6011:
--------------------------------------------

https://airflow.apache.org/concepts.html#airflowignore is the way to control this. Given we want to load dags from sub-folders I'm not sure if there is more we could do to hilight this? Is there anything you would like to have seen in the docs to make this behaviour clearer?

> Airflow treats local dependencies as DAG files if they contain the string "DAG"
> -------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-6011
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6011
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: DAG, dependencies
>    Affects Versions: 1.10.2
>            Reporter: Aaron Liblong
>            Priority: Major
>
> Our Airflow instance uses what I believe is a common pattern of having a directory for DAG dependencies within the main {{dags}} directory itself. E.g.:
> {code:java}
> dags
> | - dag1.py
> | - dag2.py
> | - dependencies
> | | - dependency1.py
> | | - dependency2.py
> {code}
>  
> Then {{dag1.py}} contains something like {{import dependency1}}.
> The dependencies are normally not treated like DAG files themselves, as expected. However, when they contain the string "DAG", even in a comment, they _will_ be treated as DAG files. I'm guessing this is the result of Airflow doing a recursive grep for that exact (case-sensitive) string. This is not intuitive behaviour for users, and is hard to debug, since users will reasonably expect non-directive comments not to have an effect on how their code will being treated.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)