[jira] [Created] (AIRFLOW-6235) Logs of dags outside `dag_dir` are placed outside log directory

["Remi Baar (Jira)" <ji...@apache.org>]

Remi Baar created AIRFLOW-6235:
----------------------------------

             Summary: Logs of dags outside `dag_dir` are placed outside log directory
                 Key: AIRFLOW-6235
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6235
             Project: Apache Airflow
          Issue Type: Bug
          Components: utils
    Affects Versions: 1.10.6
         Environment: Python 3.7, Airflow 1.10.6
            Reporter: Remi Baar


*How to reproduce*
 # Install airflow through pip (or conda).
 # Run `airflow initdb`
 # Run `airflow scheduler` 
 # Error occurs.

*Explanation*

By running `initdb` example dags are loaded in the databases. These life inside the module folder (and thus outside the `AIRFLOW_HOME` directory or the dag_dir if another is specified). 

The scheduler will create log files based on the name of the dag file. The relative path is being used form the dag_dir. So if the dag dir is '/var/lib/airflow/dags' and the dag is located is at '/var/lib/airflow/dags/foo/bar.py` the log file created is `/var/lib/airflow/logs/scheduler/2019-12-12/foor/bar.py.log`. This is perfect.

But now the dag is outside the dag directory. For example:
dag: `'/opt/miniconda3/envs/airflow/lib/python3.7/site-packages/airflow/example_dags/test_utils.py'` 
the relative path: `'../../../../opt/miniconda3/envs/airflow/lib/python3.7/site-packages/airflow/example_dags/test_utils.py'
the log file will be: `'/var/lib/airflow/logs/scheduler/2019-12-12/../../../../opt/miniconda3/envs/airflow/lib/python3.7/site-packages/airflow/example_dags/test_utils.py.log'`

As you can see the log files is being placed at a place where it never is supposed to be! 

*Lines of code causing this issue:*



--
This message was sent by Atlassian Jira
(v8.3.4#803005)