[jira] [Closed] (AIRFLOW-6859) Airflow scheduler looking for logs of example DAGs in the wrong folder

["Abhilash Kishore (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-6859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Abhilash Kishore closed AIRFLOW-6859.
-------------------------------------
    Resolution: Invalid

> Airflow scheduler looking for logs of example DAGs in the wrong folder
> ----------------------------------------------------------------------
>
>                 Key: AIRFLOW-6859
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6859
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 1.10.9
>            Reporter: Abhilash Kishore
>            Priority: Minor
>
> My AIRFLOW_HOME is set to */home/airflow*
> I am running Airflow as a service
> `cat /etc/systemd/system/airflow-scheduler.service` :
>  
> {code:java}
> [Unit]
> Description=Airflow scheduler daemon
> After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service mssql-server.service
> Wants=postgresql.service mysql.service redis.service rabbitmq-server.service mssql-server.service
> [Service]
> EnvironmentFile=/etc/sysconfig/airflow
> User=airflow
> Group=airflow
> Type=simple
> ExecStart=/usr/local/bin/airflow scheduler
> Restart=always
> RestartSec=5s
> [Install]
> WantedBy=multi-user.target
> {code}
>  
>  
> *BUG:*
> Airflow scheduler looking for logs of example DAGs in the wrong folder. Example:
>  
> {code:java}
> Feb 21 01:57:24 airflow-poc airflow[25847]: Process DagFileProcessor281-Process:
> Feb 21 01:57:24 airflow-poc airflow[25847]: Traceback (most recent call last):
> Feb 21 01:57:24 airflow-poc airflow[25847]:   File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
> Feb 21 01:57:24 airflow-poc airflow[25847]:     self.run()
> Feb 21 01:57:24 airflow-poc airflow[25847]:   File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
> Feb 21 01:57:24 airflow-poc airflow[25847]:     self._target(*self._args, **self._kwargs)
> Feb 21 01:57:24 airflow-poc airflow[25847]:   File "/usr/local/lib/python3.6/dist-packages/airflow/jobs/scheduler_job.py", line 135, in _run_file_processor
> Feb 21 01:57:24 airflow-poc airflow[25847]:     set_context(log, file_path)
> Feb 21 01:57:24 airflow-poc airflow[25847]:   File "/usr/local/lib/python3.6/dist-packages/airflow/utils/log/logging_mixin.py", line 198, in set_context
> Feb 21 01:57:24 airflow-poc airflow[25847]:     handler.set_context(value)
> Feb 21 01:57:24 airflow-poc airflow[25847]:   File "/usr/local/lib/python3.6/dist-packages/airflow/utils/log/file_processor_handler.py", line 66, in set_context
> Feb 21 01:57:24 airflow-poc airflow[25847]:     self.handler = logging.FileHandler(local_loc)
> Feb 21 01:57:24 airflow-poc airflow[25847]:   File "/usr/lib/python3.6/logging/__init__.py", line 1032, in __init__
> Feb 21 01:57:24 airflow-poc airflow[25847]:     StreamHandler.__init__(self, self._open())
> Feb 21 01:57:24 airflow-poc airflow[25847]:   File "/usr/lib/python3.6/logging/__init__.py", line 1061, in _open
> Feb 21 01:57:24 airflow-poc airflow[25847]:     return open(self.baseFilename, self.mode, encoding=self.encoding)
> Feb 21 01:57:24 airflow-poc airflow[25847]: PermissionError: [Errno 13] Permission denied: '/home/airflow/usr/local/lib/python3.6/dist-packages/airflow/example_dags/example_skip_dag.py.log'{code}
>  
>  
> The same is happening for all example DAGs (haven't loaded any other DAGs yet).
>  
> Airflow scheduler is looking for the logs of the example DAG in `/home/airflow/usr/local/lib/python3.6/dist-packages/airflow/example_dags/example_skip_dag.py.log`. It should instead look in `/usr/local/lib/python3.6/dist-packages/airflow/example_dags/example_skip_dag.py.log`  instead (without the '/home/airflow' prefix).
>  
> In the `dag` table in the database, the DAG with dag_id `example_skip_dag.` has fileloc `/usr/local/lib/python3.6/dist-packages/airflow/example_dags/example_skip_dag.py`.
>  
> Not sure why Airflow scheduler is appending `/home/airflow' to this path
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)