You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Vipul Pandey (JIRA)" <ji...@apache.org> on 2019/02/12 09:46:00 UTC
[jira] [Updated] (AIRFLOW-3877) Scheduler sending the absolute path
to celery
[ https://issues.apache.org/jira/browse/AIRFLOW-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vipul Pandey updated AIRFLOW-3877:
----------------------------------
Description:
Hi,
Upgraded the airflow version from 1.7.3 to 1.10.1. After up-gradation of the scheduler, webserver and workers, the dags have stopped working showing below error onscheduler-
{{Either the dag did not exist or it failed to parse.}}
I have not made any changes to the config. While investigating the issue the scheduler logs shows the issue. Earlier the scheduler run the task as -
Adding to queue: airflow run <dag_id> <task_id> <execution_date> --local -sd DAGS_FOLDER/<dag_filename.py>
While now it is running with absolute path -
Adding to queue: airflow run <dag_id> <task_id> <execution_date> --local -sd /<PATH_TO_DAGS_FOLDER>/<dag_filename.py>
PATH_TO_DAGS_FOLDER is like /home/<user>/Airflow/dags...
which is same as what it is pushing it to workers by since worker is running on some other user it is not able to find the dag location specified.
was:
Hi,
Upgraded the airflow version from 1.7.3 to 1.10.1. After up-gradation of the scheduler, webserver and workers the dags have stopped working saying-
{{Either the dag did not exist or it failed to parse.}}
I have not made any changes to the config. While investigating the issue the scheduler logs shows the issue. Earlier the logs run the task as -
Adding to queue: airflow run <dag_id> <task_id> <execution_date> --local -sd DAGS_FOLDER/<dag_filename.py>
While now it is running with absolute path -
Adding to queue: airflow run <dag_id> <task_id> <execution_date> --local -sd /<PATH_TO_DAGS_FOLDER>/<dag_filename.py>
PATH_TO_DAGS_FOLDER is like /home/<user>/Airflow/dags...
which is same as what it is pushing it to workers by since worker is running on some other user it is not able to find the dag location specified.
> Scheduler sending the absolute path to celery
> ---------------------------------------------
>
> Key: AIRFLOW-3877
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3877
> Project: Apache Airflow
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 1.10.1
> Reporter: Vipul Pandey
> Priority: Major
>
> Hi,
> Upgraded the airflow version from 1.7.3 to 1.10.1. After up-gradation of the scheduler, webserver and workers, the dags have stopped working showing below error onscheduler-
> {{Either the dag did not exist or it failed to parse.}}
> I have not made any changes to the config. While investigating the issue the scheduler logs shows the issue. Earlier the scheduler run the task as -
> Adding to queue: airflow run <dag_id> <task_id> <execution_date> --local -sd DAGS_FOLDER/<dag_filename.py>
> While now it is running with absolute path -
> Adding to queue: airflow run <dag_id> <task_id> <execution_date> --local -sd /<PATH_TO_DAGS_FOLDER>/<dag_filename.py>
> PATH_TO_DAGS_FOLDER is like /home/<user>/Airflow/dags...
> which is same as what it is pushing it to workers by since worker is running on some other user it is not able to find the dag location specified.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)