You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "raman (JIRA)" <ji...@apache.org> on 2017/12/17 08:00:02 UTC
[jira] [Updated] (AIRFLOW-1934) Unable to Launch Example DAG if
~/AIRFLOW_HOME/dags folder is empty
[ https://issues.apache.org/jira/browse/AIRFLOW-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
raman updated AIRFLOW-1934:
---------------------------
Description:
Steps to reproduce
1. Install airflow
2. Keep the ~/{airflow_home}/dags folder empty
3. airflow initdb
4. airflow webserver and scheduler
2. Enable a example DAG and trigger it manually from web UI.
Result: DAG run gets created in the dag_run table. task_instance table also get relevant enteries but scheduler does not pick the DAG.
Workaround: Create one sample dag in the ~/{airflow_home}/dags folder and scheduler picks it up.
The following code in jobs.py seems to be doing the trick but this code is only triggered if there is a dag inside ~/{airflow_home}/dags folder
File: jobs.py
Function: _find_executable_task_instances
ti_query = (
session
.query(TI)
.filter(TI.dag_id.in_(simple_dag_bag.dag_ids))
.outerjoin(DR,
and_(DR.dag_id == TI.dag_id,
DR.execution_date == TI.execution_date))
.filter(or_(DR.run_id == None,
not_(DR.run_id.like(BackfillJob.ID_PREFIX + '%'))))
.outerjoin(DM, DM.dag_id==TI.dag_id)
.filter(or_(DM.dag_id == None,
not_(DM.is_paused)))
)
was:
Steps to reproduce
1. Install airflow
2. Keep the ~/{airflow_home}/dags folder empty
3. airflow initdb
4. airflow webserver and scheduler
2. Enable a DAG and trigger it manually from web UI.
Result: DAG run gets created in the dag_run table. task_instance table also get relevant enteries but scheduler does not pick the DAG.
Workaround: Create one sample dag in the ~/{airflow_home}/dags folder and scheduler picks it up.
The following code in jobs.py seems to be doing the trick but this code is only triggered if there is a dag inside ~/{airflow_home}/dags folder
Function: _find_executable_task_instances
ti_query = (
session
.query(TI)
.filter(TI.dag_id.in_(simple_dag_bag.dag_ids))
.outerjoin(DR,
and_(DR.dag_id == TI.dag_id,
DR.execution_date == TI.execution_date))
.filter(or_(DR.run_id == None,
not_(DR.run_id.like(BackfillJob.ID_PREFIX + '%'))))
.outerjoin(DM, DM.dag_id==TI.dag_id)
.filter(or_(DM.dag_id == None,
not_(DM.is_paused)))
)
> Unable to Launch Example DAG if ~/AIRFLOW_HOME/dags folder is empty
> -------------------------------------------------------------------
>
> Key: AIRFLOW-1934
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1934
> Project: Apache Airflow
> Issue Type: Bug
> Components: DAG
> Affects Versions: Airflow 1.8
> Environment: RHEL
> Reporter: raman
>
> Steps to reproduce
> 1. Install airflow
> 2. Keep the ~/{airflow_home}/dags folder empty
> 3. airflow initdb
> 4. airflow webserver and scheduler
> 2. Enable a example DAG and trigger it manually from web UI.
> Result: DAG run gets created in the dag_run table. task_instance table also get relevant enteries but scheduler does not pick the DAG.
> Workaround: Create one sample dag in the ~/{airflow_home}/dags folder and scheduler picks it up.
> The following code in jobs.py seems to be doing the trick but this code is only triggered if there is a dag inside ~/{airflow_home}/dags folder
> File: jobs.py
> Function: _find_executable_task_instances
> ti_query = (
> session
> .query(TI)
> .filter(TI.dag_id.in_(simple_dag_bag.dag_ids))
> .outerjoin(DR,
> and_(DR.dag_id == TI.dag_id,
> DR.execution_date == TI.execution_date))
> .filter(or_(DR.run_id == None,
> not_(DR.run_id.like(BackfillJob.ID_PREFIX + '%'))))
> .outerjoin(DM, DM.dag_id==TI.dag_id)
> .filter(or_(DM.dag_id == None,
> not_(DM.is_paused)))
> )
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)