You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/05/14 13:39:16 UTC
[GitHub] [airflow] mik-laj commented on pull request #7597: [AIRFLOW-6497] Avoid loading DAGs in the main scheduler loop
mik-laj commented on pull request #7597:
URL: https://github.com/apache/airflow/pull/7597#issuecomment-628641784
1. For executor events
https://github.com/PolideaInternal/airflow/blob/99d237b684bbcdcf808451dfbfad425c0c7f772b/airflow/jobs/scheduler_job.py#L1430
Send to DagFileProcessorManager via DagFileProcesorAgent
https://github.com/PolideaInternal/airflow/blob/99d237b684bbcdcf808451dfbfad425c0c7f772b/airflow/jobs/scheduler_job.py#L1464-L1469
https://github.com/PolideaInternal/airflow/blob/99d237b684bbcdcf808451dfbfad425c0c7f772b/airflow/utils/dag_processing.py#L670-L673
Send to queue
https://github.com/PolideaInternal/airflow/blob/99d237b684bbcdcf808451dfbfad425c0c7f772b/airflow/utils/dag_processing.py#L737-L742
Start a new DagFileProcessor via DagFileProcessorProcess
https://github.com/PolideaInternal/airflow/blob/99d237b684bbcdcf808451dfbfad425c0c7f772b/airflow/utils/dag_processing.py#L1045-L1057
Execute callback
https://github.com/PolideaInternal/airflow/blob/99d237b684bbcdcf808451dfbfad425c0c7f772b/airflow/jobs/scheduler_job.py#L835-L838
2. For zombies
Find zombies
https://github.com/PolideaInternal/airflow/blob/99d237b684bbcdcf808451dfbfad425c0c7f772b/airflow/utils/dag_processing.py#L670-L673
Send to queue
https://github.com/PolideaInternal/airflow/blob/99d237b684bbcdcf808451dfbfad425c0c7f772b/airflow/utils/dag_processing.py#L1139
Start a new DagFileProcessor via DagFileProcessorProcess
https://github.com/PolideaInternal/airflow/blob/99d237b684bbcdcf808451dfbfad425c0c7f772b/airflow/utils/dag_processing.py#L1045-L1057
Execute callback
https://github.com/PolideaInternal/airflow/blob/99d237b684bbcdcf808451dfbfad425c0c7f772b/airflow/jobs/scheduler_job.py#L835-L838
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org