[GitHub] [airflow] ghostbody edited a comment on issue #19767: dag callbacks are missed sometimes

[GitBox <gi...@apache.org>]

ghostbody edited a comment on issue #19767:
URL: https://github.com/apache/airflow/issues/19767#issuecomment-989709988


   I had a look at the airflow sheduler logs and found this:
   
   ```
   ./logs/dag_processor_manager/dag_processor_manager.log:[2021-12-08 22:15:52,278] {manager.py:1092} ERROR - Processor for some_dag.py with PID 250 started at 2021-12-08T14:15:01.525929+00:00 has timed out, killing it.
   ./logs/dag_processor_manager/dag_processor_manager.log:[2021-12-08 22:16:43,632] {manager.py:1092} ERROR - Processor for some_dag.py with PID 581 started at 2021-12-08T14:15:53.491981+00:00 has timed out, killing it.
   ./logs/dag_processor_manager/dag_processor_manager.log:[2021-12-08 22:17:35,850] {manager.py:1092} ERROR - Processor for some_dag.py with PID 898 started at 2021-12-08T14:16:45.152570+00:00 has timed out, killing it.
   ./logs/dag_processor_manager/dag_processor_manager.log:[2021-12-08 22:18:28,167] {manager.py:1092} ERROR - Processor for some_dag.py with PID 1114 started at 2021-12-08T14:17:37.449065+00:00 has timed out, killing it.
   ```
   
   While dag callbacks are executed by the processor as the source code indicated(in the process_file function), this timeout error could lead to callbacks missed.
   
   I set up this [config](https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#dag-file-processor-timeout) (to 100 seconds or more) and this problem seems to be fixed.
   
   I think it's a not a good idea to execute callback while file procesing. Maybe to have a specialized process to handle callback is better.
   
   
   @jpradass 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org