[GitHub] [airflow] nmehraein edited a comment on pull request #17333: Regression on pid reset to allow task start after heartbeat

[GitBox <gi...@apache.org>]

nmehraein edited a comment on pull request #17333:
URL: https://github.com/apache/airflow/pull/17333#issuecomment-889818940


   > Do you have a dag that could be used to reproduce this behaviour?
   
   @ephraimbuddy IMO it's not a bug that can ben reproduced in a simply dag cause it not concern task execution code but task runner.
   To reproduce we have to slowdown the task starting process:
    - In the task pid update in https://github.com/apache/airflow/blob/main/airflow/models/taskinstance.py#L1180 
    - Or in the task fork from runner after forking https://github.com/apache/airflow/blob/main/airflow/task/task_runner/standard_task_runner.py#L54
    - Or in the raw task command line (executed by scheduler) https://github.com/apache/airflow/blob/main/airflow/cli/commands/task_command.py#L156
   
   If the starting process take more time than heartbeat (5s default), and it's not the first TI run, the problem occur.
   
   I don't know if you have another idea to reproduce that in a dag ?
   
   In my case, it occur more specifically in production at night, when the scheduler process our data of last day, and lot of task is running in parallel that generate load on our servers and our airflow db.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org