[jira] [Created] (AIRFLOW-6013) Last heartbeat check is too precise

["Alex (Jira)" <ji...@apache.org>]

Alex created AIRFLOW-6013:
-----------------------------

             Summary: Last heartbeat check is too precise
                 Key: AIRFLOW-6013
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6013
             Project: Apache Airflow
          Issue Type: Bug
          Components: worker
    Affects Versions: 1.10.6
            Reporter: Alex


Assuming the last hearbeat is not older than the heartbeat_time_limit, this warning will Always fire:

[https://github.com/apache/airflow/blob/1.10.6/airflow/jobs/local_task_job.py#L120]

There's a few commands between:

[https://github.com/apache/airflow/blob/1.10.6/airflow/jobs/base_job.py#L195]

and

[https://github.com/apache/airflow/blob/1.10.6/airflow/jobs/local_task_job.py#L111]

so _(timezone.utcnow() - self.latest_heartbeat).total_seconds()_ will always be some small but non-0 number.

 

We get many log warnings in our task-logs similar to:
{code:java}
WARNING - Time since last heartbeat(0.01 s) < heartrate(5.0 s), sleeping for 4.991735 s{code}
 

Does local_task_job need the extra check on last_heartbeat?

[https://github.com/apache/airflow/blob/1.10.6/airflow/jobs/local_task_job.py#L121]

Since base_job is already making sure to sleep through the gap:

[https://github.com/apache/airflow/blob/1.10.6/airflow/jobs/base_job.py#L187]

?

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)