[jira] [Comment Edited] (AIRFLOW-6013) Last heartbeat check does not account for execution time of session.commit()

["Oliver Frost (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16979290#comment-16979290 ] 

Oliver Frost edited comment on AIRFLOW-6013 at 11/21/19 1:57 PM:
-----------------------------------------------------------------

Just filed the same issue. I think an additional sleep time in the LocalTaskJob is fine, but should be reverted to something like the logic before commit 68b8ec5f4 [1], without interferring with the desired self-termination.

[1] https://github.com/apache/airflow/commit/68b8ec5f415795e4fa4ff7df35a3e75c712a7bad


was (Author: ofrost):
Just wanted to file the same issue. I think an additional sleep time in the LocalTaskJob is fine, but should be reverted to something like the logic before commit 68b8ec5f4 [1], without interferring with the desired self-termination.

[1] https://github.com/apache/airflow/commit/68b8ec5f415795e4fa4ff7df35a3e75c712a7bad

> Last heartbeat check does not account for execution time of session.commit()
> ----------------------------------------------------------------------------
>
>                 Key: AIRFLOW-6013
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6013
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: worker
>    Affects Versions: 1.10.6
>            Reporter: Alex B
>            Priority: Minor
>
> Assuming the last hearbeat is not older than the heartbeat_time_limit, this warning will Always fire:
> [https://github.com/apache/airflow/blob/1.10.6/airflow/jobs/local_task_job.py#L120]
> There's a few commands between:
> [https://github.com/apache/airflow/blob/1.10.6/airflow/jobs/base_job.py#L195]
> and
> [https://github.com/apache/airflow/blob/1.10.6/airflow/jobs/local_task_job.py#L111]
> so _(timezone.utcnow() - self.latest_heartbeat).total_seconds()_ will always be some small but non-0 number.
>  
> We get many log warnings in our task-logs similar to:
> {code:java}
> WARNING - Time since last heartbeat(0.01 s) < heartrate(5.0 s), sleeping for 4.991735 s{code}
>  
> Does local_task_job need the extra check on last_heartbeat?
> [https://github.com/apache/airflow/blob/1.10.6/airflow/jobs/local_task_job.py#L121]
> Since base_job is already making sure to sleep through the gap:
> [https://github.com/apache/airflow/blob/1.10.6/airflow/jobs/base_job.py#L187]
> ?
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)