You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "getaaron (via GitHub)" <gi...@apache.org> on 2023/08/01 15:02:43 UTC

[GitHub] [airflow] getaaron commented on issue #32986: 🧟 Slow Zombie query causes scheduler heartbeat failures without errors

getaaron commented on issue #32986:
URL: https://github.com/apache/airflow/issues/32986#issuecomment-1660507268

   I hope it's not a tall order, although I'm not familiar with Airflow's database code. Most database client libraries and ORMs have the ability to set timeouts on their SQL queries. A simple approach could be to add a default 60 second timeout in the session:
   
   ```sql
   SET statement_timeout = ā€˜60s’;
   ```
   
   This could be overridden if there are queries that are expected to run long, and could be configurable via an Airflow environment variable.
   
   I think it's an Airflow problem because:
   
   1. The Airflow scheduler should be designed not to hang when a database (or other dependency) is slow to respond
   2. Airflow supports health check functionality, which should be designed to be useful. The health check functionality is much less helpful if Airflow gives no indication about why the health check has failed, when it reasonably could.
   
   To be clear, I'm not asking Airflow to diagnose the problem with the database, simply to emit a log which points to the database as a culprit.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org