[GitHub] [airflow] StewartThomson edited a comment on issue #17771: Investigating expensive query when a dag has a large amount of tasks.

[GitBox <gi...@apache.org>]

StewartThomson edited a comment on issue #17771:
URL: https://github.com/apache/airflow/issues/17771#issuecomment-903184278


   Thanks for your quick response. It's mainly an issue in that it causes our database instance to spike to 100% every time the dag runs. Throw in any other dags and we end up in a situation where the CPU gets pinned at 100% and performance degrades. 
   
   https://imgur.com/gnhQTXJ
   
   Attached is a link of the RDS performance insight that shows this query as the main culprit. We can see that it gets called 20 times per second.
   
   The motivation behind wanting to investigate this is that if it is resolved, we can reduce the hardware requirements. The expense of this query also gives me concerns about scalability in the future.
   
   Thanks again for your response.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org