You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Bo Blanton (Jira)" <ji...@apache.org> on 2019/11/18 18:56:00 UTC
[jira] [Created] (AIRFLOW-6012) Kubernetes Executer very slow to
Clean finished tasks
Bo Blanton created AIRFLOW-6012:
-----------------------------------
Summary: Kubernetes Executer very slow to Clean finished tasks
Key: AIRFLOW-6012
URL: https://issues.apache.org/jira/browse/AIRFLOW-6012
Project: Apache Airflow
Issue Type: Improvement
Components: scheduler
Affects Versions: 1.10.6
Reporter: Bo Blanton
The new clean up task in the K8 executor queries the DB for all running tasks/dags for each tasks to check for safe labels (tasks and dagid) as they may have needed to be altered for k8 safety.
For hundereds of tasks finishing, this is a very long process (lots of DB calls and a large loop).
There's an improvement we can make to to quickly exit the loop if we don't need to check for the label saftly
--
This message was sent by Atlassian Jira
(v8.3.4#803005)