You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Ash Berlin-Taylor (Jira)" <ji...@apache.org> on 2019/12/12 13:37:00 UTC

[jira] [Resolved] (AIRFLOW-6012) Kubernetes Executer very slow to Clean finished tasks

     [ https://issues.apache.org/jira/browse/AIRFLOW-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ash Berlin-Taylor resolved AIRFLOW-6012.
----------------------------------------
    Resolution: Duplicate

> Kubernetes Executer very slow to Clean finished tasks
> -----------------------------------------------------
>
>                 Key: AIRFLOW-6012
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6012
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: scheduler
>    Affects Versions: 1.10.6
>            Reporter: Bo Blanton
>            Priority: Major
>
> The new clean up task in the K8 executor queries the DB for all running tasks/dags for each tasks to check for safe labels (tasks and dagid) as they may have needed to be altered for k8 safety.
> For hundereds of tasks finishing, this is a very long process (lots of DB calls and a large loop).
> There's an improvement we can make to to quickly exit the loop if we don't need to check for the label saftly



--
This message was sent by Atlassian Jira
(v8.3.4#803005)