[jira] [Assigned] (AURORA-1893) Add ability to prune tasks using aurora_admin

["David McLaughlin (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AURORA-1893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David McLaughlin reassigned AURORA-1893:
----------------------------------------

    Assignee: David McLaughlin

> Add ability to prune tasks using aurora_admin
> ---------------------------------------------
>
>                 Key: AURORA-1893
>                 URL: https://issues.apache.org/jira/browse/AURORA-1893
>             Project: Aurora
>          Issue Type: Task
>            Reporter: David McLaughlin
>            Assignee: David McLaughlin
>
> Currently task history pruning is configured with three main settings: maximum terminal tasks per job, maximum time to retain terminal tasks and a minimum time to retain terminal tasks.
> There are times where a combination of bad actors in the cluster and the minimum time to retain terminal tasks can lead to incredibly bloated task store sizes, leading to serious problems with GC pressure during task store queries, and also when creating and persisting snapshots. 
> At Twitter we've run into this and have had to respond by redeploying the Scheduler with more aggressive task pruning settings - which affects every user in the cluster. 
> What we'd like is an endpoint in aurora_admin that accepts a TaskQuery and will prune all inactive tasks that match. This should allow us to limit the pruning by role, environment and also limit the number of tasks pruned. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)