You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@yunikorn.apache.org by "Manikandan R (Jira)" <ji...@apache.org> on 2026/03/18 09:08:00 UTC
[jira] [Resolved] (YUNIKORN-3232) Preemption latency regression
[ https://issues.apache.org/jira/browse/YUNIKORN-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manikandan R resolved YUNIKORN-3232.
------------------------------------
Fix Version/s: 1.9.0
Resolution: Fixed
> Preemption latency regression
> -----------------------------
>
> Key: YUNIKORN-3232
> URL: https://issues.apache.org/jira/browse/YUNIKORN-3232
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: core - scheduler
> Reporter: Victor Zhou
> Assignee: Victor Zhou
> Priority: Major
> Labels: latency, performance, preemption, pull-request-available
> Fix For: 1.9.0
>
>
> *Summary*
> Preemption latency regression in 1.8:
> oversized queue snapshots are cloned twice in calculateVictimsByNode
>
> *Description*
> After upgrading from 1.5 to 1.8, we observed a major preemption latency increase in production-like workloads.
> In our environment:
> * total queues: 400+
> * queues that are actually valid victim candidates per cycle: typically 20–50
> observed preemption latency:
> * ~0.5s on 1.5
> * 10+ seconds on 1.8
> The latency is more noticeable as queue count grows.
>
> *Root Cause*
> The preemption queue snapshot map becomes too large because it includes queues that are already within guaranteed resource limits.
> Those queues should not be targeted as victims, and therefore should not be included in the working snapshot set for victim selection.
> This oversized snapshot map is then cloned twice in calculateVictimsByNode (first and second pass), amplifying overhead and causing significant latency regression at scale.
>
> *Fix / Improvement*
> Prune preemption snapshots to keep only relevant paths:
> * ask queue path and ancestors
> * victim-contributing leaf queues and ancestors
> * Remove leaf snapshots early when queue is within guaranteed limits
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org