[jira] [Created] (YUNIKORN-3479) [Core] Optimize preemption victim selection and trimming to mitigate over-preemption and collateral waste

["Hedger Lai (Jira)" <ji...@apache.org>]

Hedger Lai created YUNIKORN-3479:
------------------------------------

             Summary: [Core] Optimize preemption victim selection and trimming to mitigate over-preemption and collateral waste
                 Key: YUNIKORN-3479
                 URL: https://issues.apache.org/jira/browse/YUNIKORN-3479
             Project: Apache YuniKorn
          Issue Type: Improvement
          Components: core - scheduler
            Reporter: Hedger Lai
            Assignee: Hedger Lai


h3. Background & Motivation
In {{Preemptor.TryPreemption()}} ({{pkg/scheduler/objects/preemption.go:631-654}}), the final trimming loop filters {{extraVictims}} to fulfill the remaining preemption shortfall. An existing upstream TODO highlights this as an area ripe for enhancement:

{code:go}
// Since there could be more extra victims than the actual need, ensure only required victims are filtered finally
// to do: There is room for improvements especially when there are more victims. victims could be chosen based
// on different criteria. for example, victims could be picked up either from specific node (bin packing) or
// from multiple nodes (fair) given the choices.
{code}

h3. Current Limitations
The current trimming logic performs a naive, order-dependent First-Fit scan that collects any candidate that touches a needed resource dimension:
1. *Unnecessary Over-Preemption*: If a large allocation (e.g. 64 CPU) appears first in {{extraVictims}} when only a small shortfall (e.g. 2 CPU) exists, it is evicted immediately, causing avoidable workload disruption.
2. *Collateral Resource Waste*: The loop ignores unrequested scarce resources (e.g., evicting a pod with [4 CPU, 8 GPU] for a CPU-only demand even when non-GPU alternatives exist).
3. *Redundant Victim Accumulation*: Early selections are never pruned if subsequent allocations satisfy the demand, resulting in non-minimal victim sets.

h3. Proposed Direction
We plan to optimize this final trimming step with the following objectives:
1. *Resource-Aware Selection*: Select victims considering multi-dimensional resource match to decrease unnecessary overshoot and collateral waste.
2. *Redundancy Trimming*: Ensure the final set of evicted allocations is minimal (no redundant victims retained).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org