[jira] [Assigned] (SPARK-59758) Release the Kueue `Workload` of a stopped `SparkApplication` regardless of the queue label

["Dongjoon Hyun (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-59758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dongjoon Hyun reassigned SPARK-59758:
-------------------------------------

    Assignee: Peter Toth

> Release the Kueue `Workload` of a stopped `SparkApplication` regardless of the queue label
> ------------------------------------------------------------------------------------------
>
>                 Key: SPARK-59758
>                 URL: https://issues.apache.org/jira/browse/SPARK-59758
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Kubernetes
>    Affects Versions: kubernetes-operator-1.1.0
>            Reporter: Peter Toth
>            Assignee: Peter Toth
>            Priority: Major
>              Labels: pull-request-available
>
> SPARK-59754 removed the `kueue.x-k8s.io/queue-name` label check from the two suspension paths, so a `Workload` created before the label was removed is released. Two release paths in `AppCleanUpStep` still gate on the label and keep the same leak:
> * `AppCleanUpStep` releases the `Workload` of a terminating or restarting attempt only `if (KueueWorkloadFactory.hasQueueName(application))`.
> * `finishKueueWorkloadOfRetainedApp` records the Kueue `Finished` condition only under the same check.
> Measured on an application reaching `Succeeded`, with a mocked `KueueWorkloadUtils`:
> {noformat}
> labeled,   resourceRetainPolicy=Never  -> releaseWorkload called
> unlabeled, resourceRetainPolicy=Never  -> neither called
> labeled,   resourceRetainPolicy=Always -> finishWorkload called
> unlabeled, resourceRetainPolicy=Always -> neither called
> {noformat}
> So an application whose queue label was removed after its `Workload` was admitted keeps that quota until its custom resource is deleted, which with the default `ttlAfterStopMillis` of -1 may be never.
> After SPARK-59754, dropping both checks is safe: `KueueWorkloadUtils.deleteWorkloadOf` already treats a `403 Forbidden` as no `Workload` when the owner has no queue label and the `Workload` informer is disabled, so an operator that was never granted access to `Workload`s is unaffected. `releaseWorkload` swallows every other failure, and `finishWorkload` falls back to `releaseWorkload`.
> The two admission gates, `ClusterInitStep.holdForKueueAdmission` and the equivalent in `AppInitStep`, must keep the label check: an unlabeled resource has no admission to request.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org