You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2026/10/09 23:31:02 UTC
(spark-kubernetes-operator) branch main updated: [SPARK-60087] Add Kueue feature support table to `spark_custom_resources.md`
This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new fa1f279c [SPARK-60087] Add Kueue feature support table to `spark_custom_resources.md`
fa1f279c is described below
commit fa1f279cafa762f578fa46e341cd0b093633eef6
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Sat Oct 10 00:30:53 2026 +0100
[SPARK-60087] Add Kueue feature support table to `spark_custom_resources.md`
### What changes were proposed in this pull request?
This PR aims to add a table to the `Kueue` section of `docs/spark_custom_resources.md` that summarizes which Kueue features are supported. The existing items of the section are unchanged. The table also covers partial admission, Topology Aware Scheduling, MultiKueue and elastic workloads, which the docs did not mention before. None of them are supported.
### Why are the changes needed?
The section has 22 prose items and no table, so readers cannot see at a glance how much of Kueue the integration supports.
### Does this PR introduce _any_ user-facing change?
No. This is a documentation-only change.
### How was this patch tested?
Manual review against the code.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
Closes #960 from dongjoon-hyun/SPARK-60087.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
docs/spark_custom_resources.md | 31 +++++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)
diff --git a/docs/spark_custom_resources.md b/docs/spark_custom_resources.md
index 36270370..f2771ac1 100644
--- a/docs/spark_custom_resources.md
+++ b/docs/spark_custom_resources.md
@@ -651,6 +651,37 @@ spec:
sparkVersion: "4.2.0"
```
+The following table summarizes which Kueue features are supported. The items after it describe the
+behavior in detail, and
+[Kueue Admission and Workload Lifecycle](architecture.md#kueue-admission-and-workload-lifecycle)
+relates it to the reconciliation.
+
+| Feature | Support | Notes |
+|---------------------------------------------------------------------------|---------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Kueue `v1beta2` API | Supported | Kueue v0.20.0 or later is recommended, see [Compatibility](operations.md#compatibility). |
+| Queueing by the `kueue.x-k8s.io/queue-name` label | Supported | Both `SparkApplication` and `SparkCluster` are queued if `spark.kubernetes.operator.kueue.enabled` is set, e.g. by `operatorRbac.kueue.enabled` of the Helm chart, see [Optional Prerequisites](operations.md#optional-prerequisites). |
+| Waiting for the admission | Supported | The resource stays in its initializing state without its driver (or master and workers), and the `KueueAdmissionPending` and `KueueAdmitted` [events](configuration.md#kubernetes-events) report the progress. |
+| Changing the queue label | Supported | A change is followed until the `Workload` reserves quota, while the `ValidatingAdmissionPolicy` of the Helm chart rejects one once the resource started, unless it is `Suspended` or `ScheduledToRestart`. |
+| `Workload` priority | Supported | The `WorkloadPriorityClass` of the `kueue.x-k8s.io/priority-class` label takes precedence over the `priorityClassName` of the pod templates and then the `globalDefault` `PriorityClass`, like Kueue built-in integrations. |
+| `ResourceFlavor` node labels and tolerations | Supported | The `nodeLabels` and `tolerations` of the flavors which Kueue assigns to a pod set are added to its pods, like Kueue built-in integrations. |
+| Admission checks, e.g. `ProvisioningRequest` | Supported | The labels, annotations, node selector and tolerations in the `podSetUpdates` of the checks are added to the pods once Kueue admits the `Workload`. |
+| Preemption and other evictions | Supported | An evicted resource, e.g. a preempted one or one whose `ClusterQueue` is stopped with `HoldAndDrain`, is queued again with a new `Workload`, and a running one is released through `Suspended` first. |
+| `Workload` deactivation, e.g. `kueuectl stop workload` | Supported | A resource waits while its `Workload` is deactivated, and a running one is released through `Suspended` and queued again once the `Workload` is reactivated. |
+| `waitForPodsReady` | Supported | The operator records the `PodsReady` condition once the driver and `spark.executor.instances` executors, or the master and workers, are ready, and handles a `PodsReadyTimeout` like any other eviction. |
+| `spec.suspend` | Supported | It takes precedence over the queueing, so that a suspended resource releases its `Workload`, see [Suspend](#suspend). |
+| Releasing the quota of an application attempt | Supported | The `Workload` is deleted once the driver and executor pods of the attempt are gone, or gets the `Finished` condition when a `Succeeded`, `Failed` or `DriverEvicted` application [retains its resources](#delete-resources-on-termination). |
+| Requeuing backoff after an eviction | Partially supported | The resource waits for the backoff which Kueue records, but its new `Workload` starts the requeue count over, so the backoff does not grow and a `backoffLimitCount` above 0 never deactivates it. |
+| `recoveryTimeout` of `waitForPodsReady` | Not supported | The `PodsReady` condition stays `True` when a pod is lost later, e.g. an executor which Spark replaces. |
+| Maximum execution time (`kueue.x-k8s.io/max-exec-time-seconds`) | Not supported | Use the Spark native `spark.driver.timeout` instead, as in [pi-with-driver-timeout.yaml](../examples/pi-with-driver-timeout.yaml). |
+| Partial admission | Not supported | The pod sets of a `Workload` have no `minCount`, so Kueue admits all the requested pods or none. |
+| Topology Aware Scheduling | Not supported | The operator applies neither the topology assignment nor the scheduling gate of Kueue to the pods. |
+| MultiKueue | Not supported | `SparkApplication` and `SparkCluster` have no `spec.managedBy` field, which MultiKueue needs to run them on a worker cluster. |
+| Elastic workloads (`Workload` slices) | Not supported | A resource has one `Workload` whose pod set counts are fixed. |
+| `manageJobsWithoutQueueName` and `managedJobsNamespaceSelector` | Not supported | Kueue does not manage the Spark custom resources itself, so a resource without the `kueue.x-k8s.io/queue-name` label is not queued and starts right away. |
+| Dynamic allocation (`spark.dynamicAllocation.enabled`) | Not supported | Such a `SparkApplication` fails with `SchedulingFailure` instead of being queued. |
+| `SparkCluster` with `minWorkers < maxWorkers` | Not supported | Such a `SparkCluster` fails with `SchedulingFailure` instead of being queued. |
+| Pod template files (`spark.kubernetes.{driver,executor}.podTemplateFile`) | Not supported | A `SparkApplication` whose pod template is set only in such a file fails with `SchedulingFailure`, so set it in the [spec](#pod-template-support) instead. |
+
* The Helm chart sets `spark.kubernetes.operator.kueue.enabled` together with the Kueue RBAC rules
when `operatorRbac.kueue.enabled` is set. Kueue and its `LocalQueue` must exist as well. See
[Optional Prerequisites](operations.md#optional-prerequisites). Without the integration, the
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org