You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2024/08/26 19:52:43 UTC
(spark-kubernetes-operator) branch main updated: [SPARK-49395] Add `8081` port to Worker resource spec
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 80934a8 [SPARK-49395] Add `8081` port to Worker resource spec
80934a8 is described below
commit 80934a8e56d8def53b8c217716db6fdb45bce0c4
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Mon Aug 26 12:52:34 2024 -0700
[SPARK-49395] Add `8081` port to Worker resource spec
### What changes were proposed in this pull request?
This PR aims to add `8081` port to Worker resource spec.
### Why are the changes needed?
Like `Master`'s `8080` port, we can expose it for users.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manual review.
```
$ gradle build buildDockerImage spark-operator-api:relocateGeneratedCRD -x check
$ helm install spark-kubernetes-operator -f build-tools/helm/spark-kubernetes-operator/values.yaml build-tools/helm/spark-kubernetes-operator/
$ kubectl apply -f examples/qa-cluster-with-one-worker.yaml
$ kubectl get pod -l spark-role=worker -oyaml | yq '.items[0].spec.containers[0].ports'
- containerPort: 8081
name: web
protocol: TCP
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #105 from dongjoon-hyun/SPARK-49395.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
.../java/org/apache/spark/k8s/operator/SparkClusterResourceSpec.java | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkClusterResourceSpec.java b/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkClusterResourceSpec.java
index 69fa026..04e5581 100644
--- a/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkClusterResourceSpec.java
+++ b/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkClusterResourceSpec.java
@@ -261,6 +261,10 @@ public class SparkClusterResourceSpec {
+ name
+ "-master-svc:7077 && while /opt/spark/sbin/spark-daemon.sh status "
+ "org.apache.spark.deploy.worker.Worker 1; do sleep 1; done")
+ .addNewPort()
+ .withName("web")
+ .withContainerPort(8081)
+ .endPort()
.endContainer()
.endSpec()
.endTemplate()
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org