You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by "Thimme Gowda TP (Nokia)" <th...@nokia.com> on 2022/12/23 10:31:31 UTC
spark-submit fails in kubernetes 1.24.x cluster
Hello,
We are facing issue with ingress during spark-submit with kubernetes cluster 1.24.x .
We are using spark 3.3.0 to do spark-submit.
# kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.9", GitCommit:"9710807c82740b9799453677c977758becf0acbb", GitTreeState:"clean", BuildDate:"2022-12-08T10:15:09Z", GoVersion:"go1.18.9", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.9", GitCommit:"9710807c82740b9799453677c977758becf0acbb", GitTreeState:"clean", BuildDate:"2022-12-08T10:08:06Z", GoVersion:"go1.18.9", Compiler:"gc", Platform:"linux/amd64"}
Error:
{"type":"log", "level":"WARN", "time":"2022-12-23T10:04:57.536Z", "timezone":"UTC", "log":"The client is using resource type 'ingresses' with unstable version 'v1beta1'"}
Exception in thread "main" io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://xx.xx.xx.xx:6443/apis/extensions/v1beta1/namespaces/test/ingresses. Message: Not Found.
From spark code we see that spark 3.3.0 is using "<kubernetes-client.version>5.12.2</kubernetes-client.version>"
We also tried changing this to 6.x as in PR : https://github.com/apache/spark/pull/37990/files
Still we facing same issue.
Let us know if spark is tested/working with k8s 1.24.x cluster and what could be the solution to resolve this.
Thanks.
Regards
Thimme Gowda
Re: spark-submit fails in kubernetes 1.24.x cluster
Posted by Saurabh Gulati <sa...@fedex.com.INVALID>.
Hello Thimme,
Your issue is related to https://kubernetes.io/docs/reference/using-api/deprecation-guide/#ingress-v122
Deprecated API Migration Guide | Kubernetes<https://kubernetes.io/docs/reference/using-api/deprecation-guide/#ingress-v122>
As the Kubernetes API evolves, APIs are periodically reorganized or upgraded. When APIs evolve, the old API is deprecated and eventually removed. This page contains information you need to know when migrating from deprecated API versions to newer and more stable API versions. Removed APIs by release v1.29 The v1.29 release will stop serving the following deprecated API versions: Flow control ...
kubernetes.io
You will need to upgrade your endpoint to use the new general available endpoint.
Regards
Saurabh Gulati
________________________________
From: Thimme Gowda TP (Nokia) <th...@nokia.com>
Sent: 23 December 2022 11:31
To: user@spark.apache.org <us...@spark.apache.org>
Subject: [EXTERNAL] spark-submit fails in kubernetes 1.24.x cluster
Caution! This email originated outside of FedEx. Please do not open attachments or click links from an unknown or suspicious origin.
Hello,
We are facing issue with ingress during spark-submit with kubernetes cluster 1.24.x .
We are using spark 3.3.0 to do spark-submit.
# kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.9", GitCommit:"9710807c82740b9799453677c977758becf0acbb", GitTreeState:"clean", BuildDate:"2022-12-08T10:15:09Z", GoVersion:"go1.18.9", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.9", GitCommit:"9710807c82740b9799453677c977758becf0acbb", GitTreeState:"clean", BuildDate:"2022-12-08T10:08:06Z", GoVersion:"go1.18.9", Compiler:"gc", Platform:"linux/amd64"}
Error:
{"type":"log", "level":"WARN", "time":"2022-12-23T10:04:57.536Z", "timezone":"UTC", "log":"The client is using resource type 'ingresses' with unstable version 'v1beta1'"}
Exception in thread "main" io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://xx.xx.xx.xx:6443/apis/extensions/v1beta1/namespaces/test/ingresses<https://urldefense.com/v3/__https://xx.xx.xx.xx:6443/apis/extensions/v1beta1/namespaces/test/ingresses__;!!BL9GA0TyTA!YkATz9TvT8G7FBX1Vox2KJJu9Tz3DV3_InVLgRMsjiNjd88IseiBHRpDzSplA9w5xhtRlNsvUGIaqU57r_hj9bhBaExnLA$>. Message: Not Found.
From spark code we see that spark 3.3.0 is using “<kubernetes-client.version>5.12.2</kubernetes-client.version>”
We also tried changing this to 6.x as in PR : https://github.com/apache/spark/pull/37990/files<https://urldefense.com/v3/__https://github.com/apache/spark/pull/37990/files__;!!BL9GA0TyTA!YkATz9TvT8G7FBX1Vox2KJJu9Tz3DV3_InVLgRMsjiNjd88IseiBHRpDzSplA9w5xhtRlNsvUGIaqU57r_hj9bi-yg9xnQ$>
Still we facing same issue.
Let us know if spark is tested/working with k8s 1.24.x cluster and what could be the solution to resolve this.
Thanks.
Regards
Thimme Gowda