[I] [Bug] Spark on Kubernetes generated pod names can exceed kubelet pod log directory name length [kyuubi]

["ruanwenjun (via GitHub)" <gi...@apache.org>]

ruanwenjun opened a new issue, #7495:
URL: https://github.com/apache/kyuubi/issues/7495

   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   
   ### Search before asking
   
   - [x] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.
   
   ### Describe the bug
   
   When Kyuubi submits Spark applications to Kubernetes, `KubernetesUtils` generates the Spark driver pod name and executor pod name prefix from the Spark application name and engine reference id.
   
   The current length check only considers the Kubernetes pod name limit. For example, the driver pod name is allowed up to 253 characters, and the executor pod name prefix reserves only Spark's executor suffix.
   
   However, kubelet creates the pod log directory with the following format:
   
   ```text
   namespace_podName_podUID
   ```
   
   See Kubernetes' `BuildPodLogsDirectory` implementation:
   https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kuberuntime/helpers.go#L223
   
   As a result, a pod name that is valid for Kubernetes can still make the kubelet log directory name too long after the namespace, separators, and pod UID are appended. In that case, Spark submission can succeed far enough to create the pod, but the kubelet fails when creating the pod log directory on the node because the directory name exceeds the filesystem name length limit.
   
   Kyuubi should take the kubelet pod log directory format into account when generating Spark driver pod names and executor pod name prefixes for Kubernetes.
   
   ### Affects Version(s)
   
   master
   
   ### Kyuubi Server Log Output
   
   ```logtalk
   N/A
   ```
   
   ### Kyuubi Engine Log Output
   
   ```logtalk
   N/A
   ```
   
   ### Kyuubi Server Configurations
   
   ```yaml
   spark.master: k8s://...
   spark.submit.deployMode: cluster
   # The issue is easier to reproduce with a long spark.app.name and/or a long Kubernetes namespace.
   ```
   
   ### Kyuubi Engine Configurations
   
   ```yaml
   N/A
   ```
   
   ### Additional context
   
   The fix should preserve user-specified `spark.kubernetes.driver.pod.name` and `spark.kubernetes.executor.podNamePrefix`, but generated names should leave enough room for the namespace, the two `_` separators, the pod UID, and Spark's executor suffix where applicable.
   
   ### Are you willing to submit PR?
   
   - [x] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org

Re: [I] [Bug] Spark on Kubernetes generated pod names can exceed kubelet pod log directory name length [kyuubi]

["pan3793 (via GitHub)" <gi...@apache.org>]

pan3793 closed issue #7495: [Bug] Spark on Kubernetes generated pod names can exceed kubelet pod log directory name length
URL: https://github.com/apache/kyuubi/issues/7495


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org