You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "Madhukar525722 (via GitHub)" <gi...@apache.org> on 2024/10/28 15:48:44 UTC
[I] [Improvement] Launching k8s engine pods with respective users [kyuubi]
Madhukar525722 opened a new issue, #6784:
URL: https://github.com/apache/kyuubi/issues/6784
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.
### What would you like to be improved?
Spark submit
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://<k8s_cluster_endpoint>/api/v1/namespaces/genai/pods. Message: Forbidden! User doesn't have permission. pods is forbidden: User "madlnu" cannot create resource "pods" in API group "" in the namespace "genai".
Kyuubi engine launch in share level USER
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://<k8s_cluster_endpoint>/api/v1/namespaces/genai/pods. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. pods is forbidden: User "system:serviceaccount:scaas:spark" cannot create resource "pods" in API group "" in the namespace "genai".
**When an USER engine is launched in k8s cluster, it is taking the user with which kyuubi server is running rather than actual users.**
Configurations:
kyuubi.authentication=KERBEROS
kyuubi.spnego.keytab=spnego.keytab
kyuubi.spnego.principal=spnego@DOMAIN.COM
kyuubi.kinit.principal=hive@DOMAIN.COM
kyuubi.kinit.keytab=hive.keytab
spark.kubernetes.namespace=genai
kyuubi.kubernetes.master.address=k8s://https://<k8s_cluster_endpoint>
spark.master=k8s://https://<k8s_cluster_endpoint>
kyuubi.kubernetes.namespace=scaas
spark.submit.deployMode=cluster
spark.kubernetes.authenticate.serviceAccountName=spark
spark.kubernetes.authenticate.driver.serviceAccountName=spark
### How should we improve?
Expectation is user authentication should happen, while launching the engine pods.
### Are you willing to submit PR?
- [X] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
- [ ] No. I cannot submit a PR at this time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [Improvement] Launching k8s engine pods with respective users [kyuubi]
Posted by "Madhukar525722 (via GitHub)" <gi...@apache.org>.
Madhukar525722 commented on issue #6784:
URL: https://github.com/apache/kyuubi/issues/6784#issuecomment-2449123013
Hi @pan3793 , Here are some of the context
1. Official documentation - https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/
2. Medium - https://blog.devgenius.io/k8s-for-data-engineers-admission-controller-371758f90107
3. Applications like apache yunikorn uses admission controller for their use case
Ultimately, we can achieve this by mutating the owner by real user.
Sure, I will try to create a demo on a minikube cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [Improvement] Launching k8s engine pods with respective users [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #6784:
URL: https://github.com/apache/kyuubi/issues/6784#issuecomment-2448903467
I am not familiar with this area, could you provide some docs/blogs to describe this solution, and is it possible to demonstrate this solution in a minikube (our CI runs on minikube)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [Improvement] Launching k8s engine pods with respective users [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #6784:
URL: https://github.com/apache/kyuubi/issues/6784#issuecomment-2443041923
A good question, in the Spark on YARN case, we leverage the Hadoop user impersonate mechanism to avoid managing all users' keytab, how do you manage the credentials of K8s for all users?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [Improvement] Launching k8s engine pods with respective users [kyuubi]
Posted by "Madhukar525722 (via GitHub)" <gi...@apache.org>.
Madhukar525722 commented on issue #6784:
URL: https://github.com/apache/kyuubi/issues/6784#issuecomment-2447996208
Hi @pan3793, One of the thing that I came across is webhook admission controller, this can be used to achieve impersonation in k8s.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org