You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "German Schiavon Matteo (Jira)" <ji...@apache.org> on 2020/10/05 05:42:00 UTC
[jira] [Commented] (SPARK-31800) Unable to disable Kerberos when
submitting jobs to Kubernetes
[ https://issues.apache.org/jira/browse/SPARK-31800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17207827#comment-17207827 ]
German Schiavon Matteo commented on SPARK-31800:
------------------------------------------------
[~drahkar] [~jagadeesh.n] [~devaraj] [~holdenkarau]
Just to summarize, the issue is when testing spark in *local environment* and setting the k8s property in k8s
{code:java}
spark.kubernetes.file.upload.path
{code}
It uploads the jar in the *local workstation/machine* , not in the docker, hence the driver fails because it is *unable* to find the jar file, right?
While it could seem like a bug now I think is the expected behaviour?
{code:java}
spark.kubernetes.file.upload.path
{code}
I don't think the property is expected to upload the jar inside the docker. It uploads it to s3 or an HTTP server, and then your driver downloads it from there.
Let me know what do you think.
> Unable to disable Kerberos when submitting jobs to Kubernetes
> -------------------------------------------------------------
>
> Key: SPARK-31800
> URL: https://issues.apache.org/jira/browse/SPARK-31800
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 3.0.0
> Reporter: James Boylan
> Priority: Major
>
> When you attempt to submit a process to Kubernetes using spark-submit through --master, it returns the exception:
> {code:java}
> 20/05/22 20:25:54 INFO KerberosConfDriverFeatureStep: You have not specified a krb5.conf file locally or via a ConfigMap. Make sure that you have the krb5.conf locally on the driver image.
> Exception in thread "main" org.apache.spark.SparkException: Please specify spark.kubernetes.file.upload.path property.
> at org.apache.spark.deploy.k8s.KubernetesUtils$.uploadFileUri(KubernetesUtils.scala:290)
> at org.apache.spark.deploy.k8s.KubernetesUtils$.$anonfun$uploadAndTransformFileUris$1(KubernetesUtils.scala:246)
> at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
> at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
> at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
> at scala.collection.TraversableLike.map(TraversableLike.scala:238)
> at scala.collection.TraversableLike.map$(TraversableLike.scala:231)
> at scala.collection.AbstractTraversable.map(Traversable.scala:108)
> at org.apache.spark.deploy.k8s.KubernetesUtils$.uploadAndTransformFileUris(KubernetesUtils.scala:245)
> at org.apache.spark.deploy.k8s.features.BasicDriverFeatureStep.$anonfun$getAdditionalPodSystemProperties$1(BasicDriverFeatureStep.scala:165)
> at scala.collection.immutable.List.foreach(List.scala:392)
> at org.apache.spark.deploy.k8s.features.BasicDriverFeatureStep.getAdditionalPodSystemProperties(BasicDriverFeatureStep.scala:163)
> at org.apache.spark.deploy.k8s.submit.KubernetesDriverBuilder.$anonfun$buildFromFeatures$3(KubernetesDriverBuilder.scala:60)
> at scala.collection.LinearSeqOptimized.foldLeft(LinearSeqOptimized.scala:126)
> at scala.collection.LinearSeqOptimized.foldLeft$(LinearSeqOptimized.scala:122)
> at scala.collection.immutable.List.foldLeft(List.scala:89)
> at org.apache.spark.deploy.k8s.submit.KubernetesDriverBuilder.buildFromFeatures(KubernetesDriverBuilder.scala:58)
> at org.apache.spark.deploy.k8s.submit.Client.run(KubernetesClientApplication.scala:98)
> at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$4(KubernetesClientApplication.scala:221)
> at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$4$adapted(KubernetesClientApplication.scala:215)
> at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:2539)
> at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.run(KubernetesClientApplication.scala:215)
> at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.start(KubernetesClientApplication.scala:188)
> at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:928)
> at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
> at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> 20/05/22 20:25:54 INFO ShutdownHookManager: Shutdown hook called
> 20/05/22 20:25:54 INFO ShutdownHookManager: Deleting directory /private/var/folders/p1/y24myg413wx1l1l52bsdn2hr0000gq/T/spark-c94db9c5-b8a8-414d-b01d-f6369d31c9b8 {code}
> No changes in settings appear to be able to disable Kerberos. This is when running a simple execution of the SparkPi on our lab cluster. The command being used is
> {code:java}
> ./bin/spark-submit --master k8s://https://{api_hostname} --deploy-mode cluster --name spark-test --class org.apache.spark.examples.SparkPi --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark --conf spark.kubernetes.namespace=spark-jobs --conf spark.executor.instances=5 --conf spark.kubernetes.container.image={docker_registry}/spark:spark-3-test /opt/spark/examples/jars/spark-examples_2.12-3.0.0-preview2.jar{code}
> It is important to note that this same command, when run on Spark 2.4.5 works flawlessly once all of the RBAC accounts are properly setup in Kubernetes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org