You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Eric Wang <er...@gmail.com> on 2018/04/30 18:51:38 UTC
[Spark on Google Kubernetes Engine] Properties File Error
Hello all,
I've been trying to spark-submit a job to the Google Kubernetes Engine but
I keep encountering a "Exception in thread "main"
java.lang.IllegalArgumentException:
Server properties file given at /opt/spark/work-dir/driver does not exist
or is not a file."
error. I'm unsure of how to even begin debugging this so any help would be
greatly appreciated. I've attached the logs and the full spark-submit
command I'm running here: https://gist.github.com/erkkel/
c04a0b5ca60ad755cf62e9ad18e5b7ed
For reference, I've been following this guide:
https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html
Thanks,
Eric
Re: [Spark on Google Kubernetes Engine] Properties File Error
Posted by Eric Wang <er...@gmail.com>.
Thanks so much! I'll take a look at the guide right now. The versions
should all be 2.2 of spark. In my configuration, I'm using
--conf
spark.kubernetes.driver.docker.image=kubespark/spark-driver:v2.2.0-kubernetes-0.5.0
\
--conf
spark.kubernetes.executor.docker.image=kubespark/spark-executor:v2.2.0-kubernetes-0.5.0
\
and I built the version of spark located here:
https://github.com/apache-spark-on-k8s/spark which also seems to be 2.2
On Mon, Apr 30, 2018 at 12:09 PM, Yinan Li <li...@gmail.com> wrote:
> Which version of Spark are you using to run spark-submit, and which
> version of Spark your container image is based off? This looks to be caused
> my mismatched versions of Spark used for spark-submit and for the
> driver/executor at runtime.
>
> On Mon, Apr 30, 2018 at 12:00 PM, Holden Karau <ho...@pigscanfly.ca>
> wrote:
>
>> So, while its not perfect, I have a guide focused on running custom Spark
>> on GKE https://cloud.google.com/blog/big-data/2018/03/testing-f
>> uture-apache-spark-releases-and-changes-on-google-kubernetes
>> -engine-and-cloud-dataproc and if you want to run pre-built Spark on GKE
>> there is a solutions article - https://cloud.google.com/sol
>> utions/spark-on-kubernetes-engine which could be relevant.
>>
>> On Mon, Apr 30, 2018 at 7:51 PM, Eric Wang <er...@gmail.com>
>> wrote:
>>
>>> Hello all,
>>>
>>> I've been trying to spark-submit a job to the Google Kubernetes Engine
>>> but I keep encountering a "Exception in thread "main"
>>> java.lang.IllegalArgumentException: Server properties file given at
>>> /opt/spark/work-dir/driver does not exist or is not a file."
>>> error. I'm unsure of how to even begin debugging this so any help would
>>> be greatly appreciated. I've attached the logs and the full spark-submit
>>> command I'm running here: https://gist.github.com/
>>> erkkel/c04a0b5ca60ad755cf62e9ad18e5b7ed
>>>
>>> For reference, I've been following this guide: https://apache-spark-on
>>> -k8s.github.io/userdocs/running-on-kubernetes.html
>>>
>>> Thanks,
>>> Eric
>>>
>>>
>>
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>>
>
>
Re: [Spark on Google Kubernetes Engine] Properties File Error
Posted by Yinan Li <li...@gmail.com>.
Also looks like you are mixing configuration properties from different
versions of Spark on Kubernetes.
"spark.kubernetes.{driver|executor}.docker.image" is only available in the
apache-spark-on-k8s fork, whereas "spark.kubernetes.container.image" is new
in Spark 2.3.0. Please make sure you use the same version of Spark to run
spark-submit and to build your image. You also need to make sure that you
use the right properties for the version of Spark you use. BTW: we suggest
people to use the official Spark 2.3.0 release instead of the fork. The
guide is at http://spark.apache.org/docs/latest/running-on-kubernetes.html.
On Mon, Apr 30, 2018 at 12:09 PM, Yinan Li <li...@gmail.com> wrote:
> Which version of Spark are you using to run spark-submit, and which
> version of Spark your container image is based off? This looks to be caused
> my mismatched versions of Spark used for spark-submit and for the
> driver/executor at runtime.
>
> On Mon, Apr 30, 2018 at 12:00 PM, Holden Karau <ho...@pigscanfly.ca>
> wrote:
>
>> So, while its not perfect, I have a guide focused on running custom Spark
>> on GKE https://cloud.google.com/blog/big-data/2018/03/testing-f
>> uture-apache-spark-releases-and-changes-on-google-kubernetes
>> -engine-and-cloud-dataproc and if you want to run pre-built Spark on GKE
>> there is a solutions article - https://cloud.google.com/sol
>> utions/spark-on-kubernetes-engine which could be relevant.
>>
>> On Mon, Apr 30, 2018 at 7:51 PM, Eric Wang <er...@gmail.com>
>> wrote:
>>
>>> Hello all,
>>>
>>> I've been trying to spark-submit a job to the Google Kubernetes Engine
>>> but I keep encountering a "Exception in thread "main"
>>> java.lang.IllegalArgumentException: Server properties file given at
>>> /opt/spark/work-dir/driver does not exist or is not a file."
>>> error. I'm unsure of how to even begin debugging this so any help would
>>> be greatly appreciated. I've attached the logs and the full spark-submit
>>> command I'm running here: https://gist.github.com/
>>> erkkel/c04a0b5ca60ad755cf62e9ad18e5b7ed
>>>
>>> For reference, I've been following this guide: https://apache-spark-on
>>> -k8s.github.io/userdocs/running-on-kubernetes.html
>>>
>>> Thanks,
>>> Eric
>>>
>>>
>>
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>>
>
>
Re: [Spark on Google Kubernetes Engine] Properties File Error
Posted by Yinan Li <li...@gmail.com>.
Which version of Spark are you using to run spark-submit, and which
version of Spark your container image is based off? This looks to be caused
my mismatched versions of Spark used for spark-submit and for the
driver/executor at runtime.
On Mon, Apr 30, 2018 at 12:00 PM, Holden Karau <ho...@pigscanfly.ca> wrote:
> So, while its not perfect, I have a guide focused on running custom Spark
> on GKE https://cloud.google.com/blog/big-data/2018/03/testing-
> future-apache-spark-releases-and-changes-on-google-
> kubernetes-engine-and-cloud-dataproc and if you want to run pre-built
> Spark on GKE there is a solutions article - https://cloud.google.com/
> solutions/spark-on-kubernetes-engine which could be relevant.
>
> On Mon, Apr 30, 2018 at 7:51 PM, Eric Wang <er...@gmail.com>
> wrote:
>
>> Hello all,
>>
>> I've been trying to spark-submit a job to the Google Kubernetes Engine
>> but I keep encountering a "Exception in thread "main"
>> java.lang.IllegalArgumentException: Server properties file given at
>> /opt/spark/work-dir/driver does not exist or is not a file."
>> error. I'm unsure of how to even begin debugging this so any help would
>> be greatly appreciated. I've attached the logs and the full spark-submit
>> command I'm running here: https://gist.github.com/
>> erkkel/c04a0b5ca60ad755cf62e9ad18e5b7ed
>>
>> For reference, I've been following this guide: https://apache-spark-on
>> -k8s.github.io/userdocs/running-on-kubernetes.html
>>
>> Thanks,
>> Eric
>>
>>
>
>
> --
> Twitter: https://twitter.com/holdenkarau
>
Re: [Spark on Google Kubernetes Engine] Properties File Error
Posted by Holden Karau <ho...@pigscanfly.ca>.
So, while its not perfect, I have a guide focused on running custom Spark
on GKE
https://cloud.google.com/blog/big-data/2018/03/testing-future-apache-spark-releases-and-changes-on-google-kubernetes-engine-and-cloud-dataproc
and
if you want to run pre-built Spark on GKE there is a solutions article -
https://cloud.google.com/solutions/spark-on-kubernetes-engine which could
be relevant.
On Mon, Apr 30, 2018 at 7:51 PM, Eric Wang <er...@gmail.com> wrote:
> Hello all,
>
> I've been trying to spark-submit a job to the Google Kubernetes Engine but
> I keep encountering a "Exception in thread "main"
> java.lang.IllegalArgumentException: Server properties file given at
> /opt/spark/work-dir/driver does not exist or is not a file."
> error. I'm unsure of how to even begin debugging this so any help would be
> greatly appreciated. I've attached the logs and the full spark-submit
> command I'm running here: https://gist.github.com/
> erkkel/c04a0b5ca60ad755cf62e9ad18e5b7ed
>
> For reference, I've been following this guide: https://apache-spark-
> on-k8s.github.io/userdocs/running-on-kubernetes.html
>
> Thanks,
> Eric
>
>
--
Twitter: https://twitter.com/holdenkarau