Query on entrypoint.sh Kubernetes spark

[Sachit Murarka <co...@gmail.com>]

Hi All,

To run spark on kubernetes . I see following lines in entrypoint.sh script
available

case "$1" in
  driver)
    shift 1
    CMD=(
      "$SPARK_HOME/bin/spark-submit"
      --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS"
      --deploy-mode client

Could you pls suggest Why deploy-mode client is mentioned in entrypoint.sh ?
I am running spark submit using deploy mode cluster but inside
entrypoint.sh which it is mentioned like that.


Kind Regards,
Sachit Murarka

Re: Query on entrypoint.sh Kubernetes spark

[Jacek Laskowski <ja...@japila.pl>]

Hi,

I'm a beginner in Spark on Kubernetes so bear with me and watch out for
possible mistakes :)

The key to understand entrypoint.sh and -deploy-mode client is to think
about the environment where the script is executed in. That's k8s already
where the Docker image is brought to life as a container of a driver pod.
There's no point using cluster deploy mode...ever. Makes sense?

Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
"The Internals Of" Online Books <https://books.japila.pl/>
Follow me on https://twitter.com/jaceklaskowski

<https://twitter.com/jaceklaskowski>


On Thu, Jan 21, 2021 at 10:40 AM Sachit Murarka <co...@gmail.com>
wrote:

> Hi All,
>
> To run spark on kubernetes . I see following lines in entrypoint.sh script
> available
>
> case "$1" in
>   driver)
>     shift 1
>     CMD=(
>       "$SPARK_HOME/bin/spark-submit"
>       --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS"
>       --deploy-mode client
>
> Could you pls suggest Why deploy-mode client is mentioned in entrypoint.sh
> ?
> I am running spark submit using deploy mode cluster but inside
> entrypoint.sh which it is mentioned like that.
>
>
> Kind Regards,
> Sachit Murarka
>