spark-ec2 can't initialize spark-standalone module

[Nicholas Chammas <ni...@gmail.com>]

This line
<https://github.com/mesos/spark-ec2/blob/bd2b0bd89f4fe42be51b3b6431f04ca6823b43dd/setup.sh#L109>
in setup.sh initializes several modules, which are defined here
<https://github.com/apache/spark/blob/bc4418727b40c9b6ba5194ead6e2698539272280/ec2/spark_ec2.py#L575>
.

# Install / Init module
for module in $MODULES; do
  echo "Initializing $module"
  if [[ -e $module/init.sh ]]; then
    source $module/init.sh
  fi
  cd /root/spark-ec2  # guard against init.sh changing the cwd
done

One of these modules is spark-standalone. However, it does not have an
init.sh file
<https://github.com/mesos/spark-ec2/tree/bd2b0bd89f4fe42be51b3b6431f04ca6823b43dd/spark-standalone>
.

Should it have one? It’s the only module without an init.sh.

Nick
​

Re: spark-ec2 can't initialize spark-standalone module

[Shivaram Venkataraman <sh...@eecs.berkeley.edu>]

There is a check to see if init.sh file exists (` if [[ -e
$module/init.sh ]]; then`), so it just won't get called. Regarding
spark-standalone not having a init.sh that is because we dont have any
initialization work to do for it  (its not necessary for all modules
to have a init.sh) as the spark module downloads and installs Spark.

Thanks
Shivaram

On Wed, Oct 8, 2014 at 2:50 PM, Nicholas Chammas
<ni...@gmail.com> wrote:
> This line
> <https://github.com/mesos/spark-ec2/blob/bd2b0bd89f4fe42be51b3b6431f04ca6823b43dd/setup.sh#L109>
> in setup.sh initializes several modules, which are defined here
> <https://github.com/apache/spark/blob/bc4418727b40c9b6ba5194ead6e2698539272280/ec2/spark_ec2.py#L575>
> .
>
> # Install / Init module
> for module in $MODULES; do
>   echo "Initializing $module"
>   if [[ -e $module/init.sh ]]; then
>     source $module/init.sh
>   fi
>   cd /root/spark-ec2  # guard against init.sh changing the cwd
> done
>
> One of these modules is spark-standalone. However, it does not have an
> init.sh file
> <https://github.com/mesos/spark-ec2/tree/bd2b0bd89f4fe42be51b3b6431f04ca6823b43dd/spark-standalone>
> .
>
> Should it have one? It’s the only module without an init.sh.
>
> Nick
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org