You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Chen Jin <ka...@gmail.com> on 2014/01/15 20:03:58 UTC

Please help: change $SPARK_HOME/work directory for spark applications

Hi,

Currently my application jars and logs are stored in $SPARK_HOME/work,
I would like to change it to somewhere having more space. Could anyone
advise me on this? Changing the log dir is straightforward which just
to export SPARK_LOG_DIR, however, there is no environment variable for
SPARK_WORK_DIR.

Thanks a lot,

-chen

Re: Please help: change $SPARK_HOME/work directory for spark applications

Posted by Nan Zhu <zh...@gmail.com>.

Hi, Jin   

It’s SPARK_WORKER_DIR

Line 48 WorkerArguments.scala

if (System.getenv("SPARK_WORKER_DIR") != null) {
    workDir = System.getenv("SPARK_WORKER_DIR")
  }


Best,  

--  
Nan Zhu


On Wednesday, January 15, 2014 at 2:03 PM, Chen Jin wrote:

> Hi,
>  
> Currently my application jars and logs are stored in $SPARK_HOME/work,
> I would like to change it to somewhere having more space. Could anyone
> advise me on this? Changing the log dir is straightforward which just
> to export SPARK_LOG_DIR, however, there is no environment variable for
> SPARK_WORK_DIR.
>  
> Thanks a lot,
>  
> -chen