You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Nan Zhu <zh...@gmail.com> on 2014/01/07 07:13:48 UTC
is it forgotten to document how to set SPARK_WORKER_DIR?
Hi, all
I’m trying to change my worker dir to a mounted disk with larger space
but I found that no document telling me how to do this,
I have to check the source code and found the following lines
if (System.getenv("SPARK_WORKER_DIR") != null) {
workDir = System.getenv("SPARK_WORKER_DIR")
}
is it forgotten to be documented?
Best,
--
Nan Zhu
Re: is it forgotten to document how to set SPARK_WORKER_DIR?
Posted by Nan Zhu <zh...@gmail.com>.
In worker machines, you will find $SPARK_HOME/work/
that’s worker dir
--
Nan Zhu
On Tuesday, January 7, 2014 at 7:29 AM, Archit Thakur wrote:
> What do you mean by worker dir?
>
>
> On Tue, Jan 7, 2014 at 11:43 AM, Nan Zhu <zhunanmcgill@gmail.com (mailto:zhunanmcgill@gmail.com)> wrote:
> > Hi, all
> >
> > I’m trying to change my worker dir to a mounted disk with larger space
> >
> > but I found that no document telling me how to do this,
> >
> > I have to check the source code and found the following lines
> >
> > if (System.getenv("SPARK_WORKER_DIR") != null) {
> > workDir = System.getenv("SPARK_WORKER_DIR")
> > }
> >
> >
> > is it forgotten to be documented?
> >
> > Best,
> >
> > --
> > Nan Zhu
> >
> >
>
>
>
Re: is it forgotten to document how to set SPARK_WORKER_DIR?
Posted by Archit Thakur <ar...@gmail.com>.
What do you mean by worker dir?
On Tue, Jan 7, 2014 at 11:43 AM, Nan Zhu <zh...@gmail.com> wrote:
> Hi, all
>
> I’m trying to change my worker dir to a mounted disk with larger space
>
> but I found that no document telling me how to do this,
>
> I have to check the source code and found the following lines
>
> if (System.getenv("SPARK_WORKER_DIR") != null) {
> workDir = System.getenv("SPARK_WORKER_DIR")
> }
>
> is it forgotten to be documented?
>
> Best,
>
> --
> Nan Zhu
>