tmp directory for Pig

[Jakub Glapa <ja...@gmail.com>]

Hi I started running jobs in pig and I'm getting exceptions like that:
>>Caused by: java.io.IOException: No space left on device
...
I've narrowed down the problem and it looks like the /tmp is to small. I
wanted to set the java.io.tmpdir to point somewhere else like that:

pig -Djava.io.tmpdir=/foo/tmp/ script.pig


but it appears that it's not being picked by pig.
What property could i modify if I want to avoid the need to increase the
size of /tmp folder?

Ps.
pig-0.10.1
running pig scripts with python embedded if it makes any difference

--
regards,
pozdrawiam,
Jakub Glapa

Re: tmp directory for Pig

[Rohini Palaniswamy <ro...@gmail.com>]

java.io.tmpdir should work. If you are running python script, more space is
required as the jars will cached in the java.io.tmp dir. Alternatively, you
can try a different location for it by using -Dpython.cachedir=<location>
or you can skip the caching using -Dpython.cachedir.skip=true.  My guess is
that /foo/tmp also does not have space that is why java.io.tmpdir did not
take effect. Check the space availability using df.

Regards,
Rohini


On Fri, Jan 25, 2013 at 7:36 AM, Jakub Glapa <ja...@gmail.com> wrote:

> Hi I started running jobs in pig and I'm getting exceptions like that:
> >>Caused by: java.io.IOException: No space left on device
> ...
> I've narrowed down the problem and it looks like the /tmp is to small. I
> wanted to set the java.io.tmpdir to point somewhere else like that:
>
> pig -Djava.io.tmpdir=/foo/tmp/ script.pig
>
>
> but it appears that it's not being picked by pig.
> What property could i modify if I want to avoid the need to increase the
> size of /tmp folder?
>
> Ps.
> pig-0.10.1
> running pig scripts with python embedded if it makes any difference
>
> --
> regards,
> pozdrawiam,
> Jakub Glapa
>