You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by guojc <gu...@gmail.com> on 2013/11/15 19:57:47 UTC

How to override yarn default java.io.tmpdir and spark.local.dir

Hi,
   How can I override the default java.io.tmpdir and spark.local.dir in
YARN. I had tried to set SPARK_YARN_USER_ENV with SPARK_JAVA_OPTS. It seems
has no effect. The position is still from
YarnConfiguration.DEFAULT_CONTAINER_TEMP_DIR, and it is a very small disk
for me. Any suggestion?

Thanks,
Jiacheng Guo