You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Daniel Haviv <da...@veracity-group.com> on 2016/07/17 09:33:08 UTC

Spark (on Windows) not picking up HADOOP_CONF_DIR

Hi,
I'm running Spark using IntelliJ on Windows and even though I set
HADOOP_CONF_DIR it does not affect the contents of sc.hadoopConfiguration.

Anybody encountered it ?

Thanks,
Daniel

Re: Spark (on Windows) not picking up HADOOP_CONF_DIR

Posted by Jacek Laskowski <ja...@japila.pl>.

Hi,

How did you set it? How do you run the app? Use sys.env to know whether it
was set or not.

Jacek

On 17 Jul 2016 11:33 a.m., "Daniel Haviv" <da...@veracity-group.com>
wrote:

> Hi,
> I'm running Spark using IntelliJ on Windows and even though I set
> HADOOP_CONF_DIR it does not affect the contents of sc.hadoopConfiguration.
>
> Anybody encountered it ?
>
> Thanks,
> Daniel
>