Add hive-site.xml at runtime

[Shivam Sharma <28...@gmail.com>]

Hi,

I have multiple hive configurations(hive-site.xml) and because of that only
I am not able to add any hive configuration in spark *conf* directory. I
want to add this configuration file at start of any *spark-submit* or
*spark-shell*. This conf file is huge so *--conf* is not a option for me.

Note :- I have tried user@spark.apache.org and my mail was bouncing each
time so Sean Owen suggested to mail dev.(
https://issues.apache.org/jira/browse/SPARK-19546). Please give solution to
above ticket also if possible.

Thanks

-- 
Shivam Sharma

Re: Add hive-site.xml at runtime

[Ryan Blue <rb...@netflix.com.INVALID>]

Shivam,

We add hive-site.xml at runtime. We use --driver-class-path to add it to
the driver and --jars to add it for the executors.

rb

On Sun, Feb 12, 2017 at 9:10 AM, Shivam Sharma <28...@gmail.com>
wrote:

> Hi,
>
> I have multiple hive configurations(hive-site.xml) and because of that
> only I am not able to add any hive configuration in spark *conf* directory.
> I want to add this configuration file at start of any *spark-submit* or
> *spark-shell*. This conf file is huge so *--conf* is not a option for me.
>
> Note :- I have tried user@spark.apache.org and my mail was bouncing each
> time so Sean Owen suggested to mail dev.(https://issues.apache.
> org/jira/browse/SPARK-19546). Please give solution to above ticket also
> if possible.
>
> Thanks
>
> --
> Shivam Sharma
>



-- 
Ryan Blue
Software Engineer
Netflix