You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by mhd wrk <mh...@gmail.com> on 2019/11/01 18:37:57 UTC
passing extra data dynamically to spark job
What's the best way in zeppelin to pass extra data (e.g. stored in current
user's session) to spark job?
Re: passing extra data dynamically to spark job
Posted by Jeff Zhang <zj...@gmail.com>.
You can do the same thing in Zeppelin, either set that in interpreter
setting page or using inline generic configuration.
http://zeppelin.apache.org/docs/0.8.2/usage/interpreter/overview.html#generic-confinterpreter
mhd wrk <mh...@gmail.com> 于2019年11月5日周二 上午10:05写道：
> As explained here
> <https://spark.apache.org/docs/latest/configuration.html#dynamically-loading-spark-properties>,
> in spark we can use ''--conf" to load properties dynamically at submit time
> instead of hard-coding them in a configuration file. How can we
> achieve this via Zeppelin?
>
> On Mon, Nov 4, 2019 at 5:19 PM Jeff Zhang <zj...@gmail.com> wrote:
>
>> It is some kind of spark problem, you'd better to ask this kind of
>> question in spark community. If spark can, then zeppelin won't stop that.
>>
>> mhd wrk <mh...@gmail.com> 于2019年11月5日周二 上午1:02写道：
>>
>>> For example, passing a delegation and/or access token created per user
>>> authentication to the driver.
>>>
>>> On Fri, Nov 1, 2019 at 11:05 PM Jeff Zhang <zj...@gmail.com> wrote:
>>>
>>>> Could you be more specific what kind of data you want to pass to spark
>>>> job, to spark driver or to spark executor ? Giving an example would be very
>>>> helpful.
>>>>
>>>> mhd wrk <mh...@gmail.com> 于2019年11月2日周六 上午2:38写道：
>>>>
>>>>> What's the best way in zeppelin to pass extra data (e.g. stored in
>>>>> current user's session) to spark job?
>>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards
>>>>
>>>> Jeff Zhang
>>>>
>>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
--
Best Regards
Jeff Zhang
Re: passing extra data dynamically to spark job
Posted by mhd wrk <mh...@gmail.com>.
As explained here
<https://spark.apache.org/docs/latest/configuration.html#dynamically-loading-spark-properties>,
in spark we can use ''--conf" to load properties dynamically at submit time
instead of hard-coding them in a configuration file. How can we
achieve this via Zeppelin?
On Mon, Nov 4, 2019 at 5:19 PM Jeff Zhang <zj...@gmail.com> wrote:
> It is some kind of spark problem, you'd better to ask this kind of
> question in spark community. If spark can, then zeppelin won't stop that.
>
> mhd wrk <mh...@gmail.com> 于2019年11月5日周二 上午1:02写道：
>
>> For example, passing a delegation and/or access token created per user
>> authentication to the driver.
>>
>> On Fri, Nov 1, 2019 at 11:05 PM Jeff Zhang <zj...@gmail.com> wrote:
>>
>>> Could you be more specific what kind of data you want to pass to spark
>>> job, to spark driver or to spark executor ? Giving an example would be very
>>> helpful.
>>>
>>> mhd wrk <mh...@gmail.com> 于2019年11月2日周六 上午2:38写道：
>>>
>>>> What's the best way in zeppelin to pass extra data (e.g. stored in
>>>> current user's session) to spark job?
>>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>
> --
> Best Regards
>
> Jeff Zhang
>
Re: passing extra data dynamically to spark job
Posted by Jeff Zhang <zj...@gmail.com>.
It is some kind of spark problem, you'd better to ask this kind of question
in spark community. If spark can, then zeppelin won't stop that.
mhd wrk <mh...@gmail.com> 于2019年11月5日周二 上午1:02写道：
> For example, passing a delegation and/or access token created per user
> authentication to the driver.
>
> On Fri, Nov 1, 2019 at 11:05 PM Jeff Zhang <zj...@gmail.com> wrote:
>
>> Could you be more specific what kind of data you want to pass to spark
>> job, to spark driver or to spark executor ? Giving an example would be very
>> helpful.
>>
>> mhd wrk <mh...@gmail.com> 于2019年11月2日周六 上午2:38写道：
>>
>>> What's the best way in zeppelin to pass extra data (e.g. stored in
>>> current user's session) to spark job?
>>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
--
Best Regards
Jeff Zhang
Re: passing extra data dynamically to spark job
Posted by mhd wrk <mh...@gmail.com>.
For example, passing a delegation and/or access token created per user
authentication to the driver.
On Fri, Nov 1, 2019 at 11:05 PM Jeff Zhang <zj...@gmail.com> wrote:
> Could you be more specific what kind of data you want to pass to spark
> job, to spark driver or to spark executor ? Giving an example would be very
> helpful.
>
> mhd wrk <mh...@gmail.com> 于2019年11月2日周六 上午2:38写道：
>
>> What's the best way in zeppelin to pass extra data (e.g. stored in
>> current user's session) to spark job?
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
RE: passing extra data dynamically to spark job
Posted by Paul Brenner <pb...@placeiq.com>.
I don’t think you can just email the list with the word “unsubscribe”. I’m not sure why people keep doing that (not saying you did this but a lot of people do).
On the other hand if you go here: https://zeppelin.apache.org/community.html there is a link to unsubscribe next to “Users”.
Basically it just creates an email for you to the address users-unsubscribe@zeppelin.apache.org which should do the trick.
PAUL BRENNER
Head of Data Science
pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
twitter @placeiq linkedin /placeiq
On Nov 5, 2019, 4:39 PM -0500, Queimado, Fernando (Nokia - PT/Amadora) <fe...@nokia.com>, wrote:
> Hello
> I already sent several “unsubscribe” mails, but I am not able to be removed from this distribution list.
> Can you please let me know how can I be removed?
> Thank you
>
> From: Jeff Zhang <zj...@gmail.com>
> Sent: Saturday, November 2, 2019 6:05 AM
> To: users <us...@zeppelin.apache.org>
> Subject: Re: passing extra data dynamically to spark job
>
> Could you be more specific what kind of data you want to pass to spark job, to spark driver or to spark executor ? Giving an example would be very helpful.
>
> mhd wrk <mh...@gmail.com> 于2019年11月2日周六 上午2:38写道：
> > What's the best way in zeppelin to pass extra data (e.g. stored in current user's session) to spark job?
>
>
> --
> Best Regards
>
> Jeff Zhang
RE: passing extra data dynamically to spark job
Posted by "Queimado, Fernando (Nokia - PT/Amadora)" <fe...@nokia.com>.
Hello
I already sent several “unsubscribe” mails, but I am not able to be removed from this distribution list.
Can you please let me know how can I be removed?
Thank you
From: Jeff Zhang <zj...@gmail.com>
Sent: Saturday, November 2, 2019 6:05 AM
To: users <us...@zeppelin.apache.org>
Subject: Re: passing extra data dynamically to spark job
Could you be more specific what kind of data you want to pass to spark job, to spark driver or to spark executor ? Giving an example would be very helpful.
mhd wrk <mh...@gmail.com>> 于2019年11月2日周六 上午2:38写道：
What's the best way in zeppelin to pass extra data (e.g. stored in current user's session) to spark job?
--
Best Regards
Jeff Zhang
Re: passing extra data dynamically to spark job
Posted by Jeff Zhang <zj...@gmail.com>.
Could you be more specific what kind of data you want to pass to spark job,
to spark driver or to spark executor ? Giving an example would be very
helpful.
mhd wrk <mh...@gmail.com> 于2019年11月2日周六 上午2:38写道：
> What's the best way in zeppelin to pass extra data (e.g. stored in current
> user's session) to spark job?
>
--
Best Regards
Jeff Zhang