You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by "Litt, Shaun" <sl...@conversantmedia.com> on 2015/06/30 01:23:08 UTC

Zeppelin Interpreter end

Hi,  I am new to zeppelin and just got it configured to run in my YARN cluster, but I was wondering if there is a configuration or even a hard setting that shuts down interpreters after in-activity.  It seems like the interpreter (and it's yarn consumption) hang out indefinitely, ideally there would be a clean way (like logout or a shutdown button within the notebook) to shutdown these interpreters, but additionally there should be a way for an admin of zeppelin to impose and idle timeout.  As a note to the scope of this, is dynamic yarn resource allocation configured (such that once a paragraph finishes it can release vcores)?

Thanks,
Shaun






This email and any files included with it may contain privileged,
proprietary and/or confidential information that is for the sole use
of the intended recipient(s).  Any disclosure, copying, distribution,
posting, or use of the information contained in or attached to this
email is prohibited unless permitted by the sender.  If you have
received this email in error, please immediately notify the sender
via return email, telephone, or fax and destroy this original transmission
and its included files without reading or saving it in any manner.
Thank you.

Re: Zeppelin Interpreter end

Posted by Rajat Gupta <ra...@gmail.com>.
No we have not submitted it back to spark community. I can send the patches
for both spark and zeppelin. If it seems interesting enough then I can
start a PR for zeppelin part at least. I will try to do this by weekend if
it sounds interesting enough.

Rajat

On Wed, Jul 1, 2015 at 5:10 PM, Litt, Shaun <sl...@conversantmedia.com>
wrote:

>  Interesting and useful.  I would love to see your changes for reference.
>   Have you submitted this feature back to the SPARK community?  I think
> this would be useful for a lot of interactive spark use-cases in shared
> clusters.
>
>
>
> Also I have a question regarding your experience maintaining your own
> SPARK fork.  We have also been forced to maintain a very minor spark fork
> to work around some bugs in our environment, but our internal goal is to
> try to get back to the main apache branch or a vendor fork.  Have you found
> keeping up with SPARK (which is changing lightning fast – pun intended)
> challenging?
>
>
>
> Thanks,
>
> *Shaun* *Litt*
>
>
>
>
>
> *From:* Rajat Gupta [mailto:rajat.iitian@gmail.com]
> *Sent:* Wednesday, July 01, 2015 2:41 AM
> *To:* users@zeppelin.incubator.apache.org
> *Subject:* Re: Zeppelin Interpreter end
>
>
>
> We have implemented auto shutdown of spark application by adding an idle
> timeout setting directly in spark. We set the value of this setting in the
> zeppelin interpreter UI page and default it to 60 mins. For this fix we
> also added an auto lazy restart of interpreter in zeppelin if interpreter
> has shutdown because of inactivity. If you want I can send you patches for
> this both for spark and zeppelin. I did not submit the patch in zeppelin
> for this earlier because this makes sense only if spark has idle timeout in
> open source.
>
> Dynamic scaling of spark applications works fine on zeppelin with
> appropriate spark settings on interpreter page. Zeppelin just maintains
> handle to a spark context and that spark context is free to do anything
> afaik. We at Qubole are using it actively.
>
> Rajat
>
>
>
> On Tue, Jun 30, 2015 at 4:53 AM, Litt, Shaun <sl...@conversantmedia.com>
> wrote:
>
>  Hi,  I am new to zeppelin and just got it configured to run in my YARN
> cluster, but I was wondering if there is a configuration or even a hard
> setting that shuts down interpreters after in-activity.  It seems like the
> interpreter (and it’s yarn consumption) hang out indefinitely, ideally
> there would be a clean way (like logout or a shutdown button within the
> notebook) to shutdown these interpreters, but additionally there should be
> a way for an admin of zeppelin to impose and idle timeout.  As a note to
> the scope of this, is dynamic yarn resource allocation configured (such
> that once a paragraph finishes it can release vcores)?
>
>
>
> Thanks,
>
> *Shaun*
>
>
>
> This email and any files included with it may contain privileged,
> proprietary and/or confidential information that is for the sole use
> of the intended recipient(s).  Any disclosure, copying, distribution,
> posting, or use of the information contained in or attached to this
> email is prohibited unless permitted by the sender.  If you have
> received this email in error, please immediately notify the sender
> via return email, telephone, or fax and destroy this original transmission
> and its included files without reading or saving it in any manner.
> Thank you.
>
>
>
>  This email and any files included with it may contain privileged,
> proprietary and/or confidential information that is for the sole use
> of the intended recipient(s).  Any disclosure, copying, distribution,
> posting, or use of the information contained in or attached to this
> email is prohibited unless permitted by the sender.  If you have
> received this email in error, please immediately notify the sender
> via return email, telephone, or fax and destroy this original transmission
> and its included files without reading or saving it in any manner.
> Thank you.
>

RE: Zeppelin Interpreter end

Posted by "Litt, Shaun" <sl...@conversantmedia.com>.
Interesting and useful.  I would love to see your changes for reference.   Have you submitted this feature back to the SPARK community?  I think this would be useful for a lot of interactive spark use-cases in shared clusters.

Also I have a question regarding your experience maintaining your own SPARK fork.  We have also been forced to maintain a very minor spark fork to work around some bugs in our environment, but our internal goal is to try to get back to the main apache branch or a vendor fork.  Have you found keeping up with SPARK (which is changing lightning fast – pun intended) challenging?

Thanks,
Shaun Litt


From: Rajat Gupta [mailto:rajat.iitian@gmail.com]
Sent: Wednesday, July 01, 2015 2:41 AM
To: users@zeppelin.incubator.apache.org
Subject: Re: Zeppelin Interpreter end

We have implemented auto shutdown of spark application by adding an idle timeout setting directly in spark. We set the value of this setting in the zeppelin interpreter UI page and default it to 60 mins. For this fix we also added an auto lazy restart of interpreter in zeppelin if interpreter has shutdown because of inactivity. If you want I can send you patches for this both for spark and zeppelin. I did not submit the patch in zeppelin for this earlier because this makes sense only if spark has idle timeout in open source.
Dynamic scaling of spark applications works fine on zeppelin with appropriate spark settings on interpreter page. Zeppelin just maintains handle to a spark context and that spark context is free to do anything afaik. We at Qubole are using it actively.
Rajat

On Tue, Jun 30, 2015 at 4:53 AM, Litt, Shaun <sl...@conversantmedia.com>> wrote:
Hi,  I am new to zeppelin and just got it configured to run in my YARN cluster, but I was wondering if there is a configuration or even a hard setting that shuts down interpreters after in-activity.  It seems like the interpreter (and it’s yarn consumption) hang out indefinitely, ideally there would be a clean way (like logout or a shutdown button within the notebook) to shutdown these interpreters, but additionally there should be a way for an admin of zeppelin to impose and idle timeout.  As a note to the scope of this, is dynamic yarn resource allocation configured (such that once a paragraph finishes it can release vcores)?

Thanks,
Shaun


This email and any files included with it may contain privileged,
proprietary and/or confidential information that is for the sole use
of the intended recipient(s).  Any disclosure, copying, distribution,
posting, or use of the information contained in or attached to this
email is prohibited unless permitted by the sender.  If you have
received this email in error, please immediately notify the sender
via return email, telephone, or fax and destroy this original transmission
and its included files without reading or saving it in any manner.
Thank you.





This email and any files included with it may contain privileged,
proprietary and/or confidential information that is for the sole use
of the intended recipient(s).  Any disclosure, copying, distribution,
posting, or use of the information contained in or attached to this
email is prohibited unless permitted by the sender.  If you have
received this email in error, please immediately notify the sender
via return email, telephone, or fax and destroy this original transmission
and its included files without reading or saving it in any manner.
Thank you.

Re: Zeppelin Interpreter end

Posted by Rajat Gupta <ra...@gmail.com>.
We have implemented auto shutdown of spark application by adding an idle
timeout setting directly in spark. We set the value of this setting in the
zeppelin interpreter UI page and default it to 60 mins. For this fix we
also added an auto lazy restart of interpreter in zeppelin if interpreter
has shutdown because of inactivity. If you want I can send you patches for
this both for spark and zeppelin. I did not submit the patch in zeppelin
for this earlier because this makes sense only if spark has idle timeout in
open source.

Dynamic scaling of spark applications works fine on zeppelin with
appropriate spark settings on interpreter page. Zeppelin just maintains
handle to a spark context and that spark context is free to do anything
afaik. We at Qubole are using it actively.

Rajat

On Tue, Jun 30, 2015 at 4:53 AM, Litt, Shaun <sl...@conversantmedia.com>
wrote:

>  Hi,  I am new to zeppelin and just got it configured to run in my YARN
> cluster, but I was wondering if there is a configuration or even a hard
> setting that shuts down interpreters after in-activity.  It seems like the
> interpreter (and it’s yarn consumption) hang out indefinitely, ideally
> there would be a clean way (like logout or a shutdown button within the
> notebook) to shutdown these interpreters, but additionally there should be
> a way for an admin of zeppelin to impose and idle timeout.  As a note to
> the scope of this, is dynamic yarn resource allocation configured (such
> that once a paragraph finishes it can release vcores)?
>
>
>
> Thanks,
>
> *Shaun*
>
>
>
>  This email and any files included with it may contain privileged,
> proprietary and/or confidential information that is for the sole use
> of the intended recipient(s).  Any disclosure, copying, distribution,
> posting, or use of the information contained in or attached to this
> email is prohibited unless permitted by the sender.  If you have
> received this email in error, please immediately notify the sender
> via return email, telephone, or fax and destroy this original transmission
> and its included files without reading or saving it in any manner.
> Thank you.
>

Re: Zeppelin Interpreter end

Posted by IT CTO <go...@gmail.com>.
Actually reading some more about slider, I am not sure it s what I thought
it is... to fast on sending this mail.
Eran

On Tue, Jun 30, 2015 at 9:46 AM IT CTO <go...@gmail.com> wrote:

> If I am not mistaking Apache Slider is aim to handle dynamic growing and
> shrinking of applications on YARN but it is still incubating (as zeppelin)
> might be interesting writing a slider package for zeppeling.
> Eran
>
> On Tue, Jun 30, 2015 at 9:02 AM Alexander Bezzubov <ab...@nflabs.com>
> wrote:
>
>> Hi,
>>
>> right now, I'm not aware of such configuration in Zeppelin (please,
>> feel free to open the issue\submit a patch).
>>
>> AFAIK dynamic YARN resource allocation is up to the user and is not
>> configured by default right now, which looks like one possible
>> solution to the problem you describe (at least cpu-wise)
>>
>> As a workaround for your usecase, you can manually re-start Spark
>> interpreter (Interpreter menu -> restart), which, because of
>> lazy-loading, will not occupy any resources until somebody actually
>> runs it.
>>
>> Hope this helps!
>>
>>
>> On Tue, Jun 30, 2015 at 8:23 AM, Litt, Shaun <sl...@conversantmedia.com>
>> wrote:
>> > Hi,  I am new to zeppelin and just got it configured to run in my YARN
>> > cluster, but I was wondering if there is a configuration or even a hard
>> > setting that shuts down interpreters after in-activity.  It seems like
>> the
>> > interpreter (and it’s yarn consumption) hang out indefinitely, ideally
>> there
>> > would be a clean way (like logout or a shutdown button within the
>> notebook)
>> > to shutdown these interpreters, but additionally there should be a way
>> for
>> > an admin of zeppelin to impose and idle timeout.  As a note to the
>> scope of
>> > this, is dynamic yarn resource allocation configured (such that once a
>> > paragraph finishes it can release vcores)?
>> >
>> >
>> >
>> > Thanks,
>> >
>> > Shaun
>> >
>> >
>> >
>> > This email and any files included with it may contain privileged,
>> > proprietary and/or confidential information that is for the sole use
>> > of the intended recipient(s).  Any disclosure, copying, distribution,
>> > posting, or use of the information contained in or attached to this
>> > email is prohibited unless permitted by the sender.  If you have
>> > received this email in error, please immediately notify the sender
>> > via return email, telephone, or fax and destroy this original
>> transmission
>> > and its included files without reading or saving it in any manner.
>> > Thank you.
>>
>>
>>
>> --
>> --
>> Kind regards,
>> Alexander.
>>
>

Re: Zeppelin Interpreter end

Posted by IT CTO <go...@gmail.com>.
If I am not mistaking Apache Slider is aim to handle dynamic growing and
shrinking of applications on YARN but it is still incubating (as zeppelin)
might be interesting writing a slider package for zeppeling.
Eran

On Tue, Jun 30, 2015 at 9:02 AM Alexander Bezzubov <ab...@nflabs.com>
wrote:

> Hi,
>
> right now, I'm not aware of such configuration in Zeppelin (please,
> feel free to open the issue\submit a patch).
>
> AFAIK dynamic YARN resource allocation is up to the user and is not
> configured by default right now, which looks like one possible
> solution to the problem you describe (at least cpu-wise)
>
> As a workaround for your usecase, you can manually re-start Spark
> interpreter (Interpreter menu -> restart), which, because of
> lazy-loading, will not occupy any resources until somebody actually
> runs it.
>
> Hope this helps!
>
>
> On Tue, Jun 30, 2015 at 8:23 AM, Litt, Shaun <sl...@conversantmedia.com>
> wrote:
> > Hi,  I am new to zeppelin and just got it configured to run in my YARN
> > cluster, but I was wondering if there is a configuration or even a hard
> > setting that shuts down interpreters after in-activity.  It seems like
> the
> > interpreter (and it’s yarn consumption) hang out indefinitely, ideally
> there
> > would be a clean way (like logout or a shutdown button within the
> notebook)
> > to shutdown these interpreters, but additionally there should be a way
> for
> > an admin of zeppelin to impose and idle timeout.  As a note to the scope
> of
> > this, is dynamic yarn resource allocation configured (such that once a
> > paragraph finishes it can release vcores)?
> >
> >
> >
> > Thanks,
> >
> > Shaun
> >
> >
> >
> > This email and any files included with it may contain privileged,
> > proprietary and/or confidential information that is for the sole use
> > of the intended recipient(s).  Any disclosure, copying, distribution,
> > posting, or use of the information contained in or attached to this
> > email is prohibited unless permitted by the sender.  If you have
> > received this email in error, please immediately notify the sender
> > via return email, telephone, or fax and destroy this original
> transmission
> > and its included files without reading or saving it in any manner.
> > Thank you.
>
>
>
> --
> --
> Kind regards,
> Alexander.
>

Re: Zeppelin Interpreter end

Posted by Alexander Bezzubov <ab...@nflabs.com>.
Hi,

right now, I'm not aware of such configuration in Zeppelin (please,
feel free to open the issue\submit a patch).

AFAIK dynamic YARN resource allocation is up to the user and is not
configured by default right now, which looks like one possible
solution to the problem you describe (at least cpu-wise)

As a workaround for your usecase, you can manually re-start Spark
interpreter (Interpreter menu -> restart), which, because of
lazy-loading, will not occupy any resources until somebody actually
runs it.

Hope this helps!


On Tue, Jun 30, 2015 at 8:23 AM, Litt, Shaun <sl...@conversantmedia.com> wrote:
> Hi,  I am new to zeppelin and just got it configured to run in my YARN
> cluster, but I was wondering if there is a configuration or even a hard
> setting that shuts down interpreters after in-activity.  It seems like the
> interpreter (and it’s yarn consumption) hang out indefinitely, ideally there
> would be a clean way (like logout or a shutdown button within the notebook)
> to shutdown these interpreters, but additionally there should be a way for
> an admin of zeppelin to impose and idle timeout.  As a note to the scope of
> this, is dynamic yarn resource allocation configured (such that once a
> paragraph finishes it can release vcores)?
>
>
>
> Thanks,
>
> Shaun
>
>
>
> This email and any files included with it may contain privileged,
> proprietary and/or confidential information that is for the sole use
> of the intended recipient(s).  Any disclosure, copying, distribution,
> posting, or use of the information contained in or attached to this
> email is prohibited unless permitted by the sender.  If you have
> received this email in error, please immediately notify the sender
> via return email, telephone, or fax and destroy this original transmission
> and its included files without reading or saving it in any manner.
> Thank you.



-- 
--
Kind regards,
Alexander.