You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by blaubaer <re...@nzz.ch> on 2016/12/14 15:27:56 UTC
Limit on multiple concurrent interpreters / isolated notebooks?
Hi
We are running Zeppelin (0.5) on our YARN managed cluster. To allow for
multiple concurrent users, without sharing the spark context, we simply
setup one interpreter for every user. This works pretty OK, however, at some
point we seem to hit the limit of how many concurrent (spark) interpreters
the Zeppelin (daemon) service can handle. Now with the “new” (0.6) feature
of isolated notebooks, this topic should pop up with other users as well.
So, I was wondering: what are your experiences with multiple concurrent
interpreters? What are determining factors for the question how many
concurrent interpreters can run (besides from cluster resources to actually
be able to start the multiple interpreters)? Any experiences on that? For us
it seems that 2-3 is OK, 4-5 get’s critical, but that also depends on the
load of the jobs it seems.
Thx.
--
View this message in context: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Limit-on-multiple-concurrent-interpreters-isolated-notebooks-tp4732.html
Sent from the Apache Zeppelin Users (incubating) mailing list mailing list archive at Nabble.com.
Re: Limit on multiple concurrent interpreters / isolated notebooks?
Posted by "Jianfeng (Jeff) Zhang" <jz...@hortonworks.com>.
IMO, Before zeppelin-server hit bottleneck. Drivers would eat up your machine’s memory.
Best Regard,
Jeff Zhang
From: blaubaer <re...@nzz.ch>>
Reply-To: "users@zeppelin.apache.org<ma...@zeppelin.apache.org>" <us...@zeppelin.apache.org>>
Date: Thursday, December 15, 2016 at 3:43 PM
To: "users@zeppelin.incubator.apache.org<ma...@zeppelin.incubator.apache.org>" <us...@zeppelin.incubator.apache.org>>
Subject: Re: Limit on multiple concurrent interpreters / isolated notebooks?
So that is concerning the drivers, which clearly have to fit on one single machine. But what about the Zeppelin server service itself? Any bottlenecks for how many concurrent drivers processes it can handle?
-----------------------------
René Pfitzner
Lead Data Scientists
Neue Zürcher Zeitung
Sent from mobile. Please excuse brevity.
From: Jianfeng (Jeff) Zhang [via Apache Zeppelin Users (incubating) mailing list]
Sent: Thursday, December 15, 3:31 AM
Subject: Re: Limit on multiple concurrent interpreters / isolated notebooks?
To: Pfitzner René
Yes, this would be a critical performance issue for multiple user case.
Because currently zeppelin only support yarn-client mode which means the
driver JVM is in the same host
of zeppelin server. So regarding the concurrent users, it depends on the
memory you configure for the driver and how many memory you have for the
machine.
But for the long term, we should support yarn-cluster mode. Here¹s one
ticket and wiki for this
https<https://issues.apache.org/jira/browse/ZEPPELIN-1377>://issues.apache.org/jira/browse/ZEPPELIN-1377<https://issues.apache.org/jira/browse/ZEPPELIN-1377>
<a href="https://cwiki.apache.org/confluence/display/ZEPPELIN/Cluster+Manager+Propos">https<a href="https://cwiki.apache.org/confluence/display/ZEPPELIN/Cluster+Manager+Propos">://cwiki.apache.org/confluence/display/ZEPPELIN/Cluster+Manager+Propos
al
Best Regard,
Jeff Zhang
On 12/14/16, 11:27 PM, "blaubaer" <[hidden email]</user/SendEmail.jtp?type=node&node=4735&i=0>> wrote:
>Hi
>
>We are running Zeppelin (0.5) on our YARN managed cluster. To allow for
>multiple concurrent users, without sharing the spark context, we simply
>setup one interpreter for every user. This works pretty OK, however, at
>some
>point we seem to hit the limit of how many concurrent (spark) interpreters
>the Zeppelin (daemon) service can handle. Now with the ³new² (0.6) feature
>of isolated notebooks, this topic should pop up with other users as well.
>
>So, I was wondering: what are your experiences with multiple concurrent
>interpreters? What are determining factors for the question how many
>concurrent interpreters can run (besides from cluster resources to
>actually
>be able to start the multiple interpreters)? Any experiences on that? For
>us
>it seems that 2-3 is OK, 4-5 get¹s critical, but that also depends on the
>load of the jobs it seems.
>
>Thx.
>
>
>
>--
>View this message in context:
>http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/L
>imit-on-multiple-concurrent-interpreters-isolated-notebooks-tp4732.html
>Sent from the Apache Zeppelin Users (incubating) mailing list mailing
>list archive at Nabble.com.
>
If you reply to this email, your message will be added to the discussion below:
http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Limit-on-multiple-concurrent-interpreters-isolated-notebooks-tp4732p4735.html
To start a new topic under Apache Zeppelin Users (incubating) mailing list, email ml-node+[hidden email]</user/SendEmail.jtp?type=node&node=4736&i=0>
To unsubscribe from Limit on multiple concurrent interpreters / isolated notebooks?, click here.
NAML<http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
________________________________
View this message in context: Re: Limit on multiple concurrent interpreters / isolated notebooks?<http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Limit-on-multiple-concurrent-interpreters-isolated-notebooks-tp4732p4736.html>
Sent from the Apache Zeppelin Users (incubating) mailing list mailing list archive<http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/> at Nabble.com.
Re: Limit on multiple concurrent interpreters / isolated notebooks?
Posted by blaubaer <re...@nzz.ch>.
So that is concerning the drivers, which clearly have to fit on one single machine. But what about the Zeppelin server service itself? Any bottlenecks for how many concurrent drivers processes it can handle?
-----------------------------
René Pfitzner
Lead Data Scientists
Neue Zürcher Zeitung
Sent from mobile. Please excuse brevity.
From: Jianfeng (Jeff) Zhang [via Apache Zeppelin Users (incubating) mailing list]
Sent: Thursday, December 15, 3:31 AM
Subject: Re: Limit on multiple concurrent interpreters / isolated notebooks?
To: Pfitzner René
Yes, this would be a critical performance issue for multiple user case.
Because currently zeppelin only support yarn-client mode which means the
driver JVM is in the same host
of zeppelin server. So regarding the concurrent users, it depends on the
memory you configure for the driver and how many memory you have for the
machine.
But for the long term, we should support yarn-cluster mode. Here¹s one
ticket and wiki for this
https<https://issues.apache.org/jira/browse/ZEPPELIN-1377>://issues.apache.org/jira/browse/ZEPPELIN-1377<https://issues.apache.org/jira/browse/ZEPPELIN-1377>
https<https://cwiki.apache.org/confluence/display/ZEPPELIN/Cluster+Manager+Propos>://cwiki.apache.org/confluence/display/ZEPPELIN/Cluster+Manager+Propos<https://cwiki.apache.org/confluence/display/ZEPPELIN/Cluster+Manager+Propos>
al
Best Regard,
Jeff Zhang
On 12/14/16, 11:27 PM, "blaubaer" <[hidden email]</user/SendEmail.jtp?type=node&node=4735&i=0>> wrote:
>Hi
>
>We are running Zeppelin (0.5) on our YARN managed cluster. To allow for
>multiple concurrent users, without sharing the spark context, we simply
>setup one interpreter for every user. This works pretty OK, however, at
>some
>point we seem to hit the limit of how many concurrent (spark) interpreters
>the Zeppelin (daemon) service can handle. Now with the ³new² (0.6) feature
>of isolated notebooks, this topic should pop up with other users as well.
>
>So, I was wondering: what are your experiences with multiple concurrent
>interpreters? What are determining factors for the question how many
>concurrent interpreters can run (besides from cluster resources to
>actually
>be able to start the multiple interpreters)? Any experiences on that? For
>us
>it seems that 2-3 is OK, 4-5 get¹s critical, but that also depends on the
>load of the jobs it seems.
>
>Thx.
>
>
>
>--
>View this message in context:
>http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/L
>imit-on-multiple-concurrent-interpreters-isolated-notebooks-tp4732.html
>Sent from the Apache Zeppelin Users (incubating) mailing list mailing
>list archive at Nabble.com.
>
If you reply to this email, your message will be added to the discussion below:
http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Limit-on-multiple-concurrent-interpreters-isolated-notebooks-tp4732p4735.html
To start a new topic under Apache Zeppelin Users (incubating) mailing list, email ml-node+s75479n1h98@n6.nabble.com
To unsubscribe from Limit on multiple concurrent interpreters / isolated notebooks?, click here<http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4732&code=cmVuZS5wZml0em5lckBuenouY2h8NDczMnw5MzgxOTYwMjk=>.
NAML<http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Limit-on-multiple-concurrent-interpreters-isolated-notebooks-tp4732p4736.html
Sent from the Apache Zeppelin Users (incubating) mailing list mailing list archive at Nabble.com.
Re: Limit on multiple concurrent interpreters / isolated notebooks?
Posted by "Jianfeng (Jeff) Zhang" <jz...@hortonworks.com>.
Yes, this would be a critical performance issue for multiple user case.
Because currently zeppelin only support yarn-client mode which means the
driver JVM is in the same host
of zeppelin server. So regarding the concurrent users, it depends on the
memory you configure for the driver and how many memory you have for the
machine.
But for the long term, we should support yarn-cluster mode. Here¹s one
ticket and wiki for this
https://issues.apache.org/jira/browse/ZEPPELIN-1377
https://cwiki.apache.org/confluence/display/ZEPPELIN/Cluster+Manager+Propos
al
Best Regard,
Jeff Zhang
On 12/14/16, 11:27 PM, "blaubaer" <re...@nzz.ch> wrote:
>Hi
>
>We are running Zeppelin (0.5) on our YARN managed cluster. To allow for
>multiple concurrent users, without sharing the spark context, we simply
>setup one interpreter for every user. This works pretty OK, however, at
>some
>point we seem to hit the limit of how many concurrent (spark) interpreters
>the Zeppelin (daemon) service can handle. Now with the ³new² (0.6) feature
>of isolated notebooks, this topic should pop up with other users as well.
>
>So, I was wondering: what are your experiences with multiple concurrent
>interpreters? What are determining factors for the question how many
>concurrent interpreters can run (besides from cluster resources to
>actually
>be able to start the multiple interpreters)? Any experiences on that? For
>us
>it seems that 2-3 is OK, 4-5 get¹s critical, but that also depends on the
>load of the jobs it seems.
>
>Thx.
>
>
>
>--
>View this message in context:
>http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/L
>imit-on-multiple-concurrent-interpreters-isolated-notebooks-tp4732.html
>Sent from the Apache Zeppelin Users (incubating) mailing list mailing
>list archive at Nabble.com.
>