You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@livy.apache.org by Akshat Bordia <ak...@gmail.com> on 2021/04/17 17:18:46 UTC
Livy Scalability Data
Hi,
I have a few queries regarding Livy scalability. Do we have any experiments
run which addresses the following question? Appreciate any help.
1. How many sessions can a Livy server handle at a time (and with what
configuration)?
2. In particular, how many interactive sessions can be running at a time?
(Given that the driver will be launched on Livy machine in case of client
mode.)
3. Given that the JIRA for multi-active HA in Livy
<https://issues.apache.org/jira/browse/LIVY-718> is open, are there any
other recommendations to scale it horizontally?
Thanks,
Akshat
--
Regards,
Akshat Bordia
Email id : akshat.bordia31@gmail.com
Contact No.:+91-8011228476
+91-9461054710
Re: Livy Scalability Data
Posted by Akshat Bordia <ak...@gmail.com>.
Hi,
Thanks for sharing the details. Would you be able to share the result from
your Jmeter experiment? Did you run it for both interactive and batch
sessions?
Regarding scaling with K8S, is it still a single node based HA or are you
scaling horizontally somehow? If you are scaling horizontally, could you
please share some details on how you are achieving this? Appreciate your
help.
Thanks,
Akshat
On Sun, Apr 18, 2021 at 9:12 PM Jean-Baptiste Onofre <jb...@nanthrax.net>
wrote:
> Hi,
>
> Some months ago I did some performance benchmark with jmeter on the REST
> layer.
> About HA, I wrapped my Livy setup in docker and scale with k8s (still the
> REST API).
>
> Regards
> JB
>
> > Le 17 avr. 2021 à 19:18, Akshat Bordia <ak...@gmail.com> a
> écrit :
> >
> > Hi,
> >
> > I have a few queries regarding Livy scalability. Do we have any
> experiments
> > run which addresses the following question? Appreciate any help.
> >
> >
> > 1. How many sessions can a Livy server handle at a time (and with what
> > configuration)?
> > 2. In particular, how many interactive sessions can be running at a time?
> > (Given that the driver will be launched on Livy machine in case of client
> > mode.)
> > 3. Given that the JIRA for multi-active HA in Livy
> > <https://issues.apache.org/jira/browse/LIVY-718> is open, are there any
> > other recommendations to scale it horizontally?
> >
> > Thanks,
> > Akshat
> >
> > --
> > Regards,
> > Akshat Bordia
> > Email id : akshat.bordia31@gmail.com
> > Contact No.:+91-8011228476
> > +91-9461054710
>
>
--
Regards,
Akshat Bordia
Email id : akshat.bordia31@gmail.com
Contact No.:+91-8011228476
+91-9461054710
Re: Livy Scalability Data
Posted by Akshat Bordia <ak...@gmail.com>.
Hi,
Thanks for sharing the details. Would you be able to share the result from
your Jmeter experiment? Did you run it for both interactive and batch
sessions?
Regarding scaling with K8S, is it still a single node based HA or are you
scaling horizontally somehow? If you are scaling horizontally, could you
please share some details on how you are achieving this? Appreciate your
help.
Thanks,
Akshat
On Sun, Apr 18, 2021 at 9:12 PM Jean-Baptiste Onofre <jb...@nanthrax.net>
wrote:
> Hi,
>
> Some months ago I did some performance benchmark with jmeter on the REST
> layer.
> About HA, I wrapped my Livy setup in docker and scale with k8s (still the
> REST API).
>
> Regards
> JB
>
> > Le 17 avr. 2021 à 19:18, Akshat Bordia <ak...@gmail.com> a
> écrit :
> >
> > Hi,
> >
> > I have a few queries regarding Livy scalability. Do we have any
> experiments
> > run which addresses the following question? Appreciate any help.
> >
> >
> > 1. How many sessions can a Livy server handle at a time (and with what
> > configuration)?
> > 2. In particular, how many interactive sessions can be running at a time?
> > (Given that the driver will be launched on Livy machine in case of client
> > mode.)
> > 3. Given that the JIRA for multi-active HA in Livy
> > <https://issues.apache.org/jira/browse/LIVY-718> is open, are there any
> > other recommendations to scale it horizontally?
> >
> > Thanks,
> > Akshat
> >
> > --
> > Regards,
> > Akshat Bordia
> > Email id : akshat.bordia31@gmail.com
> > Contact No.:+91-8011228476
> > +91-9461054710
>
>
--
Regards,
Akshat Bordia
Email id : akshat.bordia31@gmail.com
Contact No.:+91-8011228476
+91-9461054710
Re: Livy Scalability Data
Posted by Jean-Baptiste Onofre <jb...@nanthrax.net>.
Hi,
Some months ago I did some performance benchmark with jmeter on the REST layer.
About HA, I wrapped my Livy setup in docker and scale with k8s (still the REST API).
Regards
JB
> Le 17 avr. 2021 à 19:18, Akshat Bordia <ak...@gmail.com> a écrit :
>
> Hi,
>
> I have a few queries regarding Livy scalability. Do we have any experiments
> run which addresses the following question? Appreciate any help.
>
>
> 1. How many sessions can a Livy server handle at a time (and with what
> configuration)?
> 2. In particular, how many interactive sessions can be running at a time?
> (Given that the driver will be launched on Livy machine in case of client
> mode.)
> 3. Given that the JIRA for multi-active HA in Livy
> <https://issues.apache.org/jira/browse/LIVY-718> is open, are there any
> other recommendations to scale it horizontally?
>
> Thanks,
> Akshat
>
> --
> Regards,
> Akshat Bordia
> Email id : akshat.bordia31@gmail.com
> Contact No.:+91-8011228476
> +91-9461054710
Re: Livy Scalability Data
Posted by Jean-Baptiste Onofre <jb...@nanthrax.net>.
Hi,
Some months ago I did some performance benchmark with jmeter on the REST layer.
About HA, I wrapped my Livy setup in docker and scale with k8s (still the REST API).
Regards
JB
> Le 17 avr. 2021 à 19:18, Akshat Bordia <ak...@gmail.com> a écrit :
>
> Hi,
>
> I have a few queries regarding Livy scalability. Do we have any experiments
> run which addresses the following question? Appreciate any help.
>
>
> 1. How many sessions can a Livy server handle at a time (and with what
> configuration)?
> 2. In particular, how many interactive sessions can be running at a time?
> (Given that the driver will be launched on Livy machine in case of client
> mode.)
> 3. Given that the JIRA for multi-active HA in Livy
> <https://issues.apache.org/jira/browse/LIVY-718> is open, are there any
> other recommendations to scale it horizontally?
>
> Thanks,
> Akshat
>
> --
> Regards,
> Akshat Bordia
> Email id : akshat.bordia31@gmail.com
> Contact No.:+91-8011228476
> +91-9461054710