You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Gengliang Wang <lt...@gmail.com> on 2022/11/16 08:35:35 UTC
[VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications
Hi all,
I’d like to start a vote for SPIP: "Better Spark UI scalability and Driver
stability for large applications"
The goal of the SPIP is to improve the Driver's stability by supporting
storing Spark's UI data on RocksDB. Furthermore, to fasten the read and
write operations on RocksDB, it introduces a new Protobuf serializer.
Please also refer to the following:
- Previous discussion in the dev mailing list: [DISCUSS] SPIP: Better
Spark UI scalability and Driver stability for large applications
<https://lists.apache.org/thread/f30owdts644hk2oojyf9jr308mrx2m9l>
- Design Doc: Better Spark UI scalability and Driver stability for large
applications
<https://docs.google.com/document/d/1cuKnFwlTodyVhUQPMuakq2YDaLH05jaY9FRu_aD1zMo/edit?usp=sharing>
- JIRA: SPARK-41053 <https://issues.apache.org/jira/browse/SPARK-41053>
Please vote on the SPIP for the next 72 hours:
[ ] +1: Accept the proposal as an official SPIP
[ ] +0
[ ] -1: I don’t think this is a good idea because …
Kind Regards,
Gengliang
Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications
Posted by Peter Toth <pe...@gmail.com>.
+1, thanks for the proposal!
416161436@qq.com <ru...@foxmail.com> ezt írta (időpont: 2022. nov. 17.,
Cs, 6:22):
> +1
>
> ------------------------------
> Ruifeng Zheng
> ruifengz@foxmail.com
>
> <https://wx.mail.qq.com/home/index?t=readmail_businesscard_midpage&nocheck=true&name=Ruifeng+Zheng&icon=https%3A%2F%2Fres.mail.qq.com%2Fzh_CN%2Fhtmledition%2Fimages%2Frss%2Fmale.gif%3Frand%3D1617349242&mail=ruifengz%40foxmail.com&code=>
>
>
>
> ------------------ Original ------------------
> *From:* "Wenchen Fan" <cl...@gmail.com>;
> *Date:* Thu, Nov 17, 2022 10:27 AM
> *To:* "Ye Zhou"<zh...@gmail.com>;
> *Cc:* "Jungtaek Lim"<ka...@gmail.com>;"Chris Nauroth"<
> cnauroth@apache.org>;"Maciej"<ms...@gmail.com>;"dev"<
> dev@spark.apache.org>;
> *Subject:* Re: [VOTE][SPIP] Better Spark UI scalability and Driver
> stability for large applications
>
> +1, I'm looking forward to it!
>
> On Thu, Nov 17, 2022 at 9:44 AM Ye Zhou <zh...@gmail.com> wrote:
>
>> +1 (non-binding)
>> Thanks for proposing this improvement to SHS, it resolves the main
>> performance issue within SHS.
>>
>> On Wed, Nov 16, 2022 at 1:15 PM Jungtaek Lim <
>> kabhwan.opensource@gmail.com> wrote:
>>
>>> +1
>>>
>>> Nice to see the chance for driver to reduce resource usage and increase
>>> stability, especially the fact that the driver is SPOF. It's even promising
>>> to have a future plan to pre-bake the kvstore for SHS from the driver.
>>>
>>> Thanks for driving the effort, Gengliang!
>>>
>>> On Thu, Nov 17, 2022 at 5:32 AM Chris Nauroth <cn...@apache.org>
>>> wrote:
>>>
>>>> +1 (non-binding)
>>>>
>>>> Gengliang, thank you for the SPIP.
>>>>
>>>> Chris Nauroth
>>>>
>>>>
>>>> On Wed, Nov 16, 2022 at 4:27 AM Maciej <ms...@gmail.com> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> On 11/16/22 13:19, Yuming Wang wrote:
>>>>> > +1, non-binding
>>>>> >
>>>>> > On Wed, Nov 16, 2022 at 8:12 PM Yang,Jie(INF) <yangjie01@baidu.com
>>>>> > <ma...@baidu.com>> wrote:
>>>>> >
>>>>> > +1, non-binding____
>>>>> >
>>>>> > __ __
>>>>> >
>>>>> > Yang Jie____
>>>>> >
>>>>> > __ __
>>>>> >
>>>>> > *发件人**: *Mridul Muralidharan <mridul@gmail.com
>>>>> > <ma...@gmail.com>>
>>>>> > *日期**: *2022年11月16日星期三17:35
>>>>> > *收件人**: *Kent Yao <yao@apache.org <ma...@apache.org>>
>>>>> > *抄送**: *Gengliang Wang <ltnwgl@gmail.com
>>>>> > <ma...@gmail.com>>, dev <dev@spark.apache.org
>>>>> > <ma...@spark.apache.org>>
>>>>> > *主题**: *Re: [VOTE][SPIP] Better Spark UI scalability and Driver
>>>>> > stability for large applications____
>>>>> >
>>>>> > __ __
>>>>> >
>>>>> > __ __
>>>>> >
>>>>> > +1____
>>>>> >
>>>>> > __ __
>>>>> >
>>>>> > Would be great to see history server performance improvements and
>>>>> > lower resource utilization at driver !____
>>>>> >
>>>>> > __ __
>>>>> >
>>>>> > Regards,____
>>>>> >
>>>>> > Mridul ____
>>>>> >
>>>>> > __ __
>>>>> >
>>>>> > On Wed, Nov 16, 2022 at 2:38 AM Kent Yao <yao@apache.org
>>>>> > <ma...@apache.org>> wrote:____
>>>>> >
>>>>> > +1, non-binding
>>>>> >
>>>>> > Gengliang Wang <ltnwgl@gmail.com <ma...@gmail.com>>
>>>>> 于
>>>>> > 2022年11月16日周三16:36写道：
>>>>> > >
>>>>> > > Hi all,
>>>>> > >
>>>>> > > I’d like to start a vote for SPIP: "Better Spark UI
>>>>> scalability and Driver stability for large applications"
>>>>> > >
>>>>> > > The goal of the SPIP is to improve the Driver's stability
>>>>> by supporting storing Spark's UI data on RocksDB. Furthermore, to fasten
>>>>> the read and write operations on RocksDB, it introduces a new Protobuf
>>>>> serializer.
>>>>> > >
>>>>> > > Please also refer to the following:
>>>>> > >
>>>>> > > Previous discussion in the dev mailing list: [DISCUSS]
>>>>> SPIP: Better Spark UI scalability and Driver stability for large
>>>>> applications
>>>>> > > Design Doc: Better Spark UI scalability and Driver
>>>>> stability for large applications
>>>>> > > JIRA: SPARK-41053
>>>>> > >
>>>>> > >
>>>>> > > Please vote on the SPIP for the next 72 hours:
>>>>> > >
>>>>> > > [ ] +1: Accept the proposal as an official SPIP
>>>>> > > [ ] +0
>>>>> > > [ ] -1: I don’t think this is a good idea because …
>>>>> > >
>>>>> > > Kind Regards,
>>>>> > > Gengliang
>>>>> >
>>>>> >
>>>>> ---------------------------------------------------------------------
>>>>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>> > <ma...@spark.apache.org>____
>>>>> >
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Maciej Szymkiewicz
>>>>>
>>>>> Web: https://zero323.net
>>>>> PGP: A30CEF0C31A501EC
>>>>>
>>>>>
>>
>> --
>>
>> *Zhou, Ye **周晔*
>>
>
Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications
Posted by "416161436@qq.com" <ru...@foxmail.com>.
+1
Ruifeng Zheng
ruifengz@foxmail.com
------------------ Original ------------------
From: "Wenchen Fan" <cloud0fan@gmail.com>;
Date: Thu, Nov 17, 2022 10:27 AM
To: "Ye Zhou"<zhouyejoe@gmail.com>;
Cc: "Jungtaek Lim"<kabhwan.opensource@gmail.com>;"Chris Nauroth"<cnauroth@apache.org>;"Maciej"<mszymkiewicz@gmail.com>;"dev"<dev@spark.apache.org>;
Subject: Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications
+1, I'm looking forward to it!
On Thu, Nov 17, 2022 at 9:44 AM Ye Zhou <zhouyejoe@gmail.com> wrote:
+1 (non-binding)Thanks for proposing this improvement to SHS, it resolves the main performance issue within SHS.
On Wed, Nov 16, 2022 at 1:15 PM Jungtaek Lim <kabhwan.opensource@gmail.com> wrote:
+1
Nice to see the chance for driver to reduce resource usage and increase stability, especially the fact that the driver is SPOF. It's even promising to have a future plan to pre-bake the kvstore for SHS from the driver.
Thanks for driving the effort, Gengliang!
On Thu, Nov 17, 2022 at 5:32 AM Chris Nauroth <cnauroth@apache.org> wrote:
+1 (non-binding)
Gengliang, thank you for the SPIP.
Chris Nauroth
On Wed, Nov 16, 2022 at 4:27 AM Maciej <mszymkiewicz@gmail.com> wrote:
+1
On 11/16/22 13:19, Yuming Wang wrote:
> +1, non-binding
>
> On Wed, Nov 16, 2022 at 8:12 PM Yang,Jie(INF) <yangjie01@baidu.com
> <mailto:yangjie01@baidu.com>> wrote:
>
> +1, non-binding____
>
> __ __
>
> Yang Jie____
>
> __ __
>
> *发件人**: *Mridul Muralidharan <mridul@gmail.com
> <mailto:mridul@gmail.com>>
> *日期**: *2022年11月16日星期三17:35
> *收件人**: *Kent Yao <yao@apache.org <mailto:yao@apache.org>>
> *抄送**: *Gengliang Wang <ltnwgl@gmail.com
> <mailto:ltnwgl@gmail.com>>, dev <dev@spark.apache.org
> <mailto:dev@spark.apache.org>>
> *主题**: *Re: [VOTE][SPIP] Better Spark UI scalability and Driver
> stability for large applications____
>
> __ __
>
> __ __
>
> +1____
>
> __ __
>
> Would be great to see history server performance improvements and
> lower resource utilization at driver !____
>
> __ __
>
> Regards,____
>
> Mridul ____
>
> __ __
>
> On Wed, Nov 16, 2022 at 2:38 AM Kent Yao <yao@apache.org
> <mailto:yao@apache.org>> wrote:____
>
> +1, non-binding
>
> Gengliang Wang <ltnwgl@gmail.com <mailto:ltnwgl@gmail.com>> 于
> 2022年11月16日周三16:36写道：
> >
> > Hi all,
> >
> > I’d like to start a vote for SPIP: "Better Spark UI scalability and Driver stability for large applications"
> >
> > The goal of the SPIP is to improve the Driver's stability by supporting storing Spark's UI data on RocksDB. Furthermore, to fasten the read and write operations on RocksDB, it introduces a new Protobuf serializer.
> >
> > Please also refer to the following:
> >
> > Previous discussion in the dev mailing list: [DISCUSS] SPIP: Better Spark UI scalability and Driver stability for large applications
> > Design Doc: Better Spark UI scalability and Driver stability for large applications
> > JIRA: SPARK-41053
> >
> >
> > Please vote on the SPIP for the next 72 hours:
> >
> > [ ] +1: Accept the proposal as an official SPIP
> > [ ] +0
> > [ ] -1: I don’t think this is a good idea because …
> >
> > Kind Regards,
> > Gengliang
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
> <mailto:dev-unsubscribe@spark.apache.org>____
>
--
Best regards,
Maciej Szymkiewicz
Web: https://zero323.net
PGP: A30CEF0C31A501EC
--
Zhou, Ye 周晔
Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications
Posted by Wenchen Fan <cl...@gmail.com>.
+1, I'm looking forward to it!
On Thu, Nov 17, 2022 at 9:44 AM Ye Zhou <zh...@gmail.com> wrote:
> +1 (non-binding)
> Thanks for proposing this improvement to SHS, it resolves the main
> performance issue within SHS.
>
> On Wed, Nov 16, 2022 at 1:15 PM Jungtaek Lim <ka...@gmail.com>
> wrote:
>
>> +1
>>
>> Nice to see the chance for driver to reduce resource usage and increase
>> stability, especially the fact that the driver is SPOF. It's even promising
>> to have a future plan to pre-bake the kvstore for SHS from the driver.
>>
>> Thanks for driving the effort, Gengliang!
>>
>> On Thu, Nov 17, 2022 at 5:32 AM Chris Nauroth <cn...@apache.org>
>> wrote:
>>
>>> +1 (non-binding)
>>>
>>> Gengliang, thank you for the SPIP.
>>>
>>> Chris Nauroth
>>>
>>>
>>> On Wed, Nov 16, 2022 at 4:27 AM Maciej <ms...@gmail.com> wrote:
>>>
>>>> +1
>>>>
>>>> On 11/16/22 13:19, Yuming Wang wrote:
>>>> > +1, non-binding
>>>> >
>>>> > On Wed, Nov 16, 2022 at 8:12 PM Yang,Jie(INF) <yangjie01@baidu.com
>>>> > <ma...@baidu.com>> wrote:
>>>> >
>>>> > +1, non-binding____
>>>> >
>>>> > __ __
>>>> >
>>>> > Yang Jie____
>>>> >
>>>> > __ __
>>>> >
>>>> > *发件人**: *Mridul Muralidharan <mridul@gmail.com
>>>> > <ma...@gmail.com>>
>>>> > *日期**: *2022年11月16日星期三17:35
>>>> > *收件人**: *Kent Yao <yao@apache.org <ma...@apache.org>>
>>>> > *抄送**: *Gengliang Wang <ltnwgl@gmail.com
>>>> > <ma...@gmail.com>>, dev <dev@spark.apache.org
>>>> > <ma...@spark.apache.org>>
>>>> > *主题**: *Re: [VOTE][SPIP] Better Spark UI scalability and Driver
>>>> > stability for large applications____
>>>> >
>>>> > __ __
>>>> >
>>>> > __ __
>>>> >
>>>> > +1____
>>>> >
>>>> > __ __
>>>> >
>>>> > Would be great to see history server performance improvements and
>>>> > lower resource utilization at driver !____
>>>> >
>>>> > __ __
>>>> >
>>>> > Regards,____
>>>> >
>>>> > Mridul ____
>>>> >
>>>> > __ __
>>>> >
>>>> > On Wed, Nov 16, 2022 at 2:38 AM Kent Yao <yao@apache.org
>>>> > <ma...@apache.org>> wrote:____
>>>> >
>>>> > +1, non-binding
>>>> >
>>>> > Gengliang Wang <ltnwgl@gmail.com <ma...@gmail.com>> 于
>>>> > 2022年11月16日周三16:36写道：
>>>> > >
>>>> > > Hi all,
>>>> > >
>>>> > > I’d like to start a vote for SPIP: "Better Spark UI
>>>> scalability and Driver stability for large applications"
>>>> > >
>>>> > > The goal of the SPIP is to improve the Driver's stability
>>>> by supporting storing Spark's UI data on RocksDB. Furthermore, to fasten
>>>> the read and write operations on RocksDB, it introduces a new Protobuf
>>>> serializer.
>>>> > >
>>>> > > Please also refer to the following:
>>>> > >
>>>> > > Previous discussion in the dev mailing list: [DISCUSS]
>>>> SPIP: Better Spark UI scalability and Driver stability for large
>>>> applications
>>>> > > Design Doc: Better Spark UI scalability and Driver
>>>> stability for large applications
>>>> > > JIRA: SPARK-41053
>>>> > >
>>>> > >
>>>> > > Please vote on the SPIP for the next 72 hours:
>>>> > >
>>>> > > [ ] +1: Accept the proposal as an official SPIP
>>>> > > [ ] +0
>>>> > > [ ] -1: I don’t think this is a good idea because …
>>>> > >
>>>> > > Kind Regards,
>>>> > > Gengliang
>>>> >
>>>> >
>>>> ---------------------------------------------------------------------
>>>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>> > <ma...@spark.apache.org>____
>>>> >
>>>>
>>>> --
>>>> Best regards,
>>>> Maciej Szymkiewicz
>>>>
>>>> Web: https://zero323.net
>>>> PGP: A30CEF0C31A501EC
>>>>
>>>>
>
> --
>
> *Zhou, Ye **周晔*
>
Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications
Posted by Ye Zhou <zh...@gmail.com>.
+1 (non-binding)
Thanks for proposing this improvement to SHS, it resolves the main
performance issue within SHS.
On Wed, Nov 16, 2022 at 1:15 PM Jungtaek Lim <ka...@gmail.com>
wrote:
> +1
>
> Nice to see the chance for driver to reduce resource usage and increase
> stability, especially the fact that the driver is SPOF. It's even promising
> to have a future plan to pre-bake the kvstore for SHS from the driver.
>
> Thanks for driving the effort, Gengliang!
>
> On Thu, Nov 17, 2022 at 5:32 AM Chris Nauroth <cn...@apache.org> wrote:
>
>> +1 (non-binding)
>>
>> Gengliang, thank you for the SPIP.
>>
>> Chris Nauroth
>>
>>
>> On Wed, Nov 16, 2022 at 4:27 AM Maciej <ms...@gmail.com> wrote:
>>
>>> +1
>>>
>>> On 11/16/22 13:19, Yuming Wang wrote:
>>> > +1, non-binding
>>> >
>>> > On Wed, Nov 16, 2022 at 8:12 PM Yang,Jie(INF) <yangjie01@baidu.com
>>> > <ma...@baidu.com>> wrote:
>>> >
>>> > +1, non-binding____
>>> >
>>> > __ __
>>> >
>>> > Yang Jie____
>>> >
>>> > __ __
>>> >
>>> > *发件人**: *Mridul Muralidharan <mridul@gmail.com
>>> > <ma...@gmail.com>>
>>> > *日期**: *2022年11月16日星期三17:35
>>> > *收件人**: *Kent Yao <yao@apache.org <ma...@apache.org>>
>>> > *抄送**: *Gengliang Wang <ltnwgl@gmail.com
>>> > <ma...@gmail.com>>, dev <dev@spark.apache.org
>>> > <ma...@spark.apache.org>>
>>> > *主题**: *Re: [VOTE][SPIP] Better Spark UI scalability and Driver
>>> > stability for large applications____
>>> >
>>> > __ __
>>> >
>>> > __ __
>>> >
>>> > +1____
>>> >
>>> > __ __
>>> >
>>> > Would be great to see history server performance improvements and
>>> > lower resource utilization at driver !____
>>> >
>>> > __ __
>>> >
>>> > Regards,____
>>> >
>>> > Mridul ____
>>> >
>>> > __ __
>>> >
>>> > On Wed, Nov 16, 2022 at 2:38 AM Kent Yao <yao@apache.org
>>> > <ma...@apache.org>> wrote:____
>>> >
>>> > +1, non-binding
>>> >
>>> > Gengliang Wang <ltnwgl@gmail.com <ma...@gmail.com>> 于
>>> > 2022年11月16日周三16:36写道：
>>> > >
>>> > > Hi all,
>>> > >
>>> > > I’d like to start a vote for SPIP: "Better Spark UI
>>> scalability and Driver stability for large applications"
>>> > >
>>> > > The goal of the SPIP is to improve the Driver's stability by
>>> supporting storing Spark's UI data on RocksDB. Furthermore, to fasten the
>>> read and write operations on RocksDB, it introduces a new Protobuf
>>> serializer.
>>> > >
>>> > > Please also refer to the following:
>>> > >
>>> > > Previous discussion in the dev mailing list: [DISCUSS] SPIP:
>>> Better Spark UI scalability and Driver stability for large applications
>>> > > Design Doc: Better Spark UI scalability and Driver stability
>>> for large applications
>>> > > JIRA: SPARK-41053
>>> > >
>>> > >
>>> > > Please vote on the SPIP for the next 72 hours:
>>> > >
>>> > > [ ] +1: Accept the proposal as an official SPIP
>>> > > [ ] +0
>>> > > [ ] -1: I don’t think this is a good idea because …
>>> > >
>>> > > Kind Regards,
>>> > > Gengliang
>>> >
>>> >
>>> ---------------------------------------------------------------------
>>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>> > <ma...@spark.apache.org>____
>>> >
>>>
>>> --
>>> Best regards,
>>> Maciej Szymkiewicz
>>>
>>> Web: https://zero323.net
>>> PGP: A30CEF0C31A501EC
>>>
>>>
--
*Zhou, Ye **周晔*
Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications
Posted by Jungtaek Lim <ka...@gmail.com>.
+1
Nice to see the chance for driver to reduce resource usage and increase
stability, especially the fact that the driver is SPOF. It's even promising
to have a future plan to pre-bake the kvstore for SHS from the driver.
Thanks for driving the effort, Gengliang!
On Thu, Nov 17, 2022 at 5:32 AM Chris Nauroth <cn...@apache.org> wrote:
> +1 (non-binding)
>
> Gengliang, thank you for the SPIP.
>
> Chris Nauroth
>
>
> On Wed, Nov 16, 2022 at 4:27 AM Maciej <ms...@gmail.com> wrote:
>
>> +1
>>
>> On 11/16/22 13:19, Yuming Wang wrote:
>> > +1, non-binding
>> >
>> > On Wed, Nov 16, 2022 at 8:12 PM Yang,Jie(INF) <yangjie01@baidu.com
>> > <ma...@baidu.com>> wrote:
>> >
>> > +1, non-binding____
>> >
>> > __ __
>> >
>> > Yang Jie____
>> >
>> > __ __
>> >
>> > *发件人**: *Mridul Muralidharan <mridul@gmail.com
>> > <ma...@gmail.com>>
>> > *日期**: *2022年11月16日星期三17:35
>> > *收件人**: *Kent Yao <yao@apache.org <ma...@apache.org>>
>> > *抄送**: *Gengliang Wang <ltnwgl@gmail.com
>> > <ma...@gmail.com>>, dev <dev@spark.apache.org
>> > <ma...@spark.apache.org>>
>> > *主题**: *Re: [VOTE][SPIP] Better Spark UI scalability and Driver
>> > stability for large applications____
>> >
>> > __ __
>> >
>> > __ __
>> >
>> > +1____
>> >
>> > __ __
>> >
>> > Would be great to see history server performance improvements and
>> > lower resource utilization at driver !____
>> >
>> > __ __
>> >
>> > Regards,____
>> >
>> > Mridul ____
>> >
>> > __ __
>> >
>> > On Wed, Nov 16, 2022 at 2:38 AM Kent Yao <yao@apache.org
>> > <ma...@apache.org>> wrote:____
>> >
>> > +1, non-binding
>> >
>> > Gengliang Wang <ltnwgl@gmail.com <ma...@gmail.com>> 于
>> > 2022年11月16日周三16:36写道：
>> > >
>> > > Hi all,
>> > >
>> > > I’d like to start a vote for SPIP: "Better Spark UI
>> scalability and Driver stability for large applications"
>> > >
>> > > The goal of the SPIP is to improve the Driver's stability by
>> supporting storing Spark's UI data on RocksDB. Furthermore, to fasten the
>> read and write operations on RocksDB, it introduces a new Protobuf
>> serializer.
>> > >
>> > > Please also refer to the following:
>> > >
>> > > Previous discussion in the dev mailing list: [DISCUSS] SPIP:
>> Better Spark UI scalability and Driver stability for large applications
>> > > Design Doc: Better Spark UI scalability and Driver stability
>> for large applications
>> > > JIRA: SPARK-41053
>> > >
>> > >
>> > > Please vote on the SPIP for the next 72 hours:
>> > >
>> > > [ ] +1: Accept the proposal as an official SPIP
>> > > [ ] +0
>> > > [ ] -1: I don’t think this is a good idea because …
>> > >
>> > > Kind Regards,
>> > > Gengliang
>> >
>> >
>> ---------------------------------------------------------------------
>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>> > <ma...@spark.apache.org>____
>> >
>>
>> --
>> Best regards,
>> Maciej Szymkiewicz
>>
>> Web: https://zero323.net
>> PGP: A30CEF0C31A501EC
>>
>>
Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications
Posted by Chris Nauroth <cn...@apache.org>.
+1 (non-binding)
Gengliang, thank you for the SPIP.
Chris Nauroth
On Wed, Nov 16, 2022 at 4:27 AM Maciej <ms...@gmail.com> wrote:
> +1
>
> On 11/16/22 13:19, Yuming Wang wrote:
> > +1, non-binding
> >
> > On Wed, Nov 16, 2022 at 8:12 PM Yang,Jie(INF) <yangjie01@baidu.com
> > <ma...@baidu.com>> wrote:
> >
> > +1, non-binding____
> >
> > __ __
> >
> > Yang Jie____
> >
> > __ __
> >
> > *发件人**: *Mridul Muralidharan <mridul@gmail.com
> > <ma...@gmail.com>>
> > *日期**: *2022年11月16日星期三17:35
> > *收件人**: *Kent Yao <yao@apache.org <ma...@apache.org>>
> > *抄送**: *Gengliang Wang <ltnwgl@gmail.com
> > <ma...@gmail.com>>, dev <dev@spark.apache.org
> > <ma...@spark.apache.org>>
> > *主题**: *Re: [VOTE][SPIP] Better Spark UI scalability and Driver
> > stability for large applications____
> >
> > __ __
> >
> > __ __
> >
> > +1____
> >
> > __ __
> >
> > Would be great to see history server performance improvements and
> > lower resource utilization at driver !____
> >
> > __ __
> >
> > Regards,____
> >
> > Mridul ____
> >
> > __ __
> >
> > On Wed, Nov 16, 2022 at 2:38 AM Kent Yao <yao@apache.org
> > <ma...@apache.org>> wrote:____
> >
> > +1, non-binding
> >
> > Gengliang Wang <ltnwgl@gmail.com <ma...@gmail.com>> 于
> > 2022年11月16日周三16:36写道：
> > >
> > > Hi all,
> > >
> > > I’d like to start a vote for SPIP: "Better Spark UI
> scalability and Driver stability for large applications"
> > >
> > > The goal of the SPIP is to improve the Driver's stability by
> supporting storing Spark's UI data on RocksDB. Furthermore, to fasten the
> read and write operations on RocksDB, it introduces a new Protobuf
> serializer.
> > >
> > > Please also refer to the following:
> > >
> > > Previous discussion in the dev mailing list: [DISCUSS] SPIP:
> Better Spark UI scalability and Driver stability for large applications
> > > Design Doc: Better Spark UI scalability and Driver stability
> for large applications
> > > JIRA: SPARK-41053
> > >
> > >
> > > Please vote on the SPIP for the next 72 hours:
> > >
> > > [ ] +1: Accept the proposal as an official SPIP
> > > [ ] +0
> > > [ ] -1: I don’t think this is a good idea because …
> > >
> > > Kind Regards,
> > > Gengliang
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
> > <ma...@spark.apache.org>____
> >
>
> --
> Best regards,
> Maciej Szymkiewicz
>
> Web: https://zero323.net
> PGP: A30CEF0C31A501EC
>
>
Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications
Posted by Maciej <ms...@gmail.com>.
+1
On 11/16/22 13:19, Yuming Wang wrote:
> +1, non-binding
>
> On Wed, Nov 16, 2022 at 8:12 PM Yang,Jie(INF) <yangjie01@baidu.com
> <ma...@baidu.com>> wrote:
>
> +1, non-binding____
>
> __ __
>
> Yang Jie____
>
> __ __
>
> *发件人**: *Mridul Muralidharan <mridul@gmail.com
> <ma...@gmail.com>>
> *日期**: *2022年11月16日星期三17:35
> *收件人**: *Kent Yao <yao@apache.org <ma...@apache.org>>
> *抄送**: *Gengliang Wang <ltnwgl@gmail.com
> <ma...@gmail.com>>, dev <dev@spark.apache.org
> <ma...@spark.apache.org>>
> *主题**: *Re: [VOTE][SPIP] Better Spark UI scalability and Driver
> stability for large applications____
>
> __ __
>
> __ __
>
> +1____
>
> __ __
>
> Would be great to see history server performance improvements and
> lower resource utilization at driver !____
>
> __ __
>
> Regards,____
>
> Mridul ____
>
> __ __
>
> On Wed, Nov 16, 2022 at 2:38 AM Kent Yao <yao@apache.org
> <ma...@apache.org>> wrote:____
>
> +1, non-binding
>
> Gengliang Wang <ltnwgl@gmail.com <ma...@gmail.com>> 于
> 2022年11月16日周三16:36写道：
> >
> > Hi all,
> >
> > I’d like to start a vote for SPIP: "Better Spark UI scalability and Driver stability for large applications"
> >
> > The goal of the SPIP is to improve the Driver's stability by supporting storing Spark's UI data on RocksDB. Furthermore, to fasten the read and write operations on RocksDB, it introduces a new Protobuf serializer.
> >
> > Please also refer to the following:
> >
> > Previous discussion in the dev mailing list: [DISCUSS] SPIP: Better Spark UI scalability and Driver stability for large applications
> > Design Doc: Better Spark UI scalability and Driver stability for large applications
> > JIRA: SPARK-41053
> >
> >
> > Please vote on the SPIP for the next 72 hours:
> >
> > [ ] +1: Accept the proposal as an official SPIP
> > [ ] +0
> > [ ] -1: I don’t think this is a good idea because …
> >
> > Kind Regards,
> > Gengliang
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
> <ma...@spark.apache.org>____
>
--
Best regards,
Maciej Szymkiewicz
Web: https://zero323.net
PGP: A30CEF0C31A501EC
Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications
Posted by Yuming Wang <wg...@gmail.com>.
+1, non-binding
On Wed, Nov 16, 2022 at 8:12 PM Yang,Jie(INF) <ya...@baidu.com> wrote:
> +1, non-binding
>
>
>
> Yang Jie
>
>
>
> *发件人**: *Mridul Muralidharan <mr...@gmail.com>
> *日期**: *2022年11月16日 星期三 17:35
> *收件人**: *Kent Yao <ya...@apache.org>
> *抄送**: *Gengliang Wang <lt...@gmail.com>, dev <de...@spark.apache.org>
> *主题**: *Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability
> for large applications
>
>
>
>
>
> +1
>
>
>
> Would be great to see history server performance improvements and lower
> resource utilization at driver !
>
>
>
> Regards,
>
> Mridul
>
>
>
> On Wed, Nov 16, 2022 at 2:38 AM Kent Yao <ya...@apache.org> wrote:
>
> +1, non-binding
>
> Gengliang Wang <lt...@gmail.com> 于2022年11月16日周三 16:36写道：
> >
> > Hi all,
> >
> > I’d like to start a vote for SPIP: "Better Spark UI scalability and
> Driver stability for large applications"
> >
> > The goal of the SPIP is to improve the Driver's stability by supporting
> storing Spark's UI data on RocksDB. Furthermore, to fasten the read and
> write operations on RocksDB, it introduces a new Protobuf serializer.
> >
> > Please also refer to the following:
> >
> > Previous discussion in the dev mailing list: [DISCUSS] SPIP: Better
> Spark UI scalability and Driver stability for large applications
> > Design Doc: Better Spark UI scalability and Driver stability for large
> applications
> > JIRA: SPARK-41053
> >
> >
> > Please vote on the SPIP for the next 72 hours:
> >
> > [ ] +1: Accept the proposal as an official SPIP
> > [ ] +0
> > [ ] -1: I don’t think this is a good idea because …
> >
> > Kind Regards,
> > Gengliang
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications
Posted by "Yang,Jie(INF)" <ya...@baidu.com>.
+1, non-binding
Yang Jie
发件人: Mridul Muralidharan <mr...@gmail.com>
日期: 2022年11月16日 星期三 17:35
收件人: Kent Yao <ya...@apache.org>
抄送: Gengliang Wang <lt...@gmail.com>, dev <de...@spark.apache.org>
主题: Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications
+1
Would be great to see history server performance improvements and lower resource utilization at driver !
Regards,
Mridul
On Wed, Nov 16, 2022 at 2:38 AM Kent Yao <ya...@apache.org>> wrote:
+1, non-binding
Gengliang Wang <lt...@gmail.com>> 于2022年11月16日周三 16:36写道：
>
> Hi all,
>
> I’d like to start a vote for SPIP: "Better Spark UI scalability and Driver stability for large applications"
>
> The goal of the SPIP is to improve the Driver's stability by supporting storing Spark's UI data on RocksDB. Furthermore, to fasten the read and write operations on RocksDB, it introduces a new Protobuf serializer.
>
> Please also refer to the following:
>
> Previous discussion in the dev mailing list: [DISCUSS] SPIP: Better Spark UI scalability and Driver stability for large applications
> Design Doc: Better Spark UI scalability and Driver stability for large applications
> JIRA: SPARK-41053
>
>
> Please vote on the SPIP for the next 72 hours:
>
> [ ] +1: Accept the proposal as an official SPIP
> [ ] +0
> [ ] -1: I don’t think this is a good idea because …
>
> Kind Regards,
> Gengliang
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org<ma...@spark.apache.org>
Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications
Posted by Mridul Muralidharan <mr...@gmail.com>.
+1
Would be great to see history server performance improvements and lower
resource utilization at driver !
Regards,
Mridul
On Wed, Nov 16, 2022 at 2:38 AM Kent Yao <ya...@apache.org> wrote:
> +1, non-binding
>
> Gengliang Wang <lt...@gmail.com> 于2022年11月16日周三 16:36写道：
> >
> > Hi all,
> >
> > I’d like to start a vote for SPIP: "Better Spark UI scalability and
> Driver stability for large applications"
> >
> > The goal of the SPIP is to improve the Driver's stability by supporting
> storing Spark's UI data on RocksDB. Furthermore, to fasten the read and
> write operations on RocksDB, it introduces a new Protobuf serializer.
> >
> > Please also refer to the following:
> >
> > Previous discussion in the dev mailing list: [DISCUSS] SPIP: Better
> Spark UI scalability and Driver stability for large applications
> > Design Doc: Better Spark UI scalability and Driver stability for large
> applications
> > JIRA: SPARK-41053
> >
> >
> > Please vote on the SPIP for the next 72 hours:
> >
> > [ ] +1: Accept the proposal as an official SPIP
> > [ ] +0
> > [ ] -1: I don’t think this is a good idea because …
> >
> > Kind Regards,
> > Gengliang
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications
Posted by Kent Yao <ya...@apache.org>.
+1, non-binding
Gengliang Wang <lt...@gmail.com> 于2022年11月16日周三 16:36写道：
>
> Hi all,
>
> I’d like to start a vote for SPIP: "Better Spark UI scalability and Driver stability for large applications"
>
> The goal of the SPIP is to improve the Driver's stability by supporting storing Spark's UI data on RocksDB. Furthermore, to fasten the read and write operations on RocksDB, it introduces a new Protobuf serializer.
>
> Please also refer to the following:
>
> Previous discussion in the dev mailing list: [DISCUSS] SPIP: Better Spark UI scalability and Driver stability for large applications
> Design Doc: Better Spark UI scalability and Driver stability for large applications
> JIRA: SPARK-41053
>
>
> Please vote on the SPIP for the next 72 hours:
>
> [ ] +1: Accept the proposal as an official SPIP
> [ ] +0
> [ ] -1: I don’t think this is a good idea because …
>
> Kind Regards,
> Gengliang
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org