You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@livy.apache.org by Renat Bekbolatov <re...@microsoft.com.INVALID> on 2022/10/25 02:30:00 UTC
RE: [EXTERNAL] Re: Proposal to Revive Apache Livy Community
I welcome this development - honestly, possibly long overdue.
Would it be possible to propose additional committers for a wider representation?
Thank you,
Renat
-----Original Message-----
From: larry mccay <lm...@apache.org>
Sent: Monday, October 24, 2022 4:41 PM
To: dev@livy.apache.org
Subject: [EXTERNAL] Re: Proposal to Revive Apache Livy Community
Again, I have no objections to this and will be happy to include you.
Thanks for calling out!
On Mon, Oct 24, 2022 at 5:38 PM Marco Gaido <mg...@apache.org> wrote:
> Hi all,
>
> I am also busy with different projects at the moment, as such I am
> more than happy if ai can help somehow, e.g. reviewing code in parts
> that I know, but I cannot allocate much bandwidth to it.
>
> Thanks,
> Marco
>
> Il lun 24 ott 2022, 22:07 larry mccay <lm...@apache.org> ha scritto:
>
> > Personally, I would welcome it.
> > Any notes of guidance and support as we ramp up the team and make
> > the
> first
> > few releases would be invaluable.
> >
> >
> > On Mon, Oct 24, 2022 at 3:49 PM Alex Bozarth <aj...@us.ibm.com>
> wrote:
> >
> > > I would love to stay on as a PPMC, but if my lack of availability
> > concerns
> > > the new PPMC then I'm willing to be emeritus.
> > >
> > >
> > > Alex Bozarth
> > > Jupyter Architect, IBM CODAIT
> > > GitHub: ajbozarth
> > >
> > > On 10/24/22, 2:46 PM, "larry mccay" <lm...@apache.org> wrote:
> > >
> > > Oh, very good, Alex!
> > > Be sure to let us know if you want to remain on the project in
> > > any capacity.
> > >
> > >
> > > On Mon, Oct 24, 2022 at 3:18 PM Alex Bozarth
> > > <aj...@us.ibm.com>
> > > wrote:
> > >
> > > > As the only Livy PPMC still responding to the Mentors on the
> > private
> > > list,
> > > > I have updated the Livy Podling report for November with the
> status
> > > of the
> > > > project and a request to the IPMC to review this proposal
> > > since there are
> > > > not enough active Livy PPMC members to reach a quorum to
> > > pass the proposal.
> > > >
> > > > As a current Livy PPMC I strongly support this proposal for
> > > revitalization
> > > > as I do not have enough bandwidth to dedicate to Livy.
> > > >
> > > >
> > > > Alex Bozarth
> > > > Jupyter Architect, IBM CODAIT
> > > > GitHub: ajbozarth
> > > >
> > > > On 10/24/22, 1:41 PM, "larry mccay" <lm...@apache.org> wrote:
> > > >
> > > > Gentle reminder that we need to determine next steps here.
> > > > We have an updated proposal on this thread.
> > > > Do we need a VOTE or can we move forward directly to
> adjusting
> > > the
> > > > members,
> > > > etc?
> > > >
> > > > Thanks!
> > > >
> > > > --larry
> > > >
> > > > On Thu, Oct 20, 2022 at 3:26 PM larry mccay <
> lmccay@apache.org
> > >
> > > wrote:
> > > >
> > > > > @Justin Mclean <ju...@me.com> - any insights on
> next
> > > steps
> > > > here?
> > > > >
> > > > >
> > > > > On Tue, Oct 18, 2022 at 5:44 PM larry mccay <
> > lmccay@apache.org
> > > >
> > > > wrote:
> > > > >
> > > > >> Very good, here is the latest revision with updated
> Mentors.
> > > > >> Sunil and I have been added to the IPMC as well.
> > > > >> Welcome Madhawa and thanks for stepping up as a Mentor for
> > > Livy!
> > > > >>
> > > > >> Abstract
> > > > >>
> > > > >> Livy is a web service that exposes a REST interface for
> > > managing
> > > > long
> > > > >> running Apache Spark contexts in your cluster. With Livy,
> > new
> > > > applications
> > > > >> can be built on top of Apache Spark that require fine
> > grained
> > > > interaction
> > > > >> with many Spark contexts [1].
> > > > >>
> > > > >> While this project has been well regarded and used in many
> > > contexts
> > > > as
> > > > >> the defacto standard API to Spark environments, it has
> been
> > > > incubating for
> > > > >> over 5 years without graduation to a TLP and it has become
> > > > difficult to
> > > > >> impossible for fixes and improvements to be contributed as
> > the
> > > > current
> > > > >> community seems to have moved on.
> > > > >>
> > > > >> There has been discussion regarding retirement of this
> > > podling where
> > > > >> there seems to be some increasing interest in joining and
> > > reviving
> > > > the
> > > > >> community [2].
> > > > >>
> > > > >> The intent of this proposal is to avoid retiring a well
> > > regarded,
> > > > >> actively used and rather mature project by reviving the
> PPMC
> > > and
> > > > community
> > > > >> with new folks that have a vested interest in the project
> > and
> > > > health of the
> > > > >> community.
> > > > >> Proposal
> > > > >>
> > > > >> We propose to revive the PPMC with a set of contributors
> and
> > > > maintainers
> > > > >> as mentors, PPMC members and committers.
> > > > >>
> > > > >> The retirement DISCUSS thread [2] has shown a growing
> > > interest in
> > > > >> providing new committers and bringing improvements and
> fixes
> > > from
> > > > >> organization’s internally maintained forks back to a
> revived
> > > > community.
> > > > >>
> > > > >> General Approach to Revival:
> > > > >>
> > > > >> -
> > > > >>
> > > > >> Add new Mentors
> > > > >> -
> > > > >>
> > > > >> Larry McCay, lmccay@apache.org , Cloudera
> > > > >> -
> > > > >>
> > > > >> Sunil Govindan, sunilg@apache.org, Cloudera
> > > > >> -
> > > > >>
> > > > >> Jean-Baptiste Onofré, jbonofre@apache.org, Talend
> > > > >> -
> > > > >>
> > > > >> Madhawa Gunasekara, madhawa30@gmail.com,
> Independent
> > > > >>
> > > > >>
> > > > >>
> > > > >> -
> > > > >>
> > > > >> Add new Committers/PPMC
> > > > >> -
> > > > >>
> > > > >> Larry McCay, lmccay@apache.org, Cloudera
> > > > >> -
> > > > >>
> > > > >> Vinod Kumar Vavilapalli, vinodkv@cloudera.com,
> > Cloudera
> > > > >> -
> > > > >>
> > > > >> Imran Rashid - irashid@apache.org, Cloudera
> > > > >> -
> > > > >>
> > > > >> Gyorgy Gal, ggal ,gal.gyorgy@gmail.com, Cloudera
> > > > >> -
> > > > >>
> > > > >> Wing Yew Poon, wypoon@cloudera.com, Cloudera
> > > > >> -
> > > > >>
> > > > >> Xilang Yan, xilang.yan@gmail.com, Shopee
> > > > >> -
> > > > >>
> > > > >> Jianzhen Wu, myjianzhen@gmail.com, Shopee
> > > > >> -
> > > > >>
> > > > >> Nagella Jagadeewara Rao, jnagella@visa.com, Visa
> > > > >> -
> > > > >>
> > > > >> Pralab Kumar, pralkuma@visa.com, Visa
> > > > >> -
> > > > >>
> > > > >> Prasad Shrikant, shrikant.ism@gmail.com, Visa
> > > > >> -
> > > > >>
> > > > >> Brahma Reddy Battula, brahma@apache.org, Visa
> > > > >>
> > > > >>
> > > > >>
> > > > >> -
> > > > >>
> > > > >> Invite existing PPMC members to opt-in or otherwise go
> > > emeritus
> > > > >> -
> > > > >>
> > > > >> Jean-Baptiste Onofré, jbonofre@apache.org, Talend
> > > (opted-in
> > > > via
> > > > >> Retirement DISCUSS thread [2])
> > > > >>
> > > > >>
> > > > >>
> > > > >> -
> > > > >>
> > > > >> Invite existing Committers to opt-out or otherwise
> > continue
> > > > >>
> > > > >>
> > > > >>
> > > > >> -
> > > > >>
> > > > >> Establish Roadmap via follow up DISCUSS thread
> > > > >> -
> > > > >>
> > > > >> Known Improvements from Forks which will need
> > proposals
> > > and
> > > > >> discussion:
> > > > >> -
> > > > >>
> > > > >> Adding HA for Livy
> > > > >> -
> > > > >>
> > > > >> Updating security capabilities (eg. kerberos for
> > > jdbc,
> > > > fixing
> > > > >> bugs in encryption)
> > > > >> -
> > > > >>
> > > > >> Expanding the support for kubernetes
> > > > >> -
> > > > >>
> > > > >> Responding to CVEs in dependencies (eg. log4j,
> > > thrift)
> > > > >> -
> > > > >>
> > > > >> Livy rest cluster - IS THIS SAME AS HA for Livy
> > > ABOVE?
> > > > >> -
> > > > >>
> > > > >> Support multi Spark versions
> > > > >> -
> > > > >>
> > > > >> Implemented a metrics system for Livy
> > > > >> -
> > > > >>
> > > > >> Support customize batch/interactive session
> > > lifecycle event
> > > > >> handler, default log event with log4j, very
> helpful
> > > for
> > > > troubleshooting
> > > > >> -
> > > > >>
> > > > >> Optimize log to track which session id the log
> > > message came
> > > > >> from, also very helpful for troubleshooting
> > > > >> -
> > > > >>
> > > > >> Support customize Spark config optimization
> rules,
> > > can be
> > > > used
> > > > >> to optimize config for users’ job
> > > > >> -
> > > > >>
> > > > >> A set of command line tool which can be used to
> > > replace
> > > > Spark’s
> > > > >> spark-submit, pyspark, spark-sql but actually
> > submit
> > > > application in Livy
> > > > >> -
> > > > >>
> > > > >> We are planning to implement a JDBC state store,
> > and
> > > allow
> > > > >> multi Livy Thrift sessions to share one backend
> > Spark
> > > > application in the
> > > > >> next few months.
> > > > >> -
> > > > >>
> > > > >> These items and others that are brought to community
> > > may need
> > > > >> consolidation or multiple configurable options and
> > will
> > > need
> > > > to be part of
> > > > >> the discussion
> > > > >> -
> > > > >>
> > > > >> One-pager Livy Improvement Proposals (LIP) may
> make
> > > sense
> > > > to
> > > > >> drive these discussions and convergence
> > > > >> -
> > > > >>
> > > > >> Feature Branch Strategy for large changes
> > > > >> -
> > > > >>
> > > > >> Large features are hard to review we will need
> > to
> > > > define a
> > > > >> strategy
> > > > >> -
> > > > >>
> > > > >> Determine the Improvements to be delivered across
> > first
> > > 3
> > > > Releases
> > > > >> with Target Release Dates
> > > > >>
> > > > >>
> > > > >>
> > > > >> -
> > > > >>
> > > > >> Ensure CVE and Dependency management hygiene is in
> place
> > > > >>
> > > > >>
> > > > >> The above approach will usher the community back to an
> > active
> > > > status with
> > > > >> a Roadmap of 3 or more release plans and security hygiene
> in
> > > place.
> > > > >> Development Practices
> > > > >>
> > > > >> The Livy project follows a review before commit
> philosophy.
> > > Every
> > > > commit
> > > > >> automatically runs through the unit tests and generates
> > > coverage
> > > > reports
> > > > >> presented as a pull request comment. Our experience with
> > this
> > > > process leads
> > > > >> us to believe that it helps ease new contributors into the
> > > project.
> > > > They
> > > > >> get feedback quickly on common mistakes, lowering the
> burden
> > > on
> > > > reviewers.
> > > > >> Those same reviewers get to lead by example, showing the
> new
> > > > contributors
> > > > >> that we value feedback within our community even when
> > changes
> > > are
> > > > done by
> > > > >> more experienced folks. Taken from the original Apache
> Livy
> > > > Proposal [1],
> > > > >> this should continue to be true. As mentioned, Livy is a
> > > mature
> > > > project and
> > > > >> as such RTC is the most appropriate for continued quality
> > and
> > > > awareness.
> > > > >>
> > > > >>
> > > > >>
> > > > >> 1.
> > > > >>
> > > > >> Original Apache Livy Proposal
> > > > >>
> > > >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwik
> i.apache.org%2Fconfluence%2Fdisplay%2Fincubator%2FLivyProposal&dat
> a=05%7C01%7Crebekbol%40microsoft.com%7C5ead1754350a408bceba08dab6192ee
> 9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638022516535784379%7CUn
> known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haW
> wiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=rbccZA%2BUAUHjpEolIiY8ig%2B
> 5nRw5abmw9oiwJGkz8TM%3D&reserved=0
> > > > >> 2.
> > > > >>
> > > > >> Retirement DISCUSS thread
> > > > >>
> > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fgcstsrhbp91c5mm55htqn1l3djv8m7o0&data=05%7C01%7Crebekbol%40microsoft.com%7C5ead1754350a408bceba08dab6192ee9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638022516535940162%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dgaMvjevmL84UXIvyHZ3OgCDX%2FXs7ZdKZvNXR1UR5A8%3D&reserved=0
> > > >
> > > > >>
> > > > >>
> > > > >> On Tue, Oct 18, 2022 at 5:05 PM Madhawa Gunasekara <
> > > > madhawa30@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >>> Hi Larry,
> > > > >>>
> > > > >>> I'm an IPMC Member. madhawa30 at gmail dot com is my
> > > preferred
> > > > email
> > > > >>> address.
> > > > >>> apache id: madhawa
> > > > >>>
> > > > >>> Thanks,
> > > > >>> Madhawa
> > > > >>>
> > > > >>>
> > > > >>> On Tue, Oct 18, 2022 at 10:05 PM larry mccay <
> > > lmccay@apache.org>
> > > > wrote:
> > > > >>>
> > > > >>> > Hi Madhawa -
> > > > >>> >
> > > > >>> > That's awesome!
> > > > >>> > Are you already a member of IPMC?
> > > > >>> > If not, are you an ASF member?
> > > > >>> > If you are an ASF member you can request that you be
> > added
> > > as an
> > > > IPMC
> > > > >>> > member.
> > > > >>> >
> > > > >>> > Can you provide your company affiliation for the
> proposal
> > > and
> > > > preferred
> > > > >>> > email?
> > > > >>> >
> > > > >>> > thanks!
> > > > >>> >
> > > > >>> > --larry
> > > > >>> >
> > > > >>> > On Tue, Oct 18, 2022 at 2:18 PM Madhawa Gunasekara <
> > > > >>> madhawa30@gmail.com>
> > > > >>> > wrote:
> > > > >>> >
> > > > >>> > > Hi Larry,
> > > > >>> > >
> > > > >>> > > I'm interested in working with Livy and would like to
> > > join as a
> > > > >>> Mentor.
> > > > >>> > >
> > > > >>> > > Thanks,
> > > > >>> > > Madhawa
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > On Tue, Oct 18, 2022 at 6:57 PM larry mccay <
> > > lmccay@apache.org
> > > > >
> > > > >>> wrote:
> > > > >>> > >
> > > > >>> > > > Sorry, I missed commenting on this:
> > > > >>> > > >
> > > > >>> > > > "There is also no concept as an emeritus PPMC
> member
> > > at the
> > > > ASF."
> > > > >>> > > >
> > > > >>> > > > I assume that we can remove PPMC members that do
> not
> > > opt-in
> > > > >>> explicitly
> > > > >>> > at
> > > > >>> > > > this point.
> > > > >>> > > > They will have every opportunity to rejoin.
> > > > >>> > > >
> > > > >>> > > > On Tue, Oct 18, 2022 at 12:48 PM larry mccay <
> > > > lmccay@apache.org>
> > > > >>> > wrote:
> > > > >>> > > >
> > > > >>> > > > > I will ask in a separate thread, @Justin Mclean <
> > > > >>> justinmclean@me.com
> > > > >>> > >
> > > > >>> > > -
> > > > >>> > > > > thanks.
> > > > >>> > > > > Adding JB adds another company and we are
> certainly
> > > open to
> > > > >>> anyone
> > > > >>> > else
> > > > >>> > > > > that would like to join as a mentor.
> > > > >>> > > > > At the end of the day, the mentors are for
> > > instilling the
> > > > Apache
> > > > >>> Way
> > > > >>> > > > > knowledge and steering toward graduation.
> > > > >>> > > > > I feel that this diversity, while nice to have,
> is
> > > less
> > > > important
> > > > >>> > than
> > > > >>> > > > > that of the PPMC and committers for the long term
> > > health
> > > > of the
> > > > >>> > > > community.
> > > > >>> > > > >
> > > > >>> > > > > We need to push this podling to graduation as
> > > quickly as
> > > > possible
> > > > >>> > since
> > > > >>> > > > it
> > > > >>> > > > > is rather mature and needs to get to the next
> > level.
> > > > >>> > > > >
> > > > >>> > > > > Again, any potential Mentors that would like to
> > join
> > > are
> > > > more
> > > > >>> than
> > > > >>> > > > welcome.
> > > > >>> > > > >
> > > > >>> > > > > On Tue, Oct 18, 2022 at 12:38 PM Justin Mclean <
> > > > >>> > > justin@classsoftware.com
> > > > >>> > > > >
> > > > >>> > > > > wrote:
> > > > >>> > > > >
> > > > >>> > > > >> Hi,
> > > > >>> > > > >>
> > > > >>> > > > >> I’m sorry, but Imran Rashid can’t be a mentor
> for
> > > the
> > > > project as
> > > > >>> > they
> > > > >>> > > > are
> > > > >>> > > > >> not an IPMC member. Currently, both Sunil and
> > Larry
> > > (as
> > > > they
> > > > >>> are ASF
> > > > >>> > > > >> members) need to ask to join the IPMC and NOTICE
> > > sent to
> > > > the ASF
> > > > >>> > > board.
> > > > >>> > > > I
> > > > >>> > > > >> would also prefer that mentors come from
> different
> > > > companies.
> > > > >>> > > > >>
> > > > >>> > > > >> There is also no concept as an emeritus PPMC
> > member
> > > at
> > > > the ASF.
> > > > >>> > > > >>
> > > > >>> > > > >> Kind Regards,
> > > > >>> > > > >> Justin
> > > > >>> > > > >>
> > > > >>> > > > >>
> > > > >>> > > > >>
> > > > >>> >
> > > >
> > --------------------------------------------------------------------
> > -
> > > > >>> > > > >> To unsubscribe, e-mail:
> > > > >>> general-unsubscribe@incubator.apache.org
> > > > >>> > > > >> For additional commands, e-mail:
> > > > >>> general-help@incubator.apache.org
> > > > >>> > > > >>
> > > > >>> > > > >>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > > >>
> > > >
> > > >
> > > >
> > --------------------------------------------------------------------
> > -
> > > > To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail:
> general-help@incubator.apache.org
> > > >
> > >
> > >
> >
>
Re: [EXTERNAL] Re: Proposal to Revive Apache Livy Community
Posted by larry mccay <lm...@apache.org>.
Hi Renat -
More active contributors/committers would be great!
Let's get the names before I craft the NOTICE to add new members email
later today.
If that isn't realistic then we can always add more after the fact as well.
That's what is great about re/building the community in the incubator!
thanks!
--larry
On Mon, Oct 24, 2022 at 10:30 PM Renat Bekbolatov
<re...@microsoft.com.invalid> wrote:
> I welcome this development - honestly, possibly long overdue.
> Would it be possible to propose additional committers for a wider
> representation?
>
>
> Thank you,
> Renat
>
> -----Original Message-----
> From: larry mccay <lm...@apache.org>
> Sent: Monday, October 24, 2022 4:41 PM
> To: dev@livy.apache.org
> Subject: [EXTERNAL] Re: Proposal to Revive Apache Livy Community
>
> Again, I have no objections to this and will be happy to include you.
> Thanks for calling out!
>
> On Mon, Oct 24, 2022 at 5:38 PM Marco Gaido <mg...@apache.org> wrote:
>
> > Hi all,
> >
> > I am also busy with different projects at the moment, as such I am
> > more than happy if ai can help somehow, e.g. reviewing code in parts
> > that I know, but I cannot allocate much bandwidth to it.
> >
> > Thanks,
> > Marco
> >
> > Il lun 24 ott 2022, 22:07 larry mccay <lm...@apache.org> ha scritto:
> >
> > > Personally, I would welcome it.
> > > Any notes of guidance and support as we ramp up the team and make
> > > the
> > first
> > > few releases would be invaluable.
> > >
> > >
> > > On Mon, Oct 24, 2022 at 3:49 PM Alex Bozarth <aj...@us.ibm.com>
> > wrote:
> > >
> > > > I would love to stay on as a PPMC, but if my lack of availability
> > > concerns
> > > > the new PPMC then I'm willing to be emeritus.
> > > >
> > > >
> > > > Alex Bozarth
> > > > Jupyter Architect, IBM CODAIT
> > > > GitHub: ajbozarth
> > > >
> > > > On 10/24/22, 2:46 PM, "larry mccay" <lm...@apache.org> wrote:
> > > >
> > > > Oh, very good, Alex!
> > > > Be sure to let us know if you want to remain on the project in
> > > > any capacity.
> > > >
> > > >
> > > > On Mon, Oct 24, 2022 at 3:18 PM Alex Bozarth
> > > > <aj...@us.ibm.com>
> > > > wrote:
> > > >
> > > > > As the only Livy PPMC still responding to the Mentors on the
> > > private
> > > > list,
> > > > > I have updated the Livy Podling report for November with the
> > status
> > > > of the
> > > > > project and a request to the IPMC to review this proposal
> > > > since there are
> > > > > not enough active Livy PPMC members to reach a quorum to
> > > > pass the proposal.
> > > > >
> > > > > As a current Livy PPMC I strongly support this proposal for
> > > > revitalization
> > > > > as I do not have enough bandwidth to dedicate to Livy.
> > > > >
> > > > >
> > > > > Alex Bozarth
> > > > > Jupyter Architect, IBM CODAIT
> > > > > GitHub: ajbozarth
> > > > >
> > > > > On 10/24/22, 1:41 PM, "larry mccay" <lm...@apache.org> wrote:
> > > > >
> > > > > Gentle reminder that we need to determine next steps here.
> > > > > We have an updated proposal on this thread.
> > > > > Do we need a VOTE or can we move forward directly to
> > adjusting
> > > > the
> > > > > members,
> > > > > etc?
> > > > >
> > > > > Thanks!
> > > > >
> > > > > --larry
> > > > >
> > > > > On Thu, Oct 20, 2022 at 3:26 PM larry mccay <
> > lmccay@apache.org
> > > >
> > > > wrote:
> > > > >
> > > > > > @Justin Mclean <ju...@me.com> - any insights on
> > next
> > > > steps
> > > > > here?
> > > > > >
> > > > > >
> > > > > > On Tue, Oct 18, 2022 at 5:44 PM larry mccay <
> > > lmccay@apache.org
> > > > >
> > > > > wrote:
> > > > > >
> > > > > >> Very good, here is the latest revision with updated
> > Mentors.
> > > > > >> Sunil and I have been added to the IPMC as well.
> > > > > >> Welcome Madhawa and thanks for stepping up as a Mentor
> for
> > > > Livy!
> > > > > >>
> > > > > >> Abstract
> > > > > >>
> > > > > >> Livy is a web service that exposes a REST interface for
> > > > managing
> > > > > long
> > > > > >> running Apache Spark contexts in your cluster. With
> Livy,
> > > new
> > > > > applications
> > > > > >> can be built on top of Apache Spark that require fine
> > > grained
> > > > > interaction
> > > > > >> with many Spark contexts [1].
> > > > > >>
> > > > > >> While this project has been well regarded and used in
> many
> > > > contexts
> > > > > as
> > > > > >> the defacto standard API to Spark environments, it has
> > been
> > > > > incubating for
> > > > > >> over 5 years without graduation to a TLP and it has
> become
> > > > > difficult to
> > > > > >> impossible for fixes and improvements to be contributed
> as
> > > the
> > > > > current
> > > > > >> community seems to have moved on.
> > > > > >>
> > > > > >> There has been discussion regarding retirement of this
> > > > podling where
> > > > > >> there seems to be some increasing interest in joining
> and
> > > > reviving
> > > > > the
> > > > > >> community [2].
> > > > > >>
> > > > > >> The intent of this proposal is to avoid retiring a well
> > > > regarded,
> > > > > >> actively used and rather mature project by reviving the
> > PPMC
> > > > and
> > > > > community
> > > > > >> with new folks that have a vested interest in the
> project
> > > and
> > > > > health of the
> > > > > >> community.
> > > > > >> Proposal
> > > > > >>
> > > > > >> We propose to revive the PPMC with a set of contributors
> > and
> > > > > maintainers
> > > > > >> as mentors, PPMC members and committers.
> > > > > >>
> > > > > >> The retirement DISCUSS thread [2] has shown a growing
> > > > interest in
> > > > > >> providing new committers and bringing improvements and
> > fixes
> > > > from
> > > > > >> organization’s internally maintained forks back to a
> > revived
> > > > > community.
> > > > > >>
> > > > > >> General Approach to Revival:
> > > > > >>
> > > > > >> -
> > > > > >>
> > > > > >> Add new Mentors
> > > > > >> -
> > > > > >>
> > > > > >> Larry McCay, lmccay@apache.org , Cloudera
> > > > > >> -
> > > > > >>
> > > > > >> Sunil Govindan, sunilg@apache.org, Cloudera
> > > > > >> -
> > > > > >>
> > > > > >> Jean-Baptiste Onofré, jbonofre@apache.org, Talend
> > > > > >> -
> > > > > >>
> > > > > >> Madhawa Gunasekara, madhawa30@gmail.com,
> > Independent
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> -
> > > > > >>
> > > > > >> Add new Committers/PPMC
> > > > > >> -
> > > > > >>
> > > > > >> Larry McCay, lmccay@apache.org, Cloudera
> > > > > >> -
> > > > > >>
> > > > > >> Vinod Kumar Vavilapalli, vinodkv@cloudera.com,
> > > Cloudera
> > > > > >> -
> > > > > >>
> > > > > >> Imran Rashid - irashid@apache.org, Cloudera
> > > > > >> -
> > > > > >>
> > > > > >> Gyorgy Gal, ggal ,gal.gyorgy@gmail.com, Cloudera
> > > > > >> -
> > > > > >>
> > > > > >> Wing Yew Poon, wypoon@cloudera.com, Cloudera
> > > > > >> -
> > > > > >>
> > > > > >> Xilang Yan, xilang.yan@gmail.com, Shopee
> > > > > >> -
> > > > > >>
> > > > > >> Jianzhen Wu, myjianzhen@gmail.com, Shopee
> > > > > >> -
> > > > > >>
> > > > > >> Nagella Jagadeewara Rao, jnagella@visa.com, Visa
> > > > > >> -
> > > > > >>
> > > > > >> Pralab Kumar, pralkuma@visa.com, Visa
> > > > > >> -
> > > > > >>
> > > > > >> Prasad Shrikant, shrikant.ism@gmail.com, Visa
> > > > > >> -
> > > > > >>
> > > > > >> Brahma Reddy Battula, brahma@apache.org, Visa
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> -
> > > > > >>
> > > > > >> Invite existing PPMC members to opt-in or otherwise
> go
> > > > emeritus
> > > > > >> -
> > > > > >>
> > > > > >> Jean-Baptiste Onofré, jbonofre@apache.org, Talend
> > > > (opted-in
> > > > > via
> > > > > >> Retirement DISCUSS thread [2])
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> -
> > > > > >>
> > > > > >> Invite existing Committers to opt-out or otherwise
> > > continue
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> -
> > > > > >>
> > > > > >> Establish Roadmap via follow up DISCUSS thread
> > > > > >> -
> > > > > >>
> > > > > >> Known Improvements from Forks which will need
> > > proposals
> > > > and
> > > > > >> discussion:
> > > > > >> -
> > > > > >>
> > > > > >> Adding HA for Livy
> > > > > >> -
> > > > > >>
> > > > > >> Updating security capabilities (eg. kerberos
> for
> > > > jdbc,
> > > > > fixing
> > > > > >> bugs in encryption)
> > > > > >> -
> > > > > >>
> > > > > >> Expanding the support for kubernetes
> > > > > >> -
> > > > > >>
> > > > > >> Responding to CVEs in dependencies (eg. log4j,
> > > > thrift)
> > > > > >> -
> > > > > >>
> > > > > >> Livy rest cluster - IS THIS SAME AS HA for Livy
> > > > ABOVE?
> > > > > >> -
> > > > > >>
> > > > > >> Support multi Spark versions
> > > > > >> -
> > > > > >>
> > > > > >> Implemented a metrics system for Livy
> > > > > >> -
> > > > > >>
> > > > > >> Support customize batch/interactive session
> > > > lifecycle event
> > > > > >> handler, default log event with log4j, very
> > helpful
> > > > for
> > > > > troubleshooting
> > > > > >> -
> > > > > >>
> > > > > >> Optimize log to track which session id the log
> > > > message came
> > > > > >> from, also very helpful for troubleshooting
> > > > > >> -
> > > > > >>
> > > > > >> Support customize Spark config optimization
> > rules,
> > > > can be
> > > > > used
> > > > > >> to optimize config for users’ job
> > > > > >> -
> > > > > >>
> > > > > >> A set of command line tool which can be used to
> > > > replace
> > > > > Spark’s
> > > > > >> spark-submit, pyspark, spark-sql but actually
> > > submit
> > > > > application in Livy
> > > > > >> -
> > > > > >>
> > > > > >> We are planning to implement a JDBC state
> store,
> > > and
> > > > allow
> > > > > >> multi Livy Thrift sessions to share one backend
> > > Spark
> > > > > application in the
> > > > > >> next few months.
> > > > > >> -
> > > > > >>
> > > > > >> These items and others that are brought to
> community
> > > > may need
> > > > > >> consolidation or multiple configurable options and
> > > will
> > > > need
> > > > > to be part of
> > > > > >> the discussion
> > > > > >> -
> > > > > >>
> > > > > >> One-pager Livy Improvement Proposals (LIP) may
> > make
> > > > sense
> > > > > to
> > > > > >> drive these discussions and convergence
> > > > > >> -
> > > > > >>
> > > > > >> Feature Branch Strategy for large changes
> > > > > >> -
> > > > > >>
> > > > > >> Large features are hard to review we will
> need
> > > to
> > > > > define a
> > > > > >> strategy
> > > > > >> -
> > > > > >>
> > > > > >> Determine the Improvements to be delivered across
> > > first
> > > > 3
> > > > > Releases
> > > > > >> with Target Release Dates
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> -
> > > > > >>
> > > > > >> Ensure CVE and Dependency management hygiene is in
> > place
> > > > > >>
> > > > > >>
> > > > > >> The above approach will usher the community back to an
> > > active
> > > > > status with
> > > > > >> a Roadmap of 3 or more release plans and security
> hygiene
> > in
> > > > place.
> > > > > >> Development Practices
> > > > > >>
> > > > > >> The Livy project follows a review before commit
> > philosophy.
> > > > Every
> > > > > commit
> > > > > >> automatically runs through the unit tests and generates
> > > > coverage
> > > > > reports
> > > > > >> presented as a pull request comment. Our experience with
> > > this
> > > > > process leads
> > > > > >> us to believe that it helps ease new contributors into
> the
> > > > project.
> > > > > They
> > > > > >> get feedback quickly on common mistakes, lowering the
> > burden
> > > > on
> > > > > reviewers.
> > > > > >> Those same reviewers get to lead by example, showing the
> > new
> > > > > contributors
> > > > > >> that we value feedback within our community even when
> > > changes
> > > > are
> > > > > done by
> > > > > >> more experienced folks. Taken from the original Apache
> > Livy
> > > > > Proposal [1],
> > > > > >> this should continue to be true. As mentioned, Livy is a
> > > > mature
> > > > > project and
> > > > > >> as such RTC is the most appropriate for continued
> quality
> > > and
> > > > > awareness.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> 1.
> > > > > >>
> > > > > >> Original Apache Livy Proposal
> > > > > >>
> > > > >
> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwik
> > i.apache.org%2Fconfluence%2Fdisplay%2Fincubator%2FLivyProposal&dat
> > a=05%7C01%7Crebekbol%40microsoft.com%7C5ead1754350a408bceba08dab6192ee
> > 9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638022516535784379%7CUn
> > known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haW
> > wiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=rbccZA%2BUAUHjpEolIiY8ig%2B
> > 5nRw5abmw9oiwJGkz8TM%3D&reserved=0
> > > > > >> 2.
> > > > > >>
> > > > > >> Retirement DISCUSS thread
> > > > > >>
> > > >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fgcstsrhbp91c5mm55htqn1l3djv8m7o0&data=05%7C01%7Crebekbol%40microsoft.com%7C5ead1754350a408bceba08dab6192ee9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638022516535940162%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dgaMvjevmL84UXIvyHZ3OgCDX%2FXs7ZdKZvNXR1UR5A8%3D&reserved=0
> > > > >
> > > > > >>
> > > > > >>
> > > > > >> On Tue, Oct 18, 2022 at 5:05 PM Madhawa Gunasekara <
> > > > > madhawa30@gmail.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >>> Hi Larry,
> > > > > >>>
> > > > > >>> I'm an IPMC Member. madhawa30 at gmail dot com is my
> > > > preferred
> > > > > email
> > > > > >>> address.
> > > > > >>> apache id: madhawa
> > > > > >>>
> > > > > >>> Thanks,
> > > > > >>> Madhawa
> > > > > >>>
> > > > > >>>
> > > > > >>> On Tue, Oct 18, 2022 at 10:05 PM larry mccay <
> > > > lmccay@apache.org>
> > > > > wrote:
> > > > > >>>
> > > > > >>> > Hi Madhawa -
> > > > > >>> >
> > > > > >>> > That's awesome!
> > > > > >>> > Are you already a member of IPMC?
> > > > > >>> > If not, are you an ASF member?
> > > > > >>> > If you are an ASF member you can request that you be
> > > added
> > > > as an
> > > > > IPMC
> > > > > >>> > member.
> > > > > >>> >
> > > > > >>> > Can you provide your company affiliation for the
> > proposal
> > > > and
> > > > > preferred
> > > > > >>> > email?
> > > > > >>> >
> > > > > >>> > thanks!
> > > > > >>> >
> > > > > >>> > --larry
> > > > > >>> >
> > > > > >>> > On Tue, Oct 18, 2022 at 2:18 PM Madhawa Gunasekara <
> > > > > >>> madhawa30@gmail.com>
> > > > > >>> > wrote:
> > > > > >>> >
> > > > > >>> > > Hi Larry,
> > > > > >>> > >
> > > > > >>> > > I'm interested in working with Livy and would like
> to
> > > > join as a
> > > > > >>> Mentor.
> > > > > >>> > >
> > > > > >>> > > Thanks,
> > > > > >>> > > Madhawa
> > > > > >>> > >
> > > > > >>> > >
> > > > > >>> > > On Tue, Oct 18, 2022 at 6:57 PM larry mccay <
> > > > lmccay@apache.org
> > > > > >
> > > > > >>> wrote:
> > > > > >>> > >
> > > > > >>> > > > Sorry, I missed commenting on this:
> > > > > >>> > > >
> > > > > >>> > > > "There is also no concept as an emeritus PPMC
> > member
> > > > at the
> > > > > ASF."
> > > > > >>> > > >
> > > > > >>> > > > I assume that we can remove PPMC members that do
> > not
> > > > opt-in
> > > > > >>> explicitly
> > > > > >>> > at
> > > > > >>> > > > this point.
> > > > > >>> > > > They will have every opportunity to rejoin.
> > > > > >>> > > >
> > > > > >>> > > > On Tue, Oct 18, 2022 at 12:48 PM larry mccay <
> > > > > lmccay@apache.org>
> > > > > >>> > wrote:
> > > > > >>> > > >
> > > > > >>> > > > > I will ask in a separate thread, @Justin
> Mclean <
> > > > > >>> justinmclean@me.com
> > > > > >>> > >
> > > > > >>> > > -
> > > > > >>> > > > > thanks.
> > > > > >>> > > > > Adding JB adds another company and we are
> > certainly
> > > > open to
> > > > > >>> anyone
> > > > > >>> > else
> > > > > >>> > > > > that would like to join as a mentor.
> > > > > >>> > > > > At the end of the day, the mentors are for
> > > > instilling the
> > > > > Apache
> > > > > >>> Way
> > > > > >>> > > > > knowledge and steering toward graduation.
> > > > > >>> > > > > I feel that this diversity, while nice to have,
> > is
> > > > less
> > > > > important
> > > > > >>> > than
> > > > > >>> > > > > that of the PPMC and committers for the long
> term
> > > > health
> > > > > of the
> > > > > >>> > > > community.
> > > > > >>> > > > >
> > > > > >>> > > > > We need to push this podling to graduation as
> > > > quickly as
> > > > > possible
> > > > > >>> > since
> > > > > >>> > > > it
> > > > > >>> > > > > is rather mature and needs to get to the next
> > > level.
> > > > > >>> > > > >
> > > > > >>> > > > > Again, any potential Mentors that would like to
> > > join
> > > > are
> > > > > more
> > > > > >>> than
> > > > > >>> > > > welcome.
> > > > > >>> > > > >
> > > > > >>> > > > > On Tue, Oct 18, 2022 at 12:38 PM Justin Mclean
> <
> > > > > >>> > > justin@classsoftware.com
> > > > > >>> > > > >
> > > > > >>> > > > > wrote:
> > > > > >>> > > > >
> > > > > >>> > > > >> Hi,
> > > > > >>> > > > >>
> > > > > >>> > > > >> I’m sorry, but Imran Rashid can’t be a mentor
> > for
> > > > the
> > > > > project as
> > > > > >>> > they
> > > > > >>> > > > are
> > > > > >>> > > > >> not an IPMC member. Currently, both Sunil and
> > > Larry
> > > > (as
> > > > > they
> > > > > >>> are ASF
> > > > > >>> > > > >> members) need to ask to join the IPMC and
> NOTICE
> > > > sent to
> > > > > the ASF
> > > > > >>> > > board.
> > > > > >>> > > > I
> > > > > >>> > > > >> would also prefer that mentors come from
> > different
> > > > > companies.
> > > > > >>> > > > >>
> > > > > >>> > > > >> There is also no concept as an emeritus PPMC
> > > member
> > > > at
> > > > > the ASF.
> > > > > >>> > > > >>
> > > > > >>> > > > >> Kind Regards,
> > > > > >>> > > > >> Justin
> > > > > >>> > > > >>
> > > > > >>> > > > >>
> > > > > >>> > > > >>
> > > > > >>> >
> > > > >
> > > --------------------------------------------------------------------
> > > -
> > > > > >>> > > > >> To unsubscribe, e-mail:
> > > > > >>> general-unsubscribe@incubator.apache.org
> > > > > >>> > > > >> For additional commands, e-mail:
> > > > > >>> general-help@incubator.apache.org
> > > > > >>> > > > >>
> > > > > >>> > > > >>
> > > > > >>> > > >
> > > > > >>> > >
> > > > > >>> >
> > > > > >>>
> > > > > >>
> > > > >
> > > > >
> > > > >
> > > --------------------------------------------------------------------
> > > -
> > > > > To unsubscribe, e-mail:
> general-unsubscribe@incubator.apache.org
> > > > > For additional commands, e-mail:
> > general-help@incubator.apache.org
> > > > >
> > > >
> > > >
> > >
> >
>
Re: [EXTERNAL] Re: Proposal to Revive Apache Livy Community
Posted by larry mccay <lm...@apache.org>.
Hi Renat -
More active contributors/committers would be great!
Let's get the names before I craft the NOTICE to add new members email
later today.
If that isn't realistic then we can always add more after the fact as well.
That's what is great about re/building the community in the incubator!
thanks!
--larry
On Mon, Oct 24, 2022 at 10:30 PM Renat Bekbolatov
<re...@microsoft.com.invalid> wrote:
> I welcome this development - honestly, possibly long overdue.
> Would it be possible to propose additional committers for a wider
> representation?
>
>
> Thank you,
> Renat
>
> -----Original Message-----
> From: larry mccay <lm...@apache.org>
> Sent: Monday, October 24, 2022 4:41 PM
> To: dev@livy.apache.org
> Subject: [EXTERNAL] Re: Proposal to Revive Apache Livy Community
>
> Again, I have no objections to this and will be happy to include you.
> Thanks for calling out!
>
> On Mon, Oct 24, 2022 at 5:38 PM Marco Gaido <mg...@apache.org> wrote:
>
> > Hi all,
> >
> > I am also busy with different projects at the moment, as such I am
> > more than happy if ai can help somehow, e.g. reviewing code in parts
> > that I know, but I cannot allocate much bandwidth to it.
> >
> > Thanks,
> > Marco
> >
> > Il lun 24 ott 2022, 22:07 larry mccay <lm...@apache.org> ha scritto:
> >
> > > Personally, I would welcome it.
> > > Any notes of guidance and support as we ramp up the team and make
> > > the
> > first
> > > few releases would be invaluable.
> > >
> > >
> > > On Mon, Oct 24, 2022 at 3:49 PM Alex Bozarth <aj...@us.ibm.com>
> > wrote:
> > >
> > > > I would love to stay on as a PPMC, but if my lack of availability
> > > concerns
> > > > the new PPMC then I'm willing to be emeritus.
> > > >
> > > >
> > > > Alex Bozarth
> > > > Jupyter Architect, IBM CODAIT
> > > > GitHub: ajbozarth
> > > >
> > > > On 10/24/22, 2:46 PM, "larry mccay" <lm...@apache.org> wrote:
> > > >
> > > > Oh, very good, Alex!
> > > > Be sure to let us know if you want to remain on the project in
> > > > any capacity.
> > > >
> > > >
> > > > On Mon, Oct 24, 2022 at 3:18 PM Alex Bozarth
> > > > <aj...@us.ibm.com>
> > > > wrote:
> > > >
> > > > > As the only Livy PPMC still responding to the Mentors on the
> > > private
> > > > list,
> > > > > I have updated the Livy Podling report for November with the
> > status
> > > > of the
> > > > > project and a request to the IPMC to review this proposal
> > > > since there are
> > > > > not enough active Livy PPMC members to reach a quorum to
> > > > pass the proposal.
> > > > >
> > > > > As a current Livy PPMC I strongly support this proposal for
> > > > revitalization
> > > > > as I do not have enough bandwidth to dedicate to Livy.
> > > > >
> > > > >
> > > > > Alex Bozarth
> > > > > Jupyter Architect, IBM CODAIT
> > > > > GitHub: ajbozarth
> > > > >
> > > > > On 10/24/22, 1:41 PM, "larry mccay" <lm...@apache.org> wrote:
> > > > >
> > > > > Gentle reminder that we need to determine next steps here.
> > > > > We have an updated proposal on this thread.
> > > > > Do we need a VOTE or can we move forward directly to
> > adjusting
> > > > the
> > > > > members,
> > > > > etc?
> > > > >
> > > > > Thanks!
> > > > >
> > > > > --larry
> > > > >
> > > > > On Thu, Oct 20, 2022 at 3:26 PM larry mccay <
> > lmccay@apache.org
> > > >
> > > > wrote:
> > > > >
> > > > > > @Justin Mclean <ju...@me.com> - any insights on
> > next
> > > > steps
> > > > > here?
> > > > > >
> > > > > >
> > > > > > On Tue, Oct 18, 2022 at 5:44 PM larry mccay <
> > > lmccay@apache.org
> > > > >
> > > > > wrote:
> > > > > >
> > > > > >> Very good, here is the latest revision with updated
> > Mentors.
> > > > > >> Sunil and I have been added to the IPMC as well.
> > > > > >> Welcome Madhawa and thanks for stepping up as a Mentor
> for
> > > > Livy!
> > > > > >>
> > > > > >> Abstract
> > > > > >>
> > > > > >> Livy is a web service that exposes a REST interface for
> > > > managing
> > > > > long
> > > > > >> running Apache Spark contexts in your cluster. With
> Livy,
> > > new
> > > > > applications
> > > > > >> can be built on top of Apache Spark that require fine
> > > grained
> > > > > interaction
> > > > > >> with many Spark contexts [1].
> > > > > >>
> > > > > >> While this project has been well regarded and used in
> many
> > > > contexts
> > > > > as
> > > > > >> the defacto standard API to Spark environments, it has
> > been
> > > > > incubating for
> > > > > >> over 5 years without graduation to a TLP and it has
> become
> > > > > difficult to
> > > > > >> impossible for fixes and improvements to be contributed
> as
> > > the
> > > > > current
> > > > > >> community seems to have moved on.
> > > > > >>
> > > > > >> There has been discussion regarding retirement of this
> > > > podling where
> > > > > >> there seems to be some increasing interest in joining
> and
> > > > reviving
> > > > > the
> > > > > >> community [2].
> > > > > >>
> > > > > >> The intent of this proposal is to avoid retiring a well
> > > > regarded,
> > > > > >> actively used and rather mature project by reviving the
> > PPMC
> > > > and
> > > > > community
> > > > > >> with new folks that have a vested interest in the
> project
> > > and
> > > > > health of the
> > > > > >> community.
> > > > > >> Proposal
> > > > > >>
> > > > > >> We propose to revive the PPMC with a set of contributors
> > and
> > > > > maintainers
> > > > > >> as mentors, PPMC members and committers.
> > > > > >>
> > > > > >> The retirement DISCUSS thread [2] has shown a growing
> > > > interest in
> > > > > >> providing new committers and bringing improvements and
> > fixes
> > > > from
> > > > > >> organization’s internally maintained forks back to a
> > revived
> > > > > community.
> > > > > >>
> > > > > >> General Approach to Revival:
> > > > > >>
> > > > > >> -
> > > > > >>
> > > > > >> Add new Mentors
> > > > > >> -
> > > > > >>
> > > > > >> Larry McCay, lmccay@apache.org , Cloudera
> > > > > >> -
> > > > > >>
> > > > > >> Sunil Govindan, sunilg@apache.org, Cloudera
> > > > > >> -
> > > > > >>
> > > > > >> Jean-Baptiste Onofré, jbonofre@apache.org, Talend
> > > > > >> -
> > > > > >>
> > > > > >> Madhawa Gunasekara, madhawa30@gmail.com,
> > Independent
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> -
> > > > > >>
> > > > > >> Add new Committers/PPMC
> > > > > >> -
> > > > > >>
> > > > > >> Larry McCay, lmccay@apache.org, Cloudera
> > > > > >> -
> > > > > >>
> > > > > >> Vinod Kumar Vavilapalli, vinodkv@cloudera.com,
> > > Cloudera
> > > > > >> -
> > > > > >>
> > > > > >> Imran Rashid - irashid@apache.org, Cloudera
> > > > > >> -
> > > > > >>
> > > > > >> Gyorgy Gal, ggal ,gal.gyorgy@gmail.com, Cloudera
> > > > > >> -
> > > > > >>
> > > > > >> Wing Yew Poon, wypoon@cloudera.com, Cloudera
> > > > > >> -
> > > > > >>
> > > > > >> Xilang Yan, xilang.yan@gmail.com, Shopee
> > > > > >> -
> > > > > >>
> > > > > >> Jianzhen Wu, myjianzhen@gmail.com, Shopee
> > > > > >> -
> > > > > >>
> > > > > >> Nagella Jagadeewara Rao, jnagella@visa.com, Visa
> > > > > >> -
> > > > > >>
> > > > > >> Pralab Kumar, pralkuma@visa.com, Visa
> > > > > >> -
> > > > > >>
> > > > > >> Prasad Shrikant, shrikant.ism@gmail.com, Visa
> > > > > >> -
> > > > > >>
> > > > > >> Brahma Reddy Battula, brahma@apache.org, Visa
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> -
> > > > > >>
> > > > > >> Invite existing PPMC members to opt-in or otherwise
> go
> > > > emeritus
> > > > > >> -
> > > > > >>
> > > > > >> Jean-Baptiste Onofré, jbonofre@apache.org, Talend
> > > > (opted-in
> > > > > via
> > > > > >> Retirement DISCUSS thread [2])
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> -
> > > > > >>
> > > > > >> Invite existing Committers to opt-out or otherwise
> > > continue
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> -
> > > > > >>
> > > > > >> Establish Roadmap via follow up DISCUSS thread
> > > > > >> -
> > > > > >>
> > > > > >> Known Improvements from Forks which will need
> > > proposals
> > > > and
> > > > > >> discussion:
> > > > > >> -
> > > > > >>
> > > > > >> Adding HA for Livy
> > > > > >> -
> > > > > >>
> > > > > >> Updating security capabilities (eg. kerberos
> for
> > > > jdbc,
> > > > > fixing
> > > > > >> bugs in encryption)
> > > > > >> -
> > > > > >>
> > > > > >> Expanding the support for kubernetes
> > > > > >> -
> > > > > >>
> > > > > >> Responding to CVEs in dependencies (eg. log4j,
> > > > thrift)
> > > > > >> -
> > > > > >>
> > > > > >> Livy rest cluster - IS THIS SAME AS HA for Livy
> > > > ABOVE?
> > > > > >> -
> > > > > >>
> > > > > >> Support multi Spark versions
> > > > > >> -
> > > > > >>
> > > > > >> Implemented a metrics system for Livy
> > > > > >> -
> > > > > >>
> > > > > >> Support customize batch/interactive session
> > > > lifecycle event
> > > > > >> handler, default log event with log4j, very
> > helpful
> > > > for
> > > > > troubleshooting
> > > > > >> -
> > > > > >>
> > > > > >> Optimize log to track which session id the log
> > > > message came
> > > > > >> from, also very helpful for troubleshooting
> > > > > >> -
> > > > > >>
> > > > > >> Support customize Spark config optimization
> > rules,
> > > > can be
> > > > > used
> > > > > >> to optimize config for users’ job
> > > > > >> -
> > > > > >>
> > > > > >> A set of command line tool which can be used to
> > > > replace
> > > > > Spark’s
> > > > > >> spark-submit, pyspark, spark-sql but actually
> > > submit
> > > > > application in Livy
> > > > > >> -
> > > > > >>
> > > > > >> We are planning to implement a JDBC state
> store,
> > > and
> > > > allow
> > > > > >> multi Livy Thrift sessions to share one backend
> > > Spark
> > > > > application in the
> > > > > >> next few months.
> > > > > >> -
> > > > > >>
> > > > > >> These items and others that are brought to
> community
> > > > may need
> > > > > >> consolidation or multiple configurable options and
> > > will
> > > > need
> > > > > to be part of
> > > > > >> the discussion
> > > > > >> -
> > > > > >>
> > > > > >> One-pager Livy Improvement Proposals (LIP) may
> > make
> > > > sense
> > > > > to
> > > > > >> drive these discussions and convergence
> > > > > >> -
> > > > > >>
> > > > > >> Feature Branch Strategy for large changes
> > > > > >> -
> > > > > >>
> > > > > >> Large features are hard to review we will
> need
> > > to
> > > > > define a
> > > > > >> strategy
> > > > > >> -
> > > > > >>
> > > > > >> Determine the Improvements to be delivered across
> > > first
> > > > 3
> > > > > Releases
> > > > > >> with Target Release Dates
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> -
> > > > > >>
> > > > > >> Ensure CVE and Dependency management hygiene is in
> > place
> > > > > >>
> > > > > >>
> > > > > >> The above approach will usher the community back to an
> > > active
> > > > > status with
> > > > > >> a Roadmap of 3 or more release plans and security
> hygiene
> > in
> > > > place.
> > > > > >> Development Practices
> > > > > >>
> > > > > >> The Livy project follows a review before commit
> > philosophy.
> > > > Every
> > > > > commit
> > > > > >> automatically runs through the unit tests and generates
> > > > coverage
> > > > > reports
> > > > > >> presented as a pull request comment. Our experience with
> > > this
> > > > > process leads
> > > > > >> us to believe that it helps ease new contributors into
> the
> > > > project.
> > > > > They
> > > > > >> get feedback quickly on common mistakes, lowering the
> > burden
> > > > on
> > > > > reviewers.
> > > > > >> Those same reviewers get to lead by example, showing the
> > new
> > > > > contributors
> > > > > >> that we value feedback within our community even when
> > > changes
> > > > are
> > > > > done by
> > > > > >> more experienced folks. Taken from the original Apache
> > Livy
> > > > > Proposal [1],
> > > > > >> this should continue to be true. As mentioned, Livy is a
> > > > mature
> > > > > project and
> > > > > >> as such RTC is the most appropriate for continued
> quality
> > > and
> > > > > awareness.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> 1.
> > > > > >>
> > > > > >> Original Apache Livy Proposal
> > > > > >>
> > > > >
> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwik
> > i.apache.org%2Fconfluence%2Fdisplay%2Fincubator%2FLivyProposal&dat
> > a=05%7C01%7Crebekbol%40microsoft.com%7C5ead1754350a408bceba08dab6192ee
> > 9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638022516535784379%7CUn
> > known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haW
> > wiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=rbccZA%2BUAUHjpEolIiY8ig%2B
> > 5nRw5abmw9oiwJGkz8TM%3D&reserved=0
> > > > > >> 2.
> > > > > >>
> > > > > >> Retirement DISCUSS thread
> > > > > >>
> > > >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fgcstsrhbp91c5mm55htqn1l3djv8m7o0&data=05%7C01%7Crebekbol%40microsoft.com%7C5ead1754350a408bceba08dab6192ee9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638022516535940162%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dgaMvjevmL84UXIvyHZ3OgCDX%2FXs7ZdKZvNXR1UR5A8%3D&reserved=0
> > > > >
> > > > > >>
> > > > > >>
> > > > > >> On Tue, Oct 18, 2022 at 5:05 PM Madhawa Gunasekara <
> > > > > madhawa30@gmail.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >>> Hi Larry,
> > > > > >>>
> > > > > >>> I'm an IPMC Member. madhawa30 at gmail dot com is my
> > > > preferred
> > > > > email
> > > > > >>> address.
> > > > > >>> apache id: madhawa
> > > > > >>>
> > > > > >>> Thanks,
> > > > > >>> Madhawa
> > > > > >>>
> > > > > >>>
> > > > > >>> On Tue, Oct 18, 2022 at 10:05 PM larry mccay <
> > > > lmccay@apache.org>
> > > > > wrote:
> > > > > >>>
> > > > > >>> > Hi Madhawa -
> > > > > >>> >
> > > > > >>> > That's awesome!
> > > > > >>> > Are you already a member of IPMC?
> > > > > >>> > If not, are you an ASF member?
> > > > > >>> > If you are an ASF member you can request that you be
> > > added
> > > > as an
> > > > > IPMC
> > > > > >>> > member.
> > > > > >>> >
> > > > > >>> > Can you provide your company affiliation for the
> > proposal
> > > > and
> > > > > preferred
> > > > > >>> > email?
> > > > > >>> >
> > > > > >>> > thanks!
> > > > > >>> >
> > > > > >>> > --larry
> > > > > >>> >
> > > > > >>> > On Tue, Oct 18, 2022 at 2:18 PM Madhawa Gunasekara <
> > > > > >>> madhawa30@gmail.com>
> > > > > >>> > wrote:
> > > > > >>> >
> > > > > >>> > > Hi Larry,
> > > > > >>> > >
> > > > > >>> > > I'm interested in working with Livy and would like
> to
> > > > join as a
> > > > > >>> Mentor.
> > > > > >>> > >
> > > > > >>> > > Thanks,
> > > > > >>> > > Madhawa
> > > > > >>> > >
> > > > > >>> > >
> > > > > >>> > > On Tue, Oct 18, 2022 at 6:57 PM larry mccay <
> > > > lmccay@apache.org
> > > > > >
> > > > > >>> wrote:
> > > > > >>> > >
> > > > > >>> > > > Sorry, I missed commenting on this:
> > > > > >>> > > >
> > > > > >>> > > > "There is also no concept as an emeritus PPMC
> > member
> > > > at the
> > > > > ASF."
> > > > > >>> > > >
> > > > > >>> > > > I assume that we can remove PPMC members that do
> > not
> > > > opt-in
> > > > > >>> explicitly
> > > > > >>> > at
> > > > > >>> > > > this point.
> > > > > >>> > > > They will have every opportunity to rejoin.
> > > > > >>> > > >
> > > > > >>> > > > On Tue, Oct 18, 2022 at 12:48 PM larry mccay <
> > > > > lmccay@apache.org>
> > > > > >>> > wrote:
> > > > > >>> > > >
> > > > > >>> > > > > I will ask in a separate thread, @Justin
> Mclean <
> > > > > >>> justinmclean@me.com
> > > > > >>> > >
> > > > > >>> > > -
> > > > > >>> > > > > thanks.
> > > > > >>> > > > > Adding JB adds another company and we are
> > certainly
> > > > open to
> > > > > >>> anyone
> > > > > >>> > else
> > > > > >>> > > > > that would like to join as a mentor.
> > > > > >>> > > > > At the end of the day, the mentors are for
> > > > instilling the
> > > > > Apache
> > > > > >>> Way
> > > > > >>> > > > > knowledge and steering toward graduation.
> > > > > >>> > > > > I feel that this diversity, while nice to have,
> > is
> > > > less
> > > > > important
> > > > > >>> > than
> > > > > >>> > > > > that of the PPMC and committers for the long
> term
> > > > health
> > > > > of the
> > > > > >>> > > > community.
> > > > > >>> > > > >
> > > > > >>> > > > > We need to push this podling to graduation as
> > > > quickly as
> > > > > possible
> > > > > >>> > since
> > > > > >>> > > > it
> > > > > >>> > > > > is rather mature and needs to get to the next
> > > level.
> > > > > >>> > > > >
> > > > > >>> > > > > Again, any potential Mentors that would like to
> > > join
> > > > are
> > > > > more
> > > > > >>> than
> > > > > >>> > > > welcome.
> > > > > >>> > > > >
> > > > > >>> > > > > On Tue, Oct 18, 2022 at 12:38 PM Justin Mclean
> <
> > > > > >>> > > justin@classsoftware.com
> > > > > >>> > > > >
> > > > > >>> > > > > wrote:
> > > > > >>> > > > >
> > > > > >>> > > > >> Hi,
> > > > > >>> > > > >>
> > > > > >>> > > > >> I’m sorry, but Imran Rashid can’t be a mentor
> > for
> > > > the
> > > > > project as
> > > > > >>> > they
> > > > > >>> > > > are
> > > > > >>> > > > >> not an IPMC member. Currently, both Sunil and
> > > Larry
> > > > (as
> > > > > they
> > > > > >>> are ASF
> > > > > >>> > > > >> members) need to ask to join the IPMC and
> NOTICE
> > > > sent to
> > > > > the ASF
> > > > > >>> > > board.
> > > > > >>> > > > I
> > > > > >>> > > > >> would also prefer that mentors come from
> > different
> > > > > companies.
> > > > > >>> > > > >>
> > > > > >>> > > > >> There is also no concept as an emeritus PPMC
> > > member
> > > > at
> > > > > the ASF.
> > > > > >>> > > > >>
> > > > > >>> > > > >> Kind Regards,
> > > > > >>> > > > >> Justin
> > > > > >>> > > > >>
> > > > > >>> > > > >>
> > > > > >>> > > > >>
> > > > > >>> >
> > > > >
> > > --------------------------------------------------------------------
> > > -
> > > > > >>> > > > >> To unsubscribe, e-mail:
> > > > > >>> general-unsubscribe@incubator.apache.org
> > > > > >>> > > > >> For additional commands, e-mail:
> > > > > >>> general-help@incubator.apache.org
> > > > > >>> > > > >>
> > > > > >>> > > > >>
> > > > > >>> > > >
> > > > > >>> > >
> > > > > >>> >
> > > > > >>>
> > > > > >>
> > > > >
> > > > >
> > > > >
> > > --------------------------------------------------------------------
> > > -
> > > > > To unsubscribe, e-mail:
> general-unsubscribe@incubator.apache.org
> > > > > For additional commands, e-mail:
> > general-help@incubator.apache.org
> > > > >
> > > >
> > > >
> > >
> >
>