You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@slider.apache.org by Gour Saha <gs...@hortonworks.com> on 2016/11/02 01:44:50 UTC
First draft of Apache Slider November 2016 report
Please review and provide feedback on the draft below.
=========================================================
Slider
Slider is a collection of tools and technologies to package, deploy, and
manage long running applications on Apache Hadoop YARN clusters.
Slider has been incubating since 2014-04-29.
Three most important issues to address in the move towards graduation:
1. Apache Slider community/PPMC has voted to move portions of Slider into
Apache Hadoop YARN as modules. It is possible that the remaining pieces
will be moved at a later point in time or become obsolete or evolve to
work closely with YARN. Slider PPMC will decide what makes the most
sense as we progress through this exciting time.
2. Getting more external users
3. Growth of a diverse set of developers/committers/PMC members is also
crucial towards the final state of Slider
Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware
of?
No.
How has the community developed since the last report?
The Apache Hadoop YARN community/PMC and Apache Slider community/PPMC
had decided to move portions of Slider into YARN, to make fast and significant
progress on: YARN-4692 - [Umbrella] Simplified and first-class support for
services in YARN. As a result, portions like Slider Core, Application Master,
Client and Tests had been rolled into Apache Hadoop YARN as module(s).
This effort is being captured in: YARN-5079 : Native YARN framework layer
for services. All the migrations are happening in a branch (namely
yarn-native-services) created of off Hadoop trunk. As a result of this,
we are already seeing interest in the Hadoop committers/PMC, who have
started to contribute and submit patches to Slider.
In order to support existing users of Slider, and to provide seamless
migration, there will be sufficient overlap between the time when a stable
state of long running services support is available in some future version
of YARN and the time till an independent Slider release is available. The
community/PPMC will also determine the future state of Slider as we
navigate through these changes.
The discussions on the Slider and YARN community DLs can be viewed here -
https://s.apache.org/0hoh
https://s.apache.org/MncV
For the benefit of those who would like to use Slider Core from the
Hadoop codebase, but continue to use classic Slider Agent and legacy
app-packages, a new branch (yarn-native-services) has been created
in classic Slider repo. This branch retains only the Agent (python) code
and the App Packages. It adds Slider Core module as a Hadoop
dependency. This allows users to consume the latest of Slider Core.
Future classic Slider releases can be made purely off of this branch.
Additionally an effort is being made to create an agent-less provider for
legacy app-packages, which could help users to completely migrate to
Hadoop codebase for creating long-running services with Slider. Expectation
from app-package owners would be to make minor modification of their
packages. Primary changes would be to shed the dependency on python
Agent code, effectively making packages much simpler than what they
look like today in classic Slider.
How has the project developed since the last report?
There has been fewer bug fixes on the classic Slider side as most of
the focus has been on developing features on native YARN services in
the Hadoop codebase. Work continues on support for complex services
(assemblies) and agent-less applications in Slider. The efforts on Slider
Core will continue in the new services branch created in YARN. Few
key features which were merged to classic Slider as well were -
SLIDER-875 - Ability to create an Uber application package with
capability to deploy and manage as a single business app, and
SLIDER-1107 - Generate app configuration files in AM. Several issues
identified by Coverity scans were resolved as well. Additionally few
patches were contributed by the community fixing functional and
performance issues. Slider community plans to ship a release in the
next quarter with all these features and bug fixes.
Date of last release:
2016-06-28 slider-0.91.0-incubating
When were the last committers or PMC members elected?
2015-07-07: Yu (Thomas) Liu
Re: First draft of Apache Slider November 2016 report
Posted by Josh Elser <jo...@gmail.com>.
LGTM, Gour. Thanks for writing this up.
Gour Saha wrote:
> Please review and provide feedback on the draft below.
>
> =========================================================
> Slider
>
> Slider is a collection of tools and technologies to package, deploy, and
> manage long running applications on Apache Hadoop YARN clusters.
>
> Slider has been incubating since 2014-04-29.
>
> Three most important issues to address in the move towards graduation:
>
> 1. Apache Slider community/PPMC has voted to move portions of Slider into
> Apache Hadoop YARN as modules. It is possible that the remaining pieces
> will be moved at a later point in time or become obsolete or evolve to
> work closely with YARN. Slider PPMC will decide what makes the most
> sense as we progress through this exciting time.
> 2. Getting more external users
> 3. Growth of a diverse set of developers/committers/PMC members is also
> crucial towards the final state of Slider
>
> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware
> of?
>
> No.
>
> How has the community developed since the last report?
>
> The Apache Hadoop YARN community/PMC and Apache Slider community/PPMC
> had decided to move portions of Slider into YARN, to make fast and significant
> progress on: YARN-4692 - [Umbrella] Simplified and first-class support for
> services in YARN. As a result, portions like Slider Core, Application Master,
> Client and Tests had been rolled into Apache Hadoop YARN as module(s).
> This effort is being captured in: YARN-5079 : Native YARN framework layer
> for services. All the migrations are happening in a branch (namely
> yarn-native-services) created of off Hadoop trunk. As a result of this,
> we are already seeing interest in the Hadoop committers/PMC, who have
> started to contribute and submit patches to Slider.
>
> In order to support existing users of Slider, and to provide seamless
> migration, there will be sufficient overlap between the time when a stable
> state of long running services support is available in some future version
> of YARN and the time till an independent Slider release is available. The
> community/PPMC will also determine the future state of Slider as we
> navigate through these changes.
>
> The discussions on the Slider and YARN community DLs can be viewed here -
>
> https://s.apache.org/0hoh
> https://s.apache.org/MncV
>
> For the benefit of those who would like to use Slider Core from the
> Hadoop codebase, but continue to use classic Slider Agent and legacy
> app-packages, a new branch (yarn-native-services) has been created
> in classic Slider repo. This branch retains only the Agent (python) code
> and the App Packages. It adds Slider Core module as a Hadoop
> dependency. This allows users to consume the latest of Slider Core.
> Future classic Slider releases can be made purely off of this branch.
>
> Additionally an effort is being made to create an agent-less provider for
> legacy app-packages, which could help users to completely migrate to
> Hadoop codebase for creating long-running services with Slider. Expectation
> from app-package owners would be to make minor modification of their
> packages. Primary changes would be to shed the dependency on python
> Agent code, effectively making packages much simpler than what they
> look like today in classic Slider.
>
> How has the project developed since the last report?
>
> There has been fewer bug fixes on the classic Slider side as most of
> the focus has been on developing features on native YARN services in
> the Hadoop codebase. Work continues on support for complex services
> (assemblies) and agent-less applications in Slider. The efforts on Slider
> Core will continue in the new services branch created in YARN. Few
> key features which were merged to classic Slider as well were -
> SLIDER-875 - Ability to create an Uber application package with
> capability to deploy and manage as a single business app, and
> SLIDER-1107 - Generate app configuration files in AM. Several issues
> identified by Coverity scans were resolved as well. Additionally few
> patches were contributed by the community fixing functional and
> performance issues. Slider community plans to ship a release in the
> next quarter with all these features and bug fixes.
>
> Date of last release:
>
> 2016-06-28 slider-0.91.0-incubating
>
> When were the last committers or PMC members elected?
>
> 2015-07-07: Yu (Thomas) Liu
>
>
Re: First draft of Apache Slider November 2016 report
Posted by Josh Elser <jo...@gmail.com>.
I don't have much to add here. I've been painfully absent from Slider,
and this will likely continue from me.
If you have the time/bandwidth, please feel empowered to try to help
push such a release, Manoj :)
On Wed, Nov 2, 2016 at 5:14 PM, Gour Saha <gs...@hortonworks.com> wrote:
> Not entirely sure. Given that we have big ones like SLIDER-1107 and
> SLIDER-875, we might as well release it as 1.0.0. Any reason for
> preferring one over the other?
>
> Would be good to get suggestions from other Slider PPMC members.
>
> -Gour
>
> On 11/2/16, 1:54 PM, "Manoj Samel" <ma...@gmail.com> wrote:
>
>>Hi Gour,
>>
>>Thanks for your reply. Given that 0.91 is out, will the next release be
>>1.0.0?
>>
>>
>>
>>On Wed, Nov 2, 2016 at 10:42 AM, Gour Saha <gs...@hortonworks.com> wrote:
>>
>>> Hi Manoj,
>>> It will be very difficult to provide any concrete timelines on the
>>>switch.
>>> At best, it will be incorrect.
>>>
>>> However, for the user-community of classic Slider, we will continue to
>>> support and cut releases (although the frequency will be reduced as you
>>> have already started to see). As mentioned in the report below - "Slider
>>> community plans to ship a release in the next quarter with all these
>>> features and bug fixes.²
>>>
>>> -Gour
>>>
>>> On 11/2/16, 10:29 AM, "Manoj Samel" <ma...@gmail.com> wrote:
>>>
>>> >Hi Gour,
>>> >
>>> >As user of slider (in production), it would be great to have some idea
>>>on
>>> >timelines so to plan to switch from "classic slider" to hadoop branch.
>>> >Given that hadoop version of branch itself will be not be a stable /
>>> >battle tested and that "slider classic" development is slowing down
>>>(for
>>> >right reasons) to move to yarn; the transition period is challenge for
>>> >slider users.
>>> >
>>> >Not sure if this topic belongs to the apache report but wanted to
>>>raise as
>>> >is important for user community
>>> >
>>> >Thanks,
>>> >
>>> >Manoj
>>> >
>>> >On Wed, Nov 2, 2016 at 9:49 AM, Gour Saha <gs...@hortonworks.com>
>>>wrote:
>>> >
>>> >> Josh, Billie,
>>> >> Thank you for reviewing the draft.
>>> >>
>>> >> Billie, I will make the change as you suggested. Should I also
>>> >>explicitly
>>> >> mention that the funtests have not been moved yet and needs to be
>>> >> converted to Java before we do so?
>>> >>
>>> >> -Gour
>>> >>
>>> >> On 11/2/16, 8:22 AM, "Billie Rinaldi" <bi...@gmail.com>
>>>wrote:
>>> >>
>>> >> >Thanks for the draft, Gour. This looks good. The only part I would
>>> >>change
>>> >> >is this: "As a result, portions like Slider Core, Application
>>>Master,
>>> >> >Client and Tests had been rolled into Apache Hadoop YARN as
>>> >>module(s)." It
>>> >> >isn't clear whether it includes the funtests or not, and all the
>>>slider
>>> >> >code is in a single module in YARN. I'd suggest "As a result, the
>>> >>Slider
>>> >> >Core module (including Application Master, Client and Java unit
>>>tests)
>>> >>has
>>> >> >been rolled into an Apache Hadoop YARN feature branch as a
>>> >> >hadoop-yarn-slider-core module."
>>> >> >
>>> >> >On Tue, Nov 1, 2016 at 6:44 PM, Gour Saha <gs...@hortonworks.com>
>>> >>wrote:
>>> >> >
>>> >> >> Please review and provide feedback on the draft below.
>>> >> >>
>>> >> >> =========================================================
>>> >> >> Slider
>>> >> >>
>>> >> >> Slider is a collection of tools and technologies to package,
>>>deploy,
>>> >>and
>>> >> >> manage long running applications on Apache Hadoop YARN clusters.
>>> >> >>
>>> >> >> Slider has been incubating since 2014-04-29.
>>> >> >>
>>> >> >> Three most important issues to address in the move towards
>>> >>graduation:
>>> >> >>
>>> >> >> 1. Apache Slider community/PPMC has voted to move portions of
>>>Slider
>>> >> >>into
>>> >> >> Apache Hadoop YARN as modules. It is possible that the
>>>remaining
>>> >> >>pieces
>>> >> >> will be moved at a later point in time or become obsolete or
>>> >>evolve
>>> >> >>to
>>> >> >> work closely with YARN. Slider PPMC will decide what makes the
>>> >>most
>>> >> >> sense as we progress through this exciting time.
>>> >> >> 2. Getting more external users
>>> >> >> 3. Growth of a diverse set of developers/committers/PMC members
>>>is
>>> >>also
>>> >> >> crucial towards the final state of Slider
>>> >> >>
>>> >> >> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need
>>>to be
>>> >> >>aware
>>> >> >> of?
>>> >> >>
>>> >> >> No.
>>> >> >>
>>> >> >> How has the community developed since the last report?
>>> >> >>
>>> >> >> The Apache Hadoop YARN community/PMC and Apache Slider
>>>community/PPMC
>>> >> >> had decided to move portions of Slider into YARN, to make fast and
>>> >> >> significant
>>> >> >> progress on: YARN-4692 - [Umbrella] Simplified and first-class
>>> >>support
>>> >> >>for
>>> >> >> services in YARN. As a result, portions like Slider Core,
>>>Application
>>> >> >> Master,
>>> >> >> Client and Tests had been rolled into Apache Hadoop YARN as
>>> >>module(s).
>>> >> >> This effort is being captured in: YARN-5079 : Native YARN
>>>framework
>>> >> >>layer
>>> >> >> for services. All the migrations are happening in a branch (namely
>>> >> >> yarn-native-services) created of off Hadoop trunk. As a result of
>>> >>this,
>>> >> >> we are already seeing interest in the Hadoop committers/PMC, who
>>>have
>>> >> >> started to contribute and submit patches to Slider.
>>> >> >>
>>> >> >> In order to support existing users of Slider, and to provide
>>>seamless
>>> >> >> migration, there will be sufficient overlap between the time when
>>>a
>>> >> >>stable
>>> >> >> state of long running services support is available in some future
>>> >> >>version
>>> >> >> of YARN and the time till an independent Slider release is
>>>available.
>>> >> >>The
>>> >> >> community/PPMC will also determine the future state of Slider as
>>>we
>>> >> >> navigate through these changes.
>>> >> >>
>>> >> >> The discussions on the Slider and YARN community DLs can be viewed
>>> >>here
>>> >> >>-
>>> >> >>
>>> >> >> https://s.apache.org/0hoh
>>> >> >> https://s.apache.org/MncV
>>> >> >>
>>> >> >> For the benefit of those who would like to use Slider Core from
>>>the
>>> >> >> Hadoop codebase, but continue to use classic Slider Agent and
>>>legacy
>>> >> >> app-packages, a new branch (yarn-native-services) has been created
>>> >> >> in classic Slider repo. This branch retains only the Agent
>>>(python)
>>> >>code
>>> >> >> and the App Packages. It adds Slider Core module as a Hadoop
>>> >> >> dependency. This allows users to consume the latest of Slider
>>>Core.
>>> >> >> Future classic Slider releases can be made purely off of this
>>>branch.
>>> >> >>
>>> >> >> Additionally an effort is being made to create an agent-less
>>>provider
>>> >> >>for
>>> >> >> legacy app-packages, which could help users to completely migrate
>>>to
>>> >> >> Hadoop codebase for creating long-running services with Slider.
>>> >> >>Expectation
>>> >> >> from app-package owners would be to make minor modification of
>>>their
>>> >> >> packages. Primary changes would be to shed the dependency on
>>>python
>>> >> >> Agent code, effectively making packages much simpler than what
>>>they
>>> >> >> look like today in classic Slider.
>>> >> >>
>>> >> >> How has the project developed since the last report?
>>> >> >>
>>> >> >> There has been fewer bug fixes on the classic Slider side as most
>>>of
>>> >> >> the focus has been on developing features on native YARN services
>>>in
>>> >> >> the Hadoop codebase. Work continues on support for complex
>>>services
>>> >> >> (assemblies) and agent-less applications in Slider. The efforts on
>>> >> >>Slider
>>> >> >> Core will continue in the new services branch created in YARN. Few
>>> >> >> key features which were merged to classic Slider as well were -
>>> >> >> SLIDER-875 - Ability to create an Uber application package with
>>> >> >> capability to deploy and manage as a single business app, and
>>> >> >> SLIDER-1107 - Generate app configuration files in AM. Several
>>>issues
>>> >> >> identified by Coverity scans were resolved as well. Additionally
>>>few
>>> >> >> patches were contributed by the community fixing functional and
>>> >> >> performance issues. Slider community plans to ship a release in
>>>the
>>> >> >> next quarter with all these features and bug fixes.
>>> >> >>
>>> >> >> Date of last release:
>>> >> >>
>>> >> >> 2016-06-28 slider-0.91.0-incubating
>>> >> >>
>>> >> >> When were the last committers or PMC members elected?
>>> >> >>
>>> >> >> 2015-07-07: Yu (Thomas) Liu
>>> >> >>
>>> >> >>
>>> >>
>>> >>
>>>
>>>
>
Re: First draft of Apache Slider November 2016 report
Posted by Gour Saha <gs...@hortonworks.com>.
Not entirely sure. Given that we have big ones like SLIDER-1107 and
SLIDER-875, we might as well release it as 1.0.0. Any reason for
preferring one over the other?
Would be good to get suggestions from other Slider PPMC members.
-Gour
On 11/2/16, 1:54 PM, "Manoj Samel" <ma...@gmail.com> wrote:
>Hi Gour,
>
>Thanks for your reply. Given that 0.91 is out, will the next release be
>1.0.0?
>
>
>
>On Wed, Nov 2, 2016 at 10:42 AM, Gour Saha <gs...@hortonworks.com> wrote:
>
>> Hi Manoj,
>> It will be very difficult to provide any concrete timelines on the
>>switch.
>> At best, it will be incorrect.
>>
>> However, for the user-community of classic Slider, we will continue to
>> support and cut releases (although the frequency will be reduced as you
>> have already started to see). As mentioned in the report below - "Slider
>> community plans to ship a release in the next quarter with all these
>> features and bug fixes.²
>>
>> -Gour
>>
>> On 11/2/16, 10:29 AM, "Manoj Samel" <ma...@gmail.com> wrote:
>>
>> >Hi Gour,
>> >
>> >As user of slider (in production), it would be great to have some idea
>>on
>> >timelines so to plan to switch from "classic slider" to hadoop branch.
>> >Given that hadoop version of branch itself will be not be a stable /
>> >battle tested and that "slider classic" development is slowing down
>>(for
>> >right reasons) to move to yarn; the transition period is challenge for
>> >slider users.
>> >
>> >Not sure if this topic belongs to the apache report but wanted to
>>raise as
>> >is important for user community
>> >
>> >Thanks,
>> >
>> >Manoj
>> >
>> >On Wed, Nov 2, 2016 at 9:49 AM, Gour Saha <gs...@hortonworks.com>
>>wrote:
>> >
>> >> Josh, Billie,
>> >> Thank you for reviewing the draft.
>> >>
>> >> Billie, I will make the change as you suggested. Should I also
>> >>explicitly
>> >> mention that the funtests have not been moved yet and needs to be
>> >> converted to Java before we do so?
>> >>
>> >> -Gour
>> >>
>> >> On 11/2/16, 8:22 AM, "Billie Rinaldi" <bi...@gmail.com>
>>wrote:
>> >>
>> >> >Thanks for the draft, Gour. This looks good. The only part I would
>> >>change
>> >> >is this: "As a result, portions like Slider Core, Application
>>Master,
>> >> >Client and Tests had been rolled into Apache Hadoop YARN as
>> >>module(s)." It
>> >> >isn't clear whether it includes the funtests or not, and all the
>>slider
>> >> >code is in a single module in YARN. I'd suggest "As a result, the
>> >>Slider
>> >> >Core module (including Application Master, Client and Java unit
>>tests)
>> >>has
>> >> >been rolled into an Apache Hadoop YARN feature branch as a
>> >> >hadoop-yarn-slider-core module."
>> >> >
>> >> >On Tue, Nov 1, 2016 at 6:44 PM, Gour Saha <gs...@hortonworks.com>
>> >>wrote:
>> >> >
>> >> >> Please review and provide feedback on the draft below.
>> >> >>
>> >> >> =========================================================
>> >> >> Slider
>> >> >>
>> >> >> Slider is a collection of tools and technologies to package,
>>deploy,
>> >>and
>> >> >> manage long running applications on Apache Hadoop YARN clusters.
>> >> >>
>> >> >> Slider has been incubating since 2014-04-29.
>> >> >>
>> >> >> Three most important issues to address in the move towards
>> >>graduation:
>> >> >>
>> >> >> 1. Apache Slider community/PPMC has voted to move portions of
>>Slider
>> >> >>into
>> >> >> Apache Hadoop YARN as modules. It is possible that the
>>remaining
>> >> >>pieces
>> >> >> will be moved at a later point in time or become obsolete or
>> >>evolve
>> >> >>to
>> >> >> work closely with YARN. Slider PPMC will decide what makes the
>> >>most
>> >> >> sense as we progress through this exciting time.
>> >> >> 2. Getting more external users
>> >> >> 3. Growth of a diverse set of developers/committers/PMC members
>>is
>> >>also
>> >> >> crucial towards the final state of Slider
>> >> >>
>> >> >> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need
>>to be
>> >> >>aware
>> >> >> of?
>> >> >>
>> >> >> No.
>> >> >>
>> >> >> How has the community developed since the last report?
>> >> >>
>> >> >> The Apache Hadoop YARN community/PMC and Apache Slider
>>community/PPMC
>> >> >> had decided to move portions of Slider into YARN, to make fast and
>> >> >> significant
>> >> >> progress on: YARN-4692 - [Umbrella] Simplified and first-class
>> >>support
>> >> >>for
>> >> >> services in YARN. As a result, portions like Slider Core,
>>Application
>> >> >> Master,
>> >> >> Client and Tests had been rolled into Apache Hadoop YARN as
>> >>module(s).
>> >> >> This effort is being captured in: YARN-5079 : Native YARN
>>framework
>> >> >>layer
>> >> >> for services. All the migrations are happening in a branch (namely
>> >> >> yarn-native-services) created of off Hadoop trunk. As a result of
>> >>this,
>> >> >> we are already seeing interest in the Hadoop committers/PMC, who
>>have
>> >> >> started to contribute and submit patches to Slider.
>> >> >>
>> >> >> In order to support existing users of Slider, and to provide
>>seamless
>> >> >> migration, there will be sufficient overlap between the time when
>>a
>> >> >>stable
>> >> >> state of long running services support is available in some future
>> >> >>version
>> >> >> of YARN and the time till an independent Slider release is
>>available.
>> >> >>The
>> >> >> community/PPMC will also determine the future state of Slider as
>>we
>> >> >> navigate through these changes.
>> >> >>
>> >> >> The discussions on the Slider and YARN community DLs can be viewed
>> >>here
>> >> >>-
>> >> >>
>> >> >> https://s.apache.org/0hoh
>> >> >> https://s.apache.org/MncV
>> >> >>
>> >> >> For the benefit of those who would like to use Slider Core from
>>the
>> >> >> Hadoop codebase, but continue to use classic Slider Agent and
>>legacy
>> >> >> app-packages, a new branch (yarn-native-services) has been created
>> >> >> in classic Slider repo. This branch retains only the Agent
>>(python)
>> >>code
>> >> >> and the App Packages. It adds Slider Core module as a Hadoop
>> >> >> dependency. This allows users to consume the latest of Slider
>>Core.
>> >> >> Future classic Slider releases can be made purely off of this
>>branch.
>> >> >>
>> >> >> Additionally an effort is being made to create an agent-less
>>provider
>> >> >>for
>> >> >> legacy app-packages, which could help users to completely migrate
>>to
>> >> >> Hadoop codebase for creating long-running services with Slider.
>> >> >>Expectation
>> >> >> from app-package owners would be to make minor modification of
>>their
>> >> >> packages. Primary changes would be to shed the dependency on
>>python
>> >> >> Agent code, effectively making packages much simpler than what
>>they
>> >> >> look like today in classic Slider.
>> >> >>
>> >> >> How has the project developed since the last report?
>> >> >>
>> >> >> There has been fewer bug fixes on the classic Slider side as most
>>of
>> >> >> the focus has been on developing features on native YARN services
>>in
>> >> >> the Hadoop codebase. Work continues on support for complex
>>services
>> >> >> (assemblies) and agent-less applications in Slider. The efforts on
>> >> >>Slider
>> >> >> Core will continue in the new services branch created in YARN. Few
>> >> >> key features which were merged to classic Slider as well were -
>> >> >> SLIDER-875 - Ability to create an Uber application package with
>> >> >> capability to deploy and manage as a single business app, and
>> >> >> SLIDER-1107 - Generate app configuration files in AM. Several
>>issues
>> >> >> identified by Coverity scans were resolved as well. Additionally
>>few
>> >> >> patches were contributed by the community fixing functional and
>> >> >> performance issues. Slider community plans to ship a release in
>>the
>> >> >> next quarter with all these features and bug fixes.
>> >> >>
>> >> >> Date of last release:
>> >> >>
>> >> >> 2016-06-28 slider-0.91.0-incubating
>> >> >>
>> >> >> When were the last committers or PMC members elected?
>> >> >>
>> >> >> 2015-07-07: Yu (Thomas) Liu
>> >> >>
>> >> >>
>> >>
>> >>
>>
>>
Re: First draft of Apache Slider November 2016 report
Posted by Manoj Samel <ma...@gmail.com>.
Hi Gour,
Thanks for your reply. Given that 0.91 is out, will the next release be
1.0.0?
On Wed, Nov 2, 2016 at 10:42 AM, Gour Saha <gs...@hortonworks.com> wrote:
> Hi Manoj,
> It will be very difficult to provide any concrete timelines on the switch.
> At best, it will be incorrect.
>
> However, for the user-community of classic Slider, we will continue to
> support and cut releases (although the frequency will be reduced as you
> have already started to see). As mentioned in the report below - "Slider
> community plans to ship a release in the next quarter with all these
> features and bug fixes.²
>
> -Gour
>
> On 11/2/16, 10:29 AM, "Manoj Samel" <ma...@gmail.com> wrote:
>
> >Hi Gour,
> >
> >As user of slider (in production), it would be great to have some idea on
> >timelines so to plan to switch from "classic slider" to hadoop branch.
> >Given that hadoop version of branch itself will be not be a stable /
> >battle tested and that "slider classic" development is slowing down (for
> >right reasons) to move to yarn; the transition period is challenge for
> >slider users.
> >
> >Not sure if this topic belongs to the apache report but wanted to raise as
> >is important for user community
> >
> >Thanks,
> >
> >Manoj
> >
> >On Wed, Nov 2, 2016 at 9:49 AM, Gour Saha <gs...@hortonworks.com> wrote:
> >
> >> Josh, Billie,
> >> Thank you for reviewing the draft.
> >>
> >> Billie, I will make the change as you suggested. Should I also
> >>explicitly
> >> mention that the funtests have not been moved yet and needs to be
> >> converted to Java before we do so?
> >>
> >> -Gour
> >>
> >> On 11/2/16, 8:22 AM, "Billie Rinaldi" <bi...@gmail.com> wrote:
> >>
> >> >Thanks for the draft, Gour. This looks good. The only part I would
> >>change
> >> >is this: "As a result, portions like Slider Core, Application Master,
> >> >Client and Tests had been rolled into Apache Hadoop YARN as
> >>module(s)." It
> >> >isn't clear whether it includes the funtests or not, and all the slider
> >> >code is in a single module in YARN. I'd suggest "As a result, the
> >>Slider
> >> >Core module (including Application Master, Client and Java unit tests)
> >>has
> >> >been rolled into an Apache Hadoop YARN feature branch as a
> >> >hadoop-yarn-slider-core module."
> >> >
> >> >On Tue, Nov 1, 2016 at 6:44 PM, Gour Saha <gs...@hortonworks.com>
> >>wrote:
> >> >
> >> >> Please review and provide feedback on the draft below.
> >> >>
> >> >> =========================================================
> >> >> Slider
> >> >>
> >> >> Slider is a collection of tools and technologies to package, deploy,
> >>and
> >> >> manage long running applications on Apache Hadoop YARN clusters.
> >> >>
> >> >> Slider has been incubating since 2014-04-29.
> >> >>
> >> >> Three most important issues to address in the move towards
> >>graduation:
> >> >>
> >> >> 1. Apache Slider community/PPMC has voted to move portions of Slider
> >> >>into
> >> >> Apache Hadoop YARN as modules. It is possible that the remaining
> >> >>pieces
> >> >> will be moved at a later point in time or become obsolete or
> >>evolve
> >> >>to
> >> >> work closely with YARN. Slider PPMC will decide what makes the
> >>most
> >> >> sense as we progress through this exciting time.
> >> >> 2. Getting more external users
> >> >> 3. Growth of a diverse set of developers/committers/PMC members is
> >>also
> >> >> crucial towards the final state of Slider
> >> >>
> >> >> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
> >> >>aware
> >> >> of?
> >> >>
> >> >> No.
> >> >>
> >> >> How has the community developed since the last report?
> >> >>
> >> >> The Apache Hadoop YARN community/PMC and Apache Slider community/PPMC
> >> >> had decided to move portions of Slider into YARN, to make fast and
> >> >> significant
> >> >> progress on: YARN-4692 - [Umbrella] Simplified and first-class
> >>support
> >> >>for
> >> >> services in YARN. As a result, portions like Slider Core, Application
> >> >> Master,
> >> >> Client and Tests had been rolled into Apache Hadoop YARN as
> >>module(s).
> >> >> This effort is being captured in: YARN-5079 : Native YARN framework
> >> >>layer
> >> >> for services. All the migrations are happening in a branch (namely
> >> >> yarn-native-services) created of off Hadoop trunk. As a result of
> >>this,
> >> >> we are already seeing interest in the Hadoop committers/PMC, who have
> >> >> started to contribute and submit patches to Slider.
> >> >>
> >> >> In order to support existing users of Slider, and to provide seamless
> >> >> migration, there will be sufficient overlap between the time when a
> >> >>stable
> >> >> state of long running services support is available in some future
> >> >>version
> >> >> of YARN and the time till an independent Slider release is available.
> >> >>The
> >> >> community/PPMC will also determine the future state of Slider as we
> >> >> navigate through these changes.
> >> >>
> >> >> The discussions on the Slider and YARN community DLs can be viewed
> >>here
> >> >>-
> >> >>
> >> >> https://s.apache.org/0hoh
> >> >> https://s.apache.org/MncV
> >> >>
> >> >> For the benefit of those who would like to use Slider Core from the
> >> >> Hadoop codebase, but continue to use classic Slider Agent and legacy
> >> >> app-packages, a new branch (yarn-native-services) has been created
> >> >> in classic Slider repo. This branch retains only the Agent (python)
> >>code
> >> >> and the App Packages. It adds Slider Core module as a Hadoop
> >> >> dependency. This allows users to consume the latest of Slider Core.
> >> >> Future classic Slider releases can be made purely off of this branch.
> >> >>
> >> >> Additionally an effort is being made to create an agent-less provider
> >> >>for
> >> >> legacy app-packages, which could help users to completely migrate to
> >> >> Hadoop codebase for creating long-running services with Slider.
> >> >>Expectation
> >> >> from app-package owners would be to make minor modification of their
> >> >> packages. Primary changes would be to shed the dependency on python
> >> >> Agent code, effectively making packages much simpler than what they
> >> >> look like today in classic Slider.
> >> >>
> >> >> How has the project developed since the last report?
> >> >>
> >> >> There has been fewer bug fixes on the classic Slider side as most of
> >> >> the focus has been on developing features on native YARN services in
> >> >> the Hadoop codebase. Work continues on support for complex services
> >> >> (assemblies) and agent-less applications in Slider. The efforts on
> >> >>Slider
> >> >> Core will continue in the new services branch created in YARN. Few
> >> >> key features which were merged to classic Slider as well were -
> >> >> SLIDER-875 - Ability to create an Uber application package with
> >> >> capability to deploy and manage as a single business app, and
> >> >> SLIDER-1107 - Generate app configuration files in AM. Several issues
> >> >> identified by Coverity scans were resolved as well. Additionally few
> >> >> patches were contributed by the community fixing functional and
> >> >> performance issues. Slider community plans to ship a release in the
> >> >> next quarter with all these features and bug fixes.
> >> >>
> >> >> Date of last release:
> >> >>
> >> >> 2016-06-28 slider-0.91.0-incubating
> >> >>
> >> >> When were the last committers or PMC members elected?
> >> >>
> >> >> 2015-07-07: Yu (Thomas) Liu
> >> >>
> >> >>
> >>
> >>
>
>
Re: First draft of Apache Slider November 2016 report
Posted by Gour Saha <gs...@hortonworks.com>.
Hi Manoj,
It will be very difficult to provide any concrete timelines on the switch.
At best, it will be incorrect.
However, for the user-community of classic Slider, we will continue to
support and cut releases (although the frequency will be reduced as you
have already started to see). As mentioned in the report below - "Slider
community plans to ship a release in the next quarter with all these
features and bug fixes.²
-Gour
On 11/2/16, 10:29 AM, "Manoj Samel" <ma...@gmail.com> wrote:
>Hi Gour,
>
>As user of slider (in production), it would be great to have some idea on
>timelines so to plan to switch from "classic slider" to hadoop branch.
>Given that hadoop version of branch itself will be not be a stable /
>battle tested and that "slider classic" development is slowing down (for
>right reasons) to move to yarn; the transition period is challenge for
>slider users.
>
>Not sure if this topic belongs to the apache report but wanted to raise as
>is important for user community
>
>Thanks,
>
>Manoj
>
>On Wed, Nov 2, 2016 at 9:49 AM, Gour Saha <gs...@hortonworks.com> wrote:
>
>> Josh, Billie,
>> Thank you for reviewing the draft.
>>
>> Billie, I will make the change as you suggested. Should I also
>>explicitly
>> mention that the funtests have not been moved yet and needs to be
>> converted to Java before we do so?
>>
>> -Gour
>>
>> On 11/2/16, 8:22 AM, "Billie Rinaldi" <bi...@gmail.com> wrote:
>>
>> >Thanks for the draft, Gour. This looks good. The only part I would
>>change
>> >is this: "As a result, portions like Slider Core, Application Master,
>> >Client and Tests had been rolled into Apache Hadoop YARN as
>>module(s)." It
>> >isn't clear whether it includes the funtests or not, and all the slider
>> >code is in a single module in YARN. I'd suggest "As a result, the
>>Slider
>> >Core module (including Application Master, Client and Java unit tests)
>>has
>> >been rolled into an Apache Hadoop YARN feature branch as a
>> >hadoop-yarn-slider-core module."
>> >
>> >On Tue, Nov 1, 2016 at 6:44 PM, Gour Saha <gs...@hortonworks.com>
>>wrote:
>> >
>> >> Please review and provide feedback on the draft below.
>> >>
>> >> =========================================================
>> >> Slider
>> >>
>> >> Slider is a collection of tools and technologies to package, deploy,
>>and
>> >> manage long running applications on Apache Hadoop YARN clusters.
>> >>
>> >> Slider has been incubating since 2014-04-29.
>> >>
>> >> Three most important issues to address in the move towards
>>graduation:
>> >>
>> >> 1. Apache Slider community/PPMC has voted to move portions of Slider
>> >>into
>> >> Apache Hadoop YARN as modules. It is possible that the remaining
>> >>pieces
>> >> will be moved at a later point in time or become obsolete or
>>evolve
>> >>to
>> >> work closely with YARN. Slider PPMC will decide what makes the
>>most
>> >> sense as we progress through this exciting time.
>> >> 2. Getting more external users
>> >> 3. Growth of a diverse set of developers/committers/PMC members is
>>also
>> >> crucial towards the final state of Slider
>> >>
>> >> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
>> >>aware
>> >> of?
>> >>
>> >> No.
>> >>
>> >> How has the community developed since the last report?
>> >>
>> >> The Apache Hadoop YARN community/PMC and Apache Slider community/PPMC
>> >> had decided to move portions of Slider into YARN, to make fast and
>> >> significant
>> >> progress on: YARN-4692 - [Umbrella] Simplified and first-class
>>support
>> >>for
>> >> services in YARN. As a result, portions like Slider Core, Application
>> >> Master,
>> >> Client and Tests had been rolled into Apache Hadoop YARN as
>>module(s).
>> >> This effort is being captured in: YARN-5079 : Native YARN framework
>> >>layer
>> >> for services. All the migrations are happening in a branch (namely
>> >> yarn-native-services) created of off Hadoop trunk. As a result of
>>this,
>> >> we are already seeing interest in the Hadoop committers/PMC, who have
>> >> started to contribute and submit patches to Slider.
>> >>
>> >> In order to support existing users of Slider, and to provide seamless
>> >> migration, there will be sufficient overlap between the time when a
>> >>stable
>> >> state of long running services support is available in some future
>> >>version
>> >> of YARN and the time till an independent Slider release is available.
>> >>The
>> >> community/PPMC will also determine the future state of Slider as we
>> >> navigate through these changes.
>> >>
>> >> The discussions on the Slider and YARN community DLs can be viewed
>>here
>> >>-
>> >>
>> >> https://s.apache.org/0hoh
>> >> https://s.apache.org/MncV
>> >>
>> >> For the benefit of those who would like to use Slider Core from the
>> >> Hadoop codebase, but continue to use classic Slider Agent and legacy
>> >> app-packages, a new branch (yarn-native-services) has been created
>> >> in classic Slider repo. This branch retains only the Agent (python)
>>code
>> >> and the App Packages. It adds Slider Core module as a Hadoop
>> >> dependency. This allows users to consume the latest of Slider Core.
>> >> Future classic Slider releases can be made purely off of this branch.
>> >>
>> >> Additionally an effort is being made to create an agent-less provider
>> >>for
>> >> legacy app-packages, which could help users to completely migrate to
>> >> Hadoop codebase for creating long-running services with Slider.
>> >>Expectation
>> >> from app-package owners would be to make minor modification of their
>> >> packages. Primary changes would be to shed the dependency on python
>> >> Agent code, effectively making packages much simpler than what they
>> >> look like today in classic Slider.
>> >>
>> >> How has the project developed since the last report?
>> >>
>> >> There has been fewer bug fixes on the classic Slider side as most of
>> >> the focus has been on developing features on native YARN services in
>> >> the Hadoop codebase. Work continues on support for complex services
>> >> (assemblies) and agent-less applications in Slider. The efforts on
>> >>Slider
>> >> Core will continue in the new services branch created in YARN. Few
>> >> key features which were merged to classic Slider as well were -
>> >> SLIDER-875 - Ability to create an Uber application package with
>> >> capability to deploy and manage as a single business app, and
>> >> SLIDER-1107 - Generate app configuration files in AM. Several issues
>> >> identified by Coverity scans were resolved as well. Additionally few
>> >> patches were contributed by the community fixing functional and
>> >> performance issues. Slider community plans to ship a release in the
>> >> next quarter with all these features and bug fixes.
>> >>
>> >> Date of last release:
>> >>
>> >> 2016-06-28 slider-0.91.0-incubating
>> >>
>> >> When were the last committers or PMC members elected?
>> >>
>> >> 2015-07-07: Yu (Thomas) Liu
>> >>
>> >>
>>
>>
Re: First draft of Apache Slider November 2016 report
Posted by Manoj Samel <ma...@gmail.com>.
Hi Gour,
As user of slider (in production), it would be great to have some idea on
timelines so to plan to switch from "classic slider" to hadoop branch.
Given that hadoop version of branch itself will be not be a stable /
battle tested and that "slider classic" development is slowing down (for
right reasons) to move to yarn; the transition period is challenge for
slider users.
Not sure if this topic belongs to the apache report but wanted to raise as
is important for user community
Thanks,
Manoj
On Wed, Nov 2, 2016 at 9:49 AM, Gour Saha <gs...@hortonworks.com> wrote:
> Josh, Billie,
> Thank you for reviewing the draft.
>
> Billie, I will make the change as you suggested. Should I also explicitly
> mention that the funtests have not been moved yet and needs to be
> converted to Java before we do so?
>
> -Gour
>
> On 11/2/16, 8:22 AM, "Billie Rinaldi" <bi...@gmail.com> wrote:
>
> >Thanks for the draft, Gour. This looks good. The only part I would change
> >is this: "As a result, portions like Slider Core, Application Master,
> >Client and Tests had been rolled into Apache Hadoop YARN as module(s)." It
> >isn't clear whether it includes the funtests or not, and all the slider
> >code is in a single module in YARN. I'd suggest "As a result, the Slider
> >Core module (including Application Master, Client and Java unit tests) has
> >been rolled into an Apache Hadoop YARN feature branch as a
> >hadoop-yarn-slider-core module."
> >
> >On Tue, Nov 1, 2016 at 6:44 PM, Gour Saha <gs...@hortonworks.com> wrote:
> >
> >> Please review and provide feedback on the draft below.
> >>
> >> =========================================================
> >> Slider
> >>
> >> Slider is a collection of tools and technologies to package, deploy, and
> >> manage long running applications on Apache Hadoop YARN clusters.
> >>
> >> Slider has been incubating since 2014-04-29.
> >>
> >> Three most important issues to address in the move towards graduation:
> >>
> >> 1. Apache Slider community/PPMC has voted to move portions of Slider
> >>into
> >> Apache Hadoop YARN as modules. It is possible that the remaining
> >>pieces
> >> will be moved at a later point in time or become obsolete or evolve
> >>to
> >> work closely with YARN. Slider PPMC will decide what makes the most
> >> sense as we progress through this exciting time.
> >> 2. Getting more external users
> >> 3. Growth of a diverse set of developers/committers/PMC members is also
> >> crucial towards the final state of Slider
> >>
> >> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
> >>aware
> >> of?
> >>
> >> No.
> >>
> >> How has the community developed since the last report?
> >>
> >> The Apache Hadoop YARN community/PMC and Apache Slider community/PPMC
> >> had decided to move portions of Slider into YARN, to make fast and
> >> significant
> >> progress on: YARN-4692 - [Umbrella] Simplified and first-class support
> >>for
> >> services in YARN. As a result, portions like Slider Core, Application
> >> Master,
> >> Client and Tests had been rolled into Apache Hadoop YARN as module(s).
> >> This effort is being captured in: YARN-5079 : Native YARN framework
> >>layer
> >> for services. All the migrations are happening in a branch (namely
> >> yarn-native-services) created of off Hadoop trunk. As a result of this,
> >> we are already seeing interest in the Hadoop committers/PMC, who have
> >> started to contribute and submit patches to Slider.
> >>
> >> In order to support existing users of Slider, and to provide seamless
> >> migration, there will be sufficient overlap between the time when a
> >>stable
> >> state of long running services support is available in some future
> >>version
> >> of YARN and the time till an independent Slider release is available.
> >>The
> >> community/PPMC will also determine the future state of Slider as we
> >> navigate through these changes.
> >>
> >> The discussions on the Slider and YARN community DLs can be viewed here
> >>-
> >>
> >> https://s.apache.org/0hoh
> >> https://s.apache.org/MncV
> >>
> >> For the benefit of those who would like to use Slider Core from the
> >> Hadoop codebase, but continue to use classic Slider Agent and legacy
> >> app-packages, a new branch (yarn-native-services) has been created
> >> in classic Slider repo. This branch retains only the Agent (python) code
> >> and the App Packages. It adds Slider Core module as a Hadoop
> >> dependency. This allows users to consume the latest of Slider Core.
> >> Future classic Slider releases can be made purely off of this branch.
> >>
> >> Additionally an effort is being made to create an agent-less provider
> >>for
> >> legacy app-packages, which could help users to completely migrate to
> >> Hadoop codebase for creating long-running services with Slider.
> >>Expectation
> >> from app-package owners would be to make minor modification of their
> >> packages. Primary changes would be to shed the dependency on python
> >> Agent code, effectively making packages much simpler than what they
> >> look like today in classic Slider.
> >>
> >> How has the project developed since the last report?
> >>
> >> There has been fewer bug fixes on the classic Slider side as most of
> >> the focus has been on developing features on native YARN services in
> >> the Hadoop codebase. Work continues on support for complex services
> >> (assemblies) and agent-less applications in Slider. The efforts on
> >>Slider
> >> Core will continue in the new services branch created in YARN. Few
> >> key features which were merged to classic Slider as well were -
> >> SLIDER-875 - Ability to create an Uber application package with
> >> capability to deploy and manage as a single business app, and
> >> SLIDER-1107 - Generate app configuration files in AM. Several issues
> >> identified by Coverity scans were resolved as well. Additionally few
> >> patches were contributed by the community fixing functional and
> >> performance issues. Slider community plans to ship a release in the
> >> next quarter with all these features and bug fixes.
> >>
> >> Date of last release:
> >>
> >> 2016-06-28 slider-0.91.0-incubating
> >>
> >> When were the last committers or PMC members elected?
> >>
> >> 2015-07-07: Yu (Thomas) Liu
> >>
> >>
>
>
Re: First draft of Apache Slider November 2016 report
Posted by Gour Saha <gs...@hortonworks.com>.
Josh, Billie,
Thank you for reviewing the draft.
Billie, I will make the change as you suggested. Should I also explicitly
mention that the funtests have not been moved yet and needs to be
converted to Java before we do so?
-Gour
On 11/2/16, 8:22 AM, "Billie Rinaldi" <bi...@gmail.com> wrote:
>Thanks for the draft, Gour. This looks good. The only part I would change
>is this: "As a result, portions like Slider Core, Application Master,
>Client and Tests had been rolled into Apache Hadoop YARN as module(s)." It
>isn't clear whether it includes the funtests or not, and all the slider
>code is in a single module in YARN. I'd suggest "As a result, the Slider
>Core module (including Application Master, Client and Java unit tests) has
>been rolled into an Apache Hadoop YARN feature branch as a
>hadoop-yarn-slider-core module."
>
>On Tue, Nov 1, 2016 at 6:44 PM, Gour Saha <gs...@hortonworks.com> wrote:
>
>> Please review and provide feedback on the draft below.
>>
>> =========================================================
>> Slider
>>
>> Slider is a collection of tools and technologies to package, deploy, and
>> manage long running applications on Apache Hadoop YARN clusters.
>>
>> Slider has been incubating since 2014-04-29.
>>
>> Three most important issues to address in the move towards graduation:
>>
>> 1. Apache Slider community/PPMC has voted to move portions of Slider
>>into
>> Apache Hadoop YARN as modules. It is possible that the remaining
>>pieces
>> will be moved at a later point in time or become obsolete or evolve
>>to
>> work closely with YARN. Slider PPMC will decide what makes the most
>> sense as we progress through this exciting time.
>> 2. Getting more external users
>> 3. Growth of a diverse set of developers/committers/PMC members is also
>> crucial towards the final state of Slider
>>
>> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
>>aware
>> of?
>>
>> No.
>>
>> How has the community developed since the last report?
>>
>> The Apache Hadoop YARN community/PMC and Apache Slider community/PPMC
>> had decided to move portions of Slider into YARN, to make fast and
>> significant
>> progress on: YARN-4692 - [Umbrella] Simplified and first-class support
>>for
>> services in YARN. As a result, portions like Slider Core, Application
>> Master,
>> Client and Tests had been rolled into Apache Hadoop YARN as module(s).
>> This effort is being captured in: YARN-5079 : Native YARN framework
>>layer
>> for services. All the migrations are happening in a branch (namely
>> yarn-native-services) created of off Hadoop trunk. As a result of this,
>> we are already seeing interest in the Hadoop committers/PMC, who have
>> started to contribute and submit patches to Slider.
>>
>> In order to support existing users of Slider, and to provide seamless
>> migration, there will be sufficient overlap between the time when a
>>stable
>> state of long running services support is available in some future
>>version
>> of YARN and the time till an independent Slider release is available.
>>The
>> community/PPMC will also determine the future state of Slider as we
>> navigate through these changes.
>>
>> The discussions on the Slider and YARN community DLs can be viewed here
>>-
>>
>> https://s.apache.org/0hoh
>> https://s.apache.org/MncV
>>
>> For the benefit of those who would like to use Slider Core from the
>> Hadoop codebase, but continue to use classic Slider Agent and legacy
>> app-packages, a new branch (yarn-native-services) has been created
>> in classic Slider repo. This branch retains only the Agent (python) code
>> and the App Packages. It adds Slider Core module as a Hadoop
>> dependency. This allows users to consume the latest of Slider Core.
>> Future classic Slider releases can be made purely off of this branch.
>>
>> Additionally an effort is being made to create an agent-less provider
>>for
>> legacy app-packages, which could help users to completely migrate to
>> Hadoop codebase for creating long-running services with Slider.
>>Expectation
>> from app-package owners would be to make minor modification of their
>> packages. Primary changes would be to shed the dependency on python
>> Agent code, effectively making packages much simpler than what they
>> look like today in classic Slider.
>>
>> How has the project developed since the last report?
>>
>> There has been fewer bug fixes on the classic Slider side as most of
>> the focus has been on developing features on native YARN services in
>> the Hadoop codebase. Work continues on support for complex services
>> (assemblies) and agent-less applications in Slider. The efforts on
>>Slider
>> Core will continue in the new services branch created in YARN. Few
>> key features which were merged to classic Slider as well were -
>> SLIDER-875 - Ability to create an Uber application package with
>> capability to deploy and manage as a single business app, and
>> SLIDER-1107 - Generate app configuration files in AM. Several issues
>> identified by Coverity scans were resolved as well. Additionally few
>> patches were contributed by the community fixing functional and
>> performance issues. Slider community plans to ship a release in the
>> next quarter with all these features and bug fixes.
>>
>> Date of last release:
>>
>> 2016-06-28 slider-0.91.0-incubating
>>
>> When were the last committers or PMC members elected?
>>
>> 2015-07-07: Yu (Thomas) Liu
>>
>>
Re: First draft of Apache Slider November 2016 report
Posted by Billie Rinaldi <bi...@gmail.com>.
Thanks for the draft, Gour. This looks good. The only part I would change
is this: "As a result, portions like Slider Core, Application Master,
Client and Tests had been rolled into Apache Hadoop YARN as module(s)." It
isn't clear whether it includes the funtests or not, and all the slider
code is in a single module in YARN. I'd suggest "As a result, the Slider
Core module (including Application Master, Client and Java unit tests) has
been rolled into an Apache Hadoop YARN feature branch as a
hadoop-yarn-slider-core module."
On Tue, Nov 1, 2016 at 6:44 PM, Gour Saha <gs...@hortonworks.com> wrote:
> Please review and provide feedback on the draft below.
>
> =========================================================
> Slider
>
> Slider is a collection of tools and technologies to package, deploy, and
> manage long running applications on Apache Hadoop YARN clusters.
>
> Slider has been incubating since 2014-04-29.
>
> Three most important issues to address in the move towards graduation:
>
> 1. Apache Slider community/PPMC has voted to move portions of Slider into
> Apache Hadoop YARN as modules. It is possible that the remaining pieces
> will be moved at a later point in time or become obsolete or evolve to
> work closely with YARN. Slider PPMC will decide what makes the most
> sense as we progress through this exciting time.
> 2. Getting more external users
> 3. Growth of a diverse set of developers/committers/PMC members is also
> crucial towards the final state of Slider
>
> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be aware
> of?
>
> No.
>
> How has the community developed since the last report?
>
> The Apache Hadoop YARN community/PMC and Apache Slider community/PPMC
> had decided to move portions of Slider into YARN, to make fast and
> significant
> progress on: YARN-4692 - [Umbrella] Simplified and first-class support for
> services in YARN. As a result, portions like Slider Core, Application
> Master,
> Client and Tests had been rolled into Apache Hadoop YARN as module(s).
> This effort is being captured in: YARN-5079 : Native YARN framework layer
> for services. All the migrations are happening in a branch (namely
> yarn-native-services) created of off Hadoop trunk. As a result of this,
> we are already seeing interest in the Hadoop committers/PMC, who have
> started to contribute and submit patches to Slider.
>
> In order to support existing users of Slider, and to provide seamless
> migration, there will be sufficient overlap between the time when a stable
> state of long running services support is available in some future version
> of YARN and the time till an independent Slider release is available. The
> community/PPMC will also determine the future state of Slider as we
> navigate through these changes.
>
> The discussions on the Slider and YARN community DLs can be viewed here -
>
> https://s.apache.org/0hoh
> https://s.apache.org/MncV
>
> For the benefit of those who would like to use Slider Core from the
> Hadoop codebase, but continue to use classic Slider Agent and legacy
> app-packages, a new branch (yarn-native-services) has been created
> in classic Slider repo. This branch retains only the Agent (python) code
> and the App Packages. It adds Slider Core module as a Hadoop
> dependency. This allows users to consume the latest of Slider Core.
> Future classic Slider releases can be made purely off of this branch.
>
> Additionally an effort is being made to create an agent-less provider for
> legacy app-packages, which could help users to completely migrate to
> Hadoop codebase for creating long-running services with Slider. Expectation
> from app-package owners would be to make minor modification of their
> packages. Primary changes would be to shed the dependency on python
> Agent code, effectively making packages much simpler than what they
> look like today in classic Slider.
>
> How has the project developed since the last report?
>
> There has been fewer bug fixes on the classic Slider side as most of
> the focus has been on developing features on native YARN services in
> the Hadoop codebase. Work continues on support for complex services
> (assemblies) and agent-less applications in Slider. The efforts on Slider
> Core will continue in the new services branch created in YARN. Few
> key features which were merged to classic Slider as well were -
> SLIDER-875 - Ability to create an Uber application package with
> capability to deploy and manage as a single business app, and
> SLIDER-1107 - Generate app configuration files in AM. Several issues
> identified by Coverity scans were resolved as well. Additionally few
> patches were contributed by the community fixing functional and
> performance issues. Slider community plans to ship a release in the
> next quarter with all these features and bug fixes.
>
> Date of last release:
>
> 2016-06-28 slider-0.91.0-incubating
>
> When were the last committers or PMC members elected?
>
> 2015-07-07: Yu (Thomas) Liu
>
>