You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Evans Ye <ev...@apache.org> on 2020/02/03 17:48:05 UTC
Re: [DISCUSS] Target distros on the 1.5.0 release
Hi Kengo,
Based on what you've done so far and you're already a committer of Bigtop,
would you like to be the Release Manager of Bigtop 1.5.0 release? Might got
things to do as a RM but we as a community can help and work together. I
served as RM for two times and I find it a great experience to know how
apache releases work as a whole.
Evans
Evans Ye <ev...@apache.org> 於 2019年12月22日 週日 上午2:16寫道：
> Elasticsearch has been merged[1]. Thanks to Jun for adding puppet and
> Smoke Test within that PR.
>
> [1] https://github.com/apache/bigtop/pull/566
>
> Kengo Seki <se...@apache.org> 於 2019年12月13日 週五 09:30 寫道：
>
>> Jun and Evans, sorry for my late response.
>>
>> > So if that Jira (BIGTOP-3219) can be done in next week, do you think it
>> is
>> > OK to include Elasticsearch-5.6.14 in v1.5?
>>
>> Of course! I'm looking forward to it.
>>
>> Kengo Seki <se...@apache.org>
>>
>> On Fri, Dec 6, 2019 at 8:08 PM Evans Ye <ev...@apache.org> wrote:
>> >
>> > I'm a +1 for this. I can help on the ci side so that adding one more
>> > package for release is easier.
>> >
>> > Jun HE <ju...@apache.org> 於 2019年12月6日 週五 15:41 寫道：
>> >
>> > > Hi, Kengo,
>> > >
>> > > I just noticed you updated the BOM of Bigtop v1.5 on BIGTOP-3123.
>> > >
>> > > One thing I'd like to know your and other folks' thought on
>> Elasticsearch
>> > > as Bigtop component.
>> > > There is a ticket (https://issues.apache.org/jira/browse/BIGTOP-3219)
>> for
>> > > this. And I've finished most of the sub-works
>> > > (build/packaging/deployement). Patch could be found at:
>> > > https://github.com/apache/bigtop/pull/566
>> > > For the smoke test part I think it could be done in this weekend.
>> > >
>> > > So if that Jira (BIGTOP-3219) can be done in next week, do you think
>> it is
>> > > OK to include Elasticsearch-5.6.14 in v1.5?
>> > >
>> > > Regards,
>> > >
>> > > Jun
>> > >
>> > > Kengo Seki <se...@apache.org> 于2019年11月22日周五 上午12:09写道：
>> > >
>> > > > Thanks for the comments, everyone!
>> > > > If there's still no objection in a few days, I'm going to update
>> > > > BIGTOP-3123's description.
>> > > >
>> > > > > 'contrib' module will be easier for us to maintain traditional
>> distros
>> > > > and cnb.
>> > > >
>> > > > Agreed. I think that merging the cnb branch later will be a hard
>> work
>> > > too.
>> > > >
>> > > > > What do you think about the components?
>> > > > > Is there a list of components you'd like to upgrade?
>> > > >
>> > > > I'd like to upgrade the following components:
>> > > >
>> > > > - Zookeeper: 3.4.13
>> > > > - Hadoop: 3.2.1 (or 2.10.0 if packaging Hadoop 3 is too hard, as
>> Olaf
>> > > > mentioned before)
>> > > > - HBase: 2.2.2
>> > > > - Hive: 3.1.2
>> > > > - Tez: 0.9.2
>> > > > - Spark: 2.4.4 (or 3.0.0, if GA is released before long)
>> > > > - Phoenix: 5.0.0
>> > > > - Kafka: 2.3.1
>> > > > - Ignite: 2.7.6
>> > > > - Zeppelin: 0.8.2
>> > > >
>> > > > My Teammates and I are trying to package them, and all of them
>> > > > are successfully built anyway. But we have not tested them yet,
>> > > > and I'm sure many problems will be found from now, just as Olaf
>> > > > already came across on Hadoop 3... :)
>> > > >
>> > > > > the community was lean to the direction of having important
>> component
>> > > > better supported
>> > > > > instead of spending resources for 20~30 components.
>> > > >
>> > > > Totally agreed. If we succeed to package Hadoop 3,
>> > > > I'd like to drop inactive components which can't be built with it,
>> > > > at least for now.
>> > > >
>> > > > > Just one concern about the puppet recipes compatibility across
>> multiple
>> > > > puppet versions
>> > > >
>> > > > Exactly. I think it's difficult to support Puppet 3, 4 and 5 with a
>> > > > single manifest or config file,
>> > > > so I'm thinking to create a new "puppet5" directory beside the
>> > > > existing "puppet" directory
>> > > > and put the manifests and config files for Puppet 5 into it (and
>> when
>> > > > we drop the distros
>> > > > using Puppet 3 and 4 completely in the future, we can drop the
>> > > > existing "puppet" directory
>> > > > and promote "puppet5" to "puppet").
>> > > > If it doesn't work as expected, I'll ask you the possibility to drop
>> > > > old versions in the next release again.
>> > > >
>> > > > > one source of problems was using puppet-forge for instance for
>> > > > puppet-stdlib and puppet-apt,
>> > > >
>> > > > Yeah, puppet modules seem to be installed into
>> /usr/share/puppet/modules
>> > > > via apt on Debian 9 and Ubuntu 16.04, while /etc/puppet/modules via
>> > > > `puppet module install` on CentOS 7, as you said.
>> > > > Maybe we have to consolidate them somehow, or specify both of them
>> > > > for `--modulepath` (I'm not sure if it works though), or choose
>> either of
>> > > > them
>> > > > in accordance with the distro.
>> > > >
>> > > > Kengo Seki <se...@apache.org>
>> > > >
>> > > > On Thu, Nov 21, 2019 at 3:17 PM Olaf Flebbe <of...@oflebbe.de> wrote:
>> > > > >
>> > > > > hi
>> > > > >
>> > > > > one source of problems was using puppet-forge for instance for
>> > > > puppet-stdlib and puppet-apt, since they require rather new
>> versions.
>> > > look
>> > > > out for 'puppet module install ....'. While all distros using apt
>> do
>> > > have
>> > > > matching prepackaged versions in their repository.
>> > > > >
>> > > > > other was different search paths of all these versions. we never
>> fixed
>> > > > that consistently.
>> > > > >
>> > > > > olaf
>> > > > >
>> > > > > Von meinem iPad gesendet
>> > > > >
>> > > > > > Am 21.11.2019 um 03:43 schrieb Jun HE <ju...@apache.org>:
>> > > > > >
>> > > > > > I'm fine with the new distros list. Just one concern about the
>> > > puppet
>> > > > > > recipes compatibility across multiple puppet versions (3.8.5 for
>> > > > > > Ubuntu-16.04, 4.8.2 for Debian-9, and 5.x for other new
>> distros). I
>> > > > didn't
>> > > > > > do any investigation yet. If such issues arise, I'll vote for
>> drop
>> > > > distros
>> > > > > > with older puppet.
>> > > > > >
>> > > > > > Evans Ye <ev...@apache.org> 于2019年11月21日周四 上午1:51写道：
>> > > > > >
>> > > > > >> Fine by me for the OS side.
>> > > > > >> What do you think about the components? Is there a list of
>> > > components
>> > > > you'd
>> > > > > >> like to upgrade?
>> > > > > >> We can target a subset of current supported matrix as we
>> previously
>> > > > > >> discussed about this and the community was lean to the
>> direction of
>> > > > having
>> > > > > >> important component better supported instead of spending
>> resources
>> > > for
>> > > > > >> 20~30 components.
>> > > > > >>
>> > > > > >> Youngwoo Kim (김영우) <yw...@apache.org> 於 2019年11月20日 週三
>> 上午9:41寫道：
>> > > > > >>
>> > > > > >>> Kengo,
>> > > > > >>>
>> > > > > >>> Looks good to me. I think puppet on CentOS 8 would be fine.
>> > > > > >>>
>> > > > > >>> On Cloud Native Bigtop, I believe we should consider that
>> > > components
>> > > > as a
>> > > > > >>> 'contrib' at this point.
>> > > > > >>> I'm considering about Jay's idea, making 'CNB' on master as a
>> > > contrib
>> > > > > >>> module. A development branch is good but on our "two-tracks"
>> > > > development,
>> > > > > >>> 'contrib' module will be easier for us to maintain traditional
>> > > > distros
>> > > > > >> and
>> > > > > >>> cnb.
>> > > > > >>>
>> > > > > >>> Thanks,
>> > > > > >>> Youngwoo
>> > > > > >>>
>> > > > > >>>> On Wed, Nov 20, 2019 at 9:28 AM Kengo Seki <
>> sekikn@apache.org>
>> > > > wrote:
>> > > > > >>>
>> > > > > >>>> Hi folks,
>> > > > > >>>>
>> > > > > >>>> I'd like to discuss the target distros for the next 1.5.0
>> release
>> > > > [1],
>> > > > > >>>> because over 1.5 years have passed since Ubuntu 18.04 was
>> released
>> > > > > >>>> and the next LTS will be released within half a year. In
>> addition,
>> > > > > >>>> Fedora 26 and openSUSE 42.3 have already been EOL'd.
>> > > > > >>>>
>> > > > > >>>> (I understand the "Cloud Native Bigtop" project is going on
>> > > > > >>>> and am really looking forward to it, but my customers still
>> > > requires
>> > > > > >>>> the traditional software stack :)
>> > > > > >>>>
>> > > > > >>>> Based on the past discussion [2], here's my proposal:
>> > > > > >>>>
>> > > > > >>>> - Add Debian 10, Fedora 31 and Ubuntu 18.04 as the target
>> distros
>> > > > > >>>> and use the puppet package provided by each distro, so that
>> > > > > >>>> we can support all CPU architectures (x86_64, aarch64, and
>> > > > ppc64le).
>> > > > > >>>> Their puppet versions are 5.4.0 (ubuntu) and 5.5.10 (debian
>> and
>> > > > > >>> fedora).
>> > > > > >>>>
>> > > > > >>>> Keep Debian 9 and Ubuntu 16.04 since they are still in the
>> > > support
>> > > > > >>>> period.
>> > > > > >>>>
>> > > > > >>>> Drop Fedora 26 since it has reached to the EOL on
>> 2018-05-29.
>> > > > > >>>>
>> > > > > >>>> - Add CentOS 8. Unfortunately, that version doesn't seem to
>> > > > > >>>> provide the distro's puppet package, even including EPEL.
>> > > > > >>>> Even though, I'd like to support it since that distro
>> > > > > >>>> (and RHEL8) are widely used especially in enterprise
>> systems.
>> > > > > >>>> So, as the next best option, how about using Puppet 5.5
>> provided
>> > > by
>> > > > > >>>> Puppetlabs and only supporting the x86_64 architecture on
>> this
>> > > > > >> version?
>> > > > > >>>>
>> > > > > >>>> Keep CentOS 7 since it's still in the support period.
>> > > > > >>>>
>> > > > > >>>> - Drop openSUSE 42.3 since it has reached to the EOL on
>> 2019-07-01
>> > > > > >>>> and don't add a new version of that distro, as discussed in
>> [2].
>> > > > > >>>>
>> > > > > >>>> To summarize the above, the supported distros and their
>> versions
>> > > > > >>>> in the 1.5.0 release are as follows:
>> > > > > >>>>
>> > > > > >>>> - CentOS 7, 8 (8 is only supported on x86_64)
>> > > > > >>>> - Debian 9, 10
>> > > > > >>>> - Fedora 31
>> > > > > >>>> - Ubuntu 16.04, 18.04
>> > > > > >>>>
>> > > > > >>>> Does this sound reasonable? I'd appreciate any comments or
>> > > > suggestions.
>> > > > > >>>>
>> > > > > >>>> (Honestly, I'd actually like to drop CentOS 7, Debian 9, and
>> > > Ubuntu
>> > > > > >>> 16.04,
>> > > > > >>>> so that we can consolidate the Puppet version to 5.x.
>> > > > > >>>> But it may be too aggressive for users.)
>> > > > > >>>>
>> > > > > >>>> [1]: https://issues.apache.org/jira/browse/BIGTOP-3123
>> > > > > >>>> [2]:
>> > > > > >>>>
>> > > > > >>>
>> > > > > >>
>> > > >
>> > >
>> https://lists.apache.org/thread.html/26e14cf36e9cfd61e0de581ed83bf305565c2e65234f1ce3bfb97628@%3Cdev.bigtop.apache.org%3E
>> > > > > >>>>
>> > > > > >>>> Kengo Seki <se...@apache.org>
>> > > > > >>>>
>> > > > > >>>
>> > > > > >>
>> > > >
>> > >
>>
>
Re: [DISCUSS] Target distros on the 1.5.0 release
Posted by Evans Ye <ev...@apache.org>.
Awesome! Presumably there's no -1 on this, so I bypass the vote. But if
there is please don't hesitate :)
Anyway, this will be fun and bring us to another chapter of Bigtop. Let's
get 1.5 rolled.
We ship 3 things for a release:
1. source code tgz (ASF ftp)
2. itest jars (ASF nexus)
3. binary convenient artifacts (on sponsored S3)
The full process is covered by [1].
For 1.5.0 release, I want to keep improving our release automation. Last
time we improved our CI and testing framework, this time I'd like to take
the chance to automate the binary artifacts part.
The ultimate goal is the same: release early, release often.
Please let me know what you think about the release doc and how can we
improve it.
Do you have a rough timeline on the date for first release candidate?
Evans
[1] https://cwiki.apache.org/confluence/display/BIGTOP/How+to+release
Kengo Seki <se...@apache.org> 於 2020年2月5日 週三 上午9:57寫道：
> Evans,
>
> I'd be glad to take on that role. Thank you for the invitation!
>
> Kengo Seki <se...@apache.org>
>
> On Tue, Feb 4, 2020 at 2:48 AM Evans Ye <ev...@apache.org> wrote:
> >
> > Hi Kengo,
> >
> > Based on what you've done so far and you're already a committer of
> Bigtop,
> > would you like to be the Release Manager of Bigtop 1.5.0 release? Might
> got
> > things to do as a RM but we as a community can help and work together. I
> > served as RM for two times and I find it a great experience to know how
> > apache releases work as a whole.
> >
> > Evans
> >
> > Evans Ye <ev...@apache.org> 於 2019年12月22日 週日 上午2:16寫道：
> >
> > > Elasticsearch has been merged[1]. Thanks to Jun for adding puppet and
> > > Smoke Test within that PR.
> > >
> > > [1] https://github.com/apache/bigtop/pull/566
> > >
> > > Kengo Seki <se...@apache.org> 於 2019年12月13日 週五 09:30 寫道：
> > >
> > >> Jun and Evans, sorry for my late response.
> > >>
> > >> > So if that Jira (BIGTOP-3219) can be done in next week, do you
> think it
> > >> is
> > >> > OK to include Elasticsearch-5.6.14 in v1.5?
> > >>
> > >> Of course! I'm looking forward to it.
> > >>
> > >> Kengo Seki <se...@apache.org>
> > >>
> > >> On Fri, Dec 6, 2019 at 8:08 PM Evans Ye <ev...@apache.org> wrote:
> > >> >
> > >> > I'm a +1 for this. I can help on the ci side so that adding one more
> > >> > package for release is easier.
> > >> >
> > >> > Jun HE <ju...@apache.org> 於 2019年12月6日 週五 15:41 寫道：
> > >> >
> > >> > > Hi, Kengo,
> > >> > >
> > >> > > I just noticed you updated the BOM of Bigtop v1.5 on BIGTOP-3123.
> > >> > >
> > >> > > One thing I'd like to know your and other folks' thought on
> > >> Elasticsearch
> > >> > > as Bigtop component.
> > >> > > There is a ticket (
> https://issues.apache.org/jira/browse/BIGTOP-3219)
> > >> for
> > >> > > this. And I've finished most of the sub-works
> > >> > > (build/packaging/deployement). Patch could be found at:
> > >> > > https://github.com/apache/bigtop/pull/566
> > >> > > For the smoke test part I think it could be done in this weekend.
> > >> > >
> > >> > > So if that Jira (BIGTOP-3219) can be done in next week, do you
> think
> > >> it is
> > >> > > OK to include Elasticsearch-5.6.14 in v1.5?
> > >> > >
> > >> > > Regards,
> > >> > >
> > >> > > Jun
> > >> > >
> > >> > > Kengo Seki <se...@apache.org> 于2019年11月22日周五 上午12:09写道：
> > >> > >
> > >> > > > Thanks for the comments, everyone!
> > >> > > > If there's still no objection in a few days, I'm going to update
> > >> > > > BIGTOP-3123's description.
> > >> > > >
> > >> > > > > 'contrib' module will be easier for us to maintain traditional
> > >> distros
> > >> > > > and cnb.
> > >> > > >
> > >> > > > Agreed. I think that merging the cnb branch later will be a hard
> > >> work
> > >> > > too.
> > >> > > >
> > >> > > > > What do you think about the components?
> > >> > > > > Is there a list of components you'd like to upgrade?
> > >> > > >
> > >> > > > I'd like to upgrade the following components:
> > >> > > >
> > >> > > > - Zookeeper: 3.4.13
> > >> > > > - Hadoop: 3.2.1 (or 2.10.0 if packaging Hadoop 3 is too hard, as
> > >> Olaf
> > >> > > > mentioned before)
> > >> > > > - HBase: 2.2.2
> > >> > > > - Hive: 3.1.2
> > >> > > > - Tez: 0.9.2
> > >> > > > - Spark: 2.4.4 (or 3.0.0, if GA is released before long)
> > >> > > > - Phoenix: 5.0.0
> > >> > > > - Kafka: 2.3.1
> > >> > > > - Ignite: 2.7.6
> > >> > > > - Zeppelin: 0.8.2
> > >> > > >
> > >> > > > My Teammates and I are trying to package them, and all of them
> > >> > > > are successfully built anyway. But we have not tested them yet,
> > >> > > > and I'm sure many problems will be found from now, just as Olaf
> > >> > > > already came across on Hadoop 3... :)
> > >> > > >
> > >> > > > > the community was lean to the direction of having important
> > >> component
> > >> > > > better supported
> > >> > > > > instead of spending resources for 20~30 components.
> > >> > > >
> > >> > > > Totally agreed. If we succeed to package Hadoop 3,
> > >> > > > I'd like to drop inactive components which can't be built with
> it,
> > >> > > > at least for now.
> > >> > > >
> > >> > > > > Just one concern about the puppet recipes compatibility across
> > >> multiple
> > >> > > > puppet versions
> > >> > > >
> > >> > > > Exactly. I think it's difficult to support Puppet 3, 4 and 5
> with a
> > >> > > > single manifest or config file,
> > >> > > > so I'm thinking to create a new "puppet5" directory beside the
> > >> > > > existing "puppet" directory
> > >> > > > and put the manifests and config files for Puppet 5 into it (and
> > >> when
> > >> > > > we drop the distros
> > >> > > > using Puppet 3 and 4 completely in the future, we can drop the
> > >> > > > existing "puppet" directory
> > >> > > > and promote "puppet5" to "puppet").
> > >> > > > If it doesn't work as expected, I'll ask you the possibility to
> drop
> > >> > > > old versions in the next release again.
> > >> > > >
> > >> > > > > one source of problems was using puppet-forge for instance for
> > >> > > > puppet-stdlib and puppet-apt,
> > >> > > >
> > >> > > > Yeah, puppet modules seem to be installed into
> > >> /usr/share/puppet/modules
> > >> > > > via apt on Debian 9 and Ubuntu 16.04, while /etc/puppet/modules
> via
> > >> > > > `puppet module install` on CentOS 7, as you said.
> > >> > > > Maybe we have to consolidate them somehow, or specify both of
> them
> > >> > > > for `--modulepath` (I'm not sure if it works though), or choose
> > >> either of
> > >> > > > them
> > >> > > > in accordance with the distro.
> > >> > > >
> > >> > > > Kengo Seki <se...@apache.org>
> > >> > > >
> > >> > > > On Thu, Nov 21, 2019 at 3:17 PM Olaf Flebbe <of...@oflebbe.de>
> wrote:
> > >> > > > >
> > >> > > > > hi
> > >> > > > >
> > >> > > > > one source of problems was using puppet-forge for instance for
> > >> > > > puppet-stdlib and puppet-apt, since they require rather new
> > >> versions.
> > >> > > look
> > >> > > > out for 'puppet module install ....'. While all distros using
> apt
> > >> do
> > >> > > have
> > >> > > > matching prepackaged versions in their repository.
> > >> > > > >
> > >> > > > > other was different search paths of all these versions. we
> never
> > >> fixed
> > >> > > > that consistently.
> > >> > > > >
> > >> > > > > olaf
> > >> > > > >
> > >> > > > > Von meinem iPad gesendet
> > >> > > > >
> > >> > > > > > Am 21.11.2019 um 03:43 schrieb Jun HE <ju...@apache.org>:
> > >> > > > > >
> > >> > > > > > I'm fine with the new distros list. Just one concern about
> the
> > >> > > puppet
> > >> > > > > > recipes compatibility across multiple puppet versions
> (3.8.5 for
> > >> > > > > > Ubuntu-16.04, 4.8.2 for Debian-9, and 5.x for other new
> > >> distros). I
> > >> > > > didn't
> > >> > > > > > do any investigation yet. If such issues arise, I'll vote
> for
> > >> drop
> > >> > > > distros
> > >> > > > > > with older puppet.
> > >> > > > > >
> > >> > > > > > Evans Ye <ev...@apache.org> 于2019年11月21日周四 上午1:51写道：
> > >> > > > > >
> > >> > > > > >> Fine by me for the OS side.
> > >> > > > > >> What do you think about the components? Is there a list of
> > >> > > components
> > >> > > > you'd
> > >> > > > > >> like to upgrade?
> > >> > > > > >> We can target a subset of current supported matrix as we
> > >> previously
> > >> > > > > >> discussed about this and the community was lean to the
> > >> direction of
> > >> > > > having
> > >> > > > > >> important component better supported instead of spending
> > >> resources
> > >> > > for
> > >> > > > > >> 20~30 components.
> > >> > > > > >>
> > >> > > > > >> Youngwoo Kim (김영우) <yw...@apache.org> 於 2019年11月20日 週三
> > >> 上午9:41寫道：
> > >> > > > > >>
> > >> > > > > >>> Kengo,
> > >> > > > > >>>
> > >> > > > > >>> Looks good to me. I think puppet on CentOS 8 would be
> fine.
> > >> > > > > >>>
> > >> > > > > >>> On Cloud Native Bigtop, I believe we should consider that
> > >> > > components
> > >> > > > as a
> > >> > > > > >>> 'contrib' at this point.
> > >> > > > > >>> I'm considering about Jay's idea, making 'CNB' on master
> as a
> > >> > > contrib
> > >> > > > > >>> module. A development branch is good but on our
> "two-tracks"
> > >> > > > development,
> > >> > > > > >>> 'contrib' module will be easier for us to maintain
> traditional
> > >> > > > distros
> > >> > > > > >> and
> > >> > > > > >>> cnb.
> > >> > > > > >>>
> > >> > > > > >>> Thanks,
> > >> > > > > >>> Youngwoo
> > >> > > > > >>>
> > >> > > > > >>>> On Wed, Nov 20, 2019 at 9:28 AM Kengo Seki <
> > >> sekikn@apache.org>
> > >> > > > wrote:
> > >> > > > > >>>
> > >> > > > > >>>> Hi folks,
> > >> > > > > >>>>
> > >> > > > > >>>> I'd like to discuss the target distros for the next 1.5.0
> > >> release
> > >> > > > [1],
> > >> > > > > >>>> because over 1.5 years have passed since Ubuntu 18.04 was
> > >> released
> > >> > > > > >>>> and the next LTS will be released within half a year. In
> > >> addition,
> > >> > > > > >>>> Fedora 26 and openSUSE 42.3 have already been EOL'd.
> > >> > > > > >>>>
> > >> > > > > >>>> (I understand the "Cloud Native Bigtop" project is going
> on
> > >> > > > > >>>> and am really looking forward to it, but my customers
> still
> > >> > > requires
> > >> > > > > >>>> the traditional software stack :)
> > >> > > > > >>>>
> > >> > > > > >>>> Based on the past discussion [2], here's my proposal:
> > >> > > > > >>>>
> > >> > > > > >>>> - Add Debian 10, Fedora 31 and Ubuntu 18.04 as the target
> > >> distros
> > >> > > > > >>>> and use the puppet package provided by each distro, so
> that
> > >> > > > > >>>> we can support all CPU architectures (x86_64, aarch64,
> and
> > >> > > > ppc64le).
> > >> > > > > >>>> Their puppet versions are 5.4.0 (ubuntu) and 5.5.10
> (debian
> > >> and
> > >> > > > > >>> fedora).
> > >> > > > > >>>>
> > >> > > > > >>>> Keep Debian 9 and Ubuntu 16.04 since they are still in
> the
> > >> > > support
> > >> > > > > >>>> period.
> > >> > > > > >>>>
> > >> > > > > >>>> Drop Fedora 26 since it has reached to the EOL on
> > >> 2018-05-29.
> > >> > > > > >>>>
> > >> > > > > >>>> - Add CentOS 8. Unfortunately, that version doesn't seem
> to
> > >> > > > > >>>> provide the distro's puppet package, even including
> EPEL.
> > >> > > > > >>>> Even though, I'd like to support it since that distro
> > >> > > > > >>>> (and RHEL8) are widely used especially in enterprise
> > >> systems.
> > >> > > > > >>>> So, as the next best option, how about using Puppet 5.5
> > >> provided
> > >> > > by
> > >> > > > > >>>> Puppetlabs and only supporting the x86_64 architecture
> on
> > >> this
> > >> > > > > >> version?
> > >> > > > > >>>>
> > >> > > > > >>>> Keep CentOS 7 since it's still in the support period.
> > >> > > > > >>>>
> > >> > > > > >>>> - Drop openSUSE 42.3 since it has reached to the EOL on
> > >> 2019-07-01
> > >> > > > > >>>> and don't add a new version of that distro, as
> discussed in
> > >> [2].
> > >> > > > > >>>>
> > >> > > > > >>>> To summarize the above, the supported distros and their
> > >> versions
> > >> > > > > >>>> in the 1.5.0 release are as follows:
> > >> > > > > >>>>
> > >> > > > > >>>> - CentOS 7, 8 (8 is only supported on x86_64)
> > >> > > > > >>>> - Debian 9, 10
> > >> > > > > >>>> - Fedora 31
> > >> > > > > >>>> - Ubuntu 16.04, 18.04
> > >> > > > > >>>>
> > >> > > > > >>>> Does this sound reasonable? I'd appreciate any comments
> or
> > >> > > > suggestions.
> > >> > > > > >>>>
> > >> > > > > >>>> (Honestly, I'd actually like to drop CentOS 7, Debian 9,
> and
> > >> > > Ubuntu
> > >> > > > > >>> 16.04,
> > >> > > > > >>>> so that we can consolidate the Puppet version to 5.x.
> > >> > > > > >>>> But it may be too aggressive for users.)
> > >> > > > > >>>>
> > >> > > > > >>>> [1]: https://issues.apache.org/jira/browse/BIGTOP-3123
> > >> > > > > >>>> [2]:
> > >> > > > > >>>>
> > >> > > > > >>>
> > >> > > > > >>
> > >> > > >
> > >> > >
> > >>
> https://lists.apache.org/thread.html/26e14cf36e9cfd61e0de581ed83bf305565c2e65234f1ce3bfb97628@%3Cdev.bigtop.apache.org%3E
> > >> > > > > >>>>
> > >> > > > > >>>> Kengo Seki <se...@apache.org>
> > >> > > > > >>>>
> > >> > > > > >>>
> > >> > > > > >>
> > >> > > >
> > >> > >
> > >>
> > >
>
Re: [DISCUSS] Target distros on the 1.5.0 release
Posted by Kengo Seki <se...@apache.org>.
Evans,
I'd be glad to take on that role. Thank you for the invitation!
Kengo Seki <se...@apache.org>
On Tue, Feb 4, 2020 at 2:48 AM Evans Ye <ev...@apache.org> wrote:
>
> Hi Kengo,
>
> Based on what you've done so far and you're already a committer of Bigtop,
> would you like to be the Release Manager of Bigtop 1.5.0 release? Might got
> things to do as a RM but we as a community can help and work together. I
> served as RM for two times and I find it a great experience to know how
> apache releases work as a whole.
>
> Evans
>
> Evans Ye <ev...@apache.org> 於 2019年12月22日 週日 上午2:16寫道：
>
> > Elasticsearch has been merged[1]. Thanks to Jun for adding puppet and
> > Smoke Test within that PR.
> >
> > [1] https://github.com/apache/bigtop/pull/566
> >
> > Kengo Seki <se...@apache.org> 於 2019年12月13日 週五 09:30 寫道：
> >
> >> Jun and Evans, sorry for my late response.
> >>
> >> > So if that Jira (BIGTOP-3219) can be done in next week, do you think it
> >> is
> >> > OK to include Elasticsearch-5.6.14 in v1.5?
> >>
> >> Of course! I'm looking forward to it.
> >>
> >> Kengo Seki <se...@apache.org>
> >>
> >> On Fri, Dec 6, 2019 at 8:08 PM Evans Ye <ev...@apache.org> wrote:
> >> >
> >> > I'm a +1 for this. I can help on the ci side so that adding one more
> >> > package for release is easier.
> >> >
> >> > Jun HE <ju...@apache.org> 於 2019年12月6日 週五 15:41 寫道：
> >> >
> >> > > Hi, Kengo,
> >> > >
> >> > > I just noticed you updated the BOM of Bigtop v1.5 on BIGTOP-3123.
> >> > >
> >> > > One thing I'd like to know your and other folks' thought on
> >> Elasticsearch
> >> > > as Bigtop component.
> >> > > There is a ticket (https://issues.apache.org/jira/browse/BIGTOP-3219)
> >> for
> >> > > this. And I've finished most of the sub-works
> >> > > (build/packaging/deployement). Patch could be found at:
> >> > > https://github.com/apache/bigtop/pull/566
> >> > > For the smoke test part I think it could be done in this weekend.
> >> > >
> >> > > So if that Jira (BIGTOP-3219) can be done in next week, do you think
> >> it is
> >> > > OK to include Elasticsearch-5.6.14 in v1.5?
> >> > >
> >> > > Regards,
> >> > >
> >> > > Jun
> >> > >
> >> > > Kengo Seki <se...@apache.org> 于2019年11月22日周五 上午12:09写道：
> >> > >
> >> > > > Thanks for the comments, everyone!
> >> > > > If there's still no objection in a few days, I'm going to update
> >> > > > BIGTOP-3123's description.
> >> > > >
> >> > > > > 'contrib' module will be easier for us to maintain traditional
> >> distros
> >> > > > and cnb.
> >> > > >
> >> > > > Agreed. I think that merging the cnb branch later will be a hard
> >> work
> >> > > too.
> >> > > >
> >> > > > > What do you think about the components?
> >> > > > > Is there a list of components you'd like to upgrade?
> >> > > >
> >> > > > I'd like to upgrade the following components:
> >> > > >
> >> > > > - Zookeeper: 3.4.13
> >> > > > - Hadoop: 3.2.1 (or 2.10.0 if packaging Hadoop 3 is too hard, as
> >> Olaf
> >> > > > mentioned before)
> >> > > > - HBase: 2.2.2
> >> > > > - Hive: 3.1.2
> >> > > > - Tez: 0.9.2
> >> > > > - Spark: 2.4.4 (or 3.0.0, if GA is released before long)
> >> > > > - Phoenix: 5.0.0
> >> > > > - Kafka: 2.3.1
> >> > > > - Ignite: 2.7.6
> >> > > > - Zeppelin: 0.8.2
> >> > > >
> >> > > > My Teammates and I are trying to package them, and all of them
> >> > > > are successfully built anyway. But we have not tested them yet,
> >> > > > and I'm sure many problems will be found from now, just as Olaf
> >> > > > already came across on Hadoop 3... :)
> >> > > >
> >> > > > > the community was lean to the direction of having important
> >> component
> >> > > > better supported
> >> > > > > instead of spending resources for 20~30 components.
> >> > > >
> >> > > > Totally agreed. If we succeed to package Hadoop 3,
> >> > > > I'd like to drop inactive components which can't be built with it,
> >> > > > at least for now.
> >> > > >
> >> > > > > Just one concern about the puppet recipes compatibility across
> >> multiple
> >> > > > puppet versions
> >> > > >
> >> > > > Exactly. I think it's difficult to support Puppet 3, 4 and 5 with a
> >> > > > single manifest or config file,
> >> > > > so I'm thinking to create a new "puppet5" directory beside the
> >> > > > existing "puppet" directory
> >> > > > and put the manifests and config files for Puppet 5 into it (and
> >> when
> >> > > > we drop the distros
> >> > > > using Puppet 3 and 4 completely in the future, we can drop the
> >> > > > existing "puppet" directory
> >> > > > and promote "puppet5" to "puppet").
> >> > > > If it doesn't work as expected, I'll ask you the possibility to drop
> >> > > > old versions in the next release again.
> >> > > >
> >> > > > > one source of problems was using puppet-forge for instance for
> >> > > > puppet-stdlib and puppet-apt,
> >> > > >
> >> > > > Yeah, puppet modules seem to be installed into
> >> /usr/share/puppet/modules
> >> > > > via apt on Debian 9 and Ubuntu 16.04, while /etc/puppet/modules via
> >> > > > `puppet module install` on CentOS 7, as you said.
> >> > > > Maybe we have to consolidate them somehow, or specify both of them
> >> > > > for `--modulepath` (I'm not sure if it works though), or choose
> >> either of
> >> > > > them
> >> > > > in accordance with the distro.
> >> > > >
> >> > > > Kengo Seki <se...@apache.org>
> >> > > >
> >> > > > On Thu, Nov 21, 2019 at 3:17 PM Olaf Flebbe <of...@oflebbe.de> wrote:
> >> > > > >
> >> > > > > hi
> >> > > > >
> >> > > > > one source of problems was using puppet-forge for instance for
> >> > > > puppet-stdlib and puppet-apt, since they require rather new
> >> versions.
> >> > > look
> >> > > > out for 'puppet module install ....'. While all distros using apt
> >> do
> >> > > have
> >> > > > matching prepackaged versions in their repository.
> >> > > > >
> >> > > > > other was different search paths of all these versions. we never
> >> fixed
> >> > > > that consistently.
> >> > > > >
> >> > > > > olaf
> >> > > > >
> >> > > > > Von meinem iPad gesendet
> >> > > > >
> >> > > > > > Am 21.11.2019 um 03:43 schrieb Jun HE <ju...@apache.org>:
> >> > > > > >
> >> > > > > > I'm fine with the new distros list. Just one concern about the
> >> > > puppet
> >> > > > > > recipes compatibility across multiple puppet versions (3.8.5 for
> >> > > > > > Ubuntu-16.04, 4.8.2 for Debian-9, and 5.x for other new
> >> distros). I
> >> > > > didn't
> >> > > > > > do any investigation yet. If such issues arise, I'll vote for
> >> drop
> >> > > > distros
> >> > > > > > with older puppet.
> >> > > > > >
> >> > > > > > Evans Ye <ev...@apache.org> 于2019年11月21日周四 上午1:51写道：
> >> > > > > >
> >> > > > > >> Fine by me for the OS side.
> >> > > > > >> What do you think about the components? Is there a list of
> >> > > components
> >> > > > you'd
> >> > > > > >> like to upgrade?
> >> > > > > >> We can target a subset of current supported matrix as we
> >> previously
> >> > > > > >> discussed about this and the community was lean to the
> >> direction of
> >> > > > having
> >> > > > > >> important component better supported instead of spending
> >> resources
> >> > > for
> >> > > > > >> 20~30 components.
> >> > > > > >>
> >> > > > > >> Youngwoo Kim (김영우) <yw...@apache.org> 於 2019年11月20日 週三
> >> 上午9:41寫道：
> >> > > > > >>
> >> > > > > >>> Kengo,
> >> > > > > >>>
> >> > > > > >>> Looks good to me. I think puppet on CentOS 8 would be fine.
> >> > > > > >>>
> >> > > > > >>> On Cloud Native Bigtop, I believe we should consider that
> >> > > components
> >> > > > as a
> >> > > > > >>> 'contrib' at this point.
> >> > > > > >>> I'm considering about Jay's idea, making 'CNB' on master as a
> >> > > contrib
> >> > > > > >>> module. A development branch is good but on our "two-tracks"
> >> > > > development,
> >> > > > > >>> 'contrib' module will be easier for us to maintain traditional
> >> > > > distros
> >> > > > > >> and
> >> > > > > >>> cnb.
> >> > > > > >>>
> >> > > > > >>> Thanks,
> >> > > > > >>> Youngwoo
> >> > > > > >>>
> >> > > > > >>>> On Wed, Nov 20, 2019 at 9:28 AM Kengo Seki <
> >> sekikn@apache.org>
> >> > > > wrote:
> >> > > > > >>>
> >> > > > > >>>> Hi folks,
> >> > > > > >>>>
> >> > > > > >>>> I'd like to discuss the target distros for the next 1.5.0
> >> release
> >> > > > [1],
> >> > > > > >>>> because over 1.5 years have passed since Ubuntu 18.04 was
> >> released
> >> > > > > >>>> and the next LTS will be released within half a year. In
> >> addition,
> >> > > > > >>>> Fedora 26 and openSUSE 42.3 have already been EOL'd.
> >> > > > > >>>>
> >> > > > > >>>> (I understand the "Cloud Native Bigtop" project is going on
> >> > > > > >>>> and am really looking forward to it, but my customers still
> >> > > requires
> >> > > > > >>>> the traditional software stack :)
> >> > > > > >>>>
> >> > > > > >>>> Based on the past discussion [2], here's my proposal:
> >> > > > > >>>>
> >> > > > > >>>> - Add Debian 10, Fedora 31 and Ubuntu 18.04 as the target
> >> distros
> >> > > > > >>>> and use the puppet package provided by each distro, so that
> >> > > > > >>>> we can support all CPU architectures (x86_64, aarch64, and
> >> > > > ppc64le).
> >> > > > > >>>> Their puppet versions are 5.4.0 (ubuntu) and 5.5.10 (debian
> >> and
> >> > > > > >>> fedora).
> >> > > > > >>>>
> >> > > > > >>>> Keep Debian 9 and Ubuntu 16.04 since they are still in the
> >> > > support
> >> > > > > >>>> period.
> >> > > > > >>>>
> >> > > > > >>>> Drop Fedora 26 since it has reached to the EOL on
> >> 2018-05-29.
> >> > > > > >>>>
> >> > > > > >>>> - Add CentOS 8. Unfortunately, that version doesn't seem to
> >> > > > > >>>> provide the distro's puppet package, even including EPEL.
> >> > > > > >>>> Even though, I'd like to support it since that distro
> >> > > > > >>>> (and RHEL8) are widely used especially in enterprise
> >> systems.
> >> > > > > >>>> So, as the next best option, how about using Puppet 5.5
> >> provided
> >> > > by
> >> > > > > >>>> Puppetlabs and only supporting the x86_64 architecture on
> >> this
> >> > > > > >> version?
> >> > > > > >>>>
> >> > > > > >>>> Keep CentOS 7 since it's still in the support period.
> >> > > > > >>>>
> >> > > > > >>>> - Drop openSUSE 42.3 since it has reached to the EOL on
> >> 2019-07-01
> >> > > > > >>>> and don't add a new version of that distro, as discussed in
> >> [2].
> >> > > > > >>>>
> >> > > > > >>>> To summarize the above, the supported distros and their
> >> versions
> >> > > > > >>>> in the 1.5.0 release are as follows:
> >> > > > > >>>>
> >> > > > > >>>> - CentOS 7, 8 (8 is only supported on x86_64)
> >> > > > > >>>> - Debian 9, 10
> >> > > > > >>>> - Fedora 31
> >> > > > > >>>> - Ubuntu 16.04, 18.04
> >> > > > > >>>>
> >> > > > > >>>> Does this sound reasonable? I'd appreciate any comments or
> >> > > > suggestions.
> >> > > > > >>>>
> >> > > > > >>>> (Honestly, I'd actually like to drop CentOS 7, Debian 9, and
> >> > > Ubuntu
> >> > > > > >>> 16.04,
> >> > > > > >>>> so that we can consolidate the Puppet version to 5.x.
> >> > > > > >>>> But it may be too aggressive for users.)
> >> > > > > >>>>
> >> > > > > >>>> [1]: https://issues.apache.org/jira/browse/BIGTOP-3123
> >> > > > > >>>> [2]:
> >> > > > > >>>>
> >> > > > > >>>
> >> > > > > >>
> >> > > >
> >> > >
> >> https://lists.apache.org/thread.html/26e14cf36e9cfd61e0de581ed83bf305565c2e65234f1ce3bfb97628@%3Cdev.bigtop.apache.org%3E
> >> > > > > >>>>
> >> > > > > >>>> Kengo Seki <se...@apache.org>
> >> > > > > >>>>
> >> > > > > >>>
> >> > > > > >>
> >> > > >
> >> > >
> >>
> >