You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Kengo Seki <se...@apache.org> on 2020/12/06 23:46:51 UTC
[VOTE] Release Bigtop version 1.5.0
This is the vote for release 1.5.0 of Apache Bigtop.
It fixes the following issues:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12345178
The vote will be going for at least 72 hours and will be closed on Wednesday,
December 9th, 2020 at 5pm PST. Please download, test and vote with
[ ] +1, accept RC0 as the official 1.5.0 release of Apache Bigtop
[ ] +0, I don't care either way,
[ ] -1, do not accept RC0 as the official 1.5.0 release of Apache
Bigtop, because...
Source and binary files:
https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.5.0-RC0/
Maven staging repo:
https://repository.apache.org/content/repositories/orgapachebigtop-1030/
The git tag to be voted upon is release-1.5.0
Bigtop's KEYS file containing PGP keys we use to sign the release:
https://dist.apache.org/repos/dist/release/bigtop/KEYS
For your reference, the CI result [1] is as follows. It's also
summarized in the cwiki [2].
- All components (except for Phoenix, which doesn't have a puppet
manifest and smoke test)
are successfully deployed onto all distros/platforms.
- All smoke tests except for the following combinations succeeded on CI.
The followings only partially succeeded on CI, but I confirmed that
all of them passed on my local machine and AWS m6g (arm64) instance.
- x86_64
- QFS on Ubuntu 16.04
- aarch64:
- HBase on Fedora 31
- Kibana on all distros
- Livy on all distros except for CentOS 8
- Oozie on CentOS 8
- QFS on Ubuntu 16.04
[1]: https://ci.bigtop.apache.org/view/Test/job/Bigtop-1.5.0-smoke-tests/
[2]: https://cwiki.apache.org/confluence/display/BIGTOP/Overview+of+Bigtop+1.5.0+Support+Matrix
Kengo Seki <se...@apache.org>
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Kengo Seki <se...@apache.org>.
I tried to uninstall all Vagrant plugins and re-install only
vagrant-vbguest, and came across the same problem as yours.
It seems a problem about CentOS release process, rather than
vagrant-vbguest or Vagrant provisioner itself.
Enabling 7.8 repositories in /etc/yum.repos.d/CentOS-Vault.repo inside
VM resolved this issue, following this comment.
https://github.com/dotless-de/vagrant-vbguest/issues/311#issuecomment-733202667
> Updating the default value sounds good to me. We can address it in later developments.
Thanks! Then I'm going to file it as a follow-up issue including the
above workaround, and address it after the release.
May I have your explicit +1 when your test is finished? We've still
had no binding +1, though Yuqi, Luca and Masatake gave non-binding
ones.
And other PMC members, would you take the time to verify this RC? We
still need one more binding +1.
Kengo Seki <se...@apache.org>
On Mon, Dec 14, 2020 at 3:12 AM Evans Ye <ev...@apache.org> wrote:
>
> Mine doesn't work still. Probably the env issue since I'm away from the
> field...
> Anyhow. Updating the default value sounds good to me. We can address it in
> later developments.
>
> ERROR LOG:
> ---
> $ vagrant destroy -f; vagrant up
> /Users/eyeh/.vagrant.d/gems/2.4.4/gems/rubyhacks-0.1.5/lib/rubyhacks.rb:536:
> warning: constant ::Fixnum is deprecated
> vagrant conf local repo enabled: false
> Adding output/ repo ? false
> ==> bigtop1: Forcing shutdown of VM...
> ==> bigtop1: Destroying VM and associated drives...
> ==> bigtop1: [vagrant-hostmanager:guests] Updating hosts file on active
> guest virtual machines...
> /Users/eyeh/.vagrant.d/gems/2.4.4/gems/rubyhacks-0.1.5/lib/rubyhacks.rb:536:
> warning: constant ::Fixnum is deprecated
> vagrant conf local repo enabled: false
> Adding output/ repo ? false
> Bringing machine 'bigtop1' up with 'virtualbox' provider...
> ==> bigtop1: Importing base box 'centos/7'...
> ==> bigtop1: Matching MAC address for NAT networking...
> ==> bigtop1: Checking if box 'centos/7' version '2004.01' is up to date...
> ==> bigtop1: Setting the name of the VM: vagrant_bigtop1_1607841501336_78663
> ==> bigtop1: Clearing any previously set network interfaces...
> ==> bigtop1: Preparing network interfaces based on configuration...
> bigtop1: Adapter 1: nat
> bigtop1: Adapter 2: hostonly
> ==> bigtop1: Forwarding ports...
> bigtop1: 22 (guest) => 2222 (host) (adapter 1)
> ==> bigtop1: Running 'pre-boot' VM customizations...
> ==> bigtop1: Booting VM...
> ==> bigtop1: Waiting for machine to boot. This may take a few minutes...
> bigtop1: SSH address: 127.0.0.1:2222
> bigtop1: SSH username: vagrant
> bigtop1: SSH auth method: private key
> bigtop1:
> bigtop1: Vagrant insecure key detected. Vagrant will automatically
> replace
> bigtop1: this with a newly generated keypair for better security.
> bigtop1:
> bigtop1: Inserting generated public key within guest...
> bigtop1: Removing insecure key from the guest if it's present...
> bigtop1: Key inserted! Disconnecting and reconnecting using new SSH
> key...
> ==> bigtop1: Machine booted and ready!
> [bigtop1] No Virtualbox Guest Additions installation found.
> Loaded plugins: fastestmirror
> Loading mirror speeds from cached hostfile
> * base: ftp.tc.edu.tw
> * extras: ftp.tc.edu.tw
> * updates: ftp.tc.edu.tw
> Resolving Dependencies
> --> Running transaction check
> ---> Package centos-release.x86_64 0:7-8.2003.0.el7.centos will be updated
> ---> Package centos-release.x86_64 0:7-9.2009.1.el7.centos will be an update
> --> Finished Dependency Resolution
>
> Dependencies Resolved
>
> ================================================================================
> Package Arch Version Repository
> Size
> ================================================================================
> Updating:
> centos-release x86_64 7-9.2009.1.el7.centos updates
> 27 k
>
> Transaction Summary
> ================================================================================
> Upgrade 1 Package
>
> Total download size: 27 k
> Downloading packages:
> No Presto metadata available for updates
> Public key for centos-release-7-9.2009.1.el7.centos.x86_64.rpm is not
> installed
> warning:
> /var/cache/yum/x86_64/7/updates/packages/centos-release-7-9.2009.1.el7.centos.x86_64.rpm:
> Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
> Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
> Importing GPG key 0xF4A80EB5:
> Userid : "CentOS-7 Key (CentOS 7 Official Signing Key) <
> security@centos.org>"
> Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5
> Package : centos-release-7-8.2003.0.el7.centos.x86_64 (@anaconda)
> From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
> Running transaction check
> Running transaction test
> Transaction test succeeded
> Running transaction
> Updating : centos-release-7-9.2009.1.el7.centos.x86_64
> 1/2
> Cleanup : centos-release-7-8.2003.0.el7.centos.x86_64
> 2/2
> Verifying : centos-release-7-9.2009.1.el7.centos.x86_64
> 1/2
> Verifying : centos-release-7-8.2003.0.el7.centos.x86_64
> 2/2
>
> Updated:
> centos-release.x86_64 0:7-9.2009.1.el7.centos
>
> Complete!
> Loaded plugins: fastestmirror
> Loading mirror speeds from cached hostfile
> * base: ftp.tc.edu.tw
> * extras: ftp.tc.edu.tw
> * updates: ftp.tc.edu.tw
> No package kernel-devel-3.10.0-1127.el7.x86_64 available.
> Error: Nothing to do
> Unmounting Virtualbox Guest Additions ISO from: /mnt
> umount: /mnt: not mounted
> ==> bigtop1: Checking for guest additions in VM...
> bigtop1: No guest additions were detected on the base box for this VM!
> Guest
> bigtop1: additions are required for forwarded ports, shared folders,
> host only
> bigtop1: networking, and more. If SSH fails on this machine, please
> install
> bigtop1: the guest additions and repackage the box to continue.
> bigtop1:
> bigtop1: This is not an error message; everything may continue to work
> properly,
> bigtop1: in which case you may ignore this message.
> The following SSH command responded with a non-zero exit status.
> Vagrant assumes that this means the command failed!
>
> umount /mnt
>
> Stdout from the command:
>
>
>
> Stderr from the command:
>
> umount: /mnt: not mounted
> ---
>
> Kengo Seki <se...@apache.org> 於 2020年12月13日 週日 上午8:37寫道：
>
> > Hi Evans,
> >
> > In my environment, using the latest official CentOS image instead of
> > puppetlabs' one seems to work, as follows.
> > So I don't think we have to drop the Vagrant provisioner in this
> > release and would like to propose to simply update default values in
> > vagrantconfig.yaml,
> > but I'm not also strongly against dropping it if you think we should
> > reduce maintenance cost on it in this opportunity. :)
> >
> > ```
> > ~/repos/bigtop/provisioner/vagrant$ git diff
> > diff --git a/provisioner/vagrant/vagrantconfig.yaml
> > b/provisioner/vagrant/vagrantconfig.yaml
> > index a25690d2..bee87293 100644
> > --- a/provisioner/vagrant/vagrantconfig.yaml
> > +++ b/provisioner/vagrant/vagrantconfig.yaml
> > @@ -15,8 +15,8 @@
> >
> > memory_size: 4096
> > number_cpus: 1
> > -box: "puppetlabs/centos-7.2-64-nocm"
> > -repo: "http://repos.bigtop.apache.org/releases/1.3.0/centos/7/x86_64"
> > +box: "centos/7"
> > +repo: "http://repos.bigtop.apache.org/releases/1.5.0/centos/7/x86_64"
> > num_instances: 1
> > distro: centos
> > components: [hdfs, yarn, mapreduce]
> > ~/repos/bigtop/provisioner/vagrant$ vagrant up
> >
> > (snip)
> >
> > ==> bigtop1: Notice: Roles to deploy: [resourcemanager, nodemanager,
> > mapred-app, hadoop-client, mapred-app, namenode, datanode]
> >
> > (snip)
> >
> > ==> bigtop1: Notice: Finished catalog run in 364.53 seconds
> > ==> bigtop1: Configuring cache buckets...
> > ~/repos/bigtop/provisioner/vagrant$ vagrant ssh
> >
> > (snip)
> >
> > [vagrant@bigtop1 ~]$ sudo jps
> > 3812 JobHistoryServer
> > 4212 NameNode
> > 5993 Jps
> > 3978 NodeManager
> > 3531 ResourceManager
> > 3438 WebAppProxyServer
> > 4447 DataNode
> > [vagrant@bigtop1 ~]$ hadoop version
> > Hadoop 2.10.1
> > Subversion https://gitbox.apache.org/repos/asf/bigtop.git -r
> > 81ad7b000cec0503b9a1d5521fdaf0129443b536
> > Compiled by jenkins on 2020-11-28T10:07Z
> > Compiled with protoc 2.5.0
> > From source with checksum 12fc83747448c6d239c8a4e93b4fa294
> > This command was run using /usr/lib/hadoop/hadoop-common-2.10.1.jar
> > [vagrant@bigtop1 ~]$ sudo -u yarn yarn jar
> > /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 1 1
> >
> > (snip)
> >
> > Job Finished in 29.655 seconds
> > Estimated value of Pi is 4.00000000000000000000
> > ```
> >
> > Kengo Seki <se...@apache.org>
> >
> > On Sun, Dec 13, 2020 at 12:40 AM Evans Ye <ev...@apache.org> wrote:
> > >
> > > I think having that malfunction feature removed is better to make 1.5
> > > release quality stay untainted. If go for RC1 the effort is small since
> > > all the build/repo/artifacts can stay AS-IS. Just the code need to be
> > > modified.
> > > However, you're right that it's for testing purpose so either way is ok.
> > >
> > > I don't think this worth to block the release though. I can still fire
> > up a
> > > JIRA for tracking.
> > >
> > > Luca Toscano <to...@gmail.com> 於 2020年12月12日 週六 下午7:53寫道：
> > >
> > > > Hi Evans,
> > > >
> > > > I don't know exactly what it is the typical use case for the Vagrant
> > > > provisioner, but I guess it should be mostly testing, so I think it
> > > > should be fine for the purposes of the 1.5 release to leave it aside
> > > > (and fix it later on).
> > > > Do you have more details about the systemd failure? Maybe we could
> > > > open a quick jira with details so people can chime in and check, to
> > > > speed up the release :)
> > > >
> > > > Luca
> > > >
> > > > On Sat, Dec 12, 2020 at 10:55 AM Evans Ye <ev...@apache.org> wrote:
> > > > >
> > > > > I'm still running tests but one thing need to discuss with folks
> > first.
> > > > >
> > > > > The provisioner/vagrant component is failing in my test.
> > > > >
> > > > > ==> bigtop1: Error: Could not start Service[hadoop-yarn-proxyserver]:
> > > > > Execution of '/bin/systemctl start hadoop-yarn-proxyserver' returned
> > 1:
> > > > Job
> > > > > for hadoop-yarn-proxyserver.service failed. See "systemctl status
> > > > > hadoop-yarn-proxyserver.service" and "journalctl -xe" for details.
> > > > >
> > > > > Seems it's due to the systemd changing. I searched on vagrant box
> > but I
> > > > see
> > > > > NO NEW box for centos/ubuntu from puppetlabs. Also, our CI test does
> > not
> > > > > cover this component so it's likely to have bug down the road.
> > Therefore,
> > > > > I'm proposing to remove the component in 1.5 release. How do the
> > folks
> > > > > think?
> > > > >
> > > > > Evans
> > > > >
> > > > >
> > > > > Kengo Seki <se...@apache.org> 於 2020年12月8日 週二 下午9:55寫道：
> > > > >
> > > > > > Luca and Evans,
> > > > > >
> > > > > > Thank you for testing the RC! Sure, I think we can wait for that
> > until
> > > > > > this weekend :)
> > > > > >
> > > > > > Kengo Seki <se...@apache.org>
> > > > > >
> > > > > > On Tue, Dec 8, 2020 at 6:24 PM Evans Ye <ev...@apache.org>
> > wrote:
> > > > > > >
> > > > > > > I'm really happy to see the release candidate is finally out.
> > > > > > > With this timing the community will close this year with a strong
> > > > result.
> > > > > > > Thank you all for the efforts!
> > > > > > > Let me find time this week to evaluate and vote.
> > > > > > >
> > > > > > > - Evans
> > > > > > >
> > > > > > > Luca Toscano <to...@gmail.com> 於 2020年12月8日 週二 下午4:47寫道：
> > > > > > >
> > > > > > > > Hi!
> > > > > > > >
> > > > > > > > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org>
> > > > wrote:
> > > > > > > > >
> > > > > > > > > This is the vote for release 1.5.0 of Apache Bigtop.
> > > > > > > > >
> > > > > > > > > It fixes the following issues:
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12345178
> > > > > > > > >
> > > > > > > > > The vote will be going for at least 72 hours and will be
> > closed
> > > > on
> > > > > > > > Wednesday,
> > > > > > > > > December 9th, 2020 at 5pm PST. Please download, test and vote
> > > > with
> > > > > > > >
> > > > > > > > This is a great news, really glad to see 1.5.0 finally coming
> > out!
> > > > I
> > > > > > > > might be able to try the debian-9 packages on a testing cluster
> > > > with
> > > > > > > > Bigtop 1.4 + Kerberos by the end of this week, but it would
> > delay a
> > > > > > > > bit the 72h. If this is ok, I'll keep this list posted with
> > result
> > > > (or
> > > > > > > > if I don't manage to test properly).
> > > > > > > >
> > > > > > > > Luca
> > > > > > > >
> > > > > >
> > > >
> >
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Evans Ye <ev...@apache.org>.
Mine doesn't work still. Probably the env issue since I'm away from the
field...
Anyhow. Updating the default value sounds good to me. We can address it in
later developments.
ERROR LOG:
---
$ vagrant destroy -f; vagrant up
/Users/eyeh/.vagrant.d/gems/2.4.4/gems/rubyhacks-0.1.5/lib/rubyhacks.rb:536:
warning: constant ::Fixnum is deprecated
vagrant conf local repo enabled: false
Adding output/ repo ? false
==> bigtop1: Forcing shutdown of VM...
==> bigtop1: Destroying VM and associated drives...
==> bigtop1: [vagrant-hostmanager:guests] Updating hosts file on active
guest virtual machines...
/Users/eyeh/.vagrant.d/gems/2.4.4/gems/rubyhacks-0.1.5/lib/rubyhacks.rb:536:
warning: constant ::Fixnum is deprecated
vagrant conf local repo enabled: false
Adding output/ repo ? false
Bringing machine 'bigtop1' up with 'virtualbox' provider...
==> bigtop1: Importing base box 'centos/7'...
==> bigtop1: Matching MAC address for NAT networking...
==> bigtop1: Checking if box 'centos/7' version '2004.01' is up to date...
==> bigtop1: Setting the name of the VM: vagrant_bigtop1_1607841501336_78663
==> bigtop1: Clearing any previously set network interfaces...
==> bigtop1: Preparing network interfaces based on configuration...
bigtop1: Adapter 1: nat
bigtop1: Adapter 2: hostonly
==> bigtop1: Forwarding ports...
bigtop1: 22 (guest) => 2222 (host) (adapter 1)
==> bigtop1: Running 'pre-boot' VM customizations...
==> bigtop1: Booting VM...
==> bigtop1: Waiting for machine to boot. This may take a few minutes...
bigtop1: SSH address: 127.0.0.1:2222
bigtop1: SSH username: vagrant
bigtop1: SSH auth method: private key
bigtop1:
bigtop1: Vagrant insecure key detected. Vagrant will automatically
replace
bigtop1: this with a newly generated keypair for better security.
bigtop1:
bigtop1: Inserting generated public key within guest...
bigtop1: Removing insecure key from the guest if it's present...
bigtop1: Key inserted! Disconnecting and reconnecting using new SSH
key...
==> bigtop1: Machine booted and ready!
[bigtop1] No Virtualbox Guest Additions installation found.
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: ftp.tc.edu.tw
* extras: ftp.tc.edu.tw
* updates: ftp.tc.edu.tw
Resolving Dependencies
--> Running transaction check
---> Package centos-release.x86_64 0:7-8.2003.0.el7.centos will be updated
---> Package centos-release.x86_64 0:7-9.2009.1.el7.centos will be an update
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository
Size
================================================================================
Updating:
centos-release x86_64 7-9.2009.1.el7.centos updates
27 k
Transaction Summary
================================================================================
Upgrade 1 Package
Total download size: 27 k
Downloading packages:
No Presto metadata available for updates
Public key for centos-release-7-9.2009.1.el7.centos.x86_64.rpm is not
installed
warning:
/var/cache/yum/x86_64/7/updates/packages/centos-release-7-9.2009.1.el7.centos.x86_64.rpm:
Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Importing GPG key 0xF4A80EB5:
Userid : "CentOS-7 Key (CentOS 7 Official Signing Key) <
security@centos.org>"
Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5
Package : centos-release-7-8.2003.0.el7.centos.x86_64 (@anaconda)
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : centos-release-7-9.2009.1.el7.centos.x86_64
1/2
Cleanup : centos-release-7-8.2003.0.el7.centos.x86_64
2/2
Verifying : centos-release-7-9.2009.1.el7.centos.x86_64
1/2
Verifying : centos-release-7-8.2003.0.el7.centos.x86_64
2/2
Updated:
centos-release.x86_64 0:7-9.2009.1.el7.centos
Complete!
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: ftp.tc.edu.tw
* extras: ftp.tc.edu.tw
* updates: ftp.tc.edu.tw
No package kernel-devel-3.10.0-1127.el7.x86_64 available.
Error: Nothing to do
Unmounting Virtualbox Guest Additions ISO from: /mnt
umount: /mnt: not mounted
==> bigtop1: Checking for guest additions in VM...
bigtop1: No guest additions were detected on the base box for this VM!
Guest
bigtop1: additions are required for forwarded ports, shared folders,
host only
bigtop1: networking, and more. If SSH fails on this machine, please
install
bigtop1: the guest additions and repackage the box to continue.
bigtop1:
bigtop1: This is not an error message; everything may continue to work
properly,
bigtop1: in which case you may ignore this message.
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!
umount /mnt
Stdout from the command:
Stderr from the command:
umount: /mnt: not mounted
---
Kengo Seki <se...@apache.org> 於 2020年12月13日 週日 上午8:37寫道：
> Hi Evans,
>
> In my environment, using the latest official CentOS image instead of
> puppetlabs' one seems to work, as follows.
> So I don't think we have to drop the Vagrant provisioner in this
> release and would like to propose to simply update default values in
> vagrantconfig.yaml,
> but I'm not also strongly against dropping it if you think we should
> reduce maintenance cost on it in this opportunity. :)
>
> ```
> ~/repos/bigtop/provisioner/vagrant$ git diff
> diff --git a/provisioner/vagrant/vagrantconfig.yaml
> b/provisioner/vagrant/vagrantconfig.yaml
> index a25690d2..bee87293 100644
> --- a/provisioner/vagrant/vagrantconfig.yaml
> +++ b/provisioner/vagrant/vagrantconfig.yaml
> @@ -15,8 +15,8 @@
>
> memory_size: 4096
> number_cpus: 1
> -box: "puppetlabs/centos-7.2-64-nocm"
> -repo: "http://repos.bigtop.apache.org/releases/1.3.0/centos/7/x86_64"
> +box: "centos/7"
> +repo: "http://repos.bigtop.apache.org/releases/1.5.0/centos/7/x86_64"
> num_instances: 1
> distro: centos
> components: [hdfs, yarn, mapreduce]
> ~/repos/bigtop/provisioner/vagrant$ vagrant up
>
> (snip)
>
> ==> bigtop1: Notice: Roles to deploy: [resourcemanager, nodemanager,
> mapred-app, hadoop-client, mapred-app, namenode, datanode]
>
> (snip)
>
> ==> bigtop1: Notice: Finished catalog run in 364.53 seconds
> ==> bigtop1: Configuring cache buckets...
> ~/repos/bigtop/provisioner/vagrant$ vagrant ssh
>
> (snip)
>
> [vagrant@bigtop1 ~]$ sudo jps
> 3812 JobHistoryServer
> 4212 NameNode
> 5993 Jps
> 3978 NodeManager
> 3531 ResourceManager
> 3438 WebAppProxyServer
> 4447 DataNode
> [vagrant@bigtop1 ~]$ hadoop version
> Hadoop 2.10.1
> Subversion https://gitbox.apache.org/repos/asf/bigtop.git -r
> 81ad7b000cec0503b9a1d5521fdaf0129443b536
> Compiled by jenkins on 2020-11-28T10:07Z
> Compiled with protoc 2.5.0
> From source with checksum 12fc83747448c6d239c8a4e93b4fa294
> This command was run using /usr/lib/hadoop/hadoop-common-2.10.1.jar
> [vagrant@bigtop1 ~]$ sudo -u yarn yarn jar
> /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 1 1
>
> (snip)
>
> Job Finished in 29.655 seconds
> Estimated value of Pi is 4.00000000000000000000
> ```
>
> Kengo Seki <se...@apache.org>
>
> On Sun, Dec 13, 2020 at 12:40 AM Evans Ye <ev...@apache.org> wrote:
> >
> > I think having that malfunction feature removed is better to make 1.5
> > release quality stay untainted. If go for RC1 the effort is small since
> > all the build/repo/artifacts can stay AS-IS. Just the code need to be
> > modified.
> > However, you're right that it's for testing purpose so either way is ok.
> >
> > I don't think this worth to block the release though. I can still fire
> up a
> > JIRA for tracking.
> >
> > Luca Toscano <to...@gmail.com> 於 2020年12月12日 週六 下午7:53寫道：
> >
> > > Hi Evans,
> > >
> > > I don't know exactly what it is the typical use case for the Vagrant
> > > provisioner, but I guess it should be mostly testing, so I think it
> > > should be fine for the purposes of the 1.5 release to leave it aside
> > > (and fix it later on).
> > > Do you have more details about the systemd failure? Maybe we could
> > > open a quick jira with details so people can chime in and check, to
> > > speed up the release :)
> > >
> > > Luca
> > >
> > > On Sat, Dec 12, 2020 at 10:55 AM Evans Ye <ev...@apache.org> wrote:
> > > >
> > > > I'm still running tests but one thing need to discuss with folks
> first.
> > > >
> > > > The provisioner/vagrant component is failing in my test.
> > > >
> > > > ==> bigtop1: Error: Could not start Service[hadoop-yarn-proxyserver]:
> > > > Execution of '/bin/systemctl start hadoop-yarn-proxyserver' returned
> 1:
> > > Job
> > > > for hadoop-yarn-proxyserver.service failed. See "systemctl status
> > > > hadoop-yarn-proxyserver.service" and "journalctl -xe" for details.
> > > >
> > > > Seems it's due to the systemd changing. I searched on vagrant box
> but I
> > > see
> > > > NO NEW box for centos/ubuntu from puppetlabs. Also, our CI test does
> not
> > > > cover this component so it's likely to have bug down the road.
> Therefore,
> > > > I'm proposing to remove the component in 1.5 release. How do the
> folks
> > > > think?
> > > >
> > > > Evans
> > > >
> > > >
> > > > Kengo Seki <se...@apache.org> 於 2020年12月8日 週二 下午9:55寫道：
> > > >
> > > > > Luca and Evans,
> > > > >
> > > > > Thank you for testing the RC! Sure, I think we can wait for that
> until
> > > > > this weekend :)
> > > > >
> > > > > Kengo Seki <se...@apache.org>
> > > > >
> > > > > On Tue, Dec 8, 2020 at 6:24 PM Evans Ye <ev...@apache.org>
> wrote:
> > > > > >
> > > > > > I'm really happy to see the release candidate is finally out.
> > > > > > With this timing the community will close this year with a strong
> > > result.
> > > > > > Thank you all for the efforts!
> > > > > > Let me find time this week to evaluate and vote.
> > > > > >
> > > > > > - Evans
> > > > > >
> > > > > > Luca Toscano <to...@gmail.com> 於 2020年12月8日 週二 下午4:47寫道：
> > > > > >
> > > > > > > Hi!
> > > > > > >
> > > > > > > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org>
> > > wrote:
> > > > > > > >
> > > > > > > > This is the vote for release 1.5.0 of Apache Bigtop.
> > > > > > > >
> > > > > > > > It fixes the following issues:
> > > > > > > >
> > > > > > >
> > > > >
> > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12345178
> > > > > > > >
> > > > > > > > The vote will be going for at least 72 hours and will be
> closed
> > > on
> > > > > > > Wednesday,
> > > > > > > > December 9th, 2020 at 5pm PST. Please download, test and vote
> > > with
> > > > > > >
> > > > > > > This is a great news, really glad to see 1.5.0 finally coming
> out!
> > > I
> > > > > > > might be able to try the debian-9 packages on a testing cluster
> > > with
> > > > > > > Bigtop 1.4 + Kerberos by the end of this week, but it would
> delay a
> > > > > > > bit the 72h. If this is ok, I'll keep this list posted with
> result
> > > (or
> > > > > > > if I don't manage to test properly).
> > > > > > >
> > > > > > > Luca
> > > > > > >
> > > > >
> > >
>
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Kengo Seki <se...@apache.org>.
Hi Evans,
In my environment, using the latest official CentOS image instead of
puppetlabs' one seems to work, as follows.
So I don't think we have to drop the Vagrant provisioner in this
release and would like to propose to simply update default values in
vagrantconfig.yaml,
but I'm not also strongly against dropping it if you think we should
reduce maintenance cost on it in this opportunity. :)
```
~/repos/bigtop/provisioner/vagrant$ git diff
diff --git a/provisioner/vagrant/vagrantconfig.yaml
b/provisioner/vagrant/vagrantconfig.yaml
index a25690d2..bee87293 100644
--- a/provisioner/vagrant/vagrantconfig.yaml
+++ b/provisioner/vagrant/vagrantconfig.yaml
@@ -15,8 +15,8 @@
memory_size: 4096
number_cpus: 1
-box: "puppetlabs/centos-7.2-64-nocm"
-repo: "http://repos.bigtop.apache.org/releases/1.3.0/centos/7/x86_64"
+box: "centos/7"
+repo: "http://repos.bigtop.apache.org/releases/1.5.0/centos/7/x86_64"
num_instances: 1
distro: centos
components: [hdfs, yarn, mapreduce]
~/repos/bigtop/provisioner/vagrant$ vagrant up
(snip)
==> bigtop1: Notice: Roles to deploy: [resourcemanager, nodemanager,
mapred-app, hadoop-client, mapred-app, namenode, datanode]
(snip)
==> bigtop1: Notice: Finished catalog run in 364.53 seconds
==> bigtop1: Configuring cache buckets...
~/repos/bigtop/provisioner/vagrant$ vagrant ssh
(snip)
[vagrant@bigtop1 ~]$ sudo jps
3812 JobHistoryServer
4212 NameNode
5993 Jps
3978 NodeManager
3531 ResourceManager
3438 WebAppProxyServer
4447 DataNode
[vagrant@bigtop1 ~]$ hadoop version
Hadoop 2.10.1
Subversion https://gitbox.apache.org/repos/asf/bigtop.git -r
81ad7b000cec0503b9a1d5521fdaf0129443b536
Compiled by jenkins on 2020-11-28T10:07Z
Compiled with protoc 2.5.0
From source with checksum 12fc83747448c6d239c8a4e93b4fa294
This command was run using /usr/lib/hadoop/hadoop-common-2.10.1.jar
[vagrant@bigtop1 ~]$ sudo -u yarn yarn jar
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 1 1
(snip)
Job Finished in 29.655 seconds
Estimated value of Pi is 4.00000000000000000000
```
Kengo Seki <se...@apache.org>
On Sun, Dec 13, 2020 at 12:40 AM Evans Ye <ev...@apache.org> wrote:
>
> I think having that malfunction feature removed is better to make 1.5
> release quality stay untainted. If go for RC1 the effort is small since
> all the build/repo/artifacts can stay AS-IS. Just the code need to be
> modified.
> However, you're right that it's for testing purpose so either way is ok.
>
> I don't think this worth to block the release though. I can still fire up a
> JIRA for tracking.
>
> Luca Toscano <to...@gmail.com> 於 2020年12月12日 週六 下午7:53寫道：
>
> > Hi Evans,
> >
> > I don't know exactly what it is the typical use case for the Vagrant
> > provisioner, but I guess it should be mostly testing, so I think it
> > should be fine for the purposes of the 1.5 release to leave it aside
> > (and fix it later on).
> > Do you have more details about the systemd failure? Maybe we could
> > open a quick jira with details so people can chime in and check, to
> > speed up the release :)
> >
> > Luca
> >
> > On Sat, Dec 12, 2020 at 10:55 AM Evans Ye <ev...@apache.org> wrote:
> > >
> > > I'm still running tests but one thing need to discuss with folks first.
> > >
> > > The provisioner/vagrant component is failing in my test.
> > >
> > > ==> bigtop1: Error: Could not start Service[hadoop-yarn-proxyserver]:
> > > Execution of '/bin/systemctl start hadoop-yarn-proxyserver' returned 1:
> > Job
> > > for hadoop-yarn-proxyserver.service failed. See "systemctl status
> > > hadoop-yarn-proxyserver.service" and "journalctl -xe" for details.
> > >
> > > Seems it's due to the systemd changing. I searched on vagrant box but I
> > see
> > > NO NEW box for centos/ubuntu from puppetlabs. Also, our CI test does not
> > > cover this component so it's likely to have bug down the road. Therefore,
> > > I'm proposing to remove the component in 1.5 release. How do the folks
> > > think?
> > >
> > > Evans
> > >
> > >
> > > Kengo Seki <se...@apache.org> 於 2020年12月8日 週二 下午9:55寫道：
> > >
> > > > Luca and Evans,
> > > >
> > > > Thank you for testing the RC! Sure, I think we can wait for that until
> > > > this weekend :)
> > > >
> > > > Kengo Seki <se...@apache.org>
> > > >
> > > > On Tue, Dec 8, 2020 at 6:24 PM Evans Ye <ev...@apache.org> wrote:
> > > > >
> > > > > I'm really happy to see the release candidate is finally out.
> > > > > With this timing the community will close this year with a strong
> > result.
> > > > > Thank you all for the efforts!
> > > > > Let me find time this week to evaluate and vote.
> > > > >
> > > > > - Evans
> > > > >
> > > > > Luca Toscano <to...@gmail.com> 於 2020年12月8日 週二 下午4:47寫道：
> > > > >
> > > > > > Hi!
> > > > > >
> > > > > > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org>
> > wrote:
> > > > > > >
> > > > > > > This is the vote for release 1.5.0 of Apache Bigtop.
> > > > > > >
> > > > > > > It fixes the following issues:
> > > > > > >
> > > > > >
> > > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12345178
> > > > > > >
> > > > > > > The vote will be going for at least 72 hours and will be closed
> > on
> > > > > > Wednesday,
> > > > > > > December 9th, 2020 at 5pm PST. Please download, test and vote
> > with
> > > > > >
> > > > > > This is a great news, really glad to see 1.5.0 finally coming out!
> > I
> > > > > > might be able to try the debian-9 packages on a testing cluster
> > with
> > > > > > Bigtop 1.4 + Kerberos by the end of this week, but it would delay a
> > > > > > bit the 72h. If this is ok, I'll keep this list posted with result
> > (or
> > > > > > if I don't manage to test properly).
> > > > > >
> > > > > > Luca
> > > > > >
> > > >
> >
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Evans Ye <ev...@apache.org>.
I think having that malfunction feature removed is better to make 1.5
release quality stay untainted. If go for RC1 the effort is small since
all the build/repo/artifacts can stay AS-IS. Just the code need to be
modified.
However, you're right that it's for testing purpose so either way is ok.
I don't think this worth to block the release though. I can still fire up a
JIRA for tracking.
Luca Toscano <to...@gmail.com> 於 2020年12月12日 週六 下午7:53寫道：
> Hi Evans,
>
> I don't know exactly what it is the typical use case for the Vagrant
> provisioner, but I guess it should be mostly testing, so I think it
> should be fine for the purposes of the 1.5 release to leave it aside
> (and fix it later on).
> Do you have more details about the systemd failure? Maybe we could
> open a quick jira with details so people can chime in and check, to
> speed up the release :)
>
> Luca
>
> On Sat, Dec 12, 2020 at 10:55 AM Evans Ye <ev...@apache.org> wrote:
> >
> > I'm still running tests but one thing need to discuss with folks first.
> >
> > The provisioner/vagrant component is failing in my test.
> >
> > ==> bigtop1: Error: Could not start Service[hadoop-yarn-proxyserver]:
> > Execution of '/bin/systemctl start hadoop-yarn-proxyserver' returned 1:
> Job
> > for hadoop-yarn-proxyserver.service failed. See "systemctl status
> > hadoop-yarn-proxyserver.service" and "journalctl -xe" for details.
> >
> > Seems it's due to the systemd changing. I searched on vagrant box but I
> see
> > NO NEW box for centos/ubuntu from puppetlabs. Also, our CI test does not
> > cover this component so it's likely to have bug down the road. Therefore,
> > I'm proposing to remove the component in 1.5 release. How do the folks
> > think?
> >
> > Evans
> >
> >
> > Kengo Seki <se...@apache.org> 於 2020年12月8日 週二 下午9:55寫道：
> >
> > > Luca and Evans,
> > >
> > > Thank you for testing the RC! Sure, I think we can wait for that until
> > > this weekend :)
> > >
> > > Kengo Seki <se...@apache.org>
> > >
> > > On Tue, Dec 8, 2020 at 6:24 PM Evans Ye <ev...@apache.org> wrote:
> > > >
> > > > I'm really happy to see the release candidate is finally out.
> > > > With this timing the community will close this year with a strong
> result.
> > > > Thank you all for the efforts!
> > > > Let me find time this week to evaluate and vote.
> > > >
> > > > - Evans
> > > >
> > > > Luca Toscano <to...@gmail.com> 於 2020年12月8日 週二 下午4:47寫道：
> > > >
> > > > > Hi!
> > > > >
> > > > > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org>
> wrote:
> > > > > >
> > > > > > This is the vote for release 1.5.0 of Apache Bigtop.
> > > > > >
> > > > > > It fixes the following issues:
> > > > > >
> > > > >
> > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12345178
> > > > > >
> > > > > > The vote will be going for at least 72 hours and will be closed
> on
> > > > > Wednesday,
> > > > > > December 9th, 2020 at 5pm PST. Please download, test and vote
> with
> > > > >
> > > > > This is a great news, really glad to see 1.5.0 finally coming out!
> I
> > > > > might be able to try the debian-9 packages on a testing cluster
> with
> > > > > Bigtop 1.4 + Kerberos by the end of this week, but it would delay a
> > > > > bit the 72h. If this is ok, I'll keep this list posted with result
> (or
> > > > > if I don't manage to test properly).
> > > > >
> > > > > Luca
> > > > >
> > >
>
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Luca Toscano <to...@gmail.com>.
Hi Evans,
I don't know exactly what it is the typical use case for the Vagrant
provisioner, but I guess it should be mostly testing, so I think it
should be fine for the purposes of the 1.5 release to leave it aside
(and fix it later on).
Do you have more details about the systemd failure? Maybe we could
open a quick jira with details so people can chime in and check, to
speed up the release :)
Luca
On Sat, Dec 12, 2020 at 10:55 AM Evans Ye <ev...@apache.org> wrote:
>
> I'm still running tests but one thing need to discuss with folks first.
>
> The provisioner/vagrant component is failing in my test.
>
> ==> bigtop1: Error: Could not start Service[hadoop-yarn-proxyserver]:
> Execution of '/bin/systemctl start hadoop-yarn-proxyserver' returned 1: Job
> for hadoop-yarn-proxyserver.service failed. See "systemctl status
> hadoop-yarn-proxyserver.service" and "journalctl -xe" for details.
>
> Seems it's due to the systemd changing. I searched on vagrant box but I see
> NO NEW box for centos/ubuntu from puppetlabs. Also, our CI test does not
> cover this component so it's likely to have bug down the road. Therefore,
> I'm proposing to remove the component in 1.5 release. How do the folks
> think?
>
> Evans
>
>
> Kengo Seki <se...@apache.org> 於 2020年12月8日 週二 下午9:55寫道：
>
> > Luca and Evans,
> >
> > Thank you for testing the RC! Sure, I think we can wait for that until
> > this weekend :)
> >
> > Kengo Seki <se...@apache.org>
> >
> > On Tue, Dec 8, 2020 at 6:24 PM Evans Ye <ev...@apache.org> wrote:
> > >
> > > I'm really happy to see the release candidate is finally out.
> > > With this timing the community will close this year with a strong result.
> > > Thank you all for the efforts!
> > > Let me find time this week to evaluate and vote.
> > >
> > > - Evans
> > >
> > > Luca Toscano <to...@gmail.com> 於 2020年12月8日 週二 下午4:47寫道：
> > >
> > > > Hi!
> > > >
> > > > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org> wrote:
> > > > >
> > > > > This is the vote for release 1.5.0 of Apache Bigtop.
> > > > >
> > > > > It fixes the following issues:
> > > > >
> > > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12345178
> > > > >
> > > > > The vote will be going for at least 72 hours and will be closed on
> > > > Wednesday,
> > > > > December 9th, 2020 at 5pm PST. Please download, test and vote with
> > > >
> > > > This is a great news, really glad to see 1.5.0 finally coming out! I
> > > > might be able to try the debian-9 packages on a testing cluster with
> > > > Bigtop 1.4 + Kerberos by the end of this week, but it would delay a
> > > > bit the 72h. If this is ok, I'll keep this list posted with result (or
> > > > if I don't manage to test properly).
> > > >
> > > > Luca
> > > >
> >
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Evans Ye <ev...@apache.org>.
I'm still running tests but one thing need to discuss with folks first.
The provisioner/vagrant component is failing in my test.
==> bigtop1: Error: Could not start Service[hadoop-yarn-proxyserver]:
Execution of '/bin/systemctl start hadoop-yarn-proxyserver' returned 1: Job
for hadoop-yarn-proxyserver.service failed. See "systemctl status
hadoop-yarn-proxyserver.service" and "journalctl -xe" for details.
Seems it's due to the systemd changing. I searched on vagrant box but I see
NO NEW box for centos/ubuntu from puppetlabs. Also, our CI test does not
cover this component so it's likely to have bug down the road. Therefore,
I'm proposing to remove the component in 1.5 release. How do the folks
think?
Evans
Kengo Seki <se...@apache.org> 於 2020年12月8日 週二 下午9:55寫道：
> Luca and Evans,
>
> Thank you for testing the RC! Sure, I think we can wait for that until
> this weekend :)
>
> Kengo Seki <se...@apache.org>
>
> On Tue, Dec 8, 2020 at 6:24 PM Evans Ye <ev...@apache.org> wrote:
> >
> > I'm really happy to see the release candidate is finally out.
> > With this timing the community will close this year with a strong result.
> > Thank you all for the efforts!
> > Let me find time this week to evaluate and vote.
> >
> > - Evans
> >
> > Luca Toscano <to...@gmail.com> 於 2020年12月8日 週二 下午4:47寫道：
> >
> > > Hi!
> > >
> > > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org> wrote:
> > > >
> > > > This is the vote for release 1.5.0 of Apache Bigtop.
> > > >
> > > > It fixes the following issues:
> > > >
> > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12345178
> > > >
> > > > The vote will be going for at least 72 hours and will be closed on
> > > Wednesday,
> > > > December 9th, 2020 at 5pm PST. Please download, test and vote with
> > >
> > > This is a great news, really glad to see 1.5.0 finally coming out! I
> > > might be able to try the debian-9 packages on a testing cluster with
> > > Bigtop 1.4 + Kerberos by the end of this week, but it would delay a
> > > bit the 72h. If this is ok, I'll keep this list posted with result (or
> > > if I don't manage to test properly).
> > >
> > > Luca
> > >
>
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Kengo Seki <se...@apache.org>.
Luca and Evans,
Thank you for testing the RC! Sure, I think we can wait for that until
this weekend :)
Kengo Seki <se...@apache.org>
On Tue, Dec 8, 2020 at 6:24 PM Evans Ye <ev...@apache.org> wrote:
>
> I'm really happy to see the release candidate is finally out.
> With this timing the community will close this year with a strong result.
> Thank you all for the efforts!
> Let me find time this week to evaluate and vote.
>
> - Evans
>
> Luca Toscano <to...@gmail.com> 於 2020年12月8日 週二 下午4:47寫道：
>
> > Hi!
> >
> > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org> wrote:
> > >
> > > This is the vote for release 1.5.0 of Apache Bigtop.
> > >
> > > It fixes the following issues:
> > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12345178
> > >
> > > The vote will be going for at least 72 hours and will be closed on
> > Wednesday,
> > > December 9th, 2020 at 5pm PST. Please download, test and vote with
> >
> > This is a great news, really glad to see 1.5.0 finally coming out! I
> > might be able to try the debian-9 packages on a testing cluster with
> > Bigtop 1.4 + Kerberos by the end of this week, but it would delay a
> > bit the 72h. If this is ok, I'll keep this list posted with result (or
> > if I don't manage to test properly).
> >
> > Luca
> >
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Evans Ye <ev...@apache.org>.
I'm really happy to see the release candidate is finally out.
With this timing the community will close this year with a strong result.
Thank you all for the efforts!
Let me find time this week to evaluate and vote.
- Evans
Luca Toscano <to...@gmail.com> 於 2020年12月8日 週二 下午4:47寫道：
> Hi!
>
> On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org> wrote:
> >
> > This is the vote for release 1.5.0 of Apache Bigtop.
> >
> > It fixes the following issues:
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12345178
> >
> > The vote will be going for at least 72 hours and will be closed on
> Wednesday,
> > December 9th, 2020 at 5pm PST. Please download, test and vote with
>
> This is a great news, really glad to see 1.5.0 finally coming out! I
> might be able to try the debian-9 packages on a testing cluster with
> Bigtop 1.4 + Kerberos by the end of this week, but it would delay a
> bit the 72h. If this is ok, I'll keep this list posted with result (or
> if I don't manage to test properly).
>
> Luca
>
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Luca Toscano <to...@gmail.com>.
Hi!
On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org> wrote:
>
> This is the vote for release 1.5.0 of Apache Bigtop.
>
> It fixes the following issues:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12345178
>
> The vote will be going for at least 72 hours and will be closed on Wednesday,
> December 9th, 2020 at 5pm PST. Please download, test and vote with
This is a great news, really glad to see 1.5.0 finally coming out! I
might be able to try the debian-9 packages on a testing cluster with
Bigtop 1.4 + Kerberos by the end of this week, but it would delay a
bit the 72h. If this is ok, I'll keep this list posted with result (or
if I don't manage to test properly).
Luca
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Masatake Iwasaki <iw...@oss.nttdata.co.jp>.
Thanks for putting this up, Kengo.
I'm +1(non-binding).
* verified signature and checksums
* verified products and platform important for me on CentOS 7
* built RPMs of bigtop-*, zookeeper, hadoop, hive, hbase
* ran smoke tests of hadoop, hive, hbase on 3-node cluster provisioned by docker-hadoop.sh
Masatake Iwasaki
On 2020/12/07 8:46, Kengo Seki wrote:
> This is the vote for release 1.5.0 of Apache Bigtop.
>
> It fixes the following issues:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12345178
>
> The vote will be going for at least 72 hours and will be closed on Wednesday,
> December 9th, 2020 at 5pm PST. Please download, test and vote with
>
> [ ] +1, accept RC0 as the official 1.5.0 release of Apache Bigtop
> [ ] +0, I don't care either way,
> [ ] -1, do not accept RC0 as the official 1.5.0 release of Apache
> Bigtop, because...
>
> Source and binary files:
> https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.5.0-RC0/
>
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachebigtop-1030/
>
> The git tag to be voted upon is release-1.5.0
>
> Bigtop's KEYS file containing PGP keys we use to sign the release:
> https://dist.apache.org/repos/dist/release/bigtop/KEYS
>
> For your reference, the CI result [1] is as follows. It's also
> summarized in the cwiki [2].
>
> - All components (except for Phoenix, which doesn't have a puppet
> manifest and smoke test)
> are successfully deployed onto all distros/platforms.
>
> - All smoke tests except for the following combinations succeeded on CI.
> The followings only partially succeeded on CI, but I confirmed that
> all of them passed on my local machine and AWS m6g (arm64) instance.
>
> - x86_64
>
> - QFS on Ubuntu 16.04
>
> - aarch64:
>
> - HBase on Fedora 31
> - Kibana on all distros
> - Livy on all distros except for CentOS 8
> - Oozie on CentOS 8
> - QFS on Ubuntu 16.04
>
> [1]: https://ci.bigtop.apache.org/view/Test/job/Bigtop-1.5.0-smoke-tests/
> [2]: https://cwiki.apache.org/confluence/display/BIGTOP/Overview+of+Bigtop+1.5.0+Support+Matrix
>
> Kengo Seki <se...@apache.org>
>
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Kengo Seki <se...@apache.org>.
Thank you for elaborating and filing follow-up issues, Yuqi!
Kengo Seki <se...@apache.org>
On Mon, Dec 7, 2020 at 11:57 AM Yuqi Gu <yu...@linaro.org> wrote:
>
> Thanks, Kengo Seki.
>
> As discussed in BIGTOP-3428, let's defer the issues of smoke tests after
> the 1.5.0 release.
> (BIGTOP-3428, BIGTOP-3463 and BIGTOP-3464 were created to track the issues
> respectively.)
>
> I'm +1 to this.
>
> BRs,
> Yuqi
>
>
>
> On Mon, 7 Dec 2020 at 07:47, Kengo Seki <se...@apache.org> wrote:
>
> > This is the vote for release 1.5.0 of Apache Bigtop.
> >
> > It fixes the following issues:
> >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12345178
> >
> > The vote will be going for at least 72 hours and will be closed on
> > Wednesday,
> > December 9th, 2020 at 5pm PST. Please download, test and vote with
> >
> > [ ] +1, accept RC0 as the official 1.5.0 release of Apache Bigtop
> > [ ] +0, I don't care either way,
> > [ ] -1, do not accept RC0 as the official 1.5.0 release of Apache
> > Bigtop, because...
> >
> > Source and binary files:
> > https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.5.0-RC0/
> >
> > Maven staging repo:
> >
> > https://repository.apache.org/content/repositories/orgapachebigtop-1030/
> >
> > The git tag to be voted upon is release-1.5.0
> >
> > Bigtop's KEYS file containing PGP keys we use to sign the release:
> > https://dist.apache.org/repos/dist/release/bigtop/KEYS
> >
> > For your reference, the CI result [1] is as follows. It's also
> > summarized in the cwiki [2].
> >
> > - All components (except for Phoenix, which doesn't have a puppet
> > manifest and smoke test)
> > are successfully deployed onto all distros/platforms.
> >
> > - All smoke tests except for the following combinations succeeded on CI.
> > The followings only partially succeeded on CI, but I confirmed that
> > all of them passed on my local machine and AWS m6g (arm64) instance.
> >
> > - x86_64
> >
> > - QFS on Ubuntu 16.04
> >
> > - aarch64:
> >
> > - HBase on Fedora 31
> > - Kibana on all distros
> > - Livy on all distros except for CentOS 8
> > - Oozie on CentOS 8
> > - QFS on Ubuntu 16.04
> >
> > [1]: https://ci.bigtop.apache.org/view/Test/job/Bigtop-1.5.0-smoke-tests/
> > [2]:
> > https://cwiki.apache.org/confluence/display/BIGTOP/Overview+of+Bigtop+1.5.0+Support+Matrix
> >
> > Kengo Seki <se...@apache.org>
> >
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Yuqi Gu <yu...@linaro.org>.
Thanks, Kengo Seki.
As discussed in BIGTOP-3428, let's defer the issues of smoke tests after
the 1.5.0 release.
(BIGTOP-3428, BIGTOP-3463 and BIGTOP-3464 were created to track the issues
respectively.)
I'm +1 to this.
BRs,
Yuqi
On Mon, 7 Dec 2020 at 07:47, Kengo Seki <se...@apache.org> wrote:
> This is the vote for release 1.5.0 of Apache Bigtop.
>
> It fixes the following issues:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311420&version=12345178
>
> The vote will be going for at least 72 hours and will be closed on
> Wednesday,
> December 9th, 2020 at 5pm PST. Please download, test and vote with
>
> [ ] +1, accept RC0 as the official 1.5.0 release of Apache Bigtop
> [ ] +0, I don't care either way,
> [ ] -1, do not accept RC0 as the official 1.5.0 release of Apache
> Bigtop, because...
>
> Source and binary files:
> https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.5.0-RC0/
>
> Maven staging repo:
>
> https://repository.apache.org/content/repositories/orgapachebigtop-1030/
>
> The git tag to be voted upon is release-1.5.0
>
> Bigtop's KEYS file containing PGP keys we use to sign the release:
> https://dist.apache.org/repos/dist/release/bigtop/KEYS
>
> For your reference, the CI result [1] is as follows. It's also
> summarized in the cwiki [2].
>
> - All components (except for Phoenix, which doesn't have a puppet
> manifest and smoke test)
> are successfully deployed onto all distros/platforms.
>
> - All smoke tests except for the following combinations succeeded on CI.
> The followings only partially succeeded on CI, but I confirmed that
> all of them passed on my local machine and AWS m6g (arm64) instance.
>
> - x86_64
>
> - QFS on Ubuntu 16.04
>
> - aarch64:
>
> - HBase on Fedora 31
> - Kibana on all distros
> - Livy on all distros except for CentOS 8
> - Oozie on CentOS 8
> - QFS on Ubuntu 16.04
>
> [1]: https://ci.bigtop.apache.org/view/Test/job/Bigtop-1.5.0-smoke-tests/
> [2]:
> https://cwiki.apache.org/confluence/display/BIGTOP/Overview+of+Bigtop+1.5.0+Support+Matrix
>
> Kengo Seki <se...@apache.org>
>
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Jun HE <ju...@apache.org>.
Verified following items on Arm64:
* Verified bigtop-puppet and bigtop-slaves images build
* Manually run deployment of hadoop, hbase, kafka, spark, zookeeper, solr,
flink, elasticsearch on Ubuntu-18.04/Debian-10
* Run smoke tests of hadoop, hbase, kafka, spark, zookeeper, solr, flink,
elasticsearch using ./docker-hadoop.sh on Ubuntu-18.04/Debian-10
I'm +1(non-binding) to accept RC0 as 1.5.0 release.
Thanks a lot for Kengo leading this release and all folks contributed to it!
Regards,
Jun
Evans Ye <ev...@apache.org> 于2020年12月14日周一 下午1:29写道：
> To unlock the release. I'm +1(binding) to accept RC0 as 1.5.0 release:
>
> - Checked gpg signature, sha512 sum, sha256 sum
> - build
> - Manual test zookeeper-pkg-ind (with default bigtop-slave images)
> - provision
> - Manual test docker provisioner with default centos-7 and debian-10
> config (with bigtop-puppet images)
>
> Since build, provision, and smoke test features are guarded by our CI
> already. I think just running through the checking as an end user is
> sufficient.
>
> Evans
>
> Kengo Seki <se...@apache.org> 於 2020年12月14日 週一 上午9:28寫道：
>
> > Luca,
> >
> > Thank you for the detailed checks!
> > That's really helpful and informative, so I added some of your
> > findings to the cwiki.
> >
> >
> https://cwiki.apache.org/confluence/display/BIGTOP/Overview+of+Bigtop+1.5.0+Support+Matrix
> >
> > Kengo Seki <se...@apache.org>
> >
> > On Sun, Dec 13, 2020 at 5:40 PM Luca Toscano <to...@gmail.com>
> > wrote:
> > >
> > > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org> wrote:
> > > >
> > > > This is the vote for release 1.5.0 of Apache Bigtop.
> > > >
> > > > [x] +1, accept RC0 as the official 1.5.0 release of Apache Bigtop
> > >
> > > (Community vote)
> > >
> > > I completed my tests, the new release didn't raise any blocking issue.
> > > My tests were for this use case:
> > > - Hadoop test cluster with Hive/Oozie/Spark/Jupyter/Hue on bare metal
> > hosts.
> > > - HDFS and Yarn HA setup (including journal nodes).
> > > - Bigtop 1.4 for Debian 9 for the majority of the nodes, and a client
> > > node on Debian 10.
> > > - In place upgrade from Bigtop 1.4 to 1.5 (upgrading the packages
> > > without removing the old ones first).
> > >
> > > I have tested an in place upgrade, trying to rollback/re-deploy before
> > > the HDFS finalize step, all good. Some interesting things that I
> > > noticed or had to fix in my config after the upgrade:
> > > - the HDFS Namenodes seem to be smarter when dealing with a rollback.
> > > While testing CDH -> Bigtop 1.4 (hdfs 2.6.0 -> 2.8.5) I had to follow
> > > the upstream guidelines and explicitly issue commands to rollback the
> > > hdfs state, but when testing this release it was not needed anymore.
> > > I'll dig a big more into this, but overall the upgrade/rollback
> > > procedure seemed smoother.
> > > - our Hadoop clusters never used the
> > > "yarn.resourcemanager.webapp.address.$rm-id" settings in yarn-site.xml
> > > (that seem to have a good default), but on Bigtop 1.5 it lead to NPE
> > > while running map-reduce jobs, like described in
> > > https://issues.apache.org/jira/browse/YARN-8056. Once I set the value,
> > > everything worked as expected.
> > > - while upgrading the packages, I had a little hiccup with oozie since
> > > the version on Bigtop 1.4 is higher than 1.5 (4.3.0-3 vs 4.3.0-1), so
> > > it needed a downgrade. Nothing really major, just mentioning it.
> > >
> > > About https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.5.0-RC0:
> > > - verified .asc/.shaXXX, all good.
> > > - all debian packages (9 and 10 versions) worked fine, and Kengo
> > > Seki's gpg signature was ok as well.
> > >
> > > Thanks a lot for this release!
> > >
> > > Luca
> >
>
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Evans Ye <ev...@apache.org>.
To unlock the release. I'm +1(binding) to accept RC0 as 1.5.0 release:
- Checked gpg signature, sha512 sum, sha256 sum
- build
- Manual test zookeeper-pkg-ind (with default bigtop-slave images)
- provision
- Manual test docker provisioner with default centos-7 and debian-10
config (with bigtop-puppet images)
Since build, provision, and smoke test features are guarded by our CI
already. I think just running through the checking as an end user is
sufficient.
Evans
Kengo Seki <se...@apache.org> 於 2020年12月14日 週一 上午9:28寫道：
> Luca,
>
> Thank you for the detailed checks!
> That's really helpful and informative, so I added some of your
> findings to the cwiki.
>
> https://cwiki.apache.org/confluence/display/BIGTOP/Overview+of+Bigtop+1.5.0+Support+Matrix
>
> Kengo Seki <se...@apache.org>
>
> On Sun, Dec 13, 2020 at 5:40 PM Luca Toscano <to...@gmail.com>
> wrote:
> >
> > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org> wrote:
> > >
> > > This is the vote for release 1.5.0 of Apache Bigtop.
> > >
> > > [x] +1, accept RC0 as the official 1.5.0 release of Apache Bigtop
> >
> > (Community vote)
> >
> > I completed my tests, the new release didn't raise any blocking issue.
> > My tests were for this use case:
> > - Hadoop test cluster with Hive/Oozie/Spark/Jupyter/Hue on bare metal
> hosts.
> > - HDFS and Yarn HA setup (including journal nodes).
> > - Bigtop 1.4 for Debian 9 for the majority of the nodes, and a client
> > node on Debian 10.
> > - In place upgrade from Bigtop 1.4 to 1.5 (upgrading the packages
> > without removing the old ones first).
> >
> > I have tested an in place upgrade, trying to rollback/re-deploy before
> > the HDFS finalize step, all good. Some interesting things that I
> > noticed or had to fix in my config after the upgrade:
> > - the HDFS Namenodes seem to be smarter when dealing with a rollback.
> > While testing CDH -> Bigtop 1.4 (hdfs 2.6.0 -> 2.8.5) I had to follow
> > the upstream guidelines and explicitly issue commands to rollback the
> > hdfs state, but when testing this release it was not needed anymore.
> > I'll dig a big more into this, but overall the upgrade/rollback
> > procedure seemed smoother.
> > - our Hadoop clusters never used the
> > "yarn.resourcemanager.webapp.address.$rm-id" settings in yarn-site.xml
> > (that seem to have a good default), but on Bigtop 1.5 it lead to NPE
> > while running map-reduce jobs, like described in
> > https://issues.apache.org/jira/browse/YARN-8056. Once I set the value,
> > everything worked as expected.
> > - while upgrading the packages, I had a little hiccup with oozie since
> > the version on Bigtop 1.4 is higher than 1.5 (4.3.0-3 vs 4.3.0-1), so
> > it needed a downgrade. Nothing really major, just mentioning it.
> >
> > About https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.5.0-RC0:
> > - verified .asc/.shaXXX, all good.
> > - all debian packages (9 and 10 versions) worked fine, and Kengo
> > Seki's gpg signature was ok as well.
> >
> > Thanks a lot for this release!
> >
> > Luca
>
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Kengo Seki <se...@apache.org>.
Oh I see, will remove the note about it from cwiki. Thank you for the
notification ;)
Kengo Seki <se...@apache.org>
On Tue, Dec 15, 2020 at 12:35 AM Luca Toscano <to...@gmail.com> wrote:
>
> Correction sorry - the oozie packages are ok, a downgrade was raised
> when installing the bigtop 1.5 packages since on bigtop 1.4 I have a
> version of oozie (4.3.0-3) with some extra patches in it (IIRC due to
> https://issues.apache.org/jira/browse/BIGTOP-3330).
>
> Luca
>
>
> On Mon, Dec 14, 2020 at 2:28 AM Kengo Seki <se...@apache.org> wrote:
> >
> > Luca,
> >
> > Thank you for the detailed checks!
> > That's really helpful and informative, so I added some of your
> > findings to the cwiki.
> > https://cwiki.apache.org/confluence/display/BIGTOP/Overview+of+Bigtop+1.5.0+Support+Matrix
> >
> > Kengo Seki <se...@apache.org>
> >
> > On Sun, Dec 13, 2020 at 5:40 PM Luca Toscano <to...@gmail.com> wrote:
> > >
> > > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org> wrote:
> > > >
> > > > This is the vote for release 1.5.0 of Apache Bigtop.
> > > >
> > > > [x] +1, accept RC0 as the official 1.5.0 release of Apache Bigtop
> > >
> > > (Community vote)
> > >
> > > I completed my tests, the new release didn't raise any blocking issue.
> > > My tests were for this use case:
> > > - Hadoop test cluster with Hive/Oozie/Spark/Jupyter/Hue on bare metal hosts.
> > > - HDFS and Yarn HA setup (including journal nodes).
> > > - Bigtop 1.4 for Debian 9 for the majority of the nodes, and a client
> > > node on Debian 10.
> > > - In place upgrade from Bigtop 1.4 to 1.5 (upgrading the packages
> > > without removing the old ones first).
> > >
> > > I have tested an in place upgrade, trying to rollback/re-deploy before
> > > the HDFS finalize step, all good. Some interesting things that I
> > > noticed or had to fix in my config after the upgrade:
> > > - the HDFS Namenodes seem to be smarter when dealing with a rollback.
> > > While testing CDH -> Bigtop 1.4 (hdfs 2.6.0 -> 2.8.5) I had to follow
> > > the upstream guidelines and explicitly issue commands to rollback the
> > > hdfs state, but when testing this release it was not needed anymore.
> > > I'll dig a big more into this, but overall the upgrade/rollback
> > > procedure seemed smoother.
> > > - our Hadoop clusters never used the
> > > "yarn.resourcemanager.webapp.address.$rm-id" settings in yarn-site.xml
> > > (that seem to have a good default), but on Bigtop 1.5 it lead to NPE
> > > while running map-reduce jobs, like described in
> > > https://issues.apache.org/jira/browse/YARN-8056. Once I set the value,
> > > everything worked as expected.
> > > - while upgrading the packages, I had a little hiccup with oozie since
> > > the version on Bigtop 1.4 is higher than 1.5 (4.3.0-3 vs 4.3.0-1), so
> > > it needed a downgrade. Nothing really major, just mentioning it.
> > >
> > > About https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.5.0-RC0:
> > > - verified .asc/.shaXXX, all good.
> > > - all debian packages (9 and 10 versions) worked fine, and Kengo
> > > Seki's gpg signature was ok as well.
> > >
> > > Thanks a lot for this release!
> > >
> > > Luca
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Kengo Seki <se...@apache.org>.
Thanks everyone, here's my vote:
+1 (binding): Checked release tarball contents, signature and checksums.
Also deployed some core components and ran their smoke tests on a few distros.
With three binding +1s, three non-binding +1s, and no -1 or 0 votes,
the vote for 1.5.0 release PASSED.
Binding +1s:
Evans Ye
Jun He
Kengo Seki
Non-binding +1s:
Yuqi Gu
Luca Toscano
Masatake Iwasaki
Thank you all for voting, testing, and giving feedback to make this
release happen!
I'll start to prepare the formal release and roll it out shortly.
Kengo Seki <se...@apache.org>
On Tue, Dec 15, 2020 at 11:43 AM Jun HE <ju...@apache.org> wrote:
>
> Oh, sorry I made a mistake here. Thanks for pointing that out, Evans. :)
>
> Correction for my vote:
> I'm +1(binding) to accept RC0 as 1.5.0 release.
>
> Evans Ye <ev...@apache.org> 于2020年12月15日周二 上午1:33写道：
>
> > Jun He your vote should be a binding one.
> > See http://www.apache.org/legal/release-policy.html#release-approval
> >
> > Luca Toscano <to...@gmail.com> 於 2020年12月14日 週一 下午11:36寫道：
> >
> > > Correction sorry - the oozie packages are ok, a downgrade was raised
> > > when installing the bigtop 1.5 packages since on bigtop 1.4 I have a
> > > version of oozie (4.3.0-3) with some extra patches in it (IIRC due to
> > > https://issues.apache.org/jira/browse/BIGTOP-3330).
> > >
> > > Luca
> > >
> > >
> > > On Mon, Dec 14, 2020 at 2:28 AM Kengo Seki <se...@apache.org> wrote:
> > > >
> > > > Luca,
> > > >
> > > > Thank you for the detailed checks!
> > > > That's really helpful and informative, so I added some of your
> > > > findings to the cwiki.
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/BIGTOP/Overview+of+Bigtop+1.5.0+Support+Matrix
> > > >
> > > > Kengo Seki <se...@apache.org>
> > > >
> > > > On Sun, Dec 13, 2020 at 5:40 PM Luca Toscano <to...@gmail.com>
> > > wrote:
> > > > >
> > > > > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org>
> > wrote:
> > > > > >
> > > > > > This is the vote for release 1.5.0 of Apache Bigtop.
> > > > > >
> > > > > > [x] +1, accept RC0 as the official 1.5.0 release of Apache Bigtop
> > > > >
> > > > > (Community vote)
> > > > >
> > > > > I completed my tests, the new release didn't raise any blocking
> > issue.
> > > > > My tests were for this use case:
> > > > > - Hadoop test cluster with Hive/Oozie/Spark/Jupyter/Hue on bare metal
> > > hosts.
> > > > > - HDFS and Yarn HA setup (including journal nodes).
> > > > > - Bigtop 1.4 for Debian 9 for the majority of the nodes, and a client
> > > > > node on Debian 10.
> > > > > - In place upgrade from Bigtop 1.4 to 1.5 (upgrading the packages
> > > > > without removing the old ones first).
> > > > >
> > > > > I have tested an in place upgrade, trying to rollback/re-deploy
> > before
> > > > > the HDFS finalize step, all good. Some interesting things that I
> > > > > noticed or had to fix in my config after the upgrade:
> > > > > - the HDFS Namenodes seem to be smarter when dealing with a rollback.
> > > > > While testing CDH -> Bigtop 1.4 (hdfs 2.6.0 -> 2.8.5) I had to follow
> > > > > the upstream guidelines and explicitly issue commands to rollback the
> > > > > hdfs state, but when testing this release it was not needed anymore.
> > > > > I'll dig a big more into this, but overall the upgrade/rollback
> > > > > procedure seemed smoother.
> > > > > - our Hadoop clusters never used the
> > > > > "yarn.resourcemanager.webapp.address.$rm-id" settings in
> > yarn-site.xml
> > > > > (that seem to have a good default), but on Bigtop 1.5 it lead to NPE
> > > > > while running map-reduce jobs, like described in
> > > > > https://issues.apache.org/jira/browse/YARN-8056. Once I set the
> > value,
> > > > > everything worked as expected.
> > > > > - while upgrading the packages, I had a little hiccup with oozie
> > since
> > > > > the version on Bigtop 1.4 is higher than 1.5 (4.3.0-3 vs 4.3.0-1), so
> > > > > it needed a downgrade. Nothing really major, just mentioning it.
> > > > >
> > > > > About https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.5.0-RC0
> > :
> > > > > - verified .asc/.shaXXX, all good.
> > > > > - all debian packages (9 and 10 versions) worked fine, and Kengo
> > > > > Seki's gpg signature was ok as well.
> > > > >
> > > > > Thanks a lot for this release!
> > > > >
> > > > > Luca
> > >
> >
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Jun HE <ju...@apache.org>.
Oh, sorry I made a mistake here. Thanks for pointing that out, Evans. :)
Correction for my vote:
I'm +1(binding) to accept RC0 as 1.5.0 release.
Evans Ye <ev...@apache.org> 于2020年12月15日周二 上午1:33写道：
> Jun He your vote should be a binding one.
> See http://www.apache.org/legal/release-policy.html#release-approval
>
> Luca Toscano <to...@gmail.com> 於 2020年12月14日 週一 下午11:36寫道：
>
> > Correction sorry - the oozie packages are ok, a downgrade was raised
> > when installing the bigtop 1.5 packages since on bigtop 1.4 I have a
> > version of oozie (4.3.0-3) with some extra patches in it (IIRC due to
> > https://issues.apache.org/jira/browse/BIGTOP-3330).
> >
> > Luca
> >
> >
> > On Mon, Dec 14, 2020 at 2:28 AM Kengo Seki <se...@apache.org> wrote:
> > >
> > > Luca,
> > >
> > > Thank you for the detailed checks!
> > > That's really helpful and informative, so I added some of your
> > > findings to the cwiki.
> > >
> >
> https://cwiki.apache.org/confluence/display/BIGTOP/Overview+of+Bigtop+1.5.0+Support+Matrix
> > >
> > > Kengo Seki <se...@apache.org>
> > >
> > > On Sun, Dec 13, 2020 at 5:40 PM Luca Toscano <to...@gmail.com>
> > wrote:
> > > >
> > > > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org>
> wrote:
> > > > >
> > > > > This is the vote for release 1.5.0 of Apache Bigtop.
> > > > >
> > > > > [x] +1, accept RC0 as the official 1.5.0 release of Apache Bigtop
> > > >
> > > > (Community vote)
> > > >
> > > > I completed my tests, the new release didn't raise any blocking
> issue.
> > > > My tests were for this use case:
> > > > - Hadoop test cluster with Hive/Oozie/Spark/Jupyter/Hue on bare metal
> > hosts.
> > > > - HDFS and Yarn HA setup (including journal nodes).
> > > > - Bigtop 1.4 for Debian 9 for the majority of the nodes, and a client
> > > > node on Debian 10.
> > > > - In place upgrade from Bigtop 1.4 to 1.5 (upgrading the packages
> > > > without removing the old ones first).
> > > >
> > > > I have tested an in place upgrade, trying to rollback/re-deploy
> before
> > > > the HDFS finalize step, all good. Some interesting things that I
> > > > noticed or had to fix in my config after the upgrade:
> > > > - the HDFS Namenodes seem to be smarter when dealing with a rollback.
> > > > While testing CDH -> Bigtop 1.4 (hdfs 2.6.0 -> 2.8.5) I had to follow
> > > > the upstream guidelines and explicitly issue commands to rollback the
> > > > hdfs state, but when testing this release it was not needed anymore.
> > > > I'll dig a big more into this, but overall the upgrade/rollback
> > > > procedure seemed smoother.
> > > > - our Hadoop clusters never used the
> > > > "yarn.resourcemanager.webapp.address.$rm-id" settings in
> yarn-site.xml
> > > > (that seem to have a good default), but on Bigtop 1.5 it lead to NPE
> > > > while running map-reduce jobs, like described in
> > > > https://issues.apache.org/jira/browse/YARN-8056. Once I set the
> value,
> > > > everything worked as expected.
> > > > - while upgrading the packages, I had a little hiccup with oozie
> since
> > > > the version on Bigtop 1.4 is higher than 1.5 (4.3.0-3 vs 4.3.0-1), so
> > > > it needed a downgrade. Nothing really major, just mentioning it.
> > > >
> > > > About https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.5.0-RC0
> :
> > > > - verified .asc/.shaXXX, all good.
> > > > - all debian packages (9 and 10 versions) worked fine, and Kengo
> > > > Seki's gpg signature was ok as well.
> > > >
> > > > Thanks a lot for this release!
> > > >
> > > > Luca
> >
>
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Evans Ye <ev...@apache.org>.
Jun He your vote should be a binding one.
See http://www.apache.org/legal/release-policy.html#release-approval
Luca Toscano <to...@gmail.com> 於 2020年12月14日 週一 下午11:36寫道：
> Correction sorry - the oozie packages are ok, a downgrade was raised
> when installing the bigtop 1.5 packages since on bigtop 1.4 I have a
> version of oozie (4.3.0-3) with some extra patches in it (IIRC due to
> https://issues.apache.org/jira/browse/BIGTOP-3330).
>
> Luca
>
>
> On Mon, Dec 14, 2020 at 2:28 AM Kengo Seki <se...@apache.org> wrote:
> >
> > Luca,
> >
> > Thank you for the detailed checks!
> > That's really helpful and informative, so I added some of your
> > findings to the cwiki.
> >
> https://cwiki.apache.org/confluence/display/BIGTOP/Overview+of+Bigtop+1.5.0+Support+Matrix
> >
> > Kengo Seki <se...@apache.org>
> >
> > On Sun, Dec 13, 2020 at 5:40 PM Luca Toscano <to...@gmail.com>
> wrote:
> > >
> > > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org> wrote:
> > > >
> > > > This is the vote for release 1.5.0 of Apache Bigtop.
> > > >
> > > > [x] +1, accept RC0 as the official 1.5.0 release of Apache Bigtop
> > >
> > > (Community vote)
> > >
> > > I completed my tests, the new release didn't raise any blocking issue.
> > > My tests were for this use case:
> > > - Hadoop test cluster with Hive/Oozie/Spark/Jupyter/Hue on bare metal
> hosts.
> > > - HDFS and Yarn HA setup (including journal nodes).
> > > - Bigtop 1.4 for Debian 9 for the majority of the nodes, and a client
> > > node on Debian 10.
> > > - In place upgrade from Bigtop 1.4 to 1.5 (upgrading the packages
> > > without removing the old ones first).
> > >
> > > I have tested an in place upgrade, trying to rollback/re-deploy before
> > > the HDFS finalize step, all good. Some interesting things that I
> > > noticed or had to fix in my config after the upgrade:
> > > - the HDFS Namenodes seem to be smarter when dealing with a rollback.
> > > While testing CDH -> Bigtop 1.4 (hdfs 2.6.0 -> 2.8.5) I had to follow
> > > the upstream guidelines and explicitly issue commands to rollback the
> > > hdfs state, but when testing this release it was not needed anymore.
> > > I'll dig a big more into this, but overall the upgrade/rollback
> > > procedure seemed smoother.
> > > - our Hadoop clusters never used the
> > > "yarn.resourcemanager.webapp.address.$rm-id" settings in yarn-site.xml
> > > (that seem to have a good default), but on Bigtop 1.5 it lead to NPE
> > > while running map-reduce jobs, like described in
> > > https://issues.apache.org/jira/browse/YARN-8056. Once I set the value,
> > > everything worked as expected.
> > > - while upgrading the packages, I had a little hiccup with oozie since
> > > the version on Bigtop 1.4 is higher than 1.5 (4.3.0-3 vs 4.3.0-1), so
> > > it needed a downgrade. Nothing really major, just mentioning it.
> > >
> > > About https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.5.0-RC0:
> > > - verified .asc/.shaXXX, all good.
> > > - all debian packages (9 and 10 versions) worked fine, and Kengo
> > > Seki's gpg signature was ok as well.
> > >
> > > Thanks a lot for this release!
> > >
> > > Luca
>
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Luca Toscano <to...@gmail.com>.
Correction sorry - the oozie packages are ok, a downgrade was raised
when installing the bigtop 1.5 packages since on bigtop 1.4 I have a
version of oozie (4.3.0-3) with some extra patches in it (IIRC due to
https://issues.apache.org/jira/browse/BIGTOP-3330).
Luca
On Mon, Dec 14, 2020 at 2:28 AM Kengo Seki <se...@apache.org> wrote:
>
> Luca,
>
> Thank you for the detailed checks!
> That's really helpful and informative, so I added some of your
> findings to the cwiki.
> https://cwiki.apache.org/confluence/display/BIGTOP/Overview+of+Bigtop+1.5.0+Support+Matrix
>
> Kengo Seki <se...@apache.org>
>
> On Sun, Dec 13, 2020 at 5:40 PM Luca Toscano <to...@gmail.com> wrote:
> >
> > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org> wrote:
> > >
> > > This is the vote for release 1.5.0 of Apache Bigtop.
> > >
> > > [x] +1, accept RC0 as the official 1.5.0 release of Apache Bigtop
> >
> > (Community vote)
> >
> > I completed my tests, the new release didn't raise any blocking issue.
> > My tests were for this use case:
> > - Hadoop test cluster with Hive/Oozie/Spark/Jupyter/Hue on bare metal hosts.
> > - HDFS and Yarn HA setup (including journal nodes).
> > - Bigtop 1.4 for Debian 9 for the majority of the nodes, and a client
> > node on Debian 10.
> > - In place upgrade from Bigtop 1.4 to 1.5 (upgrading the packages
> > without removing the old ones first).
> >
> > I have tested an in place upgrade, trying to rollback/re-deploy before
> > the HDFS finalize step, all good. Some interesting things that I
> > noticed or had to fix in my config after the upgrade:
> > - the HDFS Namenodes seem to be smarter when dealing with a rollback.
> > While testing CDH -> Bigtop 1.4 (hdfs 2.6.0 -> 2.8.5) I had to follow
> > the upstream guidelines and explicitly issue commands to rollback the
> > hdfs state, but when testing this release it was not needed anymore.
> > I'll dig a big more into this, but overall the upgrade/rollback
> > procedure seemed smoother.
> > - our Hadoop clusters never used the
> > "yarn.resourcemanager.webapp.address.$rm-id" settings in yarn-site.xml
> > (that seem to have a good default), but on Bigtop 1.5 it lead to NPE
> > while running map-reduce jobs, like described in
> > https://issues.apache.org/jira/browse/YARN-8056. Once I set the value,
> > everything worked as expected.
> > - while upgrading the packages, I had a little hiccup with oozie since
> > the version on Bigtop 1.4 is higher than 1.5 (4.3.0-3 vs 4.3.0-1), so
> > it needed a downgrade. Nothing really major, just mentioning it.
> >
> > About https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.5.0-RC0:
> > - verified .asc/.shaXXX, all good.
> > - all debian packages (9 and 10 versions) worked fine, and Kengo
> > Seki's gpg signature was ok as well.
> >
> > Thanks a lot for this release!
> >
> > Luca
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Kengo Seki <se...@apache.org>.
Luca,
Thank you for the detailed checks!
That's really helpful and informative, so I added some of your
findings to the cwiki.
https://cwiki.apache.org/confluence/display/BIGTOP/Overview+of+Bigtop+1.5.0+Support+Matrix
Kengo Seki <se...@apache.org>
On Sun, Dec 13, 2020 at 5:40 PM Luca Toscano <to...@gmail.com> wrote:
>
> On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org> wrote:
> >
> > This is the vote for release 1.5.0 of Apache Bigtop.
> >
> > [x] +1, accept RC0 as the official 1.5.0 release of Apache Bigtop
>
> (Community vote)
>
> I completed my tests, the new release didn't raise any blocking issue.
> My tests were for this use case:
> - Hadoop test cluster with Hive/Oozie/Spark/Jupyter/Hue on bare metal hosts.
> - HDFS and Yarn HA setup (including journal nodes).
> - Bigtop 1.4 for Debian 9 for the majority of the nodes, and a client
> node on Debian 10.
> - In place upgrade from Bigtop 1.4 to 1.5 (upgrading the packages
> without removing the old ones first).
>
> I have tested an in place upgrade, trying to rollback/re-deploy before
> the HDFS finalize step, all good. Some interesting things that I
> noticed or had to fix in my config after the upgrade:
> - the HDFS Namenodes seem to be smarter when dealing with a rollback.
> While testing CDH -> Bigtop 1.4 (hdfs 2.6.0 -> 2.8.5) I had to follow
> the upstream guidelines and explicitly issue commands to rollback the
> hdfs state, but when testing this release it was not needed anymore.
> I'll dig a big more into this, but overall the upgrade/rollback
> procedure seemed smoother.
> - our Hadoop clusters never used the
> "yarn.resourcemanager.webapp.address.$rm-id" settings in yarn-site.xml
> (that seem to have a good default), but on Bigtop 1.5 it lead to NPE
> while running map-reduce jobs, like described in
> https://issues.apache.org/jira/browse/YARN-8056. Once I set the value,
> everything worked as expected.
> - while upgrading the packages, I had a little hiccup with oozie since
> the version on Bigtop 1.4 is higher than 1.5 (4.3.0-3 vs 4.3.0-1), so
> it needed a downgrade. Nothing really major, just mentioning it.
>
> About https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.5.0-RC0:
> - verified .asc/.shaXXX, all good.
> - all debian packages (9 and 10 versions) worked fine, and Kengo
> Seki's gpg signature was ok as well.
>
> Thanks a lot for this release!
>
> Luca
Re: [VOTE] Release Bigtop version 1.5.0
Posted by Luca Toscano <to...@gmail.com>.
On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <se...@apache.org> wrote:
>
> This is the vote for release 1.5.0 of Apache Bigtop.
>
> [x] +1, accept RC0 as the official 1.5.0 release of Apache Bigtop
(Community vote)
I completed my tests, the new release didn't raise any blocking issue.
My tests were for this use case:
- Hadoop test cluster with Hive/Oozie/Spark/Jupyter/Hue on bare metal hosts.
- HDFS and Yarn HA setup (including journal nodes).
- Bigtop 1.4 for Debian 9 for the majority of the nodes, and a client
node on Debian 10.
- In place upgrade from Bigtop 1.4 to 1.5 (upgrading the packages
without removing the old ones first).
I have tested an in place upgrade, trying to rollback/re-deploy before
the HDFS finalize step, all good. Some interesting things that I
noticed or had to fix in my config after the upgrade:
- the HDFS Namenodes seem to be smarter when dealing with a rollback.
While testing CDH -> Bigtop 1.4 (hdfs 2.6.0 -> 2.8.5) I had to follow
the upstream guidelines and explicitly issue commands to rollback the
hdfs state, but when testing this release it was not needed anymore.
I'll dig a big more into this, but overall the upgrade/rollback
procedure seemed smoother.
- our Hadoop clusters never used the
"yarn.resourcemanager.webapp.address.$rm-id" settings in yarn-site.xml
(that seem to have a good default), but on Bigtop 1.5 it lead to NPE
while running map-reduce jobs, like described in
https://issues.apache.org/jira/browse/YARN-8056. Once I set the value,
everything worked as expected.
- while upgrading the packages, I had a little hiccup with oozie since
the version on Bigtop 1.4 is higher than 1.5 (4.3.0-3 vs 4.3.0-1), so
it needed a downgrade. Nothing really major, just mentioning it.
About https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.5.0-RC0:
- verified .asc/.shaXXX, all good.
- all debian packages (9 and 10 versions) worked fine, and Kengo
Seki's gpg signature was ok as well.
Thanks a lot for this release!
Luca