You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by Andrew Wang <an...@cloudera.com> on 2017/12/08 20:31:09 UTC

[VOTE] Release Apache Hadoop 3.0.0 RC1

Hi all,

Let me start, as always, by thanking the efforts of all the contributors
who contributed to this release, especially those who jumped on the issues
found in RC0.

I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
fixed JIRAs since the previous 3.0.0-beta1 release.

You can find the artifacts here:

http://home.apache.org/~wang/3.0.0-RC1/

I've done the traditional testing of building from the source tarball and
running a Pi job on a single node cluster. I also verified that the shaded
jars are not empty.

Found one issue that create-release (probably due to the mvn deploy change)
didn't sign the artifacts, but I fixed that by calling mvn one more time.
Available here:

https://repository.apache.org/content/repositories/orgapachehadoop-1075/

This release will run the standard 5 days, closing on Dec 13th at 12:31pm
Pacific. My +1 to start.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Iñigo Goiri <el...@gmail.com>.

+1 (non-binding)

I tested it in a deployment with 24 nodes across 8 subclusters.
Tested a few jobs reading and writing data through HDFS Router-based
federation.
However, jobs failed to run when setting RBF as the default filesystem
because after MAPREDUCE-6954, it tries to invoke setErasureCodingPolicy
while is not implemented.
I filed HDFS-12919 to track this but I don't think is a blocker.

Thanks,
Inigo

On Tue, Dec 12, 2017 at 2:43 PM, Elek, Marton <hd...@anzix.net> wrote:

> +1 (non-binding)
>
>  * built from the source tarball (archlinux) / verified signature
>  * Deployed to a kubernetes cluster (10/10 datanode/nodemanager pods)
>  * Enabled ec on hdfs directory (hdfs cli)
>  * Started example yarn jobs (pi/terragen)
>  * checked yarn ui/ui2
>
> Thanks for all the efforts.
>
> Marton
>
>
>
> On 12/08/2017 09:31 PM, Andrew Wang wrote:
>
>> Hi all,
>>
>> Let me start, as always, by thanking the efforts of all the contributors
>> who contributed to this release, especially those who jumped on the issues
>> found in RC0.
>>
>> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
>> fixed JIRAs since the previous 3.0.0-beta1 release.
>>
>> You can find the artifacts here:
>>
>> http://home.apache.org/~wang/3.0.0-RC1/
>>
>> I've done the traditional testing of building from the source tarball and
>> running a Pi job on a single node cluster. I also verified that the shaded
>> jars are not empty.
>>
>> Found one issue that create-release (probably due to the mvn deploy
>> change)
>> didn't sign the artifacts, but I fixed that by calling mvn one more time.
>> Available here:
>>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>>
>> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
>> Pacific. My +1 to start.
>>
>> Best,
>> Andrew
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Iñigo Goiri <el...@gmail.com>.

+1 (non-binding)

I tested it in a deployment with 24 nodes across 8 subclusters.
Tested a few jobs reading and writing data through HDFS Router-based
federation.
However, jobs failed to run when setting RBF as the default filesystem
because after MAPREDUCE-6954, it tries to invoke setErasureCodingPolicy
while is not implemented.
I filed HDFS-12919 to track this but I don't think is a blocker.

Thanks,
Inigo

On Tue, Dec 12, 2017 at 2:43 PM, Elek, Marton <hd...@anzix.net> wrote:

> +1 (non-binding)
>
>  * built from the source tarball (archlinux) / verified signature
>  * Deployed to a kubernetes cluster (10/10 datanode/nodemanager pods)
>  * Enabled ec on hdfs directory (hdfs cli)
>  * Started example yarn jobs (pi/terragen)
>  * checked yarn ui/ui2
>
> Thanks for all the efforts.
>
> Marton
>
>
>
> On 12/08/2017 09:31 PM, Andrew Wang wrote:
>
>> Hi all,
>>
>> Let me start, as always, by thanking the efforts of all the contributors
>> who contributed to this release, especially those who jumped on the issues
>> found in RC0.
>>
>> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
>> fixed JIRAs since the previous 3.0.0-beta1 release.
>>
>> You can find the artifacts here:
>>
>> http://home.apache.org/~wang/3.0.0-RC1/
>>
>> I've done the traditional testing of building from the source tarball and
>> running a Pi job on a single node cluster. I also verified that the shaded
>> jars are not empty.
>>
>> Found one issue that create-release (probably due to the mvn deploy
>> change)
>> didn't sign the artifacts, but I fixed that by calling mvn one more time.
>> Available here:
>>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>>
>> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
>> Pacific. My +1 to start.
>>
>> Best,
>> Andrew
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Iñigo Goiri <el...@gmail.com>.

+1 (non-binding)

I tested it in a deployment with 24 nodes across 8 subclusters.
Tested a few jobs reading and writing data through HDFS Router-based
federation.
However, jobs failed to run when setting RBF as the default filesystem
because after MAPREDUCE-6954, it tries to invoke setErasureCodingPolicy
while is not implemented.
I filed HDFS-12919 to track this but I don't think is a blocker.

Thanks,
Inigo

On Tue, Dec 12, 2017 at 2:43 PM, Elek, Marton <hd...@anzix.net> wrote:

> +1 (non-binding)
>
>  * built from the source tarball (archlinux) / verified signature
>  * Deployed to a kubernetes cluster (10/10 datanode/nodemanager pods)
>  * Enabled ec on hdfs directory (hdfs cli)
>  * Started example yarn jobs (pi/terragen)
>  * checked yarn ui/ui2
>
> Thanks for all the efforts.
>
> Marton
>
>
>
> On 12/08/2017 09:31 PM, Andrew Wang wrote:
>
>> Hi all,
>>
>> Let me start, as always, by thanking the efforts of all the contributors
>> who contributed to this release, especially those who jumped on the issues
>> found in RC0.
>>
>> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
>> fixed JIRAs since the previous 3.0.0-beta1 release.
>>
>> You can find the artifacts here:
>>
>> http://home.apache.org/~wang/3.0.0-RC1/
>>
>> I've done the traditional testing of building from the source tarball and
>> running a Pi job on a single node cluster. I also verified that the shaded
>> jars are not empty.
>>
>> Found one issue that create-release (probably due to the mvn deploy
>> change)
>> didn't sign the artifacts, but I fixed that by calling mvn one more time.
>> Available here:
>>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>>
>> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
>> Pacific. My +1 to start.
>>
>> Best,
>> Andrew
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Iñigo Goiri <el...@gmail.com>.

+1 (non-binding)

I tested it in a deployment with 24 nodes across 8 subclusters.
Tested a few jobs reading and writing data through HDFS Router-based
federation.
However, jobs failed to run when setting RBF as the default filesystem
because after MAPREDUCE-6954, it tries to invoke setErasureCodingPolicy
while is not implemented.
I filed HDFS-12919 to track this but I don't think is a blocker.

Thanks,
Inigo

On Tue, Dec 12, 2017 at 2:43 PM, Elek, Marton <hd...@anzix.net> wrote:

> +1 (non-binding)
>
>  * built from the source tarball (archlinux) / verified signature
>  * Deployed to a kubernetes cluster (10/10 datanode/nodemanager pods)
>  * Enabled ec on hdfs directory (hdfs cli)
>  * Started example yarn jobs (pi/terragen)
>  * checked yarn ui/ui2
>
> Thanks for all the efforts.
>
> Marton
>
>
>
> On 12/08/2017 09:31 PM, Andrew Wang wrote:
>
>> Hi all,
>>
>> Let me start, as always, by thanking the efforts of all the contributors
>> who contributed to this release, especially those who jumped on the issues
>> found in RC0.
>>
>> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
>> fixed JIRAs since the previous 3.0.0-beta1 release.
>>
>> You can find the artifacts here:
>>
>> http://home.apache.org/~wang/3.0.0-RC1/
>>
>> I've done the traditional testing of building from the source tarball and
>> running a Pi job on a single node cluster. I also verified that the shaded
>> jars are not empty.
>>
>> Found one issue that create-release (probably due to the mvn deploy
>> change)
>> didn't sign the artifacts, but I fixed that by calling mvn one more time.
>> Available here:
>>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>>
>> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
>> Pacific. My +1 to start.
>>
>> Best,
>> Andrew
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by "Elek, Marton" <hd...@anzix.net>.

+1 (non-binding)

  * built from the source tarball (archlinux) / verified signature
  * Deployed to a kubernetes cluster (10/10 datanode/nodemanager pods)
  * Enabled ec on hdfs directory (hdfs cli)
  * Started example yarn jobs (pi/terragen)
  * checked yarn ui/ui2

Thanks for all the efforts.

Marton


On 12/08/2017 09:31 PM, Andrew Wang wrote:
> Hi all,
> 
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
> 
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
> 
> You can find the artifacts here:
> 
> http://home.apache.org/~wang/3.0.0-RC1/
> 
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
> 
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
> 
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> 
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
> 
> Best,
> Andrew
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Rohith Sharma K S <ro...@apache.org>.

+1 (binding)

- built from source and deployed 3 node cluster
- installed RM HA cluster along with ATSv2 enabled and new YARN UI.
- verified for
-- RM HA switch / RM Restart / RM Work preserving restart
-- NM work preserving restart
-- Ran sample MR jobs and Distributed shell along with multiple RM and NM
switch
- verified for ATSv2 entities data, REST API's validation as HBase-1.2.6 as
back end.
- verified for priority. timeout feature of RM.
- verified for new YARN UI and pages along with atsv2 integration

Thanks & Regards
Rohith Sharma K S


On 9 December 2017 at 02:01, Andrew Wang <an...@cloudera.com> wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by "Elek, Marton" <hd...@anzix.net>.

+1 (non-binding)

  * built from the source tarball (archlinux) / verified signature
  * Deployed to a kubernetes cluster (10/10 datanode/nodemanager pods)
  * Enabled ec on hdfs directory (hdfs cli)
  * Started example yarn jobs (pi/terragen)
  * checked yarn ui/ui2

Thanks for all the efforts.

Marton


On 12/08/2017 09:31 PM, Andrew Wang wrote:
> Hi all,
> 
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
> 
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
> 
> You can find the artifacts here:
> 
> http://home.apache.org/~wang/3.0.0-RC1/
> 
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
> 
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
> 
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> 
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
> 
> Best,
> Andrew
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sangjin Lee <sj...@apache.org>.

+1 (binding)

- downloaded the binary tarball and the source tarball and checked
signatures
- verified that the source builds cleanly
- verified that the shaded client jars are correct
- checked the basic pseudo-distributed cluster set-up and checked UI and
logs (hdfs and YARN)
- ran some test jobs successfully
- enabled Timeline Service v.2 and tested the writer/reader with test jobs


On Tue, Dec 12, 2017 at 6:14 PM, Jonathan Hung <jy...@gmail.com> wrote:

> Thanks Andrew for the huge effort.
>
> +1 (non-binding)
> - Downloaded binary tarball and verified md5
> - Ran RM HA and verified manual failover
> - Verified add/remove/update scheduler configuration API (CLI/REST) works
> for leveldb/zookeeper backend
> - Verified scheduler configuration changes persisted on restart/failover
> - Verified "yarn rmadmin -refreshQueues" works when scheduler configuration
> API disabled, and does not work when scheduler configuration API enabled
>
>
> Jonathan Hung
>
> On Tue, Dec 12, 2017 at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get
> > chance to validate new RC now.
> >
> > Basically, I found two critical issues with the same rolling upgrade
> > scenario as where HADOOP-15059 get found previously:
> > HDFS-12920, we changed value format for some hdfs configurations that old
> > version MR client doesn't understand when fetching these configurations.
> > Some quick workarounds are to add old value (without time unit) in
> > hdfs-site.xml to override new default values but will generate many
> > annoying warnings. I provided my fix suggestions on the JIRA already for
> > more discussion.
> > The other one is YARN-7646. After we workaround HDFS-12920, will hit the
> > issue that old version MR AppMaster cannot communicate with new version
> of
> > YARN RM - could be related to resource profile changes from YARN side but
> > root cause are still in investigation.
> >
> > The first issue may not belong to a blocker given we can workaround this
> > without code change. I am not sure if we can workaround 2nd issue so far.
> > If not, we may have to fix this or compromise with withdrawing support of
> > rolling upgrade or calling it a stable release.
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Robert Kanter <rk...@cloudera.com>
> > Sent: Tuesday, December 12, 2017 3:10 PM
> > To: Arun Suresh
> > Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T.
> > Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
> >
> > +1 (binding)
> >
> > + Downloaded the binary release
> > + Deployed on a 3 node cluster on CentOS 7.3
> > + Ran some MR jobs, clicked around the UI, etc
> > + Ran some CLI commands (yarn logs, etc)
> >
> > Good job everyone on Hadoop 3!
> >
> >
> > - Robert
> >
> > On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
> >
> > > +1 (binding)
> > >
> > > - Verified signatures of the source tarball.
> > > - built from source - using the docker build environment.
> > > - set up a pseudo-distributed test cluster.
> > > - ran basic HDFS commands
> > > - ran some basic MR jobs
> > >
> > > Cheers
> > > -Arun
> > >
> > > On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <andrew.wang@cloudera.com
> >
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > As a reminder, this vote closes tomorrow at 12:31pm, so please give
> it
> > a
> > > > whack if you have time. There are already enough binding +1s to pass
> > this
> > > > vote, but it'd be great to get additional validation.
> > > >
> > > > Thanks to everyone who's voted thus far!
> > > >
> > > > Best,
> > > > Andrew
> > > >
> > > >
> > > >
> > > > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > * Verified src tarball and bin tarball, verified md5 of each.
> > > > > * Build source with -Pdist,native
> > > > > * Started a pseudo cluster
> > > > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run
> hdfs
> > > > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > > > >
> > > > > Thanks Andrew for this great effort!
> > > > >
> > > > > Best,
> > > > >
> > > > >
> > > > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <
> > andrew.wang@cloudera.com
> > > >
> > > > > wrote:
> > > > > > Hi Wei-Chiu,
> > > > > >
> > > > > > The patchprocess directory is left over from the create-release
> > > > process,
> > > > > > and it looks empty to me. We should still file a create-release
> > JIRA
> > > to
> > > > > fix
> > > > > > this, but I think this is not a blocker. Would you agree?
> > > > > >
> > > > > > Best,
> > > > > > Andrew
> > > > > >
> > > > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> > > weichiu@cloudera.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Andrew, thanks the tremendous effort.
> > > > > >> I found an empty "patchprocess" directory in the source tarball,
> > > that
> > > > is
> > > > > >> not there if you clone from github. Any chance you might have
> some
> > > > > leftover
> > > > > >> trash when you made the tarball?
> > > > > >> Not wanting to nitpicking, but you might want to double check so
> > we
> > > > > don't
> > > > > >> ship anything private to you in public :)
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > > > ajay.kumar@hortonworks.com
> > > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >>> +1 (non-binding)
> > > > > >>> Thanks for driving this, Andrew Wang!!
> > > > > >>>
> > > > > >>> - downloaded the src tarball and verified md5 checksum
> > > > > >>> - built from source with jdk 1.8.0_111-b14
> > > > > >>> - brought up a pseudo distributed cluster
> > > > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > > > >>> confirmed that everything was working
> > > > > >>> - Run word count, pi and DFSIOTest
> > > > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > > > >>>
> > > > > >>> Cheers,
> > > > > >>> Ajay
> > > > > >>>
> > > > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > > > >>>
> > > > > >>>     +1 (binding)
> > > > > >>>
> > > > > >>>     - downloaded src tarball, verified md5
> > > > > >>>     - built from source with jdk1.8.0_112
> > > > > >>>     - started a pseudo cluster with hdfs and kms
> > > > > >>>     - sanity checked encryption related operations working
> > > > > >>>     - sanity checked webui and logs.
> > > > > >>>
> > > > > >>>     -Xiao
> > > > > >>>
> > > > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> > > atm@apache.org>
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>>     > +1 (binding)
> > > > > >>>     >
> > > > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > > > -Pnative)
> > > > > >>>     > - verified the checksum
> > > > > >>>     > - brought up a secure pseudo distributed cluster
> > > > > >>>     > - did some basic file system operations (mkdir, list,
> put,
> > > cat)
> > > > > and
> > > > > >>>     > confirmed that everything was working
> > > > > >>>     > - confirmed that the web UI worked
> > > > > >>>     >
> > > > > >>>     > Best,
> > > > > >>>     > Aaron
> > > > > >>>     >
> > > > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > > > >>> andrew.wang@cloudera.com>
> > > > > >>>     > wrote:
> > > > > >>>     >
> > > > > >>>     > > Hi all,
> > > > > >>>     > >
> > > > > >>>     > > Let me start, as always, by thanking the efforts of all
> > the
> > > > > >>> contributors
> > > > > >>>     > > who contributed to this release, especially those who
> > > jumped
> > > > on
> > > > > >>> the
> > > > > >>>     > issues
> > > > > >>>     > > found in RC0.
> > > > > >>>     > >
> > > > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > > > >>> incorporates 302
> > > > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > > > >>>     > >
> > > > > >>>     > > You can find the artifacts here:
> > > > > >>>     > >
> > > > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > > > >>>     > >
> > > > > >>>     > > I've done the traditional testing of building from the
> > > source
> > > > > >>> tarball and
> > > > > >>>     > > running a Pi job on a single node cluster. I also
> > verified
> > > > that
> > > > > >>> the
> > > > > >>>     > shaded
> > > > > >>>     > > jars are not empty.
> > > > > >>>     > >
> > > > > >>>     > > Found one issue that create-release (probably due to
> the
> > > mvn
> > > > > >>> deploy
> > > > > >>>     > change)
> > > > > >>>     > > didn't sign the artifacts, but I fixed that by calling
> > mvn
> > > > one
> > > > > >>> more time.
> > > > > >>>     > > Available here:
> > > > > >>>     > >
> > > > > >>>     > > https://repository.apache.org/
> > > content/repositories/orgapache
> > > > > >>> hadoop-1075/
> > > > > >>>     > >
> > > > > >>>     > > This release will run the standard 5 days, closing on
> Dec
> > > > 13th
> > > > > at
> > > > > >>> 12:31pm
> > > > > >>>     > > Pacific. My +1 to start.
> > > > > >>>     > >
> > > > > >>>     > > Best,
> > > > > >>>     > > Andrew
> > > > > >>>     > >
> > > > > >>>     >
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>> ------------------------------------------------------------
> > > > ---------
> > > > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.
> apache.org
> > > > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.
> > org
> > > > > >>>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Lei (Eddy) Xu
> > > > > Software Engineer, Cloudera
> > > > >
> > > >
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sangjin Lee <sj...@apache.org>.

+1 (binding)

- downloaded the binary tarball and the source tarball and checked
signatures
- verified that the source builds cleanly
- verified that the shaded client jars are correct
- checked the basic pseudo-distributed cluster set-up and checked UI and
logs (hdfs and YARN)
- ran some test jobs successfully
- enabled Timeline Service v.2 and tested the writer/reader with test jobs


On Tue, Dec 12, 2017 at 6:14 PM, Jonathan Hung <jy...@gmail.com> wrote:

> Thanks Andrew for the huge effort.
>
> +1 (non-binding)
> - Downloaded binary tarball and verified md5
> - Ran RM HA and verified manual failover
> - Verified add/remove/update scheduler configuration API (CLI/REST) works
> for leveldb/zookeeper backend
> - Verified scheduler configuration changes persisted on restart/failover
> - Verified "yarn rmadmin -refreshQueues" works when scheduler configuration
> API disabled, and does not work when scheduler configuration API enabled
>
>
> Jonathan Hung
>
> On Tue, Dec 12, 2017 at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get
> > chance to validate new RC now.
> >
> > Basically, I found two critical issues with the same rolling upgrade
> > scenario as where HADOOP-15059 get found previously:
> > HDFS-12920, we changed value format for some hdfs configurations that old
> > version MR client doesn't understand when fetching these configurations.
> > Some quick workarounds are to add old value (without time unit) in
> > hdfs-site.xml to override new default values but will generate many
> > annoying warnings. I provided my fix suggestions on the JIRA already for
> > more discussion.
> > The other one is YARN-7646. After we workaround HDFS-12920, will hit the
> > issue that old version MR AppMaster cannot communicate with new version
> of
> > YARN RM - could be related to resource profile changes from YARN side but
> > root cause are still in investigation.
> >
> > The first issue may not belong to a blocker given we can workaround this
> > without code change. I am not sure if we can workaround 2nd issue so far.
> > If not, we may have to fix this or compromise with withdrawing support of
> > rolling upgrade or calling it a stable release.
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Robert Kanter <rk...@cloudera.com>
> > Sent: Tuesday, December 12, 2017 3:10 PM
> > To: Arun Suresh
> > Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T.
> > Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
> >
> > +1 (binding)
> >
> > + Downloaded the binary release
> > + Deployed on a 3 node cluster on CentOS 7.3
> > + Ran some MR jobs, clicked around the UI, etc
> > + Ran some CLI commands (yarn logs, etc)
> >
> > Good job everyone on Hadoop 3!
> >
> >
> > - Robert
> >
> > On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
> >
> > > +1 (binding)
> > >
> > > - Verified signatures of the source tarball.
> > > - built from source - using the docker build environment.
> > > - set up a pseudo-distributed test cluster.
> > > - ran basic HDFS commands
> > > - ran some basic MR jobs
> > >
> > > Cheers
> > > -Arun
> > >
> > > On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <andrew.wang@cloudera.com
> >
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > As a reminder, this vote closes tomorrow at 12:31pm, so please give
> it
> > a
> > > > whack if you have time. There are already enough binding +1s to pass
> > this
> > > > vote, but it'd be great to get additional validation.
> > > >
> > > > Thanks to everyone who's voted thus far!
> > > >
> > > > Best,
> > > > Andrew
> > > >
> > > >
> > > >
> > > > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > * Verified src tarball and bin tarball, verified md5 of each.
> > > > > * Build source with -Pdist,native
> > > > > * Started a pseudo cluster
> > > > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run
> hdfs
> > > > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > > > >
> > > > > Thanks Andrew for this great effort!
> > > > >
> > > > > Best,
> > > > >
> > > > >
> > > > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <
> > andrew.wang@cloudera.com
> > > >
> > > > > wrote:
> > > > > > Hi Wei-Chiu,
> > > > > >
> > > > > > The patchprocess directory is left over from the create-release
> > > > process,
> > > > > > and it looks empty to me. We should still file a create-release
> > JIRA
> > > to
> > > > > fix
> > > > > > this, but I think this is not a blocker. Would you agree?
> > > > > >
> > > > > > Best,
> > > > > > Andrew
> > > > > >
> > > > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> > > weichiu@cloudera.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Andrew, thanks the tremendous effort.
> > > > > >> I found an empty "patchprocess" directory in the source tarball,
> > > that
> > > > is
> > > > > >> not there if you clone from github. Any chance you might have
> some
> > > > > leftover
> > > > > >> trash when you made the tarball?
> > > > > >> Not wanting to nitpicking, but you might want to double check so
> > we
> > > > > don't
> > > > > >> ship anything private to you in public :)
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > > > ajay.kumar@hortonworks.com
> > > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >>> +1 (non-binding)
> > > > > >>> Thanks for driving this, Andrew Wang!!
> > > > > >>>
> > > > > >>> - downloaded the src tarball and verified md5 checksum
> > > > > >>> - built from source with jdk 1.8.0_111-b14
> > > > > >>> - brought up a pseudo distributed cluster
> > > > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > > > >>> confirmed that everything was working
> > > > > >>> - Run word count, pi and DFSIOTest
> > > > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > > > >>>
> > > > > >>> Cheers,
> > > > > >>> Ajay
> > > > > >>>
> > > > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > > > >>>
> > > > > >>>     +1 (binding)
> > > > > >>>
> > > > > >>>     - downloaded src tarball, verified md5
> > > > > >>>     - built from source with jdk1.8.0_112
> > > > > >>>     - started a pseudo cluster with hdfs and kms
> > > > > >>>     - sanity checked encryption related operations working
> > > > > >>>     - sanity checked webui and logs.
> > > > > >>>
> > > > > >>>     -Xiao
> > > > > >>>
> > > > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> > > atm@apache.org>
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>>     > +1 (binding)
> > > > > >>>     >
> > > > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > > > -Pnative)
> > > > > >>>     > - verified the checksum
> > > > > >>>     > - brought up a secure pseudo distributed cluster
> > > > > >>>     > - did some basic file system operations (mkdir, list,
> put,
> > > cat)
> > > > > and
> > > > > >>>     > confirmed that everything was working
> > > > > >>>     > - confirmed that the web UI worked
> > > > > >>>     >
> > > > > >>>     > Best,
> > > > > >>>     > Aaron
> > > > > >>>     >
> > > > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > > > >>> andrew.wang@cloudera.com>
> > > > > >>>     > wrote:
> > > > > >>>     >
> > > > > >>>     > > Hi all,
> > > > > >>>     > >
> > > > > >>>     > > Let me start, as always, by thanking the efforts of all
> > the
> > > > > >>> contributors
> > > > > >>>     > > who contributed to this release, especially those who
> > > jumped
> > > > on
> > > > > >>> the
> > > > > >>>     > issues
> > > > > >>>     > > found in RC0.
> > > > > >>>     > >
> > > > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > > > >>> incorporates 302
> > > > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > > > >>>     > >
> > > > > >>>     > > You can find the artifacts here:
> > > > > >>>     > >
> > > > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > > > >>>     > >
> > > > > >>>     > > I've done the traditional testing of building from the
> > > source
> > > > > >>> tarball and
> > > > > >>>     > > running a Pi job on a single node cluster. I also
> > verified
> > > > that
> > > > > >>> the
> > > > > >>>     > shaded
> > > > > >>>     > > jars are not empty.
> > > > > >>>     > >
> > > > > >>>     > > Found one issue that create-release (probably due to
> the
> > > mvn
> > > > > >>> deploy
> > > > > >>>     > change)
> > > > > >>>     > > didn't sign the artifacts, but I fixed that by calling
> > mvn
> > > > one
> > > > > >>> more time.
> > > > > >>>     > > Available here:
> > > > > >>>     > >
> > > > > >>>     > > https://repository.apache.org/
> > > content/repositories/orgapache
> > > > > >>> hadoop-1075/
> > > > > >>>     > >
> > > > > >>>     > > This release will run the standard 5 days, closing on
> Dec
> > > > 13th
> > > > > at
> > > > > >>> 12:31pm
> > > > > >>>     > > Pacific. My +1 to start.
> > > > > >>>     > >
> > > > > >>>     > > Best,
> > > > > >>>     > > Andrew
> > > > > >>>     > >
> > > > > >>>     >
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>> ------------------------------------------------------------
> > > > ---------
> > > > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.
> apache.org
> > > > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.
> > org
> > > > > >>>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Lei (Eddy) Xu
> > > > > Software Engineer, Cloudera
> > > > >
> > > >
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sangjin Lee <sj...@apache.org>.

+1 (binding)

- downloaded the binary tarball and the source tarball and checked
signatures
- verified that the source builds cleanly
- verified that the shaded client jars are correct
- checked the basic pseudo-distributed cluster set-up and checked UI and
logs (hdfs and YARN)
- ran some test jobs successfully
- enabled Timeline Service v.2 and tested the writer/reader with test jobs


On Tue, Dec 12, 2017 at 6:14 PM, Jonathan Hung <jy...@gmail.com> wrote:

> Thanks Andrew for the huge effort.
>
> +1 (non-binding)
> - Downloaded binary tarball and verified md5
> - Ran RM HA and verified manual failover
> - Verified add/remove/update scheduler configuration API (CLI/REST) works
> for leveldb/zookeeper backend
> - Verified scheduler configuration changes persisted on restart/failover
> - Verified "yarn rmadmin -refreshQueues" works when scheduler configuration
> API disabled, and does not work when scheduler configuration API enabled
>
>
> Jonathan Hung
>
> On Tue, Dec 12, 2017 at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get
> > chance to validate new RC now.
> >
> > Basically, I found two critical issues with the same rolling upgrade
> > scenario as where HADOOP-15059 get found previously:
> > HDFS-12920, we changed value format for some hdfs configurations that old
> > version MR client doesn't understand when fetching these configurations.
> > Some quick workarounds are to add old value (without time unit) in
> > hdfs-site.xml to override new default values but will generate many
> > annoying warnings. I provided my fix suggestions on the JIRA already for
> > more discussion.
> > The other one is YARN-7646. After we workaround HDFS-12920, will hit the
> > issue that old version MR AppMaster cannot communicate with new version
> of
> > YARN RM - could be related to resource profile changes from YARN side but
> > root cause are still in investigation.
> >
> > The first issue may not belong to a blocker given we can workaround this
> > without code change. I am not sure if we can workaround 2nd issue so far.
> > If not, we may have to fix this or compromise with withdrawing support of
> > rolling upgrade or calling it a stable release.
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Robert Kanter <rk...@cloudera.com>
> > Sent: Tuesday, December 12, 2017 3:10 PM
> > To: Arun Suresh
> > Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T.
> > Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
> >
> > +1 (binding)
> >
> > + Downloaded the binary release
> > + Deployed on a 3 node cluster on CentOS 7.3
> > + Ran some MR jobs, clicked around the UI, etc
> > + Ran some CLI commands (yarn logs, etc)
> >
> > Good job everyone on Hadoop 3!
> >
> >
> > - Robert
> >
> > On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
> >
> > > +1 (binding)
> > >
> > > - Verified signatures of the source tarball.
> > > - built from source - using the docker build environment.
> > > - set up a pseudo-distributed test cluster.
> > > - ran basic HDFS commands
> > > - ran some basic MR jobs
> > >
> > > Cheers
> > > -Arun
> > >
> > > On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <andrew.wang@cloudera.com
> >
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > As a reminder, this vote closes tomorrow at 12:31pm, so please give
> it
> > a
> > > > whack if you have time. There are already enough binding +1s to pass
> > this
> > > > vote, but it'd be great to get additional validation.
> > > >
> > > > Thanks to everyone who's voted thus far!
> > > >
> > > > Best,
> > > > Andrew
> > > >
> > > >
> > > >
> > > > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > * Verified src tarball and bin tarball, verified md5 of each.
> > > > > * Build source with -Pdist,native
> > > > > * Started a pseudo cluster
> > > > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run
> hdfs
> > > > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > > > >
> > > > > Thanks Andrew for this great effort!
> > > > >
> > > > > Best,
> > > > >
> > > > >
> > > > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <
> > andrew.wang@cloudera.com
> > > >
> > > > > wrote:
> > > > > > Hi Wei-Chiu,
> > > > > >
> > > > > > The patchprocess directory is left over from the create-release
> > > > process,
> > > > > > and it looks empty to me. We should still file a create-release
> > JIRA
> > > to
> > > > > fix
> > > > > > this, but I think this is not a blocker. Would you agree?
> > > > > >
> > > > > > Best,
> > > > > > Andrew
> > > > > >
> > > > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> > > weichiu@cloudera.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Andrew, thanks the tremendous effort.
> > > > > >> I found an empty "patchprocess" directory in the source tarball,
> > > that
> > > > is
> > > > > >> not there if you clone from github. Any chance you might have
> some
> > > > > leftover
> > > > > >> trash when you made the tarball?
> > > > > >> Not wanting to nitpicking, but you might want to double check so
> > we
> > > > > don't
> > > > > >> ship anything private to you in public :)
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > > > ajay.kumar@hortonworks.com
> > > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >>> +1 (non-binding)
> > > > > >>> Thanks for driving this, Andrew Wang!!
> > > > > >>>
> > > > > >>> - downloaded the src tarball and verified md5 checksum
> > > > > >>> - built from source with jdk 1.8.0_111-b14
> > > > > >>> - brought up a pseudo distributed cluster
> > > > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > > > >>> confirmed that everything was working
> > > > > >>> - Run word count, pi and DFSIOTest
> > > > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > > > >>>
> > > > > >>> Cheers,
> > > > > >>> Ajay
> > > > > >>>
> > > > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > > > >>>
> > > > > >>>     +1 (binding)
> > > > > >>>
> > > > > >>>     - downloaded src tarball, verified md5
> > > > > >>>     - built from source with jdk1.8.0_112
> > > > > >>>     - started a pseudo cluster with hdfs and kms
> > > > > >>>     - sanity checked encryption related operations working
> > > > > >>>     - sanity checked webui and logs.
> > > > > >>>
> > > > > >>>     -Xiao
> > > > > >>>
> > > > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> > > atm@apache.org>
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>>     > +1 (binding)
> > > > > >>>     >
> > > > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > > > -Pnative)
> > > > > >>>     > - verified the checksum
> > > > > >>>     > - brought up a secure pseudo distributed cluster
> > > > > >>>     > - did some basic file system operations (mkdir, list,
> put,
> > > cat)
> > > > > and
> > > > > >>>     > confirmed that everything was working
> > > > > >>>     > - confirmed that the web UI worked
> > > > > >>>     >
> > > > > >>>     > Best,
> > > > > >>>     > Aaron
> > > > > >>>     >
> > > > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > > > >>> andrew.wang@cloudera.com>
> > > > > >>>     > wrote:
> > > > > >>>     >
> > > > > >>>     > > Hi all,
> > > > > >>>     > >
> > > > > >>>     > > Let me start, as always, by thanking the efforts of all
> > the
> > > > > >>> contributors
> > > > > >>>     > > who contributed to this release, especially those who
> > > jumped
> > > > on
> > > > > >>> the
> > > > > >>>     > issues
> > > > > >>>     > > found in RC0.
> > > > > >>>     > >
> > > > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > > > >>> incorporates 302
> > > > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > > > >>>     > >
> > > > > >>>     > > You can find the artifacts here:
> > > > > >>>     > >
> > > > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > > > >>>     > >
> > > > > >>>     > > I've done the traditional testing of building from the
> > > source
> > > > > >>> tarball and
> > > > > >>>     > > running a Pi job on a single node cluster. I also
> > verified
> > > > that
> > > > > >>> the
> > > > > >>>     > shaded
> > > > > >>>     > > jars are not empty.
> > > > > >>>     > >
> > > > > >>>     > > Found one issue that create-release (probably due to
> the
> > > mvn
> > > > > >>> deploy
> > > > > >>>     > change)
> > > > > >>>     > > didn't sign the artifacts, but I fixed that by calling
> > mvn
> > > > one
> > > > > >>> more time.
> > > > > >>>     > > Available here:
> > > > > >>>     > >
> > > > > >>>     > > https://repository.apache.org/
> > > content/repositories/orgapache
> > > > > >>> hadoop-1075/
> > > > > >>>     > >
> > > > > >>>     > > This release will run the standard 5 days, closing on
> Dec
> > > > 13th
> > > > > at
> > > > > >>> 12:31pm
> > > > > >>>     > > Pacific. My +1 to start.
> > > > > >>>     > >
> > > > > >>>     > > Best,
> > > > > >>>     > > Andrew
> > > > > >>>     > >
> > > > > >>>     >
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>> ------------------------------------------------------------
> > > > ---------
> > > > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.
> apache.org
> > > > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.
> > org
> > > > > >>>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Lei (Eddy) Xu
> > > > > Software Engineer, Cloudera
> > > > >
> > > >
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sangjin Lee <sj...@apache.org>.

+1 (binding)

- downloaded the binary tarball and the source tarball and checked
signatures
- verified that the source builds cleanly
- verified that the shaded client jars are correct
- checked the basic pseudo-distributed cluster set-up and checked UI and
logs (hdfs and YARN)
- ran some test jobs successfully
- enabled Timeline Service v.2 and tested the writer/reader with test jobs


On Tue, Dec 12, 2017 at 6:14 PM, Jonathan Hung <jy...@gmail.com> wrote:

> Thanks Andrew for the huge effort.
>
> +1 (non-binding)
> - Downloaded binary tarball and verified md5
> - Ran RM HA and verified manual failover
> - Verified add/remove/update scheduler configuration API (CLI/REST) works
> for leveldb/zookeeper backend
> - Verified scheduler configuration changes persisted on restart/failover
> - Verified "yarn rmadmin -refreshQueues" works when scheduler configuration
> API disabled, and does not work when scheduler configuration API enabled
>
>
> Jonathan Hung
>
> On Tue, Dec 12, 2017 at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get
> > chance to validate new RC now.
> >
> > Basically, I found two critical issues with the same rolling upgrade
> > scenario as where HADOOP-15059 get found previously:
> > HDFS-12920, we changed value format for some hdfs configurations that old
> > version MR client doesn't understand when fetching these configurations.
> > Some quick workarounds are to add old value (without time unit) in
> > hdfs-site.xml to override new default values but will generate many
> > annoying warnings. I provided my fix suggestions on the JIRA already for
> > more discussion.
> > The other one is YARN-7646. After we workaround HDFS-12920, will hit the
> > issue that old version MR AppMaster cannot communicate with new version
> of
> > YARN RM - could be related to resource profile changes from YARN side but
> > root cause are still in investigation.
> >
> > The first issue may not belong to a blocker given we can workaround this
> > without code change. I am not sure if we can workaround 2nd issue so far.
> > If not, we may have to fix this or compromise with withdrawing support of
> > rolling upgrade or calling it a stable release.
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Robert Kanter <rk...@cloudera.com>
> > Sent: Tuesday, December 12, 2017 3:10 PM
> > To: Arun Suresh
> > Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T.
> > Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
> >
> > +1 (binding)
> >
> > + Downloaded the binary release
> > + Deployed on a 3 node cluster on CentOS 7.3
> > + Ran some MR jobs, clicked around the UI, etc
> > + Ran some CLI commands (yarn logs, etc)
> >
> > Good job everyone on Hadoop 3!
> >
> >
> > - Robert
> >
> > On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
> >
> > > +1 (binding)
> > >
> > > - Verified signatures of the source tarball.
> > > - built from source - using the docker build environment.
> > > - set up a pseudo-distributed test cluster.
> > > - ran basic HDFS commands
> > > - ran some basic MR jobs
> > >
> > > Cheers
> > > -Arun
> > >
> > > On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <andrew.wang@cloudera.com
> >
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > As a reminder, this vote closes tomorrow at 12:31pm, so please give
> it
> > a
> > > > whack if you have time. There are already enough binding +1s to pass
> > this
> > > > vote, but it'd be great to get additional validation.
> > > >
> > > > Thanks to everyone who's voted thus far!
> > > >
> > > > Best,
> > > > Andrew
> > > >
> > > >
> > > >
> > > > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > * Verified src tarball and bin tarball, verified md5 of each.
> > > > > * Build source with -Pdist,native
> > > > > * Started a pseudo cluster
> > > > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run
> hdfs
> > > > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > > > >
> > > > > Thanks Andrew for this great effort!
> > > > >
> > > > > Best,
> > > > >
> > > > >
> > > > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <
> > andrew.wang@cloudera.com
> > > >
> > > > > wrote:
> > > > > > Hi Wei-Chiu,
> > > > > >
> > > > > > The patchprocess directory is left over from the create-release
> > > > process,
> > > > > > and it looks empty to me. We should still file a create-release
> > JIRA
> > > to
> > > > > fix
> > > > > > this, but I think this is not a blocker. Would you agree?
> > > > > >
> > > > > > Best,
> > > > > > Andrew
> > > > > >
> > > > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> > > weichiu@cloudera.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Andrew, thanks the tremendous effort.
> > > > > >> I found an empty "patchprocess" directory in the source tarball,
> > > that
> > > > is
> > > > > >> not there if you clone from github. Any chance you might have
> some
> > > > > leftover
> > > > > >> trash when you made the tarball?
> > > > > >> Not wanting to nitpicking, but you might want to double check so
> > we
> > > > > don't
> > > > > >> ship anything private to you in public :)
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > > > ajay.kumar@hortonworks.com
> > > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >>> +1 (non-binding)
> > > > > >>> Thanks for driving this, Andrew Wang!!
> > > > > >>>
> > > > > >>> - downloaded the src tarball and verified md5 checksum
> > > > > >>> - built from source with jdk 1.8.0_111-b14
> > > > > >>> - brought up a pseudo distributed cluster
> > > > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > > > >>> confirmed that everything was working
> > > > > >>> - Run word count, pi and DFSIOTest
> > > > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > > > >>>
> > > > > >>> Cheers,
> > > > > >>> Ajay
> > > > > >>>
> > > > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > > > >>>
> > > > > >>>     +1 (binding)
> > > > > >>>
> > > > > >>>     - downloaded src tarball, verified md5
> > > > > >>>     - built from source with jdk1.8.0_112
> > > > > >>>     - started a pseudo cluster with hdfs and kms
> > > > > >>>     - sanity checked encryption related operations working
> > > > > >>>     - sanity checked webui and logs.
> > > > > >>>
> > > > > >>>     -Xiao
> > > > > >>>
> > > > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> > > atm@apache.org>
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>>     > +1 (binding)
> > > > > >>>     >
> > > > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > > > -Pnative)
> > > > > >>>     > - verified the checksum
> > > > > >>>     > - brought up a secure pseudo distributed cluster
> > > > > >>>     > - did some basic file system operations (mkdir, list,
> put,
> > > cat)
> > > > > and
> > > > > >>>     > confirmed that everything was working
> > > > > >>>     > - confirmed that the web UI worked
> > > > > >>>     >
> > > > > >>>     > Best,
> > > > > >>>     > Aaron
> > > > > >>>     >
> > > > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > > > >>> andrew.wang@cloudera.com>
> > > > > >>>     > wrote:
> > > > > >>>     >
> > > > > >>>     > > Hi all,
> > > > > >>>     > >
> > > > > >>>     > > Let me start, as always, by thanking the efforts of all
> > the
> > > > > >>> contributors
> > > > > >>>     > > who contributed to this release, especially those who
> > > jumped
> > > > on
> > > > > >>> the
> > > > > >>>     > issues
> > > > > >>>     > > found in RC0.
> > > > > >>>     > >
> > > > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > > > >>> incorporates 302
> > > > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > > > >>>     > >
> > > > > >>>     > > You can find the artifacts here:
> > > > > >>>     > >
> > > > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > > > >>>     > >
> > > > > >>>     > > I've done the traditional testing of building from the
> > > source
> > > > > >>> tarball and
> > > > > >>>     > > running a Pi job on a single node cluster. I also
> > verified
> > > > that
> > > > > >>> the
> > > > > >>>     > shaded
> > > > > >>>     > > jars are not empty.
> > > > > >>>     > >
> > > > > >>>     > > Found one issue that create-release (probably due to
> the
> > > mvn
> > > > > >>> deploy
> > > > > >>>     > change)
> > > > > >>>     > > didn't sign the artifacts, but I fixed that by calling
> > mvn
> > > > one
> > > > > >>> more time.
> > > > > >>>     > > Available here:
> > > > > >>>     > >
> > > > > >>>     > > https://repository.apache.org/
> > > content/repositories/orgapache
> > > > > >>> hadoop-1075/
> > > > > >>>     > >
> > > > > >>>     > > This release will run the standard 5 days, closing on
> Dec
> > > > 13th
> > > > > at
> > > > > >>> 12:31pm
> > > > > >>>     > > Pacific. My +1 to start.
> > > > > >>>     > >
> > > > > >>>     > > Best,
> > > > > >>>     > > Andrew
> > > > > >>>     > >
> > > > > >>>     >
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>> ------------------------------------------------------------
> > > > ---------
> > > > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.
> apache.org
> > > > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.
> > org
> > > > > >>>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Lei (Eddy) Xu
> > > > > Software Engineer, Cloudera
> > > > >
> > > >
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Jonathan Hung <jy...@gmail.com>.

Thanks Andrew for the huge effort.

+1 (non-binding)
- Downloaded binary tarball and verified md5
- Ran RM HA and verified manual failover
- Verified add/remove/update scheduler configuration API (CLI/REST) works
for leveldb/zookeeper backend
- Verified scheduler configuration changes persisted on restart/failover
- Verified "yarn rmadmin -refreshQueues" works when scheduler configuration
API disabled, and does not work when scheduler configuration API enabled


Jonathan Hung

On Tue, Dec 12, 2017 at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:

> Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get
> chance to validate new RC now.
>
> Basically, I found two critical issues with the same rolling upgrade
> scenario as where HADOOP-15059 get found previously:
> HDFS-12920, we changed value format for some hdfs configurations that old
> version MR client doesn't understand when fetching these configurations.
> Some quick workarounds are to add old value (without time unit) in
> hdfs-site.xml to override new default values but will generate many
> annoying warnings. I provided my fix suggestions on the JIRA already for
> more discussion.
> The other one is YARN-7646. After we workaround HDFS-12920, will hit the
> issue that old version MR AppMaster cannot communicate with new version of
> YARN RM - could be related to resource profile changes from YARN side but
> root cause are still in investigation.
>
> The first issue may not belong to a blocker given we can workaround this
> without code change. I am not sure if we can workaround 2nd issue so far.
> If not, we may have to fix this or compromise with withdrawing support of
> rolling upgrade or calling it a stable release.
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Robert Kanter <rk...@cloudera.com>
> Sent: Tuesday, December 12, 2017 3:10 PM
> To: Arun Suresh
> Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T.
> Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
>
> +1 (binding)
>
> + Downloaded the binary release
> + Deployed on a 3 node cluster on CentOS 7.3
> + Ran some MR jobs, clicked around the UI, etc
> + Ran some CLI commands (yarn logs, etc)
>
> Good job everyone on Hadoop 3!
>
>
> - Robert
>
> On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
>
> > +1 (binding)
> >
> > - Verified signatures of the source tarball.
> > - built from source - using the docker build environment.
> > - set up a pseudo-distributed test cluster.
> > - ran basic HDFS commands
> > - ran some basic MR jobs
> >
> > Cheers
> > -Arun
> >
> > On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> > wrote:
> >
> > > Hi everyone,
> > >
> > > As a reminder, this vote closes tomorrow at 12:31pm, so please give it
> a
> > > whack if you have time. There are already enough binding +1s to pass
> this
> > > vote, but it'd be great to get additional validation.
> > >
> > > Thanks to everyone who's voted thus far!
> > >
> > > Best,
> > > Andrew
> > >
> > >
> > >
> > > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > * Verified src tarball and bin tarball, verified md5 of each.
> > > > * Build source with -Pdist,native
> > > > * Started a pseudo cluster
> > > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > > >
> > > > Thanks Andrew for this great effort!
> > > >
> > > > Best,
> > > >
> > > >
> > > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <
> andrew.wang@cloudera.com
> > >
> > > > wrote:
> > > > > Hi Wei-Chiu,
> > > > >
> > > > > The patchprocess directory is left over from the create-release
> > > process,
> > > > > and it looks empty to me. We should still file a create-release
> JIRA
> > to
> > > > fix
> > > > > this, but I think this is not a blocker. Would you agree?
> > > > >
> > > > > Best,
> > > > > Andrew
> > > > >
> > > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> > weichiu@cloudera.com
> > > >
> > > > > wrote:
> > > > >
> > > > >> Hi Andrew, thanks the tremendous effort.
> > > > >> I found an empty "patchprocess" directory in the source tarball,
> > that
> > > is
> > > > >> not there if you clone from github. Any chance you might have some
> > > > leftover
> > > > >> trash when you made the tarball?
> > > > >> Not wanting to nitpicking, but you might want to double check so
> we
> > > > don't
> > > > >> ship anything private to you in public :)
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > > ajay.kumar@hortonworks.com
> > > > >
> > > > >> wrote:
> > > > >>
> > > > >>> +1 (non-binding)
> > > > >>> Thanks for driving this, Andrew Wang!!
> > > > >>>
> > > > >>> - downloaded the src tarball and verified md5 checksum
> > > > >>> - built from source with jdk 1.8.0_111-b14
> > > > >>> - brought up a pseudo distributed cluster
> > > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > > >>> confirmed that everything was working
> > > > >>> - Run word count, pi and DFSIOTest
> > > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > > >>>
> > > > >>> Cheers,
> > > > >>> Ajay
> > > > >>>
> > > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > > >>>
> > > > >>>     +1 (binding)
> > > > >>>
> > > > >>>     - downloaded src tarball, verified md5
> > > > >>>     - built from source with jdk1.8.0_112
> > > > >>>     - started a pseudo cluster with hdfs and kms
> > > > >>>     - sanity checked encryption related operations working
> > > > >>>     - sanity checked webui and logs.
> > > > >>>
> > > > >>>     -Xiao
> > > > >>>
> > > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> > atm@apache.org>
> > > > >>> wrote:
> > > > >>>
> > > > >>>     > +1 (binding)
> > > > >>>     >
> > > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > > -Pnative)
> > > > >>>     > - verified the checksum
> > > > >>>     > - brought up a secure pseudo distributed cluster
> > > > >>>     > - did some basic file system operations (mkdir, list, put,
> > cat)
> > > > and
> > > > >>>     > confirmed that everything was working
> > > > >>>     > - confirmed that the web UI worked
> > > > >>>     >
> > > > >>>     > Best,
> > > > >>>     > Aaron
> > > > >>>     >
> > > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > > >>> andrew.wang@cloudera.com>
> > > > >>>     > wrote:
> > > > >>>     >
> > > > >>>     > > Hi all,
> > > > >>>     > >
> > > > >>>     > > Let me start, as always, by thanking the efforts of all
> the
> > > > >>> contributors
> > > > >>>     > > who contributed to this release, especially those who
> > jumped
> > > on
> > > > >>> the
> > > > >>>     > issues
> > > > >>>     > > found in RC0.
> > > > >>>     > >
> > > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > > >>> incorporates 302
> > > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > > >>>     > >
> > > > >>>     > > You can find the artifacts here:
> > > > >>>     > >
> > > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > > >>>     > >
> > > > >>>     > > I've done the traditional testing of building from the
> > source
> > > > >>> tarball and
> > > > >>>     > > running a Pi job on a single node cluster. I also
> verified
> > > that
> > > > >>> the
> > > > >>>     > shaded
> > > > >>>     > > jars are not empty.
> > > > >>>     > >
> > > > >>>     > > Found one issue that create-release (probably due to the
> > mvn
> > > > >>> deploy
> > > > >>>     > change)
> > > > >>>     > > didn't sign the artifacts, but I fixed that by calling
> mvn
> > > one
> > > > >>> more time.
> > > > >>>     > > Available here:
> > > > >>>     > >
> > > > >>>     > > https://repository.apache.org/
> > content/repositories/orgapache
> > > > >>> hadoop-1075/
> > > > >>>     > >
> > > > >>>     > > This release will run the standard 5 days, closing on Dec
> > > 13th
> > > > at
> > > > >>> 12:31pm
> > > > >>>     > > Pacific. My +1 to start.
> > > > >>>     > >
> > > > >>>     > > Best,
> > > > >>>     > > Andrew
> > > > >>>     > >
> > > > >>>     >
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> ------------------------------------------------------------
> > > ---------
> > > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.
> org
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Lei (Eddy) Xu
> > > > Software Engineer, Cloudera
> > > >
> > >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Jonathan Hung <jy...@gmail.com>.

Thanks Andrew for the huge effort.

+1 (non-binding)
- Downloaded binary tarball and verified md5
- Ran RM HA and verified manual failover
- Verified add/remove/update scheduler configuration API (CLI/REST) works
for leveldb/zookeeper backend
- Verified scheduler configuration changes persisted on restart/failover
- Verified "yarn rmadmin -refreshQueues" works when scheduler configuration
API disabled, and does not work when scheduler configuration API enabled


Jonathan Hung

On Tue, Dec 12, 2017 at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:

> Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get
> chance to validate new RC now.
>
> Basically, I found two critical issues with the same rolling upgrade
> scenario as where HADOOP-15059 get found previously:
> HDFS-12920, we changed value format for some hdfs configurations that old
> version MR client doesn't understand when fetching these configurations.
> Some quick workarounds are to add old value (without time unit) in
> hdfs-site.xml to override new default values but will generate many
> annoying warnings. I provided my fix suggestions on the JIRA already for
> more discussion.
> The other one is YARN-7646. After we workaround HDFS-12920, will hit the
> issue that old version MR AppMaster cannot communicate with new version of
> YARN RM - could be related to resource profile changes from YARN side but
> root cause are still in investigation.
>
> The first issue may not belong to a blocker given we can workaround this
> without code change. I am not sure if we can workaround 2nd issue so far.
> If not, we may have to fix this or compromise with withdrawing support of
> rolling upgrade or calling it a stable release.
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Robert Kanter <rk...@cloudera.com>
> Sent: Tuesday, December 12, 2017 3:10 PM
> To: Arun Suresh
> Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T.
> Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
>
> +1 (binding)
>
> + Downloaded the binary release
> + Deployed on a 3 node cluster on CentOS 7.3
> + Ran some MR jobs, clicked around the UI, etc
> + Ran some CLI commands (yarn logs, etc)
>
> Good job everyone on Hadoop 3!
>
>
> - Robert
>
> On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
>
> > +1 (binding)
> >
> > - Verified signatures of the source tarball.
> > - built from source - using the docker build environment.
> > - set up a pseudo-distributed test cluster.
> > - ran basic HDFS commands
> > - ran some basic MR jobs
> >
> > Cheers
> > -Arun
> >
> > On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> > wrote:
> >
> > > Hi everyone,
> > >
> > > As a reminder, this vote closes tomorrow at 12:31pm, so please give it
> a
> > > whack if you have time. There are already enough binding +1s to pass
> this
> > > vote, but it'd be great to get additional validation.
> > >
> > > Thanks to everyone who's voted thus far!
> > >
> > > Best,
> > > Andrew
> > >
> > >
> > >
> > > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > * Verified src tarball and bin tarball, verified md5 of each.
> > > > * Build source with -Pdist,native
> > > > * Started a pseudo cluster
> > > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > > >
> > > > Thanks Andrew for this great effort!
> > > >
> > > > Best,
> > > >
> > > >
> > > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <
> andrew.wang@cloudera.com
> > >
> > > > wrote:
> > > > > Hi Wei-Chiu,
> > > > >
> > > > > The patchprocess directory is left over from the create-release
> > > process,
> > > > > and it looks empty to me. We should still file a create-release
> JIRA
> > to
> > > > fix
> > > > > this, but I think this is not a blocker. Would you agree?
> > > > >
> > > > > Best,
> > > > > Andrew
> > > > >
> > > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> > weichiu@cloudera.com
> > > >
> > > > > wrote:
> > > > >
> > > > >> Hi Andrew, thanks the tremendous effort.
> > > > >> I found an empty "patchprocess" directory in the source tarball,
> > that
> > > is
> > > > >> not there if you clone from github. Any chance you might have some
> > > > leftover
> > > > >> trash when you made the tarball?
> > > > >> Not wanting to nitpicking, but you might want to double check so
> we
> > > > don't
> > > > >> ship anything private to you in public :)
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > > ajay.kumar@hortonworks.com
> > > > >
> > > > >> wrote:
> > > > >>
> > > > >>> +1 (non-binding)
> > > > >>> Thanks for driving this, Andrew Wang!!
> > > > >>>
> > > > >>> - downloaded the src tarball and verified md5 checksum
> > > > >>> - built from source with jdk 1.8.0_111-b14
> > > > >>> - brought up a pseudo distributed cluster
> > > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > > >>> confirmed that everything was working
> > > > >>> - Run word count, pi and DFSIOTest
> > > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > > >>>
> > > > >>> Cheers,
> > > > >>> Ajay
> > > > >>>
> > > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > > >>>
> > > > >>>     +1 (binding)
> > > > >>>
> > > > >>>     - downloaded src tarball, verified md5
> > > > >>>     - built from source with jdk1.8.0_112
> > > > >>>     - started a pseudo cluster with hdfs and kms
> > > > >>>     - sanity checked encryption related operations working
> > > > >>>     - sanity checked webui and logs.
> > > > >>>
> > > > >>>     -Xiao
> > > > >>>
> > > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> > atm@apache.org>
> > > > >>> wrote:
> > > > >>>
> > > > >>>     > +1 (binding)
> > > > >>>     >
> > > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > > -Pnative)
> > > > >>>     > - verified the checksum
> > > > >>>     > - brought up a secure pseudo distributed cluster
> > > > >>>     > - did some basic file system operations (mkdir, list, put,
> > cat)
> > > > and
> > > > >>>     > confirmed that everything was working
> > > > >>>     > - confirmed that the web UI worked
> > > > >>>     >
> > > > >>>     > Best,
> > > > >>>     > Aaron
> > > > >>>     >
> > > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > > >>> andrew.wang@cloudera.com>
> > > > >>>     > wrote:
> > > > >>>     >
> > > > >>>     > > Hi all,
> > > > >>>     > >
> > > > >>>     > > Let me start, as always, by thanking the efforts of all
> the
> > > > >>> contributors
> > > > >>>     > > who contributed to this release, especially those who
> > jumped
> > > on
> > > > >>> the
> > > > >>>     > issues
> > > > >>>     > > found in RC0.
> > > > >>>     > >
> > > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > > >>> incorporates 302
> > > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > > >>>     > >
> > > > >>>     > > You can find the artifacts here:
> > > > >>>     > >
> > > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > > >>>     > >
> > > > >>>     > > I've done the traditional testing of building from the
> > source
> > > > >>> tarball and
> > > > >>>     > > running a Pi job on a single node cluster. I also
> verified
> > > that
> > > > >>> the
> > > > >>>     > shaded
> > > > >>>     > > jars are not empty.
> > > > >>>     > >
> > > > >>>     > > Found one issue that create-release (probably due to the
> > mvn
> > > > >>> deploy
> > > > >>>     > change)
> > > > >>>     > > didn't sign the artifacts, but I fixed that by calling
> mvn
> > > one
> > > > >>> more time.
> > > > >>>     > > Available here:
> > > > >>>     > >
> > > > >>>     > > https://repository.apache.org/
> > content/repositories/orgapache
> > > > >>> hadoop-1075/
> > > > >>>     > >
> > > > >>>     > > This release will run the standard 5 days, closing on Dec
> > > 13th
> > > > at
> > > > >>> 12:31pm
> > > > >>>     > > Pacific. My +1 to start.
> > > > >>>     > >
> > > > >>>     > > Best,
> > > > >>>     > > Andrew
> > > > >>>     > >
> > > > >>>     >
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> ------------------------------------------------------------
> > > ---------
> > > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.
> org
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Lei (Eddy) Xu
> > > > Software Engineer, Cloudera
> > > >
> > >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

Good stuff Andrew, and thanks everyone!

+Vinod

> On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com> wrote:
> 
> To close this out, the vote passes successfully with 13 binding +1s, 5 non-binding +1s, and no -1s. Thanks everyone for voting! I'll work on staging.
>

回复：[VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by "郑锴(铁杰)" <zh...@alibaba-inc.com>.

Thanks Andrew for the hard driving!!
I downloaded the source tar bao and built it successfully on a MacOS. I haven't got the chance to try it yet, so mynon-binding +1. 
Regards,Kai
------------------------------------------------------------------发件人：John Zhuge <jo...@gmail.com>发送时间：2017年12月14日(星期四) 11:06收件人：Vinod Kumar Vavilapalli <vi...@apache.org>抄　送：Andrew Wang <an...@cloudera.com>; Junping Du <jd...@hortonworks.com>; Robert Kanter <rk...@cloudera.com>; Arun Suresh <as...@apache.org>; Lei Xu <le...@cloudera.com>; Wei-Chiu Chuang <we...@cloudera.com>; Ajay Kumar <aj...@hortonworks.com>; Xiao Chen <xi...@cloudera.com>; Aaron T. Myers <at...@apache.org>; common-dev@hadoop.apache.org <co...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org <hd...@hadoop.apache.org>; yarn-dev@hadoop.apache.org <ya...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org <ma...@hadoop.apache.org>主　题：Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
Thanks Andrew for the great effort! Here is my late vote.


+1 (binding)

   - Verified checksums and signatures of tarballs
   - Built source with native, Oracle Java 1.8.0_152 on Mac OS X 10.13.2
   - Verified cloud connectors:
      - S3A integration tests (perf tests skipped)
   - Deployed both binary and built source to a pseudo cluster, passed the
   following sanity tests in insecure and SSL mode:
      - HDFS basic and ACL
      - DistCp basic
      - MapReduce wordcount
      - KMS and HttpFS basic
      - Balancer start/stop


On Wed, Dec 13, 2017 at 6:12 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:

> Yes, JIRAs will be filed, the wiki-page idea from YARN meetup is to record
> all combinations of testing that need to be done and correspondingly
> capture all the testing that someone in the community has already done and
> record it for future perusal.
>
> From what you are saying, I guess we haven't advertised to the public yet
> on rolling upgrades, but in our meetups etc so far, you have been saying
> that rolling upgrades is supported - so I assumed we did put it in our
> messaging.
>
> The important question is if we are or are not allowed to make potentially
> incompatible changes to fix bugs in the process of supporting 2.x to 3.x
> upgrades whether rolling or not.
>
> +Vinod
>
> > On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com>
> wrote:
> >
> > I'm hoping we can address YARN-7588 and any remaining rolling upgrade
> issues in 3.0.x maintenance releases. Beyond a wiki page, it would be
> really great to get JIRAs filed and targeted for tracking as soon as
> possible.
> >
> > Vinod, what do you think we need to do regarding caveating rolling
> upgrade support? We haven't advertised rolling upgrade support between
> major releases outside of dev lists and JIRA. As a new major release, our
> compat guidelines allow us to break compatibility, so I don't think it's
> expected by users.
> >
>
>


-- 
John

回复：[VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by "郑锴(铁杰)" <zh...@alibaba-inc.com>.

Thanks Andrew for the hard driving!!
I downloaded the source tar bao and built it successfully on a MacOS. I haven't got the chance to try it yet, so mynon-binding +1. 
Regards,Kai
------------------------------------------------------------------发件人：John Zhuge <jo...@gmail.com>发送时间：2017年12月14日(星期四) 11:06收件人：Vinod Kumar Vavilapalli <vi...@apache.org>抄　送：Andrew Wang <an...@cloudera.com>; Junping Du <jd...@hortonworks.com>; Robert Kanter <rk...@cloudera.com>; Arun Suresh <as...@apache.org>; Lei Xu <le...@cloudera.com>; Wei-Chiu Chuang <we...@cloudera.com>; Ajay Kumar <aj...@hortonworks.com>; Xiao Chen <xi...@cloudera.com>; Aaron T. Myers <at...@apache.org>; common-dev@hadoop.apache.org <co...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org <hd...@hadoop.apache.org>; yarn-dev@hadoop.apache.org <ya...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org <ma...@hadoop.apache.org>主　题：Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
Thanks Andrew for the great effort! Here is my late vote.


+1 (binding)

   - Verified checksums and signatures of tarballs
   - Built source with native, Oracle Java 1.8.0_152 on Mac OS X 10.13.2
   - Verified cloud connectors:
      - S3A integration tests (perf tests skipped)
   - Deployed both binary and built source to a pseudo cluster, passed the
   following sanity tests in insecure and SSL mode:
      - HDFS basic and ACL
      - DistCp basic
      - MapReduce wordcount
      - KMS and HttpFS basic
      - Balancer start/stop


On Wed, Dec 13, 2017 at 6:12 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:

> Yes, JIRAs will be filed, the wiki-page idea from YARN meetup is to record
> all combinations of testing that need to be done and correspondingly
> capture all the testing that someone in the community has already done and
> record it for future perusal.
>
> From what you are saying, I guess we haven't advertised to the public yet
> on rolling upgrades, but in our meetups etc so far, you have been saying
> that rolling upgrades is supported - so I assumed we did put it in our
> messaging.
>
> The important question is if we are or are not allowed to make potentially
> incompatible changes to fix bugs in the process of supporting 2.x to 3.x
> upgrades whether rolling or not.
>
> +Vinod
>
> > On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com>
> wrote:
> >
> > I'm hoping we can address YARN-7588 and any remaining rolling upgrade
> issues in 3.0.x maintenance releases. Beyond a wiki page, it would be
> really great to get JIRAs filed and targeted for tracking as soon as
> possible.
> >
> > Vinod, what do you think we need to do regarding caveating rolling
> upgrade support? We haven't advertised rolling upgrade support between
> major releases outside of dev lists and JIRA. As a new major release, our
> compat guidelines allow us to break compatibility, so I don't think it's
> expected by users.
> >
>
>


-- 
John

回复：[VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by "郑锴(铁杰)" <zh...@alibaba-inc.com>.

Thanks Andrew for the hard driving!!
I downloaded the source tar bao and built it successfully on a MacOS. I haven't got the chance to try it yet, so mynon-binding +1. 
Regards,Kai
------------------------------------------------------------------发件人：John Zhuge <jo...@gmail.com>发送时间：2017年12月14日(星期四) 11:06收件人：Vinod Kumar Vavilapalli <vi...@apache.org>抄　送：Andrew Wang <an...@cloudera.com>; Junping Du <jd...@hortonworks.com>; Robert Kanter <rk...@cloudera.com>; Arun Suresh <as...@apache.org>; Lei Xu <le...@cloudera.com>; Wei-Chiu Chuang <we...@cloudera.com>; Ajay Kumar <aj...@hortonworks.com>; Xiao Chen <xi...@cloudera.com>; Aaron T. Myers <at...@apache.org>; common-dev@hadoop.apache.org <co...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org <hd...@hadoop.apache.org>; yarn-dev@hadoop.apache.org <ya...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org <ma...@hadoop.apache.org>主　题：Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
Thanks Andrew for the great effort! Here is my late vote.


+1 (binding)

   - Verified checksums and signatures of tarballs
   - Built source with native, Oracle Java 1.8.0_152 on Mac OS X 10.13.2
   - Verified cloud connectors:
      - S3A integration tests (perf tests skipped)
   - Deployed both binary and built source to a pseudo cluster, passed the
   following sanity tests in insecure and SSL mode:
      - HDFS basic and ACL
      - DistCp basic
      - MapReduce wordcount
      - KMS and HttpFS basic
      - Balancer start/stop


On Wed, Dec 13, 2017 at 6:12 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:

> Yes, JIRAs will be filed, the wiki-page idea from YARN meetup is to record
> all combinations of testing that need to be done and correspondingly
> capture all the testing that someone in the community has already done and
> record it for future perusal.
>
> From what you are saying, I guess we haven't advertised to the public yet
> on rolling upgrades, but in our meetups etc so far, you have been saying
> that rolling upgrades is supported - so I assumed we did put it in our
> messaging.
>
> The important question is if we are or are not allowed to make potentially
> incompatible changes to fix bugs in the process of supporting 2.x to 3.x
> upgrades whether rolling or not.
>
> +Vinod
>
> > On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com>
> wrote:
> >
> > I'm hoping we can address YARN-7588 and any remaining rolling upgrade
> issues in 3.0.x maintenance releases. Beyond a wiki page, it would be
> really great to get JIRAs filed and targeted for tracking as soon as
> possible.
> >
> > Vinod, what do you think we need to do regarding caveating rolling
> upgrade support? We haven't advertised rolling upgrade support between
> major releases outside of dev lists and JIRA. As a new major release, our
> compat guidelines allow us to break compatibility, so I don't think it's
> expected by users.
> >
>
>


-- 
John

回复：[VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by "郑锴(铁杰)" <zh...@alibaba-inc.com>.

Thanks Andrew for the hard driving!!
I downloaded the source tar bao and built it successfully on a MacOS. I haven't got the chance to try it yet, so mynon-binding +1. 
Regards,Kai
------------------------------------------------------------------发件人：John Zhuge <jo...@gmail.com>发送时间：2017年12月14日(星期四) 11:06收件人：Vinod Kumar Vavilapalli <vi...@apache.org>抄　送：Andrew Wang <an...@cloudera.com>; Junping Du <jd...@hortonworks.com>; Robert Kanter <rk...@cloudera.com>; Arun Suresh <as...@apache.org>; Lei Xu <le...@cloudera.com>; Wei-Chiu Chuang <we...@cloudera.com>; Ajay Kumar <aj...@hortonworks.com>; Xiao Chen <xi...@cloudera.com>; Aaron T. Myers <at...@apache.org>; common-dev@hadoop.apache.org <co...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org <hd...@hadoop.apache.org>; yarn-dev@hadoop.apache.org <ya...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org <ma...@hadoop.apache.org>主　题：Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
Thanks Andrew for the great effort! Here is my late vote.


+1 (binding)

   - Verified checksums and signatures of tarballs
   - Built source with native, Oracle Java 1.8.0_152 on Mac OS X 10.13.2
   - Verified cloud connectors:
      - S3A integration tests (perf tests skipped)
   - Deployed both binary and built source to a pseudo cluster, passed the
   following sanity tests in insecure and SSL mode:
      - HDFS basic and ACL
      - DistCp basic
      - MapReduce wordcount
      - KMS and HttpFS basic
      - Balancer start/stop


On Wed, Dec 13, 2017 at 6:12 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:

> Yes, JIRAs will be filed, the wiki-page idea from YARN meetup is to record
> all combinations of testing that need to be done and correspondingly
> capture all the testing that someone in the community has already done and
> record it for future perusal.
>
> From what you are saying, I guess we haven't advertised to the public yet
> on rolling upgrades, but in our meetups etc so far, you have been saying
> that rolling upgrades is supported - so I assumed we did put it in our
> messaging.
>
> The important question is if we are or are not allowed to make potentially
> incompatible changes to fix bugs in the process of supporting 2.x to 3.x
> upgrades whether rolling or not.
>
> +Vinod
>
> > On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com>
> wrote:
> >
> > I'm hoping we can address YARN-7588 and any remaining rolling upgrade
> issues in 3.0.x maintenance releases. Beyond a wiki page, it would be
> really great to get JIRAs filed and targeted for tracking as soon as
> possible.
> >
> > Vinod, what do you think we need to do regarding caveating rolling
> upgrade support? We haven't advertised rolling upgrade support between
> major releases outside of dev lists and JIRA. As a new major release, our
> compat guidelines allow us to break compatibility, so I don't think it's
> expected by users.
> >
>
>


-- 
John

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by John Zhuge <jo...@gmail.com>.

Thanks Andrew for the great effort! Here is my late vote.


+1 (binding)

   - Verified checksums and signatures of tarballs
   - Built source with native, Oracle Java 1.8.0_152 on Mac OS X 10.13.2
   - Verified cloud connectors:
      - S3A integration tests (perf tests skipped)
   - Deployed both binary and built source to a pseudo cluster, passed the
   following sanity tests in insecure and SSL mode:
      - HDFS basic and ACL
      - DistCp basic
      - MapReduce wordcount
      - KMS and HttpFS basic
      - Balancer start/stop


On Wed, Dec 13, 2017 at 6:12 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:

> Yes, JIRAs will be filed, the wiki-page idea from YARN meetup is to record
> all combinations of testing that need to be done and correspondingly
> capture all the testing that someone in the community has already done and
> record it for future perusal.
>
> From what you are saying, I guess we haven't advertised to the public yet
> on rolling upgrades, but in our meetups etc so far, you have been saying
> that rolling upgrades is supported - so I assumed we did put it in our
> messaging.
>
> The important question is if we are or are not allowed to make potentially
> incompatible changes to fix bugs in the process of supporting 2.x to 3.x
> upgrades whether rolling or not.
>
> +Vinod
>
> > On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com>
> wrote:
> >
> > I'm hoping we can address YARN-7588 and any remaining rolling upgrade
> issues in 3.0.x maintenance releases. Beyond a wiki page, it would be
> really great to get JIRAs filed and targeted for tracking as soon as
> possible.
> >
> > Vinod, what do you think we need to do regarding caveating rolling
> upgrade support? We haven't advertised rolling upgrade support between
> major releases outside of dev lists and JIRA. As a new major release, our
> compat guidelines allow us to break compatibility, so I don't think it's
> expected by users.
> >
>
>


-- 
John

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by John Zhuge <jo...@gmail.com>.

Thanks Andrew for the great effort! Here is my late vote.


+1 (binding)

   - Verified checksums and signatures of tarballs
   - Built source with native, Oracle Java 1.8.0_152 on Mac OS X 10.13.2
   - Verified cloud connectors:
      - S3A integration tests (perf tests skipped)
   - Deployed both binary and built source to a pseudo cluster, passed the
   following sanity tests in insecure and SSL mode:
      - HDFS basic and ACL
      - DistCp basic
      - MapReduce wordcount
      - KMS and HttpFS basic
      - Balancer start/stop


On Wed, Dec 13, 2017 at 6:12 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:

> Yes, JIRAs will be filed, the wiki-page idea from YARN meetup is to record
> all combinations of testing that need to be done and correspondingly
> capture all the testing that someone in the community has already done and
> record it for future perusal.
>
> From what you are saying, I guess we haven't advertised to the public yet
> on rolling upgrades, but in our meetups etc so far, you have been saying
> that rolling upgrades is supported - so I assumed we did put it in our
> messaging.
>
> The important question is if we are or are not allowed to make potentially
> incompatible changes to fix bugs in the process of supporting 2.x to 3.x
> upgrades whether rolling or not.
>
> +Vinod
>
> > On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com>
> wrote:
> >
> > I'm hoping we can address YARN-7588 and any remaining rolling upgrade
> issues in 3.0.x maintenance releases. Beyond a wiki page, it would be
> really great to get JIRAs filed and targeted for tracking as soon as
> possible.
> >
> > Vinod, what do you think we need to do regarding caveating rolling
> upgrade support? We haven't advertised rolling upgrade support between
> major releases outside of dev lists and JIRA. As a new major release, our
> compat guidelines allow us to break compatibility, so I don't think it's
> expected by users.
> >
>
>


-- 
John

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by John Zhuge <jo...@gmail.com>.

Thanks Andrew for the great effort! Here is my late vote.


+1 (binding)

   - Verified checksums and signatures of tarballs
   - Built source with native, Oracle Java 1.8.0_152 on Mac OS X 10.13.2
   - Verified cloud connectors:
      - S3A integration tests (perf tests skipped)
   - Deployed both binary and built source to a pseudo cluster, passed the
   following sanity tests in insecure and SSL mode:
      - HDFS basic and ACL
      - DistCp basic
      - MapReduce wordcount
      - KMS and HttpFS basic
      - Balancer start/stop


On Wed, Dec 13, 2017 at 6:12 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:

> Yes, JIRAs will be filed, the wiki-page idea from YARN meetup is to record
> all combinations of testing that need to be done and correspondingly
> capture all the testing that someone in the community has already done and
> record it for future perusal.
>
> From what you are saying, I guess we haven't advertised to the public yet
> on rolling upgrades, but in our meetups etc so far, you have been saying
> that rolling upgrades is supported - so I assumed we did put it in our
> messaging.
>
> The important question is if we are or are not allowed to make potentially
> incompatible changes to fix bugs in the process of supporting 2.x to 3.x
> upgrades whether rolling or not.
>
> +Vinod
>
> > On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com>
> wrote:
> >
> > I'm hoping we can address YARN-7588 and any remaining rolling upgrade
> issues in 3.0.x maintenance releases. Beyond a wiki page, it would be
> really great to get JIRAs filed and targeted for tracking as soon as
> possible.
> >
> > Vinod, what do you think we need to do regarding caveating rolling
> upgrade support? We haven't advertised rolling upgrade support between
> major releases outside of dev lists and JIRA. As a new major release, our
> compat guidelines allow us to break compatibility, so I don't think it's
> expected by users.
> >
>
>


-- 
John

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by John Zhuge <jo...@gmail.com>.

Thanks Andrew for the great effort! Here is my late vote.


+1 (binding)

   - Verified checksums and signatures of tarballs
   - Built source with native, Oracle Java 1.8.0_152 on Mac OS X 10.13.2
   - Verified cloud connectors:
      - S3A integration tests (perf tests skipped)
   - Deployed both binary and built source to a pseudo cluster, passed the
   following sanity tests in insecure and SSL mode:
      - HDFS basic and ACL
      - DistCp basic
      - MapReduce wordcount
      - KMS and HttpFS basic
      - Balancer start/stop


On Wed, Dec 13, 2017 at 6:12 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:

> Yes, JIRAs will be filed, the wiki-page idea from YARN meetup is to record
> all combinations of testing that need to be done and correspondingly
> capture all the testing that someone in the community has already done and
> record it for future perusal.
>
> From what you are saying, I guess we haven't advertised to the public yet
> on rolling upgrades, but in our meetups etc so far, you have been saying
> that rolling upgrades is supported - so I assumed we did put it in our
> messaging.
>
> The important question is if we are or are not allowed to make potentially
> incompatible changes to fix bugs in the process of supporting 2.x to 3.x
> upgrades whether rolling or not.
>
> +Vinod
>
> > On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com>
> wrote:
> >
> > I'm hoping we can address YARN-7588 and any remaining rolling upgrade
> issues in 3.0.x maintenance releases. Beyond a wiki page, it would be
> really great to get JIRAs filed and targeted for tracking as soon as
> possible.
> >
> > Vinod, what do you think we need to do regarding caveating rolling
> upgrade support? We haven't advertised rolling upgrade support between
> major releases outside of dev lists and JIRA. As a new major release, our
> compat guidelines allow us to break compatibility, so I don't think it's
> expected by users.
> >
>
>


-- 
John

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

Yes, JIRAs will be filed, the wiki-page idea from YARN meetup is to record all combinations of testing that need to be done and correspondingly capture all the testing that someone in the community has already done and record it for future perusal.

From what you are saying, I guess we haven't advertised to the public yet on rolling upgrades, but in our meetups etc so far, you have been saying that rolling upgrades is supported - so I assumed we did put it in our messaging.

The important question is if we are or are not allowed to make potentially incompatible changes to fix bugs in the process of supporting 2.x to 3.x upgrades whether rolling or not.

+Vinod

> On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com> wrote:
> 
> I'm hoping we can address YARN-7588 and any remaining rolling upgrade issues in 3.0.x maintenance releases. Beyond a wiki page, it would be really great to get JIRAs filed and targeted for tracking as soon as possible.
> 
> Vinod, what do you think we need to do regarding caveating rolling upgrade support? We haven't advertised rolling upgrade support between major releases outside of dev lists and JIRA. As a new major release, our compat guidelines allow us to break compatibility, so I don't think it's expected by users.
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

Yes, JIRAs will be filed, the wiki-page idea from YARN meetup is to record all combinations of testing that need to be done and correspondingly capture all the testing that someone in the community has already done and record it for future perusal.

From what you are saying, I guess we haven't advertised to the public yet on rolling upgrades, but in our meetups etc so far, you have been saying that rolling upgrades is supported - so I assumed we did put it in our messaging.

The important question is if we are or are not allowed to make potentially incompatible changes to fix bugs in the process of supporting 2.x to 3.x upgrades whether rolling or not.

+Vinod

> On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com> wrote:
> 
> I'm hoping we can address YARN-7588 and any remaining rolling upgrade issues in 3.0.x maintenance releases. Beyond a wiki page, it would be really great to get JIRAs filed and targeted for tracking as soon as possible.
> 
> Vinod, what do you think we need to do regarding caveating rolling upgrade support? We haven't advertised rolling upgrade support between major releases outside of dev lists and JIRA. As a new major release, our compat guidelines allow us to break compatibility, so I don't think it's expected by users.
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

Good stuff Andrew, and thanks everyone!

+Vinod

> On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com> wrote:
> 
> To close this out, the vote passes successfully with 13 binding +1s, 5 non-binding +1s, and no -1s. Thanks everyone for voting! I'll work on staging.
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by "Gangumalla, Uma" <um...@intel.com>.

Here is my +1(binding) too. 
Sorry for late vote.

Verified signatures of the source tarball.
built from source.
set up a 2-node test cluster.
Tested via HDFS commands and java API – Written bunch of files and read back. 
Ran basic MR job

Thanks Andrew and others for the hard work for getting Hadoop 3.0 out.

Regards,
Uma

On 12/13/17, 1:05 PM, "Andrew Wang" <an...@cloudera.com> wrote:

    Hi folks,
    
    To close this out, the vote passes successfully with 13 binding +1s, 5
    non-binding +1s, and no -1s. Thanks everyone for voting! I'll work on
    staging.
    
    I'm hoping we can address YARN-7588 and any remaining rolling upgrade
    issues in 3.0.x maintenance releases. Beyond a wiki page, it would be
    really great to get JIRAs filed and targeted for tracking as soon as
    possible.
    
    Vinod, what do you think we need to do regarding caveating rolling upgrade
    support? We haven't advertised rolling upgrade support between major
    releases outside of dev lists and JIRA. As a new major release, our compat
    guidelines allow us to break compatibility, so I don't think it's expected
    by users.
    
    Best,
    Andrew
    
    On Wed, Dec 13, 2017 at 12:37 PM, Vinod Kumar Vavilapalli <
    vinodkv@apache.org> wrote:
    
    > I was waiting for Daniel to post the minutes from YARN meetup to talk
    > about this. Anyways, in that discussion, we identified a bunch of key
    > upgrade related scenarios that no-one seems to have validated - atleast
    > from the representation in the YARN meetup. I'm going to create a wiki-page
    > listing all these scenarios.
    >
    > But back to the bug that Junping raised. At this point, we don't have a
    > clear path towards running 2.x applications on 3.0.0 clusters. So, our
    > claim of rolling-upgrades already working is not accurate.
    >
    > One of the two options that Junping proposed should be pursued before we
    > close the release. I'm in favor of calling out rolling-upgrade support be
    > with-drawn or caveated and push for progress instead of blocking the
    > release.
    >
    > Thanks
    > +Vinod
    >
    > > On Dec 12, 2017, at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:
    > >
    > > Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just
    > get chance to validate new RC now.
    > >
    > > Basically, I found two critical issues with the same rolling upgrade
    > scenario as where HADOOP-15059 get found previously:
    > > HDFS-12920, we changed value format for some hdfs configurations that
    > old version MR client doesn't understand when fetching these
    > configurations. Some quick workarounds are to add old value (without time
    > unit) in hdfs-site.xml to override new default values but will generate
    > many annoying warnings. I provided my fix suggestions on the JIRA already
    > for more discussion.
    > > The other one is YARN-7646. After we workaround HDFS-12920, will hit the
    > issue that old version MR AppMaster cannot communicate with new version of
    > YARN RM - could be related to resource profile changes from YARN side but
    > root cause are still in investigation.
    > >
    > > The first issue may not belong to a blocker given we can workaround this
    > without code change. I am not sure if we can workaround 2nd issue so far.
    > If not, we may have to fix this or compromise with withdrawing support of
    > rolling upgrade or calling it a stable release.
    > >
    > >
    > > Thanks,
    > >
    > > Junping
    > >
    > > ________________________________________
    > > From: Robert Kanter <rk...@cloudera.com>
    > > Sent: Tuesday, December 12, 2017 3:10 PM
    > > To: Arun Suresh
    > > Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron
    > T. Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
    > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
    > > Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
    > >
    > > +1 (binding)
    > >
    > > + Downloaded the binary release
    > > + Deployed on a 3 node cluster on CentOS 7.3
    > > + Ran some MR jobs, clicked around the UI, etc
    > > + Ran some CLI commands (yarn logs, etc)
    > >
    > > Good job everyone on Hadoop 3!
    > >
    > >
    > > - Robert
    > >
    > > On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
    > >
    > >> +1 (binding)
    > >>
    > >> - Verified signatures of the source tarball.
    > >> - built from source - using the docker build environment.
    > >> - set up a pseudo-distributed test cluster.
    > >> - ran basic HDFS commands
    > >> - ran some basic MR jobs
    > >>
    > >> Cheers
    > >> -Arun
    > >>
    > >> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
    > >> wrote:
    > >>
    > >>> Hi everyone,
    > >>>
    > >>> As a reminder, this vote closes tomorrow at 12:31pm, so please give it
    > a
    > >>> whack if you have time. There are already enough binding +1s to pass
    > this
    > >>> vote, but it'd be great to get additional validation.
    > >>>
    > >>> Thanks to everyone who's voted thus far!
    > >>>
    > >>> Best,
    > >>> Andrew
    > >>>
    > >>>
    > >>>
    > >>> On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
    > >>>
    > >>>> +1 (binding)
    > >>>>
    > >>>> * Verified src tarball and bin tarball, verified md5 of each.
    > >>>> * Build source with -Pdist,native
    > >>>> * Started a pseudo cluster
    > >>>> * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
    > >>>> dfs put/get/cat on "/" with XOR-2-1 policy.
    > >>>>
    > >>>> Thanks Andrew for this great effort!
    > >>>>
    > >>>> Best,
    > >>>>
    > >>>>
    > >>>> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <
    > andrew.wang@cloudera.com
    > >>>
    > >>>> wrote:
    > >>>>> Hi Wei-Chiu,
    > >>>>>
    > >>>>> The patchprocess directory is left over from the create-release
    > >>> process,
    > >>>>> and it looks empty to me. We should still file a create-release JIRA
    > >> to
    > >>>> fix
    > >>>>> this, but I think this is not a blocker. Would you agree?
    > >>>>>
    > >>>>> Best,
    > >>>>> Andrew
    > >>>>>
    > >>>>> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
    > >> weichiu@cloudera.com
    > >>>>
    > >>>>> wrote:
    > >>>>>
    > >>>>>> Hi Andrew, thanks the tremendous effort.
    > >>>>>> I found an empty "patchprocess" directory in the source tarball,
    > >> that
    > >>> is
    > >>>>>> not there if you clone from github. Any chance you might have some
    > >>>> leftover
    > >>>>>> trash when you made the tarball?
    > >>>>>> Not wanting to nitpicking, but you might want to double check so we
    > >>>> don't
    > >>>>>> ship anything private to you in public :)
    > >>>>>>
    > >>>>>>
    > >>>>>>
    > >>>>>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
    > >>> ajay.kumar@hortonworks.com
    > >>>>>
    > >>>>>> wrote:
    > >>>>>>
    > >>>>>>> +1 (non-binding)
    > >>>>>>> Thanks for driving this, Andrew Wang!!
    > >>>>>>>
    > >>>>>>> - downloaded the src tarball and verified md5 checksum
    > >>>>>>> - built from source with jdk 1.8.0_111-b14
    > >>>>>>> - brought up a pseudo distributed cluster
    > >>>>>>> - did basic file system operations (mkdir, list, put, cat) and
    > >>>>>>> confirmed that everything was working
    > >>>>>>> - Run word count, pi and DFSIOTest
    > >>>>>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
    > >>>>>>>
    > >>>>>>> Cheers,
    > >>>>>>> Ajay
    > >>>>>>>
    > >>>>>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
    > >>>>>>>
    > >>>>>>>    +1 (binding)
    > >>>>>>>
    > >>>>>>>    - downloaded src tarball, verified md5
    > >>>>>>>    - built from source with jdk1.8.0_112
    > >>>>>>>    - started a pseudo cluster with hdfs and kms
    > >>>>>>>    - sanity checked encryption related operations working
    > >>>>>>>    - sanity checked webui and logs.
    > >>>>>>>
    > >>>>>>>    -Xiao
    > >>>>>>>
    > >>>>>>>    On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
    > >> atm@apache.org>
    > >>>>>>> wrote:
    > >>>>>>>
    > >>>>>>>> +1 (binding)
    > >>>>>>>>
    > >>>>>>>> - downloaded the src tarball and built the source (-Pdist
    > >>>> -Pnative)
    > >>>>>>>> - verified the checksum
    > >>>>>>>> - brought up a secure pseudo distributed cluster
    > >>>>>>>> - did some basic file system operations (mkdir, list, put,
    > >> cat)
    > >>>> and
    > >>>>>>>> confirmed that everything was working
    > >>>>>>>> - confirmed that the web UI worked
    > >>>>>>>>
    > >>>>>>>> Best,
    > >>>>>>>> Aaron
    > >>>>>>>>
    > >>>>>>>> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
    > >>>>>>> andrew.wang@cloudera.com>
    > >>>>>>>> wrote:
    > >>>>>>>>
    > >>>>>>>>> Hi all,
    > >>>>>>>>>
    > >>>>>>>>> Let me start, as always, by thanking the efforts of all the
    > >>>>>>> contributors
    > >>>>>>>>> who contributed to this release, especially those who
    > >> jumped
    > >>> on
    > >>>>>>> the
    > >>>>>>>> issues
    > >>>>>>>>> found in RC0.
    > >>>>>>>>>
    > >>>>>>>>> I've prepared RC1 for Apache Hadoop 3.0.0. This release
    > >>>>>>> incorporates 302
    > >>>>>>>>> fixed JIRAs since the previous 3.0.0-beta1 release.
    > >>>>>>>>>
    > >>>>>>>>> You can find the artifacts here:
    > >>>>>>>>>
    > >>>>>>>>> http://home.apache.org/~wang/3.0.0-RC1/
    > >>>>>>>>>
    > >>>>>>>>> I've done the traditional testing of building from the
    > >> source
    > >>>>>>> tarball and
    > >>>>>>>>> running a Pi job on a single node cluster. I also verified
    > >>> that
    > >>>>>>> the
    > >>>>>>>> shaded
    > >>>>>>>>> jars are not empty.
    > >>>>>>>>>
    > >>>>>>>>> Found one issue that create-release (probably due to the
    > >> mvn
    > >>>>>>> deploy
    > >>>>>>>> change)
    > >>>>>>>>> didn't sign the artifacts, but I fixed that by calling mvn
    > >>> one
    > >>>>>>> more time.
    > >>>>>>>>> Available here:
    > >>>>>>>>>
    > >>>>>>>>> https://repository.apache.org/
    > >> content/repositories/orgapache
    > >>>>>>> hadoop-1075/
    > >>>>>>>>>
    > >>>>>>>>> This release will run the standard 5 days, closing on Dec
    > >>> 13th
    > >>>> at
    > >>>>>>> 12:31pm
    > >>>>>>>>> Pacific. My +1 to start.
    > >>>>>>>>>
    > >>>>>>>>> Best,
    > >>>>>>>>> Andrew
    > >>>>>>>>>
    > >>>>>>>>
    > >>>>>>>
    > >>>>>>>
    > >>>>>>>
    > >>>>>>>
    > >>>>>>>
    > >>>>>>>
    > >>>>>>>
    > >>>>>>> ------------------------------------------------------------
    > >>> ---------
    > >>>>>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
    > >>>>>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
    > >>>>>>>
    > >>>>>>
    > >>>>>>
    > >>>>>>
    > >>>>>>
    > >>>>
    > >>>>
    > >>>>
    > >>>> --
    > >>>> Lei (Eddy) Xu
    > >>>> Software Engineer, Cloudera
    > >>>>
    > >>>
    > >>
    > >
    > >
    > > ---------------------------------------------------------------------
    > > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
    > > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
    > >
    >
    >

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by "Gangumalla, Uma" <um...@intel.com>.

Here is my +1(binding) too. 
Sorry for late vote.

Verified signatures of the source tarball.
built from source.
set up a 2-node test cluster.
Tested via HDFS commands and java API – Written bunch of files and read back. 
Ran basic MR job

Thanks Andrew and others for the hard work for getting Hadoop 3.0 out.

Regards,
Uma

On 12/13/17, 1:05 PM, "Andrew Wang" <an...@cloudera.com> wrote:

    Hi folks,
    
    To close this out, the vote passes successfully with 13 binding +1s, 5
    non-binding +1s, and no -1s. Thanks everyone for voting! I'll work on
    staging.
    
    I'm hoping we can address YARN-7588 and any remaining rolling upgrade
    issues in 3.0.x maintenance releases. Beyond a wiki page, it would be
    really great to get JIRAs filed and targeted for tracking as soon as
    possible.
    
    Vinod, what do you think we need to do regarding caveating rolling upgrade
    support? We haven't advertised rolling upgrade support between major
    releases outside of dev lists and JIRA. As a new major release, our compat
    guidelines allow us to break compatibility, so I don't think it's expected
    by users.
    
    Best,
    Andrew
    
    On Wed, Dec 13, 2017 at 12:37 PM, Vinod Kumar Vavilapalli <
    vinodkv@apache.org> wrote:
    
    > I was waiting for Daniel to post the minutes from YARN meetup to talk
    > about this. Anyways, in that discussion, we identified a bunch of key
    > upgrade related scenarios that no-one seems to have validated - atleast
    > from the representation in the YARN meetup. I'm going to create a wiki-page
    > listing all these scenarios.
    >
    > But back to the bug that Junping raised. At this point, we don't have a
    > clear path towards running 2.x applications on 3.0.0 clusters. So, our
    > claim of rolling-upgrades already working is not accurate.
    >
    > One of the two options that Junping proposed should be pursued before we
    > close the release. I'm in favor of calling out rolling-upgrade support be
    > with-drawn or caveated and push for progress instead of blocking the
    > release.
    >
    > Thanks
    > +Vinod
    >
    > > On Dec 12, 2017, at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:
    > >
    > > Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just
    > get chance to validate new RC now.
    > >
    > > Basically, I found two critical issues with the same rolling upgrade
    > scenario as where HADOOP-15059 get found previously:
    > > HDFS-12920, we changed value format for some hdfs configurations that
    > old version MR client doesn't understand when fetching these
    > configurations. Some quick workarounds are to add old value (without time
    > unit) in hdfs-site.xml to override new default values but will generate
    > many annoying warnings. I provided my fix suggestions on the JIRA already
    > for more discussion.
    > > The other one is YARN-7646. After we workaround HDFS-12920, will hit the
    > issue that old version MR AppMaster cannot communicate with new version of
    > YARN RM - could be related to resource profile changes from YARN side but
    > root cause are still in investigation.
    > >
    > > The first issue may not belong to a blocker given we can workaround this
    > without code change. I am not sure if we can workaround 2nd issue so far.
    > If not, we may have to fix this or compromise with withdrawing support of
    > rolling upgrade or calling it a stable release.
    > >
    > >
    > > Thanks,
    > >
    > > Junping
    > >
    > > ________________________________________
    > > From: Robert Kanter <rk...@cloudera.com>
    > > Sent: Tuesday, December 12, 2017 3:10 PM
    > > To: Arun Suresh
    > > Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron
    > T. Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
    > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
    > > Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
    > >
    > > +1 (binding)
    > >
    > > + Downloaded the binary release
    > > + Deployed on a 3 node cluster on CentOS 7.3
    > > + Ran some MR jobs, clicked around the UI, etc
    > > + Ran some CLI commands (yarn logs, etc)
    > >
    > > Good job everyone on Hadoop 3!
    > >
    > >
    > > - Robert
    > >
    > > On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
    > >
    > >> +1 (binding)
    > >>
    > >> - Verified signatures of the source tarball.
    > >> - built from source - using the docker build environment.
    > >> - set up a pseudo-distributed test cluster.
    > >> - ran basic HDFS commands
    > >> - ran some basic MR jobs
    > >>
    > >> Cheers
    > >> -Arun
    > >>
    > >> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
    > >> wrote:
    > >>
    > >>> Hi everyone,
    > >>>
    > >>> As a reminder, this vote closes tomorrow at 12:31pm, so please give it
    > a
    > >>> whack if you have time. There are already enough binding +1s to pass
    > this
    > >>> vote, but it'd be great to get additional validation.
    > >>>
    > >>> Thanks to everyone who's voted thus far!
    > >>>
    > >>> Best,
    > >>> Andrew
    > >>>
    > >>>
    > >>>
    > >>> On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
    > >>>
    > >>>> +1 (binding)
    > >>>>
    > >>>> * Verified src tarball and bin tarball, verified md5 of each.
    > >>>> * Build source with -Pdist,native
    > >>>> * Started a pseudo cluster
    > >>>> * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
    > >>>> dfs put/get/cat on "/" with XOR-2-1 policy.
    > >>>>
    > >>>> Thanks Andrew for this great effort!
    > >>>>
    > >>>> Best,
    > >>>>
    > >>>>
    > >>>> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <
    > andrew.wang@cloudera.com
    > >>>
    > >>>> wrote:
    > >>>>> Hi Wei-Chiu,
    > >>>>>
    > >>>>> The patchprocess directory is left over from the create-release
    > >>> process,
    > >>>>> and it looks empty to me. We should still file a create-release JIRA
    > >> to
    > >>>> fix
    > >>>>> this, but I think this is not a blocker. Would you agree?
    > >>>>>
    > >>>>> Best,
    > >>>>> Andrew
    > >>>>>
    > >>>>> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
    > >> weichiu@cloudera.com
    > >>>>
    > >>>>> wrote:
    > >>>>>
    > >>>>>> Hi Andrew, thanks the tremendous effort.
    > >>>>>> I found an empty "patchprocess" directory in the source tarball,
    > >> that
    > >>> is
    > >>>>>> not there if you clone from github. Any chance you might have some
    > >>>> leftover
    > >>>>>> trash when you made the tarball?
    > >>>>>> Not wanting to nitpicking, but you might want to double check so we
    > >>>> don't
    > >>>>>> ship anything private to you in public :)
    > >>>>>>
    > >>>>>>
    > >>>>>>
    > >>>>>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
    > >>> ajay.kumar@hortonworks.com
    > >>>>>
    > >>>>>> wrote:
    > >>>>>>
    > >>>>>>> +1 (non-binding)
    > >>>>>>> Thanks for driving this, Andrew Wang!!
    > >>>>>>>
    > >>>>>>> - downloaded the src tarball and verified md5 checksum
    > >>>>>>> - built from source with jdk 1.8.0_111-b14
    > >>>>>>> - brought up a pseudo distributed cluster
    > >>>>>>> - did basic file system operations (mkdir, list, put, cat) and
    > >>>>>>> confirmed that everything was working
    > >>>>>>> - Run word count, pi and DFSIOTest
    > >>>>>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
    > >>>>>>>
    > >>>>>>> Cheers,
    > >>>>>>> Ajay
    > >>>>>>>
    > >>>>>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
    > >>>>>>>
    > >>>>>>>    +1 (binding)
    > >>>>>>>
    > >>>>>>>    - downloaded src tarball, verified md5
    > >>>>>>>    - built from source with jdk1.8.0_112
    > >>>>>>>    - started a pseudo cluster with hdfs and kms
    > >>>>>>>    - sanity checked encryption related operations working
    > >>>>>>>    - sanity checked webui and logs.
    > >>>>>>>
    > >>>>>>>    -Xiao
    > >>>>>>>
    > >>>>>>>    On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
    > >> atm@apache.org>
    > >>>>>>> wrote:
    > >>>>>>>
    > >>>>>>>> +1 (binding)
    > >>>>>>>>
    > >>>>>>>> - downloaded the src tarball and built the source (-Pdist
    > >>>> -Pnative)
    > >>>>>>>> - verified the checksum
    > >>>>>>>> - brought up a secure pseudo distributed cluster
    > >>>>>>>> - did some basic file system operations (mkdir, list, put,
    > >> cat)
    > >>>> and
    > >>>>>>>> confirmed that everything was working
    > >>>>>>>> - confirmed that the web UI worked
    > >>>>>>>>
    > >>>>>>>> Best,
    > >>>>>>>> Aaron
    > >>>>>>>>
    > >>>>>>>> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
    > >>>>>>> andrew.wang@cloudera.com>
    > >>>>>>>> wrote:
    > >>>>>>>>
    > >>>>>>>>> Hi all,
    > >>>>>>>>>
    > >>>>>>>>> Let me start, as always, by thanking the efforts of all the
    > >>>>>>> contributors
    > >>>>>>>>> who contributed to this release, especially those who
    > >> jumped
    > >>> on
    > >>>>>>> the
    > >>>>>>>> issues
    > >>>>>>>>> found in RC0.
    > >>>>>>>>>
    > >>>>>>>>> I've prepared RC1 for Apache Hadoop 3.0.0. This release
    > >>>>>>> incorporates 302
    > >>>>>>>>> fixed JIRAs since the previous 3.0.0-beta1 release.
    > >>>>>>>>>
    > >>>>>>>>> You can find the artifacts here:
    > >>>>>>>>>
    > >>>>>>>>> http://home.apache.org/~wang/3.0.0-RC1/
    > >>>>>>>>>
    > >>>>>>>>> I've done the traditional testing of building from the
    > >> source
    > >>>>>>> tarball and
    > >>>>>>>>> running a Pi job on a single node cluster. I also verified
    > >>> that
    > >>>>>>> the
    > >>>>>>>> shaded
    > >>>>>>>>> jars are not empty.
    > >>>>>>>>>
    > >>>>>>>>> Found one issue that create-release (probably due to the
    > >> mvn
    > >>>>>>> deploy
    > >>>>>>>> change)
    > >>>>>>>>> didn't sign the artifacts, but I fixed that by calling mvn
    > >>> one
    > >>>>>>> more time.
    > >>>>>>>>> Available here:
    > >>>>>>>>>
    > >>>>>>>>> https://repository.apache.org/
    > >> content/repositories/orgapache
    > >>>>>>> hadoop-1075/
    > >>>>>>>>>
    > >>>>>>>>> This release will run the standard 5 days, closing on Dec
    > >>> 13th
    > >>>> at
    > >>>>>>> 12:31pm
    > >>>>>>>>> Pacific. My +1 to start.
    > >>>>>>>>>
    > >>>>>>>>> Best,
    > >>>>>>>>> Andrew
    > >>>>>>>>>
    > >>>>>>>>
    > >>>>>>>
    > >>>>>>>
    > >>>>>>>
    > >>>>>>>
    > >>>>>>>
    > >>>>>>>
    > >>>>>>>
    > >>>>>>> ------------------------------------------------------------
    > >>> ---------
    > >>>>>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
    > >>>>>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
    > >>>>>>>
    > >>>>>>
    > >>>>>>
    > >>>>>>
    > >>>>>>
    > >>>>
    > >>>>
    > >>>>
    > >>>> --
    > >>>> Lei (Eddy) Xu
    > >>>> Software Engineer, Cloudera
    > >>>>
    > >>>
    > >>
    > >
    > >
    > > ---------------------------------------------------------------------
    > > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
    > > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
    > >
    >
    >

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

Good stuff Andrew, and thanks everyone!

+Vinod

> On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com> wrote:
> 
> To close this out, the vote passes successfully with 13 binding +1s, 5 non-binding +1s, and no -1s. Thanks everyone for voting! I'll work on staging.
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

Yes, JIRAs will be filed, the wiki-page idea from YARN meetup is to record all combinations of testing that need to be done and correspondingly capture all the testing that someone in the community has already done and record it for future perusal.

From what you are saying, I guess we haven't advertised to the public yet on rolling upgrades, but in our meetups etc so far, you have been saying that rolling upgrades is supported - so I assumed we did put it in our messaging.

The important question is if we are or are not allowed to make potentially incompatible changes to fix bugs in the process of supporting 2.x to 3.x upgrades whether rolling or not.

+Vinod

> On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com> wrote:
> 
> I'm hoping we can address YARN-7588 and any remaining rolling upgrade issues in 3.0.x maintenance releases. Beyond a wiki page, it would be really great to get JIRAs filed and targeted for tracking as soon as possible.
> 
> Vinod, what do you think we need to do regarding caveating rolling upgrade support? We haven't advertised rolling upgrade support between major releases outside of dev lists and JIRA. As a new major release, our compat guidelines allow us to break compatibility, so I don't think it's expected by users.
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

Yes, JIRAs will be filed, the wiki-page idea from YARN meetup is to record all combinations of testing that need to be done and correspondingly capture all the testing that someone in the community has already done and record it for future perusal.

From what you are saying, I guess we haven't advertised to the public yet on rolling upgrades, but in our meetups etc so far, you have been saying that rolling upgrades is supported - so I assumed we did put it in our messaging.

The important question is if we are or are not allowed to make potentially incompatible changes to fix bugs in the process of supporting 2.x to 3.x upgrades whether rolling or not.

+Vinod

> On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com> wrote:
> 
> I'm hoping we can address YARN-7588 and any remaining rolling upgrade issues in 3.0.x maintenance releases. Beyond a wiki page, it would be really great to get JIRAs filed and targeted for tracking as soon as possible.
> 
> Vinod, what do you think we need to do regarding caveating rolling upgrade support? We haven't advertised rolling upgrade support between major releases outside of dev lists and JIRA. As a new major release, our compat guidelines allow us to break compatibility, so I don't think it's expected by users.
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

Good stuff Andrew, and thanks everyone!

+Vinod

> On Dec 13, 2017, at 1:05 PM, Andrew Wang <an...@cloudera.com> wrote:
> 
> To close this out, the vote passes successfully with 13 binding +1s, 5 non-binding +1s, and no -1s. Thanks everyone for voting! I'll work on staging.
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Hi folks,

To close this out, the vote passes successfully with 13 binding +1s, 5
non-binding +1s, and no -1s. Thanks everyone for voting! I'll work on
staging.

I'm hoping we can address YARN-7588 and any remaining rolling upgrade
issues in 3.0.x maintenance releases. Beyond a wiki page, it would be
really great to get JIRAs filed and targeted for tracking as soon as
possible.

Vinod, what do you think we need to do regarding caveating rolling upgrade
support? We haven't advertised rolling upgrade support between major
releases outside of dev lists and JIRA. As a new major release, our compat
guidelines allow us to break compatibility, so I don't think it's expected
by users.

Best,
Andrew

On Wed, Dec 13, 2017 at 12:37 PM, Vinod Kumar Vavilapalli <
vinodkv@apache.org> wrote:

> I was waiting for Daniel to post the minutes from YARN meetup to talk
> about this. Anyways, in that discussion, we identified a bunch of key
> upgrade related scenarios that no-one seems to have validated - atleast
> from the representation in the YARN meetup. I'm going to create a wiki-page
> listing all these scenarios.
>
> But back to the bug that Junping raised. At this point, we don't have a
> clear path towards running 2.x applications on 3.0.0 clusters. So, our
> claim of rolling-upgrades already working is not accurate.
>
> One of the two options that Junping proposed should be pursued before we
> close the release. I'm in favor of calling out rolling-upgrade support be
> with-drawn or caveated and push for progress instead of blocking the
> release.
>
> Thanks
> +Vinod
>
> > On Dec 12, 2017, at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:
> >
> > Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just
> get chance to validate new RC now.
> >
> > Basically, I found two critical issues with the same rolling upgrade
> scenario as where HADOOP-15059 get found previously:
> > HDFS-12920, we changed value format for some hdfs configurations that
> old version MR client doesn't understand when fetching these
> configurations. Some quick workarounds are to add old value (without time
> unit) in hdfs-site.xml to override new default values but will generate
> many annoying warnings. I provided my fix suggestions on the JIRA already
> for more discussion.
> > The other one is YARN-7646. After we workaround HDFS-12920, will hit the
> issue that old version MR AppMaster cannot communicate with new version of
> YARN RM - could be related to resource profile changes from YARN side but
> root cause are still in investigation.
> >
> > The first issue may not belong to a blocker given we can workaround this
> without code change. I am not sure if we can workaround 2nd issue so far.
> If not, we may have to fix this or compromise with withdrawing support of
> rolling upgrade or calling it a stable release.
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Robert Kanter <rk...@cloudera.com>
> > Sent: Tuesday, December 12, 2017 3:10 PM
> > To: Arun Suresh
> > Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron
> T. Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
> >
> > +1 (binding)
> >
> > + Downloaded the binary release
> > + Deployed on a 3 node cluster on CentOS 7.3
> > + Ran some MR jobs, clicked around the UI, etc
> > + Ran some CLI commands (yarn logs, etc)
> >
> > Good job everyone on Hadoop 3!
> >
> >
> > - Robert
> >
> > On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
> >
> >> +1 (binding)
> >>
> >> - Verified signatures of the source tarball.
> >> - built from source - using the docker build environment.
> >> - set up a pseudo-distributed test cluster.
> >> - ran basic HDFS commands
> >> - ran some basic MR jobs
> >>
> >> Cheers
> >> -Arun
> >>
> >> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> >> wrote:
> >>
> >>> Hi everyone,
> >>>
> >>> As a reminder, this vote closes tomorrow at 12:31pm, so please give it
> a
> >>> whack if you have time. There are already enough binding +1s to pass
> this
> >>> vote, but it'd be great to get additional validation.
> >>>
> >>> Thanks to everyone who's voted thus far!
> >>>
> >>> Best,
> >>> Andrew
> >>>
> >>>
> >>>
> >>> On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> >>>
> >>>> +1 (binding)
> >>>>
> >>>> * Verified src tarball and bin tarball, verified md5 of each.
> >>>> * Build source with -Pdist,native
> >>>> * Started a pseudo cluster
> >>>> * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> >>>> dfs put/get/cat on "/" with XOR-2-1 policy.
> >>>>
> >>>> Thanks Andrew for this great effort!
> >>>>
> >>>> Best,
> >>>>
> >>>>
> >>>> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <
> andrew.wang@cloudera.com
> >>>
> >>>> wrote:
> >>>>> Hi Wei-Chiu,
> >>>>>
> >>>>> The patchprocess directory is left over from the create-release
> >>> process,
> >>>>> and it looks empty to me. We should still file a create-release JIRA
> >> to
> >>>> fix
> >>>>> this, but I think this is not a blocker. Would you agree?
> >>>>>
> >>>>> Best,
> >>>>> Andrew
> >>>>>
> >>>>> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> >> weichiu@cloudera.com
> >>>>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi Andrew, thanks the tremendous effort.
> >>>>>> I found an empty "patchprocess" directory in the source tarball,
> >> that
> >>> is
> >>>>>> not there if you clone from github. Any chance you might have some
> >>>> leftover
> >>>>>> trash when you made the tarball?
> >>>>>> Not wanting to nitpicking, but you might want to double check so we
> >>>> don't
> >>>>>> ship anything private to you in public :)
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> >>> ajay.kumar@hortonworks.com
> >>>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> +1 (non-binding)
> >>>>>>> Thanks for driving this, Andrew Wang!!
> >>>>>>>
> >>>>>>> - downloaded the src tarball and verified md5 checksum
> >>>>>>> - built from source with jdk 1.8.0_111-b14
> >>>>>>> - brought up a pseudo distributed cluster
> >>>>>>> - did basic file system operations (mkdir, list, put, cat) and
> >>>>>>> confirmed that everything was working
> >>>>>>> - Run word count, pi and DFSIOTest
> >>>>>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>> Ajay
> >>>>>>>
> >>>>>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> >>>>>>>
> >>>>>>>    +1 (binding)
> >>>>>>>
> >>>>>>>    - downloaded src tarball, verified md5
> >>>>>>>    - built from source with jdk1.8.0_112
> >>>>>>>    - started a pseudo cluster with hdfs and kms
> >>>>>>>    - sanity checked encryption related operations working
> >>>>>>>    - sanity checked webui and logs.
> >>>>>>>
> >>>>>>>    -Xiao
> >>>>>>>
> >>>>>>>    On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> >> atm@apache.org>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> +1 (binding)
> >>>>>>>>
> >>>>>>>> - downloaded the src tarball and built the source (-Pdist
> >>>> -Pnative)
> >>>>>>>> - verified the checksum
> >>>>>>>> - brought up a secure pseudo distributed cluster
> >>>>>>>> - did some basic file system operations (mkdir, list, put,
> >> cat)
> >>>> and
> >>>>>>>> confirmed that everything was working
> >>>>>>>> - confirmed that the web UI worked
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Aaron
> >>>>>>>>
> >>>>>>>> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> >>>>>>> andrew.wang@cloudera.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi all,
> >>>>>>>>>
> >>>>>>>>> Let me start, as always, by thanking the efforts of all the
> >>>>>>> contributors
> >>>>>>>>> who contributed to this release, especially those who
> >> jumped
> >>> on
> >>>>>>> the
> >>>>>>>> issues
> >>>>>>>>> found in RC0.
> >>>>>>>>>
> >>>>>>>>> I've prepared RC1 for Apache Hadoop 3.0.0. This release
> >>>>>>> incorporates 302
> >>>>>>>>> fixed JIRAs since the previous 3.0.0-beta1 release.
> >>>>>>>>>
> >>>>>>>>> You can find the artifacts here:
> >>>>>>>>>
> >>>>>>>>> http://home.apache.org/~wang/3.0.0-RC1/
> >>>>>>>>>
> >>>>>>>>> I've done the traditional testing of building from the
> >> source
> >>>>>>> tarball and
> >>>>>>>>> running a Pi job on a single node cluster. I also verified
> >>> that
> >>>>>>> the
> >>>>>>>> shaded
> >>>>>>>>> jars are not empty.
> >>>>>>>>>
> >>>>>>>>> Found one issue that create-release (probably due to the
> >> mvn
> >>>>>>> deploy
> >>>>>>>> change)
> >>>>>>>>> didn't sign the artifacts, but I fixed that by calling mvn
> >>> one
> >>>>>>> more time.
> >>>>>>>>> Available here:
> >>>>>>>>>
> >>>>>>>>> https://repository.apache.org/
> >> content/repositories/orgapache
> >>>>>>> hadoop-1075/
> >>>>>>>>>
> >>>>>>>>> This release will run the standard 5 days, closing on Dec
> >>> 13th
> >>>> at
> >>>>>>> 12:31pm
> >>>>>>>>> Pacific. My +1 to start.
> >>>>>>>>>
> >>>>>>>>> Best,
> >>>>>>>>> Andrew
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> ------------------------------------------------------------
> >>> ---------
> >>>>>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >>>>>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Lei (Eddy) Xu
> >>>> Software Engineer, Cloudera
> >>>>
> >>>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Hi folks,

To close this out, the vote passes successfully with 13 binding +1s, 5
non-binding +1s, and no -1s. Thanks everyone for voting! I'll work on
staging.

I'm hoping we can address YARN-7588 and any remaining rolling upgrade
issues in 3.0.x maintenance releases. Beyond a wiki page, it would be
really great to get JIRAs filed and targeted for tracking as soon as
possible.

Vinod, what do you think we need to do regarding caveating rolling upgrade
support? We haven't advertised rolling upgrade support between major
releases outside of dev lists and JIRA. As a new major release, our compat
guidelines allow us to break compatibility, so I don't think it's expected
by users.

Best,
Andrew

On Wed, Dec 13, 2017 at 12:37 PM, Vinod Kumar Vavilapalli <
vinodkv@apache.org> wrote:

> I was waiting for Daniel to post the minutes from YARN meetup to talk
> about this. Anyways, in that discussion, we identified a bunch of key
> upgrade related scenarios that no-one seems to have validated - atleast
> from the representation in the YARN meetup. I'm going to create a wiki-page
> listing all these scenarios.
>
> But back to the bug that Junping raised. At this point, we don't have a
> clear path towards running 2.x applications on 3.0.0 clusters. So, our
> claim of rolling-upgrades already working is not accurate.
>
> One of the two options that Junping proposed should be pursued before we
> close the release. I'm in favor of calling out rolling-upgrade support be
> with-drawn or caveated and push for progress instead of blocking the
> release.
>
> Thanks
> +Vinod
>
> > On Dec 12, 2017, at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:
> >
> > Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just
> get chance to validate new RC now.
> >
> > Basically, I found two critical issues with the same rolling upgrade
> scenario as where HADOOP-15059 get found previously:
> > HDFS-12920, we changed value format for some hdfs configurations that
> old version MR client doesn't understand when fetching these
> configurations. Some quick workarounds are to add old value (without time
> unit) in hdfs-site.xml to override new default values but will generate
> many annoying warnings. I provided my fix suggestions on the JIRA already
> for more discussion.
> > The other one is YARN-7646. After we workaround HDFS-12920, will hit the
> issue that old version MR AppMaster cannot communicate with new version of
> YARN RM - could be related to resource profile changes from YARN side but
> root cause are still in investigation.
> >
> > The first issue may not belong to a blocker given we can workaround this
> without code change. I am not sure if we can workaround 2nd issue so far.
> If not, we may have to fix this or compromise with withdrawing support of
> rolling upgrade or calling it a stable release.
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Robert Kanter <rk...@cloudera.com>
> > Sent: Tuesday, December 12, 2017 3:10 PM
> > To: Arun Suresh
> > Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron
> T. Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
> >
> > +1 (binding)
> >
> > + Downloaded the binary release
> > + Deployed on a 3 node cluster on CentOS 7.3
> > + Ran some MR jobs, clicked around the UI, etc
> > + Ran some CLI commands (yarn logs, etc)
> >
> > Good job everyone on Hadoop 3!
> >
> >
> > - Robert
> >
> > On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
> >
> >> +1 (binding)
> >>
> >> - Verified signatures of the source tarball.
> >> - built from source - using the docker build environment.
> >> - set up a pseudo-distributed test cluster.
> >> - ran basic HDFS commands
> >> - ran some basic MR jobs
> >>
> >> Cheers
> >> -Arun
> >>
> >> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> >> wrote:
> >>
> >>> Hi everyone,
> >>>
> >>> As a reminder, this vote closes tomorrow at 12:31pm, so please give it
> a
> >>> whack if you have time. There are already enough binding +1s to pass
> this
> >>> vote, but it'd be great to get additional validation.
> >>>
> >>> Thanks to everyone who's voted thus far!
> >>>
> >>> Best,
> >>> Andrew
> >>>
> >>>
> >>>
> >>> On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> >>>
> >>>> +1 (binding)
> >>>>
> >>>> * Verified src tarball and bin tarball, verified md5 of each.
> >>>> * Build source with -Pdist,native
> >>>> * Started a pseudo cluster
> >>>> * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> >>>> dfs put/get/cat on "/" with XOR-2-1 policy.
> >>>>
> >>>> Thanks Andrew for this great effort!
> >>>>
> >>>> Best,
> >>>>
> >>>>
> >>>> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <
> andrew.wang@cloudera.com
> >>>
> >>>> wrote:
> >>>>> Hi Wei-Chiu,
> >>>>>
> >>>>> The patchprocess directory is left over from the create-release
> >>> process,
> >>>>> and it looks empty to me. We should still file a create-release JIRA
> >> to
> >>>> fix
> >>>>> this, but I think this is not a blocker. Would you agree?
> >>>>>
> >>>>> Best,
> >>>>> Andrew
> >>>>>
> >>>>> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> >> weichiu@cloudera.com
> >>>>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi Andrew, thanks the tremendous effort.
> >>>>>> I found an empty "patchprocess" directory in the source tarball,
> >> that
> >>> is
> >>>>>> not there if you clone from github. Any chance you might have some
> >>>> leftover
> >>>>>> trash when you made the tarball?
> >>>>>> Not wanting to nitpicking, but you might want to double check so we
> >>>> don't
> >>>>>> ship anything private to you in public :)
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> >>> ajay.kumar@hortonworks.com
> >>>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> +1 (non-binding)
> >>>>>>> Thanks for driving this, Andrew Wang!!
> >>>>>>>
> >>>>>>> - downloaded the src tarball and verified md5 checksum
> >>>>>>> - built from source with jdk 1.8.0_111-b14
> >>>>>>> - brought up a pseudo distributed cluster
> >>>>>>> - did basic file system operations (mkdir, list, put, cat) and
> >>>>>>> confirmed that everything was working
> >>>>>>> - Run word count, pi and DFSIOTest
> >>>>>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>> Ajay
> >>>>>>>
> >>>>>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> >>>>>>>
> >>>>>>>    +1 (binding)
> >>>>>>>
> >>>>>>>    - downloaded src tarball, verified md5
> >>>>>>>    - built from source with jdk1.8.0_112
> >>>>>>>    - started a pseudo cluster with hdfs and kms
> >>>>>>>    - sanity checked encryption related operations working
> >>>>>>>    - sanity checked webui and logs.
> >>>>>>>
> >>>>>>>    -Xiao
> >>>>>>>
> >>>>>>>    On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> >> atm@apache.org>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> +1 (binding)
> >>>>>>>>
> >>>>>>>> - downloaded the src tarball and built the source (-Pdist
> >>>> -Pnative)
> >>>>>>>> - verified the checksum
> >>>>>>>> - brought up a secure pseudo distributed cluster
> >>>>>>>> - did some basic file system operations (mkdir, list, put,
> >> cat)
> >>>> and
> >>>>>>>> confirmed that everything was working
> >>>>>>>> - confirmed that the web UI worked
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Aaron
> >>>>>>>>
> >>>>>>>> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> >>>>>>> andrew.wang@cloudera.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi all,
> >>>>>>>>>
> >>>>>>>>> Let me start, as always, by thanking the efforts of all the
> >>>>>>> contributors
> >>>>>>>>> who contributed to this release, especially those who
> >> jumped
> >>> on
> >>>>>>> the
> >>>>>>>> issues
> >>>>>>>>> found in RC0.
> >>>>>>>>>
> >>>>>>>>> I've prepared RC1 for Apache Hadoop 3.0.0. This release
> >>>>>>> incorporates 302
> >>>>>>>>> fixed JIRAs since the previous 3.0.0-beta1 release.
> >>>>>>>>>
> >>>>>>>>> You can find the artifacts here:
> >>>>>>>>>
> >>>>>>>>> http://home.apache.org/~wang/3.0.0-RC1/
> >>>>>>>>>
> >>>>>>>>> I've done the traditional testing of building from the
> >> source
> >>>>>>> tarball and
> >>>>>>>>> running a Pi job on a single node cluster. I also verified
> >>> that
> >>>>>>> the
> >>>>>>>> shaded
> >>>>>>>>> jars are not empty.
> >>>>>>>>>
> >>>>>>>>> Found one issue that create-release (probably due to the
> >> mvn
> >>>>>>> deploy
> >>>>>>>> change)
> >>>>>>>>> didn't sign the artifacts, but I fixed that by calling mvn
> >>> one
> >>>>>>> more time.
> >>>>>>>>> Available here:
> >>>>>>>>>
> >>>>>>>>> https://repository.apache.org/
> >> content/repositories/orgapache
> >>>>>>> hadoop-1075/
> >>>>>>>>>
> >>>>>>>>> This release will run the standard 5 days, closing on Dec
> >>> 13th
> >>>> at
> >>>>>>> 12:31pm
> >>>>>>>>> Pacific. My +1 to start.
> >>>>>>>>>
> >>>>>>>>> Best,
> >>>>>>>>> Andrew
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> ------------------------------------------------------------
> >>> ---------
> >>>>>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >>>>>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Lei (Eddy) Xu
> >>>> Software Engineer, Cloudera
> >>>>
> >>>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Hi folks,

To close this out, the vote passes successfully with 13 binding +1s, 5
non-binding +1s, and no -1s. Thanks everyone for voting! I'll work on
staging.

I'm hoping we can address YARN-7588 and any remaining rolling upgrade
issues in 3.0.x maintenance releases. Beyond a wiki page, it would be
really great to get JIRAs filed and targeted for tracking as soon as
possible.

Vinod, what do you think we need to do regarding caveating rolling upgrade
support? We haven't advertised rolling upgrade support between major
releases outside of dev lists and JIRA. As a new major release, our compat
guidelines allow us to break compatibility, so I don't think it's expected
by users.

Best,
Andrew

On Wed, Dec 13, 2017 at 12:37 PM, Vinod Kumar Vavilapalli <
vinodkv@apache.org> wrote:

> I was waiting for Daniel to post the minutes from YARN meetup to talk
> about this. Anyways, in that discussion, we identified a bunch of key
> upgrade related scenarios that no-one seems to have validated - atleast
> from the representation in the YARN meetup. I'm going to create a wiki-page
> listing all these scenarios.
>
> But back to the bug that Junping raised. At this point, we don't have a
> clear path towards running 2.x applications on 3.0.0 clusters. So, our
> claim of rolling-upgrades already working is not accurate.
>
> One of the two options that Junping proposed should be pursued before we
> close the release. I'm in favor of calling out rolling-upgrade support be
> with-drawn or caveated and push for progress instead of blocking the
> release.
>
> Thanks
> +Vinod
>
> > On Dec 12, 2017, at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:
> >
> > Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just
> get chance to validate new RC now.
> >
> > Basically, I found two critical issues with the same rolling upgrade
> scenario as where HADOOP-15059 get found previously:
> > HDFS-12920, we changed value format for some hdfs configurations that
> old version MR client doesn't understand when fetching these
> configurations. Some quick workarounds are to add old value (without time
> unit) in hdfs-site.xml to override new default values but will generate
> many annoying warnings. I provided my fix suggestions on the JIRA already
> for more discussion.
> > The other one is YARN-7646. After we workaround HDFS-12920, will hit the
> issue that old version MR AppMaster cannot communicate with new version of
> YARN RM - could be related to resource profile changes from YARN side but
> root cause are still in investigation.
> >
> > The first issue may not belong to a blocker given we can workaround this
> without code change. I am not sure if we can workaround 2nd issue so far.
> If not, we may have to fix this or compromise with withdrawing support of
> rolling upgrade or calling it a stable release.
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Robert Kanter <rk...@cloudera.com>
> > Sent: Tuesday, December 12, 2017 3:10 PM
> > To: Arun Suresh
> > Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron
> T. Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
> >
> > +1 (binding)
> >
> > + Downloaded the binary release
> > + Deployed on a 3 node cluster on CentOS 7.3
> > + Ran some MR jobs, clicked around the UI, etc
> > + Ran some CLI commands (yarn logs, etc)
> >
> > Good job everyone on Hadoop 3!
> >
> >
> > - Robert
> >
> > On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
> >
> >> +1 (binding)
> >>
> >> - Verified signatures of the source tarball.
> >> - built from source - using the docker build environment.
> >> - set up a pseudo-distributed test cluster.
> >> - ran basic HDFS commands
> >> - ran some basic MR jobs
> >>
> >> Cheers
> >> -Arun
> >>
> >> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> >> wrote:
> >>
> >>> Hi everyone,
> >>>
> >>> As a reminder, this vote closes tomorrow at 12:31pm, so please give it
> a
> >>> whack if you have time. There are already enough binding +1s to pass
> this
> >>> vote, but it'd be great to get additional validation.
> >>>
> >>> Thanks to everyone who's voted thus far!
> >>>
> >>> Best,
> >>> Andrew
> >>>
> >>>
> >>>
> >>> On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> >>>
> >>>> +1 (binding)
> >>>>
> >>>> * Verified src tarball and bin tarball, verified md5 of each.
> >>>> * Build source with -Pdist,native
> >>>> * Started a pseudo cluster
> >>>> * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> >>>> dfs put/get/cat on "/" with XOR-2-1 policy.
> >>>>
> >>>> Thanks Andrew for this great effort!
> >>>>
> >>>> Best,
> >>>>
> >>>>
> >>>> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <
> andrew.wang@cloudera.com
> >>>
> >>>> wrote:
> >>>>> Hi Wei-Chiu,
> >>>>>
> >>>>> The patchprocess directory is left over from the create-release
> >>> process,
> >>>>> and it looks empty to me. We should still file a create-release JIRA
> >> to
> >>>> fix
> >>>>> this, but I think this is not a blocker. Would you agree?
> >>>>>
> >>>>> Best,
> >>>>> Andrew
> >>>>>
> >>>>> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> >> weichiu@cloudera.com
> >>>>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi Andrew, thanks the tremendous effort.
> >>>>>> I found an empty "patchprocess" directory in the source tarball,
> >> that
> >>> is
> >>>>>> not there if you clone from github. Any chance you might have some
> >>>> leftover
> >>>>>> trash when you made the tarball?
> >>>>>> Not wanting to nitpicking, but you might want to double check so we
> >>>> don't
> >>>>>> ship anything private to you in public :)
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> >>> ajay.kumar@hortonworks.com
> >>>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> +1 (non-binding)
> >>>>>>> Thanks for driving this, Andrew Wang!!
> >>>>>>>
> >>>>>>> - downloaded the src tarball and verified md5 checksum
> >>>>>>> - built from source with jdk 1.8.0_111-b14
> >>>>>>> - brought up a pseudo distributed cluster
> >>>>>>> - did basic file system operations (mkdir, list, put, cat) and
> >>>>>>> confirmed that everything was working
> >>>>>>> - Run word count, pi and DFSIOTest
> >>>>>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>> Ajay
> >>>>>>>
> >>>>>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> >>>>>>>
> >>>>>>>    +1 (binding)
> >>>>>>>
> >>>>>>>    - downloaded src tarball, verified md5
> >>>>>>>    - built from source with jdk1.8.0_112
> >>>>>>>    - started a pseudo cluster with hdfs and kms
> >>>>>>>    - sanity checked encryption related operations working
> >>>>>>>    - sanity checked webui and logs.
> >>>>>>>
> >>>>>>>    -Xiao
> >>>>>>>
> >>>>>>>    On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> >> atm@apache.org>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> +1 (binding)
> >>>>>>>>
> >>>>>>>> - downloaded the src tarball and built the source (-Pdist
> >>>> -Pnative)
> >>>>>>>> - verified the checksum
> >>>>>>>> - brought up a secure pseudo distributed cluster
> >>>>>>>> - did some basic file system operations (mkdir, list, put,
> >> cat)
> >>>> and
> >>>>>>>> confirmed that everything was working
> >>>>>>>> - confirmed that the web UI worked
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Aaron
> >>>>>>>>
> >>>>>>>> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> >>>>>>> andrew.wang@cloudera.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi all,
> >>>>>>>>>
> >>>>>>>>> Let me start, as always, by thanking the efforts of all the
> >>>>>>> contributors
> >>>>>>>>> who contributed to this release, especially those who
> >> jumped
> >>> on
> >>>>>>> the
> >>>>>>>> issues
> >>>>>>>>> found in RC0.
> >>>>>>>>>
> >>>>>>>>> I've prepared RC1 for Apache Hadoop 3.0.0. This release
> >>>>>>> incorporates 302
> >>>>>>>>> fixed JIRAs since the previous 3.0.0-beta1 release.
> >>>>>>>>>
> >>>>>>>>> You can find the artifacts here:
> >>>>>>>>>
> >>>>>>>>> http://home.apache.org/~wang/3.0.0-RC1/
> >>>>>>>>>
> >>>>>>>>> I've done the traditional testing of building from the
> >> source
> >>>>>>> tarball and
> >>>>>>>>> running a Pi job on a single node cluster. I also verified
> >>> that
> >>>>>>> the
> >>>>>>>> shaded
> >>>>>>>>> jars are not empty.
> >>>>>>>>>
> >>>>>>>>> Found one issue that create-release (probably due to the
> >> mvn
> >>>>>>> deploy
> >>>>>>>> change)
> >>>>>>>>> didn't sign the artifacts, but I fixed that by calling mvn
> >>> one
> >>>>>>> more time.
> >>>>>>>>> Available here:
> >>>>>>>>>
> >>>>>>>>> https://repository.apache.org/
> >> content/repositories/orgapache
> >>>>>>> hadoop-1075/
> >>>>>>>>>
> >>>>>>>>> This release will run the standard 5 days, closing on Dec
> >>> 13th
> >>>> at
> >>>>>>> 12:31pm
> >>>>>>>>> Pacific. My +1 to start.
> >>>>>>>>>
> >>>>>>>>> Best,
> >>>>>>>>> Andrew
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> ------------------------------------------------------------
> >>> ---------
> >>>>>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >>>>>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Lei (Eddy) Xu
> >>>> Software Engineer, Cloudera
> >>>>
> >>>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Hi folks,

To close this out, the vote passes successfully with 13 binding +1s, 5
non-binding +1s, and no -1s. Thanks everyone for voting! I'll work on
staging.

I'm hoping we can address YARN-7588 and any remaining rolling upgrade
issues in 3.0.x maintenance releases. Beyond a wiki page, it would be
really great to get JIRAs filed and targeted for tracking as soon as
possible.

Vinod, what do you think we need to do regarding caveating rolling upgrade
support? We haven't advertised rolling upgrade support between major
releases outside of dev lists and JIRA. As a new major release, our compat
guidelines allow us to break compatibility, so I don't think it's expected
by users.

Best,
Andrew

On Wed, Dec 13, 2017 at 12:37 PM, Vinod Kumar Vavilapalli <
vinodkv@apache.org> wrote:

> I was waiting for Daniel to post the minutes from YARN meetup to talk
> about this. Anyways, in that discussion, we identified a bunch of key
> upgrade related scenarios that no-one seems to have validated - atleast
> from the representation in the YARN meetup. I'm going to create a wiki-page
> listing all these scenarios.
>
> But back to the bug that Junping raised. At this point, we don't have a
> clear path towards running 2.x applications on 3.0.0 clusters. So, our
> claim of rolling-upgrades already working is not accurate.
>
> One of the two options that Junping proposed should be pursued before we
> close the release. I'm in favor of calling out rolling-upgrade support be
> with-drawn or caveated and push for progress instead of blocking the
> release.
>
> Thanks
> +Vinod
>
> > On Dec 12, 2017, at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:
> >
> > Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just
> get chance to validate new RC now.
> >
> > Basically, I found two critical issues with the same rolling upgrade
> scenario as where HADOOP-15059 get found previously:
> > HDFS-12920, we changed value format for some hdfs configurations that
> old version MR client doesn't understand when fetching these
> configurations. Some quick workarounds are to add old value (without time
> unit) in hdfs-site.xml to override new default values but will generate
> many annoying warnings. I provided my fix suggestions on the JIRA already
> for more discussion.
> > The other one is YARN-7646. After we workaround HDFS-12920, will hit the
> issue that old version MR AppMaster cannot communicate with new version of
> YARN RM - could be related to resource profile changes from YARN side but
> root cause are still in investigation.
> >
> > The first issue may not belong to a blocker given we can workaround this
> without code change. I am not sure if we can workaround 2nd issue so far.
> If not, we may have to fix this or compromise with withdrawing support of
> rolling upgrade or calling it a stable release.
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Robert Kanter <rk...@cloudera.com>
> > Sent: Tuesday, December 12, 2017 3:10 PM
> > To: Arun Suresh
> > Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron
> T. Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
> >
> > +1 (binding)
> >
> > + Downloaded the binary release
> > + Deployed on a 3 node cluster on CentOS 7.3
> > + Ran some MR jobs, clicked around the UI, etc
> > + Ran some CLI commands (yarn logs, etc)
> >
> > Good job everyone on Hadoop 3!
> >
> >
> > - Robert
> >
> > On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
> >
> >> +1 (binding)
> >>
> >> - Verified signatures of the source tarball.
> >> - built from source - using the docker build environment.
> >> - set up a pseudo-distributed test cluster.
> >> - ran basic HDFS commands
> >> - ran some basic MR jobs
> >>
> >> Cheers
> >> -Arun
> >>
> >> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> >> wrote:
> >>
> >>> Hi everyone,
> >>>
> >>> As a reminder, this vote closes tomorrow at 12:31pm, so please give it
> a
> >>> whack if you have time. There are already enough binding +1s to pass
> this
> >>> vote, but it'd be great to get additional validation.
> >>>
> >>> Thanks to everyone who's voted thus far!
> >>>
> >>> Best,
> >>> Andrew
> >>>
> >>>
> >>>
> >>> On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> >>>
> >>>> +1 (binding)
> >>>>
> >>>> * Verified src tarball and bin tarball, verified md5 of each.
> >>>> * Build source with -Pdist,native
> >>>> * Started a pseudo cluster
> >>>> * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> >>>> dfs put/get/cat on "/" with XOR-2-1 policy.
> >>>>
> >>>> Thanks Andrew for this great effort!
> >>>>
> >>>> Best,
> >>>>
> >>>>
> >>>> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <
> andrew.wang@cloudera.com
> >>>
> >>>> wrote:
> >>>>> Hi Wei-Chiu,
> >>>>>
> >>>>> The patchprocess directory is left over from the create-release
> >>> process,
> >>>>> and it looks empty to me. We should still file a create-release JIRA
> >> to
> >>>> fix
> >>>>> this, but I think this is not a blocker. Would you agree?
> >>>>>
> >>>>> Best,
> >>>>> Andrew
> >>>>>
> >>>>> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> >> weichiu@cloudera.com
> >>>>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi Andrew, thanks the tremendous effort.
> >>>>>> I found an empty "patchprocess" directory in the source tarball,
> >> that
> >>> is
> >>>>>> not there if you clone from github. Any chance you might have some
> >>>> leftover
> >>>>>> trash when you made the tarball?
> >>>>>> Not wanting to nitpicking, but you might want to double check so we
> >>>> don't
> >>>>>> ship anything private to you in public :)
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> >>> ajay.kumar@hortonworks.com
> >>>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> +1 (non-binding)
> >>>>>>> Thanks for driving this, Andrew Wang!!
> >>>>>>>
> >>>>>>> - downloaded the src tarball and verified md5 checksum
> >>>>>>> - built from source with jdk 1.8.0_111-b14
> >>>>>>> - brought up a pseudo distributed cluster
> >>>>>>> - did basic file system operations (mkdir, list, put, cat) and
> >>>>>>> confirmed that everything was working
> >>>>>>> - Run word count, pi and DFSIOTest
> >>>>>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>> Ajay
> >>>>>>>
> >>>>>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> >>>>>>>
> >>>>>>>    +1 (binding)
> >>>>>>>
> >>>>>>>    - downloaded src tarball, verified md5
> >>>>>>>    - built from source with jdk1.8.0_112
> >>>>>>>    - started a pseudo cluster with hdfs and kms
> >>>>>>>    - sanity checked encryption related operations working
> >>>>>>>    - sanity checked webui and logs.
> >>>>>>>
> >>>>>>>    -Xiao
> >>>>>>>
> >>>>>>>    On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> >> atm@apache.org>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> +1 (binding)
> >>>>>>>>
> >>>>>>>> - downloaded the src tarball and built the source (-Pdist
> >>>> -Pnative)
> >>>>>>>> - verified the checksum
> >>>>>>>> - brought up a secure pseudo distributed cluster
> >>>>>>>> - did some basic file system operations (mkdir, list, put,
> >> cat)
> >>>> and
> >>>>>>>> confirmed that everything was working
> >>>>>>>> - confirmed that the web UI worked
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Aaron
> >>>>>>>>
> >>>>>>>> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> >>>>>>> andrew.wang@cloudera.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi all,
> >>>>>>>>>
> >>>>>>>>> Let me start, as always, by thanking the efforts of all the
> >>>>>>> contributors
> >>>>>>>>> who contributed to this release, especially those who
> >> jumped
> >>> on
> >>>>>>> the
> >>>>>>>> issues
> >>>>>>>>> found in RC0.
> >>>>>>>>>
> >>>>>>>>> I've prepared RC1 for Apache Hadoop 3.0.0. This release
> >>>>>>> incorporates 302
> >>>>>>>>> fixed JIRAs since the previous 3.0.0-beta1 release.
> >>>>>>>>>
> >>>>>>>>> You can find the artifacts here:
> >>>>>>>>>
> >>>>>>>>> http://home.apache.org/~wang/3.0.0-RC1/
> >>>>>>>>>
> >>>>>>>>> I've done the traditional testing of building from the
> >> source
> >>>>>>> tarball and
> >>>>>>>>> running a Pi job on a single node cluster. I also verified
> >>> that
> >>>>>>> the
> >>>>>>>> shaded
> >>>>>>>>> jars are not empty.
> >>>>>>>>>
> >>>>>>>>> Found one issue that create-release (probably due to the
> >> mvn
> >>>>>>> deploy
> >>>>>>>> change)
> >>>>>>>>> didn't sign the artifacts, but I fixed that by calling mvn
> >>> one
> >>>>>>> more time.
> >>>>>>>>> Available here:
> >>>>>>>>>
> >>>>>>>>> https://repository.apache.org/
> >> content/repositories/orgapache
> >>>>>>> hadoop-1075/
> >>>>>>>>>
> >>>>>>>>> This release will run the standard 5 days, closing on Dec
> >>> 13th
> >>>> at
> >>>>>>> 12:31pm
> >>>>>>>>> Pacific. My +1 to start.
> >>>>>>>>>
> >>>>>>>>> Best,
> >>>>>>>>> Andrew
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> ------------------------------------------------------------
> >>> ---------
> >>>>>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >>>>>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Lei (Eddy) Xu
> >>>> Software Engineer, Cloudera
> >>>>
> >>>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

I was waiting for Daniel to post the minutes from YARN meetup to talk about this. Anyways, in that discussion, we identified a bunch of key upgrade related scenarios that no-one seems to have validated - atleast from the representation in the YARN meetup. I'm going to create a wiki-page listing all these scenarios.

But back to the bug that Junping raised. At this point, we don't have a clear path towards running 2.x applications on 3.0.0 clusters. So, our claim of rolling-upgrades already working is not accurate.

One of the two options that Junping proposed should be pursued before we close the release. I'm in favor of calling out rolling-upgrade support be with-drawn or caveated and push for progress instead of blocking the release.

Thanks
+Vinod

> On Dec 12, 2017, at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:
> 
> Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get chance to validate new RC now.
> 
> Basically, I found two critical issues with the same rolling upgrade scenario as where HADOOP-15059 get found previously:
> HDFS-12920, we changed value format for some hdfs configurations that old version MR client doesn't understand when fetching these configurations. Some quick workarounds are to add old value (without time unit) in hdfs-site.xml to override new default values but will generate many annoying warnings. I provided my fix suggestions on the JIRA already for more discussion.
> The other one is YARN-7646. After we workaround HDFS-12920, will hit the issue that old version MR AppMaster cannot communicate with new version of YARN RM - could be related to resource profile changes from YARN side but root cause are still in investigation.
> 
> The first issue may not belong to a blocker given we can workaround this without code change. I am not sure if we can workaround 2nd issue so far. If not, we may have to fix this or compromise with withdrawing support of rolling upgrade or calling it a stable release.
> 
> 
> Thanks,
> 
> Junping
> 
> ________________________________________
> From: Robert Kanter <rk...@cloudera.com>
> Sent: Tuesday, December 12, 2017 3:10 PM
> To: Arun Suresh
> Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T. Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
> 
> +1 (binding)
> 
> + Downloaded the binary release
> + Deployed on a 3 node cluster on CentOS 7.3
> + Ran some MR jobs, clicked around the UI, etc
> + Ran some CLI commands (yarn logs, etc)
> 
> Good job everyone on Hadoop 3!
> 
> 
> - Robert
> 
> On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
> 
>> +1 (binding)
>> 
>> - Verified signatures of the source tarball.
>> - built from source - using the docker build environment.
>> - set up a pseudo-distributed test cluster.
>> - ran basic HDFS commands
>> - ran some basic MR jobs
>> 
>> Cheers
>> -Arun
>> 
>> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
>> wrote:
>> 
>>> Hi everyone,
>>> 
>>> As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
>>> whack if you have time. There are already enough binding +1s to pass this
>>> vote, but it'd be great to get additional validation.
>>> 
>>> Thanks to everyone who's voted thus far!
>>> 
>>> Best,
>>> Andrew
>>> 
>>> 
>>> 
>>> On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
>>> 
>>>> +1 (binding)
>>>> 
>>>> * Verified src tarball and bin tarball, verified md5 of each.
>>>> * Build source with -Pdist,native
>>>> * Started a pseudo cluster
>>>> * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
>>>> dfs put/get/cat on "/" with XOR-2-1 policy.
>>>> 
>>>> Thanks Andrew for this great effort!
>>>> 
>>>> Best,
>>>> 
>>>> 
>>>> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <andrew.wang@cloudera.com
>>> 
>>>> wrote:
>>>>> Hi Wei-Chiu,
>>>>> 
>>>>> The patchprocess directory is left over from the create-release
>>> process,
>>>>> and it looks empty to me. We should still file a create-release JIRA
>> to
>>>> fix
>>>>> this, but I think this is not a blocker. Would you agree?
>>>>> 
>>>>> Best,
>>>>> Andrew
>>>>> 
>>>>> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
>> weichiu@cloudera.com
>>>> 
>>>>> wrote:
>>>>> 
>>>>>> Hi Andrew, thanks the tremendous effort.
>>>>>> I found an empty "patchprocess" directory in the source tarball,
>> that
>>> is
>>>>>> not there if you clone from github. Any chance you might have some
>>>> leftover
>>>>>> trash when you made the tarball?
>>>>>> Not wanting to nitpicking, but you might want to double check so we
>>>> don't
>>>>>> ship anything private to you in public :)
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
>>> ajay.kumar@hortonworks.com
>>>>> 
>>>>>> wrote:
>>>>>> 
>>>>>>> +1 (non-binding)
>>>>>>> Thanks for driving this, Andrew Wang!!
>>>>>>> 
>>>>>>> - downloaded the src tarball and verified md5 checksum
>>>>>>> - built from source with jdk 1.8.0_111-b14
>>>>>>> - brought up a pseudo distributed cluster
>>>>>>> - did basic file system operations (mkdir, list, put, cat) and
>>>>>>> confirmed that everything was working
>>>>>>> - Run word count, pi and DFSIOTest
>>>>>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> Ajay
>>>>>>> 
>>>>>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>>>>>> 
>>>>>>>    +1 (binding)
>>>>>>> 
>>>>>>>    - downloaded src tarball, verified md5
>>>>>>>    - built from source with jdk1.8.0_112
>>>>>>>    - started a pseudo cluster with hdfs and kms
>>>>>>>    - sanity checked encryption related operations working
>>>>>>>    - sanity checked webui and logs.
>>>>>>> 
>>>>>>>    -Xiao
>>>>>>> 
>>>>>>>    On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
>> atm@apache.org>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> +1 (binding)
>>>>>>>> 
>>>>>>>> - downloaded the src tarball and built the source (-Pdist
>>>> -Pnative)
>>>>>>>> - verified the checksum
>>>>>>>> - brought up a secure pseudo distributed cluster
>>>>>>>> - did some basic file system operations (mkdir, list, put,
>> cat)
>>>> and
>>>>>>>> confirmed that everything was working
>>>>>>>> - confirmed that the web UI worked
>>>>>>>> 
>>>>>>>> Best,
>>>>>>>> Aaron
>>>>>>>> 
>>>>>>>> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>>>>>>> andrew.wang@cloudera.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hi all,
>>>>>>>>> 
>>>>>>>>> Let me start, as always, by thanking the efforts of all the
>>>>>>> contributors
>>>>>>>>> who contributed to this release, especially those who
>> jumped
>>> on
>>>>>>> the
>>>>>>>> issues
>>>>>>>>> found in RC0.
>>>>>>>>> 
>>>>>>>>> I've prepared RC1 for Apache Hadoop 3.0.0. This release
>>>>>>> incorporates 302
>>>>>>>>> fixed JIRAs since the previous 3.0.0-beta1 release.
>>>>>>>>> 
>>>>>>>>> You can find the artifacts here:
>>>>>>>>> 
>>>>>>>>> http://home.apache.org/~wang/3.0.0-RC1/
>>>>>>>>> 
>>>>>>>>> I've done the traditional testing of building from the
>> source
>>>>>>> tarball and
>>>>>>>>> running a Pi job on a single node cluster. I also verified
>>> that
>>>>>>> the
>>>>>>>> shaded
>>>>>>>>> jars are not empty.
>>>>>>>>> 
>>>>>>>>> Found one issue that create-release (probably due to the
>> mvn
>>>>>>> deploy
>>>>>>>> change)
>>>>>>>>> didn't sign the artifacts, but I fixed that by calling mvn
>>> one
>>>>>>> more time.
>>>>>>>>> Available here:
>>>>>>>>> 
>>>>>>>>> https://repository.apache.org/
>> content/repositories/orgapache
>>>>>>> hadoop-1075/
>>>>>>>>> 
>>>>>>>>> This release will run the standard 5 days, closing on Dec
>>> 13th
>>>> at
>>>>>>> 12:31pm
>>>>>>>>> Pacific. My +1 to start.
>>>>>>>>> 
>>>>>>>>> Best,
>>>>>>>>> Andrew
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> ------------------------------------------------------------
>>> ---------
>>>>>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>>>>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Lei (Eddy) Xu
>>>> Software Engineer, Cloudera
>>>> 
>>> 
>> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Jonathan Hung <jy...@gmail.com>.

Thanks Andrew for the huge effort.

+1 (non-binding)
- Downloaded binary tarball and verified md5
- Ran RM HA and verified manual failover
- Verified add/remove/update scheduler configuration API (CLI/REST) works
for leveldb/zookeeper backend
- Verified scheduler configuration changes persisted on restart/failover
- Verified "yarn rmadmin -refreshQueues" works when scheduler configuration
API disabled, and does not work when scheduler configuration API enabled


Jonathan Hung

On Tue, Dec 12, 2017 at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:

> Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get
> chance to validate new RC now.
>
> Basically, I found two critical issues with the same rolling upgrade
> scenario as where HADOOP-15059 get found previously:
> HDFS-12920, we changed value format for some hdfs configurations that old
> version MR client doesn't understand when fetching these configurations.
> Some quick workarounds are to add old value (without time unit) in
> hdfs-site.xml to override new default values but will generate many
> annoying warnings. I provided my fix suggestions on the JIRA already for
> more discussion.
> The other one is YARN-7646. After we workaround HDFS-12920, will hit the
> issue that old version MR AppMaster cannot communicate with new version of
> YARN RM - could be related to resource profile changes from YARN side but
> root cause are still in investigation.
>
> The first issue may not belong to a blocker given we can workaround this
> without code change. I am not sure if we can workaround 2nd issue so far.
> If not, we may have to fix this or compromise with withdrawing support of
> rolling upgrade or calling it a stable release.
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Robert Kanter <rk...@cloudera.com>
> Sent: Tuesday, December 12, 2017 3:10 PM
> To: Arun Suresh
> Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T.
> Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
>
> +1 (binding)
>
> + Downloaded the binary release
> + Deployed on a 3 node cluster on CentOS 7.3
> + Ran some MR jobs, clicked around the UI, etc
> + Ran some CLI commands (yarn logs, etc)
>
> Good job everyone on Hadoop 3!
>
>
> - Robert
>
> On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
>
> > +1 (binding)
> >
> > - Verified signatures of the source tarball.
> > - built from source - using the docker build environment.
> > - set up a pseudo-distributed test cluster.
> > - ran basic HDFS commands
> > - ran some basic MR jobs
> >
> > Cheers
> > -Arun
> >
> > On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> > wrote:
> >
> > > Hi everyone,
> > >
> > > As a reminder, this vote closes tomorrow at 12:31pm, so please give it
> a
> > > whack if you have time. There are already enough binding +1s to pass
> this
> > > vote, but it'd be great to get additional validation.
> > >
> > > Thanks to everyone who's voted thus far!
> > >
> > > Best,
> > > Andrew
> > >
> > >
> > >
> > > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > * Verified src tarball and bin tarball, verified md5 of each.
> > > > * Build source with -Pdist,native
> > > > * Started a pseudo cluster
> > > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > > >
> > > > Thanks Andrew for this great effort!
> > > >
> > > > Best,
> > > >
> > > >
> > > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <
> andrew.wang@cloudera.com
> > >
> > > > wrote:
> > > > > Hi Wei-Chiu,
> > > > >
> > > > > The patchprocess directory is left over from the create-release
> > > process,
> > > > > and it looks empty to me. We should still file a create-release
> JIRA
> > to
> > > > fix
> > > > > this, but I think this is not a blocker. Would you agree?
> > > > >
> > > > > Best,
> > > > > Andrew
> > > > >
> > > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> > weichiu@cloudera.com
> > > >
> > > > > wrote:
> > > > >
> > > > >> Hi Andrew, thanks the tremendous effort.
> > > > >> I found an empty "patchprocess" directory in the source tarball,
> > that
> > > is
> > > > >> not there if you clone from github. Any chance you might have some
> > > > leftover
> > > > >> trash when you made the tarball?
> > > > >> Not wanting to nitpicking, but you might want to double check so
> we
> > > > don't
> > > > >> ship anything private to you in public :)
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > > ajay.kumar@hortonworks.com
> > > > >
> > > > >> wrote:
> > > > >>
> > > > >>> +1 (non-binding)
> > > > >>> Thanks for driving this, Andrew Wang!!
> > > > >>>
> > > > >>> - downloaded the src tarball and verified md5 checksum
> > > > >>> - built from source with jdk 1.8.0_111-b14
> > > > >>> - brought up a pseudo distributed cluster
> > > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > > >>> confirmed that everything was working
> > > > >>> - Run word count, pi and DFSIOTest
> > > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > > >>>
> > > > >>> Cheers,
> > > > >>> Ajay
> > > > >>>
> > > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > > >>>
> > > > >>>     +1 (binding)
> > > > >>>
> > > > >>>     - downloaded src tarball, verified md5
> > > > >>>     - built from source with jdk1.8.0_112
> > > > >>>     - started a pseudo cluster with hdfs and kms
> > > > >>>     - sanity checked encryption related operations working
> > > > >>>     - sanity checked webui and logs.
> > > > >>>
> > > > >>>     -Xiao
> > > > >>>
> > > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> > atm@apache.org>
> > > > >>> wrote:
> > > > >>>
> > > > >>>     > +1 (binding)
> > > > >>>     >
> > > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > > -Pnative)
> > > > >>>     > - verified the checksum
> > > > >>>     > - brought up a secure pseudo distributed cluster
> > > > >>>     > - did some basic file system operations (mkdir, list, put,
> > cat)
> > > > and
> > > > >>>     > confirmed that everything was working
> > > > >>>     > - confirmed that the web UI worked
> > > > >>>     >
> > > > >>>     > Best,
> > > > >>>     > Aaron
> > > > >>>     >
> > > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > > >>> andrew.wang@cloudera.com>
> > > > >>>     > wrote:
> > > > >>>     >
> > > > >>>     > > Hi all,
> > > > >>>     > >
> > > > >>>     > > Let me start, as always, by thanking the efforts of all
> the
> > > > >>> contributors
> > > > >>>     > > who contributed to this release, especially those who
> > jumped
> > > on
> > > > >>> the
> > > > >>>     > issues
> > > > >>>     > > found in RC0.
> > > > >>>     > >
> > > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > > >>> incorporates 302
> > > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > > >>>     > >
> > > > >>>     > > You can find the artifacts here:
> > > > >>>     > >
> > > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > > >>>     > >
> > > > >>>     > > I've done the traditional testing of building from the
> > source
> > > > >>> tarball and
> > > > >>>     > > running a Pi job on a single node cluster. I also
> verified
> > > that
> > > > >>> the
> > > > >>>     > shaded
> > > > >>>     > > jars are not empty.
> > > > >>>     > >
> > > > >>>     > > Found one issue that create-release (probably due to the
> > mvn
> > > > >>> deploy
> > > > >>>     > change)
> > > > >>>     > > didn't sign the artifacts, but I fixed that by calling
> mvn
> > > one
> > > > >>> more time.
> > > > >>>     > > Available here:
> > > > >>>     > >
> > > > >>>     > > https://repository.apache.org/
> > content/repositories/orgapache
> > > > >>> hadoop-1075/
> > > > >>>     > >
> > > > >>>     > > This release will run the standard 5 days, closing on Dec
> > > 13th
> > > > at
> > > > >>> 12:31pm
> > > > >>>     > > Pacific. My +1 to start.
> > > > >>>     > >
> > > > >>>     > > Best,
> > > > >>>     > > Andrew
> > > > >>>     > >
> > > > >>>     >
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> ------------------------------------------------------------
> > > ---------
> > > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.
> org
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Lei (Eddy) Xu
> > > > Software Engineer, Cloudera
> > > >
> > >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

I was waiting for Daniel to post the minutes from YARN meetup to talk about this. Anyways, in that discussion, we identified a bunch of key upgrade related scenarios that no-one seems to have validated - atleast from the representation in the YARN meetup. I'm going to create a wiki-page listing all these scenarios.

But back to the bug that Junping raised. At this point, we don't have a clear path towards running 2.x applications on 3.0.0 clusters. So, our claim of rolling-upgrades already working is not accurate.

One of the two options that Junping proposed should be pursued before we close the release. I'm in favor of calling out rolling-upgrade support be with-drawn or caveated and push for progress instead of blocking the release.

Thanks
+Vinod

> On Dec 12, 2017, at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:
> 
> Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get chance to validate new RC now.
> 
> Basically, I found two critical issues with the same rolling upgrade scenario as where HADOOP-15059 get found previously:
> HDFS-12920, we changed value format for some hdfs configurations that old version MR client doesn't understand when fetching these configurations. Some quick workarounds are to add old value (without time unit) in hdfs-site.xml to override new default values but will generate many annoying warnings. I provided my fix suggestions on the JIRA already for more discussion.
> The other one is YARN-7646. After we workaround HDFS-12920, will hit the issue that old version MR AppMaster cannot communicate with new version of YARN RM - could be related to resource profile changes from YARN side but root cause are still in investigation.
> 
> The first issue may not belong to a blocker given we can workaround this without code change. I am not sure if we can workaround 2nd issue so far. If not, we may have to fix this or compromise with withdrawing support of rolling upgrade or calling it a stable release.
> 
> 
> Thanks,
> 
> Junping
> 
> ________________________________________
> From: Robert Kanter <rk...@cloudera.com>
> Sent: Tuesday, December 12, 2017 3:10 PM
> To: Arun Suresh
> Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T. Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
> 
> +1 (binding)
> 
> + Downloaded the binary release
> + Deployed on a 3 node cluster on CentOS 7.3
> + Ran some MR jobs, clicked around the UI, etc
> + Ran some CLI commands (yarn logs, etc)
> 
> Good job everyone on Hadoop 3!
> 
> 
> - Robert
> 
> On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
> 
>> +1 (binding)
>> 
>> - Verified signatures of the source tarball.
>> - built from source - using the docker build environment.
>> - set up a pseudo-distributed test cluster.
>> - ran basic HDFS commands
>> - ran some basic MR jobs
>> 
>> Cheers
>> -Arun
>> 
>> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
>> wrote:
>> 
>>> Hi everyone,
>>> 
>>> As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
>>> whack if you have time. There are already enough binding +1s to pass this
>>> vote, but it'd be great to get additional validation.
>>> 
>>> Thanks to everyone who's voted thus far!
>>> 
>>> Best,
>>> Andrew
>>> 
>>> 
>>> 
>>> On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
>>> 
>>>> +1 (binding)
>>>> 
>>>> * Verified src tarball and bin tarball, verified md5 of each.
>>>> * Build source with -Pdist,native
>>>> * Started a pseudo cluster
>>>> * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
>>>> dfs put/get/cat on "/" with XOR-2-1 policy.
>>>> 
>>>> Thanks Andrew for this great effort!
>>>> 
>>>> Best,
>>>> 
>>>> 
>>>> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <andrew.wang@cloudera.com
>>> 
>>>> wrote:
>>>>> Hi Wei-Chiu,
>>>>> 
>>>>> The patchprocess directory is left over from the create-release
>>> process,
>>>>> and it looks empty to me. We should still file a create-release JIRA
>> to
>>>> fix
>>>>> this, but I think this is not a blocker. Would you agree?
>>>>> 
>>>>> Best,
>>>>> Andrew
>>>>> 
>>>>> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
>> weichiu@cloudera.com
>>>> 
>>>>> wrote:
>>>>> 
>>>>>> Hi Andrew, thanks the tremendous effort.
>>>>>> I found an empty "patchprocess" directory in the source tarball,
>> that
>>> is
>>>>>> not there if you clone from github. Any chance you might have some
>>>> leftover
>>>>>> trash when you made the tarball?
>>>>>> Not wanting to nitpicking, but you might want to double check so we
>>>> don't
>>>>>> ship anything private to you in public :)
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
>>> ajay.kumar@hortonworks.com
>>>>> 
>>>>>> wrote:
>>>>>> 
>>>>>>> +1 (non-binding)
>>>>>>> Thanks for driving this, Andrew Wang!!
>>>>>>> 
>>>>>>> - downloaded the src tarball and verified md5 checksum
>>>>>>> - built from source with jdk 1.8.0_111-b14
>>>>>>> - brought up a pseudo distributed cluster
>>>>>>> - did basic file system operations (mkdir, list, put, cat) and
>>>>>>> confirmed that everything was working
>>>>>>> - Run word count, pi and DFSIOTest
>>>>>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> Ajay
>>>>>>> 
>>>>>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>>>>>> 
>>>>>>>    +1 (binding)
>>>>>>> 
>>>>>>>    - downloaded src tarball, verified md5
>>>>>>>    - built from source with jdk1.8.0_112
>>>>>>>    - started a pseudo cluster with hdfs and kms
>>>>>>>    - sanity checked encryption related operations working
>>>>>>>    - sanity checked webui and logs.
>>>>>>> 
>>>>>>>    -Xiao
>>>>>>> 
>>>>>>>    On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
>> atm@apache.org>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> +1 (binding)
>>>>>>>> 
>>>>>>>> - downloaded the src tarball and built the source (-Pdist
>>>> -Pnative)
>>>>>>>> - verified the checksum
>>>>>>>> - brought up a secure pseudo distributed cluster
>>>>>>>> - did some basic file system operations (mkdir, list, put,
>> cat)
>>>> and
>>>>>>>> confirmed that everything was working
>>>>>>>> - confirmed that the web UI worked
>>>>>>>> 
>>>>>>>> Best,
>>>>>>>> Aaron
>>>>>>>> 
>>>>>>>> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>>>>>>> andrew.wang@cloudera.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hi all,
>>>>>>>>> 
>>>>>>>>> Let me start, as always, by thanking the efforts of all the
>>>>>>> contributors
>>>>>>>>> who contributed to this release, especially those who
>> jumped
>>> on
>>>>>>> the
>>>>>>>> issues
>>>>>>>>> found in RC0.
>>>>>>>>> 
>>>>>>>>> I've prepared RC1 for Apache Hadoop 3.0.0. This release
>>>>>>> incorporates 302
>>>>>>>>> fixed JIRAs since the previous 3.0.0-beta1 release.
>>>>>>>>> 
>>>>>>>>> You can find the artifacts here:
>>>>>>>>> 
>>>>>>>>> http://home.apache.org/~wang/3.0.0-RC1/
>>>>>>>>> 
>>>>>>>>> I've done the traditional testing of building from the
>> source
>>>>>>> tarball and
>>>>>>>>> running a Pi job on a single node cluster. I also verified
>>> that
>>>>>>> the
>>>>>>>> shaded
>>>>>>>>> jars are not empty.
>>>>>>>>> 
>>>>>>>>> Found one issue that create-release (probably due to the
>> mvn
>>>>>>> deploy
>>>>>>>> change)
>>>>>>>>> didn't sign the artifacts, but I fixed that by calling mvn
>>> one
>>>>>>> more time.
>>>>>>>>> Available here:
>>>>>>>>> 
>>>>>>>>> https://repository.apache.org/
>> content/repositories/orgapache
>>>>>>> hadoop-1075/
>>>>>>>>> 
>>>>>>>>> This release will run the standard 5 days, closing on Dec
>>> 13th
>>>> at
>>>>>>> 12:31pm
>>>>>>>>> Pacific. My +1 to start.
>>>>>>>>> 
>>>>>>>>> Best,
>>>>>>>>> Andrew
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> ------------------------------------------------------------
>>> ---------
>>>>>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>>>>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Lei (Eddy) Xu
>>>> Software Engineer, Cloudera
>>>> 
>>> 
>> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

I was waiting for Daniel to post the minutes from YARN meetup to talk about this. Anyways, in that discussion, we identified a bunch of key upgrade related scenarios that no-one seems to have validated - atleast from the representation in the YARN meetup. I'm going to create a wiki-page listing all these scenarios.

But back to the bug that Junping raised. At this point, we don't have a clear path towards running 2.x applications on 3.0.0 clusters. So, our claim of rolling-upgrades already working is not accurate.

One of the two options that Junping proposed should be pursued before we close the release. I'm in favor of calling out rolling-upgrade support be with-drawn or caveated and push for progress instead of blocking the release.

Thanks
+Vinod

> On Dec 12, 2017, at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:
> 
> Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get chance to validate new RC now.
> 
> Basically, I found two critical issues with the same rolling upgrade scenario as where HADOOP-15059 get found previously:
> HDFS-12920, we changed value format for some hdfs configurations that old version MR client doesn't understand when fetching these configurations. Some quick workarounds are to add old value (without time unit) in hdfs-site.xml to override new default values but will generate many annoying warnings. I provided my fix suggestions on the JIRA already for more discussion.
> The other one is YARN-7646. After we workaround HDFS-12920, will hit the issue that old version MR AppMaster cannot communicate with new version of YARN RM - could be related to resource profile changes from YARN side but root cause are still in investigation.
> 
> The first issue may not belong to a blocker given we can workaround this without code change. I am not sure if we can workaround 2nd issue so far. If not, we may have to fix this or compromise with withdrawing support of rolling upgrade or calling it a stable release.
> 
> 
> Thanks,
> 
> Junping
> 
> ________________________________________
> From: Robert Kanter <rk...@cloudera.com>
> Sent: Tuesday, December 12, 2017 3:10 PM
> To: Arun Suresh
> Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T. Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
> 
> +1 (binding)
> 
> + Downloaded the binary release
> + Deployed on a 3 node cluster on CentOS 7.3
> + Ran some MR jobs, clicked around the UI, etc
> + Ran some CLI commands (yarn logs, etc)
> 
> Good job everyone on Hadoop 3!
> 
> 
> - Robert
> 
> On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
> 
>> +1 (binding)
>> 
>> - Verified signatures of the source tarball.
>> - built from source - using the docker build environment.
>> - set up a pseudo-distributed test cluster.
>> - ran basic HDFS commands
>> - ran some basic MR jobs
>> 
>> Cheers
>> -Arun
>> 
>> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
>> wrote:
>> 
>>> Hi everyone,
>>> 
>>> As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
>>> whack if you have time. There are already enough binding +1s to pass this
>>> vote, but it'd be great to get additional validation.
>>> 
>>> Thanks to everyone who's voted thus far!
>>> 
>>> Best,
>>> Andrew
>>> 
>>> 
>>> 
>>> On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
>>> 
>>>> +1 (binding)
>>>> 
>>>> * Verified src tarball and bin tarball, verified md5 of each.
>>>> * Build source with -Pdist,native
>>>> * Started a pseudo cluster
>>>> * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
>>>> dfs put/get/cat on "/" with XOR-2-1 policy.
>>>> 
>>>> Thanks Andrew for this great effort!
>>>> 
>>>> Best,
>>>> 
>>>> 
>>>> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <andrew.wang@cloudera.com
>>> 
>>>> wrote:
>>>>> Hi Wei-Chiu,
>>>>> 
>>>>> The patchprocess directory is left over from the create-release
>>> process,
>>>>> and it looks empty to me. We should still file a create-release JIRA
>> to
>>>> fix
>>>>> this, but I think this is not a blocker. Would you agree?
>>>>> 
>>>>> Best,
>>>>> Andrew
>>>>> 
>>>>> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
>> weichiu@cloudera.com
>>>> 
>>>>> wrote:
>>>>> 
>>>>>> Hi Andrew, thanks the tremendous effort.
>>>>>> I found an empty "patchprocess" directory in the source tarball,
>> that
>>> is
>>>>>> not there if you clone from github. Any chance you might have some
>>>> leftover
>>>>>> trash when you made the tarball?
>>>>>> Not wanting to nitpicking, but you might want to double check so we
>>>> don't
>>>>>> ship anything private to you in public :)
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
>>> ajay.kumar@hortonworks.com
>>>>> 
>>>>>> wrote:
>>>>>> 
>>>>>>> +1 (non-binding)
>>>>>>> Thanks for driving this, Andrew Wang!!
>>>>>>> 
>>>>>>> - downloaded the src tarball and verified md5 checksum
>>>>>>> - built from source with jdk 1.8.0_111-b14
>>>>>>> - brought up a pseudo distributed cluster
>>>>>>> - did basic file system operations (mkdir, list, put, cat) and
>>>>>>> confirmed that everything was working
>>>>>>> - Run word count, pi and DFSIOTest
>>>>>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> Ajay
>>>>>>> 
>>>>>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>>>>>> 
>>>>>>>    +1 (binding)
>>>>>>> 
>>>>>>>    - downloaded src tarball, verified md5
>>>>>>>    - built from source with jdk1.8.0_112
>>>>>>>    - started a pseudo cluster with hdfs and kms
>>>>>>>    - sanity checked encryption related operations working
>>>>>>>    - sanity checked webui and logs.
>>>>>>> 
>>>>>>>    -Xiao
>>>>>>> 
>>>>>>>    On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
>> atm@apache.org>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> +1 (binding)
>>>>>>>> 
>>>>>>>> - downloaded the src tarball and built the source (-Pdist
>>>> -Pnative)
>>>>>>>> - verified the checksum
>>>>>>>> - brought up a secure pseudo distributed cluster
>>>>>>>> - did some basic file system operations (mkdir, list, put,
>> cat)
>>>> and
>>>>>>>> confirmed that everything was working
>>>>>>>> - confirmed that the web UI worked
>>>>>>>> 
>>>>>>>> Best,
>>>>>>>> Aaron
>>>>>>>> 
>>>>>>>> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>>>>>>> andrew.wang@cloudera.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hi all,
>>>>>>>>> 
>>>>>>>>> Let me start, as always, by thanking the efforts of all the
>>>>>>> contributors
>>>>>>>>> who contributed to this release, especially those who
>> jumped
>>> on
>>>>>>> the
>>>>>>>> issues
>>>>>>>>> found in RC0.
>>>>>>>>> 
>>>>>>>>> I've prepared RC1 for Apache Hadoop 3.0.0. This release
>>>>>>> incorporates 302
>>>>>>>>> fixed JIRAs since the previous 3.0.0-beta1 release.
>>>>>>>>> 
>>>>>>>>> You can find the artifacts here:
>>>>>>>>> 
>>>>>>>>> http://home.apache.org/~wang/3.0.0-RC1/
>>>>>>>>> 
>>>>>>>>> I've done the traditional testing of building from the
>> source
>>>>>>> tarball and
>>>>>>>>> running a Pi job on a single node cluster. I also verified
>>> that
>>>>>>> the
>>>>>>>> shaded
>>>>>>>>> jars are not empty.
>>>>>>>>> 
>>>>>>>>> Found one issue that create-release (probably due to the
>> mvn
>>>>>>> deploy
>>>>>>>> change)
>>>>>>>>> didn't sign the artifacts, but I fixed that by calling mvn
>>> one
>>>>>>> more time.
>>>>>>>>> Available here:
>>>>>>>>> 
>>>>>>>>> https://repository.apache.org/
>> content/repositories/orgapache
>>>>>>> hadoop-1075/
>>>>>>>>> 
>>>>>>>>> This release will run the standard 5 days, closing on Dec
>>> 13th
>>>> at
>>>>>>> 12:31pm
>>>>>>>>> Pacific. My +1 to start.
>>>>>>>>> 
>>>>>>>>> Best,
>>>>>>>>> Andrew
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> ------------------------------------------------------------
>>> ---------
>>>>>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>>>>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Lei (Eddy) Xu
>>>> Software Engineer, Cloudera
>>>> 
>>> 
>> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

I was waiting for Daniel to post the minutes from YARN meetup to talk about this. Anyways, in that discussion, we identified a bunch of key upgrade related scenarios that no-one seems to have validated - atleast from the representation in the YARN meetup. I'm going to create a wiki-page listing all these scenarios.

But back to the bug that Junping raised. At this point, we don't have a clear path towards running 2.x applications on 3.0.0 clusters. So, our claim of rolling-upgrades already working is not accurate.

One of the two options that Junping proposed should be pursued before we close the release. I'm in favor of calling out rolling-upgrade support be with-drawn or caveated and push for progress instead of blocking the release.

Thanks
+Vinod

> On Dec 12, 2017, at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:
> 
> Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get chance to validate new RC now.
> 
> Basically, I found two critical issues with the same rolling upgrade scenario as where HADOOP-15059 get found previously:
> HDFS-12920, we changed value format for some hdfs configurations that old version MR client doesn't understand when fetching these configurations. Some quick workarounds are to add old value (without time unit) in hdfs-site.xml to override new default values but will generate many annoying warnings. I provided my fix suggestions on the JIRA already for more discussion.
> The other one is YARN-7646. After we workaround HDFS-12920, will hit the issue that old version MR AppMaster cannot communicate with new version of YARN RM - could be related to resource profile changes from YARN side but root cause are still in investigation.
> 
> The first issue may not belong to a blocker given we can workaround this without code change. I am not sure if we can workaround 2nd issue so far. If not, we may have to fix this or compromise with withdrawing support of rolling upgrade or calling it a stable release.
> 
> 
> Thanks,
> 
> Junping
> 
> ________________________________________
> From: Robert Kanter <rk...@cloudera.com>
> Sent: Tuesday, December 12, 2017 3:10 PM
> To: Arun Suresh
> Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T. Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
> 
> +1 (binding)
> 
> + Downloaded the binary release
> + Deployed on a 3 node cluster on CentOS 7.3
> + Ran some MR jobs, clicked around the UI, etc
> + Ran some CLI commands (yarn logs, etc)
> 
> Good job everyone on Hadoop 3!
> 
> 
> - Robert
> 
> On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
> 
>> +1 (binding)
>> 
>> - Verified signatures of the source tarball.
>> - built from source - using the docker build environment.
>> - set up a pseudo-distributed test cluster.
>> - ran basic HDFS commands
>> - ran some basic MR jobs
>> 
>> Cheers
>> -Arun
>> 
>> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
>> wrote:
>> 
>>> Hi everyone,
>>> 
>>> As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
>>> whack if you have time. There are already enough binding +1s to pass this
>>> vote, but it'd be great to get additional validation.
>>> 
>>> Thanks to everyone who's voted thus far!
>>> 
>>> Best,
>>> Andrew
>>> 
>>> 
>>> 
>>> On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
>>> 
>>>> +1 (binding)
>>>> 
>>>> * Verified src tarball and bin tarball, verified md5 of each.
>>>> * Build source with -Pdist,native
>>>> * Started a pseudo cluster
>>>> * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
>>>> dfs put/get/cat on "/" with XOR-2-1 policy.
>>>> 
>>>> Thanks Andrew for this great effort!
>>>> 
>>>> Best,
>>>> 
>>>> 
>>>> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <andrew.wang@cloudera.com
>>> 
>>>> wrote:
>>>>> Hi Wei-Chiu,
>>>>> 
>>>>> The patchprocess directory is left over from the create-release
>>> process,
>>>>> and it looks empty to me. We should still file a create-release JIRA
>> to
>>>> fix
>>>>> this, but I think this is not a blocker. Would you agree?
>>>>> 
>>>>> Best,
>>>>> Andrew
>>>>> 
>>>>> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
>> weichiu@cloudera.com
>>>> 
>>>>> wrote:
>>>>> 
>>>>>> Hi Andrew, thanks the tremendous effort.
>>>>>> I found an empty "patchprocess" directory in the source tarball,
>> that
>>> is
>>>>>> not there if you clone from github. Any chance you might have some
>>>> leftover
>>>>>> trash when you made the tarball?
>>>>>> Not wanting to nitpicking, but you might want to double check so we
>>>> don't
>>>>>> ship anything private to you in public :)
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
>>> ajay.kumar@hortonworks.com
>>>>> 
>>>>>> wrote:
>>>>>> 
>>>>>>> +1 (non-binding)
>>>>>>> Thanks for driving this, Andrew Wang!!
>>>>>>> 
>>>>>>> - downloaded the src tarball and verified md5 checksum
>>>>>>> - built from source with jdk 1.8.0_111-b14
>>>>>>> - brought up a pseudo distributed cluster
>>>>>>> - did basic file system operations (mkdir, list, put, cat) and
>>>>>>> confirmed that everything was working
>>>>>>> - Run word count, pi and DFSIOTest
>>>>>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> Ajay
>>>>>>> 
>>>>>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>>>>>> 
>>>>>>>    +1 (binding)
>>>>>>> 
>>>>>>>    - downloaded src tarball, verified md5
>>>>>>>    - built from source with jdk1.8.0_112
>>>>>>>    - started a pseudo cluster with hdfs and kms
>>>>>>>    - sanity checked encryption related operations working
>>>>>>>    - sanity checked webui and logs.
>>>>>>> 
>>>>>>>    -Xiao
>>>>>>> 
>>>>>>>    On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
>> atm@apache.org>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> +1 (binding)
>>>>>>>> 
>>>>>>>> - downloaded the src tarball and built the source (-Pdist
>>>> -Pnative)
>>>>>>>> - verified the checksum
>>>>>>>> - brought up a secure pseudo distributed cluster
>>>>>>>> - did some basic file system operations (mkdir, list, put,
>> cat)
>>>> and
>>>>>>>> confirmed that everything was working
>>>>>>>> - confirmed that the web UI worked
>>>>>>>> 
>>>>>>>> Best,
>>>>>>>> Aaron
>>>>>>>> 
>>>>>>>> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>>>>>>> andrew.wang@cloudera.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Hi all,
>>>>>>>>> 
>>>>>>>>> Let me start, as always, by thanking the efforts of all the
>>>>>>> contributors
>>>>>>>>> who contributed to this release, especially those who
>> jumped
>>> on
>>>>>>> the
>>>>>>>> issues
>>>>>>>>> found in RC0.
>>>>>>>>> 
>>>>>>>>> I've prepared RC1 for Apache Hadoop 3.0.0. This release
>>>>>>> incorporates 302
>>>>>>>>> fixed JIRAs since the previous 3.0.0-beta1 release.
>>>>>>>>> 
>>>>>>>>> You can find the artifacts here:
>>>>>>>>> 
>>>>>>>>> http://home.apache.org/~wang/3.0.0-RC1/
>>>>>>>>> 
>>>>>>>>> I've done the traditional testing of building from the
>> source
>>>>>>> tarball and
>>>>>>>>> running a Pi job on a single node cluster. I also verified
>>> that
>>>>>>> the
>>>>>>>> shaded
>>>>>>>>> jars are not empty.
>>>>>>>>> 
>>>>>>>>> Found one issue that create-release (probably due to the
>> mvn
>>>>>>> deploy
>>>>>>>> change)
>>>>>>>>> didn't sign the artifacts, but I fixed that by calling mvn
>>> one
>>>>>>> more time.
>>>>>>>>> Available here:
>>>>>>>>> 
>>>>>>>>> https://repository.apache.org/
>> content/repositories/orgapache
>>>>>>> hadoop-1075/
>>>>>>>>> 
>>>>>>>>> This release will run the standard 5 days, closing on Dec
>>> 13th
>>>> at
>>>>>>> 12:31pm
>>>>>>>>> Pacific. My +1 to start.
>>>>>>>>> 
>>>>>>>>> Best,
>>>>>>>>> Andrew
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> ------------------------------------------------------------
>>> ---------
>>>>>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>>>>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Lei (Eddy) Xu
>>>> Software Engineer, Cloudera
>>>> 
>>> 
>> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Jonathan Hung <jy...@gmail.com>.

Thanks Andrew for the huge effort.

+1 (non-binding)
- Downloaded binary tarball and verified md5
- Ran RM HA and verified manual failover
- Verified add/remove/update scheduler configuration API (CLI/REST) works
for leveldb/zookeeper backend
- Verified scheduler configuration changes persisted on restart/failover
- Verified "yarn rmadmin -refreshQueues" works when scheduler configuration
API disabled, and does not work when scheduler configuration API enabled


Jonathan Hung

On Tue, Dec 12, 2017 at 5:44 PM, Junping Du <jd...@hortonworks.com> wrote:

> Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get
> chance to validate new RC now.
>
> Basically, I found two critical issues with the same rolling upgrade
> scenario as where HADOOP-15059 get found previously:
> HDFS-12920, we changed value format for some hdfs configurations that old
> version MR client doesn't understand when fetching these configurations.
> Some quick workarounds are to add old value (without time unit) in
> hdfs-site.xml to override new default values but will generate many
> annoying warnings. I provided my fix suggestions on the JIRA already for
> more discussion.
> The other one is YARN-7646. After we workaround HDFS-12920, will hit the
> issue that old version MR AppMaster cannot communicate with new version of
> YARN RM - could be related to resource profile changes from YARN side but
> root cause are still in investigation.
>
> The first issue may not belong to a blocker given we can workaround this
> without code change. I am not sure if we can workaround 2nd issue so far.
> If not, we may have to fix this or compromise with withdrawing support of
> rolling upgrade or calling it a stable release.
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Robert Kanter <rk...@cloudera.com>
> Sent: Tuesday, December 12, 2017 3:10 PM
> To: Arun Suresh
> Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T.
> Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1
>
> +1 (binding)
>
> + Downloaded the binary release
> + Deployed on a 3 node cluster on CentOS 7.3
> + Ran some MR jobs, clicked around the UI, etc
> + Ran some CLI commands (yarn logs, etc)
>
> Good job everyone on Hadoop 3!
>
>
> - Robert
>
> On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:
>
> > +1 (binding)
> >
> > - Verified signatures of the source tarball.
> > - built from source - using the docker build environment.
> > - set up a pseudo-distributed test cluster.
> > - ran basic HDFS commands
> > - ran some basic MR jobs
> >
> > Cheers
> > -Arun
> >
> > On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> > wrote:
> >
> > > Hi everyone,
> > >
> > > As a reminder, this vote closes tomorrow at 12:31pm, so please give it
> a
> > > whack if you have time. There are already enough binding +1s to pass
> this
> > > vote, but it'd be great to get additional validation.
> > >
> > > Thanks to everyone who's voted thus far!
> > >
> > > Best,
> > > Andrew
> > >
> > >
> > >
> > > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > * Verified src tarball and bin tarball, verified md5 of each.
> > > > * Build source with -Pdist,native
> > > > * Started a pseudo cluster
> > > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > > >
> > > > Thanks Andrew for this great effort!
> > > >
> > > > Best,
> > > >
> > > >
> > > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <
> andrew.wang@cloudera.com
> > >
> > > > wrote:
> > > > > Hi Wei-Chiu,
> > > > >
> > > > > The patchprocess directory is left over from the create-release
> > > process,
> > > > > and it looks empty to me. We should still file a create-release
> JIRA
> > to
> > > > fix
> > > > > this, but I think this is not a blocker. Would you agree?
> > > > >
> > > > > Best,
> > > > > Andrew
> > > > >
> > > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> > weichiu@cloudera.com
> > > >
> > > > > wrote:
> > > > >
> > > > >> Hi Andrew, thanks the tremendous effort.
> > > > >> I found an empty "patchprocess" directory in the source tarball,
> > that
> > > is
> > > > >> not there if you clone from github. Any chance you might have some
> > > > leftover
> > > > >> trash when you made the tarball?
> > > > >> Not wanting to nitpicking, but you might want to double check so
> we
> > > > don't
> > > > >> ship anything private to you in public :)
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > > ajay.kumar@hortonworks.com
> > > > >
> > > > >> wrote:
> > > > >>
> > > > >>> +1 (non-binding)
> > > > >>> Thanks for driving this, Andrew Wang!!
> > > > >>>
> > > > >>> - downloaded the src tarball and verified md5 checksum
> > > > >>> - built from source with jdk 1.8.0_111-b14
> > > > >>> - brought up a pseudo distributed cluster
> > > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > > >>> confirmed that everything was working
> > > > >>> - Run word count, pi and DFSIOTest
> > > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > > >>>
> > > > >>> Cheers,
> > > > >>> Ajay
> > > > >>>
> > > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > > >>>
> > > > >>>     +1 (binding)
> > > > >>>
> > > > >>>     - downloaded src tarball, verified md5
> > > > >>>     - built from source with jdk1.8.0_112
> > > > >>>     - started a pseudo cluster with hdfs and kms
> > > > >>>     - sanity checked encryption related operations working
> > > > >>>     - sanity checked webui and logs.
> > > > >>>
> > > > >>>     -Xiao
> > > > >>>
> > > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> > atm@apache.org>
> > > > >>> wrote:
> > > > >>>
> > > > >>>     > +1 (binding)
> > > > >>>     >
> > > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > > -Pnative)
> > > > >>>     > - verified the checksum
> > > > >>>     > - brought up a secure pseudo distributed cluster
> > > > >>>     > - did some basic file system operations (mkdir, list, put,
> > cat)
> > > > and
> > > > >>>     > confirmed that everything was working
> > > > >>>     > - confirmed that the web UI worked
> > > > >>>     >
> > > > >>>     > Best,
> > > > >>>     > Aaron
> > > > >>>     >
> > > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > > >>> andrew.wang@cloudera.com>
> > > > >>>     > wrote:
> > > > >>>     >
> > > > >>>     > > Hi all,
> > > > >>>     > >
> > > > >>>     > > Let me start, as always, by thanking the efforts of all
> the
> > > > >>> contributors
> > > > >>>     > > who contributed to this release, especially those who
> > jumped
> > > on
> > > > >>> the
> > > > >>>     > issues
> > > > >>>     > > found in RC0.
> > > > >>>     > >
> > > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > > >>> incorporates 302
> > > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > > >>>     > >
> > > > >>>     > > You can find the artifacts here:
> > > > >>>     > >
> > > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > > >>>     > >
> > > > >>>     > > I've done the traditional testing of building from the
> > source
> > > > >>> tarball and
> > > > >>>     > > running a Pi job on a single node cluster. I also
> verified
> > > that
> > > > >>> the
> > > > >>>     > shaded
> > > > >>>     > > jars are not empty.
> > > > >>>     > >
> > > > >>>     > > Found one issue that create-release (probably due to the
> > mvn
> > > > >>> deploy
> > > > >>>     > change)
> > > > >>>     > > didn't sign the artifacts, but I fixed that by calling
> mvn
> > > one
> > > > >>> more time.
> > > > >>>     > > Available here:
> > > > >>>     > >
> > > > >>>     > > https://repository.apache.org/
> > content/repositories/orgapache
> > > > >>> hadoop-1075/
> > > > >>>     > >
> > > > >>>     > > This release will run the standard 5 days, closing on Dec
> > > 13th
> > > > at
> > > > >>> 12:31pm
> > > > >>>     > > Pacific. My +1 to start.
> > > > >>>     > >
> > > > >>>     > > Best,
> > > > >>>     > > Andrew
> > > > >>>     > >
> > > > >>>     >
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> ------------------------------------------------------------
> > > ---------
> > > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.
> org
> > > > >>>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Lei (Eddy) Xu
> > > > Software Engineer, Cloudera
> > > >
> > >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Junping Du <jd...@hortonworks.com>.

Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get chance to validate new RC now.

Basically, I found two critical issues with the same rolling upgrade scenario as where HADOOP-15059 get found previously:
HDFS-12920, we changed value format for some hdfs configurations that old version MR client doesn't understand when fetching these configurations. Some quick workarounds are to add old value (without time unit) in hdfs-site.xml to override new default values but will generate many annoying warnings. I provided my fix suggestions on the JIRA already for more discussion.
The other one is YARN-7646. After we workaround HDFS-12920, will hit the issue that old version MR AppMaster cannot communicate with new version of YARN RM - could be related to resource profile changes from YARN side but root cause are still in investigation.

The first issue may not belong to a blocker given we can workaround this without code change. I am not sure if we can workaround 2nd issue so far. If not, we may have to fix this or compromise with withdrawing support of rolling upgrade or calling it a stable release.


Thanks,

Junping

________________________________________
From: Robert Kanter <rk...@cloudera.com>
Sent: Tuesday, December 12, 2017 3:10 PM
To: Arun Suresh
Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T. Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

+1 (binding)

+ Downloaded the binary release
+ Deployed on a 3 node cluster on CentOS 7.3
+ Ran some MR jobs, clicked around the UI, etc
+ Ran some CLI commands (yarn logs, etc)

Good job everyone on Hadoop 3!


- Robert

On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:

> +1 (binding)
>
> - Verified signatures of the source tarball.
> - built from source - using the docker build environment.
> - set up a pseudo-distributed test cluster.
> - ran basic HDFS commands
> - ran some basic MR jobs
>
> Cheers
> -Arun
>
> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi everyone,
> >
> > As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
> > whack if you have time. There are already enough binding +1s to pass this
> > vote, but it'd be great to get additional validation.
> >
> > Thanks to everyone who's voted thus far!
> >
> > Best,
> > Andrew
> >
> >
> >
> > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> >
> > > +1 (binding)
> > >
> > > * Verified src tarball and bin tarball, verified md5 of each.
> > > * Build source with -Pdist,native
> > > * Started a pseudo cluster
> > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > >
> > > Thanks Andrew for this great effort!
> > >
> > > Best,
> > >
> > >
> > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <andrew.wang@cloudera.com
> >
> > > wrote:
> > > > Hi Wei-Chiu,
> > > >
> > > > The patchprocess directory is left over from the create-release
> > process,
> > > > and it looks empty to me. We should still file a create-release JIRA
> to
> > > fix
> > > > this, but I think this is not a blocker. Would you agree?
> > > >
> > > > Best,
> > > > Andrew
> > > >
> > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> weichiu@cloudera.com
> > >
> > > > wrote:
> > > >
> > > >> Hi Andrew, thanks the tremendous effort.
> > > >> I found an empty "patchprocess" directory in the source tarball,
> that
> > is
> > > >> not there if you clone from github. Any chance you might have some
> > > leftover
> > > >> trash when you made the tarball?
> > > >> Not wanting to nitpicking, but you might want to double check so we
> > > don't
> > > >> ship anything private to you in public :)
> > > >>
> > > >>
> > > >>
> > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > ajay.kumar@hortonworks.com
> > > >
> > > >> wrote:
> > > >>
> > > >>> +1 (non-binding)
> > > >>> Thanks for driving this, Andrew Wang!!
> > > >>>
> > > >>> - downloaded the src tarball and verified md5 checksum
> > > >>> - built from source with jdk 1.8.0_111-b14
> > > >>> - brought up a pseudo distributed cluster
> > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > >>> confirmed that everything was working
> > > >>> - Run word count, pi and DFSIOTest
> > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > >>>
> > > >>> Cheers,
> > > >>> Ajay
> > > >>>
> > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > >>>
> > > >>>     +1 (binding)
> > > >>>
> > > >>>     - downloaded src tarball, verified md5
> > > >>>     - built from source with jdk1.8.0_112
> > > >>>     - started a pseudo cluster with hdfs and kms
> > > >>>     - sanity checked encryption related operations working
> > > >>>     - sanity checked webui and logs.
> > > >>>
> > > >>>     -Xiao
> > > >>>
> > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> atm@apache.org>
> > > >>> wrote:
> > > >>>
> > > >>>     > +1 (binding)
> > > >>>     >
> > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > -Pnative)
> > > >>>     > - verified the checksum
> > > >>>     > - brought up a secure pseudo distributed cluster
> > > >>>     > - did some basic file system operations (mkdir, list, put,
> cat)
> > > and
> > > >>>     > confirmed that everything was working
> > > >>>     > - confirmed that the web UI worked
> > > >>>     >
> > > >>>     > Best,
> > > >>>     > Aaron
> > > >>>     >
> > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > >>> andrew.wang@cloudera.com>
> > > >>>     > wrote:
> > > >>>     >
> > > >>>     > > Hi all,
> > > >>>     > >
> > > >>>     > > Let me start, as always, by thanking the efforts of all the
> > > >>> contributors
> > > >>>     > > who contributed to this release, especially those who
> jumped
> > on
> > > >>> the
> > > >>>     > issues
> > > >>>     > > found in RC0.
> > > >>>     > >
> > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > >>> incorporates 302
> > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > >>>     > >
> > > >>>     > > You can find the artifacts here:
> > > >>>     > >
> > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > >>>     > >
> > > >>>     > > I've done the traditional testing of building from the
> source
> > > >>> tarball and
> > > >>>     > > running a Pi job on a single node cluster. I also verified
> > that
> > > >>> the
> > > >>>     > shaded
> > > >>>     > > jars are not empty.
> > > >>>     > >
> > > >>>     > > Found one issue that create-release (probably due to the
> mvn
> > > >>> deploy
> > > >>>     > change)
> > > >>>     > > didn't sign the artifacts, but I fixed that by calling mvn
> > one
> > > >>> more time.
> > > >>>     > > Available here:
> > > >>>     > >
> > > >>>     > > https://repository.apache.org/
> content/repositories/orgapache
> > > >>> hadoop-1075/
> > > >>>     > >
> > > >>>     > > This release will run the standard 5 days, closing on Dec
> > 13th
> > > at
> > > >>> 12:31pm
> > > >>>     > > Pacific. My +1 to start.
> > > >>>     > >
> > > >>>     > > Best,
> > > >>>     > > Andrew
> > > >>>     > >
> > > >>>     >
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> ------------------------------------------------------------
> > ---------
> > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >>
> > >
> > >
> > >
> > > --
> > > Lei (Eddy) Xu
> > > Software Engineer, Cloudera
> > >
> >
>


---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Junping Du <jd...@hortonworks.com>.

Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get chance to validate new RC now.

Basically, I found two critical issues with the same rolling upgrade scenario as where HADOOP-15059 get found previously:
HDFS-12920, we changed value format for some hdfs configurations that old version MR client doesn't understand when fetching these configurations. Some quick workarounds are to add old value (without time unit) in hdfs-site.xml to override new default values but will generate many annoying warnings. I provided my fix suggestions on the JIRA already for more discussion.
The other one is YARN-7646. After we workaround HDFS-12920, will hit the issue that old version MR AppMaster cannot communicate with new version of YARN RM - could be related to resource profile changes from YARN side but root cause are still in investigation.

The first issue may not belong to a blocker given we can workaround this without code change. I am not sure if we can workaround 2nd issue so far. If not, we may have to fix this or compromise with withdrawing support of rolling upgrade or calling it a stable release.


Thanks,

Junping

________________________________________
From: Robert Kanter <rk...@cloudera.com>
Sent: Tuesday, December 12, 2017 3:10 PM
To: Arun Suresh
Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T. Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

+1 (binding)

+ Downloaded the binary release
+ Deployed on a 3 node cluster on CentOS 7.3
+ Ran some MR jobs, clicked around the UI, etc
+ Ran some CLI commands (yarn logs, etc)

Good job everyone on Hadoop 3!


- Robert

On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:

> +1 (binding)
>
> - Verified signatures of the source tarball.
> - built from source - using the docker build environment.
> - set up a pseudo-distributed test cluster.
> - ran basic HDFS commands
> - ran some basic MR jobs
>
> Cheers
> -Arun
>
> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi everyone,
> >
> > As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
> > whack if you have time. There are already enough binding +1s to pass this
> > vote, but it'd be great to get additional validation.
> >
> > Thanks to everyone who's voted thus far!
> >
> > Best,
> > Andrew
> >
> >
> >
> > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> >
> > > +1 (binding)
> > >
> > > * Verified src tarball and bin tarball, verified md5 of each.
> > > * Build source with -Pdist,native
> > > * Started a pseudo cluster
> > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > >
> > > Thanks Andrew for this great effort!
> > >
> > > Best,
> > >
> > >
> > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <andrew.wang@cloudera.com
> >
> > > wrote:
> > > > Hi Wei-Chiu,
> > > >
> > > > The patchprocess directory is left over from the create-release
> > process,
> > > > and it looks empty to me. We should still file a create-release JIRA
> to
> > > fix
> > > > this, but I think this is not a blocker. Would you agree?
> > > >
> > > > Best,
> > > > Andrew
> > > >
> > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> weichiu@cloudera.com
> > >
> > > > wrote:
> > > >
> > > >> Hi Andrew, thanks the tremendous effort.
> > > >> I found an empty "patchprocess" directory in the source tarball,
> that
> > is
> > > >> not there if you clone from github. Any chance you might have some
> > > leftover
> > > >> trash when you made the tarball?
> > > >> Not wanting to nitpicking, but you might want to double check so we
> > > don't
> > > >> ship anything private to you in public :)
> > > >>
> > > >>
> > > >>
> > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > ajay.kumar@hortonworks.com
> > > >
> > > >> wrote:
> > > >>
> > > >>> +1 (non-binding)
> > > >>> Thanks for driving this, Andrew Wang!!
> > > >>>
> > > >>> - downloaded the src tarball and verified md5 checksum
> > > >>> - built from source with jdk 1.8.0_111-b14
> > > >>> - brought up a pseudo distributed cluster
> > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > >>> confirmed that everything was working
> > > >>> - Run word count, pi and DFSIOTest
> > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > >>>
> > > >>> Cheers,
> > > >>> Ajay
> > > >>>
> > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > >>>
> > > >>>     +1 (binding)
> > > >>>
> > > >>>     - downloaded src tarball, verified md5
> > > >>>     - built from source with jdk1.8.0_112
> > > >>>     - started a pseudo cluster with hdfs and kms
> > > >>>     - sanity checked encryption related operations working
> > > >>>     - sanity checked webui and logs.
> > > >>>
> > > >>>     -Xiao
> > > >>>
> > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> atm@apache.org>
> > > >>> wrote:
> > > >>>
> > > >>>     > +1 (binding)
> > > >>>     >
> > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > -Pnative)
> > > >>>     > - verified the checksum
> > > >>>     > - brought up a secure pseudo distributed cluster
> > > >>>     > - did some basic file system operations (mkdir, list, put,
> cat)
> > > and
> > > >>>     > confirmed that everything was working
> > > >>>     > - confirmed that the web UI worked
> > > >>>     >
> > > >>>     > Best,
> > > >>>     > Aaron
> > > >>>     >
> > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > >>> andrew.wang@cloudera.com>
> > > >>>     > wrote:
> > > >>>     >
> > > >>>     > > Hi all,
> > > >>>     > >
> > > >>>     > > Let me start, as always, by thanking the efforts of all the
> > > >>> contributors
> > > >>>     > > who contributed to this release, especially those who
> jumped
> > on
> > > >>> the
> > > >>>     > issues
> > > >>>     > > found in RC0.
> > > >>>     > >
> > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > >>> incorporates 302
> > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > >>>     > >
> > > >>>     > > You can find the artifacts here:
> > > >>>     > >
> > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > >>>     > >
> > > >>>     > > I've done the traditional testing of building from the
> source
> > > >>> tarball and
> > > >>>     > > running a Pi job on a single node cluster. I also verified
> > that
> > > >>> the
> > > >>>     > shaded
> > > >>>     > > jars are not empty.
> > > >>>     > >
> > > >>>     > > Found one issue that create-release (probably due to the
> mvn
> > > >>> deploy
> > > >>>     > change)
> > > >>>     > > didn't sign the artifacts, but I fixed that by calling mvn
> > one
> > > >>> more time.
> > > >>>     > > Available here:
> > > >>>     > >
> > > >>>     > > https://repository.apache.org/
> content/repositories/orgapache
> > > >>> hadoop-1075/
> > > >>>     > >
> > > >>>     > > This release will run the standard 5 days, closing on Dec
> > 13th
> > > at
> > > >>> 12:31pm
> > > >>>     > > Pacific. My +1 to start.
> > > >>>     > >
> > > >>>     > > Best,
> > > >>>     > > Andrew
> > > >>>     > >
> > > >>>     >
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> ------------------------------------------------------------
> > ---------
> > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >>
> > >
> > >
> > >
> > > --
> > > Lei (Eddy) Xu
> > > Software Engineer, Cloudera
> > >
> >
>


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Junping Du <jd...@hortonworks.com>.

Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get chance to validate new RC now.

Basically, I found two critical issues with the same rolling upgrade scenario as where HADOOP-15059 get found previously:
HDFS-12920, we changed value format for some hdfs configurations that old version MR client doesn't understand when fetching these configurations. Some quick workarounds are to add old value (without time unit) in hdfs-site.xml to override new default values but will generate many annoying warnings. I provided my fix suggestions on the JIRA already for more discussion.
The other one is YARN-7646. After we workaround HDFS-12920, will hit the issue that old version MR AppMaster cannot communicate with new version of YARN RM - could be related to resource profile changes from YARN side but root cause are still in investigation.

The first issue may not belong to a blocker given we can workaround this without code change. I am not sure if we can workaround 2nd issue so far. If not, we may have to fix this or compromise with withdrawing support of rolling upgrade or calling it a stable release.


Thanks,

Junping

________________________________________
From: Robert Kanter <rk...@cloudera.com>
Sent: Tuesday, December 12, 2017 3:10 PM
To: Arun Suresh
Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T. Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

+1 (binding)

+ Downloaded the binary release
+ Deployed on a 3 node cluster on CentOS 7.3
+ Ran some MR jobs, clicked around the UI, etc
+ Ran some CLI commands (yarn logs, etc)

Good job everyone on Hadoop 3!


- Robert

On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:

> +1 (binding)
>
> - Verified signatures of the source tarball.
> - built from source - using the docker build environment.
> - set up a pseudo-distributed test cluster.
> - ran basic HDFS commands
> - ran some basic MR jobs
>
> Cheers
> -Arun
>
> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi everyone,
> >
> > As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
> > whack if you have time. There are already enough binding +1s to pass this
> > vote, but it'd be great to get additional validation.
> >
> > Thanks to everyone who's voted thus far!
> >
> > Best,
> > Andrew
> >
> >
> >
> > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> >
> > > +1 (binding)
> > >
> > > * Verified src tarball and bin tarball, verified md5 of each.
> > > * Build source with -Pdist,native
> > > * Started a pseudo cluster
> > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > >
> > > Thanks Andrew for this great effort!
> > >
> > > Best,
> > >
> > >
> > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <andrew.wang@cloudera.com
> >
> > > wrote:
> > > > Hi Wei-Chiu,
> > > >
> > > > The patchprocess directory is left over from the create-release
> > process,
> > > > and it looks empty to me. We should still file a create-release JIRA
> to
> > > fix
> > > > this, but I think this is not a blocker. Would you agree?
> > > >
> > > > Best,
> > > > Andrew
> > > >
> > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> weichiu@cloudera.com
> > >
> > > > wrote:
> > > >
> > > >> Hi Andrew, thanks the tremendous effort.
> > > >> I found an empty "patchprocess" directory in the source tarball,
> that
> > is
> > > >> not there if you clone from github. Any chance you might have some
> > > leftover
> > > >> trash when you made the tarball?
> > > >> Not wanting to nitpicking, but you might want to double check so we
> > > don't
> > > >> ship anything private to you in public :)
> > > >>
> > > >>
> > > >>
> > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > ajay.kumar@hortonworks.com
> > > >
> > > >> wrote:
> > > >>
> > > >>> +1 (non-binding)
> > > >>> Thanks for driving this, Andrew Wang!!
> > > >>>
> > > >>> - downloaded the src tarball and verified md5 checksum
> > > >>> - built from source with jdk 1.8.0_111-b14
> > > >>> - brought up a pseudo distributed cluster
> > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > >>> confirmed that everything was working
> > > >>> - Run word count, pi and DFSIOTest
> > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > >>>
> > > >>> Cheers,
> > > >>> Ajay
> > > >>>
> > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > >>>
> > > >>>     +1 (binding)
> > > >>>
> > > >>>     - downloaded src tarball, verified md5
> > > >>>     - built from source with jdk1.8.0_112
> > > >>>     - started a pseudo cluster with hdfs and kms
> > > >>>     - sanity checked encryption related operations working
> > > >>>     - sanity checked webui and logs.
> > > >>>
> > > >>>     -Xiao
> > > >>>
> > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> atm@apache.org>
> > > >>> wrote:
> > > >>>
> > > >>>     > +1 (binding)
> > > >>>     >
> > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > -Pnative)
> > > >>>     > - verified the checksum
> > > >>>     > - brought up a secure pseudo distributed cluster
> > > >>>     > - did some basic file system operations (mkdir, list, put,
> cat)
> > > and
> > > >>>     > confirmed that everything was working
> > > >>>     > - confirmed that the web UI worked
> > > >>>     >
> > > >>>     > Best,
> > > >>>     > Aaron
> > > >>>     >
> > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > >>> andrew.wang@cloudera.com>
> > > >>>     > wrote:
> > > >>>     >
> > > >>>     > > Hi all,
> > > >>>     > >
> > > >>>     > > Let me start, as always, by thanking the efforts of all the
> > > >>> contributors
> > > >>>     > > who contributed to this release, especially those who
> jumped
> > on
> > > >>> the
> > > >>>     > issues
> > > >>>     > > found in RC0.
> > > >>>     > >
> > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > >>> incorporates 302
> > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > >>>     > >
> > > >>>     > > You can find the artifacts here:
> > > >>>     > >
> > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > >>>     > >
> > > >>>     > > I've done the traditional testing of building from the
> source
> > > >>> tarball and
> > > >>>     > > running a Pi job on a single node cluster. I also verified
> > that
> > > >>> the
> > > >>>     > shaded
> > > >>>     > > jars are not empty.
> > > >>>     > >
> > > >>>     > > Found one issue that create-release (probably due to the
> mvn
> > > >>> deploy
> > > >>>     > change)
> > > >>>     > > didn't sign the artifacts, but I fixed that by calling mvn
> > one
> > > >>> more time.
> > > >>>     > > Available here:
> > > >>>     > >
> > > >>>     > > https://repository.apache.org/
> content/repositories/orgapache
> > > >>> hadoop-1075/
> > > >>>     > >
> > > >>>     > > This release will run the standard 5 days, closing on Dec
> > 13th
> > > at
> > > >>> 12:31pm
> > > >>>     > > Pacific. My +1 to start.
> > > >>>     > >
> > > >>>     > > Best,
> > > >>>     > > Andrew
> > > >>>     > >
> > > >>>     >
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> ------------------------------------------------------------
> > ---------
> > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >>
> > >
> > >
> > >
> > > --
> > > Lei (Eddy) Xu
> > > Software Engineer, Cloudera
> > >
> >
>


---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Junping Du <jd...@hortonworks.com>.

Thanks Andrew for pushing new RC for 3.0.0. I was out last week, just get chance to validate new RC now.

Basically, I found two critical issues with the same rolling upgrade scenario as where HADOOP-15059 get found previously:
HDFS-12920, we changed value format for some hdfs configurations that old version MR client doesn't understand when fetching these configurations. Some quick workarounds are to add old value (without time unit) in hdfs-site.xml to override new default values but will generate many annoying warnings. I provided my fix suggestions on the JIRA already for more discussion.
The other one is YARN-7646. After we workaround HDFS-12920, will hit the issue that old version MR AppMaster cannot communicate with new version of YARN RM - could be related to resource profile changes from YARN side but root cause are still in investigation.

The first issue may not belong to a blocker given we can workaround this without code change. I am not sure if we can workaround 2nd issue so far. If not, we may have to fix this or compromise with withdrawing support of rolling upgrade or calling it a stable release.


Thanks,

Junping

________________________________________
From: Robert Kanter <rk...@cloudera.com>
Sent: Tuesday, December 12, 2017 3:10 PM
To: Arun Suresh
Cc: Andrew Wang; Lei Xu; Wei-Chiu Chuang; Ajay Kumar; Xiao Chen; Aaron T. Myers; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

+1 (binding)

+ Downloaded the binary release
+ Deployed on a 3 node cluster on CentOS 7.3
+ Ran some MR jobs, clicked around the UI, etc
+ Ran some CLI commands (yarn logs, etc)

Good job everyone on Hadoop 3!


- Robert

On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:

> +1 (binding)
>
> - Verified signatures of the source tarball.
> - built from source - using the docker build environment.
> - set up a pseudo-distributed test cluster.
> - ran basic HDFS commands
> - ran some basic MR jobs
>
> Cheers
> -Arun
>
> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi everyone,
> >
> > As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
> > whack if you have time. There are already enough binding +1s to pass this
> > vote, but it'd be great to get additional validation.
> >
> > Thanks to everyone who's voted thus far!
> >
> > Best,
> > Andrew
> >
> >
> >
> > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> >
> > > +1 (binding)
> > >
> > > * Verified src tarball and bin tarball, verified md5 of each.
> > > * Build source with -Pdist,native
> > > * Started a pseudo cluster
> > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > >
> > > Thanks Andrew for this great effort!
> > >
> > > Best,
> > >
> > >
> > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <andrew.wang@cloudera.com
> >
> > > wrote:
> > > > Hi Wei-Chiu,
> > > >
> > > > The patchprocess directory is left over from the create-release
> > process,
> > > > and it looks empty to me. We should still file a create-release JIRA
> to
> > > fix
> > > > this, but I think this is not a blocker. Would you agree?
> > > >
> > > > Best,
> > > > Andrew
> > > >
> > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> weichiu@cloudera.com
> > >
> > > > wrote:
> > > >
> > > >> Hi Andrew, thanks the tremendous effort.
> > > >> I found an empty "patchprocess" directory in the source tarball,
> that
> > is
> > > >> not there if you clone from github. Any chance you might have some
> > > leftover
> > > >> trash when you made the tarball?
> > > >> Not wanting to nitpicking, but you might want to double check so we
> > > don't
> > > >> ship anything private to you in public :)
> > > >>
> > > >>
> > > >>
> > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > ajay.kumar@hortonworks.com
> > > >
> > > >> wrote:
> > > >>
> > > >>> +1 (non-binding)
> > > >>> Thanks for driving this, Andrew Wang!!
> > > >>>
> > > >>> - downloaded the src tarball and verified md5 checksum
> > > >>> - built from source with jdk 1.8.0_111-b14
> > > >>> - brought up a pseudo distributed cluster
> > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > >>> confirmed that everything was working
> > > >>> - Run word count, pi and DFSIOTest
> > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > >>>
> > > >>> Cheers,
> > > >>> Ajay
> > > >>>
> > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > >>>
> > > >>>     +1 (binding)
> > > >>>
> > > >>>     - downloaded src tarball, verified md5
> > > >>>     - built from source with jdk1.8.0_112
> > > >>>     - started a pseudo cluster with hdfs and kms
> > > >>>     - sanity checked encryption related operations working
> > > >>>     - sanity checked webui and logs.
> > > >>>
> > > >>>     -Xiao
> > > >>>
> > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> atm@apache.org>
> > > >>> wrote:
> > > >>>
> > > >>>     > +1 (binding)
> > > >>>     >
> > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > -Pnative)
> > > >>>     > - verified the checksum
> > > >>>     > - brought up a secure pseudo distributed cluster
> > > >>>     > - did some basic file system operations (mkdir, list, put,
> cat)
> > > and
> > > >>>     > confirmed that everything was working
> > > >>>     > - confirmed that the web UI worked
> > > >>>     >
> > > >>>     > Best,
> > > >>>     > Aaron
> > > >>>     >
> > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > >>> andrew.wang@cloudera.com>
> > > >>>     > wrote:
> > > >>>     >
> > > >>>     > > Hi all,
> > > >>>     > >
> > > >>>     > > Let me start, as always, by thanking the efforts of all the
> > > >>> contributors
> > > >>>     > > who contributed to this release, especially those who
> jumped
> > on
> > > >>> the
> > > >>>     > issues
> > > >>>     > > found in RC0.
> > > >>>     > >
> > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > >>> incorporates 302
> > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > >>>     > >
> > > >>>     > > You can find the artifacts here:
> > > >>>     > >
> > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > >>>     > >
> > > >>>     > > I've done the traditional testing of building from the
> source
> > > >>> tarball and
> > > >>>     > > running a Pi job on a single node cluster. I also verified
> > that
> > > >>> the
> > > >>>     > shaded
> > > >>>     > > jars are not empty.
> > > >>>     > >
> > > >>>     > > Found one issue that create-release (probably due to the
> mvn
> > > >>> deploy
> > > >>>     > change)
> > > >>>     > > didn't sign the artifacts, but I fixed that by calling mvn
> > one
> > > >>> more time.
> > > >>>     > > Available here:
> > > >>>     > >
> > > >>>     > > https://repository.apache.org/
> content/repositories/orgapache
> > > >>> hadoop-1075/
> > > >>>     > >
> > > >>>     > > This release will run the standard 5 days, closing on Dec
> > 13th
> > > at
> > > >>> 12:31pm
> > > >>>     > > Pacific. My +1 to start.
> > > >>>     > >
> > > >>>     > > Best,
> > > >>>     > > Andrew
> > > >>>     > >
> > > >>>     >
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> ------------------------------------------------------------
> > ---------
> > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >>
> > >
> > >
> > >
> > > --
> > > Lei (Eddy) Xu
> > > Software Engineer, Cloudera
> > >
> >
>


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Robert Kanter <rk...@cloudera.com>.

+1 (binding)

+ Downloaded the binary release
+ Deployed on a 3 node cluster on CentOS 7.3
+ Ran some MR jobs, clicked around the UI, etc
+ Ran some CLI commands (yarn logs, etc)

Good job everyone on Hadoop 3!


- Robert

On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:

> +1 (binding)
>
> - Verified signatures of the source tarball.
> - built from source - using the docker build environment.
> - set up a pseudo-distributed test cluster.
> - ran basic HDFS commands
> - ran some basic MR jobs
>
> Cheers
> -Arun
>
> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi everyone,
> >
> > As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
> > whack if you have time. There are already enough binding +1s to pass this
> > vote, but it'd be great to get additional validation.
> >
> > Thanks to everyone who's voted thus far!
> >
> > Best,
> > Andrew
> >
> >
> >
> > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> >
> > > +1 (binding)
> > >
> > > * Verified src tarball and bin tarball, verified md5 of each.
> > > * Build source with -Pdist,native
> > > * Started a pseudo cluster
> > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > >
> > > Thanks Andrew for this great effort!
> > >
> > > Best,
> > >
> > >
> > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <andrew.wang@cloudera.com
> >
> > > wrote:
> > > > Hi Wei-Chiu,
> > > >
> > > > The patchprocess directory is left over from the create-release
> > process,
> > > > and it looks empty to me. We should still file a create-release JIRA
> to
> > > fix
> > > > this, but I think this is not a blocker. Would you agree?
> > > >
> > > > Best,
> > > > Andrew
> > > >
> > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> weichiu@cloudera.com
> > >
> > > > wrote:
> > > >
> > > >> Hi Andrew, thanks the tremendous effort.
> > > >> I found an empty "patchprocess" directory in the source tarball,
> that
> > is
> > > >> not there if you clone from github. Any chance you might have some
> > > leftover
> > > >> trash when you made the tarball?
> > > >> Not wanting to nitpicking, but you might want to double check so we
> > > don't
> > > >> ship anything private to you in public :)
> > > >>
> > > >>
> > > >>
> > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > ajay.kumar@hortonworks.com
> > > >
> > > >> wrote:
> > > >>
> > > >>> +1 (non-binding)
> > > >>> Thanks for driving this, Andrew Wang!!
> > > >>>
> > > >>> - downloaded the src tarball and verified md5 checksum
> > > >>> - built from source with jdk 1.8.0_111-b14
> > > >>> - brought up a pseudo distributed cluster
> > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > >>> confirmed that everything was working
> > > >>> - Run word count, pi and DFSIOTest
> > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > >>>
> > > >>> Cheers,
> > > >>> Ajay
> > > >>>
> > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > >>>
> > > >>>     +1 (binding)
> > > >>>
> > > >>>     - downloaded src tarball, verified md5
> > > >>>     - built from source with jdk1.8.0_112
> > > >>>     - started a pseudo cluster with hdfs and kms
> > > >>>     - sanity checked encryption related operations working
> > > >>>     - sanity checked webui and logs.
> > > >>>
> > > >>>     -Xiao
> > > >>>
> > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> atm@apache.org>
> > > >>> wrote:
> > > >>>
> > > >>>     > +1 (binding)
> > > >>>     >
> > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > -Pnative)
> > > >>>     > - verified the checksum
> > > >>>     > - brought up a secure pseudo distributed cluster
> > > >>>     > - did some basic file system operations (mkdir, list, put,
> cat)
> > > and
> > > >>>     > confirmed that everything was working
> > > >>>     > - confirmed that the web UI worked
> > > >>>     >
> > > >>>     > Best,
> > > >>>     > Aaron
> > > >>>     >
> > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > >>> andrew.wang@cloudera.com>
> > > >>>     > wrote:
> > > >>>     >
> > > >>>     > > Hi all,
> > > >>>     > >
> > > >>>     > > Let me start, as always, by thanking the efforts of all the
> > > >>> contributors
> > > >>>     > > who contributed to this release, especially those who
> jumped
> > on
> > > >>> the
> > > >>>     > issues
> > > >>>     > > found in RC0.
> > > >>>     > >
> > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > >>> incorporates 302
> > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > >>>     > >
> > > >>>     > > You can find the artifacts here:
> > > >>>     > >
> > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > >>>     > >
> > > >>>     > > I've done the traditional testing of building from the
> source
> > > >>> tarball and
> > > >>>     > > running a Pi job on a single node cluster. I also verified
> > that
> > > >>> the
> > > >>>     > shaded
> > > >>>     > > jars are not empty.
> > > >>>     > >
> > > >>>     > > Found one issue that create-release (probably due to the
> mvn
> > > >>> deploy
> > > >>>     > change)
> > > >>>     > > didn't sign the artifacts, but I fixed that by calling mvn
> > one
> > > >>> more time.
> > > >>>     > > Available here:
> > > >>>     > >
> > > >>>     > > https://repository.apache.org/
> content/repositories/orgapache
> > > >>> hadoop-1075/
> > > >>>     > >
> > > >>>     > > This release will run the standard 5 days, closing on Dec
> > 13th
> > > at
> > > >>> 12:31pm
> > > >>>     > > Pacific. My +1 to start.
> > > >>>     > >
> > > >>>     > > Best,
> > > >>>     > > Andrew
> > > >>>     > >
> > > >>>     >
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> ------------------------------------------------------------
> > ---------
> > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >>
> > >
> > >
> > >
> > > --
> > > Lei (Eddy) Xu
> > > Software Engineer, Cloudera
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Robert Kanter <rk...@cloudera.com>.

+1 (binding)

+ Downloaded the binary release
+ Deployed on a 3 node cluster on CentOS 7.3
+ Ran some MR jobs, clicked around the UI, etc
+ Ran some CLI commands (yarn logs, etc)

Good job everyone on Hadoop 3!


- Robert

On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:

> +1 (binding)
>
> - Verified signatures of the source tarball.
> - built from source - using the docker build environment.
> - set up a pseudo-distributed test cluster.
> - ran basic HDFS commands
> - ran some basic MR jobs
>
> Cheers
> -Arun
>
> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi everyone,
> >
> > As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
> > whack if you have time. There are already enough binding +1s to pass this
> > vote, but it'd be great to get additional validation.
> >
> > Thanks to everyone who's voted thus far!
> >
> > Best,
> > Andrew
> >
> >
> >
> > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> >
> > > +1 (binding)
> > >
> > > * Verified src tarball and bin tarball, verified md5 of each.
> > > * Build source with -Pdist,native
> > > * Started a pseudo cluster
> > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > >
> > > Thanks Andrew for this great effort!
> > >
> > > Best,
> > >
> > >
> > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <andrew.wang@cloudera.com
> >
> > > wrote:
> > > > Hi Wei-Chiu,
> > > >
> > > > The patchprocess directory is left over from the create-release
> > process,
> > > > and it looks empty to me. We should still file a create-release JIRA
> to
> > > fix
> > > > this, but I think this is not a blocker. Would you agree?
> > > >
> > > > Best,
> > > > Andrew
> > > >
> > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> weichiu@cloudera.com
> > >
> > > > wrote:
> > > >
> > > >> Hi Andrew, thanks the tremendous effort.
> > > >> I found an empty "patchprocess" directory in the source tarball,
> that
> > is
> > > >> not there if you clone from github. Any chance you might have some
> > > leftover
> > > >> trash when you made the tarball?
> > > >> Not wanting to nitpicking, but you might want to double check so we
> > > don't
> > > >> ship anything private to you in public :)
> > > >>
> > > >>
> > > >>
> > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > ajay.kumar@hortonworks.com
> > > >
> > > >> wrote:
> > > >>
> > > >>> +1 (non-binding)
> > > >>> Thanks for driving this, Andrew Wang!!
> > > >>>
> > > >>> - downloaded the src tarball and verified md5 checksum
> > > >>> - built from source with jdk 1.8.0_111-b14
> > > >>> - brought up a pseudo distributed cluster
> > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > >>> confirmed that everything was working
> > > >>> - Run word count, pi and DFSIOTest
> > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > >>>
> > > >>> Cheers,
> > > >>> Ajay
> > > >>>
> > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > >>>
> > > >>>     +1 (binding)
> > > >>>
> > > >>>     - downloaded src tarball, verified md5
> > > >>>     - built from source with jdk1.8.0_112
> > > >>>     - started a pseudo cluster with hdfs and kms
> > > >>>     - sanity checked encryption related operations working
> > > >>>     - sanity checked webui and logs.
> > > >>>
> > > >>>     -Xiao
> > > >>>
> > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> atm@apache.org>
> > > >>> wrote:
> > > >>>
> > > >>>     > +1 (binding)
> > > >>>     >
> > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > -Pnative)
> > > >>>     > - verified the checksum
> > > >>>     > - brought up a secure pseudo distributed cluster
> > > >>>     > - did some basic file system operations (mkdir, list, put,
> cat)
> > > and
> > > >>>     > confirmed that everything was working
> > > >>>     > - confirmed that the web UI worked
> > > >>>     >
> > > >>>     > Best,
> > > >>>     > Aaron
> > > >>>     >
> > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > >>> andrew.wang@cloudera.com>
> > > >>>     > wrote:
> > > >>>     >
> > > >>>     > > Hi all,
> > > >>>     > >
> > > >>>     > > Let me start, as always, by thanking the efforts of all the
> > > >>> contributors
> > > >>>     > > who contributed to this release, especially those who
> jumped
> > on
> > > >>> the
> > > >>>     > issues
> > > >>>     > > found in RC0.
> > > >>>     > >
> > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > >>> incorporates 302
> > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > >>>     > >
> > > >>>     > > You can find the artifacts here:
> > > >>>     > >
> > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > >>>     > >
> > > >>>     > > I've done the traditional testing of building from the
> source
> > > >>> tarball and
> > > >>>     > > running a Pi job on a single node cluster. I also verified
> > that
> > > >>> the
> > > >>>     > shaded
> > > >>>     > > jars are not empty.
> > > >>>     > >
> > > >>>     > > Found one issue that create-release (probably due to the
> mvn
> > > >>> deploy
> > > >>>     > change)
> > > >>>     > > didn't sign the artifacts, but I fixed that by calling mvn
> > one
> > > >>> more time.
> > > >>>     > > Available here:
> > > >>>     > >
> > > >>>     > > https://repository.apache.org/
> content/repositories/orgapache
> > > >>> hadoop-1075/
> > > >>>     > >
> > > >>>     > > This release will run the standard 5 days, closing on Dec
> > 13th
> > > at
> > > >>> 12:31pm
> > > >>>     > > Pacific. My +1 to start.
> > > >>>     > >
> > > >>>     > > Best,
> > > >>>     > > Andrew
> > > >>>     > >
> > > >>>     >
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> ------------------------------------------------------------
> > ---------
> > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >>
> > >
> > >
> > >
> > > --
> > > Lei (Eddy) Xu
> > > Software Engineer, Cloudera
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Robert Kanter <rk...@cloudera.com>.

+1 (binding)

+ Downloaded the binary release
+ Deployed on a 3 node cluster on CentOS 7.3
+ Ran some MR jobs, clicked around the UI, etc
+ Ran some CLI commands (yarn logs, etc)

Good job everyone on Hadoop 3!


- Robert

On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:

> +1 (binding)
>
> - Verified signatures of the source tarball.
> - built from source - using the docker build environment.
> - set up a pseudo-distributed test cluster.
> - ran basic HDFS commands
> - ran some basic MR jobs
>
> Cheers
> -Arun
>
> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi everyone,
> >
> > As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
> > whack if you have time. There are already enough binding +1s to pass this
> > vote, but it'd be great to get additional validation.
> >
> > Thanks to everyone who's voted thus far!
> >
> > Best,
> > Andrew
> >
> >
> >
> > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> >
> > > +1 (binding)
> > >
> > > * Verified src tarball and bin tarball, verified md5 of each.
> > > * Build source with -Pdist,native
> > > * Started a pseudo cluster
> > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > >
> > > Thanks Andrew for this great effort!
> > >
> > > Best,
> > >
> > >
> > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <andrew.wang@cloudera.com
> >
> > > wrote:
> > > > Hi Wei-Chiu,
> > > >
> > > > The patchprocess directory is left over from the create-release
> > process,
> > > > and it looks empty to me. We should still file a create-release JIRA
> to
> > > fix
> > > > this, but I think this is not a blocker. Would you agree?
> > > >
> > > > Best,
> > > > Andrew
> > > >
> > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> weichiu@cloudera.com
> > >
> > > > wrote:
> > > >
> > > >> Hi Andrew, thanks the tremendous effort.
> > > >> I found an empty "patchprocess" directory in the source tarball,
> that
> > is
> > > >> not there if you clone from github. Any chance you might have some
> > > leftover
> > > >> trash when you made the tarball?
> > > >> Not wanting to nitpicking, but you might want to double check so we
> > > don't
> > > >> ship anything private to you in public :)
> > > >>
> > > >>
> > > >>
> > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > ajay.kumar@hortonworks.com
> > > >
> > > >> wrote:
> > > >>
> > > >>> +1 (non-binding)
> > > >>> Thanks for driving this, Andrew Wang!!
> > > >>>
> > > >>> - downloaded the src tarball and verified md5 checksum
> > > >>> - built from source with jdk 1.8.0_111-b14
> > > >>> - brought up a pseudo distributed cluster
> > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > >>> confirmed that everything was working
> > > >>> - Run word count, pi and DFSIOTest
> > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > >>>
> > > >>> Cheers,
> > > >>> Ajay
> > > >>>
> > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > >>>
> > > >>>     +1 (binding)
> > > >>>
> > > >>>     - downloaded src tarball, verified md5
> > > >>>     - built from source with jdk1.8.0_112
> > > >>>     - started a pseudo cluster with hdfs and kms
> > > >>>     - sanity checked encryption related operations working
> > > >>>     - sanity checked webui and logs.
> > > >>>
> > > >>>     -Xiao
> > > >>>
> > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> atm@apache.org>
> > > >>> wrote:
> > > >>>
> > > >>>     > +1 (binding)
> > > >>>     >
> > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > -Pnative)
> > > >>>     > - verified the checksum
> > > >>>     > - brought up a secure pseudo distributed cluster
> > > >>>     > - did some basic file system operations (mkdir, list, put,
> cat)
> > > and
> > > >>>     > confirmed that everything was working
> > > >>>     > - confirmed that the web UI worked
> > > >>>     >
> > > >>>     > Best,
> > > >>>     > Aaron
> > > >>>     >
> > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > >>> andrew.wang@cloudera.com>
> > > >>>     > wrote:
> > > >>>     >
> > > >>>     > > Hi all,
> > > >>>     > >
> > > >>>     > > Let me start, as always, by thanking the efforts of all the
> > > >>> contributors
> > > >>>     > > who contributed to this release, especially those who
> jumped
> > on
> > > >>> the
> > > >>>     > issues
> > > >>>     > > found in RC0.
> > > >>>     > >
> > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > >>> incorporates 302
> > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > >>>     > >
> > > >>>     > > You can find the artifacts here:
> > > >>>     > >
> > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > >>>     > >
> > > >>>     > > I've done the traditional testing of building from the
> source
> > > >>> tarball and
> > > >>>     > > running a Pi job on a single node cluster. I also verified
> > that
> > > >>> the
> > > >>>     > shaded
> > > >>>     > > jars are not empty.
> > > >>>     > >
> > > >>>     > > Found one issue that create-release (probably due to the
> mvn
> > > >>> deploy
> > > >>>     > change)
> > > >>>     > > didn't sign the artifacts, but I fixed that by calling mvn
> > one
> > > >>> more time.
> > > >>>     > > Available here:
> > > >>>     > >
> > > >>>     > > https://repository.apache.org/
> content/repositories/orgapache
> > > >>> hadoop-1075/
> > > >>>     > >
> > > >>>     > > This release will run the standard 5 days, closing on Dec
> > 13th
> > > at
> > > >>> 12:31pm
> > > >>>     > > Pacific. My +1 to start.
> > > >>>     > >
> > > >>>     > > Best,
> > > >>>     > > Andrew
> > > >>>     > >
> > > >>>     >
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> ------------------------------------------------------------
> > ---------
> > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >>
> > >
> > >
> > >
> > > --
> > > Lei (Eddy) Xu
> > > Software Engineer, Cloudera
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Robert Kanter <rk...@cloudera.com>.

+1 (binding)

+ Downloaded the binary release
+ Deployed on a 3 node cluster on CentOS 7.3
+ Ran some MR jobs, clicked around the UI, etc
+ Ran some CLI commands (yarn logs, etc)

Good job everyone on Hadoop 3!


- Robert

On Tue, Dec 12, 2017 at 1:56 PM, Arun Suresh <as...@apache.org> wrote:

> +1 (binding)
>
> - Verified signatures of the source tarball.
> - built from source - using the docker build environment.
> - set up a pseudo-distributed test cluster.
> - ran basic HDFS commands
> - ran some basic MR jobs
>
> Cheers
> -Arun
>
> On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi everyone,
> >
> > As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
> > whack if you have time. There are already enough binding +1s to pass this
> > vote, but it'd be great to get additional validation.
> >
> > Thanks to everyone who's voted thus far!
> >
> > Best,
> > Andrew
> >
> >
> >
> > On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
> >
> > > +1 (binding)
> > >
> > > * Verified src tarball and bin tarball, verified md5 of each.
> > > * Build source with -Pdist,native
> > > * Started a pseudo cluster
> > > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > > dfs put/get/cat on "/" with XOR-2-1 policy.
> > >
> > > Thanks Andrew for this great effort!
> > >
> > > Best,
> > >
> > >
> > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <andrew.wang@cloudera.com
> >
> > > wrote:
> > > > Hi Wei-Chiu,
> > > >
> > > > The patchprocess directory is left over from the create-release
> > process,
> > > > and it looks empty to me. We should still file a create-release JIRA
> to
> > > fix
> > > > this, but I think this is not a blocker. Would you agree?
> > > >
> > > > Best,
> > > > Andrew
> > > >
> > > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <
> weichiu@cloudera.com
> > >
> > > > wrote:
> > > >
> > > >> Hi Andrew, thanks the tremendous effort.
> > > >> I found an empty "patchprocess" directory in the source tarball,
> that
> > is
> > > >> not there if you clone from github. Any chance you might have some
> > > leftover
> > > >> trash when you made the tarball?
> > > >> Not wanting to nitpicking, but you might want to double check so we
> > > don't
> > > >> ship anything private to you in public :)
> > > >>
> > > >>
> > > >>
> > > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> > ajay.kumar@hortonworks.com
> > > >
> > > >> wrote:
> > > >>
> > > >>> +1 (non-binding)
> > > >>> Thanks for driving this, Andrew Wang!!
> > > >>>
> > > >>> - downloaded the src tarball and verified md5 checksum
> > > >>> - built from source with jdk 1.8.0_111-b14
> > > >>> - brought up a pseudo distributed cluster
> > > >>> - did basic file system operations (mkdir, list, put, cat) and
> > > >>> confirmed that everything was working
> > > >>> - Run word count, pi and DFSIOTest
> > > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > > >>>
> > > >>> Cheers,
> > > >>> Ajay
> > > >>>
> > > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > > >>>
> > > >>>     +1 (binding)
> > > >>>
> > > >>>     - downloaded src tarball, verified md5
> > > >>>     - built from source with jdk1.8.0_112
> > > >>>     - started a pseudo cluster with hdfs and kms
> > > >>>     - sanity checked encryption related operations working
> > > >>>     - sanity checked webui and logs.
> > > >>>
> > > >>>     -Xiao
> > > >>>
> > > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <
> atm@apache.org>
> > > >>> wrote:
> > > >>>
> > > >>>     > +1 (binding)
> > > >>>     >
> > > >>>     > - downloaded the src tarball and built the source (-Pdist
> > > -Pnative)
> > > >>>     > - verified the checksum
> > > >>>     > - brought up a secure pseudo distributed cluster
> > > >>>     > - did some basic file system operations (mkdir, list, put,
> cat)
> > > and
> > > >>>     > confirmed that everything was working
> > > >>>     > - confirmed that the web UI worked
> > > >>>     >
> > > >>>     > Best,
> > > >>>     > Aaron
> > > >>>     >
> > > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > > >>> andrew.wang@cloudera.com>
> > > >>>     > wrote:
> > > >>>     >
> > > >>>     > > Hi all,
> > > >>>     > >
> > > >>>     > > Let me start, as always, by thanking the efforts of all the
> > > >>> contributors
> > > >>>     > > who contributed to this release, especially those who
> jumped
> > on
> > > >>> the
> > > >>>     > issues
> > > >>>     > > found in RC0.
> > > >>>     > >
> > > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > > >>> incorporates 302
> > > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > > >>>     > >
> > > >>>     > > You can find the artifacts here:
> > > >>>     > >
> > > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > > >>>     > >
> > > >>>     > > I've done the traditional testing of building from the
> source
> > > >>> tarball and
> > > >>>     > > running a Pi job on a single node cluster. I also verified
> > that
> > > >>> the
> > > >>>     > shaded
> > > >>>     > > jars are not empty.
> > > >>>     > >
> > > >>>     > > Found one issue that create-release (probably due to the
> mvn
> > > >>> deploy
> > > >>>     > change)
> > > >>>     > > didn't sign the artifacts, but I fixed that by calling mvn
> > one
> > > >>> more time.
> > > >>>     > > Available here:
> > > >>>     > >
> > > >>>     > > https://repository.apache.org/
> content/repositories/orgapache
> > > >>> hadoop-1075/
> > > >>>     > >
> > > >>>     > > This release will run the standard 5 days, closing on Dec
> > 13th
> > > at
> > > >>> 12:31pm
> > > >>>     > > Pacific. My +1 to start.
> > > >>>     > >
> > > >>>     > > Best,
> > > >>>     > > Andrew
> > > >>>     > >
> > > >>>     >
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> ------------------------------------------------------------
> > ---------
> > > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >>
> > >
> > >
> > >
> > > --
> > > Lei (Eddy) Xu
> > > Software Engineer, Cloudera
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Arun Suresh <as...@apache.org>.

+1 (binding)

- Verified signatures of the source tarball.
- built from source - using the docker build environment.
- set up a pseudo-distributed test cluster.
- ran basic HDFS commands
- ran some basic MR jobs

Cheers
-Arun

On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi everyone,
>
> As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
> whack if you have time. There are already enough binding +1s to pass this
> vote, but it'd be great to get additional validation.
>
> Thanks to everyone who's voted thus far!
>
> Best,
> Andrew
>
>
>
> On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
>
> > +1 (binding)
> >
> > * Verified src tarball and bin tarball, verified md5 of each.
> > * Build source with -Pdist,native
> > * Started a pseudo cluster
> > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > dfs put/get/cat on "/" with XOR-2-1 policy.
> >
> > Thanks Andrew for this great effort!
> >
> > Best,
> >
> >
> > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
> > wrote:
> > > Hi Wei-Chiu,
> > >
> > > The patchprocess directory is left over from the create-release
> process,
> > > and it looks empty to me. We should still file a create-release JIRA to
> > fix
> > > this, but I think this is not a blocker. Would you agree?
> > >
> > > Best,
> > > Andrew
> > >
> > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <weichiu@cloudera.com
> >
> > > wrote:
> > >
> > >> Hi Andrew, thanks the tremendous effort.
> > >> I found an empty "patchprocess" directory in the source tarball, that
> is
> > >> not there if you clone from github. Any chance you might have some
> > leftover
> > >> trash when you made the tarball?
> > >> Not wanting to nitpicking, but you might want to double check so we
> > don't
> > >> ship anything private to you in public :)
> > >>
> > >>
> > >>
> > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> ajay.kumar@hortonworks.com
> > >
> > >> wrote:
> > >>
> > >>> +1 (non-binding)
> > >>> Thanks for driving this, Andrew Wang!!
> > >>>
> > >>> - downloaded the src tarball and verified md5 checksum
> > >>> - built from source with jdk 1.8.0_111-b14
> > >>> - brought up a pseudo distributed cluster
> > >>> - did basic file system operations (mkdir, list, put, cat) and
> > >>> confirmed that everything was working
> > >>> - Run word count, pi and DFSIOTest
> > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > >>>
> > >>> Cheers,
> > >>> Ajay
> > >>>
> > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > >>>
> > >>>     +1 (binding)
> > >>>
> > >>>     - downloaded src tarball, verified md5
> > >>>     - built from source with jdk1.8.0_112
> > >>>     - started a pseudo cluster with hdfs and kms
> > >>>     - sanity checked encryption related operations working
> > >>>     - sanity checked webui and logs.
> > >>>
> > >>>     -Xiao
> > >>>
> > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> > >>> wrote:
> > >>>
> > >>>     > +1 (binding)
> > >>>     >
> > >>>     > - downloaded the src tarball and built the source (-Pdist
> > -Pnative)
> > >>>     > - verified the checksum
> > >>>     > - brought up a secure pseudo distributed cluster
> > >>>     > - did some basic file system operations (mkdir, list, put, cat)
> > and
> > >>>     > confirmed that everything was working
> > >>>     > - confirmed that the web UI worked
> > >>>     >
> > >>>     > Best,
> > >>>     > Aaron
> > >>>     >
> > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > >>> andrew.wang@cloudera.com>
> > >>>     > wrote:
> > >>>     >
> > >>>     > > Hi all,
> > >>>     > >
> > >>>     > > Let me start, as always, by thanking the efforts of all the
> > >>> contributors
> > >>>     > > who contributed to this release, especially those who jumped
> on
> > >>> the
> > >>>     > issues
> > >>>     > > found in RC0.
> > >>>     > >
> > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > >>> incorporates 302
> > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > >>>     > >
> > >>>     > > You can find the artifacts here:
> > >>>     > >
> > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > >>>     > >
> > >>>     > > I've done the traditional testing of building from the source
> > >>> tarball and
> > >>>     > > running a Pi job on a single node cluster. I also verified
> that
> > >>> the
> > >>>     > shaded
> > >>>     > > jars are not empty.
> > >>>     > >
> > >>>     > > Found one issue that create-release (probably due to the mvn
> > >>> deploy
> > >>>     > change)
> > >>>     > > didn't sign the artifacts, but I fixed that by calling mvn
> one
> > >>> more time.
> > >>>     > > Available here:
> > >>>     > >
> > >>>     > > https://repository.apache.org/content/repositories/orgapache
> > >>> hadoop-1075/
> > >>>     > >
> > >>>     > > This release will run the standard 5 days, closing on Dec
> 13th
> > at
> > >>> 12:31pm
> > >>>     > > Pacific. My +1 to start.
> > >>>     > >
> > >>>     > > Best,
> > >>>     > > Andrew
> > >>>     > >
> > >>>     >
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> ------------------------------------------------------------
> ---------
> > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >>>
> > >>
> > >>
> > >>
> > >>
> >
> >
> >
> > --
> > Lei (Eddy) Xu
> > Software Engineer, Cloudera
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Arun Suresh <as...@apache.org>.

+1 (binding)

- Verified signatures of the source tarball.
- built from source - using the docker build environment.
- set up a pseudo-distributed test cluster.
- ran basic HDFS commands
- ran some basic MR jobs

Cheers
-Arun

On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi everyone,
>
> As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
> whack if you have time. There are already enough binding +1s to pass this
> vote, but it'd be great to get additional validation.
>
> Thanks to everyone who's voted thus far!
>
> Best,
> Andrew
>
>
>
> On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
>
> > +1 (binding)
> >
> > * Verified src tarball and bin tarball, verified md5 of each.
> > * Build source with -Pdist,native
> > * Started a pseudo cluster
> > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > dfs put/get/cat on "/" with XOR-2-1 policy.
> >
> > Thanks Andrew for this great effort!
> >
> > Best,
> >
> >
> > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
> > wrote:
> > > Hi Wei-Chiu,
> > >
> > > The patchprocess directory is left over from the create-release
> process,
> > > and it looks empty to me. We should still file a create-release JIRA to
> > fix
> > > this, but I think this is not a blocker. Would you agree?
> > >
> > > Best,
> > > Andrew
> > >
> > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <weichiu@cloudera.com
> >
> > > wrote:
> > >
> > >> Hi Andrew, thanks the tremendous effort.
> > >> I found an empty "patchprocess" directory in the source tarball, that
> is
> > >> not there if you clone from github. Any chance you might have some
> > leftover
> > >> trash when you made the tarball?
> > >> Not wanting to nitpicking, but you might want to double check so we
> > don't
> > >> ship anything private to you in public :)
> > >>
> > >>
> > >>
> > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> ajay.kumar@hortonworks.com
> > >
> > >> wrote:
> > >>
> > >>> +1 (non-binding)
> > >>> Thanks for driving this, Andrew Wang!!
> > >>>
> > >>> - downloaded the src tarball and verified md5 checksum
> > >>> - built from source with jdk 1.8.0_111-b14
> > >>> - brought up a pseudo distributed cluster
> > >>> - did basic file system operations (mkdir, list, put, cat) and
> > >>> confirmed that everything was working
> > >>> - Run word count, pi and DFSIOTest
> > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > >>>
> > >>> Cheers,
> > >>> Ajay
> > >>>
> > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > >>>
> > >>>     +1 (binding)
> > >>>
> > >>>     - downloaded src tarball, verified md5
> > >>>     - built from source with jdk1.8.0_112
> > >>>     - started a pseudo cluster with hdfs and kms
> > >>>     - sanity checked encryption related operations working
> > >>>     - sanity checked webui and logs.
> > >>>
> > >>>     -Xiao
> > >>>
> > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> > >>> wrote:
> > >>>
> > >>>     > +1 (binding)
> > >>>     >
> > >>>     > - downloaded the src tarball and built the source (-Pdist
> > -Pnative)
> > >>>     > - verified the checksum
> > >>>     > - brought up a secure pseudo distributed cluster
> > >>>     > - did some basic file system operations (mkdir, list, put, cat)
> > and
> > >>>     > confirmed that everything was working
> > >>>     > - confirmed that the web UI worked
> > >>>     >
> > >>>     > Best,
> > >>>     > Aaron
> > >>>     >
> > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > >>> andrew.wang@cloudera.com>
> > >>>     > wrote:
> > >>>     >
> > >>>     > > Hi all,
> > >>>     > >
> > >>>     > > Let me start, as always, by thanking the efforts of all the
> > >>> contributors
> > >>>     > > who contributed to this release, especially those who jumped
> on
> > >>> the
> > >>>     > issues
> > >>>     > > found in RC0.
> > >>>     > >
> > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > >>> incorporates 302
> > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > >>>     > >
> > >>>     > > You can find the artifacts here:
> > >>>     > >
> > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > >>>     > >
> > >>>     > > I've done the traditional testing of building from the source
> > >>> tarball and
> > >>>     > > running a Pi job on a single node cluster. I also verified
> that
> > >>> the
> > >>>     > shaded
> > >>>     > > jars are not empty.
> > >>>     > >
> > >>>     > > Found one issue that create-release (probably due to the mvn
> > >>> deploy
> > >>>     > change)
> > >>>     > > didn't sign the artifacts, but I fixed that by calling mvn
> one
> > >>> more time.
> > >>>     > > Available here:
> > >>>     > >
> > >>>     > > https://repository.apache.org/content/repositories/orgapache
> > >>> hadoop-1075/
> > >>>     > >
> > >>>     > > This release will run the standard 5 days, closing on Dec
> 13th
> > at
> > >>> 12:31pm
> > >>>     > > Pacific. My +1 to start.
> > >>>     > >
> > >>>     > > Best,
> > >>>     > > Andrew
> > >>>     > >
> > >>>     >
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> ------------------------------------------------------------
> ---------
> > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >>>
> > >>
> > >>
> > >>
> > >>
> >
> >
> >
> > --
> > Lei (Eddy) Xu
> > Software Engineer, Cloudera
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Arun Suresh <as...@apache.org>.

+1 (binding)

- Verified signatures of the source tarball.
- built from source - using the docker build environment.
- set up a pseudo-distributed test cluster.
- ran basic HDFS commands
- ran some basic MR jobs

Cheers
-Arun

On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi everyone,
>
> As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
> whack if you have time. There are already enough binding +1s to pass this
> vote, but it'd be great to get additional validation.
>
> Thanks to everyone who's voted thus far!
>
> Best,
> Andrew
>
>
>
> On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
>
> > +1 (binding)
> >
> > * Verified src tarball and bin tarball, verified md5 of each.
> > * Build source with -Pdist,native
> > * Started a pseudo cluster
> > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > dfs put/get/cat on "/" with XOR-2-1 policy.
> >
> > Thanks Andrew for this great effort!
> >
> > Best,
> >
> >
> > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
> > wrote:
> > > Hi Wei-Chiu,
> > >
> > > The patchprocess directory is left over from the create-release
> process,
> > > and it looks empty to me. We should still file a create-release JIRA to
> > fix
> > > this, but I think this is not a blocker. Would you agree?
> > >
> > > Best,
> > > Andrew
> > >
> > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <weichiu@cloudera.com
> >
> > > wrote:
> > >
> > >> Hi Andrew, thanks the tremendous effort.
> > >> I found an empty "patchprocess" directory in the source tarball, that
> is
> > >> not there if you clone from github. Any chance you might have some
> > leftover
> > >> trash when you made the tarball?
> > >> Not wanting to nitpicking, but you might want to double check so we
> > don't
> > >> ship anything private to you in public :)
> > >>
> > >>
> > >>
> > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> ajay.kumar@hortonworks.com
> > >
> > >> wrote:
> > >>
> > >>> +1 (non-binding)
> > >>> Thanks for driving this, Andrew Wang!!
> > >>>
> > >>> - downloaded the src tarball and verified md5 checksum
> > >>> - built from source with jdk 1.8.0_111-b14
> > >>> - brought up a pseudo distributed cluster
> > >>> - did basic file system operations (mkdir, list, put, cat) and
> > >>> confirmed that everything was working
> > >>> - Run word count, pi and DFSIOTest
> > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > >>>
> > >>> Cheers,
> > >>> Ajay
> > >>>
> > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > >>>
> > >>>     +1 (binding)
> > >>>
> > >>>     - downloaded src tarball, verified md5
> > >>>     - built from source with jdk1.8.0_112
> > >>>     - started a pseudo cluster with hdfs and kms
> > >>>     - sanity checked encryption related operations working
> > >>>     - sanity checked webui and logs.
> > >>>
> > >>>     -Xiao
> > >>>
> > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> > >>> wrote:
> > >>>
> > >>>     > +1 (binding)
> > >>>     >
> > >>>     > - downloaded the src tarball and built the source (-Pdist
> > -Pnative)
> > >>>     > - verified the checksum
> > >>>     > - brought up a secure pseudo distributed cluster
> > >>>     > - did some basic file system operations (mkdir, list, put, cat)
> > and
> > >>>     > confirmed that everything was working
> > >>>     > - confirmed that the web UI worked
> > >>>     >
> > >>>     > Best,
> > >>>     > Aaron
> > >>>     >
> > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > >>> andrew.wang@cloudera.com>
> > >>>     > wrote:
> > >>>     >
> > >>>     > > Hi all,
> > >>>     > >
> > >>>     > > Let me start, as always, by thanking the efforts of all the
> > >>> contributors
> > >>>     > > who contributed to this release, especially those who jumped
> on
> > >>> the
> > >>>     > issues
> > >>>     > > found in RC0.
> > >>>     > >
> > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > >>> incorporates 302
> > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > >>>     > >
> > >>>     > > You can find the artifacts here:
> > >>>     > >
> > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > >>>     > >
> > >>>     > > I've done the traditional testing of building from the source
> > >>> tarball and
> > >>>     > > running a Pi job on a single node cluster. I also verified
> that
> > >>> the
> > >>>     > shaded
> > >>>     > > jars are not empty.
> > >>>     > >
> > >>>     > > Found one issue that create-release (probably due to the mvn
> > >>> deploy
> > >>>     > change)
> > >>>     > > didn't sign the artifacts, but I fixed that by calling mvn
> one
> > >>> more time.
> > >>>     > > Available here:
> > >>>     > >
> > >>>     > > https://repository.apache.org/content/repositories/orgapache
> > >>> hadoop-1075/
> > >>>     > >
> > >>>     > > This release will run the standard 5 days, closing on Dec
> 13th
> > at
> > >>> 12:31pm
> > >>>     > > Pacific. My +1 to start.
> > >>>     > >
> > >>>     > > Best,
> > >>>     > > Andrew
> > >>>     > >
> > >>>     >
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> ------------------------------------------------------------
> ---------
> > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >>>
> > >>
> > >>
> > >>
> > >>
> >
> >
> >
> > --
> > Lei (Eddy) Xu
> > Software Engineer, Cloudera
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Arun Suresh <as...@apache.org>.

+1 (binding)

- Verified signatures of the source tarball.
- built from source - using the docker build environment.
- set up a pseudo-distributed test cluster.
- ran basic HDFS commands
- ran some basic MR jobs

Cheers
-Arun

On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi everyone,
>
> As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
> whack if you have time. There are already enough binding +1s to pass this
> vote, but it'd be great to get additional validation.
>
> Thanks to everyone who's voted thus far!
>
> Best,
> Andrew
>
>
>
> On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:
>
> > +1 (binding)
> >
> > * Verified src tarball and bin tarball, verified md5 of each.
> > * Build source with -Pdist,native
> > * Started a pseudo cluster
> > * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> > dfs put/get/cat on "/" with XOR-2-1 policy.
> >
> > Thanks Andrew for this great effort!
> >
> > Best,
> >
> >
> > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
> > wrote:
> > > Hi Wei-Chiu,
> > >
> > > The patchprocess directory is left over from the create-release
> process,
> > > and it looks empty to me. We should still file a create-release JIRA to
> > fix
> > > this, but I think this is not a blocker. Would you agree?
> > >
> > > Best,
> > > Andrew
> > >
> > > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <weichiu@cloudera.com
> >
> > > wrote:
> > >
> > >> Hi Andrew, thanks the tremendous effort.
> > >> I found an empty "patchprocess" directory in the source tarball, that
> is
> > >> not there if you clone from github. Any chance you might have some
> > leftover
> > >> trash when you made the tarball?
> > >> Not wanting to nitpicking, but you might want to double check so we
> > don't
> > >> ship anything private to you in public :)
> > >>
> > >>
> > >>
> > >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <
> ajay.kumar@hortonworks.com
> > >
> > >> wrote:
> > >>
> > >>> +1 (non-binding)
> > >>> Thanks for driving this, Andrew Wang!!
> > >>>
> > >>> - downloaded the src tarball and verified md5 checksum
> > >>> - built from source with jdk 1.8.0_111-b14
> > >>> - brought up a pseudo distributed cluster
> > >>> - did basic file system operations (mkdir, list, put, cat) and
> > >>> confirmed that everything was working
> > >>> - Run word count, pi and DFSIOTest
> > >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> > >>>
> > >>> Cheers,
> > >>> Ajay
> > >>>
> > >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> > >>>
> > >>>     +1 (binding)
> > >>>
> > >>>     - downloaded src tarball, verified md5
> > >>>     - built from source with jdk1.8.0_112
> > >>>     - started a pseudo cluster with hdfs and kms
> > >>>     - sanity checked encryption related operations working
> > >>>     - sanity checked webui and logs.
> > >>>
> > >>>     -Xiao
> > >>>
> > >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> > >>> wrote:
> > >>>
> > >>>     > +1 (binding)
> > >>>     >
> > >>>     > - downloaded the src tarball and built the source (-Pdist
> > -Pnative)
> > >>>     > - verified the checksum
> > >>>     > - brought up a secure pseudo distributed cluster
> > >>>     > - did some basic file system operations (mkdir, list, put, cat)
> > and
> > >>>     > confirmed that everything was working
> > >>>     > - confirmed that the web UI worked
> > >>>     >
> > >>>     > Best,
> > >>>     > Aaron
> > >>>     >
> > >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> > >>> andrew.wang@cloudera.com>
> > >>>     > wrote:
> > >>>     >
> > >>>     > > Hi all,
> > >>>     > >
> > >>>     > > Let me start, as always, by thanking the efforts of all the
> > >>> contributors
> > >>>     > > who contributed to this release, especially those who jumped
> on
> > >>> the
> > >>>     > issues
> > >>>     > > found in RC0.
> > >>>     > >
> > >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> > >>> incorporates 302
> > >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> > >>>     > >
> > >>>     > > You can find the artifacts here:
> > >>>     > >
> > >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> > >>>     > >
> > >>>     > > I've done the traditional testing of building from the source
> > >>> tarball and
> > >>>     > > running a Pi job on a single node cluster. I also verified
> that
> > >>> the
> > >>>     > shaded
> > >>>     > > jars are not empty.
> > >>>     > >
> > >>>     > > Found one issue that create-release (probably due to the mvn
> > >>> deploy
> > >>>     > change)
> > >>>     > > didn't sign the artifacts, but I fixed that by calling mvn
> one
> > >>> more time.
> > >>>     > > Available here:
> > >>>     > >
> > >>>     > > https://repository.apache.org/content/repositories/orgapache
> > >>> hadoop-1075/
> > >>>     > >
> > >>>     > > This release will run the standard 5 days, closing on Dec
> 13th
> > at
> > >>> 12:31pm
> > >>>     > > Pacific. My +1 to start.
> > >>>     > >
> > >>>     > > Best,
> > >>>     > > Andrew
> > >>>     > >
> > >>>     >
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> ------------------------------------------------------------
> ---------
> > >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >>>
> > >>
> > >>
> > >>
> > >>
> >
> >
> >
> > --
> > Lei (Eddy) Xu
> > Software Engineer, Cloudera
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Hi everyone,

As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
whack if you have time. There are already enough binding +1s to pass this
vote, but it'd be great to get additional validation.

Thanks to everyone who's voted thus far!

Best,
Andrew



On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:

> +1 (binding)
>
> * Verified src tarball and bin tarball, verified md5 of each.
> * Build source with -Pdist,native
> * Started a pseudo cluster
> * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> dfs put/get/cat on "/" with XOR-2-1 policy.
>
> Thanks Andrew for this great effort!
>
> Best,
>
>
> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
> wrote:
> > Hi Wei-Chiu,
> >
> > The patchprocess directory is left over from the create-release process,
> > and it looks empty to me. We should still file a create-release JIRA to
> fix
> > this, but I think this is not a blocker. Would you agree?
> >
> > Best,
> > Andrew
> >
> > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> > wrote:
> >
> >> Hi Andrew, thanks the tremendous effort.
> >> I found an empty "patchprocess" directory in the source tarball, that is
> >> not there if you clone from github. Any chance you might have some
> leftover
> >> trash when you made the tarball?
> >> Not wanting to nitpicking, but you might want to double check so we
> don't
> >> ship anything private to you in public :)
> >>
> >>
> >>
> >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <ajay.kumar@hortonworks.com
> >
> >> wrote:
> >>
> >>> +1 (non-binding)
> >>> Thanks for driving this, Andrew Wang!!
> >>>
> >>> - downloaded the src tarball and verified md5 checksum
> >>> - built from source with jdk 1.8.0_111-b14
> >>> - brought up a pseudo distributed cluster
> >>> - did basic file system operations (mkdir, list, put, cat) and
> >>> confirmed that everything was working
> >>> - Run word count, pi and DFSIOTest
> >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> >>>
> >>> Cheers,
> >>> Ajay
> >>>
> >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> >>>
> >>>     +1 (binding)
> >>>
> >>>     - downloaded src tarball, verified md5
> >>>     - built from source with jdk1.8.0_112
> >>>     - started a pseudo cluster with hdfs and kms
> >>>     - sanity checked encryption related operations working
> >>>     - sanity checked webui and logs.
> >>>
> >>>     -Xiao
> >>>
> >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> >>> wrote:
> >>>
> >>>     > +1 (binding)
> >>>     >
> >>>     > - downloaded the src tarball and built the source (-Pdist
> -Pnative)
> >>>     > - verified the checksum
> >>>     > - brought up a secure pseudo distributed cluster
> >>>     > - did some basic file system operations (mkdir, list, put, cat)
> and
> >>>     > confirmed that everything was working
> >>>     > - confirmed that the web UI worked
> >>>     >
> >>>     > Best,
> >>>     > Aaron
> >>>     >
> >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> >>> andrew.wang@cloudera.com>
> >>>     > wrote:
> >>>     >
> >>>     > > Hi all,
> >>>     > >
> >>>     > > Let me start, as always, by thanking the efforts of all the
> >>> contributors
> >>>     > > who contributed to this release, especially those who jumped on
> >>> the
> >>>     > issues
> >>>     > > found in RC0.
> >>>     > >
> >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> >>> incorporates 302
> >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> >>>     > >
> >>>     > > You can find the artifacts here:
> >>>     > >
> >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> >>>     > >
> >>>     > > I've done the traditional testing of building from the source
> >>> tarball and
> >>>     > > running a Pi job on a single node cluster. I also verified that
> >>> the
> >>>     > shaded
> >>>     > > jars are not empty.
> >>>     > >
> >>>     > > Found one issue that create-release (probably due to the mvn
> >>> deploy
> >>>     > change)
> >>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
> >>> more time.
> >>>     > > Available here:
> >>>     > >
> >>>     > > https://repository.apache.org/content/repositories/orgapache
> >>> hadoop-1075/
> >>>     > >
> >>>     > > This release will run the standard 5 days, closing on Dec 13th
> at
> >>> 12:31pm
> >>>     > > Pacific. My +1 to start.
> >>>     > >
> >>>     > > Best,
> >>>     > > Andrew
> >>>     > >
> >>>     >
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>>
> >>
> >>
> >>
> >>
>
>
>
> --
> Lei (Eddy) Xu
> Software Engineer, Cloudera
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Hi everyone,

As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
whack if you have time. There are already enough binding +1s to pass this
vote, but it'd be great to get additional validation.

Thanks to everyone who's voted thus far!

Best,
Andrew



On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:

> +1 (binding)
>
> * Verified src tarball and bin tarball, verified md5 of each.
> * Build source with -Pdist,native
> * Started a pseudo cluster
> * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> dfs put/get/cat on "/" with XOR-2-1 policy.
>
> Thanks Andrew for this great effort!
>
> Best,
>
>
> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
> wrote:
> > Hi Wei-Chiu,
> >
> > The patchprocess directory is left over from the create-release process,
> > and it looks empty to me. We should still file a create-release JIRA to
> fix
> > this, but I think this is not a blocker. Would you agree?
> >
> > Best,
> > Andrew
> >
> > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> > wrote:
> >
> >> Hi Andrew, thanks the tremendous effort.
> >> I found an empty "patchprocess" directory in the source tarball, that is
> >> not there if you clone from github. Any chance you might have some
> leftover
> >> trash when you made the tarball?
> >> Not wanting to nitpicking, but you might want to double check so we
> don't
> >> ship anything private to you in public :)
> >>
> >>
> >>
> >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <ajay.kumar@hortonworks.com
> >
> >> wrote:
> >>
> >>> +1 (non-binding)
> >>> Thanks for driving this, Andrew Wang!!
> >>>
> >>> - downloaded the src tarball and verified md5 checksum
> >>> - built from source with jdk 1.8.0_111-b14
> >>> - brought up a pseudo distributed cluster
> >>> - did basic file system operations (mkdir, list, put, cat) and
> >>> confirmed that everything was working
> >>> - Run word count, pi and DFSIOTest
> >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> >>>
> >>> Cheers,
> >>> Ajay
> >>>
> >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> >>>
> >>>     +1 (binding)
> >>>
> >>>     - downloaded src tarball, verified md5
> >>>     - built from source with jdk1.8.0_112
> >>>     - started a pseudo cluster with hdfs and kms
> >>>     - sanity checked encryption related operations working
> >>>     - sanity checked webui and logs.
> >>>
> >>>     -Xiao
> >>>
> >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> >>> wrote:
> >>>
> >>>     > +1 (binding)
> >>>     >
> >>>     > - downloaded the src tarball and built the source (-Pdist
> -Pnative)
> >>>     > - verified the checksum
> >>>     > - brought up a secure pseudo distributed cluster
> >>>     > - did some basic file system operations (mkdir, list, put, cat)
> and
> >>>     > confirmed that everything was working
> >>>     > - confirmed that the web UI worked
> >>>     >
> >>>     > Best,
> >>>     > Aaron
> >>>     >
> >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> >>> andrew.wang@cloudera.com>
> >>>     > wrote:
> >>>     >
> >>>     > > Hi all,
> >>>     > >
> >>>     > > Let me start, as always, by thanking the efforts of all the
> >>> contributors
> >>>     > > who contributed to this release, especially those who jumped on
> >>> the
> >>>     > issues
> >>>     > > found in RC0.
> >>>     > >
> >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> >>> incorporates 302
> >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> >>>     > >
> >>>     > > You can find the artifacts here:
> >>>     > >
> >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> >>>     > >
> >>>     > > I've done the traditional testing of building from the source
> >>> tarball and
> >>>     > > running a Pi job on a single node cluster. I also verified that
> >>> the
> >>>     > shaded
> >>>     > > jars are not empty.
> >>>     > >
> >>>     > > Found one issue that create-release (probably due to the mvn
> >>> deploy
> >>>     > change)
> >>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
> >>> more time.
> >>>     > > Available here:
> >>>     > >
> >>>     > > https://repository.apache.org/content/repositories/orgapache
> >>> hadoop-1075/
> >>>     > >
> >>>     > > This release will run the standard 5 days, closing on Dec 13th
> at
> >>> 12:31pm
> >>>     > > Pacific. My +1 to start.
> >>>     > >
> >>>     > > Best,
> >>>     > > Andrew
> >>>     > >
> >>>     >
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>>
> >>
> >>
> >>
> >>
>
>
>
> --
> Lei (Eddy) Xu
> Software Engineer, Cloudera
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Hi everyone,

As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
whack if you have time. There are already enough binding +1s to pass this
vote, but it'd be great to get additional validation.

Thanks to everyone who's voted thus far!

Best,
Andrew



On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:

> +1 (binding)
>
> * Verified src tarball and bin tarball, verified md5 of each.
> * Build source with -Pdist,native
> * Started a pseudo cluster
> * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> dfs put/get/cat on "/" with XOR-2-1 policy.
>
> Thanks Andrew for this great effort!
>
> Best,
>
>
> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
> wrote:
> > Hi Wei-Chiu,
> >
> > The patchprocess directory is left over from the create-release process,
> > and it looks empty to me. We should still file a create-release JIRA to
> fix
> > this, but I think this is not a blocker. Would you agree?
> >
> > Best,
> > Andrew
> >
> > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> > wrote:
> >
> >> Hi Andrew, thanks the tremendous effort.
> >> I found an empty "patchprocess" directory in the source tarball, that is
> >> not there if you clone from github. Any chance you might have some
> leftover
> >> trash when you made the tarball?
> >> Not wanting to nitpicking, but you might want to double check so we
> don't
> >> ship anything private to you in public :)
> >>
> >>
> >>
> >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <ajay.kumar@hortonworks.com
> >
> >> wrote:
> >>
> >>> +1 (non-binding)
> >>> Thanks for driving this, Andrew Wang!!
> >>>
> >>> - downloaded the src tarball and verified md5 checksum
> >>> - built from source with jdk 1.8.0_111-b14
> >>> - brought up a pseudo distributed cluster
> >>> - did basic file system operations (mkdir, list, put, cat) and
> >>> confirmed that everything was working
> >>> - Run word count, pi and DFSIOTest
> >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> >>>
> >>> Cheers,
> >>> Ajay
> >>>
> >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> >>>
> >>>     +1 (binding)
> >>>
> >>>     - downloaded src tarball, verified md5
> >>>     - built from source with jdk1.8.0_112
> >>>     - started a pseudo cluster with hdfs and kms
> >>>     - sanity checked encryption related operations working
> >>>     - sanity checked webui and logs.
> >>>
> >>>     -Xiao
> >>>
> >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> >>> wrote:
> >>>
> >>>     > +1 (binding)
> >>>     >
> >>>     > - downloaded the src tarball and built the source (-Pdist
> -Pnative)
> >>>     > - verified the checksum
> >>>     > - brought up a secure pseudo distributed cluster
> >>>     > - did some basic file system operations (mkdir, list, put, cat)
> and
> >>>     > confirmed that everything was working
> >>>     > - confirmed that the web UI worked
> >>>     >
> >>>     > Best,
> >>>     > Aaron
> >>>     >
> >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> >>> andrew.wang@cloudera.com>
> >>>     > wrote:
> >>>     >
> >>>     > > Hi all,
> >>>     > >
> >>>     > > Let me start, as always, by thanking the efforts of all the
> >>> contributors
> >>>     > > who contributed to this release, especially those who jumped on
> >>> the
> >>>     > issues
> >>>     > > found in RC0.
> >>>     > >
> >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> >>> incorporates 302
> >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> >>>     > >
> >>>     > > You can find the artifacts here:
> >>>     > >
> >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> >>>     > >
> >>>     > > I've done the traditional testing of building from the source
> >>> tarball and
> >>>     > > running a Pi job on a single node cluster. I also verified that
> >>> the
> >>>     > shaded
> >>>     > > jars are not empty.
> >>>     > >
> >>>     > > Found one issue that create-release (probably due to the mvn
> >>> deploy
> >>>     > change)
> >>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
> >>> more time.
> >>>     > > Available here:
> >>>     > >
> >>>     > > https://repository.apache.org/content/repositories/orgapache
> >>> hadoop-1075/
> >>>     > >
> >>>     > > This release will run the standard 5 days, closing on Dec 13th
> at
> >>> 12:31pm
> >>>     > > Pacific. My +1 to start.
> >>>     > >
> >>>     > > Best,
> >>>     > > Andrew
> >>>     > >
> >>>     >
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>>
> >>
> >>
> >>
> >>
>
>
>
> --
> Lei (Eddy) Xu
> Software Engineer, Cloudera
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Hi everyone,

As a reminder, this vote closes tomorrow at 12:31pm, so please give it a
whack if you have time. There are already enough binding +1s to pass this
vote, but it'd be great to get additional validation.

Thanks to everyone who's voted thus far!

Best,
Andrew



On Tue, Dec 12, 2017 at 11:08 AM, Lei Xu <le...@cloudera.com> wrote:

> +1 (binding)
>
> * Verified src tarball and bin tarball, verified md5 of each.
> * Build source with -Pdist,native
> * Started a pseudo cluster
> * Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
> dfs put/get/cat on "/" with XOR-2-1 policy.
>
> Thanks Andrew for this great effort!
>
> Best,
>
>
> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
> wrote:
> > Hi Wei-Chiu,
> >
> > The patchprocess directory is left over from the create-release process,
> > and it looks empty to me. We should still file a create-release JIRA to
> fix
> > this, but I think this is not a blocker. Would you agree?
> >
> > Best,
> > Andrew
> >
> > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> > wrote:
> >
> >> Hi Andrew, thanks the tremendous effort.
> >> I found an empty "patchprocess" directory in the source tarball, that is
> >> not there if you clone from github. Any chance you might have some
> leftover
> >> trash when you made the tarball?
> >> Not wanting to nitpicking, but you might want to double check so we
> don't
> >> ship anything private to you in public :)
> >>
> >>
> >>
> >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <ajay.kumar@hortonworks.com
> >
> >> wrote:
> >>
> >>> +1 (non-binding)
> >>> Thanks for driving this, Andrew Wang!!
> >>>
> >>> - downloaded the src tarball and verified md5 checksum
> >>> - built from source with jdk 1.8.0_111-b14
> >>> - brought up a pseudo distributed cluster
> >>> - did basic file system operations (mkdir, list, put, cat) and
> >>> confirmed that everything was working
> >>> - Run word count, pi and DFSIOTest
> >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> >>>
> >>> Cheers,
> >>> Ajay
> >>>
> >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> >>>
> >>>     +1 (binding)
> >>>
> >>>     - downloaded src tarball, verified md5
> >>>     - built from source with jdk1.8.0_112
> >>>     - started a pseudo cluster with hdfs and kms
> >>>     - sanity checked encryption related operations working
> >>>     - sanity checked webui and logs.
> >>>
> >>>     -Xiao
> >>>
> >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> >>> wrote:
> >>>
> >>>     > +1 (binding)
> >>>     >
> >>>     > - downloaded the src tarball and built the source (-Pdist
> -Pnative)
> >>>     > - verified the checksum
> >>>     > - brought up a secure pseudo distributed cluster
> >>>     > - did some basic file system operations (mkdir, list, put, cat)
> and
> >>>     > confirmed that everything was working
> >>>     > - confirmed that the web UI worked
> >>>     >
> >>>     > Best,
> >>>     > Aaron
> >>>     >
> >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> >>> andrew.wang@cloudera.com>
> >>>     > wrote:
> >>>     >
> >>>     > > Hi all,
> >>>     > >
> >>>     > > Let me start, as always, by thanking the efforts of all the
> >>> contributors
> >>>     > > who contributed to this release, especially those who jumped on
> >>> the
> >>>     > issues
> >>>     > > found in RC0.
> >>>     > >
> >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> >>> incorporates 302
> >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> >>>     > >
> >>>     > > You can find the artifacts here:
> >>>     > >
> >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> >>>     > >
> >>>     > > I've done the traditional testing of building from the source
> >>> tarball and
> >>>     > > running a Pi job on a single node cluster. I also verified that
> >>> the
> >>>     > shaded
> >>>     > > jars are not empty.
> >>>     > >
> >>>     > > Found one issue that create-release (probably due to the mvn
> >>> deploy
> >>>     > change)
> >>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
> >>> more time.
> >>>     > > Available here:
> >>>     > >
> >>>     > > https://repository.apache.org/content/repositories/orgapache
> >>> hadoop-1075/
> >>>     > >
> >>>     > > This release will run the standard 5 days, closing on Dec 13th
> at
> >>> 12:31pm
> >>>     > > Pacific. My +1 to start.
> >>>     > >
> >>>     > > Best,
> >>>     > > Andrew
> >>>     > >
> >>>     >
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>>
> >>
> >>
> >>
> >>
>
>
>
> --
> Lei (Eddy) Xu
> Software Engineer, Cloudera
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Lei Xu <le...@cloudera.com>.

+1 (binding)

* Verified src tarball and bin tarball, verified md5 of each.
* Build source with -Pdist,native
* Started a pseudo cluster
* Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
dfs put/get/cat on "/" with XOR-2-1 policy.

Thanks Andrew for this great effort!

Best,


On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com> wrote:
> Hi Wei-Chiu,
>
> The patchprocess directory is left over from the create-release process,
> and it looks empty to me. We should still file a create-release JIRA to fix
> this, but I think this is not a blocker. Would you agree?
>
> Best,
> Andrew
>
> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> wrote:
>
>> Hi Andrew, thanks the tremendous effort.
>> I found an empty "patchprocess" directory in the source tarball, that is
>> not there if you clone from github. Any chance you might have some leftover
>> trash when you made the tarball?
>> Not wanting to nitpicking, but you might want to double check so we don't
>> ship anything private to you in public :)
>>
>>
>>
>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
>> wrote:
>>
>>> +1 (non-binding)
>>> Thanks for driving this, Andrew Wang!!
>>>
>>> - downloaded the src tarball and verified md5 checksum
>>> - built from source with jdk 1.8.0_111-b14
>>> - brought up a pseudo distributed cluster
>>> - did basic file system operations (mkdir, list, put, cat) and
>>> confirmed that everything was working
>>> - Run word count, pi and DFSIOTest
>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>>
>>> Cheers,
>>> Ajay
>>>
>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>>
>>>     +1 (binding)
>>>
>>>     - downloaded src tarball, verified md5
>>>     - built from source with jdk1.8.0_112
>>>     - started a pseudo cluster with hdfs and kms
>>>     - sanity checked encryption related operations working
>>>     - sanity checked webui and logs.
>>>
>>>     -Xiao
>>>
>>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
>>> wrote:
>>>
>>>     > +1 (binding)
>>>     >
>>>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>>>     > - verified the checksum
>>>     > - brought up a secure pseudo distributed cluster
>>>     > - did some basic file system operations (mkdir, list, put, cat) and
>>>     > confirmed that everything was working
>>>     > - confirmed that the web UI worked
>>>     >
>>>     > Best,
>>>     > Aaron
>>>     >
>>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>>> andrew.wang@cloudera.com>
>>>     > wrote:
>>>     >
>>>     > > Hi all,
>>>     > >
>>>     > > Let me start, as always, by thanking the efforts of all the
>>> contributors
>>>     > > who contributed to this release, especially those who jumped on
>>> the
>>>     > issues
>>>     > > found in RC0.
>>>     > >
>>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
>>> incorporates 302
>>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>>>     > >
>>>     > > You can find the artifacts here:
>>>     > >
>>>     > > http://home.apache.org/~wang/3.0.0-RC1/
>>>     > >
>>>     > > I've done the traditional testing of building from the source
>>> tarball and
>>>     > > running a Pi job on a single node cluster. I also verified that
>>> the
>>>     > shaded
>>>     > > jars are not empty.
>>>     > >
>>>     > > Found one issue that create-release (probably due to the mvn
>>> deploy
>>>     > change)
>>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
>>> more time.
>>>     > > Available here:
>>>     > >
>>>     > > https://repository.apache.org/content/repositories/orgapache
>>> hadoop-1075/
>>>     > >
>>>     > > This release will run the standard 5 days, closing on Dec 13th at
>>> 12:31pm
>>>     > > Pacific. My +1 to start.
>>>     > >
>>>     > > Best,
>>>     > > Andrew
>>>     > >
>>>     >
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>
>>
>>
>>
>>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Subramaniam V K <su...@gmail.com>.

Thanks to everyone who pushed on this release, really great to see a
release coming off trunk.

+1 (binding).

Deployed the RC on a federated YARN cluster consisting of 8 sub-clusters on
CentOs 7.4 running Java 1.8.0_144.

Hit YARN-7652[1] which is not a blocker as I was able to run multiple jobs
successfully once I cleaned up the StateStore.

[1] https://issues.apache.org/jira/browse/YARN-7652

On Wed, Dec 13, 2017 at 11:25 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:

> Thanks Andrew again,
>
> +1 (binding)
>
> * Downloaded source tarball, and compiled with native libs
> successfully: -Pdist,native -Drequire.isal -Drequire.snappy
> -Drequire.openssl -Drequire.zstd.
> * hadoop checknative ran successfully.
> * Upgraded a fsimage that was previously used in a Hadoop 2.6-based CDH
> production cluster (fsimage size: 7.3 GB) to Hadoop 3.0.0 RC1 successfully.
> (Command used: hdfs namenode -upgrade)
> Interestingly, after the upgrade, the fsimage grew from 7.3 GB to 7.5 GB.
> I'm not really sure what went into fsimage between 2.6 and 3.0, but this is
> definitely not a big problem.
>
> In addition,
> * Built HBase 2.0.0 alpha4 against Hadoop 3.0.0 RC1 successfully.
> * Built Hive master branch against Hadoop 3.0.0 RC1 successfully.
>
>
> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi Wei-Chiu,
> >
> > The patchprocess directory is left over from the create-release process,
> > and it looks empty to me. We should still file a create-release JIRA to
> fix
> > this, but I think this is not a blocker. Would you agree?
> >
> > Best,
> > Andrew
> >
> > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> > wrote:
> >
> >> Hi Andrew, thanks the tremendous effort.
> >> I found an empty "patchprocess" directory in the source tarball, that is
> >> not there if you clone from github. Any chance you might have some
> leftover
> >> trash when you made the tarball?
> >> Not wanting to nitpicking, but you might want to double check so we
> don't
> >> ship anything private to you in public :)
> >>
> >>
> >>
> >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <ajay.kumar@hortonworks.com
> >
> >> wrote:
> >>
> >>> +1 (non-binding)
> >>> Thanks for driving this, Andrew Wang!!
> >>>
> >>> - downloaded the src tarball and verified md5 checksum
> >>> - built from source with jdk 1.8.0_111-b14
> >>> - brought up a pseudo distributed cluster
> >>> - did basic file system operations (mkdir, list, put, cat) and
> >>> confirmed that everything was working
> >>> - Run word count, pi and DFSIOTest
> >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> >>>
> >>> Cheers,
> >>> Ajay
> >>>
> >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> >>>
> >>>     +1 (binding)
> >>>
> >>>     - downloaded src tarball, verified md5
> >>>     - built from source with jdk1.8.0_112
> >>>     - started a pseudo cluster with hdfs and kms
> >>>     - sanity checked encryption related operations working
> >>>     - sanity checked webui and logs.
> >>>
> >>>     -Xiao
> >>>
> >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> >>> wrote:
> >>>
> >>>     > +1 (binding)
> >>>     >
> >>>     > - downloaded the src tarball and built the source (-Pdist
> -Pnative)
> >>>     > - verified the checksum
> >>>     > - brought up a secure pseudo distributed cluster
> >>>     > - did some basic file system operations (mkdir, list, put, cat)
> and
> >>>     > confirmed that everything was working
> >>>     > - confirmed that the web UI worked
> >>>     >
> >>>     > Best,
> >>>     > Aaron
> >>>     >
> >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> >>> andrew.wang@cloudera.com>
> >>>     > wrote:
> >>>     >
> >>>     > > Hi all,
> >>>     > >
> >>>     > > Let me start, as always, by thanking the efforts of all the
> >>> contributors
> >>>     > > who contributed to this release, especially those who jumped on
> >>> the
> >>>     > issues
> >>>     > > found in RC0.
> >>>     > >
> >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> >>> incorporates 302
> >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> >>>     > >
> >>>     > > You can find the artifacts here:
> >>>     > >
> >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> >>>     > >
> >>>     > > I've done the traditional testing of building from the source
> >>> tarball and
> >>>     > > running a Pi job on a single node cluster. I also verified that
> >>> the
> >>>     > shaded
> >>>     > > jars are not empty.
> >>>     > >
> >>>     > > Found one issue that create-release (probably due to the mvn
> >>> deploy
> >>>     > change)
> >>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
> >>> more time.
> >>>     > > Available here:
> >>>     > >
> >>>     > > https://repository.apache.org/content/repositories/orgapache
> >>> hadoop-1075/
> >>>     > >
> >>>     > > This release will run the standard 5 days, closing on Dec 13th
> >>> at 12:31pm
> >>>     > > Pacific. My +1 to start.
> >>>     > >
> >>>     > > Best,
> >>>     > > Andrew
> >>>     > >
> >>>     >
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>>
> >>
> >>
> >>
> >>
> >
>
>
> --
> A very happy Clouderan
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Subramaniam V K <su...@gmail.com>.

Thanks to everyone who pushed on this release, really great to see a
release coming off trunk.

+1 (binding).

Deployed the RC on a federated YARN cluster consisting of 8 sub-clusters on
CentOs 7.4 running Java 1.8.0_144.

Hit YARN-7652[1] which is not a blocker as I was able to run multiple jobs
successfully once I cleaned up the StateStore.

[1] https://issues.apache.org/jira/browse/YARN-7652

On Wed, Dec 13, 2017 at 11:25 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:

> Thanks Andrew again,
>
> +1 (binding)
>
> * Downloaded source tarball, and compiled with native libs
> successfully: -Pdist,native -Drequire.isal -Drequire.snappy
> -Drequire.openssl -Drequire.zstd.
> * hadoop checknative ran successfully.
> * Upgraded a fsimage that was previously used in a Hadoop 2.6-based CDH
> production cluster (fsimage size: 7.3 GB) to Hadoop 3.0.0 RC1 successfully.
> (Command used: hdfs namenode -upgrade)
> Interestingly, after the upgrade, the fsimage grew from 7.3 GB to 7.5 GB.
> I'm not really sure what went into fsimage between 2.6 and 3.0, but this is
> definitely not a big problem.
>
> In addition,
> * Built HBase 2.0.0 alpha4 against Hadoop 3.0.0 RC1 successfully.
> * Built Hive master branch against Hadoop 3.0.0 RC1 successfully.
>
>
> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi Wei-Chiu,
> >
> > The patchprocess directory is left over from the create-release process,
> > and it looks empty to me. We should still file a create-release JIRA to
> fix
> > this, but I think this is not a blocker. Would you agree?
> >
> > Best,
> > Andrew
> >
> > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> > wrote:
> >
> >> Hi Andrew, thanks the tremendous effort.
> >> I found an empty "patchprocess" directory in the source tarball, that is
> >> not there if you clone from github. Any chance you might have some
> leftover
> >> trash when you made the tarball?
> >> Not wanting to nitpicking, but you might want to double check so we
> don't
> >> ship anything private to you in public :)
> >>
> >>
> >>
> >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <ajay.kumar@hortonworks.com
> >
> >> wrote:
> >>
> >>> +1 (non-binding)
> >>> Thanks for driving this, Andrew Wang!!
> >>>
> >>> - downloaded the src tarball and verified md5 checksum
> >>> - built from source with jdk 1.8.0_111-b14
> >>> - brought up a pseudo distributed cluster
> >>> - did basic file system operations (mkdir, list, put, cat) and
> >>> confirmed that everything was working
> >>> - Run word count, pi and DFSIOTest
> >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> >>>
> >>> Cheers,
> >>> Ajay
> >>>
> >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> >>>
> >>>     +1 (binding)
> >>>
> >>>     - downloaded src tarball, verified md5
> >>>     - built from source with jdk1.8.0_112
> >>>     - started a pseudo cluster with hdfs and kms
> >>>     - sanity checked encryption related operations working
> >>>     - sanity checked webui and logs.
> >>>
> >>>     -Xiao
> >>>
> >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> >>> wrote:
> >>>
> >>>     > +1 (binding)
> >>>     >
> >>>     > - downloaded the src tarball and built the source (-Pdist
> -Pnative)
> >>>     > - verified the checksum
> >>>     > - brought up a secure pseudo distributed cluster
> >>>     > - did some basic file system operations (mkdir, list, put, cat)
> and
> >>>     > confirmed that everything was working
> >>>     > - confirmed that the web UI worked
> >>>     >
> >>>     > Best,
> >>>     > Aaron
> >>>     >
> >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> >>> andrew.wang@cloudera.com>
> >>>     > wrote:
> >>>     >
> >>>     > > Hi all,
> >>>     > >
> >>>     > > Let me start, as always, by thanking the efforts of all the
> >>> contributors
> >>>     > > who contributed to this release, especially those who jumped on
> >>> the
> >>>     > issues
> >>>     > > found in RC0.
> >>>     > >
> >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> >>> incorporates 302
> >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> >>>     > >
> >>>     > > You can find the artifacts here:
> >>>     > >
> >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> >>>     > >
> >>>     > > I've done the traditional testing of building from the source
> >>> tarball and
> >>>     > > running a Pi job on a single node cluster. I also verified that
> >>> the
> >>>     > shaded
> >>>     > > jars are not empty.
> >>>     > >
> >>>     > > Found one issue that create-release (probably due to the mvn
> >>> deploy
> >>>     > change)
> >>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
> >>> more time.
> >>>     > > Available here:
> >>>     > >
> >>>     > > https://repository.apache.org/content/repositories/orgapache
> >>> hadoop-1075/
> >>>     > >
> >>>     > > This release will run the standard 5 days, closing on Dec 13th
> >>> at 12:31pm
> >>>     > > Pacific. My +1 to start.
> >>>     > >
> >>>     > > Best,
> >>>     > > Andrew
> >>>     > >
> >>>     >
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>>
> >>
> >>
> >>
> >>
> >
>
>
> --
> A very happy Clouderan
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Subramaniam V K <su...@gmail.com>.

Thanks to everyone who pushed on this release, really great to see a
release coming off trunk.

+1 (binding).

Deployed the RC on a federated YARN cluster consisting of 8 sub-clusters on
CentOs 7.4 running Java 1.8.0_144.

Hit YARN-7652[1] which is not a blocker as I was able to run multiple jobs
successfully once I cleaned up the StateStore.

[1] https://issues.apache.org/jira/browse/YARN-7652

On Wed, Dec 13, 2017 at 11:25 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:

> Thanks Andrew again,
>
> +1 (binding)
>
> * Downloaded source tarball, and compiled with native libs
> successfully: -Pdist,native -Drequire.isal -Drequire.snappy
> -Drequire.openssl -Drequire.zstd.
> * hadoop checknative ran successfully.
> * Upgraded a fsimage that was previously used in a Hadoop 2.6-based CDH
> production cluster (fsimage size: 7.3 GB) to Hadoop 3.0.0 RC1 successfully.
> (Command used: hdfs namenode -upgrade)
> Interestingly, after the upgrade, the fsimage grew from 7.3 GB to 7.5 GB.
> I'm not really sure what went into fsimage between 2.6 and 3.0, but this is
> definitely not a big problem.
>
> In addition,
> * Built HBase 2.0.0 alpha4 against Hadoop 3.0.0 RC1 successfully.
> * Built Hive master branch against Hadoop 3.0.0 RC1 successfully.
>
>
> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi Wei-Chiu,
> >
> > The patchprocess directory is left over from the create-release process,
> > and it looks empty to me. We should still file a create-release JIRA to
> fix
> > this, but I think this is not a blocker. Would you agree?
> >
> > Best,
> > Andrew
> >
> > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> > wrote:
> >
> >> Hi Andrew, thanks the tremendous effort.
> >> I found an empty "patchprocess" directory in the source tarball, that is
> >> not there if you clone from github. Any chance you might have some
> leftover
> >> trash when you made the tarball?
> >> Not wanting to nitpicking, but you might want to double check so we
> don't
> >> ship anything private to you in public :)
> >>
> >>
> >>
> >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <ajay.kumar@hortonworks.com
> >
> >> wrote:
> >>
> >>> +1 (non-binding)
> >>> Thanks for driving this, Andrew Wang!!
> >>>
> >>> - downloaded the src tarball and verified md5 checksum
> >>> - built from source with jdk 1.8.0_111-b14
> >>> - brought up a pseudo distributed cluster
> >>> - did basic file system operations (mkdir, list, put, cat) and
> >>> confirmed that everything was working
> >>> - Run word count, pi and DFSIOTest
> >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> >>>
> >>> Cheers,
> >>> Ajay
> >>>
> >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> >>>
> >>>     +1 (binding)
> >>>
> >>>     - downloaded src tarball, verified md5
> >>>     - built from source with jdk1.8.0_112
> >>>     - started a pseudo cluster with hdfs and kms
> >>>     - sanity checked encryption related operations working
> >>>     - sanity checked webui and logs.
> >>>
> >>>     -Xiao
> >>>
> >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> >>> wrote:
> >>>
> >>>     > +1 (binding)
> >>>     >
> >>>     > - downloaded the src tarball and built the source (-Pdist
> -Pnative)
> >>>     > - verified the checksum
> >>>     > - brought up a secure pseudo distributed cluster
> >>>     > - did some basic file system operations (mkdir, list, put, cat)
> and
> >>>     > confirmed that everything was working
> >>>     > - confirmed that the web UI worked
> >>>     >
> >>>     > Best,
> >>>     > Aaron
> >>>     >
> >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> >>> andrew.wang@cloudera.com>
> >>>     > wrote:
> >>>     >
> >>>     > > Hi all,
> >>>     > >
> >>>     > > Let me start, as always, by thanking the efforts of all the
> >>> contributors
> >>>     > > who contributed to this release, especially those who jumped on
> >>> the
> >>>     > issues
> >>>     > > found in RC0.
> >>>     > >
> >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> >>> incorporates 302
> >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> >>>     > >
> >>>     > > You can find the artifacts here:
> >>>     > >
> >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> >>>     > >
> >>>     > > I've done the traditional testing of building from the source
> >>> tarball and
> >>>     > > running a Pi job on a single node cluster. I also verified that
> >>> the
> >>>     > shaded
> >>>     > > jars are not empty.
> >>>     > >
> >>>     > > Found one issue that create-release (probably due to the mvn
> >>> deploy
> >>>     > change)
> >>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
> >>> more time.
> >>>     > > Available here:
> >>>     > >
> >>>     > > https://repository.apache.org/content/repositories/orgapache
> >>> hadoop-1075/
> >>>     > >
> >>>     > > This release will run the standard 5 days, closing on Dec 13th
> >>> at 12:31pm
> >>>     > > Pacific. My +1 to start.
> >>>     > >
> >>>     > > Best,
> >>>     > > Andrew
> >>>     > >
> >>>     >
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>>
> >>
> >>
> >>
> >>
> >
>
>
> --
> A very happy Clouderan
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Subramaniam V K <su...@gmail.com>.

Thanks to everyone who pushed on this release, really great to see a
release coming off trunk.

+1 (binding).

Deployed the RC on a federated YARN cluster consisting of 8 sub-clusters on
CentOs 7.4 running Java 1.8.0_144.

Hit YARN-7652[1] which is not a blocker as I was able to run multiple jobs
successfully once I cleaned up the StateStore.

[1] https://issues.apache.org/jira/browse/YARN-7652

On Wed, Dec 13, 2017 at 11:25 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:

> Thanks Andrew again,
>
> +1 (binding)
>
> * Downloaded source tarball, and compiled with native libs
> successfully: -Pdist,native -Drequire.isal -Drequire.snappy
> -Drequire.openssl -Drequire.zstd.
> * hadoop checknative ran successfully.
> * Upgraded a fsimage that was previously used in a Hadoop 2.6-based CDH
> production cluster (fsimage size: 7.3 GB) to Hadoop 3.0.0 RC1 successfully.
> (Command used: hdfs namenode -upgrade)
> Interestingly, after the upgrade, the fsimage grew from 7.3 GB to 7.5 GB.
> I'm not really sure what went into fsimage between 2.6 and 3.0, but this is
> definitely not a big problem.
>
> In addition,
> * Built HBase 2.0.0 alpha4 against Hadoop 3.0.0 RC1 successfully.
> * Built Hive master branch against Hadoop 3.0.0 RC1 successfully.
>
>
> On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi Wei-Chiu,
> >
> > The patchprocess directory is left over from the create-release process,
> > and it looks empty to me. We should still file a create-release JIRA to
> fix
> > this, but I think this is not a blocker. Would you agree?
> >
> > Best,
> > Andrew
> >
> > On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> > wrote:
> >
> >> Hi Andrew, thanks the tremendous effort.
> >> I found an empty "patchprocess" directory in the source tarball, that is
> >> not there if you clone from github. Any chance you might have some
> leftover
> >> trash when you made the tarball?
> >> Not wanting to nitpicking, but you might want to double check so we
> don't
> >> ship anything private to you in public :)
> >>
> >>
> >>
> >> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <ajay.kumar@hortonworks.com
> >
> >> wrote:
> >>
> >>> +1 (non-binding)
> >>> Thanks for driving this, Andrew Wang!!
> >>>
> >>> - downloaded the src tarball and verified md5 checksum
> >>> - built from source with jdk 1.8.0_111-b14
> >>> - brought up a pseudo distributed cluster
> >>> - did basic file system operations (mkdir, list, put, cat) and
> >>> confirmed that everything was working
> >>> - Run word count, pi and DFSIOTest
> >>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
> >>>
> >>> Cheers,
> >>> Ajay
> >>>
> >>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
> >>>
> >>>     +1 (binding)
> >>>
> >>>     - downloaded src tarball, verified md5
> >>>     - built from source with jdk1.8.0_112
> >>>     - started a pseudo cluster with hdfs and kms
> >>>     - sanity checked encryption related operations working
> >>>     - sanity checked webui and logs.
> >>>
> >>>     -Xiao
> >>>
> >>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> >>> wrote:
> >>>
> >>>     > +1 (binding)
> >>>     >
> >>>     > - downloaded the src tarball and built the source (-Pdist
> -Pnative)
> >>>     > - verified the checksum
> >>>     > - brought up a secure pseudo distributed cluster
> >>>     > - did some basic file system operations (mkdir, list, put, cat)
> and
> >>>     > confirmed that everything was working
> >>>     > - confirmed that the web UI worked
> >>>     >
> >>>     > Best,
> >>>     > Aaron
> >>>     >
> >>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> >>> andrew.wang@cloudera.com>
> >>>     > wrote:
> >>>     >
> >>>     > > Hi all,
> >>>     > >
> >>>     > > Let me start, as always, by thanking the efforts of all the
> >>> contributors
> >>>     > > who contributed to this release, especially those who jumped on
> >>> the
> >>>     > issues
> >>>     > > found in RC0.
> >>>     > >
> >>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> >>> incorporates 302
> >>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
> >>>     > >
> >>>     > > You can find the artifacts here:
> >>>     > >
> >>>     > > http://home.apache.org/~wang/3.0.0-RC1/
> >>>     > >
> >>>     > > I've done the traditional testing of building from the source
> >>> tarball and
> >>>     > > running a Pi job on a single node cluster. I also verified that
> >>> the
> >>>     > shaded
> >>>     > > jars are not empty.
> >>>     > >
> >>>     > > Found one issue that create-release (probably due to the mvn
> >>> deploy
> >>>     > change)
> >>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
> >>> more time.
> >>>     > > Available here:
> >>>     > >
> >>>     > > https://repository.apache.org/content/repositories/orgapache
> >>> hadoop-1075/
> >>>     > >
> >>>     > > This release will run the standard 5 days, closing on Dec 13th
> >>> at 12:31pm
> >>>     > > Pacific. My +1 to start.
> >>>     > >
> >>>     > > Best,
> >>>     > > Andrew
> >>>     > >
> >>>     >
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>>
> >>
> >>
> >>
> >>
> >
>
>
> --
> A very happy Clouderan
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Wei-Chiu Chuang <we...@cloudera.com>.

Thanks Andrew again,

+1 (binding)

* Downloaded source tarball, and compiled with native libs
successfully: -Pdist,native -Drequire.isal -Drequire.snappy
-Drequire.openssl -Drequire.zstd.
* hadoop checknative ran successfully.
* Upgraded a fsimage that was previously used in a Hadoop 2.6-based CDH
production cluster (fsimage size: 7.3 GB) to Hadoop 3.0.0 RC1 successfully.
(Command used: hdfs namenode -upgrade)
Interestingly, after the upgrade, the fsimage grew from 7.3 GB to 7.5 GB.
I'm not really sure what went into fsimage between 2.6 and 3.0, but this is
definitely not a big problem.

In addition,
* Built HBase 2.0.0 alpha4 against Hadoop 3.0.0 RC1 successfully.
* Built Hive master branch against Hadoop 3.0.0 RC1 successfully.


On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi Wei-Chiu,
>
> The patchprocess directory is left over from the create-release process,
> and it looks empty to me. We should still file a create-release JIRA to fix
> this, but I think this is not a blocker. Would you agree?
>
> Best,
> Andrew
>
> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> wrote:
>
>> Hi Andrew, thanks the tremendous effort.
>> I found an empty "patchprocess" directory in the source tarball, that is
>> not there if you clone from github. Any chance you might have some leftover
>> trash when you made the tarball?
>> Not wanting to nitpicking, but you might want to double check so we don't
>> ship anything private to you in public :)
>>
>>
>>
>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
>> wrote:
>>
>>> +1 (non-binding)
>>> Thanks for driving this, Andrew Wang!!
>>>
>>> - downloaded the src tarball and verified md5 checksum
>>> - built from source with jdk 1.8.0_111-b14
>>> - brought up a pseudo distributed cluster
>>> - did basic file system operations (mkdir, list, put, cat) and
>>> confirmed that everything was working
>>> - Run word count, pi and DFSIOTest
>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>>
>>> Cheers,
>>> Ajay
>>>
>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>>
>>>     +1 (binding)
>>>
>>>     - downloaded src tarball, verified md5
>>>     - built from source with jdk1.8.0_112
>>>     - started a pseudo cluster with hdfs and kms
>>>     - sanity checked encryption related operations working
>>>     - sanity checked webui and logs.
>>>
>>>     -Xiao
>>>
>>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
>>> wrote:
>>>
>>>     > +1 (binding)
>>>     >
>>>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>>>     > - verified the checksum
>>>     > - brought up a secure pseudo distributed cluster
>>>     > - did some basic file system operations (mkdir, list, put, cat) and
>>>     > confirmed that everything was working
>>>     > - confirmed that the web UI worked
>>>     >
>>>     > Best,
>>>     > Aaron
>>>     >
>>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>>> andrew.wang@cloudera.com>
>>>     > wrote:
>>>     >
>>>     > > Hi all,
>>>     > >
>>>     > > Let me start, as always, by thanking the efforts of all the
>>> contributors
>>>     > > who contributed to this release, especially those who jumped on
>>> the
>>>     > issues
>>>     > > found in RC0.
>>>     > >
>>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
>>> incorporates 302
>>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>>>     > >
>>>     > > You can find the artifacts here:
>>>     > >
>>>     > > http://home.apache.org/~wang/3.0.0-RC1/
>>>     > >
>>>     > > I've done the traditional testing of building from the source
>>> tarball and
>>>     > > running a Pi job on a single node cluster. I also verified that
>>> the
>>>     > shaded
>>>     > > jars are not empty.
>>>     > >
>>>     > > Found one issue that create-release (probably due to the mvn
>>> deploy
>>>     > change)
>>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
>>> more time.
>>>     > > Available here:
>>>     > >
>>>     > > https://repository.apache.org/content/repositories/orgapache
>>> hadoop-1075/
>>>     > >
>>>     > > This release will run the standard 5 days, closing on Dec 13th
>>> at 12:31pm
>>>     > > Pacific. My +1 to start.
>>>     > >
>>>     > > Best,
>>>     > > Andrew
>>>     > >
>>>     >
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>
>>
>>
>>
>>
>


-- 
A very happy Clouderan

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Lei Xu <le...@cloudera.com>.

+1 (binding)

* Verified src tarball and bin tarball, verified md5 of each.
* Build source with -Pdist,native
* Started a pseudo cluster
* Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
dfs put/get/cat on "/" with XOR-2-1 policy.

Thanks Andrew for this great effort!

Best,


On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com> wrote:
> Hi Wei-Chiu,
>
> The patchprocess directory is left over from the create-release process,
> and it looks empty to me. We should still file a create-release JIRA to fix
> this, but I think this is not a blocker. Would you agree?
>
> Best,
> Andrew
>
> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> wrote:
>
>> Hi Andrew, thanks the tremendous effort.
>> I found an empty "patchprocess" directory in the source tarball, that is
>> not there if you clone from github. Any chance you might have some leftover
>> trash when you made the tarball?
>> Not wanting to nitpicking, but you might want to double check so we don't
>> ship anything private to you in public :)
>>
>>
>>
>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
>> wrote:
>>
>>> +1 (non-binding)
>>> Thanks for driving this, Andrew Wang!!
>>>
>>> - downloaded the src tarball and verified md5 checksum
>>> - built from source with jdk 1.8.0_111-b14
>>> - brought up a pseudo distributed cluster
>>> - did basic file system operations (mkdir, list, put, cat) and
>>> confirmed that everything was working
>>> - Run word count, pi and DFSIOTest
>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>>
>>> Cheers,
>>> Ajay
>>>
>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>>
>>>     +1 (binding)
>>>
>>>     - downloaded src tarball, verified md5
>>>     - built from source with jdk1.8.0_112
>>>     - started a pseudo cluster with hdfs and kms
>>>     - sanity checked encryption related operations working
>>>     - sanity checked webui and logs.
>>>
>>>     -Xiao
>>>
>>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
>>> wrote:
>>>
>>>     > +1 (binding)
>>>     >
>>>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>>>     > - verified the checksum
>>>     > - brought up a secure pseudo distributed cluster
>>>     > - did some basic file system operations (mkdir, list, put, cat) and
>>>     > confirmed that everything was working
>>>     > - confirmed that the web UI worked
>>>     >
>>>     > Best,
>>>     > Aaron
>>>     >
>>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>>> andrew.wang@cloudera.com>
>>>     > wrote:
>>>     >
>>>     > > Hi all,
>>>     > >
>>>     > > Let me start, as always, by thanking the efforts of all the
>>> contributors
>>>     > > who contributed to this release, especially those who jumped on
>>> the
>>>     > issues
>>>     > > found in RC0.
>>>     > >
>>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
>>> incorporates 302
>>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>>>     > >
>>>     > > You can find the artifacts here:
>>>     > >
>>>     > > http://home.apache.org/~wang/3.0.0-RC1/
>>>     > >
>>>     > > I've done the traditional testing of building from the source
>>> tarball and
>>>     > > running a Pi job on a single node cluster. I also verified that
>>> the
>>>     > shaded
>>>     > > jars are not empty.
>>>     > >
>>>     > > Found one issue that create-release (probably due to the mvn
>>> deploy
>>>     > change)
>>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
>>> more time.
>>>     > > Available here:
>>>     > >
>>>     > > https://repository.apache.org/content/repositories/orgapache
>>> hadoop-1075/
>>>     > >
>>>     > > This release will run the standard 5 days, closing on Dec 13th at
>>> 12:31pm
>>>     > > Pacific. My +1 to start.
>>>     > >
>>>     > > Best,
>>>     > > Andrew
>>>     > >
>>>     >
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>
>>
>>
>>
>>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Wei-Chiu Chuang <we...@cloudera.com>.

Thanks Andrew again,

+1 (binding)

* Downloaded source tarball, and compiled with native libs
successfully: -Pdist,native -Drequire.isal -Drequire.snappy
-Drequire.openssl -Drequire.zstd.
* hadoop checknative ran successfully.
* Upgraded a fsimage that was previously used in a Hadoop 2.6-based CDH
production cluster (fsimage size: 7.3 GB) to Hadoop 3.0.0 RC1 successfully.
(Command used: hdfs namenode -upgrade)
Interestingly, after the upgrade, the fsimage grew from 7.3 GB to 7.5 GB.
I'm not really sure what went into fsimage between 2.6 and 3.0, but this is
definitely not a big problem.

In addition,
* Built HBase 2.0.0 alpha4 against Hadoop 3.0.0 RC1 successfully.
* Built Hive master branch against Hadoop 3.0.0 RC1 successfully.


On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi Wei-Chiu,
>
> The patchprocess directory is left over from the create-release process,
> and it looks empty to me. We should still file a create-release JIRA to fix
> this, but I think this is not a blocker. Would you agree?
>
> Best,
> Andrew
>
> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> wrote:
>
>> Hi Andrew, thanks the tremendous effort.
>> I found an empty "patchprocess" directory in the source tarball, that is
>> not there if you clone from github. Any chance you might have some leftover
>> trash when you made the tarball?
>> Not wanting to nitpicking, but you might want to double check so we don't
>> ship anything private to you in public :)
>>
>>
>>
>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
>> wrote:
>>
>>> +1 (non-binding)
>>> Thanks for driving this, Andrew Wang!!
>>>
>>> - downloaded the src tarball and verified md5 checksum
>>> - built from source with jdk 1.8.0_111-b14
>>> - brought up a pseudo distributed cluster
>>> - did basic file system operations (mkdir, list, put, cat) and
>>> confirmed that everything was working
>>> - Run word count, pi and DFSIOTest
>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>>
>>> Cheers,
>>> Ajay
>>>
>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>>
>>>     +1 (binding)
>>>
>>>     - downloaded src tarball, verified md5
>>>     - built from source with jdk1.8.0_112
>>>     - started a pseudo cluster with hdfs and kms
>>>     - sanity checked encryption related operations working
>>>     - sanity checked webui and logs.
>>>
>>>     -Xiao
>>>
>>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
>>> wrote:
>>>
>>>     > +1 (binding)
>>>     >
>>>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>>>     > - verified the checksum
>>>     > - brought up a secure pseudo distributed cluster
>>>     > - did some basic file system operations (mkdir, list, put, cat) and
>>>     > confirmed that everything was working
>>>     > - confirmed that the web UI worked
>>>     >
>>>     > Best,
>>>     > Aaron
>>>     >
>>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>>> andrew.wang@cloudera.com>
>>>     > wrote:
>>>     >
>>>     > > Hi all,
>>>     > >
>>>     > > Let me start, as always, by thanking the efforts of all the
>>> contributors
>>>     > > who contributed to this release, especially those who jumped on
>>> the
>>>     > issues
>>>     > > found in RC0.
>>>     > >
>>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
>>> incorporates 302
>>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>>>     > >
>>>     > > You can find the artifacts here:
>>>     > >
>>>     > > http://home.apache.org/~wang/3.0.0-RC1/
>>>     > >
>>>     > > I've done the traditional testing of building from the source
>>> tarball and
>>>     > > running a Pi job on a single node cluster. I also verified that
>>> the
>>>     > shaded
>>>     > > jars are not empty.
>>>     > >
>>>     > > Found one issue that create-release (probably due to the mvn
>>> deploy
>>>     > change)
>>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
>>> more time.
>>>     > > Available here:
>>>     > >
>>>     > > https://repository.apache.org/content/repositories/orgapache
>>> hadoop-1075/
>>>     > >
>>>     > > This release will run the standard 5 days, closing on Dec 13th
>>> at 12:31pm
>>>     > > Pacific. My +1 to start.
>>>     > >
>>>     > > Best,
>>>     > > Andrew
>>>     > >
>>>     >
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>
>>
>>
>>
>>
>


-- 
A very happy Clouderan

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Lei Xu <le...@cloudera.com>.

+1 (binding)

* Verified src tarball and bin tarball, verified md5 of each.
* Build source with -Pdist,native
* Started a pseudo cluster
* Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
dfs put/get/cat on "/" with XOR-2-1 policy.

Thanks Andrew for this great effort!

Best,


On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com> wrote:
> Hi Wei-Chiu,
>
> The patchprocess directory is left over from the create-release process,
> and it looks empty to me. We should still file a create-release JIRA to fix
> this, but I think this is not a blocker. Would you agree?
>
> Best,
> Andrew
>
> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> wrote:
>
>> Hi Andrew, thanks the tremendous effort.
>> I found an empty "patchprocess" directory in the source tarball, that is
>> not there if you clone from github. Any chance you might have some leftover
>> trash when you made the tarball?
>> Not wanting to nitpicking, but you might want to double check so we don't
>> ship anything private to you in public :)
>>
>>
>>
>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
>> wrote:
>>
>>> +1 (non-binding)
>>> Thanks for driving this, Andrew Wang!!
>>>
>>> - downloaded the src tarball and verified md5 checksum
>>> - built from source with jdk 1.8.0_111-b14
>>> - brought up a pseudo distributed cluster
>>> - did basic file system operations (mkdir, list, put, cat) and
>>> confirmed that everything was working
>>> - Run word count, pi and DFSIOTest
>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>>
>>> Cheers,
>>> Ajay
>>>
>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>>
>>>     +1 (binding)
>>>
>>>     - downloaded src tarball, verified md5
>>>     - built from source with jdk1.8.0_112
>>>     - started a pseudo cluster with hdfs and kms
>>>     - sanity checked encryption related operations working
>>>     - sanity checked webui and logs.
>>>
>>>     -Xiao
>>>
>>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
>>> wrote:
>>>
>>>     > +1 (binding)
>>>     >
>>>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>>>     > - verified the checksum
>>>     > - brought up a secure pseudo distributed cluster
>>>     > - did some basic file system operations (mkdir, list, put, cat) and
>>>     > confirmed that everything was working
>>>     > - confirmed that the web UI worked
>>>     >
>>>     > Best,
>>>     > Aaron
>>>     >
>>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>>> andrew.wang@cloudera.com>
>>>     > wrote:
>>>     >
>>>     > > Hi all,
>>>     > >
>>>     > > Let me start, as always, by thanking the efforts of all the
>>> contributors
>>>     > > who contributed to this release, especially those who jumped on
>>> the
>>>     > issues
>>>     > > found in RC0.
>>>     > >
>>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
>>> incorporates 302
>>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>>>     > >
>>>     > > You can find the artifacts here:
>>>     > >
>>>     > > http://home.apache.org/~wang/3.0.0-RC1/
>>>     > >
>>>     > > I've done the traditional testing of building from the source
>>> tarball and
>>>     > > running a Pi job on a single node cluster. I also verified that
>>> the
>>>     > shaded
>>>     > > jars are not empty.
>>>     > >
>>>     > > Found one issue that create-release (probably due to the mvn
>>> deploy
>>>     > change)
>>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
>>> more time.
>>>     > > Available here:
>>>     > >
>>>     > > https://repository.apache.org/content/repositories/orgapache
>>> hadoop-1075/
>>>     > >
>>>     > > This release will run the standard 5 days, closing on Dec 13th at
>>> 12:31pm
>>>     > > Pacific. My +1 to start.
>>>     > >
>>>     > > Best,
>>>     > > Andrew
>>>     > >
>>>     >
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>
>>
>>
>>
>>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Wei-Chiu Chuang <we...@cloudera.com>.

Thanks Andrew again,

+1 (binding)

* Downloaded source tarball, and compiled with native libs
successfully: -Pdist,native -Drequire.isal -Drequire.snappy
-Drequire.openssl -Drequire.zstd.
* hadoop checknative ran successfully.
* Upgraded a fsimage that was previously used in a Hadoop 2.6-based CDH
production cluster (fsimage size: 7.3 GB) to Hadoop 3.0.0 RC1 successfully.
(Command used: hdfs namenode -upgrade)
Interestingly, after the upgrade, the fsimage grew from 7.3 GB to 7.5 GB.
I'm not really sure what went into fsimage between 2.6 and 3.0, but this is
definitely not a big problem.

In addition,
* Built HBase 2.0.0 alpha4 against Hadoop 3.0.0 RC1 successfully.
* Built Hive master branch against Hadoop 3.0.0 RC1 successfully.


On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi Wei-Chiu,
>
> The patchprocess directory is left over from the create-release process,
> and it looks empty to me. We should still file a create-release JIRA to fix
> this, but I think this is not a blocker. Would you agree?
>
> Best,
> Andrew
>
> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> wrote:
>
>> Hi Andrew, thanks the tremendous effort.
>> I found an empty "patchprocess" directory in the source tarball, that is
>> not there if you clone from github. Any chance you might have some leftover
>> trash when you made the tarball?
>> Not wanting to nitpicking, but you might want to double check so we don't
>> ship anything private to you in public :)
>>
>>
>>
>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
>> wrote:
>>
>>> +1 (non-binding)
>>> Thanks for driving this, Andrew Wang!!
>>>
>>> - downloaded the src tarball and verified md5 checksum
>>> - built from source with jdk 1.8.0_111-b14
>>> - brought up a pseudo distributed cluster
>>> - did basic file system operations (mkdir, list, put, cat) and
>>> confirmed that everything was working
>>> - Run word count, pi and DFSIOTest
>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>>
>>> Cheers,
>>> Ajay
>>>
>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>>
>>>     +1 (binding)
>>>
>>>     - downloaded src tarball, verified md5
>>>     - built from source with jdk1.8.0_112
>>>     - started a pseudo cluster with hdfs and kms
>>>     - sanity checked encryption related operations working
>>>     - sanity checked webui and logs.
>>>
>>>     -Xiao
>>>
>>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
>>> wrote:
>>>
>>>     > +1 (binding)
>>>     >
>>>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>>>     > - verified the checksum
>>>     > - brought up a secure pseudo distributed cluster
>>>     > - did some basic file system operations (mkdir, list, put, cat) and
>>>     > confirmed that everything was working
>>>     > - confirmed that the web UI worked
>>>     >
>>>     > Best,
>>>     > Aaron
>>>     >
>>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>>> andrew.wang@cloudera.com>
>>>     > wrote:
>>>     >
>>>     > > Hi all,
>>>     > >
>>>     > > Let me start, as always, by thanking the efforts of all the
>>> contributors
>>>     > > who contributed to this release, especially those who jumped on
>>> the
>>>     > issues
>>>     > > found in RC0.
>>>     > >
>>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
>>> incorporates 302
>>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>>>     > >
>>>     > > You can find the artifacts here:
>>>     > >
>>>     > > http://home.apache.org/~wang/3.0.0-RC1/
>>>     > >
>>>     > > I've done the traditional testing of building from the source
>>> tarball and
>>>     > > running a Pi job on a single node cluster. I also verified that
>>> the
>>>     > shaded
>>>     > > jars are not empty.
>>>     > >
>>>     > > Found one issue that create-release (probably due to the mvn
>>> deploy
>>>     > change)
>>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
>>> more time.
>>>     > > Available here:
>>>     > >
>>>     > > https://repository.apache.org/content/repositories/orgapache
>>> hadoop-1075/
>>>     > >
>>>     > > This release will run the standard 5 days, closing on Dec 13th
>>> at 12:31pm
>>>     > > Pacific. My +1 to start.
>>>     > >
>>>     > > Best,
>>>     > > Andrew
>>>     > >
>>>     >
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>
>>
>>
>>
>>
>


-- 
A very happy Clouderan

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Lei Xu <le...@cloudera.com>.

+1 (binding)

* Verified src tarball and bin tarball, verified md5 of each.
* Build source with -Pdist,native
* Started a pseudo cluster
* Run ec -listPolicies / -getPolicy / -setPolicy on /  , and run hdfs
dfs put/get/cat on "/" with XOR-2-1 policy.

Thanks Andrew for this great effort!

Best,


On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com> wrote:
> Hi Wei-Chiu,
>
> The patchprocess directory is left over from the create-release process,
> and it looks empty to me. We should still file a create-release JIRA to fix
> this, but I think this is not a blocker. Would you agree?
>
> Best,
> Andrew
>
> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> wrote:
>
>> Hi Andrew, thanks the tremendous effort.
>> I found an empty "patchprocess" directory in the source tarball, that is
>> not there if you clone from github. Any chance you might have some leftover
>> trash when you made the tarball?
>> Not wanting to nitpicking, but you might want to double check so we don't
>> ship anything private to you in public :)
>>
>>
>>
>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
>> wrote:
>>
>>> +1 (non-binding)
>>> Thanks for driving this, Andrew Wang!!
>>>
>>> - downloaded the src tarball and verified md5 checksum
>>> - built from source with jdk 1.8.0_111-b14
>>> - brought up a pseudo distributed cluster
>>> - did basic file system operations (mkdir, list, put, cat) and
>>> confirmed that everything was working
>>> - Run word count, pi and DFSIOTest
>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>>
>>> Cheers,
>>> Ajay
>>>
>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>>
>>>     +1 (binding)
>>>
>>>     - downloaded src tarball, verified md5
>>>     - built from source with jdk1.8.0_112
>>>     - started a pseudo cluster with hdfs and kms
>>>     - sanity checked encryption related operations working
>>>     - sanity checked webui and logs.
>>>
>>>     -Xiao
>>>
>>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
>>> wrote:
>>>
>>>     > +1 (binding)
>>>     >
>>>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>>>     > - verified the checksum
>>>     > - brought up a secure pseudo distributed cluster
>>>     > - did some basic file system operations (mkdir, list, put, cat) and
>>>     > confirmed that everything was working
>>>     > - confirmed that the web UI worked
>>>     >
>>>     > Best,
>>>     > Aaron
>>>     >
>>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>>> andrew.wang@cloudera.com>
>>>     > wrote:
>>>     >
>>>     > > Hi all,
>>>     > >
>>>     > > Let me start, as always, by thanking the efforts of all the
>>> contributors
>>>     > > who contributed to this release, especially those who jumped on
>>> the
>>>     > issues
>>>     > > found in RC0.
>>>     > >
>>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
>>> incorporates 302
>>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>>>     > >
>>>     > > You can find the artifacts here:
>>>     > >
>>>     > > http://home.apache.org/~wang/3.0.0-RC1/
>>>     > >
>>>     > > I've done the traditional testing of building from the source
>>> tarball and
>>>     > > running a Pi job on a single node cluster. I also verified that
>>> the
>>>     > shaded
>>>     > > jars are not empty.
>>>     > >
>>>     > > Found one issue that create-release (probably due to the mvn
>>> deploy
>>>     > change)
>>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
>>> more time.
>>>     > > Available here:
>>>     > >
>>>     > > https://repository.apache.org/content/repositories/orgapache
>>> hadoop-1075/
>>>     > >
>>>     > > This release will run the standard 5 days, closing on Dec 13th at
>>> 12:31pm
>>>     > > Pacific. My +1 to start.
>>>     > >
>>>     > > Best,
>>>     > > Andrew
>>>     > >
>>>     >
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>
>>
>>
>>
>>



-- 
Lei (Eddy) Xu
Software Engineer, Cloudera

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Wei-Chiu Chuang <we...@cloudera.com>.

Thanks Andrew again,

+1 (binding)

* Downloaded source tarball, and compiled with native libs
successfully: -Pdist,native -Drequire.isal -Drequire.snappy
-Drequire.openssl -Drequire.zstd.
* hadoop checknative ran successfully.
* Upgraded a fsimage that was previously used in a Hadoop 2.6-based CDH
production cluster (fsimage size: 7.3 GB) to Hadoop 3.0.0 RC1 successfully.
(Command used: hdfs namenode -upgrade)
Interestingly, after the upgrade, the fsimage grew from 7.3 GB to 7.5 GB.
I'm not really sure what went into fsimage between 2.6 and 3.0, but this is
definitely not a big problem.

In addition,
* Built HBase 2.0.0 alpha4 against Hadoop 3.0.0 RC1 successfully.
* Built Hive master branch against Hadoop 3.0.0 RC1 successfully.


On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi Wei-Chiu,
>
> The patchprocess directory is left over from the create-release process,
> and it looks empty to me. We should still file a create-release JIRA to fix
> this, but I think this is not a blocker. Would you agree?
>
> Best,
> Andrew
>
> On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
> wrote:
>
>> Hi Andrew, thanks the tremendous effort.
>> I found an empty "patchprocess" directory in the source tarball, that is
>> not there if you clone from github. Any chance you might have some leftover
>> trash when you made the tarball?
>> Not wanting to nitpicking, but you might want to double check so we don't
>> ship anything private to you in public :)
>>
>>
>>
>> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
>> wrote:
>>
>>> +1 (non-binding)
>>> Thanks for driving this, Andrew Wang!!
>>>
>>> - downloaded the src tarball and verified md5 checksum
>>> - built from source with jdk 1.8.0_111-b14
>>> - brought up a pseudo distributed cluster
>>> - did basic file system operations (mkdir, list, put, cat) and
>>> confirmed that everything was working
>>> - Run word count, pi and DFSIOTest
>>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>>
>>> Cheers,
>>> Ajay
>>>
>>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>>
>>>     +1 (binding)
>>>
>>>     - downloaded src tarball, verified md5
>>>     - built from source with jdk1.8.0_112
>>>     - started a pseudo cluster with hdfs and kms
>>>     - sanity checked encryption related operations working
>>>     - sanity checked webui and logs.
>>>
>>>     -Xiao
>>>
>>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
>>> wrote:
>>>
>>>     > +1 (binding)
>>>     >
>>>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>>>     > - verified the checksum
>>>     > - brought up a secure pseudo distributed cluster
>>>     > - did some basic file system operations (mkdir, list, put, cat) and
>>>     > confirmed that everything was working
>>>     > - confirmed that the web UI worked
>>>     >
>>>     > Best,
>>>     > Aaron
>>>     >
>>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>>> andrew.wang@cloudera.com>
>>>     > wrote:
>>>     >
>>>     > > Hi all,
>>>     > >
>>>     > > Let me start, as always, by thanking the efforts of all the
>>> contributors
>>>     > > who contributed to this release, especially those who jumped on
>>> the
>>>     > issues
>>>     > > found in RC0.
>>>     > >
>>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
>>> incorporates 302
>>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>>>     > >
>>>     > > You can find the artifacts here:
>>>     > >
>>>     > > http://home.apache.org/~wang/3.0.0-RC1/
>>>     > >
>>>     > > I've done the traditional testing of building from the source
>>> tarball and
>>>     > > running a Pi job on a single node cluster. I also verified that
>>> the
>>>     > shaded
>>>     > > jars are not empty.
>>>     > >
>>>     > > Found one issue that create-release (probably due to the mvn
>>> deploy
>>>     > change)
>>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
>>> more time.
>>>     > > Available here:
>>>     > >
>>>     > > https://repository.apache.org/content/repositories/orgapache
>>> hadoop-1075/
>>>     > >
>>>     > > This release will run the standard 5 days, closing on Dec 13th
>>> at 12:31pm
>>>     > > Pacific. My +1 to start.
>>>     > >
>>>     > > Best,
>>>     > > Andrew
>>>     > >
>>>     >
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>
>>
>>
>>
>>
>


-- 
A very happy Clouderan

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Hi Wei-Chiu,

The patchprocess directory is left over from the create-release process,
and it looks empty to me. We should still file a create-release JIRA to fix
this, but I think this is not a blocker. Would you agree?

Best,
Andrew

On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:

> Hi Andrew, thanks the tremendous effort.
> I found an empty "patchprocess" directory in the source tarball, that is
> not there if you clone from github. Any chance you might have some leftover
> trash when you made the tarball?
> Not wanting to nitpicking, but you might want to double check so we don't
> ship anything private to you in public :)
>
>
>
> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
> wrote:
>
>> +1 (non-binding)
>> Thanks for driving this, Andrew Wang!!
>>
>> - downloaded the src tarball and verified md5 checksum
>> - built from source with jdk 1.8.0_111-b14
>> - brought up a pseudo distributed cluster
>> - did basic file system operations (mkdir, list, put, cat) and
>> confirmed that everything was working
>> - Run word count, pi and DFSIOTest
>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>
>> Cheers,
>> Ajay
>>
>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>
>>     +1 (binding)
>>
>>     - downloaded src tarball, verified md5
>>     - built from source with jdk1.8.0_112
>>     - started a pseudo cluster with hdfs and kms
>>     - sanity checked encryption related operations working
>>     - sanity checked webui and logs.
>>
>>     -Xiao
>>
>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
>> wrote:
>>
>>     > +1 (binding)
>>     >
>>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>>     > - verified the checksum
>>     > - brought up a secure pseudo distributed cluster
>>     > - did some basic file system operations (mkdir, list, put, cat) and
>>     > confirmed that everything was working
>>     > - confirmed that the web UI worked
>>     >
>>     > Best,
>>     > Aaron
>>     >
>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>> andrew.wang@cloudera.com>
>>     > wrote:
>>     >
>>     > > Hi all,
>>     > >
>>     > > Let me start, as always, by thanking the efforts of all the
>> contributors
>>     > > who contributed to this release, especially those who jumped on
>> the
>>     > issues
>>     > > found in RC0.
>>     > >
>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
>> incorporates 302
>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>>     > >
>>     > > You can find the artifacts here:
>>     > >
>>     > > http://home.apache.org/~wang/3.0.0-RC1/
>>     > >
>>     > > I've done the traditional testing of building from the source
>> tarball and
>>     > > running a Pi job on a single node cluster. I also verified that
>> the
>>     > shaded
>>     > > jars are not empty.
>>     > >
>>     > > Found one issue that create-release (probably due to the mvn
>> deploy
>>     > change)
>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
>> more time.
>>     > > Available here:
>>     > >
>>     > > https://repository.apache.org/content/repositories/orgapache
>> hadoop-1075/
>>     > >
>>     > > This release will run the standard 5 days, closing on Dec 13th at
>> 12:31pm
>>     > > Pacific. My +1 to start.
>>     > >
>>     > > Best,
>>     > > Andrew
>>     > >
>>     >
>>
>>
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>
>
>
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Hi Wei-Chiu,

The patchprocess directory is left over from the create-release process,
and it looks empty to me. We should still file a create-release JIRA to fix
this, but I think this is not a blocker. Would you agree?

Best,
Andrew

On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:

> Hi Andrew, thanks the tremendous effort.
> I found an empty "patchprocess" directory in the source tarball, that is
> not there if you clone from github. Any chance you might have some leftover
> trash when you made the tarball?
> Not wanting to nitpicking, but you might want to double check so we don't
> ship anything private to you in public :)
>
>
>
> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
> wrote:
>
>> +1 (non-binding)
>> Thanks for driving this, Andrew Wang!!
>>
>> - downloaded the src tarball and verified md5 checksum
>> - built from source with jdk 1.8.0_111-b14
>> - brought up a pseudo distributed cluster
>> - did basic file system operations (mkdir, list, put, cat) and
>> confirmed that everything was working
>> - Run word count, pi and DFSIOTest
>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>
>> Cheers,
>> Ajay
>>
>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>
>>     +1 (binding)
>>
>>     - downloaded src tarball, verified md5
>>     - built from source with jdk1.8.0_112
>>     - started a pseudo cluster with hdfs and kms
>>     - sanity checked encryption related operations working
>>     - sanity checked webui and logs.
>>
>>     -Xiao
>>
>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
>> wrote:
>>
>>     > +1 (binding)
>>     >
>>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>>     > - verified the checksum
>>     > - brought up a secure pseudo distributed cluster
>>     > - did some basic file system operations (mkdir, list, put, cat) and
>>     > confirmed that everything was working
>>     > - confirmed that the web UI worked
>>     >
>>     > Best,
>>     > Aaron
>>     >
>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>> andrew.wang@cloudera.com>
>>     > wrote:
>>     >
>>     > > Hi all,
>>     > >
>>     > > Let me start, as always, by thanking the efforts of all the
>> contributors
>>     > > who contributed to this release, especially those who jumped on
>> the
>>     > issues
>>     > > found in RC0.
>>     > >
>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
>> incorporates 302
>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>>     > >
>>     > > You can find the artifacts here:
>>     > >
>>     > > http://home.apache.org/~wang/3.0.0-RC1/
>>     > >
>>     > > I've done the traditional testing of building from the source
>> tarball and
>>     > > running a Pi job on a single node cluster. I also verified that
>> the
>>     > shaded
>>     > > jars are not empty.
>>     > >
>>     > > Found one issue that create-release (probably due to the mvn
>> deploy
>>     > change)
>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
>> more time.
>>     > > Available here:
>>     > >
>>     > > https://repository.apache.org/content/repositories/orgapache
>> hadoop-1075/
>>     > >
>>     > > This release will run the standard 5 days, closing on Dec 13th at
>> 12:31pm
>>     > > Pacific. My +1 to start.
>>     > >
>>     > > Best,
>>     > > Andrew
>>     > >
>>     >
>>
>>
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>
>
>
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Hi Wei-Chiu,

The patchprocess directory is left over from the create-release process,
and it looks empty to me. We should still file a create-release JIRA to fix
this, but I think this is not a blocker. Would you agree?

Best,
Andrew

On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:

> Hi Andrew, thanks the tremendous effort.
> I found an empty "patchprocess" directory in the source tarball, that is
> not there if you clone from github. Any chance you might have some leftover
> trash when you made the tarball?
> Not wanting to nitpicking, but you might want to double check so we don't
> ship anything private to you in public :)
>
>
>
> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
> wrote:
>
>> +1 (non-binding)
>> Thanks for driving this, Andrew Wang!!
>>
>> - downloaded the src tarball and verified md5 checksum
>> - built from source with jdk 1.8.0_111-b14
>> - brought up a pseudo distributed cluster
>> - did basic file system operations (mkdir, list, put, cat) and
>> confirmed that everything was working
>> - Run word count, pi and DFSIOTest
>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>
>> Cheers,
>> Ajay
>>
>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>
>>     +1 (binding)
>>
>>     - downloaded src tarball, verified md5
>>     - built from source with jdk1.8.0_112
>>     - started a pseudo cluster with hdfs and kms
>>     - sanity checked encryption related operations working
>>     - sanity checked webui and logs.
>>
>>     -Xiao
>>
>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
>> wrote:
>>
>>     > +1 (binding)
>>     >
>>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>>     > - verified the checksum
>>     > - brought up a secure pseudo distributed cluster
>>     > - did some basic file system operations (mkdir, list, put, cat) and
>>     > confirmed that everything was working
>>     > - confirmed that the web UI worked
>>     >
>>     > Best,
>>     > Aaron
>>     >
>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>> andrew.wang@cloudera.com>
>>     > wrote:
>>     >
>>     > > Hi all,
>>     > >
>>     > > Let me start, as always, by thanking the efforts of all the
>> contributors
>>     > > who contributed to this release, especially those who jumped on
>> the
>>     > issues
>>     > > found in RC0.
>>     > >
>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
>> incorporates 302
>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>>     > >
>>     > > You can find the artifacts here:
>>     > >
>>     > > http://home.apache.org/~wang/3.0.0-RC1/
>>     > >
>>     > > I've done the traditional testing of building from the source
>> tarball and
>>     > > running a Pi job on a single node cluster. I also verified that
>> the
>>     > shaded
>>     > > jars are not empty.
>>     > >
>>     > > Found one issue that create-release (probably due to the mvn
>> deploy
>>     > change)
>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
>> more time.
>>     > > Available here:
>>     > >
>>     > > https://repository.apache.org/content/repositories/orgapache
>> hadoop-1075/
>>     > >
>>     > > This release will run the standard 5 days, closing on Dec 13th at
>> 12:31pm
>>     > > Pacific. My +1 to start.
>>     > >
>>     > > Best,
>>     > > Andrew
>>     > >
>>     >
>>
>>
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>
>
>
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Hi Wei-Chiu,

The patchprocess directory is left over from the create-release process,
and it looks empty to me. We should still file a create-release JIRA to fix
this, but I think this is not a blocker. Would you agree?

Best,
Andrew

On Tue, Dec 12, 2017 at 9:44 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:

> Hi Andrew, thanks the tremendous effort.
> I found an empty "patchprocess" directory in the source tarball, that is
> not there if you clone from github. Any chance you might have some leftover
> trash when you made the tarball?
> Not wanting to nitpicking, but you might want to double check so we don't
> ship anything private to you in public :)
>
>
>
> On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
> wrote:
>
>> +1 (non-binding)
>> Thanks for driving this, Andrew Wang!!
>>
>> - downloaded the src tarball and verified md5 checksum
>> - built from source with jdk 1.8.0_111-b14
>> - brought up a pseudo distributed cluster
>> - did basic file system operations (mkdir, list, put, cat) and
>> confirmed that everything was working
>> - Run word count, pi and DFSIOTest
>> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>>
>> Cheers,
>> Ajay
>>
>> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>>
>>     +1 (binding)
>>
>>     - downloaded src tarball, verified md5
>>     - built from source with jdk1.8.0_112
>>     - started a pseudo cluster with hdfs and kms
>>     - sanity checked encryption related operations working
>>     - sanity checked webui and logs.
>>
>>     -Xiao
>>
>>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
>> wrote:
>>
>>     > +1 (binding)
>>     >
>>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>>     > - verified the checksum
>>     > - brought up a secure pseudo distributed cluster
>>     > - did some basic file system operations (mkdir, list, put, cat) and
>>     > confirmed that everything was working
>>     > - confirmed that the web UI worked
>>     >
>>     > Best,
>>     > Aaron
>>     >
>>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
>> andrew.wang@cloudera.com>
>>     > wrote:
>>     >
>>     > > Hi all,
>>     > >
>>     > > Let me start, as always, by thanking the efforts of all the
>> contributors
>>     > > who contributed to this release, especially those who jumped on
>> the
>>     > issues
>>     > > found in RC0.
>>     > >
>>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
>> incorporates 302
>>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>>     > >
>>     > > You can find the artifacts here:
>>     > >
>>     > > http://home.apache.org/~wang/3.0.0-RC1/
>>     > >
>>     > > I've done the traditional testing of building from the source
>> tarball and
>>     > > running a Pi job on a single node cluster. I also verified that
>> the
>>     > shaded
>>     > > jars are not empty.
>>     > >
>>     > > Found one issue that create-release (probably due to the mvn
>> deploy
>>     > change)
>>     > > didn't sign the artifacts, but I fixed that by calling mvn one
>> more time.
>>     > > Available here:
>>     > >
>>     > > https://repository.apache.org/content/repositories/orgapache
>> hadoop-1075/
>>     > >
>>     > > This release will run the standard 5 days, closing on Dec 13th at
>> 12:31pm
>>     > > Pacific. My +1 to start.
>>     > >
>>     > > Best,
>>     > > Andrew
>>     > >
>>     >
>>
>>
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>
>
>
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Wei-Chiu Chuang <we...@cloudera.com>.

Hi Andrew, thanks the tremendous effort.
I found an empty "patchprocess" directory in the source tarball, that is
not there if you clone from github. Any chance you might have some leftover
trash when you made the tarball?
Not wanting to nitpicking, but you might want to double check so we don't
ship anything private to you in public :)



On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
wrote:

> +1 (non-binding)
> Thanks for driving this, Andrew Wang!!
>
> - downloaded the src tarball and verified md5 checksum
> - built from source with jdk 1.8.0_111-b14
> - brought up a pseudo distributed cluster
> - did basic file system operations (mkdir, list, put, cat) and
> confirmed that everything was working
> - Run word count, pi and DFSIOTest
> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>
> Cheers,
> Ajay
>
> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>
>     +1 (binding)
>
>     - downloaded src tarball, verified md5
>     - built from source with jdk1.8.0_112
>     - started a pseudo cluster with hdfs and kms
>     - sanity checked encryption related operations working
>     - sanity checked webui and logs.
>
>     -Xiao
>
>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> wrote:
>
>     > +1 (binding)
>     >
>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>     > - verified the checksum
>     > - brought up a secure pseudo distributed cluster
>     > - did some basic file system operations (mkdir, list, put, cat) and
>     > confirmed that everything was working
>     > - confirmed that the web UI worked
>     >
>     > Best,
>     > Aaron
>     >
>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> andrew.wang@cloudera.com>
>     > wrote:
>     >
>     > > Hi all,
>     > >
>     > > Let me start, as always, by thanking the efforts of all the
> contributors
>     > > who contributed to this release, especially those who jumped on the
>     > issues
>     > > found in RC0.
>     > >
>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> incorporates 302
>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>     > >
>     > > You can find the artifacts here:
>     > >
>     > > http://home.apache.org/~wang/3.0.0-RC1/
>     > >
>     > > I've done the traditional testing of building from the source
> tarball and
>     > > running a Pi job on a single node cluster. I also verified that the
>     > shaded
>     > > jars are not empty.
>     > >
>     > > Found one issue that create-release (probably due to the mvn deploy
>     > change)
>     > > didn't sign the artifacts, but I fixed that by calling mvn one
> more time.
>     > > Available here:
>     > >
>     > > https://repository.apache.org/content/repositories/orgapache
> hadoop-1075/
>     > >
>     > > This release will run the standard 5 days, closing on Dec 13th at
> 12:31pm
>     > > Pacific. My +1 to start.
>     > >
>     > > Best,
>     > > Andrew
>     > >
>     >
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Wei-Chiu Chuang <we...@cloudera.com>.

Hi Andrew, thanks the tremendous effort.
I found an empty "patchprocess" directory in the source tarball, that is
not there if you clone from github. Any chance you might have some leftover
trash when you made the tarball?
Not wanting to nitpicking, but you might want to double check so we don't
ship anything private to you in public :)



On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
wrote:

> +1 (non-binding)
> Thanks for driving this, Andrew Wang!!
>
> - downloaded the src tarball and verified md5 checksum
> - built from source with jdk 1.8.0_111-b14
> - brought up a pseudo distributed cluster
> - did basic file system operations (mkdir, list, put, cat) and
> confirmed that everything was working
> - Run word count, pi and DFSIOTest
> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>
> Cheers,
> Ajay
>
> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>
>     +1 (binding)
>
>     - downloaded src tarball, verified md5
>     - built from source with jdk1.8.0_112
>     - started a pseudo cluster with hdfs and kms
>     - sanity checked encryption related operations working
>     - sanity checked webui and logs.
>
>     -Xiao
>
>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> wrote:
>
>     > +1 (binding)
>     >
>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>     > - verified the checksum
>     > - brought up a secure pseudo distributed cluster
>     > - did some basic file system operations (mkdir, list, put, cat) and
>     > confirmed that everything was working
>     > - confirmed that the web UI worked
>     >
>     > Best,
>     > Aaron
>     >
>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> andrew.wang@cloudera.com>
>     > wrote:
>     >
>     > > Hi all,
>     > >
>     > > Let me start, as always, by thanking the efforts of all the
> contributors
>     > > who contributed to this release, especially those who jumped on the
>     > issues
>     > > found in RC0.
>     > >
>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> incorporates 302
>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>     > >
>     > > You can find the artifacts here:
>     > >
>     > > http://home.apache.org/~wang/3.0.0-RC1/
>     > >
>     > > I've done the traditional testing of building from the source
> tarball and
>     > > running a Pi job on a single node cluster. I also verified that the
>     > shaded
>     > > jars are not empty.
>     > >
>     > > Found one issue that create-release (probably due to the mvn deploy
>     > change)
>     > > didn't sign the artifacts, but I fixed that by calling mvn one
> more time.
>     > > Available here:
>     > >
>     > > https://repository.apache.org/content/repositories/orgapache
> hadoop-1075/
>     > >
>     > > This release will run the standard 5 days, closing on Dec 13th at
> 12:31pm
>     > > Pacific. My +1 to start.
>     > >
>     > > Best,
>     > > Andrew
>     > >
>     >
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Wei-Chiu Chuang <we...@cloudera.com>.

Hi Andrew, thanks the tremendous effort.
I found an empty "patchprocess" directory in the source tarball, that is
not there if you clone from github. Any chance you might have some leftover
trash when you made the tarball?
Not wanting to nitpicking, but you might want to double check so we don't
ship anything private to you in public :)



On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
wrote:

> +1 (non-binding)
> Thanks for driving this, Andrew Wang!!
>
> - downloaded the src tarball and verified md5 checksum
> - built from source with jdk 1.8.0_111-b14
> - brought up a pseudo distributed cluster
> - did basic file system operations (mkdir, list, put, cat) and
> confirmed that everything was working
> - Run word count, pi and DFSIOTest
> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>
> Cheers,
> Ajay
>
> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>
>     +1 (binding)
>
>     - downloaded src tarball, verified md5
>     - built from source with jdk1.8.0_112
>     - started a pseudo cluster with hdfs and kms
>     - sanity checked encryption related operations working
>     - sanity checked webui and logs.
>
>     -Xiao
>
>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> wrote:
>
>     > +1 (binding)
>     >
>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>     > - verified the checksum
>     > - brought up a secure pseudo distributed cluster
>     > - did some basic file system operations (mkdir, list, put, cat) and
>     > confirmed that everything was working
>     > - confirmed that the web UI worked
>     >
>     > Best,
>     > Aaron
>     >
>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> andrew.wang@cloudera.com>
>     > wrote:
>     >
>     > > Hi all,
>     > >
>     > > Let me start, as always, by thanking the efforts of all the
> contributors
>     > > who contributed to this release, especially those who jumped on the
>     > issues
>     > > found in RC0.
>     > >
>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> incorporates 302
>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>     > >
>     > > You can find the artifacts here:
>     > >
>     > > http://home.apache.org/~wang/3.0.0-RC1/
>     > >
>     > > I've done the traditional testing of building from the source
> tarball and
>     > > running a Pi job on a single node cluster. I also verified that the
>     > shaded
>     > > jars are not empty.
>     > >
>     > > Found one issue that create-release (probably due to the mvn deploy
>     > change)
>     > > didn't sign the artifacts, but I fixed that by calling mvn one
> more time.
>     > > Available here:
>     > >
>     > > https://repository.apache.org/content/repositories/orgapache
> hadoop-1075/
>     > >
>     > > This release will run the standard 5 days, closing on Dec 13th at
> 12:31pm
>     > > Pacific. My +1 to start.
>     > >
>     > > Best,
>     > > Andrew
>     > >
>     >
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Wei-Chiu Chuang <we...@cloudera.com>.

Hi Andrew, thanks the tremendous effort.
I found an empty "patchprocess" directory in the source tarball, that is
not there if you clone from github. Any chance you might have some leftover
trash when you made the tarball?
Not wanting to nitpicking, but you might want to double check so we don't
ship anything private to you in public :)



On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar <aj...@hortonworks.com>
wrote:

> +1 (non-binding)
> Thanks for driving this, Andrew Wang!!
>
> - downloaded the src tarball and verified md5 checksum
> - built from source with jdk 1.8.0_111-b14
> - brought up a pseudo distributed cluster
> - did basic file system operations (mkdir, list, put, cat) and
> confirmed that everything was working
> - Run word count, pi and DFSIOTest
> - run hdfs and yarn, confirmed that the NN, RM web UI worked
>
> Cheers,
> Ajay
>
> On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:
>
>     +1 (binding)
>
>     - downloaded src tarball, verified md5
>     - built from source with jdk1.8.0_112
>     - started a pseudo cluster with hdfs and kms
>     - sanity checked encryption related operations working
>     - sanity checked webui and logs.
>
>     -Xiao
>
>     On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org>
> wrote:
>
>     > +1 (binding)
>     >
>     > - downloaded the src tarball and built the source (-Pdist -Pnative)
>     > - verified the checksum
>     > - brought up a secure pseudo distributed cluster
>     > - did some basic file system operations (mkdir, list, put, cat) and
>     > confirmed that everything was working
>     > - confirmed that the web UI worked
>     >
>     > Best,
>     > Aaron
>     >
>     > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <
> andrew.wang@cloudera.com>
>     > wrote:
>     >
>     > > Hi all,
>     > >
>     > > Let me start, as always, by thanking the efforts of all the
> contributors
>     > > who contributed to this release, especially those who jumped on the
>     > issues
>     > > found in RC0.
>     > >
>     > > I've prepared RC1 for Apache Hadoop 3.0.0. This release
> incorporates 302
>     > > fixed JIRAs since the previous 3.0.0-beta1 release.
>     > >
>     > > You can find the artifacts here:
>     > >
>     > > http://home.apache.org/~wang/3.0.0-RC1/
>     > >
>     > > I've done the traditional testing of building from the source
> tarball and
>     > > running a Pi job on a single node cluster. I also verified that the
>     > shaded
>     > > jars are not empty.
>     > >
>     > > Found one issue that create-release (probably due to the mvn deploy
>     > change)
>     > > didn't sign the artifacts, but I fixed that by calling mvn one
> more time.
>     > > Available here:
>     > >
>     > > https://repository.apache.org/content/repositories/orgapache
> hadoop-1075/
>     > >
>     > > This release will run the standard 5 days, closing on Dec 13th at
> 12:31pm
>     > > Pacific. My +1 to start.
>     > >
>     > > Best,
>     > > Andrew
>     > >
>     >
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Ajay Kumar <aj...@hortonworks.com>.

+1 (non-binding)
Thanks for driving this, Andrew Wang!!

- downloaded the src tarball and verified md5 checksum 
- built from source with jdk 1.8.0_111-b14
- brought up a pseudo distributed cluster
- did basic file system operations (mkdir, list, put, cat) and
confirmed that everything was working
- Run word count, pi and DFSIOTest 
- run hdfs and yarn, confirmed that the NN, RM web UI worked

Cheers,
Ajay

On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:

    +1 (binding)
    
    - downloaded src tarball, verified md5
    - built from source with jdk1.8.0_112
    - started a pseudo cluster with hdfs and kms
    - sanity checked encryption related operations working
    - sanity checked webui and logs.
    
    -Xiao
    
    On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org> wrote:
    
    > +1 (binding)
    >
    > - downloaded the src tarball and built the source (-Pdist -Pnative)
    > - verified the checksum
    > - brought up a secure pseudo distributed cluster
    > - did some basic file system operations (mkdir, list, put, cat) and
    > confirmed that everything was working
    > - confirmed that the web UI worked
    >
    > Best,
    > Aaron
    >
    > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
    > wrote:
    >
    > > Hi all,
    > >
    > > Let me start, as always, by thanking the efforts of all the contributors
    > > who contributed to this release, especially those who jumped on the
    > issues
    > > found in RC0.
    > >
    > > I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
    > > fixed JIRAs since the previous 3.0.0-beta1 release.
    > >
    > > You can find the artifacts here:
    > >
    > > http://home.apache.org/~wang/3.0.0-RC1/
    > >
    > > I've done the traditional testing of building from the source tarball and
    > > running a Pi job on a single node cluster. I also verified that the
    > shaded
    > > jars are not empty.
    > >
    > > Found one issue that create-release (probably due to the mvn deploy
    > change)
    > > didn't sign the artifacts, but I fixed that by calling mvn one more time.
    > > Available here:
    > >
    > > https://repository.apache.org/content/repositories/orgapachehadoop-1075/
    > >
    > > This release will run the standard 5 days, closing on Dec 13th at 12:31pm
    > > Pacific. My +1 to start.
    > >
    > > Best,
    > > Andrew
    > >
    >
    






---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Ajay Kumar <aj...@hortonworks.com>.

+1 (non-binding)
Thanks for driving this, Andrew Wang!!

- downloaded the src tarball and verified md5 checksum 
- built from source with jdk 1.8.0_111-b14
- brought up a pseudo distributed cluster
- did basic file system operations (mkdir, list, put, cat) and
confirmed that everything was working
- Run word count, pi and DFSIOTest 
- run hdfs and yarn, confirmed that the NN, RM web UI worked

Cheers,
Ajay

On 12/11/17, 9:35 PM, "Xiao Chen" <xi...@cloudera.com> wrote:

    +1 (binding)
    
    - downloaded src tarball, verified md5
    - built from source with jdk1.8.0_112
    - started a pseudo cluster with hdfs and kms
    - sanity checked encryption related operations working
    - sanity checked webui and logs.
    
    -Xiao
    
    On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org> wrote:
    
    > +1 (binding)
    >
    > - downloaded the src tarball and built the source (-Pdist -Pnative)
    > - verified the checksum
    > - brought up a secure pseudo distributed cluster
    > - did some basic file system operations (mkdir, list, put, cat) and
    > confirmed that everything was working
    > - confirmed that the web UI worked
    >
    > Best,
    > Aaron
    >
    > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
    > wrote:
    >
    > > Hi all,
    > >
    > > Let me start, as always, by thanking the efforts of all the contributors
    > > who contributed to this release, especially those who jumped on the
    > issues
    > > found in RC0.
    > >
    > > I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
    > > fixed JIRAs since the previous 3.0.0-beta1 release.
    > >
    > > You can find the artifacts here:
    > >
    > > http://home.apache.org/~wang/3.0.0-RC1/
    > >
    > > I've done the traditional testing of building from the source tarball and
    > > running a Pi job on a single node cluster. I also verified that the
    > shaded
    > > jars are not empty.
    > >
    > > Found one issue that create-release (probably due to the mvn deploy
    > change)
    > > didn't sign the artifacts, but I fixed that by calling mvn one more time.
    > > Available here:
    > >
    > > https://repository.apache.org/content/repositories/orgapachehadoop-1075/
    > >
    > > This release will run the standard 5 days, closing on Dec 13th at 12:31pm
    > > Pacific. My +1 to start.
    > >
    > > Best,
    > > Andrew
    > >
    >
    






---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Xiao Chen <xi...@cloudera.com>.

+1 (binding)

- downloaded src tarball, verified md5
- built from source with jdk1.8.0_112
- started a pseudo cluster with hdfs and kms
- sanity checked encryption related operations working
- sanity checked webui and logs.

-Xiao

On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org> wrote:

> +1 (binding)
>
> - downloaded the src tarball and built the source (-Pdist -Pnative)
> - verified the checksum
> - brought up a secure pseudo distributed cluster
> - did some basic file system operations (mkdir, list, put, cat) and
> confirmed that everything was working
> - confirmed that the web UI worked
>
> Best,
> Aaron
>
> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi all,
> >
> > Let me start, as always, by thanking the efforts of all the contributors
> > who contributed to this release, especially those who jumped on the
> issues
> > found in RC0.
> >
> > I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> > fixed JIRAs since the previous 3.0.0-beta1 release.
> >
> > You can find the artifacts here:
> >
> > http://home.apache.org/~wang/3.0.0-RC1/
> >
> > I've done the traditional testing of building from the source tarball and
> > running a Pi job on a single node cluster. I also verified that the
> shaded
> > jars are not empty.
> >
> > Found one issue that create-release (probably due to the mvn deploy
> change)
> > didn't sign the artifacts, but I fixed that by calling mvn one more time.
> > Available here:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> >
> > This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> > Pacific. My +1 to start.
> >
> > Best,
> > Andrew
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Xiao Chen <xi...@cloudera.com>.

+1 (binding)

- downloaded src tarball, verified md5
- built from source with jdk1.8.0_112
- started a pseudo cluster with hdfs and kms
- sanity checked encryption related operations working
- sanity checked webui and logs.

-Xiao

On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org> wrote:

> +1 (binding)
>
> - downloaded the src tarball and built the source (-Pdist -Pnative)
> - verified the checksum
> - brought up a secure pseudo distributed cluster
> - did some basic file system operations (mkdir, list, put, cat) and
> confirmed that everything was working
> - confirmed that the web UI worked
>
> Best,
> Aaron
>
> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi all,
> >
> > Let me start, as always, by thanking the efforts of all the contributors
> > who contributed to this release, especially those who jumped on the
> issues
> > found in RC0.
> >
> > I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> > fixed JIRAs since the previous 3.0.0-beta1 release.
> >
> > You can find the artifacts here:
> >
> > http://home.apache.org/~wang/3.0.0-RC1/
> >
> > I've done the traditional testing of building from the source tarball and
> > running a Pi job on a single node cluster. I also verified that the
> shaded
> > jars are not empty.
> >
> > Found one issue that create-release (probably due to the mvn deploy
> change)
> > didn't sign the artifacts, but I fixed that by calling mvn one more time.
> > Available here:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> >
> > This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> > Pacific. My +1 to start.
> >
> > Best,
> > Andrew
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Xiao Chen <xi...@cloudera.com>.

+1 (binding)

- downloaded src tarball, verified md5
- built from source with jdk1.8.0_112
- started a pseudo cluster with hdfs and kms
- sanity checked encryption related operations working
- sanity checked webui and logs.

-Xiao

On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org> wrote:

> +1 (binding)
>
> - downloaded the src tarball and built the source (-Pdist -Pnative)
> - verified the checksum
> - brought up a secure pseudo distributed cluster
> - did some basic file system operations (mkdir, list, put, cat) and
> confirmed that everything was working
> - confirmed that the web UI worked
>
> Best,
> Aaron
>
> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi all,
> >
> > Let me start, as always, by thanking the efforts of all the contributors
> > who contributed to this release, especially those who jumped on the
> issues
> > found in RC0.
> >
> > I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> > fixed JIRAs since the previous 3.0.0-beta1 release.
> >
> > You can find the artifacts here:
> >
> > http://home.apache.org/~wang/3.0.0-RC1/
> >
> > I've done the traditional testing of building from the source tarball and
> > running a Pi job on a single node cluster. I also verified that the
> shaded
> > jars are not empty.
> >
> > Found one issue that create-release (probably due to the mvn deploy
> change)
> > didn't sign the artifacts, but I fixed that by calling mvn one more time.
> > Available here:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> >
> > This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> > Pacific. My +1 to start.
> >
> > Best,
> > Andrew
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Xiao Chen <xi...@cloudera.com>.

+1 (binding)

- downloaded src tarball, verified md5
- built from source with jdk1.8.0_112
- started a pseudo cluster with hdfs and kms
- sanity checked encryption related operations working
- sanity checked webui and logs.

-Xiao

On Mon, Dec 11, 2017 at 6:10 PM, Aaron T. Myers <at...@apache.org> wrote:

> +1 (binding)
>
> - downloaded the src tarball and built the source (-Pdist -Pnative)
> - verified the checksum
> - brought up a secure pseudo distributed cluster
> - did some basic file system operations (mkdir, list, put, cat) and
> confirmed that everything was working
> - confirmed that the web UI worked
>
> Best,
> Aaron
>
> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi all,
> >
> > Let me start, as always, by thanking the efforts of all the contributors
> > who contributed to this release, especially those who jumped on the
> issues
> > found in RC0.
> >
> > I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> > fixed JIRAs since the previous 3.0.0-beta1 release.
> >
> > You can find the artifacts here:
> >
> > http://home.apache.org/~wang/3.0.0-RC1/
> >
> > I've done the traditional testing of building from the source tarball and
> > running a Pi job on a single node cluster. I also verified that the
> shaded
> > jars are not empty.
> >
> > Found one issue that create-release (probably due to the mvn deploy
> change)
> > didn't sign the artifacts, but I fixed that by calling mvn one more time.
> > Available here:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> >
> > This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> > Pacific. My +1 to start.
> >
> > Best,
> > Andrew
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by "Aaron T. Myers" <at...@apache.org>.

+1 (binding)

- downloaded the src tarball and built the source (-Pdist -Pnative)
- verified the checksum
- brought up a secure pseudo distributed cluster
- did some basic file system operations (mkdir, list, put, cat) and
confirmed that everything was working
- confirmed that the web UI worked

Best,
Aaron

On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Good point on the mutability. Release tags are immutable, RCs are not.

On Mon, Dec 11, 2017 at 1:39 PM, Sangjin Lee <sj...@apache.org> wrote:

> Thanks Andrew. For the record, the commit id would be
> c25427ceca461ee979d30edd7a4b0f50718e6533. I mention that for completeness
> because of the mutability of tags.
>
> On Mon, Dec 11, 2017 at 10:31 AM, Andrew Wang <an...@cloudera.com>
> wrote:
>
>> Sorry, forgot to push the tag. It's up there now.
>>
>> On Sun, Dec 10, 2017 at 8:31 PM, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org> wrote:
>>
>>> I couldn't find the release tag for RC1 either - is it just me or has
>>> the release-process changed?
>>>
>>> +Vinod
>>>
>>> > On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
>>> >
>>> > Hi Andrew,
>>> >
>>> > Thanks much for your effort! Just to be clear, could you please state
>>> the
>>> > git commit id of the RC1 we're voting for?
>>> >
>>> > Sangjin
>>> >
>>> > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <andrew.wang@cloudera.com
>>> >
>>> > wrote:
>>> >
>>> >> Hi all,
>>> >>
>>> >> Let me start, as always, by thanking the efforts of all the
>>> contributors
>>> >> who contributed to this release, especially those who jumped on the
>>> issues
>>> >> found in RC0.
>>> >>
>>> >> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates
>>> 302
>>> >> fixed JIRAs since the previous 3.0.0-beta1 release.
>>> >>
>>> >> You can find the artifacts here:
>>> >>
>>> >> http://home.apache.org/~wang/3.0.0-RC1/
>>> >>
>>> >> I've done the traditional testing of building from the source tarball
>>> and
>>> >> running a Pi job on a single node cluster. I also verified that the
>>> shaded
>>> >> jars are not empty.
>>> >>
>>> >> Found one issue that create-release (probably due to the mvn deploy
>>> change)
>>> >> didn't sign the artifacts, but I fixed that by calling mvn one more
>>> time.
>>> >> Available here:
>>> >>
>>> >> https://repository.apache.org/content/repositories/orgapache
>>> hadoop-1075/
>>> >>
>>> >> This release will run the standard 5 days, closing on Dec 13th at
>>> 12:31pm
>>> >> Pacific. My +1 to start.
>>> >>
>>> >> Best,
>>> >> Andrew
>>> >>
>>>
>>>
>>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Good point on the mutability. Release tags are immutable, RCs are not.

On Mon, Dec 11, 2017 at 1:39 PM, Sangjin Lee <sj...@apache.org> wrote:

> Thanks Andrew. For the record, the commit id would be
> c25427ceca461ee979d30edd7a4b0f50718e6533. I mention that for completeness
> because of the mutability of tags.
>
> On Mon, Dec 11, 2017 at 10:31 AM, Andrew Wang <an...@cloudera.com>
> wrote:
>
>> Sorry, forgot to push the tag. It's up there now.
>>
>> On Sun, Dec 10, 2017 at 8:31 PM, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org> wrote:
>>
>>> I couldn't find the release tag for RC1 either - is it just me or has
>>> the release-process changed?
>>>
>>> +Vinod
>>>
>>> > On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
>>> >
>>> > Hi Andrew,
>>> >
>>> > Thanks much for your effort! Just to be clear, could you please state
>>> the
>>> > git commit id of the RC1 we're voting for?
>>> >
>>> > Sangjin
>>> >
>>> > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <andrew.wang@cloudera.com
>>> >
>>> > wrote:
>>> >
>>> >> Hi all,
>>> >>
>>> >> Let me start, as always, by thanking the efforts of all the
>>> contributors
>>> >> who contributed to this release, especially those who jumped on the
>>> issues
>>> >> found in RC0.
>>> >>
>>> >> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates
>>> 302
>>> >> fixed JIRAs since the previous 3.0.0-beta1 release.
>>> >>
>>> >> You can find the artifacts here:
>>> >>
>>> >> http://home.apache.org/~wang/3.0.0-RC1/
>>> >>
>>> >> I've done the traditional testing of building from the source tarball
>>> and
>>> >> running a Pi job on a single node cluster. I also verified that the
>>> shaded
>>> >> jars are not empty.
>>> >>
>>> >> Found one issue that create-release (probably due to the mvn deploy
>>> change)
>>> >> didn't sign the artifacts, but I fixed that by calling mvn one more
>>> time.
>>> >> Available here:
>>> >>
>>> >> https://repository.apache.org/content/repositories/orgapache
>>> hadoop-1075/
>>> >>
>>> >> This release will run the standard 5 days, closing on Dec 13th at
>>> 12:31pm
>>> >> Pacific. My +1 to start.
>>> >>
>>> >> Best,
>>> >> Andrew
>>> >>
>>>
>>>
>>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Good point on the mutability. Release tags are immutable, RCs are not.

On Mon, Dec 11, 2017 at 1:39 PM, Sangjin Lee <sj...@apache.org> wrote:

> Thanks Andrew. For the record, the commit id would be
> c25427ceca461ee979d30edd7a4b0f50718e6533. I mention that for completeness
> because of the mutability of tags.
>
> On Mon, Dec 11, 2017 at 10:31 AM, Andrew Wang <an...@cloudera.com>
> wrote:
>
>> Sorry, forgot to push the tag. It's up there now.
>>
>> On Sun, Dec 10, 2017 at 8:31 PM, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org> wrote:
>>
>>> I couldn't find the release tag for RC1 either - is it just me or has
>>> the release-process changed?
>>>
>>> +Vinod
>>>
>>> > On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
>>> >
>>> > Hi Andrew,
>>> >
>>> > Thanks much for your effort! Just to be clear, could you please state
>>> the
>>> > git commit id of the RC1 we're voting for?
>>> >
>>> > Sangjin
>>> >
>>> > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <andrew.wang@cloudera.com
>>> >
>>> > wrote:
>>> >
>>> >> Hi all,
>>> >>
>>> >> Let me start, as always, by thanking the efforts of all the
>>> contributors
>>> >> who contributed to this release, especially those who jumped on the
>>> issues
>>> >> found in RC0.
>>> >>
>>> >> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates
>>> 302
>>> >> fixed JIRAs since the previous 3.0.0-beta1 release.
>>> >>
>>> >> You can find the artifacts here:
>>> >>
>>> >> http://home.apache.org/~wang/3.0.0-RC1/
>>> >>
>>> >> I've done the traditional testing of building from the source tarball
>>> and
>>> >> running a Pi job on a single node cluster. I also verified that the
>>> shaded
>>> >> jars are not empty.
>>> >>
>>> >> Found one issue that create-release (probably due to the mvn deploy
>>> change)
>>> >> didn't sign the artifacts, but I fixed that by calling mvn one more
>>> time.
>>> >> Available here:
>>> >>
>>> >> https://repository.apache.org/content/repositories/orgapache
>>> hadoop-1075/
>>> >>
>>> >> This release will run the standard 5 days, closing on Dec 13th at
>>> 12:31pm
>>> >> Pacific. My +1 to start.
>>> >>
>>> >> Best,
>>> >> Andrew
>>> >>
>>>
>>>
>>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Good point on the mutability. Release tags are immutable, RCs are not.

On Mon, Dec 11, 2017 at 1:39 PM, Sangjin Lee <sj...@apache.org> wrote:

> Thanks Andrew. For the record, the commit id would be
> c25427ceca461ee979d30edd7a4b0f50718e6533. I mention that for completeness
> because of the mutability of tags.
>
> On Mon, Dec 11, 2017 at 10:31 AM, Andrew Wang <an...@cloudera.com>
> wrote:
>
>> Sorry, forgot to push the tag. It's up there now.
>>
>> On Sun, Dec 10, 2017 at 8:31 PM, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org> wrote:
>>
>>> I couldn't find the release tag for RC1 either - is it just me or has
>>> the release-process changed?
>>>
>>> +Vinod
>>>
>>> > On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
>>> >
>>> > Hi Andrew,
>>> >
>>> > Thanks much for your effort! Just to be clear, could you please state
>>> the
>>> > git commit id of the RC1 we're voting for?
>>> >
>>> > Sangjin
>>> >
>>> > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <andrew.wang@cloudera.com
>>> >
>>> > wrote:
>>> >
>>> >> Hi all,
>>> >>
>>> >> Let me start, as always, by thanking the efforts of all the
>>> contributors
>>> >> who contributed to this release, especially those who jumped on the
>>> issues
>>> >> found in RC0.
>>> >>
>>> >> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates
>>> 302
>>> >> fixed JIRAs since the previous 3.0.0-beta1 release.
>>> >>
>>> >> You can find the artifacts here:
>>> >>
>>> >> http://home.apache.org/~wang/3.0.0-RC1/
>>> >>
>>> >> I've done the traditional testing of building from the source tarball
>>> and
>>> >> running a Pi job on a single node cluster. I also verified that the
>>> shaded
>>> >> jars are not empty.
>>> >>
>>> >> Found one issue that create-release (probably due to the mvn deploy
>>> change)
>>> >> didn't sign the artifacts, but I fixed that by calling mvn one more
>>> time.
>>> >> Available here:
>>> >>
>>> >> https://repository.apache.org/content/repositories/orgapache
>>> hadoop-1075/
>>> >>
>>> >> This release will run the standard 5 days, closing on Dec 13th at
>>> 12:31pm
>>> >> Pacific. My +1 to start.
>>> >>
>>> >> Best,
>>> >> Andrew
>>> >>
>>>
>>>
>>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sangjin Lee <sj...@apache.org>.

Thanks Andrew. For the record, the commit id would be
c25427ceca461ee979d30edd7a4b0f50718e6533. I mention that for completeness
because of the mutability of tags.

On Mon, Dec 11, 2017 at 10:31 AM, Andrew Wang <an...@cloudera.com>
wrote:

> Sorry, forgot to push the tag. It's up there now.
>
> On Sun, Dec 10, 2017 at 8:31 PM, Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
>> I couldn't find the release tag for RC1 either - is it just me or has the
>> release-process changed?
>>
>> +Vinod
>>
>> > On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
>> >
>> > Hi Andrew,
>> >
>> > Thanks much for your effort! Just to be clear, could you please state
>> the
>> > git commit id of the RC1 we're voting for?
>> >
>> > Sangjin
>> >
>> > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
>> > wrote:
>> >
>> >> Hi all,
>> >>
>> >> Let me start, as always, by thanking the efforts of all the
>> contributors
>> >> who contributed to this release, especially those who jumped on the
>> issues
>> >> found in RC0.
>> >>
>> >> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates
>> 302
>> >> fixed JIRAs since the previous 3.0.0-beta1 release.
>> >>
>> >> You can find the artifacts here:
>> >>
>> >> http://home.apache.org/~wang/3.0.0-RC1/
>> >>
>> >> I've done the traditional testing of building from the source tarball
>> and
>> >> running a Pi job on a single node cluster. I also verified that the
>> shaded
>> >> jars are not empty.
>> >>
>> >> Found one issue that create-release (probably due to the mvn deploy
>> change)
>> >> didn't sign the artifacts, but I fixed that by calling mvn one more
>> time.
>> >> Available here:
>> >>
>> >> https://repository.apache.org/content/repositories/orgapache
>> hadoop-1075/
>> >>
>> >> This release will run the standard 5 days, closing on Dec 13th at
>> 12:31pm
>> >> Pacific. My +1 to start.
>> >>
>> >> Best,
>> >> Andrew
>> >>
>>
>>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sangjin Lee <sj...@apache.org>.

Thanks Andrew. For the record, the commit id would be
c25427ceca461ee979d30edd7a4b0f50718e6533. I mention that for completeness
because of the mutability of tags.

On Mon, Dec 11, 2017 at 10:31 AM, Andrew Wang <an...@cloudera.com>
wrote:

> Sorry, forgot to push the tag. It's up there now.
>
> On Sun, Dec 10, 2017 at 8:31 PM, Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
>> I couldn't find the release tag for RC1 either - is it just me or has the
>> release-process changed?
>>
>> +Vinod
>>
>> > On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
>> >
>> > Hi Andrew,
>> >
>> > Thanks much for your effort! Just to be clear, could you please state
>> the
>> > git commit id of the RC1 we're voting for?
>> >
>> > Sangjin
>> >
>> > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
>> > wrote:
>> >
>> >> Hi all,
>> >>
>> >> Let me start, as always, by thanking the efforts of all the
>> contributors
>> >> who contributed to this release, especially those who jumped on the
>> issues
>> >> found in RC0.
>> >>
>> >> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates
>> 302
>> >> fixed JIRAs since the previous 3.0.0-beta1 release.
>> >>
>> >> You can find the artifacts here:
>> >>
>> >> http://home.apache.org/~wang/3.0.0-RC1/
>> >>
>> >> I've done the traditional testing of building from the source tarball
>> and
>> >> running a Pi job on a single node cluster. I also verified that the
>> shaded
>> >> jars are not empty.
>> >>
>> >> Found one issue that create-release (probably due to the mvn deploy
>> change)
>> >> didn't sign the artifacts, but I fixed that by calling mvn one more
>> time.
>> >> Available here:
>> >>
>> >> https://repository.apache.org/content/repositories/orgapache
>> hadoop-1075/
>> >>
>> >> This release will run the standard 5 days, closing on Dec 13th at
>> 12:31pm
>> >> Pacific. My +1 to start.
>> >>
>> >> Best,
>> >> Andrew
>> >>
>>
>>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sangjin Lee <sj...@apache.org>.

Thanks Andrew. For the record, the commit id would be
c25427ceca461ee979d30edd7a4b0f50718e6533. I mention that for completeness
because of the mutability of tags.

On Mon, Dec 11, 2017 at 10:31 AM, Andrew Wang <an...@cloudera.com>
wrote:

> Sorry, forgot to push the tag. It's up there now.
>
> On Sun, Dec 10, 2017 at 8:31 PM, Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
>> I couldn't find the release tag for RC1 either - is it just me or has the
>> release-process changed?
>>
>> +Vinod
>>
>> > On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
>> >
>> > Hi Andrew,
>> >
>> > Thanks much for your effort! Just to be clear, could you please state
>> the
>> > git commit id of the RC1 we're voting for?
>> >
>> > Sangjin
>> >
>> > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
>> > wrote:
>> >
>> >> Hi all,
>> >>
>> >> Let me start, as always, by thanking the efforts of all the
>> contributors
>> >> who contributed to this release, especially those who jumped on the
>> issues
>> >> found in RC0.
>> >>
>> >> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates
>> 302
>> >> fixed JIRAs since the previous 3.0.0-beta1 release.
>> >>
>> >> You can find the artifacts here:
>> >>
>> >> http://home.apache.org/~wang/3.0.0-RC1/
>> >>
>> >> I've done the traditional testing of building from the source tarball
>> and
>> >> running a Pi job on a single node cluster. I also verified that the
>> shaded
>> >> jars are not empty.
>> >>
>> >> Found one issue that create-release (probably due to the mvn deploy
>> change)
>> >> didn't sign the artifacts, but I fixed that by calling mvn one more
>> time.
>> >> Available here:
>> >>
>> >> https://repository.apache.org/content/repositories/orgapache
>> hadoop-1075/
>> >>
>> >> This release will run the standard 5 days, closing on Dec 13th at
>> 12:31pm
>> >> Pacific. My +1 to start.
>> >>
>> >> Best,
>> >> Andrew
>> >>
>>
>>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sangjin Lee <sj...@apache.org>.

Thanks Andrew. For the record, the commit id would be
c25427ceca461ee979d30edd7a4b0f50718e6533. I mention that for completeness
because of the mutability of tags.

On Mon, Dec 11, 2017 at 10:31 AM, Andrew Wang <an...@cloudera.com>
wrote:

> Sorry, forgot to push the tag. It's up there now.
>
> On Sun, Dec 10, 2017 at 8:31 PM, Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
>> I couldn't find the release tag for RC1 either - is it just me or has the
>> release-process changed?
>>
>> +Vinod
>>
>> > On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
>> >
>> > Hi Andrew,
>> >
>> > Thanks much for your effort! Just to be clear, could you please state
>> the
>> > git commit id of the RC1 we're voting for?
>> >
>> > Sangjin
>> >
>> > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
>> > wrote:
>> >
>> >> Hi all,
>> >>
>> >> Let me start, as always, by thanking the efforts of all the
>> contributors
>> >> who contributed to this release, especially those who jumped on the
>> issues
>> >> found in RC0.
>> >>
>> >> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates
>> 302
>> >> fixed JIRAs since the previous 3.0.0-beta1 release.
>> >>
>> >> You can find the artifacts here:
>> >>
>> >> http://home.apache.org/~wang/3.0.0-RC1/
>> >>
>> >> I've done the traditional testing of building from the source tarball
>> and
>> >> running a Pi job on a single node cluster. I also verified that the
>> shaded
>> >> jars are not empty.
>> >>
>> >> Found one issue that create-release (probably due to the mvn deploy
>> change)
>> >> didn't sign the artifacts, but I fixed that by calling mvn one more
>> time.
>> >> Available here:
>> >>
>> >> https://repository.apache.org/content/repositories/orgapache
>> hadoop-1075/
>> >>
>> >> This release will run the standard 5 days, closing on Dec 13th at
>> 12:31pm
>> >> Pacific. My +1 to start.
>> >>
>> >> Best,
>> >> Andrew
>> >>
>>
>>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Sorry, forgot to push the tag. It's up there now.

On Sun, Dec 10, 2017 at 8:31 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:

> I couldn't find the release tag for RC1 either - is it just me or has the
> release-process changed?
>
> +Vinod
>
> > On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
> >
> > Hi Andrew,
> >
> > Thanks much for your effort! Just to be clear, could you please state the
> > git commit id of the RC1 we're voting for?
> >
> > Sangjin
> >
> > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
> > wrote:
> >
> >> Hi all,
> >>
> >> Let me start, as always, by thanking the efforts of all the contributors
> >> who contributed to this release, especially those who jumped on the
> issues
> >> found in RC0.
> >>
> >> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> >> fixed JIRAs since the previous 3.0.0-beta1 release.
> >>
> >> You can find the artifacts here:
> >>
> >> http://home.apache.org/~wang/3.0.0-RC1/
> >>
> >> I've done the traditional testing of building from the source tarball
> and
> >> running a Pi job on a single node cluster. I also verified that the
> shaded
> >> jars are not empty.
> >>
> >> Found one issue that create-release (probably due to the mvn deploy
> change)
> >> didn't sign the artifacts, but I fixed that by calling mvn one more
> time.
> >> Available here:
> >>
> >> https://repository.apache.org/content/repositories/
> orgapachehadoop-1075/
> >>
> >> This release will run the standard 5 days, closing on Dec 13th at
> 12:31pm
> >> Pacific. My +1 to start.
> >>
> >> Best,
> >> Andrew
> >>
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Sorry, forgot to push the tag. It's up there now.

On Sun, Dec 10, 2017 at 8:31 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:

> I couldn't find the release tag for RC1 either - is it just me or has the
> release-process changed?
>
> +Vinod
>
> > On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
> >
> > Hi Andrew,
> >
> > Thanks much for your effort! Just to be clear, could you please state the
> > git commit id of the RC1 we're voting for?
> >
> > Sangjin
> >
> > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
> > wrote:
> >
> >> Hi all,
> >>
> >> Let me start, as always, by thanking the efforts of all the contributors
> >> who contributed to this release, especially those who jumped on the
> issues
> >> found in RC0.
> >>
> >> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> >> fixed JIRAs since the previous 3.0.0-beta1 release.
> >>
> >> You can find the artifacts here:
> >>
> >> http://home.apache.org/~wang/3.0.0-RC1/
> >>
> >> I've done the traditional testing of building from the source tarball
> and
> >> running a Pi job on a single node cluster. I also verified that the
> shaded
> >> jars are not empty.
> >>
> >> Found one issue that create-release (probably due to the mvn deploy
> change)
> >> didn't sign the artifacts, but I fixed that by calling mvn one more
> time.
> >> Available here:
> >>
> >> https://repository.apache.org/content/repositories/
> orgapachehadoop-1075/
> >>
> >> This release will run the standard 5 days, closing on Dec 13th at
> 12:31pm
> >> Pacific. My +1 to start.
> >>
> >> Best,
> >> Andrew
> >>
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Sorry, forgot to push the tag. It's up there now.

On Sun, Dec 10, 2017 at 8:31 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:

> I couldn't find the release tag for RC1 either - is it just me or has the
> release-process changed?
>
> +Vinod
>
> > On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
> >
> > Hi Andrew,
> >
> > Thanks much for your effort! Just to be clear, could you please state the
> > git commit id of the RC1 we're voting for?
> >
> > Sangjin
> >
> > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
> > wrote:
> >
> >> Hi all,
> >>
> >> Let me start, as always, by thanking the efforts of all the contributors
> >> who contributed to this release, especially those who jumped on the
> issues
> >> found in RC0.
> >>
> >> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> >> fixed JIRAs since the previous 3.0.0-beta1 release.
> >>
> >> You can find the artifacts here:
> >>
> >> http://home.apache.org/~wang/3.0.0-RC1/
> >>
> >> I've done the traditional testing of building from the source tarball
> and
> >> running a Pi job on a single node cluster. I also verified that the
> shaded
> >> jars are not empty.
> >>
> >> Found one issue that create-release (probably due to the mvn deploy
> change)
> >> didn't sign the artifacts, but I fixed that by calling mvn one more
> time.
> >> Available here:
> >>
> >> https://repository.apache.org/content/repositories/
> orgapachehadoop-1075/
> >>
> >> This release will run the standard 5 days, closing on Dec 13th at
> 12:31pm
> >> Pacific. My +1 to start.
> >>
> >> Best,
> >> Andrew
> >>
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Andrew Wang <an...@cloudera.com>.

Sorry, forgot to push the tag. It's up there now.

On Sun, Dec 10, 2017 at 8:31 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:

> I couldn't find the release tag for RC1 either - is it just me or has the
> release-process changed?
>
> +Vinod
>
> > On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
> >
> > Hi Andrew,
> >
> > Thanks much for your effort! Just to be clear, could you please state the
> > git commit id of the RC1 we're voting for?
> >
> > Sangjin
> >
> > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
> > wrote:
> >
> >> Hi all,
> >>
> >> Let me start, as always, by thanking the efforts of all the contributors
> >> who contributed to this release, especially those who jumped on the
> issues
> >> found in RC0.
> >>
> >> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> >> fixed JIRAs since the previous 3.0.0-beta1 release.
> >>
> >> You can find the artifacts here:
> >>
> >> http://home.apache.org/~wang/3.0.0-RC1/
> >>
> >> I've done the traditional testing of building from the source tarball
> and
> >> running a Pi job on a single node cluster. I also verified that the
> shaded
> >> jars are not empty.
> >>
> >> Found one issue that create-release (probably due to the mvn deploy
> change)
> >> didn't sign the artifacts, but I fixed that by calling mvn one more
> time.
> >> Available here:
> >>
> >> https://repository.apache.org/content/repositories/
> orgapachehadoop-1075/
> >>
> >> This release will run the standard 5 days, closing on Dec 13th at
> 12:31pm
> >> Pacific. My +1 to start.
> >>
> >> Best,
> >> Andrew
> >>
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

I couldn't find the release tag for RC1 either - is it just me or has the release-process changed?

+Vinod

> On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
> 
> Hi Andrew,
> 
> Thanks much for your effort! Just to be clear, could you please state the
> git commit id of the RC1 we're voting for?
> 
> Sangjin
> 
> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
> wrote:
> 
>> Hi all,
>> 
>> Let me start, as always, by thanking the efforts of all the contributors
>> who contributed to this release, especially those who jumped on the issues
>> found in RC0.
>> 
>> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
>> fixed JIRAs since the previous 3.0.0-beta1 release.
>> 
>> You can find the artifacts here:
>> 
>> http://home.apache.org/~wang/3.0.0-RC1/
>> 
>> I've done the traditional testing of building from the source tarball and
>> running a Pi job on a single node cluster. I also verified that the shaded
>> jars are not empty.
>> 
>> Found one issue that create-release (probably due to the mvn deploy change)
>> didn't sign the artifacts, but I fixed that by calling mvn one more time.
>> Available here:
>> 
>> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>> 
>> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
>> Pacific. My +1 to start.
>> 
>> Best,
>> Andrew
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

I couldn't find the release tag for RC1 either - is it just me or has the release-process changed?

+Vinod

> On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
> 
> Hi Andrew,
> 
> Thanks much for your effort! Just to be clear, could you please state the
> git commit id of the RC1 we're voting for?
> 
> Sangjin
> 
> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
> wrote:
> 
>> Hi all,
>> 
>> Let me start, as always, by thanking the efforts of all the contributors
>> who contributed to this release, especially those who jumped on the issues
>> found in RC0.
>> 
>> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
>> fixed JIRAs since the previous 3.0.0-beta1 release.
>> 
>> You can find the artifacts here:
>> 
>> http://home.apache.org/~wang/3.0.0-RC1/
>> 
>> I've done the traditional testing of building from the source tarball and
>> running a Pi job on a single node cluster. I also verified that the shaded
>> jars are not empty.
>> 
>> Found one issue that create-release (probably due to the mvn deploy change)
>> didn't sign the artifacts, but I fixed that by calling mvn one more time.
>> Available here:
>> 
>> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>> 
>> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
>> Pacific. My +1 to start.
>> 
>> Best,
>> Andrew
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

I couldn't find the release tag for RC1 either - is it just me or has the release-process changed?

+Vinod

> On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
> 
> Hi Andrew,
> 
> Thanks much for your effort! Just to be clear, could you please state the
> git commit id of the RC1 we're voting for?
> 
> Sangjin
> 
> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
> wrote:
> 
>> Hi all,
>> 
>> Let me start, as always, by thanking the efforts of all the contributors
>> who contributed to this release, especially those who jumped on the issues
>> found in RC0.
>> 
>> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
>> fixed JIRAs since the previous 3.0.0-beta1 release.
>> 
>> You can find the artifacts here:
>> 
>> http://home.apache.org/~wang/3.0.0-RC1/
>> 
>> I've done the traditional testing of building from the source tarball and
>> running a Pi job on a single node cluster. I also verified that the shaded
>> jars are not empty.
>> 
>> Found one issue that create-release (probably due to the mvn deploy change)
>> didn't sign the artifacts, but I fixed that by calling mvn one more time.
>> Available here:
>> 
>> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>> 
>> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
>> Pacific. My +1 to start.
>> 
>> Best,
>> Andrew
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

I couldn't find the release tag for RC1 either - is it just me or has the release-process changed?

+Vinod

> On Dec 10, 2017, at 4:31 PM, Sangjin Lee <sj...@apache.org> wrote:
> 
> Hi Andrew,
> 
> Thanks much for your effort! Just to be clear, could you please state the
> git commit id of the RC1 we're voting for?
> 
> Sangjin
> 
> On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
> wrote:
> 
>> Hi all,
>> 
>> Let me start, as always, by thanking the efforts of all the contributors
>> who contributed to this release, especially those who jumped on the issues
>> found in RC0.
>> 
>> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
>> fixed JIRAs since the previous 3.0.0-beta1 release.
>> 
>> You can find the artifacts here:
>> 
>> http://home.apache.org/~wang/3.0.0-RC1/
>> 
>> I've done the traditional testing of building from the source tarball and
>> running a Pi job on a single node cluster. I also verified that the shaded
>> jars are not empty.
>> 
>> Found one issue that create-release (probably due to the mvn deploy change)
>> didn't sign the artifacts, but I fixed that by calling mvn one more time.
>> Available here:
>> 
>> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>> 
>> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
>> Pacific. My +1 to start.
>> 
>> Best,
>> Andrew
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sangjin Lee <sj...@apache.org>.

Hi Andrew,

Thanks much for your effort! Just to be clear, could you please state the
git commit id of the RC1 we're voting for?

Sangjin

On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by "Elek, Marton" <hd...@anzix.net>.

+1 (non-binding)

  * built from the source tarball (archlinux) / verified signature
  * Deployed to a kubernetes cluster (10/10 datanode/nodemanager pods)
  * Enabled ec on hdfs directory (hdfs cli)
  * Started example yarn jobs (pi/terragen)
  * checked yarn ui/ui2

Thanks for all the efforts.

Marton


On 12/08/2017 09:31 PM, Andrew Wang wrote:
> Hi all,
> 
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
> 
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
> 
> You can find the artifacts here:
> 
> http://home.apache.org/~wang/3.0.0-RC1/
> 
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
> 
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
> 
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> 
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
> 
> Best,
> Andrew
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by "Aaron T. Myers" <at...@apache.org>.

+1 (binding)

- downloaded the src tarball and built the source (-Pdist -Pnative)
- verified the checksum
- brought up a secure pseudo distributed cluster
- did some basic file system operations (mkdir, list, put, cat) and
confirmed that everything was working
- confirmed that the web UI worked

Best,
Aaron

On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Wangda Tan <wh...@gmail.com>.

Thanks Andrew for driving this.

+1 (Binding)

Ran SLS + CS's Perf unit test and saw similar performance compared to
trunk.

Compiled from source, deployed single node cluster and ran several jobs.

Best,
Wangda




On Wed, Dec 13, 2017 at 7:29 AM, Sunil G <su...@apache.org> wrote:

> +1 (binding)
>
> Thanks Andrew Wang for driving this effort and also thanks to all others
> who helped in this release. Kudos!!!
>
> I tested this RC by building it from source. I met with couple of issues
> (not blocker) HADOOP-15116 and YARN-7650. This could be tracked separately.
>
>
>    - Ran many MR apps and verified both new YARN UI and old RM UI.
>    - Tested below feature sanity and got results as per the behavior
>       - Application priority (verified CLI/REST/UI etc)
>       - Application timeout
>       - Intra Queue preemption with priority based
>       - Inter Queue preemption
>    - Tested basic NodeLabel scenarios.
>       - Added couple of labels to few of nodes and behavior is coming
>       correct.
>       - Verified old UI  and new YARN UI for labels.
>       - Submitted apps to labelled cluster and it works fine.
>       - Also performed few cli commands related to nodelabel.
>    - Test basic HA cases and seems correct. However I got one issue.
>    Raised HADOOP-15116 as its not a blocker.
>    - Also tested new YARN UI . All pages are getting loaded correctly.
>    (User must enable CORS to access NodeManager pages)
>    - *Performance test*: I ran a tight loop perf test on CS
>    TestCapacitySchedulerPerf#testUserLimitThroughputForTwoResources.
>    Results are a bit off w.r.t 2.8  (~5% less). I will open a ticket and
>    investigate by doing more tests to see if its to be addressed or not.
>
>
> - Sunil G
>
>
>
> On Sat, Dec 9, 2017 at 2:01 AM Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi all,
> >
> > Let me start, as always, by thanking the efforts of all the contributors
> > who contributed to this release, especially those who jumped on the
> issues
> > found in RC0.
> >
> > I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> > fixed JIRAs since the previous 3.0.0-beta1 release.
> >
> > You can find the artifacts here:
> >
> > http://home.apache.org/~wang/3.0.0-RC1/
> >
> > I've done the traditional testing of building from the source tarball and
> > running a Pi job on a single node cluster. I also verified that the
> shaded
> > jars are not empty.
> >
> > Found one issue that create-release (probably due to the mvn deploy
> change)
> > didn't sign the artifacts, but I fixed that by calling mvn one more time.
> > Available here:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> >
> > This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> > Pacific. My +1 to start.
> >
> > Best,
> > Andrew
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Wangda Tan <wh...@gmail.com>.

Thanks Andrew for driving this.

+1 (Binding)

Ran SLS + CS's Perf unit test and saw similar performance compared to
trunk.

Compiled from source, deployed single node cluster and ran several jobs.

Best,
Wangda




On Wed, Dec 13, 2017 at 7:29 AM, Sunil G <su...@apache.org> wrote:

> +1 (binding)
>
> Thanks Andrew Wang for driving this effort and also thanks to all others
> who helped in this release. Kudos!!!
>
> I tested this RC by building it from source. I met with couple of issues
> (not blocker) HADOOP-15116 and YARN-7650. This could be tracked separately.
>
>
>    - Ran many MR apps and verified both new YARN UI and old RM UI.
>    - Tested below feature sanity and got results as per the behavior
>       - Application priority (verified CLI/REST/UI etc)
>       - Application timeout
>       - Intra Queue preemption with priority based
>       - Inter Queue preemption
>    - Tested basic NodeLabel scenarios.
>       - Added couple of labels to few of nodes and behavior is coming
>       correct.
>       - Verified old UI  and new YARN UI for labels.
>       - Submitted apps to labelled cluster and it works fine.
>       - Also performed few cli commands related to nodelabel.
>    - Test basic HA cases and seems correct. However I got one issue.
>    Raised HADOOP-15116 as its not a blocker.
>    - Also tested new YARN UI . All pages are getting loaded correctly.
>    (User must enable CORS to access NodeManager pages)
>    - *Performance test*: I ran a tight loop perf test on CS
>    TestCapacitySchedulerPerf#testUserLimitThroughputForTwoResources.
>    Results are a bit off w.r.t 2.8  (~5% less). I will open a ticket and
>    investigate by doing more tests to see if its to be addressed or not.
>
>
> - Sunil G
>
>
>
> On Sat, Dec 9, 2017 at 2:01 AM Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi all,
> >
> > Let me start, as always, by thanking the efforts of all the contributors
> > who contributed to this release, especially those who jumped on the
> issues
> > found in RC0.
> >
> > I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> > fixed JIRAs since the previous 3.0.0-beta1 release.
> >
> > You can find the artifacts here:
> >
> > http://home.apache.org/~wang/3.0.0-RC1/
> >
> > I've done the traditional testing of building from the source tarball and
> > running a Pi job on a single node cluster. I also verified that the
> shaded
> > jars are not empty.
> >
> > Found one issue that create-release (probably due to the mvn deploy
> change)
> > didn't sign the artifacts, but I fixed that by calling mvn one more time.
> > Available here:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> >
> > This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> > Pacific. My +1 to start.
> >
> > Best,
> > Andrew
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Wangda Tan <wh...@gmail.com>.

Thanks Andrew for driving this.

+1 (Binding)

Ran SLS + CS's Perf unit test and saw similar performance compared to
trunk.

Compiled from source, deployed single node cluster and ran several jobs.

Best,
Wangda




On Wed, Dec 13, 2017 at 7:29 AM, Sunil G <su...@apache.org> wrote:

> +1 (binding)
>
> Thanks Andrew Wang for driving this effort and also thanks to all others
> who helped in this release. Kudos!!!
>
> I tested this RC by building it from source. I met with couple of issues
> (not blocker) HADOOP-15116 and YARN-7650. This could be tracked separately.
>
>
>    - Ran many MR apps and verified both new YARN UI and old RM UI.
>    - Tested below feature sanity and got results as per the behavior
>       - Application priority (verified CLI/REST/UI etc)
>       - Application timeout
>       - Intra Queue preemption with priority based
>       - Inter Queue preemption
>    - Tested basic NodeLabel scenarios.
>       - Added couple of labels to few of nodes and behavior is coming
>       correct.
>       - Verified old UI  and new YARN UI for labels.
>       - Submitted apps to labelled cluster and it works fine.
>       - Also performed few cli commands related to nodelabel.
>    - Test basic HA cases and seems correct. However I got one issue.
>    Raised HADOOP-15116 as its not a blocker.
>    - Also tested new YARN UI . All pages are getting loaded correctly.
>    (User must enable CORS to access NodeManager pages)
>    - *Performance test*: I ran a tight loop perf test on CS
>    TestCapacitySchedulerPerf#testUserLimitThroughputForTwoResources.
>    Results are a bit off w.r.t 2.8  (~5% less). I will open a ticket and
>    investigate by doing more tests to see if its to be addressed or not.
>
>
> - Sunil G
>
>
>
> On Sat, Dec 9, 2017 at 2:01 AM Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi all,
> >
> > Let me start, as always, by thanking the efforts of all the contributors
> > who contributed to this release, especially those who jumped on the
> issues
> > found in RC0.
> >
> > I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> > fixed JIRAs since the previous 3.0.0-beta1 release.
> >
> > You can find the artifacts here:
> >
> > http://home.apache.org/~wang/3.0.0-RC1/
> >
> > I've done the traditional testing of building from the source tarball and
> > running a Pi job on a single node cluster. I also verified that the
> shaded
> > jars are not empty.
> >
> > Found one issue that create-release (probably due to the mvn deploy
> change)
> > didn't sign the artifacts, but I fixed that by calling mvn one more time.
> > Available here:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> >
> > This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> > Pacific. My +1 to start.
> >
> > Best,
> > Andrew
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Wangda Tan <wh...@gmail.com>.

Thanks Andrew for driving this.

+1 (Binding)

Ran SLS + CS's Perf unit test and saw similar performance compared to
trunk.

Compiled from source, deployed single node cluster and ran several jobs.

Best,
Wangda




On Wed, Dec 13, 2017 at 7:29 AM, Sunil G <su...@apache.org> wrote:

> +1 (binding)
>
> Thanks Andrew Wang for driving this effort and also thanks to all others
> who helped in this release. Kudos!!!
>
> I tested this RC by building it from source. I met with couple of issues
> (not blocker) HADOOP-15116 and YARN-7650. This could be tracked separately.
>
>
>    - Ran many MR apps and verified both new YARN UI and old RM UI.
>    - Tested below feature sanity and got results as per the behavior
>       - Application priority (verified CLI/REST/UI etc)
>       - Application timeout
>       - Intra Queue preemption with priority based
>       - Inter Queue preemption
>    - Tested basic NodeLabel scenarios.
>       - Added couple of labels to few of nodes and behavior is coming
>       correct.
>       - Verified old UI  and new YARN UI for labels.
>       - Submitted apps to labelled cluster and it works fine.
>       - Also performed few cli commands related to nodelabel.
>    - Test basic HA cases and seems correct. However I got one issue.
>    Raised HADOOP-15116 as its not a blocker.
>    - Also tested new YARN UI . All pages are getting loaded correctly.
>    (User must enable CORS to access NodeManager pages)
>    - *Performance test*: I ran a tight loop perf test on CS
>    TestCapacitySchedulerPerf#testUserLimitThroughputForTwoResources.
>    Results are a bit off w.r.t 2.8  (~5% less). I will open a ticket and
>    investigate by doing more tests to see if its to be addressed or not.
>
>
> - Sunil G
>
>
>
> On Sat, Dec 9, 2017 at 2:01 AM Andrew Wang <an...@cloudera.com>
> wrote:
>
> > Hi all,
> >
> > Let me start, as always, by thanking the efforts of all the contributors
> > who contributed to this release, especially those who jumped on the
> issues
> > found in RC0.
> >
> > I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> > fixed JIRAs since the previous 3.0.0-beta1 release.
> >
> > You can find the artifacts here:
> >
> > http://home.apache.org/~wang/3.0.0-RC1/
> >
> > I've done the traditional testing of building from the source tarball and
> > running a Pi job on a single node cluster. I also verified that the
> shaded
> > jars are not empty.
> >
> > Found one issue that create-release (probably due to the mvn deploy
> change)
> > didn't sign the artifacts, but I fixed that by calling mvn one more time.
> > Available here:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> >
> > This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> > Pacific. My +1 to start.
> >
> > Best,
> > Andrew
> >
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sunil G <su...@apache.org>.

+1 (binding)

Thanks Andrew Wang for driving this effort and also thanks to all others
who helped in this release. Kudos!!!

I tested this RC by building it from source. I met with couple of issues
(not blocker) HADOOP-15116 and YARN-7650. This could be tracked separately.


   - Ran many MR apps and verified both new YARN UI and old RM UI.
   - Tested below feature sanity and got results as per the behavior
      - Application priority (verified CLI/REST/UI etc)
      - Application timeout
      - Intra Queue preemption with priority based
      - Inter Queue preemption
   - Tested basic NodeLabel scenarios.
      - Added couple of labels to few of nodes and behavior is coming
      correct.
      - Verified old UI  and new YARN UI for labels.
      - Submitted apps to labelled cluster and it works fine.
      - Also performed few cli commands related to nodelabel.
   - Test basic HA cases and seems correct. However I got one issue.
   Raised HADOOP-15116 as its not a blocker.
   - Also tested new YARN UI . All pages are getting loaded correctly.
   (User must enable CORS to access NodeManager pages)
   - *Performance test*: I ran a tight loop perf test on CS
   TestCapacitySchedulerPerf#testUserLimitThroughputForTwoResources.
   Results are a bit off w.r.t 2.8  (~5% less). I will open a ticket and
   investigate by doing more tests to see if its to be addressed or not.


- Sunil G



On Sat, Dec 9, 2017 at 2:01 AM Andrew Wang <an...@cloudera.com> wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Rohith Sharma K S <ro...@apache.org>.

+1 (binding)

- built from source and deployed 3 node cluster
- installed RM HA cluster along with ATSv2 enabled and new YARN UI.
- verified for
-- RM HA switch / RM Restart / RM Work preserving restart
-- NM work preserving restart
-- Ran sample MR jobs and Distributed shell along with multiple RM and NM
switch
- verified for ATSv2 entities data, REST API's validation as HBase-1.2.6 as
back end.
- verified for priority. timeout feature of RM.
- verified for new YARN UI and pages along with atsv2 integration

Thanks & Regards
Rohith Sharma K S


On 9 December 2017 at 02:01, Andrew Wang <an...@cloudera.com> wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sangjin Lee <sj...@apache.org>.

Hi Andrew,

Thanks much for your effort! Just to be clear, could you please state the
git commit id of the RC1 we're voting for?

Sangjin

On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

Looked at RC1. Went through my usual check-list. Here's my summary.

+1 (binding) overall

Verification
- [Check] Successful recompilation from source tar-ball
- [Check] Signature verification
- [Check] Generating dist tarballs from source tar-ball
- [Check] Validating the layout of the binary tar-ball
- [Check] Testing
   -- Start NN, DN, RM, NM, JHS, Timeline Service
   -- Ran dist-shell example, MR sleep, wordcount, randomwriter, sort, grep, pi
   -- Tested CLIs to print nodes, apps etc and also navigated UIs

Few issues as before found during testing, but shouldn't be blockers
 - The previously supported way of being able to use different tar-balls for different sub-modules is completely broken - common and HDFS tar.gz are completely empty. Will file a ticket.
 - resourcemanager-metrics.out is going into current directory instead of log directory. Will file a ticket.

One thing I want to make sure folks agree to. Allen filed a ticket to remove yarn-historyserver option per our previous discussion - https://issues.apache.org/jira/browse/YARN-7588 <https://issues.apache.org/jira/browse/YARN-7588>. It isn't done - I am hoping this 'incompatible change' can be put in 3.0.1 and not 4.0.

Thanks
+Vinod


> On Dec 8, 2017, at 12:31 PM, Andrew Wang <an...@cloudera.com> wrote:
> 
> Hi all,
> 
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
> 
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
> 
> You can find the artifacts here:
> 
> http://home.apache.org/~wang/3.0.0-RC1/
> 
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
> 
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
> 
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> 
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
> 
> Best,
> Andrew

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sean Mackrory <ma...@gmail.com>.

+1 (non-binding)

* Verified md5 of all artifacts
* Built with -Pdist
* Ran several s3a shell commands
* Started a pseudo-distributed HDFS and YARN cluster
* Ran grep and pi MR examples
* Sanity-checked contents of release notes, rat report, and changes

On Fri, Dec 8, 2017 at 1:31 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sean Mackrory <ma...@gmail.com>.

+1 (non-binding)

* Verified md5 of all artifacts
* Built with -Pdist
* Ran several s3a shell commands
* Started a pseudo-distributed HDFS and YARN cluster
* Ran grep and pi MR examples
* Sanity-checked contents of release notes, rat report, and changes

On Fri, Dec 8, 2017 at 1:31 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

RE: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Brahma Reddy Battula <br...@huawei.com>.

+1 (non-binding),Thanks Andrew wang for driving this.

---Built from the source
--Installed 3 Node HA cluster
--Verified Basic shell commands 
--Ran Sample jobs like pi,wordcount
--Verified the UI's



--Brahma Reddy Battula


-----Original Message-----
From: Andrew Wang [mailto:andrew.wang@cloudera.com] 
Sent: 09 December 2017 02:01
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 3.0.0 RC1

Hi all,

Let me start, as always, by thanking the efforts of all the contributors who contributed to this release, especially those who jumped on the issues found in RC0.

I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302 fixed JIRAs since the previous 3.0.0-beta1 release.

You can find the artifacts here:

http://home.apache.org/~wang/3.0.0-RC1/

I've done the traditional testing of building from the source tarball and running a Pi job on a single node cluster. I also verified that the shaded jars are not empty.

Found one issue that create-release (probably due to the mvn deploy change) didn't sign the artifacts, but I fixed that by calling mvn one more time.
Available here:

https://repository.apache.org/content/repositories/orgapachehadoop-1075/

This release will run the standard 5 days, closing on Dec 13th at 12:31pm Pacific. My +1 to start.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

Looked at RC1. Went through my usual check-list. Here's my summary.

+1 (binding) overall

Verification
- [Check] Successful recompilation from source tar-ball
- [Check] Signature verification
- [Check] Generating dist tarballs from source tar-ball
- [Check] Validating the layout of the binary tar-ball
- [Check] Testing
   -- Start NN, DN, RM, NM, JHS, Timeline Service
   -- Ran dist-shell example, MR sleep, wordcount, randomwriter, sort, grep, pi
   -- Tested CLIs to print nodes, apps etc and also navigated UIs

Few issues as before found during testing, but shouldn't be blockers
 - The previously supported way of being able to use different tar-balls for different sub-modules is completely broken - common and HDFS tar.gz are completely empty. Will file a ticket.
 - resourcemanager-metrics.out is going into current directory instead of log directory. Will file a ticket.

One thing I want to make sure folks agree to. Allen filed a ticket to remove yarn-historyserver option per our previous discussion - https://issues.apache.org/jira/browse/YARN-7588 <https://issues.apache.org/jira/browse/YARN-7588>. It isn't done - I am hoping this 'incompatible change' can be put in 3.0.1 and not 4.0.

Thanks
+Vinod


> On Dec 8, 2017, at 12:31 PM, Andrew Wang <an...@cloudera.com> wrote:
> 
> Hi all,
> 
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
> 
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
> 
> You can find the artifacts here:
> 
> http://home.apache.org/~wang/3.0.0-RC1/
> 
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
> 
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
> 
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> 
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
> 
> Best,
> Andrew

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

Looked at RC1. Went through my usual check-list. Here's my summary.

+1 (binding) overall

Verification
- [Check] Successful recompilation from source tar-ball
- [Check] Signature verification
- [Check] Generating dist tarballs from source tar-ball
- [Check] Validating the layout of the binary tar-ball
- [Check] Testing
   -- Start NN, DN, RM, NM, JHS, Timeline Service
   -- Ran dist-shell example, MR sleep, wordcount, randomwriter, sort, grep, pi
   -- Tested CLIs to print nodes, apps etc and also navigated UIs

Few issues as before found during testing, but shouldn't be blockers
 - The previously supported way of being able to use different tar-balls for different sub-modules is completely broken - common and HDFS tar.gz are completely empty. Will file a ticket.
 - resourcemanager-metrics.out is going into current directory instead of log directory. Will file a ticket.

One thing I want to make sure folks agree to. Allen filed a ticket to remove yarn-historyserver option per our previous discussion - https://issues.apache.org/jira/browse/YARN-7588 <https://issues.apache.org/jira/browse/YARN-7588>. It isn't done - I am hoping this 'incompatible change' can be put in 3.0.1 and not 4.0.

Thanks
+Vinod


> On Dec 8, 2017, at 12:31 PM, Andrew Wang <an...@cloudera.com> wrote:
> 
> Hi all,
> 
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
> 
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
> 
> You can find the artifacts here:
> 
> http://home.apache.org/~wang/3.0.0-RC1/
> 
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
> 
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
> 
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> 
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
> 
> Best,
> Andrew

RE: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Brahma Reddy Battula <br...@huawei.com>.

+1 (non-binding),Thanks Andrew wang for driving this.

---Built from the source
--Installed 3 Node HA cluster
--Verified Basic shell commands 
--Ran Sample jobs like pi,wordcount
--Verified the UI's



--Brahma Reddy Battula


-----Original Message-----
From: Andrew Wang [mailto:andrew.wang@cloudera.com] 
Sent: 09 December 2017 02:01
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 3.0.0 RC1

Hi all,

Let me start, as always, by thanking the efforts of all the contributors who contributed to this release, especially those who jumped on the issues found in RC0.

I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302 fixed JIRAs since the previous 3.0.0-beta1 release.

You can find the artifacts here:

http://home.apache.org/~wang/3.0.0-RC1/

I've done the traditional testing of building from the source tarball and running a Pi job on a single node cluster. I also verified that the shaded jars are not empty.

Found one issue that create-release (probably due to the mvn deploy change) didn't sign the artifacts, but I fixed that by calling mvn one more time.
Available here:

https://repository.apache.org/content/repositories/orgapachehadoop-1075/

This release will run the standard 5 days, closing on Dec 13th at 12:31pm Pacific. My +1 to start.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sean Mackrory <ma...@gmail.com>.

+1 (non-binding)

* Verified md5 of all artifacts
* Built with -Pdist
* Ran several s3a shell commands
* Started a pseudo-distributed HDFS and YARN cluster
* Ran grep and pi MR examples
* Sanity-checked contents of release notes, rat report, and changes

On Fri, Dec 8, 2017 at 1:31 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Rohith Sharma K S <ro...@apache.org>.

+1 (binding)

- built from source and deployed 3 node cluster
- installed RM HA cluster along with ATSv2 enabled and new YARN UI.
- verified for
-- RM HA switch / RM Restart / RM Work preserving restart
-- NM work preserving restart
-- Ran sample MR jobs and Distributed shell along with multiple RM and NM
switch
- verified for ATSv2 entities data, REST API's validation as HBase-1.2.6 as
back end.
- verified for priority. timeout feature of RM.
- verified for new YARN UI and pages along with atsv2 integration

Thanks & Regards
Rohith Sharma K S


On 9 December 2017 at 02:01, Andrew Wang <an...@cloudera.com> wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

RE: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Brahma Reddy Battula <br...@huawei.com>.

+1 (non-binding),Thanks Andrew wang for driving this.

---Built from the source
--Installed 3 Node HA cluster
--Verified Basic shell commands 
--Ran Sample jobs like pi,wordcount
--Verified the UI's



--Brahma Reddy Battula


-----Original Message-----
From: Andrew Wang [mailto:andrew.wang@cloudera.com] 
Sent: 09 December 2017 02:01
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 3.0.0 RC1

Hi all,

Let me start, as always, by thanking the efforts of all the contributors who contributed to this release, especially those who jumped on the issues found in RC0.

I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302 fixed JIRAs since the previous 3.0.0-beta1 release.

You can find the artifacts here:

http://home.apache.org/~wang/3.0.0-RC1/

I've done the traditional testing of building from the source tarball and running a Pi job on a single node cluster. I also verified that the shaded jars are not empty.

Found one issue that create-release (probably due to the mvn deploy change) didn't sign the artifacts, but I fixed that by calling mvn one more time.
Available here:

https://repository.apache.org/content/repositories/orgapachehadoop-1075/

This release will run the standard 5 days, closing on Dec 13th at 12:31pm Pacific. My +1 to start.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sangjin Lee <sj...@apache.org>.

Hi Andrew,

Thanks much for your effort! Just to be clear, could you please state the
git commit id of the RC1 we're voting for?

Sangjin

On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sean Mackrory <ma...@gmail.com>.

+1 (non-binding)

* Verified md5 of all artifacts
* Built with -Pdist
* Ran several s3a shell commands
* Started a pseudo-distributed HDFS and YARN cluster
* Ran grep and pi MR examples
* Sanity-checked contents of release notes, rat report, and changes

On Fri, Dec 8, 2017 at 1:31 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sunil G <su...@apache.org>.

+1 (binding)

Thanks Andrew Wang for driving this effort and also thanks to all others
who helped in this release. Kudos!!!

I tested this RC by building it from source. I met with couple of issues
(not blocker) HADOOP-15116 and YARN-7650. This could be tracked separately.


   - Ran many MR apps and verified both new YARN UI and old RM UI.
   - Tested below feature sanity and got results as per the behavior
      - Application priority (verified CLI/REST/UI etc)
      - Application timeout
      - Intra Queue preemption with priority based
      - Inter Queue preemption
   - Tested basic NodeLabel scenarios.
      - Added couple of labels to few of nodes and behavior is coming
      correct.
      - Verified old UI  and new YARN UI for labels.
      - Submitted apps to labelled cluster and it works fine.
      - Also performed few cli commands related to nodelabel.
   - Test basic HA cases and seems correct. However I got one issue.
   Raised HADOOP-15116 as its not a blocker.
   - Also tested new YARN UI . All pages are getting loaded correctly.
   (User must enable CORS to access NodeManager pages)
   - *Performance test*: I ran a tight loop perf test on CS
   TestCapacitySchedulerPerf#testUserLimitThroughputForTwoResources.
   Results are a bit off w.r.t 2.8  (~5% less). I will open a ticket and
   investigate by doing more tests to see if its to be addressed or not.


- Sunil G



On Sat, Dec 9, 2017 at 2:01 AM Andrew Wang <an...@cloudera.com> wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Rohith Sharma K S <ro...@apache.org>.

+1 (binding)

- built from source and deployed 3 node cluster
- installed RM HA cluster along with ATSv2 enabled and new YARN UI.
- verified for
-- RM HA switch / RM Restart / RM Work preserving restart
-- NM work preserving restart
-- Ran sample MR jobs and Distributed shell along with multiple RM and NM
switch
- verified for ATSv2 entities data, REST API's validation as HBase-1.2.6 as
back end.
- verified for priority. timeout feature of RM.
- verified for new YARN UI and pages along with atsv2 integration

Thanks & Regards
Rohith Sharma K S


On 9 December 2017 at 02:01, Andrew Wang <an...@cloudera.com> wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sunil G <su...@apache.org>.

+1 (binding)

Thanks Andrew Wang for driving this effort and also thanks to all others
who helped in this release. Kudos!!!

I tested this RC by building it from source. I met with couple of issues
(not blocker) HADOOP-15116 and YARN-7650. This could be tracked separately.


   - Ran many MR apps and verified both new YARN UI and old RM UI.
   - Tested below feature sanity and got results as per the behavior
      - Application priority (verified CLI/REST/UI etc)
      - Application timeout
      - Intra Queue preemption with priority based
      - Inter Queue preemption
   - Tested basic NodeLabel scenarios.
      - Added couple of labels to few of nodes and behavior is coming
      correct.
      - Verified old UI  and new YARN UI for labels.
      - Submitted apps to labelled cluster and it works fine.
      - Also performed few cli commands related to nodelabel.
   - Test basic HA cases and seems correct. However I got one issue.
   Raised HADOOP-15116 as its not a blocker.
   - Also tested new YARN UI . All pages are getting loaded correctly.
   (User must enable CORS to access NodeManager pages)
   - *Performance test*: I ran a tight loop perf test on CS
   TestCapacitySchedulerPerf#testUserLimitThroughputForTwoResources.
   Results are a bit off w.r.t 2.8  (~5% less). I will open a ticket and
   investigate by doing more tests to see if its to be addressed or not.


- Sunil G



On Sat, Dec 9, 2017 at 2:01 AM Andrew Wang <an...@cloudera.com> wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by "Elek, Marton" <hd...@anzix.net>.

+1 (non-binding)

  * built from the source tarball (archlinux) / verified signature
  * Deployed to a kubernetes cluster (10/10 datanode/nodemanager pods)
  * Enabled ec on hdfs directory (hdfs cli)
  * Started example yarn jobs (pi/terragen)
  * checked yarn ui/ui2

Thanks for all the efforts.

Marton


On 12/08/2017 09:31 PM, Andrew Wang wrote:
> Hi all,
> 
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
> 
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
> 
> You can find the artifacts here:
> 
> http://home.apache.org/~wang/3.0.0-RC1/
> 
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
> 
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
> 
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> 
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
> 
> Best,
> Andrew
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

RE: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Brahma Reddy Battula <br...@huawei.com>.

+1 (non-binding),Thanks Andrew wang for driving this.

---Built from the source
--Installed 3 Node HA cluster
--Verified Basic shell commands 
--Ran Sample jobs like pi,wordcount
--Verified the UI's



--Brahma Reddy Battula


-----Original Message-----
From: Andrew Wang [mailto:andrew.wang@cloudera.com] 
Sent: 09 December 2017 02:01
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 3.0.0 RC1

Hi all,

Let me start, as always, by thanking the efforts of all the contributors who contributed to this release, especially those who jumped on the issues found in RC0.

I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302 fixed JIRAs since the previous 3.0.0-beta1 release.

You can find the artifacts here:

http://home.apache.org/~wang/3.0.0-RC1/

I've done the traditional testing of building from the source tarball and running a Pi job on a single node cluster. I also verified that the shaded jars are not empty.

Found one issue that create-release (probably due to the mvn deploy change) didn't sign the artifacts, but I fixed that by calling mvn one more time.
Available here:

https://repository.apache.org/content/repositories/orgapachehadoop-1075/

This release will run the standard 5 days, closing on Dec 13th at 12:31pm Pacific. My +1 to start.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

Looked at RC1. Went through my usual check-list. Here's my summary.

+1 (binding) overall

Verification
- [Check] Successful recompilation from source tar-ball
- [Check] Signature verification
- [Check] Generating dist tarballs from source tar-ball
- [Check] Validating the layout of the binary tar-ball
- [Check] Testing
   -- Start NN, DN, RM, NM, JHS, Timeline Service
   -- Ran dist-shell example, MR sleep, wordcount, randomwriter, sort, grep, pi
   -- Tested CLIs to print nodes, apps etc and also navigated UIs

Few issues as before found during testing, but shouldn't be blockers
 - The previously supported way of being able to use different tar-balls for different sub-modules is completely broken - common and HDFS tar.gz are completely empty. Will file a ticket.
 - resourcemanager-metrics.out is going into current directory instead of log directory. Will file a ticket.

One thing I want to make sure folks agree to. Allen filed a ticket to remove yarn-historyserver option per our previous discussion - https://issues.apache.org/jira/browse/YARN-7588 <https://issues.apache.org/jira/browse/YARN-7588>. It isn't done - I am hoping this 'incompatible change' can be put in 3.0.1 and not 4.0.

Thanks
+Vinod


> On Dec 8, 2017, at 12:31 PM, Andrew Wang <an...@cloudera.com> wrote:
> 
> Hi all,
> 
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
> 
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
> 
> You can find the artifacts here:
> 
> http://home.apache.org/~wang/3.0.0-RC1/
> 
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
> 
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
> 
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
> 
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
> 
> Best,
> Andrew

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sunil G <su...@apache.org>.

+1 (binding)

Thanks Andrew Wang for driving this effort and also thanks to all others
who helped in this release. Kudos!!!

I tested this RC by building it from source. I met with couple of issues
(not blocker) HADOOP-15116 and YARN-7650. This could be tracked separately.


   - Ran many MR apps and verified both new YARN UI and old RM UI.
   - Tested below feature sanity and got results as per the behavior
      - Application priority (verified CLI/REST/UI etc)
      - Application timeout
      - Intra Queue preemption with priority based
      - Inter Queue preemption
   - Tested basic NodeLabel scenarios.
      - Added couple of labels to few of nodes and behavior is coming
      correct.
      - Verified old UI  and new YARN UI for labels.
      - Submitted apps to labelled cluster and it works fine.
      - Also performed few cli commands related to nodelabel.
   - Test basic HA cases and seems correct. However I got one issue.
   Raised HADOOP-15116 as its not a blocker.
   - Also tested new YARN UI . All pages are getting loaded correctly.
   (User must enable CORS to access NodeManager pages)
   - *Performance test*: I ran a tight loop perf test on CS
   TestCapacitySchedulerPerf#testUserLimitThroughputForTwoResources.
   Results are a bit off w.r.t 2.8  (~5% less). I will open a ticket and
   investigate by doing more tests to see if its to be addressed or not.


- Sunil G



On Sat, Dec 9, 2017 at 2:01 AM Andrew Wang <an...@cloudera.com> wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by "Aaron T. Myers" <at...@apache.org>.

+1 (binding)

- downloaded the src tarball and built the source (-Pdist -Pnative)
- verified the checksum
- brought up a secure pseudo distributed cluster
- did some basic file system operations (mkdir, list, put, cat) and
confirmed that everything was working
- confirmed that the web UI worked

Best,
Aaron

On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by "Aaron T. Myers" <at...@apache.org>.

+1 (binding)

- downloaded the src tarball and built the source (-Pdist -Pnative)
- verified the checksum
- brought up a secure pseudo distributed cluster
- did some basic file system operations (mkdir, list, put, cat) and
confirmed that everything was working
- confirmed that the web UI worked

Best,
Aaron

On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Posted by Sangjin Lee <sj...@apache.org>.

Hi Andrew,

Thanks much for your effort! Just to be clear, could you please state the
git commit id of the RC1 we're voting for?

Sangjin

On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang <an...@cloudera.com>
wrote:

> Hi all,
>
> Let me start, as always, by thanking the efforts of all the contributors
> who contributed to this release, especially those who jumped on the issues
> found in RC0.
>
> I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302
> fixed JIRAs since the previous 3.0.0-beta1 release.
>
> You can find the artifacts here:
>
> http://home.apache.org/~wang/3.0.0-RC1/
>
> I've done the traditional testing of building from the source tarball and
> running a Pi job on a single node cluster. I also verified that the shaded
> jars are not empty.
>
> Found one issue that create-release (probably due to the mvn deploy change)
> didn't sign the artifacts, but I fixed that by calling mvn one more time.
> Available here:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1075/
>
> This release will run the standard 5 days, closing on Dec 13th at 12:31pm
> Pacific. My +1 to start.
>
> Best,
> Andrew
>