[VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Hi folks,

This is the first release candidate for the first release of Apache Hadoop
2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It includes
features such as:

- User-defined resource types
- Native GPU support as a schedulable resource type
- Consistent reads from standby node
- Namenode port based selective encryption
- Improvements related to rolling upgrade support from 2.x to 3.x

The RC0 artifacts are at: http://home.apache.org/~jhung/hadoop-2.10.0-RC0/

RC tag is release-2.10.0-RC0.

The maven artifacts are hosted here:
https://repository.apache.org/content/repositories/orgapachehadoop-1241/

My public key is available here:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS

The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm
PDT.

Thanks,
Jonathan Hung

[1]
https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

Jonathan,

I actually did all my testing on RC1. Sorry for the confusion. I'll respond on the RC1 thread.

-Eric

 On Monday, October 28, 2019, 8:00:20 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 

Thanks Eric! I sent out an RC1 earlier last week, not sure if you saw that. The only diff between RC1 and RC0 is HDFS-14667. If RC1 looks good to you then it'd be great to get your testing results on that thread.

Jonathan Hung


On Mon, Oct 28, 2019 at 1:06 PM epayne@apache.org <ep...@apache.org> wrote:
> Compatibility testing has gone well for me.
> 
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.8.5 and 2.10.0
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.10.0 and trunk
> - With one 4-node cluster running 2.10.0 and one 4-node cluster running trunk, I ran a word count job in each cluster whose inputs and outputs were from and to the opposite cluster.
> - I verified that HDFS replication works as expected in a trunk cluster that has one 2.10.0 datanode.
> 
> Thanks,
> -Eric
> 
> On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 
> 
> 
> 
> 
> 
> Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
> 2.10.0 clients and datanodes. Everything worked as expected.
> 
> Jonathan Hung
> 
> 
> On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
> wrote:
> 
>> Hi Jonathan,
>>
>> Thanks for putting this RC together. You stated that there are
>> improvements related to rolling upgrades from 2.x to 3.x and I know I have
>> seen multiple JIRAs getting committed to that effect. Could you describe
>> any tests that you have done to verify rolling upgrade compatibility
>> for 3.x servers talking to 2.x clients and vice versa?
>>
>> Thanks,
>>
>> Eric
>>
>> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
>> wrote:
>>
>>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>>> (HDFS-14667). Since this is the first of a minor release, we would like to
>>> get it into 2.10.0.
>>>
>>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>>> shortly.
>>>
>>> Jonathan Hung
>>>
>>>
>>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>>
>>> > Thanks for the effort, Jonathan!
>>> >
>>> > +1 (non-binding) on RC0.
>>> >  - Set up a single node cluster with the binary tarball
>>> >  - Run Spark Pi and pySpark job
>>> >
>>> > BR,
>>> > Zhankun
>>> >
>>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>>> >
>>> > wrote:
>>> >
>>> >> +1 on RC0.
>>> >> - Verified signatures
>>> >> - Built from sources
>>> >> - Ran unit tests for new features
>>> >> - Checked artifacts on Nexus, made sure the sources are present.
>>> >>
>>> >> Thanks
>>> >> --Konstantin
>>> >>
>>> >>
>>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>>> >> wrote:
>>> >>
>>> >> > Hi folks,
>>> >> >
>>> >> > This is the first release candidate for the first release of Apache
>>> >> Hadoop
>>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>>> includes
>>> >> > features such as:
>>> >> >
>>> >> > - User-defined resource types
>>> >> > - Native GPU support as a schedulable resource type
>>> >> > - Consistent reads from standby node
>>> >> > - Namenode port based selective encryption
>>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>>> >> >
>>> >> > The RC0 artifacts are at:
>>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>>> >> >
>>> >> > RC tag is release-2.10.0-RC0.
>>> >> >
>>> >> > The maven artifacts are hosted here:
>>> >> >
>>> >>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>>> >> >
>>> >> > My public key is available here:
>>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>> >> >
>>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>>> 6:00 pm
>>> >> > PDT.
>>> >> >
>>> >> > Thanks,
>>> >> > Jonathan Hung
>>> >> >
>>> >> > [1]
>>> >> >
>>> >> >
>>> >>
>>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>>> >> >
>>> >>
>>> >
>>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

Jonathan,

I actually did all my testing on RC1. Sorry for the confusion. I'll respond on the RC1 thread.

-Eric

 On Monday, October 28, 2019, 8:00:20 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 

Thanks Eric! I sent out an RC1 earlier last week, not sure if you saw that. The only diff between RC1 and RC0 is HDFS-14667. If RC1 looks good to you then it'd be great to get your testing results on that thread.

Jonathan Hung


On Mon, Oct 28, 2019 at 1:06 PM epayne@apache.org <ep...@apache.org> wrote:
> Compatibility testing has gone well for me.
> 
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.8.5 and 2.10.0
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.10.0 and trunk
> - With one 4-node cluster running 2.10.0 and one 4-node cluster running trunk, I ran a word count job in each cluster whose inputs and outputs were from and to the opposite cluster.
> - I verified that HDFS replication works as expected in a trunk cluster that has one 2.10.0 datanode.
> 
> Thanks,
> -Eric
> 
> On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 
> 
> 
> 
> 
> 
> Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
> 2.10.0 clients and datanodes. Everything worked as expected.
> 
> Jonathan Hung
> 
> 
> On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
> wrote:
> 
>> Hi Jonathan,
>>
>> Thanks for putting this RC together. You stated that there are
>> improvements related to rolling upgrades from 2.x to 3.x and I know I have
>> seen multiple JIRAs getting committed to that effect. Could you describe
>> any tests that you have done to verify rolling upgrade compatibility
>> for 3.x servers talking to 2.x clients and vice versa?
>>
>> Thanks,
>>
>> Eric
>>
>> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
>> wrote:
>>
>>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>>> (HDFS-14667). Since this is the first of a minor release, we would like to
>>> get it into 2.10.0.
>>>
>>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>>> shortly.
>>>
>>> Jonathan Hung
>>>
>>>
>>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>>
>>> > Thanks for the effort, Jonathan!
>>> >
>>> > +1 (non-binding) on RC0.
>>> >  - Set up a single node cluster with the binary tarball
>>> >  - Run Spark Pi and pySpark job
>>> >
>>> > BR,
>>> > Zhankun
>>> >
>>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>>> >
>>> > wrote:
>>> >
>>> >> +1 on RC0.
>>> >> - Verified signatures
>>> >> - Built from sources
>>> >> - Ran unit tests for new features
>>> >> - Checked artifacts on Nexus, made sure the sources are present.
>>> >>
>>> >> Thanks
>>> >> --Konstantin
>>> >>
>>> >>
>>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>>> >> wrote:
>>> >>
>>> >> > Hi folks,
>>> >> >
>>> >> > This is the first release candidate for the first release of Apache
>>> >> Hadoop
>>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>>> includes
>>> >> > features such as:
>>> >> >
>>> >> > - User-defined resource types
>>> >> > - Native GPU support as a schedulable resource type
>>> >> > - Consistent reads from standby node
>>> >> > - Namenode port based selective encryption
>>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>>> >> >
>>> >> > The RC0 artifacts are at:
>>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>>> >> >
>>> >> > RC tag is release-2.10.0-RC0.
>>> >> >
>>> >> > The maven artifacts are hosted here:
>>> >> >
>>> >>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>>> >> >
>>> >> > My public key is available here:
>>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>> >> >
>>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>>> 6:00 pm
>>> >> > PDT.
>>> >> >
>>> >> > Thanks,
>>> >> > Jonathan Hung
>>> >> >
>>> >> > [1]
>>> >> >
>>> >> >
>>> >>
>>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>>> >> >
>>> >>
>>> >
>>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

Jonathan,

I actually did all my testing on RC1. Sorry for the confusion. I'll respond on the RC1 thread.

-Eric

 On Monday, October 28, 2019, 8:00:20 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 

Thanks Eric! I sent out an RC1 earlier last week, not sure if you saw that. The only diff between RC1 and RC0 is HDFS-14667. If RC1 looks good to you then it'd be great to get your testing results on that thread.

Jonathan Hung


On Mon, Oct 28, 2019 at 1:06 PM epayne@apache.org <ep...@apache.org> wrote:
> Compatibility testing has gone well for me.
> 
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.8.5 and 2.10.0
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.10.0 and trunk
> - With one 4-node cluster running 2.10.0 and one 4-node cluster running trunk, I ran a word count job in each cluster whose inputs and outputs were from and to the opposite cluster.
> - I verified that HDFS replication works as expected in a trunk cluster that has one 2.10.0 datanode.
> 
> Thanks,
> -Eric
> 
> On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 
> 
> 
> 
> 
> 
> Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
> 2.10.0 clients and datanodes. Everything worked as expected.
> 
> Jonathan Hung
> 
> 
> On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
> wrote:
> 
>> Hi Jonathan,
>>
>> Thanks for putting this RC together. You stated that there are
>> improvements related to rolling upgrades from 2.x to 3.x and I know I have
>> seen multiple JIRAs getting committed to that effect. Could you describe
>> any tests that you have done to verify rolling upgrade compatibility
>> for 3.x servers talking to 2.x clients and vice versa?
>>
>> Thanks,
>>
>> Eric
>>
>> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
>> wrote:
>>
>>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>>> (HDFS-14667). Since this is the first of a minor release, we would like to
>>> get it into 2.10.0.
>>>
>>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>>> shortly.
>>>
>>> Jonathan Hung
>>>
>>>
>>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>>
>>> > Thanks for the effort, Jonathan!
>>> >
>>> > +1 (non-binding) on RC0.
>>> >  - Set up a single node cluster with the binary tarball
>>> >  - Run Spark Pi and pySpark job
>>> >
>>> > BR,
>>> > Zhankun
>>> >
>>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>>> >
>>> > wrote:
>>> >
>>> >> +1 on RC0.
>>> >> - Verified signatures
>>> >> - Built from sources
>>> >> - Ran unit tests for new features
>>> >> - Checked artifacts on Nexus, made sure the sources are present.
>>> >>
>>> >> Thanks
>>> >> --Konstantin
>>> >>
>>> >>
>>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>>> >> wrote:
>>> >>
>>> >> > Hi folks,
>>> >> >
>>> >> > This is the first release candidate for the first release of Apache
>>> >> Hadoop
>>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>>> includes
>>> >> > features such as:
>>> >> >
>>> >> > - User-defined resource types
>>> >> > - Native GPU support as a schedulable resource type
>>> >> > - Consistent reads from standby node
>>> >> > - Namenode port based selective encryption
>>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>>> >> >
>>> >> > The RC0 artifacts are at:
>>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>>> >> >
>>> >> > RC tag is release-2.10.0-RC0.
>>> >> >
>>> >> > The maven artifacts are hosted here:
>>> >> >
>>> >>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>>> >> >
>>> >> > My public key is available here:
>>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>> >> >
>>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>>> 6:00 pm
>>> >> > PDT.
>>> >> >
>>> >> > Thanks,
>>> >> > Jonathan Hung
>>> >> >
>>> >> > [1]
>>> >> >
>>> >> >
>>> >>
>>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>>> >> >
>>> >>
>>> >
>>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

Jonathan,

I actually did all my testing on RC1. Sorry for the confusion. I'll respond on the RC1 thread.

-Eric

 On Monday, October 28, 2019, 8:00:20 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 

Thanks Eric! I sent out an RC1 earlier last week, not sure if you saw that. The only diff between RC1 and RC0 is HDFS-14667. If RC1 looks good to you then it'd be great to get your testing results on that thread.

Jonathan Hung


On Mon, Oct 28, 2019 at 1:06 PM epayne@apache.org <ep...@apache.org> wrote:
> Compatibility testing has gone well for me.
> 
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.8.5 and 2.10.0
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.10.0 and trunk
> - With one 4-node cluster running 2.10.0 and one 4-node cluster running trunk, I ran a word count job in each cluster whose inputs and outputs were from and to the opposite cluster.
> - I verified that HDFS replication works as expected in a trunk cluster that has one 2.10.0 datanode.
> 
> Thanks,
> -Eric
> 
> On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 
> 
> 
> 
> 
> 
> Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
> 2.10.0 clients and datanodes. Everything worked as expected.
> 
> Jonathan Hung
> 
> 
> On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
> wrote:
> 
>> Hi Jonathan,
>>
>> Thanks for putting this RC together. You stated that there are
>> improvements related to rolling upgrades from 2.x to 3.x and I know I have
>> seen multiple JIRAs getting committed to that effect. Could you describe
>> any tests that you have done to verify rolling upgrade compatibility
>> for 3.x servers talking to 2.x clients and vice versa?
>>
>> Thanks,
>>
>> Eric
>>
>> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
>> wrote:
>>
>>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>>> (HDFS-14667). Since this is the first of a minor release, we would like to
>>> get it into 2.10.0.
>>>
>>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>>> shortly.
>>>
>>> Jonathan Hung
>>>
>>>
>>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>>
>>> > Thanks for the effort, Jonathan!
>>> >
>>> > +1 (non-binding) on RC0.
>>> >  - Set up a single node cluster with the binary tarball
>>> >  - Run Spark Pi and pySpark job
>>> >
>>> > BR,
>>> > Zhankun
>>> >
>>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>>> >
>>> > wrote:
>>> >
>>> >> +1 on RC0.
>>> >> - Verified signatures
>>> >> - Built from sources
>>> >> - Ran unit tests for new features
>>> >> - Checked artifacts on Nexus, made sure the sources are present.
>>> >>
>>> >> Thanks
>>> >> --Konstantin
>>> >>
>>> >>
>>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>>> >> wrote:
>>> >>
>>> >> > Hi folks,
>>> >> >
>>> >> > This is the first release candidate for the first release of Apache
>>> >> Hadoop
>>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>>> includes
>>> >> > features such as:
>>> >> >
>>> >> > - User-defined resource types
>>> >> > - Native GPU support as a schedulable resource type
>>> >> > - Consistent reads from standby node
>>> >> > - Namenode port based selective encryption
>>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>>> >> >
>>> >> > The RC0 artifacts are at:
>>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>>> >> >
>>> >> > RC tag is release-2.10.0-RC0.
>>> >> >
>>> >> > The maven artifacts are hosted here:
>>> >> >
>>> >>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>>> >> >
>>> >> > My public key is available here:
>>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>> >> >
>>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>>> 6:00 pm
>>> >> > PDT.
>>> >> >
>>> >> > Thanks,
>>> >> > Jonathan Hung
>>> >> >
>>> >> > [1]
>>> >> >
>>> >> >
>>> >>
>>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>>> >> >
>>> >>
>>> >
>>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Thanks Eric! I sent out an RC1 earlier last week, not sure if you saw that.
The only diff between RC1 and RC0 is HDFS-14667. If RC1 looks good to you
then it'd be great to get your testing results on that thread.

Jonathan Hung


On Mon, Oct 28, 2019 at 1:06 PM epayne@apache.org <ep...@apache.org> wrote:

> Compatibility testing has gone well for me.
>
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.8.5 and
> 2.10.0
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.10.0 and
> trunk
> - With one 4-node cluster running 2.10.0 and one 4-node cluster running
> trunk, I ran a word count job in each cluster whose inputs and outputs were
> from and to the opposite cluster.
> - I verified that HDFS replication works as expected in a trunk cluster
> that has one 2.10.0 datanode.
>
> Thanks,
> -Eric
>
> On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <
> jyhung2357@gmail.com> wrote:
>
>
>
>
>
> Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
> 2.10.0 clients and datanodes. Everything worked as expected.
>
> Jonathan Hung
>
>
> On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
> wrote:
>
> > Hi Jonathan,
> >
> > Thanks for putting this RC together. You stated that there are
> > improvements related to rolling upgrades from 2.x to 3.x and I know I
> have
> > seen multiple JIRAs getting committed to that effect. Could you describe
> > any tests that you have done to verify rolling upgrade compatibility
> > for 3.x servers talking to 2.x clients and vice versa?
> >
> > Thanks,
> >
> > Eric
> >
> > On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> > wrote:
> >
> >> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
> >> (HDFS-14667). Since this is the first of a minor release, we would like
> to
> >> get it into 2.10.0.
> >>
> >> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
> >> shortly.
> >>
> >> Jonathan Hung
> >>
> >>
> >> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
> >>
> >> > Thanks for the effort, Jonathan!
> >> >
> >> > +1 (non-binding) on RC0.
> >> >  - Set up a single node cluster with the binary tarball
> >> >  - Run Spark Pi and pySpark job
> >> >
> >> > BR,
> >> > Zhankun
> >> >
> >> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <
> shv.hadoop@gmail.com
> >> >
> >> > wrote:
> >> >
> >> >> +1 on RC0.
> >> >> - Verified signatures
> >> >> - Built from sources
> >> >> - Ran unit tests for new features
> >> >> - Checked artifacts on Nexus, made sure the sources are present.
> >> >>
> >> >> Thanks
> >> >> --Konstantin
> >> >>
> >> >>
> >> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> >> >> wrote:
> >> >>
> >> >> > Hi folks,
> >> >> >
> >> >> > This is the first release candidate for the first release of Apache
> >> >> Hadoop
> >> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
> >> includes
> >> >> > features such as:
> >> >> >
> >> >> > - User-defined resource types
> >> >> > - Native GPU support as a schedulable resource type
> >> >> > - Consistent reads from standby node
> >> >> > - Namenode port based selective encryption
> >> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >> >> >
> >> >> > The RC0 artifacts are at:
> >> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >> >> >
> >> >> > RC tag is release-2.10.0-RC0.
> >> >> >
> >> >> > The maven artifacts are hosted here:
> >> >> >
> >> >>
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >> >> >
> >> >> > My public key is available here:
> >> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >> >> >
> >> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
> >> 6:00 pm
> >> >> > PDT.
> >> >> >
> >> >> > Thanks,
> >> >> > Jonathan Hung
> >> >> >
> >> >> > [1]
> >> >> >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >> >> >
> >> >>
> >> >
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Thanks Eric! I sent out an RC1 earlier last week, not sure if you saw that.
The only diff between RC1 and RC0 is HDFS-14667. If RC1 looks good to you
then it'd be great to get your testing results on that thread.

Jonathan Hung


On Mon, Oct 28, 2019 at 1:06 PM epayne@apache.org <ep...@apache.org> wrote:

> Compatibility testing has gone well for me.
>
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.8.5 and
> 2.10.0
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.10.0 and
> trunk
> - With one 4-node cluster running 2.10.0 and one 4-node cluster running
> trunk, I ran a word count job in each cluster whose inputs and outputs were
> from and to the opposite cluster.
> - I verified that HDFS replication works as expected in a trunk cluster
> that has one 2.10.0 datanode.
>
> Thanks,
> -Eric
>
> On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <
> jyhung2357@gmail.com> wrote:
>
>
>
>
>
> Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
> 2.10.0 clients and datanodes. Everything worked as expected.
>
> Jonathan Hung
>
>
> On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
> wrote:
>
> > Hi Jonathan,
> >
> > Thanks for putting this RC together. You stated that there are
> > improvements related to rolling upgrades from 2.x to 3.x and I know I
> have
> > seen multiple JIRAs getting committed to that effect. Could you describe
> > any tests that you have done to verify rolling upgrade compatibility
> > for 3.x servers talking to 2.x clients and vice versa?
> >
> > Thanks,
> >
> > Eric
> >
> > On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> > wrote:
> >
> >> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
> >> (HDFS-14667). Since this is the first of a minor release, we would like
> to
> >> get it into 2.10.0.
> >>
> >> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
> >> shortly.
> >>
> >> Jonathan Hung
> >>
> >>
> >> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
> >>
> >> > Thanks for the effort, Jonathan!
> >> >
> >> > +1 (non-binding) on RC0.
> >> >  - Set up a single node cluster with the binary tarball
> >> >  - Run Spark Pi and pySpark job
> >> >
> >> > BR,
> >> > Zhankun
> >> >
> >> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <
> shv.hadoop@gmail.com
> >> >
> >> > wrote:
> >> >
> >> >> +1 on RC0.
> >> >> - Verified signatures
> >> >> - Built from sources
> >> >> - Ran unit tests for new features
> >> >> - Checked artifacts on Nexus, made sure the sources are present.
> >> >>
> >> >> Thanks
> >> >> --Konstantin
> >> >>
> >> >>
> >> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> >> >> wrote:
> >> >>
> >> >> > Hi folks,
> >> >> >
> >> >> > This is the first release candidate for the first release of Apache
> >> >> Hadoop
> >> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
> >> includes
> >> >> > features such as:
> >> >> >
> >> >> > - User-defined resource types
> >> >> > - Native GPU support as a schedulable resource type
> >> >> > - Consistent reads from standby node
> >> >> > - Namenode port based selective encryption
> >> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >> >> >
> >> >> > The RC0 artifacts are at:
> >> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >> >> >
> >> >> > RC tag is release-2.10.0-RC0.
> >> >> >
> >> >> > The maven artifacts are hosted here:
> >> >> >
> >> >>
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >> >> >
> >> >> > My public key is available here:
> >> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >> >> >
> >> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
> >> 6:00 pm
> >> >> > PDT.
> >> >> >
> >> >> > Thanks,
> >> >> > Jonathan Hung
> >> >> >
> >> >> > [1]
> >> >> >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >> >> >
> >> >>
> >> >
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Thanks Eric! I sent out an RC1 earlier last week, not sure if you saw that.
The only diff between RC1 and RC0 is HDFS-14667. If RC1 looks good to you
then it'd be great to get your testing results on that thread.

Jonathan Hung


On Mon, Oct 28, 2019 at 1:06 PM epayne@apache.org <ep...@apache.org> wrote:

> Compatibility testing has gone well for me.
>
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.8.5 and
> 2.10.0
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.10.0 and
> trunk
> - With one 4-node cluster running 2.10.0 and one 4-node cluster running
> trunk, I ran a word count job in each cluster whose inputs and outputs were
> from and to the opposite cluster.
> - I verified that HDFS replication works as expected in a trunk cluster
> that has one 2.10.0 datanode.
>
> Thanks,
> -Eric
>
> On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <
> jyhung2357@gmail.com> wrote:
>
>
>
>
>
> Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
> 2.10.0 clients and datanodes. Everything worked as expected.
>
> Jonathan Hung
>
>
> On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
> wrote:
>
> > Hi Jonathan,
> >
> > Thanks for putting this RC together. You stated that there are
> > improvements related to rolling upgrades from 2.x to 3.x and I know I
> have
> > seen multiple JIRAs getting committed to that effect. Could you describe
> > any tests that you have done to verify rolling upgrade compatibility
> > for 3.x servers talking to 2.x clients and vice versa?
> >
> > Thanks,
> >
> > Eric
> >
> > On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> > wrote:
> >
> >> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
> >> (HDFS-14667). Since this is the first of a minor release, we would like
> to
> >> get it into 2.10.0.
> >>
> >> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
> >> shortly.
> >>
> >> Jonathan Hung
> >>
> >>
> >> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
> >>
> >> > Thanks for the effort, Jonathan!
> >> >
> >> > +1 (non-binding) on RC0.
> >> >  - Set up a single node cluster with the binary tarball
> >> >  - Run Spark Pi and pySpark job
> >> >
> >> > BR,
> >> > Zhankun
> >> >
> >> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <
> shv.hadoop@gmail.com
> >> >
> >> > wrote:
> >> >
> >> >> +1 on RC0.
> >> >> - Verified signatures
> >> >> - Built from sources
> >> >> - Ran unit tests for new features
> >> >> - Checked artifacts on Nexus, made sure the sources are present.
> >> >>
> >> >> Thanks
> >> >> --Konstantin
> >> >>
> >> >>
> >> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> >> >> wrote:
> >> >>
> >> >> > Hi folks,
> >> >> >
> >> >> > This is the first release candidate for the first release of Apache
> >> >> Hadoop
> >> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
> >> includes
> >> >> > features such as:
> >> >> >
> >> >> > - User-defined resource types
> >> >> > - Native GPU support as a schedulable resource type
> >> >> > - Consistent reads from standby node
> >> >> > - Namenode port based selective encryption
> >> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >> >> >
> >> >> > The RC0 artifacts are at:
> >> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >> >> >
> >> >> > RC tag is release-2.10.0-RC0.
> >> >> >
> >> >> > The maven artifacts are hosted here:
> >> >> >
> >> >>
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >> >> >
> >> >> > My public key is available here:
> >> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >> >> >
> >> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
> >> 6:00 pm
> >> >> > PDT.
> >> >> >
> >> >> > Thanks,
> >> >> > Jonathan Hung
> >> >> >
> >> >> > [1]
> >> >> >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >> >> >
> >> >>
> >> >
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Thanks Eric! I sent out an RC1 earlier last week, not sure if you saw that.
The only diff between RC1 and RC0 is HDFS-14667. If RC1 looks good to you
then it'd be great to get your testing results on that thread.

Jonathan Hung


On Mon, Oct 28, 2019 at 1:06 PM epayne@apache.org <ep...@apache.org> wrote:

> Compatibility testing has gone well for me.
>
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.8.5 and
> 2.10.0
> - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.10.0 and
> trunk
> - With one 4-node cluster running 2.10.0 and one 4-node cluster running
> trunk, I ran a word count job in each cluster whose inputs and outputs were
> from and to the opposite cluster.
> - I verified that HDFS replication works as expected in a trunk cluster
> that has one 2.10.0 datanode.
>
> Thanks,
> -Eric
>
> On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <
> jyhung2357@gmail.com> wrote:
>
>
>
>
>
> Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
> 2.10.0 clients and datanodes. Everything worked as expected.
>
> Jonathan Hung
>
>
> On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
> wrote:
>
> > Hi Jonathan,
> >
> > Thanks for putting this RC together. You stated that there are
> > improvements related to rolling upgrades from 2.x to 3.x and I know I
> have
> > seen multiple JIRAs getting committed to that effect. Could you describe
> > any tests that you have done to verify rolling upgrade compatibility
> > for 3.x servers talking to 2.x clients and vice versa?
> >
> > Thanks,
> >
> > Eric
> >
> > On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> > wrote:
> >
> >> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
> >> (HDFS-14667). Since this is the first of a minor release, we would like
> to
> >> get it into 2.10.0.
> >>
> >> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
> >> shortly.
> >>
> >> Jonathan Hung
> >>
> >>
> >> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
> >>
> >> > Thanks for the effort, Jonathan!
> >> >
> >> > +1 (non-binding) on RC0.
> >> >  - Set up a single node cluster with the binary tarball
> >> >  - Run Spark Pi and pySpark job
> >> >
> >> > BR,
> >> > Zhankun
> >> >
> >> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <
> shv.hadoop@gmail.com
> >> >
> >> > wrote:
> >> >
> >> >> +1 on RC0.
> >> >> - Verified signatures
> >> >> - Built from sources
> >> >> - Ran unit tests for new features
> >> >> - Checked artifacts on Nexus, made sure the sources are present.
> >> >>
> >> >> Thanks
> >> >> --Konstantin
> >> >>
> >> >>
> >> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> >> >> wrote:
> >> >>
> >> >> > Hi folks,
> >> >> >
> >> >> > This is the first release candidate for the first release of Apache
> >> >> Hadoop
> >> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
> >> includes
> >> >> > features such as:
> >> >> >
> >> >> > - User-defined resource types
> >> >> > - Native GPU support as a schedulable resource type
> >> >> > - Consistent reads from standby node
> >> >> > - Namenode port based selective encryption
> >> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >> >> >
> >> >> > The RC0 artifacts are at:
> >> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >> >> >
> >> >> > RC tag is release-2.10.0-RC0.
> >> >> >
> >> >> > The maven artifacts are hosted here:
> >> >> >
> >> >>
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >> >> >
> >> >> > My public key is available here:
> >> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >> >> >
> >> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
> >> 6:00 pm
> >> >> > PDT.
> >> >> >
> >> >> > Thanks,
> >> >> > Jonathan Hung
> >> >> >
> >> >> > [1]
> >> >> >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >> >> >
> >> >>
> >> >
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

Compatibility testing has gone well for me.

- In a 4-node cluster, I ran YARN rolling upgrade tests between 2.8.5 and 2.10.0
- In a 4-node cluster, I ran YARN rolling upgrade tests between 2.10.0 and trunk
- With one 4-node cluster running 2.10.0 and one 4-node cluster running trunk, I ran a word count job in each cluster whose inputs and outputs were from and to the opposite cluster.
- I verified that HDFS replication works as expected in a trunk cluster that has one 2.10.0 datanode.

Thanks,
-Eric

On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 





Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

Compatibility testing has gone well for me.

- In a 4-node cluster, I ran YARN rolling upgrade tests between 2.8.5 and 2.10.0
- In a 4-node cluster, I ran YARN rolling upgrade tests between 2.10.0 and trunk
- With one 4-node cluster running 2.10.0 and one 4-node cluster running trunk, I ran a word count job in each cluster whose inputs and outputs were from and to the opposite cluster.
- I verified that HDFS replication works as expected in a trunk cluster that has one 2.10.0 datanode.

Thanks,
-Eric

On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 





Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

 Ah! Yes! That makes sense. I will use the mapredonhdfs framework in my next set of tests.
The other compatibility tests that I ran worked as expected.
-Eric

    On Saturday, October 26, 2019, 12:29:54 PM CDT, Jonathan Hung <jy...@gmail.com> wrote:  
 
 Hi Eric, I took a quick look, are you using mapreduce.application.framework.path to run your MR jobs? If not, this seems like expected behavior if AM and tasks get launched on different NMs with different locally installed hadoop versions?

Jonathan Hung

On Sat, Oct 26, 2019 at 8:55 AM epayne@apache.org <ep...@apache.org> wrote:

I ran a few compatibility tests between 2.10.0 and 3.3.0 (trunk)

Unfortunately, I ran into the following problem:

Running with 2.10 RM and 3.3.0 (trunk) NM fails attempts with the following error:

2019-10-26 15:44:06,885 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RPC$VersionMismatch): Protocol org.apache.hadoop.mapred.TaskUmbilicalProtocol version mismatch. (client = 19, server = 21)

The AM happened to launch on the 3.3.0 node.

Is this a protobuf issue? I thought we addressed that?

-Eric Payne



On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 





Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

  

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

 Ah! Yes! That makes sense. I will use the mapredonhdfs framework in my next set of tests.
The other compatibility tests that I ran worked as expected.
-Eric

    On Saturday, October 26, 2019, 12:29:54 PM CDT, Jonathan Hung <jy...@gmail.com> wrote:  
 
 Hi Eric, I took a quick look, are you using mapreduce.application.framework.path to run your MR jobs? If not, this seems like expected behavior if AM and tasks get launched on different NMs with different locally installed hadoop versions?

Jonathan Hung

On Sat, Oct 26, 2019 at 8:55 AM epayne@apache.org <ep...@apache.org> wrote:

I ran a few compatibility tests between 2.10.0 and 3.3.0 (trunk)

Unfortunately, I ran into the following problem:

Running with 2.10 RM and 3.3.0 (trunk) NM fails attempts with the following error:

2019-10-26 15:44:06,885 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RPC$VersionMismatch): Protocol org.apache.hadoop.mapred.TaskUmbilicalProtocol version mismatch. (client = 19, server = 21)

The AM happened to launch on the 3.3.0 node.

Is this a protobuf issue? I thought we addressed that?

-Eric Payne



On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 





Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

  

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

 Ah! Yes! That makes sense. I will use the mapredonhdfs framework in my next set of tests.
The other compatibility tests that I ran worked as expected.
-Eric

    On Saturday, October 26, 2019, 12:29:54 PM CDT, Jonathan Hung <jy...@gmail.com> wrote:  
 
 Hi Eric, I took a quick look, are you using mapreduce.application.framework.path to run your MR jobs? If not, this seems like expected behavior if AM and tasks get launched on different NMs with different locally installed hadoop versions?

Jonathan Hung

On Sat, Oct 26, 2019 at 8:55 AM epayne@apache.org <ep...@apache.org> wrote:

I ran a few compatibility tests between 2.10.0 and 3.3.0 (trunk)

Unfortunately, I ran into the following problem:

Running with 2.10 RM and 3.3.0 (trunk) NM fails attempts with the following error:

2019-10-26 15:44:06,885 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RPC$VersionMismatch): Protocol org.apache.hadoop.mapred.TaskUmbilicalProtocol version mismatch. (client = 19, server = 21)

The AM happened to launch on the 3.3.0 node.

Is this a protobuf issue? I thought we addressed that?

-Eric Payne



On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 





Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

  

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

 Ah! Yes! That makes sense. I will use the mapredonhdfs framework in my next set of tests.
The other compatibility tests that I ran worked as expected.
-Eric

    On Saturday, October 26, 2019, 12:29:54 PM CDT, Jonathan Hung <jy...@gmail.com> wrote:  
 
 Hi Eric, I took a quick look, are you using mapreduce.application.framework.path to run your MR jobs? If not, this seems like expected behavior if AM and tasks get launched on different NMs with different locally installed hadoop versions?

Jonathan Hung

On Sat, Oct 26, 2019 at 8:55 AM epayne@apache.org <ep...@apache.org> wrote:

I ran a few compatibility tests between 2.10.0 and 3.3.0 (trunk)

Unfortunately, I ran into the following problem:

Running with 2.10 RM and 3.3.0 (trunk) NM fails attempts with the following error:

2019-10-26 15:44:06,885 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RPC$VersionMismatch): Protocol org.apache.hadoop.mapred.TaskUmbilicalProtocol version mismatch. (client = 19, server = 21)

The AM happened to launch on the 3.3.0 node.

Is this a protobuf issue? I thought we addressed that?

-Eric Payne



On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 





Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

  

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Hi Eric, I took a quick look, are you using
mapreduce.application.framework.path to run your MR jobs? If not, this
seems like expected behavior if AM and tasks get launched on different NMs
with different locally installed hadoop versions?

Jonathan Hung


On Sat, Oct 26, 2019 at 8:55 AM epayne@apache.org <ep...@apache.org> wrote:

> I ran a few compatibility tests between 2.10.0 and 3.3.0 (trunk)
>
> Unfortunately, I ran into the following problem:
>
> Running with 2.10 RM and 3.3.0 (trunk) NM fails attempts with the
> following error:
>
> 2019-10-26 15:44:06,885 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child :
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RPC$VersionMismatch):
> Protocol org.apache.hadoop.mapred.TaskUmbilicalProtocol version mismatch.
> (client = 19, server = 21)
>
> The AM happened to launch on the 3.3.0 node.
>
> Is this a protobuf issue? I thought we addressed that?
>
> -Eric Payne
>
>
>
> On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <
> jyhung2357@gmail.com> wrote:
>
>
>
>
>
> Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
> 2.10.0 clients and datanodes. Everything worked as expected.
>
> Jonathan Hung
>
>
> On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
> wrote:
>
> > Hi Jonathan,
> >
> > Thanks for putting this RC together. You stated that there are
> > improvements related to rolling upgrades from 2.x to 3.x and I know I
> have
> > seen multiple JIRAs getting committed to that effect. Could you describe
> > any tests that you have done to verify rolling upgrade compatibility
> > for 3.x servers talking to 2.x clients and vice versa?
> >
> > Thanks,
> >
> > Eric
> >
> > On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> > wrote:
> >
> >> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
> >> (HDFS-14667). Since this is the first of a minor release, we would like
> to
> >> get it into 2.10.0.
> >>
> >> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
> >> shortly.
> >>
> >> Jonathan Hung
> >>
> >>
> >> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
> >>
> >> > Thanks for the effort, Jonathan!
> >> >
> >> > +1 (non-binding) on RC0.
> >> >  - Set up a single node cluster with the binary tarball
> >> >  - Run Spark Pi and pySpark job
> >> >
> >> > BR,
> >> > Zhankun
> >> >
> >> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <
> shv.hadoop@gmail.com
> >> >
> >> > wrote:
> >> >
> >> >> +1 on RC0.
> >> >> - Verified signatures
> >> >> - Built from sources
> >> >> - Ran unit tests for new features
> >> >> - Checked artifacts on Nexus, made sure the sources are present.
> >> >>
> >> >> Thanks
> >> >> --Konstantin
> >> >>
> >> >>
> >> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> >> >> wrote:
> >> >>
> >> >> > Hi folks,
> >> >> >
> >> >> > This is the first release candidate for the first release of Apache
> >> >> Hadoop
> >> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
> >> includes
> >> >> > features such as:
> >> >> >
> >> >> > - User-defined resource types
> >> >> > - Native GPU support as a schedulable resource type
> >> >> > - Consistent reads from standby node
> >> >> > - Namenode port based selective encryption
> >> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >> >> >
> >> >> > The RC0 artifacts are at:
> >> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >> >> >
> >> >> > RC tag is release-2.10.0-RC0.
> >> >> >
> >> >> > The maven artifacts are hosted here:
> >> >> >
> >> >>
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >> >> >
> >> >> > My public key is available here:
> >> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >> >> >
> >> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
> >> 6:00 pm
> >> >> > PDT.
> >> >> >
> >> >> > Thanks,
> >> >> > Jonathan Hung
> >> >> >
> >> >> > [1]
> >> >> >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >> >> >
> >> >>
> >> >
> >>
> >
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Hi Eric, I took a quick look, are you using
mapreduce.application.framework.path to run your MR jobs? If not, this
seems like expected behavior if AM and tasks get launched on different NMs
with different locally installed hadoop versions?

Jonathan Hung


On Sat, Oct 26, 2019 at 8:55 AM epayne@apache.org <ep...@apache.org> wrote:

> I ran a few compatibility tests between 2.10.0 and 3.3.0 (trunk)
>
> Unfortunately, I ran into the following problem:
>
> Running with 2.10 RM and 3.3.0 (trunk) NM fails attempts with the
> following error:
>
> 2019-10-26 15:44:06,885 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child :
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RPC$VersionMismatch):
> Protocol org.apache.hadoop.mapred.TaskUmbilicalProtocol version mismatch.
> (client = 19, server = 21)
>
> The AM happened to launch on the 3.3.0 node.
>
> Is this a protobuf issue? I thought we addressed that?
>
> -Eric Payne
>
>
>
> On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <
> jyhung2357@gmail.com> wrote:
>
>
>
>
>
> Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
> 2.10.0 clients and datanodes. Everything worked as expected.
>
> Jonathan Hung
>
>
> On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
> wrote:
>
> > Hi Jonathan,
> >
> > Thanks for putting this RC together. You stated that there are
> > improvements related to rolling upgrades from 2.x to 3.x and I know I
> have
> > seen multiple JIRAs getting committed to that effect. Could you describe
> > any tests that you have done to verify rolling upgrade compatibility
> > for 3.x servers talking to 2.x clients and vice versa?
> >
> > Thanks,
> >
> > Eric
> >
> > On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> > wrote:
> >
> >> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
> >> (HDFS-14667). Since this is the first of a minor release, we would like
> to
> >> get it into 2.10.0.
> >>
> >> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
> >> shortly.
> >>
> >> Jonathan Hung
> >>
> >>
> >> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
> >>
> >> > Thanks for the effort, Jonathan!
> >> >
> >> > +1 (non-binding) on RC0.
> >> >  - Set up a single node cluster with the binary tarball
> >> >  - Run Spark Pi and pySpark job
> >> >
> >> > BR,
> >> > Zhankun
> >> >
> >> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <
> shv.hadoop@gmail.com
> >> >
> >> > wrote:
> >> >
> >> >> +1 on RC0.
> >> >> - Verified signatures
> >> >> - Built from sources
> >> >> - Ran unit tests for new features
> >> >> - Checked artifacts on Nexus, made sure the sources are present.
> >> >>
> >> >> Thanks
> >> >> --Konstantin
> >> >>
> >> >>
> >> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> >> >> wrote:
> >> >>
> >> >> > Hi folks,
> >> >> >
> >> >> > This is the first release candidate for the first release of Apache
> >> >> Hadoop
> >> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
> >> includes
> >> >> > features such as:
> >> >> >
> >> >> > - User-defined resource types
> >> >> > - Native GPU support as a schedulable resource type
> >> >> > - Consistent reads from standby node
> >> >> > - Namenode port based selective encryption
> >> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >> >> >
> >> >> > The RC0 artifacts are at:
> >> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >> >> >
> >> >> > RC tag is release-2.10.0-RC0.
> >> >> >
> >> >> > The maven artifacts are hosted here:
> >> >> >
> >> >>
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >> >> >
> >> >> > My public key is available here:
> >> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >> >> >
> >> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
> >> 6:00 pm
> >> >> > PDT.
> >> >> >
> >> >> > Thanks,
> >> >> > Jonathan Hung
> >> >> >
> >> >> > [1]
> >> >> >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >> >> >
> >> >>
> >> >
> >>
> >
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Hi Eric, I took a quick look, are you using
mapreduce.application.framework.path to run your MR jobs? If not, this
seems like expected behavior if AM and tasks get launched on different NMs
with different locally installed hadoop versions?

Jonathan Hung


On Sat, Oct 26, 2019 at 8:55 AM epayne@apache.org <ep...@apache.org> wrote:

> I ran a few compatibility tests between 2.10.0 and 3.3.0 (trunk)
>
> Unfortunately, I ran into the following problem:
>
> Running with 2.10 RM and 3.3.0 (trunk) NM fails attempts with the
> following error:
>
> 2019-10-26 15:44:06,885 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child :
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RPC$VersionMismatch):
> Protocol org.apache.hadoop.mapred.TaskUmbilicalProtocol version mismatch.
> (client = 19, server = 21)
>
> The AM happened to launch on the 3.3.0 node.
>
> Is this a protobuf issue? I thought we addressed that?
>
> -Eric Payne
>
>
>
> On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <
> jyhung2357@gmail.com> wrote:
>
>
>
>
>
> Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
> 2.10.0 clients and datanodes. Everything worked as expected.
>
> Jonathan Hung
>
>
> On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
> wrote:
>
> > Hi Jonathan,
> >
> > Thanks for putting this RC together. You stated that there are
> > improvements related to rolling upgrades from 2.x to 3.x and I know I
> have
> > seen multiple JIRAs getting committed to that effect. Could you describe
> > any tests that you have done to verify rolling upgrade compatibility
> > for 3.x servers talking to 2.x clients and vice versa?
> >
> > Thanks,
> >
> > Eric
> >
> > On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> > wrote:
> >
> >> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
> >> (HDFS-14667). Since this is the first of a minor release, we would like
> to
> >> get it into 2.10.0.
> >>
> >> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
> >> shortly.
> >>
> >> Jonathan Hung
> >>
> >>
> >> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
> >>
> >> > Thanks for the effort, Jonathan!
> >> >
> >> > +1 (non-binding) on RC0.
> >> >  - Set up a single node cluster with the binary tarball
> >> >  - Run Spark Pi and pySpark job
> >> >
> >> > BR,
> >> > Zhankun
> >> >
> >> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <
> shv.hadoop@gmail.com
> >> >
> >> > wrote:
> >> >
> >> >> +1 on RC0.
> >> >> - Verified signatures
> >> >> - Built from sources
> >> >> - Ran unit tests for new features
> >> >> - Checked artifacts on Nexus, made sure the sources are present.
> >> >>
> >> >> Thanks
> >> >> --Konstantin
> >> >>
> >> >>
> >> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> >> >> wrote:
> >> >>
> >> >> > Hi folks,
> >> >> >
> >> >> > This is the first release candidate for the first release of Apache
> >> >> Hadoop
> >> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
> >> includes
> >> >> > features such as:
> >> >> >
> >> >> > - User-defined resource types
> >> >> > - Native GPU support as a schedulable resource type
> >> >> > - Consistent reads from standby node
> >> >> > - Namenode port based selective encryption
> >> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >> >> >
> >> >> > The RC0 artifacts are at:
> >> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >> >> >
> >> >> > RC tag is release-2.10.0-RC0.
> >> >> >
> >> >> > The maven artifacts are hosted here:
> >> >> >
> >> >>
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >> >> >
> >> >> > My public key is available here:
> >> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >> >> >
> >> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
> >> 6:00 pm
> >> >> > PDT.
> >> >> >
> >> >> > Thanks,
> >> >> > Jonathan Hung
> >> >> >
> >> >> > [1]
> >> >> >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >> >> >
> >> >>
> >> >
> >>
> >
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Hi Eric, I took a quick look, are you using
mapreduce.application.framework.path to run your MR jobs? If not, this
seems like expected behavior if AM and tasks get launched on different NMs
with different locally installed hadoop versions?

Jonathan Hung


On Sat, Oct 26, 2019 at 8:55 AM epayne@apache.org <ep...@apache.org> wrote:

> I ran a few compatibility tests between 2.10.0 and 3.3.0 (trunk)
>
> Unfortunately, I ran into the following problem:
>
> Running with 2.10 RM and 3.3.0 (trunk) NM fails attempts with the
> following error:
>
> 2019-10-26 15:44:06,885 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child :
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RPC$VersionMismatch):
> Protocol org.apache.hadoop.mapred.TaskUmbilicalProtocol version mismatch.
> (client = 19, server = 21)
>
> The AM happened to launch on the 3.3.0 node.
>
> Is this a protobuf issue? I thought we addressed that?
>
> -Eric Payne
>
>
>
> On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <
> jyhung2357@gmail.com> wrote:
>
>
>
>
>
> Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
> 2.10.0 clients and datanodes. Everything worked as expected.
>
> Jonathan Hung
>
>
> On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
> wrote:
>
> > Hi Jonathan,
> >
> > Thanks for putting this RC together. You stated that there are
> > improvements related to rolling upgrades from 2.x to 3.x and I know I
> have
> > seen multiple JIRAs getting committed to that effect. Could you describe
> > any tests that you have done to verify rolling upgrade compatibility
> > for 3.x servers talking to 2.x clients and vice versa?
> >
> > Thanks,
> >
> > Eric
> >
> > On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> > wrote:
> >
> >> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
> >> (HDFS-14667). Since this is the first of a minor release, we would like
> to
> >> get it into 2.10.0.
> >>
> >> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
> >> shortly.
> >>
> >> Jonathan Hung
> >>
> >>
> >> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
> >>
> >> > Thanks for the effort, Jonathan!
> >> >
> >> > +1 (non-binding) on RC0.
> >> >  - Set up a single node cluster with the binary tarball
> >> >  - Run Spark Pi and pySpark job
> >> >
> >> > BR,
> >> > Zhankun
> >> >
> >> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <
> shv.hadoop@gmail.com
> >> >
> >> > wrote:
> >> >
> >> >> +1 on RC0.
> >> >> - Verified signatures
> >> >> - Built from sources
> >> >> - Ran unit tests for new features
> >> >> - Checked artifacts on Nexus, made sure the sources are present.
> >> >>
> >> >> Thanks
> >> >> --Konstantin
> >> >>
> >> >>
> >> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> >> >> wrote:
> >> >>
> >> >> > Hi folks,
> >> >> >
> >> >> > This is the first release candidate for the first release of Apache
> >> >> Hadoop
> >> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
> >> includes
> >> >> > features such as:
> >> >> >
> >> >> > - User-defined resource types
> >> >> > - Native GPU support as a schedulable resource type
> >> >> > - Consistent reads from standby node
> >> >> > - Namenode port based selective encryption
> >> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >> >> >
> >> >> > The RC0 artifacts are at:
> >> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >> >> >
> >> >> > RC tag is release-2.10.0-RC0.
> >> >> >
> >> >> > The maven artifacts are hosted here:
> >> >> >
> >> >>
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >> >> >
> >> >> > My public key is available here:
> >> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >> >> >
> >> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
> >> 6:00 pm
> >> >> > PDT.
> >> >> >
> >> >> > Thanks,
> >> >> > Jonathan Hung
> >> >> >
> >> >> > [1]
> >> >> >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >> >> >
> >> >>
> >> >
> >>
> >
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

I ran a few compatibility tests between 2.10.0 and 3.3.0 (trunk)

Unfortunately, I ran into the following problem:

Running with 2.10 RM and 3.3.0 (trunk) NM fails attempts with the following error:

2019-10-26 15:44:06,885 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RPC$VersionMismatch): Protocol org.apache.hadoop.mapred.TaskUmbilicalProtocol version mismatch. (client = 19, server = 21)

The AM happened to launch on the 3.3.0 node.

Is this a protobuf issue? I thought we addressed that?

-Eric Payne



On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 





Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

I ran a few compatibility tests between 2.10.0 and 3.3.0 (trunk)

Unfortunately, I ran into the following problem:

Running with 2.10 RM and 3.3.0 (trunk) NM fails attempts with the following error:

2019-10-26 15:44:06,885 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RPC$VersionMismatch): Protocol org.apache.hadoop.mapred.TaskUmbilicalProtocol version mismatch. (client = 19, server = 21)

The AM happened to launch on the 3.3.0 node.

Is this a protobuf issue? I thought we addressed that?

-Eric Payne



On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 





Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

Compatibility testing has gone well for me.

- In a 4-node cluster, I ran YARN rolling upgrade tests between 2.8.5 and 2.10.0
- In a 4-node cluster, I ran YARN rolling upgrade tests between 2.10.0 and trunk
- With one 4-node cluster running 2.10.0 and one 4-node cluster running trunk, I ran a word count job in each cluster whose inputs and outputs were from and to the opposite cluster.
- I verified that HDFS replication works as expected in a trunk cluster that has one 2.10.0 datanode.

Thanks,
-Eric

On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 





Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

Compatibility testing has gone well for me.

- In a 4-node cluster, I ran YARN rolling upgrade tests between 2.8.5 and 2.10.0
- In a 4-node cluster, I ran YARN rolling upgrade tests between 2.10.0 and trunk
- With one 4-node cluster running 2.10.0 and one 4-node cluster running trunk, I ran a word count job in each cluster whose inputs and outputs were from and to the opposite cluster.
- I verified that HDFS replication works as expected in a trunk cluster that has one 2.10.0 datanode.

Thanks,
-Eric

On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 





Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

I ran a few compatibility tests between 2.10.0 and 3.3.0 (trunk)

Unfortunately, I ran into the following problem:

Running with 2.10 RM and 3.3.0 (trunk) NM fails attempts with the following error:

2019-10-26 15:44:06,885 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RPC$VersionMismatch): Protocol org.apache.hadoop.mapred.TaskUmbilicalProtocol version mismatch. (client = 19, server = 21)

The AM happened to launch on the 3.3.0 node.

Is this a protobuf issue? I thought we addressed that?

-Eric Payne



On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 





Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

["epayne@apache.org" <ep...@apache.org>]

I ran a few compatibility tests between 2.10.0 and 3.3.0 (trunk)

Unfortunately, I ran into the following problem:

Running with 2.10 RM and 3.3.0 (trunk) NM fails attempts with the following error:

2019-10-26 15:44:06,885 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RPC$VersionMismatch): Protocol org.apache.hadoop.mapred.TaskUmbilicalProtocol version mismatch. (client = 19, server = 21)

The AM happened to launch on the 3.3.0 node.

Is this a protobuf issue? I thought we addressed that?

-Eric Payne



On Tuesday, October 22, 2019, 8:39:38 PM CDT, Jonathan Hung <jy...@gmail.com> wrote: 





Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and
2.10.0 clients and datanodes. Everything worked as expected.

Jonathan Hung


On Tue, Oct 22, 2019 at 3:04 PM Eric Badger <eb...@verizonmedia.com>
wrote:

> Hi Jonathan,
>
> Thanks for putting this RC together. You stated that there are
> improvements related to rolling upgrades from 2.x to 3.x and I know I have
> seen multiple JIRAs getting committed to that effect. Could you describe
> any tests that you have done to verify rolling upgrade compatibility
> for 3.x servers talking to 2.x clients and vice versa?
>
> Thanks,
>
> Eric
>
> On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
>> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
>> (HDFS-14667). Since this is the first of a minor release, we would like to
>> get it into 2.10.0.
>>
>> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
>> shortly.
>>
>> Jonathan Hung
>>
>>
>> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>>
>> > Thanks for the effort, Jonathan!
>> >
>> > +1 (non-binding) on RC0.
>> >  - Set up a single node cluster with the binary tarball
>> >  - Run Spark Pi and pySpark job
>> >
>> > BR,
>> > Zhankun
>> >
>> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <shv.hadoop@gmail.com
>> >
>> > wrote:
>> >
>> >> +1 on RC0.
>> >> - Verified signatures
>> >> - Built from sources
>> >> - Ran unit tests for new features
>> >> - Checked artifacts on Nexus, made sure the sources are present.
>> >>
>> >> Thanks
>> >> --Konstantin
>> >>
>> >>
>> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi folks,
>> >> >
>> >> > This is the first release candidate for the first release of Apache
>> >> Hadoop
>> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
>> includes
>> >> > features such as:
>> >> >
>> >> > - User-defined resource types
>> >> > - Native GPU support as a schedulable resource type
>> >> > - Consistent reads from standby node
>> >> > - Namenode port based selective encryption
>> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >> >
>> >> > The RC0 artifacts are at:
>> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >> >
>> >> > RC tag is release-2.10.0-RC0.
>> >> >
>> >> > The maven artifacts are hosted here:
>> >> >
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >> >
>> >> > My public key is available here:
>> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >> >
>> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at
>> 6:00 pm
>> >> > PDT.
>> >> >
>> >> > Thanks,
>> >> > Jonathan Hung
>> >> >
>> >> > [1]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >> >
>> >>
>> >
>>
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Eric Badger <eb...@verizonmedia.com.INVALID>]

Hi Jonathan,

Thanks for putting this RC together. You stated that there are improvements
related to rolling upgrades from 2.x to 3.x and I know I have seen multiple
JIRAs getting committed to that effect. Could you describe any tests that
you have done to verify rolling upgrade compatibility for 3.x servers
talking to 2.x clients and vice versa?

Thanks,

Eric

On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com> wrote:

> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
> (HDFS-14667). Since this is the first of a minor release, we would like to
> get it into 2.10.0.
>
> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
> shortly.
>
> Jonathan Hung
>
>
> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>
> > Thanks for the effort, Jonathan!
> >
> > +1 (non-binding) on RC0.
> >  - Set up a single node cluster with the binary tarball
> >  - Run Spark Pi and pySpark job
> >
> > BR,
> > Zhankun
> >
> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <sh...@gmail.com>
> > wrote:
> >
> >> +1 on RC0.
> >> - Verified signatures
> >> - Built from sources
> >> - Ran unit tests for new features
> >> - Checked artifacts on Nexus, made sure the sources are present.
> >>
> >> Thanks
> >> --Konstantin
> >>
> >>
> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> >> wrote:
> >>
> >> > Hi folks,
> >> >
> >> > This is the first release candidate for the first release of Apache
> >> Hadoop
> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
> includes
> >> > features such as:
> >> >
> >> > - User-defined resource types
> >> > - Native GPU support as a schedulable resource type
> >> > - Consistent reads from standby node
> >> > - Namenode port based selective encryption
> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >> >
> >> > The RC0 artifacts are at:
> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >> >
> >> > RC tag is release-2.10.0-RC0.
> >> >
> >> > The maven artifacts are hosted here:
> >> >
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >> >
> >> > My public key is available here:
> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >> >
> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00
> pm
> >> > PDT.
> >> >
> >> > Thanks,
> >> > Jonathan Hung
> >> >
> >> > [1]
> >> >
> >> >
> >>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >> >
> >>
> >
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Eric Badger <eb...@verizonmedia.com.INVALID>]

Hi Jonathan,

Thanks for putting this RC together. You stated that there are improvements
related to rolling upgrades from 2.x to 3.x and I know I have seen multiple
JIRAs getting committed to that effect. Could you describe any tests that
you have done to verify rolling upgrade compatibility for 3.x servers
talking to 2.x clients and vice versa?

Thanks,

Eric

On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com> wrote:

> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
> (HDFS-14667). Since this is the first of a minor release, we would like to
> get it into 2.10.0.
>
> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
> shortly.
>
> Jonathan Hung
>
>
> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>
> > Thanks for the effort, Jonathan!
> >
> > +1 (non-binding) on RC0.
> >  - Set up a single node cluster with the binary tarball
> >  - Run Spark Pi and pySpark job
> >
> > BR,
> > Zhankun
> >
> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <sh...@gmail.com>
> > wrote:
> >
> >> +1 on RC0.
> >> - Verified signatures
> >> - Built from sources
> >> - Ran unit tests for new features
> >> - Checked artifacts on Nexus, made sure the sources are present.
> >>
> >> Thanks
> >> --Konstantin
> >>
> >>
> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> >> wrote:
> >>
> >> > Hi folks,
> >> >
> >> > This is the first release candidate for the first release of Apache
> >> Hadoop
> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
> includes
> >> > features such as:
> >> >
> >> > - User-defined resource types
> >> > - Native GPU support as a schedulable resource type
> >> > - Consistent reads from standby node
> >> > - Namenode port based selective encryption
> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >> >
> >> > The RC0 artifacts are at:
> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >> >
> >> > RC tag is release-2.10.0-RC0.
> >> >
> >> > The maven artifacts are hosted here:
> >> >
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >> >
> >> > My public key is available here:
> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >> >
> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00
> pm
> >> > PDT.
> >> >
> >> > Thanks,
> >> > Jonathan Hung
> >> >
> >> > [1]
> >> >
> >> >
> >>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >> >
> >>
> >
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Eric Badger <eb...@verizonmedia.com.INVALID>]

Hi Jonathan,

Thanks for putting this RC together. You stated that there are improvements
related to rolling upgrades from 2.x to 3.x and I know I have seen multiple
JIRAs getting committed to that effect. Could you describe any tests that
you have done to verify rolling upgrade compatibility for 3.x servers
talking to 2.x clients and vice versa?

Thanks,

Eric

On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com> wrote:

> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
> (HDFS-14667). Since this is the first of a minor release, we would like to
> get it into 2.10.0.
>
> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
> shortly.
>
> Jonathan Hung
>
>
> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>
> > Thanks for the effort, Jonathan!
> >
> > +1 (non-binding) on RC0.
> >  - Set up a single node cluster with the binary tarball
> >  - Run Spark Pi and pySpark job
> >
> > BR,
> > Zhankun
> >
> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <sh...@gmail.com>
> > wrote:
> >
> >> +1 on RC0.
> >> - Verified signatures
> >> - Built from sources
> >> - Ran unit tests for new features
> >> - Checked artifacts on Nexus, made sure the sources are present.
> >>
> >> Thanks
> >> --Konstantin
> >>
> >>
> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> >> wrote:
> >>
> >> > Hi folks,
> >> >
> >> > This is the first release candidate for the first release of Apache
> >> Hadoop
> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
> includes
> >> > features such as:
> >> >
> >> > - User-defined resource types
> >> > - Native GPU support as a schedulable resource type
> >> > - Consistent reads from standby node
> >> > - Namenode port based selective encryption
> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >> >
> >> > The RC0 artifacts are at:
> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >> >
> >> > RC tag is release-2.10.0-RC0.
> >> >
> >> > The maven artifacts are hosted here:
> >> >
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >> >
> >> > My public key is available here:
> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >> >
> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00
> pm
> >> > PDT.
> >> >
> >> > Thanks,
> >> > Jonathan Hung
> >> >
> >> > [1]
> >> >
> >> >
> >>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >> >
> >>
> >
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Eric Badger <eb...@verizonmedia.com.INVALID>]

Hi Jonathan,

Thanks for putting this RC together. You stated that there are improvements
related to rolling upgrades from 2.x to 3.x and I know I have seen multiple
JIRAs getting committed to that effect. Could you describe any tests that
you have done to verify rolling upgrade compatibility for 3.x servers
talking to 2.x clients and vice versa?

Thanks,

Eric

On Tue, Oct 22, 2019 at 1:49 PM Jonathan Hung <jy...@gmail.com> wrote:

> Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
> (HDFS-14667). Since this is the first of a minor release, we would like to
> get it into 2.10.0.
>
> HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
> shortly.
>
> Jonathan Hung
>
>
> On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:
>
> > Thanks for the effort, Jonathan!
> >
> > +1 (non-binding) on RC0.
> >  - Set up a single node cluster with the binary tarball
> >  - Run Spark Pi and pySpark job
> >
> > BR,
> > Zhankun
> >
> > On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <sh...@gmail.com>
> > wrote:
> >
> >> +1 on RC0.
> >> - Verified signatures
> >> - Built from sources
> >> - Ran unit tests for new features
> >> - Checked artifacts on Nexus, made sure the sources are present.
> >>
> >> Thanks
> >> --Konstantin
> >>
> >>
> >> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> >> wrote:
> >>
> >> > Hi folks,
> >> >
> >> > This is the first release candidate for the first release of Apache
> >> Hadoop
> >> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It
> includes
> >> > features such as:
> >> >
> >> > - User-defined resource types
> >> > - Native GPU support as a schedulable resource type
> >> > - Consistent reads from standby node
> >> > - Namenode port based selective encryption
> >> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >> >
> >> > The RC0 artifacts are at:
> >> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >> >
> >> > RC tag is release-2.10.0-RC0.
> >> >
> >> > The maven artifacts are hosted here:
> >> >
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >> >
> >> > My public key is available here:
> >> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >> >
> >> > The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00
> pm
> >> > PDT.
> >> >
> >> > Thanks,
> >> > Jonathan Hung
> >> >
> >> > [1]
> >> >
> >> >
> >>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >> >
> >>
> >
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
(HDFS-14667). Since this is the first of a minor release, we would like to
get it into 2.10.0.

HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
shortly.

Jonathan Hung


On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:

> Thanks for the effort, Jonathan!
>
> +1 (non-binding) on RC0.
>  - Set up a single node cluster with the binary tarball
>  - Run Spark Pi and pySpark job
>
> BR,
> Zhankun
>
> On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <sh...@gmail.com>
> wrote:
>
>> +1 on RC0.
>> - Verified signatures
>> - Built from sources
>> - Ran unit tests for new features
>> - Checked artifacts on Nexus, made sure the sources are present.
>>
>> Thanks
>> --Konstantin
>>
>>
>> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> wrote:
>>
>> > Hi folks,
>> >
>> > This is the first release candidate for the first release of Apache
>> Hadoop
>> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It includes
>> > features such as:
>> >
>> > - User-defined resource types
>> > - Native GPU support as a schedulable resource type
>> > - Consistent reads from standby node
>> > - Namenode port based selective encryption
>> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >
>> > The RC0 artifacts are at:
>> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >
>> > RC tag is release-2.10.0-RC0.
>> >
>> > The maven artifacts are hosted here:
>> >
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >
>> > My public key is available here:
>> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >
>> > The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm
>> > PDT.
>> >
>> > Thanks,
>> > Jonathan Hung
>> >
>> > [1]
>> >
>> >
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >
>>
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
(HDFS-14667). Since this is the first of a minor release, we would like to
get it into 2.10.0.

HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
shortly.

Jonathan Hung


On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:

> Thanks for the effort, Jonathan!
>
> +1 (non-binding) on RC0.
>  - Set up a single node cluster with the binary tarball
>  - Run Spark Pi and pySpark job
>
> BR,
> Zhankun
>
> On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <sh...@gmail.com>
> wrote:
>
>> +1 on RC0.
>> - Verified signatures
>> - Built from sources
>> - Ran unit tests for new features
>> - Checked artifacts on Nexus, made sure the sources are present.
>>
>> Thanks
>> --Konstantin
>>
>>
>> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> wrote:
>>
>> > Hi folks,
>> >
>> > This is the first release candidate for the first release of Apache
>> Hadoop
>> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It includes
>> > features such as:
>> >
>> > - User-defined resource types
>> > - Native GPU support as a schedulable resource type
>> > - Consistent reads from standby node
>> > - Namenode port based selective encryption
>> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >
>> > The RC0 artifacts are at:
>> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >
>> > RC tag is release-2.10.0-RC0.
>> >
>> > The maven artifacts are hosted here:
>> >
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >
>> > My public key is available here:
>> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >
>> > The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm
>> > PDT.
>> >
>> > Thanks,
>> > Jonathan Hung
>> >
>> > [1]
>> >
>> >
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >
>>
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
(HDFS-14667). Since this is the first of a minor release, we would like to
get it into 2.10.0.

HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
shortly.

Jonathan Hung


On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:

> Thanks for the effort, Jonathan!
>
> +1 (non-binding) on RC0.
>  - Set up a single node cluster with the binary tarball
>  - Run Spark Pi and pySpark job
>
> BR,
> Zhankun
>
> On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <sh...@gmail.com>
> wrote:
>
>> +1 on RC0.
>> - Verified signatures
>> - Built from sources
>> - Ran unit tests for new features
>> - Checked artifacts on Nexus, made sure the sources are present.
>>
>> Thanks
>> --Konstantin
>>
>>
>> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> wrote:
>>
>> > Hi folks,
>> >
>> > This is the first release candidate for the first release of Apache
>> Hadoop
>> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It includes
>> > features such as:
>> >
>> > - User-defined resource types
>> > - Native GPU support as a schedulable resource type
>> > - Consistent reads from standby node
>> > - Namenode port based selective encryption
>> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >
>> > The RC0 artifacts are at:
>> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >
>> > RC tag is release-2.10.0-RC0.
>> >
>> > The maven artifacts are hosted here:
>> >
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >
>> > My public key is available here:
>> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >
>> > The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm
>> > PDT.
>> >
>> > Thanks,
>> > Jonathan Hung
>> >
>> > [1]
>> >
>> >
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >
>>
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Jonathan Hung <jy...@gmail.com>]

Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar
(HDFS-14667). Since this is the first of a minor release, we would like to
get it into 2.10.0.

HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1
shortly.

Jonathan Hung


On Tue, Oct 22, 2019 at 1:39 AM Zhankun Tang <zt...@apache.org> wrote:

> Thanks for the effort, Jonathan!
>
> +1 (non-binding) on RC0.
>  - Set up a single node cluster with the binary tarball
>  - Run Spark Pi and pySpark job
>
> BR,
> Zhankun
>
> On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <sh...@gmail.com>
> wrote:
>
>> +1 on RC0.
>> - Verified signatures
>> - Built from sources
>> - Ran unit tests for new features
>> - Checked artifacts on Nexus, made sure the sources are present.
>>
>> Thanks
>> --Konstantin
>>
>>
>> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
>> wrote:
>>
>> > Hi folks,
>> >
>> > This is the first release candidate for the first release of Apache
>> Hadoop
>> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It includes
>> > features such as:
>> >
>> > - User-defined resource types
>> > - Native GPU support as a schedulable resource type
>> > - Consistent reads from standby node
>> > - Namenode port based selective encryption
>> > - Improvements related to rolling upgrade support from 2.x to 3.x
>> >
>> > The RC0 artifacts are at:
>> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>> >
>> > RC tag is release-2.10.0-RC0.
>> >
>> > The maven artifacts are hosted here:
>> >
>> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>> >
>> > My public key is available here:
>> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >
>> > The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm
>> > PDT.
>> >
>> > Thanks,
>> > Jonathan Hung
>> >
>> > [1]
>> >
>> >
>> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>> >
>>
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Zhankun Tang <zt...@apache.org>]

Thanks for the effort, Jonathan!

+1 (non-binding) on RC0.
 - Set up a single node cluster with the binary tarball
 - Run Spark Pi and pySpark job

BR,
Zhankun

On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <sh...@gmail.com>
wrote:

> +1 on RC0.
> - Verified signatures
> - Built from sources
> - Ran unit tests for new features
> - Checked artifacts on Nexus, made sure the sources are present.
>
> Thanks
> --Konstantin
>
>
> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
> > Hi folks,
> >
> > This is the first release candidate for the first release of Apache
> Hadoop
> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It includes
> > features such as:
> >
> > - User-defined resource types
> > - Native GPU support as a schedulable resource type
> > - Consistent reads from standby node
> > - Namenode port based selective encryption
> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >
> > The RC0 artifacts are at:
> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >
> > RC tag is release-2.10.0-RC0.
> >
> > The maven artifacts are hosted here:
> > https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >
> > My public key is available here:
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> > The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm
> > PDT.
> >
> > Thanks,
> > Jonathan Hung
> >
> > [1]
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Zhankun Tang <zt...@apache.org>]

Thanks for the effort, Jonathan!

+1 (non-binding) on RC0.
 - Set up a single node cluster with the binary tarball
 - Run Spark Pi and pySpark job

BR,
Zhankun

On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <sh...@gmail.com>
wrote:

> +1 on RC0.
> - Verified signatures
> - Built from sources
> - Ran unit tests for new features
> - Checked artifacts on Nexus, made sure the sources are present.
>
> Thanks
> --Konstantin
>
>
> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
> > Hi folks,
> >
> > This is the first release candidate for the first release of Apache
> Hadoop
> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It includes
> > features such as:
> >
> > - User-defined resource types
> > - Native GPU support as a schedulable resource type
> > - Consistent reads from standby node
> > - Namenode port based selective encryption
> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >
> > The RC0 artifacts are at:
> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >
> > RC tag is release-2.10.0-RC0.
> >
> > The maven artifacts are hosted here:
> > https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >
> > My public key is available here:
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> > The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm
> > PDT.
> >
> > Thanks,
> > Jonathan Hung
> >
> > [1]
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Zhankun Tang <zt...@apache.org>]

Thanks for the effort, Jonathan!

+1 (non-binding) on RC0.
 - Set up a single node cluster with the binary tarball
 - Run Spark Pi and pySpark job

BR,
Zhankun

On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <sh...@gmail.com>
wrote:

> +1 on RC0.
> - Verified signatures
> - Built from sources
> - Ran unit tests for new features
> - Checked artifacts on Nexus, made sure the sources are present.
>
> Thanks
> --Konstantin
>
>
> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
> > Hi folks,
> >
> > This is the first release candidate for the first release of Apache
> Hadoop
> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It includes
> > features such as:
> >
> > - User-defined resource types
> > - Native GPU support as a schedulable resource type
> > - Consistent reads from standby node
> > - Namenode port based selective encryption
> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >
> > The RC0 artifacts are at:
> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >
> > RC tag is release-2.10.0-RC0.
> >
> > The maven artifacts are hosted here:
> > https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >
> > My public key is available here:
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> > The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm
> > PDT.
> >
> > Thanks,
> > Jonathan Hung
> >
> > [1]
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Zhankun Tang <zt...@apache.org>]

Thanks for the effort, Jonathan!

+1 (non-binding) on RC0.
 - Set up a single node cluster with the binary tarball
 - Run Spark Pi and pySpark job

BR,
Zhankun

On Tue, 22 Oct 2019 at 14:31, Konstantin Shvachko <sh...@gmail.com>
wrote:

> +1 on RC0.
> - Verified signatures
> - Built from sources
> - Ran unit tests for new features
> - Checked artifacts on Nexus, made sure the sources are present.
>
> Thanks
> --Konstantin
>
>
> On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com>
> wrote:
>
> > Hi folks,
> >
> > This is the first release candidate for the first release of Apache
> Hadoop
> > 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It includes
> > features such as:
> >
> > - User-defined resource types
> > - Native GPU support as a schedulable resource type
> > - Consistent reads from standby node
> > - Namenode port based selective encryption
> > - Improvements related to rolling upgrade support from 2.x to 3.x
> >
> > The RC0 artifacts are at:
> http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
> >
> > RC tag is release-2.10.0-RC0.
> >
> > The maven artifacts are hosted here:
> > https://repository.apache.org/content/repositories/orgapachehadoop-1241/
> >
> > My public key is available here:
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> > The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm
> > PDT.
> >
> > Thanks,
> > Jonathan Hung
> >
> > [1]
> >
> >
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
> >
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Konstantin Shvachko <sh...@gmail.com>]

+1 on RC0.
- Verified signatures
- Built from sources
- Ran unit tests for new features
- Checked artifacts on Nexus, made sure the sources are present.

Thanks
--Konstantin


On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com> wrote:

> Hi folks,
>
> This is the first release candidate for the first release of Apache Hadoop
> 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It includes
> features such as:
>
> - User-defined resource types
> - Native GPU support as a schedulable resource type
> - Consistent reads from standby node
> - Namenode port based selective encryption
> - Improvements related to rolling upgrade support from 2.x to 3.x
>
> The RC0 artifacts are at: http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>
> RC tag is release-2.10.0-RC0.
>
> The maven artifacts are hosted here:
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>
> My public key is available here:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm
> PDT.
>
> Thanks,
> Jonathan Hung
>
> [1]
>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Konstantin Shvachko <sh...@gmail.com>]

+1 on RC0.
- Verified signatures
- Built from sources
- Ran unit tests for new features
- Checked artifacts on Nexus, made sure the sources are present.

Thanks
--Konstantin


On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com> wrote:

> Hi folks,
>
> This is the first release candidate for the first release of Apache Hadoop
> 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It includes
> features such as:
>
> - User-defined resource types
> - Native GPU support as a schedulable resource type
> - Consistent reads from standby node
> - Namenode port based selective encryption
> - Improvements related to rolling upgrade support from 2.x to 3.x
>
> The RC0 artifacts are at: http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>
> RC tag is release-2.10.0-RC0.
>
> The maven artifacts are hosted here:
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>
> My public key is available here:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm
> PDT.
>
> Thanks,
> Jonathan Hung
>
> [1]
>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Konstantin Shvachko <sh...@gmail.com>]

+1 on RC0.
- Verified signatures
- Built from sources
- Ran unit tests for new features
- Checked artifacts on Nexus, made sure the sources are present.

Thanks
--Konstantin


On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com> wrote:

> Hi folks,
>
> This is the first release candidate for the first release of Apache Hadoop
> 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It includes
> features such as:
>
> - User-defined resource types
> - Native GPU support as a schedulable resource type
> - Consistent reads from standby node
> - Namenode port based selective encryption
> - Improvements related to rolling upgrade support from 2.x to 3.x
>
> The RC0 artifacts are at: http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>
> RC tag is release-2.10.0-RC0.
>
> The maven artifacts are hosted here:
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>
> My public key is available here:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm
> PDT.
>
> Thanks,
> Jonathan Hung
>
> [1]
>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

[Konstantin Shvachko <sh...@gmail.com>]

+1 on RC0.
- Verified signatures
- Built from sources
- Ran unit tests for new features
- Checked artifacts on Nexus, made sure the sources are present.

Thanks
--Konstantin


On Wed, Oct 16, 2019 at 6:01 PM Jonathan Hung <jy...@gmail.com> wrote:

> Hi folks,
>
> This is the first release candidate for the first release of Apache Hadoop
> 2.10 line. It contains 361 fixes/improvements since 2.9 [1]. It includes
> features such as:
>
> - User-defined resource types
> - Native GPU support as a schedulable resource type
> - Consistent reads from standby node
> - Namenode port based selective encryption
> - Improvements related to rolling upgrade support from 2.x to 3.x
>
> The RC0 artifacts are at: http://home.apache.org/~jhung/hadoop-2.10.0-RC0/
>
> RC tag is release-2.10.0-RC0.
>
> The maven artifacts are hosted here:
> https://repository.apache.org/content/repositories/orgapachehadoop-1241/
>
> My public key is available here:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm
> PDT.
>
> Thanks,
> Jonathan Hung
>
> [1]
>
> https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.0%20AND%20fixVersion%20not%20in%20(2.9.2%2C%202.9.1%2C%202.9.0)
>