You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Wei-Chiu Chuang <we...@apache.org> on 2021/01/28 01:28:16 UTC
[DISCUSS] Hadoop 3.3.1 release
Hi all,
Hadoop 3.3.0 was released half a year ago, and as of now we've accumulated
more than 400 changes in the branch-3.3. A number of downstreamers are
eagerly waiting for 3.3.1 which addresses the guava version conflict issue.
https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
We should start the release work for 3.3.1 before the diff becomes even
larger.
I believe there are currently only two real blockers for a 3.3.1 (using
this filter
https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
)
1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
2.
1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112>
2.
Is there anyone who would volunteer to be the 3.3.1 RM?
Also, the HowToRelease wiki does not describe the ARM build process. That's
going to be important for future releases.
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Without looking into details, I would say that new features like IPv6
support should get into 3.4.0 (and hopefully spend a little time to
stabilize) before backporting to 3.3.x or lower branches.
On Thu, Mar 11, 2021 at 1:21 AM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> I'm going to argue its too late to do IPv6 support close to a release, as
> it's best if its on developer machines for some time to let all quirks
> surface. It's not so much IPv6 itself, but do we cause any regressions on
> IPv4?
>
> But: it can/should go into trunk and stabilize there
>
> On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> muralikrishna.dmmkr@gmail.com> wrote:
>
> > Hi Brahma,
> >
> > I have missed out mentioning about the IPV6 feature in the last mail,
> > Support for IPV6 has been in development since 2015 and We have done a
> good
> > amount of testing at our organisation, the feature is stable and used by
> > our customers extensively in the last one year. I think it is a good time
> > to add the IPV6 support to 3.3.1.
> >
> > https://issues.apache.org/jira/browse/HADOOP-11890
> >
> > Thanks
> > D M Murali Krishna Reddy
> >
> > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > muralikrishna.dmmkr@gmail.com> wrote:
> >
> > > Hi Brahma,
> > >
> > > Can we have this new feature "YARN Registry based AM discovery with
> retry
> > > and in-flight task persistent via JHS" in the upcoming 3.1.1 release. I
> > > have also attached a test-report in the below jira.
> > >
> > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > >
> > >
> > > Thanks,
> > > D M Murali Krishna Reddy
> > >
> > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > > wrote:
> > >
> > >> Hi Bilwa,
> > >>
> > >> I have commented on the jira's you mentioned. Based on the stability
> we
> > >> can
> > >> plan this.But needs to be merged ASAP.
> > >>
> > >>
> > >>
> > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> > >>
> > >> > Hi Brahma,
> > >> >
> > >> > Can we have below features in 3.3.1 release? We have been using
> these
> > >> > features for a long time. They are stable and tested in bigger
> > clusters.
> > >> >
> > >> > 1. Container reuse -
> > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > >> > 2. Speculative attempts should not run on the same node -
> > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > >> >
> > >> > Thanks,
> > >> > Bilwa
> > >> >
> > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > >> > wrote:
> > >> >
> > >> >> Sorry for the late reply..
> > >> >>
> > >> >> I will come up with a plan.. Please let me know if anybody has some
> > >> >> features/improvements/bugs that need to be included.
> > >> >>
> > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org>
> > >> wrote:
> > >> >>
> > >> >> > Hi Wei-Chiu,
> > >> >> >
> > >> >> > What will be the next steps here for 3.3.1 planning?
> > >> >> >
> > >> >> > Thanks
> > >> >> > Sunil
> > >> >> >
> > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> > >> >> >
> > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > >> >> > <stevel@cloudera.com.invalid
> > >> >> > > >
> > >> >> > > wrote:
> > >> >> > >
> > >> >> > > >
> > >> >> > > > Regarding blockers &c: how about we have a little hackathon
> > >> where we
> > >> >> > try
> > >> >> > > > and get things in. This means a promise of review time from
> the
> > >> >> people
> > >> >> > > with
> > >> >> > > > commit rights and other people who understand the code
> (Stack?)
> > >> >> > > >
> > >> >> > > >
> > >> >> > >
> > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> testing).
> > >> >> > > Thanks,
> > >> >> > > S
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > > > -steve
> > >> >> > > >
> > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> ayushtkn@gmail.com
> > >
> > >> >> wrote:
> > >> >> > > >
> > >> >> > > > > +1
> > >> >> > > > > Just to mention we would need to release hadoop-thirdparty
> > too
> > >> >> > before.
> > >> >> > > > > Presently we are using the snapshot version of it.
> > >> >> > > > >
> > >> >> > > > > -Ayush
> > >> >> > > > >
> > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > >> weichiu@apache.org
> > >> >> >
> > >> >> > > > wrote:
> > >> >> > > > > >
> > >> >> > > > > > Hi all,
> > >> >> > > > > >
> > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now
> > >> we've
> > >> >> > > > > accumulated
> > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > >> >> downstreamers
> > >> >> > > are
> > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> version
> > >> >> > conflict
> > >> >> > > > > issue.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > >> >> > > > > >
> > >> >> > > > > > We should start the release work for 3.3.1 before the
> diff
> > >> >> becomes
> > >> >> > > even
> > >> >> > > > > > larger.
> > >> >> > > > > >
> > >> >> > > > > > I believe there are currently only two real blockers
> for a
> > >> >> 3.3.1
> > >> >> > > > (using
> > >> >> > > > > > this filter
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > >> >> > > > > > )
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > 1. HDFS-15566 <
> > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > >> >> > > > > > 2.
> > >> >> > > > > > 1. HADOOP-17112 <
> > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > >> >> > > > > >
> > >> >> > > > > > 2.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > >> >> > > > > >
> > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> build
> > >> >> > process.
> > >> >> > > > > That's
> > >> >> > > > > > going to be important for future releases.
> > >> >> > > > >
> > >> >> > > > >
> > >> >>
> ---------------------------------------------------------------------
> > >> >> > > > > To unsubscribe, e-mail:
> > hdfs-dev-unsubscribe@hadoop.apache.org
> > >> >> > > > > For additional commands, e-mail:
> > >> hdfs-dev-help@hadoop.apache.org
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >>
> > >> >> --
> > >> >>
> > >> >>
> > >> >>
> > >> >> --Brahma Reddy Battula
> > >> >>
> > >> >
> > >>
> > >> --
> > >>
> > >>
> > >>
> > >> --Brahma Reddy Battula
> > >>
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Without looking into details, I would say that new features like IPv6
support should get into 3.4.0 (and hopefully spend a little time to
stabilize) before backporting to 3.3.x or lower branches.
On Thu, Mar 11, 2021 at 1:21 AM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> I'm going to argue its too late to do IPv6 support close to a release, as
> it's best if its on developer machines for some time to let all quirks
> surface. It's not so much IPv6 itself, but do we cause any regressions on
> IPv4?
>
> But: it can/should go into trunk and stabilize there
>
> On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> muralikrishna.dmmkr@gmail.com> wrote:
>
> > Hi Brahma,
> >
> > I have missed out mentioning about the IPV6 feature in the last mail,
> > Support for IPV6 has been in development since 2015 and We have done a
> good
> > amount of testing at our organisation, the feature is stable and used by
> > our customers extensively in the last one year. I think it is a good time
> > to add the IPV6 support to 3.3.1.
> >
> > https://issues.apache.org/jira/browse/HADOOP-11890
> >
> > Thanks
> > D M Murali Krishna Reddy
> >
> > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > muralikrishna.dmmkr@gmail.com> wrote:
> >
> > > Hi Brahma,
> > >
> > > Can we have this new feature "YARN Registry based AM discovery with
> retry
> > > and in-flight task persistent via JHS" in the upcoming 3.1.1 release. I
> > > have also attached a test-report in the below jira.
> > >
> > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > >
> > >
> > > Thanks,
> > > D M Murali Krishna Reddy
> > >
> > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > > wrote:
> > >
> > >> Hi Bilwa,
> > >>
> > >> I have commented on the jira's you mentioned. Based on the stability
> we
> > >> can
> > >> plan this.But needs to be merged ASAP.
> > >>
> > >>
> > >>
> > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> > >>
> > >> > Hi Brahma,
> > >> >
> > >> > Can we have below features in 3.3.1 release? We have been using
> these
> > >> > features for a long time. They are stable and tested in bigger
> > clusters.
> > >> >
> > >> > 1. Container reuse -
> > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > >> > 2. Speculative attempts should not run on the same node -
> > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > >> >
> > >> > Thanks,
> > >> > Bilwa
> > >> >
> > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > >> > wrote:
> > >> >
> > >> >> Sorry for the late reply..
> > >> >>
> > >> >> I will come up with a plan.. Please let me know if anybody has some
> > >> >> features/improvements/bugs that need to be included.
> > >> >>
> > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org>
> > >> wrote:
> > >> >>
> > >> >> > Hi Wei-Chiu,
> > >> >> >
> > >> >> > What will be the next steps here for 3.3.1 planning?
> > >> >> >
> > >> >> > Thanks
> > >> >> > Sunil
> > >> >> >
> > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> > >> >> >
> > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > >> >> > <stevel@cloudera.com.invalid
> > >> >> > > >
> > >> >> > > wrote:
> > >> >> > >
> > >> >> > > >
> > >> >> > > > Regarding blockers &c: how about we have a little hackathon
> > >> where we
> > >> >> > try
> > >> >> > > > and get things in. This means a promise of review time from
> the
> > >> >> people
> > >> >> > > with
> > >> >> > > > commit rights and other people who understand the code
> (Stack?)
> > >> >> > > >
> > >> >> > > >
> > >> >> > >
> > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> testing).
> > >> >> > > Thanks,
> > >> >> > > S
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > > > -steve
> > >> >> > > >
> > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> ayushtkn@gmail.com
> > >
> > >> >> wrote:
> > >> >> > > >
> > >> >> > > > > +1
> > >> >> > > > > Just to mention we would need to release hadoop-thirdparty
> > too
> > >> >> > before.
> > >> >> > > > > Presently we are using the snapshot version of it.
> > >> >> > > > >
> > >> >> > > > > -Ayush
> > >> >> > > > >
> > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > >> weichiu@apache.org
> > >> >> >
> > >> >> > > > wrote:
> > >> >> > > > > >
> > >> >> > > > > > Hi all,
> > >> >> > > > > >
> > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now
> > >> we've
> > >> >> > > > > accumulated
> > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > >> >> downstreamers
> > >> >> > > are
> > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> version
> > >> >> > conflict
> > >> >> > > > > issue.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > >> >> > > > > >
> > >> >> > > > > > We should start the release work for 3.3.1 before the
> diff
> > >> >> becomes
> > >> >> > > even
> > >> >> > > > > > larger.
> > >> >> > > > > >
> > >> >> > > > > > I believe there are currently only two real blockers
> for a
> > >> >> 3.3.1
> > >> >> > > > (using
> > >> >> > > > > > this filter
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > >> >> > > > > > )
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > 1. HDFS-15566 <
> > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > >> >> > > > > > 2.
> > >> >> > > > > > 1. HADOOP-17112 <
> > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > >> >> > > > > >
> > >> >> > > > > > 2.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > >> >> > > > > >
> > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> build
> > >> >> > process.
> > >> >> > > > > That's
> > >> >> > > > > > going to be important for future releases.
> > >> >> > > > >
> > >> >> > > > >
> > >> >>
> ---------------------------------------------------------------------
> > >> >> > > > > To unsubscribe, e-mail:
> > hdfs-dev-unsubscribe@hadoop.apache.org
> > >> >> > > > > For additional commands, e-mail:
> > >> hdfs-dev-help@hadoop.apache.org
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >>
> > >> >> --
> > >> >>
> > >> >>
> > >> >>
> > >> >> --Brahma Reddy Battula
> > >> >>
> > >> >
> > >>
> > >> --
> > >>
> > >>
> > >>
> > >> --Brahma Reddy Battula
> > >>
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Back at this. The Hadoop thirdparty 1.1.0 is released.
We have one real blocker for the 3.3.1 release (HDFS-15790 Make
ProtobufRpcEngineProtos and ProtobufRpcEngineProtos2 Co-Exist
<https://issues.apache.org/jira/browse/HDFS-15790>)
I tried to rebase the PR but it didn't work like I wanted. Some help is
needed there. If either Vinay comes back to rebase the PR, or if someone
else can take over the PR, that would be great.
Meanwhile, I'll cut a branch for 3.3.1 later. If everything works out fine,
I should be able to make a RC0 by tonight (which doesn't contain
HDFS-15790) so you guys can quickly check out.
The first RC will probably not have the aarch64 binary. I don't have much
experience with that.
Cheers,
Wei-Chiu
On Thu, May 13, 2021 at 9:04 AM Wei-Chiu Chuang <we...@cloudera.com>
wrote:
> Hello it's me again.
>
> While working on the hadoop-thirdparty release, I would like to make
> progress on the Hadoop 3.3.1 release in parallel as well.
> Looking at the jira dashboard, I think the only real blocker is HDFS-15790
> https://github.com/apache/hadoop/pull/2767
>
> I'll try to review the remaining jiras and push out those that don't have
> much progress today.
>
> Thanks all!
> Weichiu
>
>
> On Tue, Apr 20, 2021 at 12:57 PM Wei-Chiu Chuang <we...@cloudera.com>
> wrote:
>
>> Billie and Viraj,
>>
>> As of today we are at 657 resolved jiras in 3.3.1
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20status%20in%20(Resolved)%20AND%20fixVersion%20in%20(3.3.1)
>> There are so many new features (
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.3+Release) I
>> counted 10, that it is barely a dot release.
>>
>> Please make every effort to backport the said jiras to branch-3.3. I'd
>> like to cut the branch sooner than later. I'd also call for more frequent
>> releases. At the current development pace, a dot release every quarter
>> would make sense IMO.
>>
>> Given the number of release blockers, I propose a tentative code freeze
>> date: end of next Friday. After that I'd like to cut the branch and make an
>> RC.
>>
>> Thoughts? I can't do this all alone. If you find things that are worth
>> stopping the release, please tag the jira with the label 'release-blocker'.
>>
>>
>>
>> On Tue, Apr 20, 2021 at 2:44 AM Billie Rinaldi <bi...@apache.org> wrote:
>>
>>> I was thinking of backporting HADOOP-16948 for 3.3.1.
>>>
>>> Billie
>>>
>>> On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
>>> <we...@cloudera.com.invalid> wrote:
>>>
>>> > Hello, reviving this thread.
>>> >
>>> > I created a dashboard for Hadoop 3.3.1 release.
>>> >
>>> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
>>> > Also a jira to track the release work: HADOOP-17647
>>> > <https://issues.apache.org/jira/browse/HADOOP-17647>
>>> >
>>> > We are currently at 5 release blockers and 3 critical issues for Hadoop
>>> > 3.3.1. I'll go through each of them and push out the ones that aren't
>>> > really blocking us.
>>> >
>>> > If you believe there are more features/bug fixes we should include in
>>> 3.3.1
>>> > (I spent the past few weeks backporting jiras but I'm sure I missed
>>> some)
>>> > please shout out.
>>> >
>>> > Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
>>> There
>>> > are a number of tasks to be done there too. Let's start another thread
>>> for
>>> > hadoop-thirdparty 1.1.0 release.
>>> >
>>> > On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <
>>> hemanthboyina224@gmail.com
>>> > >
>>> > wrote:
>>> >
>>> > > Hi Steve and Wei-Chiu
>>> > >
>>> > > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
>>> > trunk
>>> > > and tried to work out with IPV6, we have faced some issues and have
>>> made
>>> > > the required changes for ipv6 to work.After the changes were made we
>>> have
>>> > > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
>>> > > rigorously.Its been quite a some time these changes were deployed in
>>> > > production cluster and have been in use for extensive purpose.
>>> > >
>>> > > I think it's good time to add this feature.
>>> > >
>>> > > Thanks
>>> > > Hemanth Boyina
>>> > >
>>> > >
>>> > >
>>> > > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
>>> > wrote:
>>> > >
>>> > > > Hi David,
>>> > > >
>>> > > > >> Still hoping for help here:
>>> > > >
>>> > > > >> https://issues.apache.org/jira/browse/HDFS-15790
>>> > > >
>>> > > > I will raise a PR for the said solution soon (in a day or two).
>>> > > >
>>> > > > -Vinay
>>> > > >
>>> > > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
>>> > > >
>>> > > > > Hello,
>>> > > > >
>>> > > > > Still hoping for help here:
>>> > > > >
>>> > > > > https://issues.apache.org/jira/browse/HDFS-15790
>>> > > > >
>>> > > > > Looks like it has been worked on, not sure how to best move it
>>> > forward.
>>> > > > >
>>> > > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
>>> > > > <stevel@cloudera.com.invalid
>>> > > > > >
>>> > > > > wrote:
>>> > > > >
>>> > > > > > I'm going to argue its too late to do IPv6 support close to a
>>> > > release,
>>> > > > as
>>> > > > > > it's best if its on developer machines for some time to let all
>>> > > quirks
>>> > > > > > surface. It's not so much IPv6 itself, but do we cause any
>>> > > regressions
>>> > > > on
>>> > > > > > IPv4?
>>> > > > > >
>>> > > > > > But: it can/should go into trunk and stabilize there
>>> > > > > >
>>> > > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
>>> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>>> > > > > >
>>> > > > > > > Hi Brahma,
>>> > > > > > >
>>> > > > > > > I have missed out mentioning about the IPV6 feature in the
>>> last
>>> > > mail,
>>> > > > > > > Support for IPV6 has been in development since 2015 and We
>>> have
>>> > > done
>>> > > > a
>>> > > > > > good
>>> > > > > > > amount of testing at our organisation, the feature is stable
>>> and
>>> > > used
>>> > > > > by
>>> > > > > > > our customers extensively in the last one year. I think it
>>> is a
>>> > > good
>>> > > > > time
>>> > > > > > > to add the IPV6 support to 3.3.1.
>>> > > > > > >
>>> > > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
>>> > > > > > >
>>> > > > > > > Thanks
>>> > > > > > > D M Murali Krishna Reddy
>>> > > > > > >
>>> > > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
>>> > > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>>> > > > > > >
>>> > > > > > > > Hi Brahma,
>>> > > > > > > >
>>> > > > > > > > Can we have this new feature "YARN Registry based AM
>>> discovery
>>> > > with
>>> > > > > > retry
>>> > > > > > > > and in-flight task persistent via JHS" in the upcoming
>>> 3.1.1
>>> > > > > release. I
>>> > > > > > > > have also attached a test-report in the below jira.
>>> > > > > > > >
>>> > > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > Thanks,
>>> > > > > > > > D M Murali Krishna Reddy
>>> > > > > > > >
>>> > > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
>>> > > > > > brahma@apache.org
>>> > > > > > > >
>>> > > > > > > > wrote:
>>> > > > > > > >
>>> > > > > > > >> Hi Bilwa,
>>> > > > > > > >>
>>> > > > > > > >> I have commented on the jira's you mentioned. Based on the
>>> > > > stability
>>> > > > > > we
>>> > > > > > > >> can
>>> > > > > > > >> plan this.But needs to be merged ASAP.
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <
>>> stbilwa@gmail.com>
>>> > > > wrote:
>>> > > > > > > >>
>>> > > > > > > >> > Hi Brahma,
>>> > > > > > > >> >
>>> > > > > > > >> > Can we have below features in 3.3.1 release? We have
>>> been
>>> > > using
>>> > > > > > these
>>> > > > > > > >> > features for a long time. They are stable and tested in
>>> > bigger
>>> > > > > > > clusters.
>>> > > > > > > >> >
>>> > > > > > > >> > 1. Container reuse -
>>> > > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
>>> > > > > > > >> > 2. Speculative attempts should not run on the same node
>>> -
>>> > > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
>>> > > > > > > >> >
>>> > > > > > > >> > Thanks,
>>> > > > > > > >> > Bilwa
>>> > > > > > > >> >
>>> > > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
>>> > > > > > brahma@apache.org
>>> > > > > > > >
>>> > > > > > > >> > wrote:
>>> > > > > > > >> >
>>> > > > > > > >> >> Sorry for the late reply..
>>> > > > > > > >> >>
>>> > > > > > > >> >> I will come up with a plan.. Please let me know if
>>> anybody
>>> > > has
>>> > > > > some
>>> > > > > > > >> >> features/improvements/bugs that need to be included.
>>> > > > > > > >> >>
>>> > > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
>>> > > > > sunilg@apache.org>
>>> > > > > > > >> wrote:
>>> > > > > > > >> >>
>>> > > > > > > >> >> > Hi Wei-Chiu,
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > Thanks
>>> > > > > > > >> >> > Sunil
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <
>>> stack@duboce.net>
>>> > > > > wrote:
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
>>> > > > > > > >> >> > <stevel@cloudera.com.invalid
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > wrote:
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > > Regarding blockers &c: how about we have a little
>>> > > > hackathon
>>> > > > > > > >> where we
>>> > > > > > > >> >> > try
>>> > > > > > > >> >> > > > and get things in. This means a promise of review
>>> > time
>>> > > > from
>>> > > > > > the
>>> > > > > > > >> >> people
>>> > > > > > > >> >> > > with
>>> > > > > > > >> >> > > > commit rights and other people who understand the
>>> > code
>>> > > > > > (Stack?)
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing,
>>> hackathon,
>>> > > > > > testing).
>>> > > > > > > >> >> > > Thanks,
>>> > > > > > > >> >> > > S
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > > > -steve
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
>>> > > > > > ayushtkn@gmail.com
>>> > > > > > > >
>>> > > > > > > >> >> wrote:
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > > > +1
>>> > > > > > > >> >> > > > > Just to mention we would need to release
>>> > > > > hadoop-thirdparty
>>> > > > > > > too
>>> > > > > > > >> >> > before.
>>> > > > > > > >> >> > > > > Presently we are using the snapshot version of
>>> it.
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > > -Ayush
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
>>> > > > > > > >> weichiu@apache.org
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > > > wrote:
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Hi all,
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago,
>>> and as
>>> > > of
>>> > > > > now
>>> > > > > > > >> we've
>>> > > > > > > >> >> > > > > accumulated
>>> > > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A
>>> number
>>> > > of
>>> > > > > > > >> >> downstreamers
>>> > > > > > > >> >> > > are
>>> > > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
>>> > guava
>>> > > > > > version
>>> > > > > > > >> >> > conflict
>>> > > > > > > >> >> > > > > issue.
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> >
>>> > > > > > > >> >>
>>> > > > > > > >>
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > We should start the release work for 3.3.1
>>> before
>>> > > the
>>> > > > > > diff
>>> > > > > > > >> >> becomes
>>> > > > > > > >> >> > > even
>>> > > > > > > >> >> > > > > > larger.
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > I believe there are currently only two real
>>> > > blockers
>>> > > > > > for a
>>> > > > > > > >> >> 3.3.1
>>> > > > > > > >> >> > > > (using
>>> > > > > > > >> >> > > > > > this filter
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> >
>>> > > > > > > >> >>
>>> > > > > > > >>
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
>>> > > > > > > >> >> > > > > > )
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > 1. HDFS-15566 <
>>> > > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
>>> > > > > > > >> >> > > > > > 2.
>>> > > > > > > >> >> > > > > > 1. HADOOP-17112 <
>>> > > > > > > >> >> > > >
>>> https://issues.apache.org/jira/browse/HADOOP-17112
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > 2.
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
>>> > 3.3.1
>>> > > > RM?
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Also, the HowToRelease wiki does not
>>> describe the
>>> > > ARM
>>> > > > > > build
>>> > > > > > > >> >> > process.
>>> > > > > > > >> >> > > > > That's
>>> > > > > > > >> >> > > > > > going to be important for future releases.
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >>
>>> > > > > >
>>> > ---------------------------------------------------------------------
>>> > > > > > > >> >> > > > > To unsubscribe, e-mail:
>>> > > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
>>> > > > > > > >> >> > > > > For additional commands, e-mail:
>>> > > > > > > >> hdfs-dev-help@hadoop.apache.org
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> >
>>> > > > > > > >> >>
>>> > > > > > > >> >>
>>> > > > > > > >> >> --
>>> > > > > > > >> >>
>>> > > > > > > >> >>
>>> > > > > > > >> >>
>>> > > > > > > >> >> --Brahma Reddy Battula
>>> > > > > > > >> >>
>>> > > > > > > >> >
>>> > > > > > > >>
>>> > > > > > > >> --
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >> --Brahma Reddy Battula
>>> > > > > > > >>
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > > --
>>> > > > -Vinay
>>> > > >
>>> > >
>>> >
>>>
>>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Back at this. The Hadoop thirdparty 1.1.0 is released.
We have one real blocker for the 3.3.1 release (HDFS-15790 Make
ProtobufRpcEngineProtos and ProtobufRpcEngineProtos2 Co-Exist
<https://issues.apache.org/jira/browse/HDFS-15790>)
I tried to rebase the PR but it didn't work like I wanted. Some help is
needed there. If either Vinay comes back to rebase the PR, or if someone
else can take over the PR, that would be great.
Meanwhile, I'll cut a branch for 3.3.1 later. If everything works out fine,
I should be able to make a RC0 by tonight (which doesn't contain
HDFS-15790) so you guys can quickly check out.
The first RC will probably not have the aarch64 binary. I don't have much
experience with that.
Cheers,
Wei-Chiu
On Thu, May 13, 2021 at 9:04 AM Wei-Chiu Chuang <we...@cloudera.com>
wrote:
> Hello it's me again.
>
> While working on the hadoop-thirdparty release, I would like to make
> progress on the Hadoop 3.3.1 release in parallel as well.
> Looking at the jira dashboard, I think the only real blocker is HDFS-15790
> https://github.com/apache/hadoop/pull/2767
>
> I'll try to review the remaining jiras and push out those that don't have
> much progress today.
>
> Thanks all!
> Weichiu
>
>
> On Tue, Apr 20, 2021 at 12:57 PM Wei-Chiu Chuang <we...@cloudera.com>
> wrote:
>
>> Billie and Viraj,
>>
>> As of today we are at 657 resolved jiras in 3.3.1
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20status%20in%20(Resolved)%20AND%20fixVersion%20in%20(3.3.1)
>> There are so many new features (
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.3+Release) I
>> counted 10, that it is barely a dot release.
>>
>> Please make every effort to backport the said jiras to branch-3.3. I'd
>> like to cut the branch sooner than later. I'd also call for more frequent
>> releases. At the current development pace, a dot release every quarter
>> would make sense IMO.
>>
>> Given the number of release blockers, I propose a tentative code freeze
>> date: end of next Friday. After that I'd like to cut the branch and make an
>> RC.
>>
>> Thoughts? I can't do this all alone. If you find things that are worth
>> stopping the release, please tag the jira with the label 'release-blocker'.
>>
>>
>>
>> On Tue, Apr 20, 2021 at 2:44 AM Billie Rinaldi <bi...@apache.org> wrote:
>>
>>> I was thinking of backporting HADOOP-16948 for 3.3.1.
>>>
>>> Billie
>>>
>>> On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
>>> <we...@cloudera.com.invalid> wrote:
>>>
>>> > Hello, reviving this thread.
>>> >
>>> > I created a dashboard for Hadoop 3.3.1 release.
>>> >
>>> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
>>> > Also a jira to track the release work: HADOOP-17647
>>> > <https://issues.apache.org/jira/browse/HADOOP-17647>
>>> >
>>> > We are currently at 5 release blockers and 3 critical issues for Hadoop
>>> > 3.3.1. I'll go through each of them and push out the ones that aren't
>>> > really blocking us.
>>> >
>>> > If you believe there are more features/bug fixes we should include in
>>> 3.3.1
>>> > (I spent the past few weeks backporting jiras but I'm sure I missed
>>> some)
>>> > please shout out.
>>> >
>>> > Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
>>> There
>>> > are a number of tasks to be done there too. Let's start another thread
>>> for
>>> > hadoop-thirdparty 1.1.0 release.
>>> >
>>> > On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <
>>> hemanthboyina224@gmail.com
>>> > >
>>> > wrote:
>>> >
>>> > > Hi Steve and Wei-Chiu
>>> > >
>>> > > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
>>> > trunk
>>> > > and tried to work out with IPV6, we have faced some issues and have
>>> made
>>> > > the required changes for ipv6 to work.After the changes were made we
>>> have
>>> > > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
>>> > > rigorously.Its been quite a some time these changes were deployed in
>>> > > production cluster and have been in use for extensive purpose.
>>> > >
>>> > > I think it's good time to add this feature.
>>> > >
>>> > > Thanks
>>> > > Hemanth Boyina
>>> > >
>>> > >
>>> > >
>>> > > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
>>> > wrote:
>>> > >
>>> > > > Hi David,
>>> > > >
>>> > > > >> Still hoping for help here:
>>> > > >
>>> > > > >> https://issues.apache.org/jira/browse/HDFS-15790
>>> > > >
>>> > > > I will raise a PR for the said solution soon (in a day or two).
>>> > > >
>>> > > > -Vinay
>>> > > >
>>> > > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
>>> > > >
>>> > > > > Hello,
>>> > > > >
>>> > > > > Still hoping for help here:
>>> > > > >
>>> > > > > https://issues.apache.org/jira/browse/HDFS-15790
>>> > > > >
>>> > > > > Looks like it has been worked on, not sure how to best move it
>>> > forward.
>>> > > > >
>>> > > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
>>> > > > <stevel@cloudera.com.invalid
>>> > > > > >
>>> > > > > wrote:
>>> > > > >
>>> > > > > > I'm going to argue its too late to do IPv6 support close to a
>>> > > release,
>>> > > > as
>>> > > > > > it's best if its on developer machines for some time to let all
>>> > > quirks
>>> > > > > > surface. It's not so much IPv6 itself, but do we cause any
>>> > > regressions
>>> > > > on
>>> > > > > > IPv4?
>>> > > > > >
>>> > > > > > But: it can/should go into trunk and stabilize there
>>> > > > > >
>>> > > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
>>> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>>> > > > > >
>>> > > > > > > Hi Brahma,
>>> > > > > > >
>>> > > > > > > I have missed out mentioning about the IPV6 feature in the
>>> last
>>> > > mail,
>>> > > > > > > Support for IPV6 has been in development since 2015 and We
>>> have
>>> > > done
>>> > > > a
>>> > > > > > good
>>> > > > > > > amount of testing at our organisation, the feature is stable
>>> and
>>> > > used
>>> > > > > by
>>> > > > > > > our customers extensively in the last one year. I think it
>>> is a
>>> > > good
>>> > > > > time
>>> > > > > > > to add the IPV6 support to 3.3.1.
>>> > > > > > >
>>> > > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
>>> > > > > > >
>>> > > > > > > Thanks
>>> > > > > > > D M Murali Krishna Reddy
>>> > > > > > >
>>> > > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
>>> > > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>>> > > > > > >
>>> > > > > > > > Hi Brahma,
>>> > > > > > > >
>>> > > > > > > > Can we have this new feature "YARN Registry based AM
>>> discovery
>>> > > with
>>> > > > > > retry
>>> > > > > > > > and in-flight task persistent via JHS" in the upcoming
>>> 3.1.1
>>> > > > > release. I
>>> > > > > > > > have also attached a test-report in the below jira.
>>> > > > > > > >
>>> > > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > Thanks,
>>> > > > > > > > D M Murali Krishna Reddy
>>> > > > > > > >
>>> > > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
>>> > > > > > brahma@apache.org
>>> > > > > > > >
>>> > > > > > > > wrote:
>>> > > > > > > >
>>> > > > > > > >> Hi Bilwa,
>>> > > > > > > >>
>>> > > > > > > >> I have commented on the jira's you mentioned. Based on the
>>> > > > stability
>>> > > > > > we
>>> > > > > > > >> can
>>> > > > > > > >> plan this.But needs to be merged ASAP.
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <
>>> stbilwa@gmail.com>
>>> > > > wrote:
>>> > > > > > > >>
>>> > > > > > > >> > Hi Brahma,
>>> > > > > > > >> >
>>> > > > > > > >> > Can we have below features in 3.3.1 release? We have
>>> been
>>> > > using
>>> > > > > > these
>>> > > > > > > >> > features for a long time. They are stable and tested in
>>> > bigger
>>> > > > > > > clusters.
>>> > > > > > > >> >
>>> > > > > > > >> > 1. Container reuse -
>>> > > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
>>> > > > > > > >> > 2. Speculative attempts should not run on the same node
>>> -
>>> > > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
>>> > > > > > > >> >
>>> > > > > > > >> > Thanks,
>>> > > > > > > >> > Bilwa
>>> > > > > > > >> >
>>> > > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
>>> > > > > > brahma@apache.org
>>> > > > > > > >
>>> > > > > > > >> > wrote:
>>> > > > > > > >> >
>>> > > > > > > >> >> Sorry for the late reply..
>>> > > > > > > >> >>
>>> > > > > > > >> >> I will come up with a plan.. Please let me know if
>>> anybody
>>> > > has
>>> > > > > some
>>> > > > > > > >> >> features/improvements/bugs that need to be included.
>>> > > > > > > >> >>
>>> > > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
>>> > > > > sunilg@apache.org>
>>> > > > > > > >> wrote:
>>> > > > > > > >> >>
>>> > > > > > > >> >> > Hi Wei-Chiu,
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > Thanks
>>> > > > > > > >> >> > Sunil
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <
>>> stack@duboce.net>
>>> > > > > wrote:
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
>>> > > > > > > >> >> > <stevel@cloudera.com.invalid
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > wrote:
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > > Regarding blockers &c: how about we have a little
>>> > > > hackathon
>>> > > > > > > >> where we
>>> > > > > > > >> >> > try
>>> > > > > > > >> >> > > > and get things in. This means a promise of review
>>> > time
>>> > > > from
>>> > > > > > the
>>> > > > > > > >> >> people
>>> > > > > > > >> >> > > with
>>> > > > > > > >> >> > > > commit rights and other people who understand the
>>> > code
>>> > > > > > (Stack?)
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing,
>>> hackathon,
>>> > > > > > testing).
>>> > > > > > > >> >> > > Thanks,
>>> > > > > > > >> >> > > S
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > > > -steve
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
>>> > > > > > ayushtkn@gmail.com
>>> > > > > > > >
>>> > > > > > > >> >> wrote:
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > > > +1
>>> > > > > > > >> >> > > > > Just to mention we would need to release
>>> > > > > hadoop-thirdparty
>>> > > > > > > too
>>> > > > > > > >> >> > before.
>>> > > > > > > >> >> > > > > Presently we are using the snapshot version of
>>> it.
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > > -Ayush
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
>>> > > > > > > >> weichiu@apache.org
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > > > wrote:
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Hi all,
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago,
>>> and as
>>> > > of
>>> > > > > now
>>> > > > > > > >> we've
>>> > > > > > > >> >> > > > > accumulated
>>> > > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A
>>> number
>>> > > of
>>> > > > > > > >> >> downstreamers
>>> > > > > > > >> >> > > are
>>> > > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
>>> > guava
>>> > > > > > version
>>> > > > > > > >> >> > conflict
>>> > > > > > > >> >> > > > > issue.
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> >
>>> > > > > > > >> >>
>>> > > > > > > >>
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > We should start the release work for 3.3.1
>>> before
>>> > > the
>>> > > > > > diff
>>> > > > > > > >> >> becomes
>>> > > > > > > >> >> > > even
>>> > > > > > > >> >> > > > > > larger.
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > I believe there are currently only two real
>>> > > blockers
>>> > > > > > for a
>>> > > > > > > >> >> 3.3.1
>>> > > > > > > >> >> > > > (using
>>> > > > > > > >> >> > > > > > this filter
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> >
>>> > > > > > > >> >>
>>> > > > > > > >>
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
>>> > > > > > > >> >> > > > > > )
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > 1. HDFS-15566 <
>>> > > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
>>> > > > > > > >> >> > > > > > 2.
>>> > > > > > > >> >> > > > > > 1. HADOOP-17112 <
>>> > > > > > > >> >> > > >
>>> https://issues.apache.org/jira/browse/HADOOP-17112
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > 2.
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
>>> > 3.3.1
>>> > > > RM?
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Also, the HowToRelease wiki does not
>>> describe the
>>> > > ARM
>>> > > > > > build
>>> > > > > > > >> >> > process.
>>> > > > > > > >> >> > > > > That's
>>> > > > > > > >> >> > > > > > going to be important for future releases.
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >>
>>> > > > > >
>>> > ---------------------------------------------------------------------
>>> > > > > > > >> >> > > > > To unsubscribe, e-mail:
>>> > > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
>>> > > > > > > >> >> > > > > For additional commands, e-mail:
>>> > > > > > > >> hdfs-dev-help@hadoop.apache.org
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> >
>>> > > > > > > >> >>
>>> > > > > > > >> >>
>>> > > > > > > >> >> --
>>> > > > > > > >> >>
>>> > > > > > > >> >>
>>> > > > > > > >> >>
>>> > > > > > > >> >> --Brahma Reddy Battula
>>> > > > > > > >> >>
>>> > > > > > > >> >
>>> > > > > > > >>
>>> > > > > > > >> --
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >> --Brahma Reddy Battula
>>> > > > > > > >>
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > > --
>>> > > > -Vinay
>>> > > >
>>> > >
>>> >
>>>
>>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Back at this. The Hadoop thirdparty 1.1.0 is released.
We have one real blocker for the 3.3.1 release (HDFS-15790 Make
ProtobufRpcEngineProtos and ProtobufRpcEngineProtos2 Co-Exist
<https://issues.apache.org/jira/browse/HDFS-15790>)
I tried to rebase the PR but it didn't work like I wanted. Some help is
needed there. If either Vinay comes back to rebase the PR, or if someone
else can take over the PR, that would be great.
Meanwhile, I'll cut a branch for 3.3.1 later. If everything works out fine,
I should be able to make a RC0 by tonight (which doesn't contain
HDFS-15790) so you guys can quickly check out.
The first RC will probably not have the aarch64 binary. I don't have much
experience with that.
Cheers,
Wei-Chiu
On Thu, May 13, 2021 at 9:04 AM Wei-Chiu Chuang <we...@cloudera.com>
wrote:
> Hello it's me again.
>
> While working on the hadoop-thirdparty release, I would like to make
> progress on the Hadoop 3.3.1 release in parallel as well.
> Looking at the jira dashboard, I think the only real blocker is HDFS-15790
> https://github.com/apache/hadoop/pull/2767
>
> I'll try to review the remaining jiras and push out those that don't have
> much progress today.
>
> Thanks all!
> Weichiu
>
>
> On Tue, Apr 20, 2021 at 12:57 PM Wei-Chiu Chuang <we...@cloudera.com>
> wrote:
>
>> Billie and Viraj,
>>
>> As of today we are at 657 resolved jiras in 3.3.1
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20status%20in%20(Resolved)%20AND%20fixVersion%20in%20(3.3.1)
>> There are so many new features (
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.3+Release) I
>> counted 10, that it is barely a dot release.
>>
>> Please make every effort to backport the said jiras to branch-3.3. I'd
>> like to cut the branch sooner than later. I'd also call for more frequent
>> releases. At the current development pace, a dot release every quarter
>> would make sense IMO.
>>
>> Given the number of release blockers, I propose a tentative code freeze
>> date: end of next Friday. After that I'd like to cut the branch and make an
>> RC.
>>
>> Thoughts? I can't do this all alone. If you find things that are worth
>> stopping the release, please tag the jira with the label 'release-blocker'.
>>
>>
>>
>> On Tue, Apr 20, 2021 at 2:44 AM Billie Rinaldi <bi...@apache.org> wrote:
>>
>>> I was thinking of backporting HADOOP-16948 for 3.3.1.
>>>
>>> Billie
>>>
>>> On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
>>> <we...@cloudera.com.invalid> wrote:
>>>
>>> > Hello, reviving this thread.
>>> >
>>> > I created a dashboard for Hadoop 3.3.1 release.
>>> >
>>> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
>>> > Also a jira to track the release work: HADOOP-17647
>>> > <https://issues.apache.org/jira/browse/HADOOP-17647>
>>> >
>>> > We are currently at 5 release blockers and 3 critical issues for Hadoop
>>> > 3.3.1. I'll go through each of them and push out the ones that aren't
>>> > really blocking us.
>>> >
>>> > If you believe there are more features/bug fixes we should include in
>>> 3.3.1
>>> > (I spent the past few weeks backporting jiras but I'm sure I missed
>>> some)
>>> > please shout out.
>>> >
>>> > Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
>>> There
>>> > are a number of tasks to be done there too. Let's start another thread
>>> for
>>> > hadoop-thirdparty 1.1.0 release.
>>> >
>>> > On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <
>>> hemanthboyina224@gmail.com
>>> > >
>>> > wrote:
>>> >
>>> > > Hi Steve and Wei-Chiu
>>> > >
>>> > > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
>>> > trunk
>>> > > and tried to work out with IPV6, we have faced some issues and have
>>> made
>>> > > the required changes for ipv6 to work.After the changes were made we
>>> have
>>> > > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
>>> > > rigorously.Its been quite a some time these changes were deployed in
>>> > > production cluster and have been in use for extensive purpose.
>>> > >
>>> > > I think it's good time to add this feature.
>>> > >
>>> > > Thanks
>>> > > Hemanth Boyina
>>> > >
>>> > >
>>> > >
>>> > > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
>>> > wrote:
>>> > >
>>> > > > Hi David,
>>> > > >
>>> > > > >> Still hoping for help here:
>>> > > >
>>> > > > >> https://issues.apache.org/jira/browse/HDFS-15790
>>> > > >
>>> > > > I will raise a PR for the said solution soon (in a day or two).
>>> > > >
>>> > > > -Vinay
>>> > > >
>>> > > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
>>> > > >
>>> > > > > Hello,
>>> > > > >
>>> > > > > Still hoping for help here:
>>> > > > >
>>> > > > > https://issues.apache.org/jira/browse/HDFS-15790
>>> > > > >
>>> > > > > Looks like it has been worked on, not sure how to best move it
>>> > forward.
>>> > > > >
>>> > > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
>>> > > > <stevel@cloudera.com.invalid
>>> > > > > >
>>> > > > > wrote:
>>> > > > >
>>> > > > > > I'm going to argue its too late to do IPv6 support close to a
>>> > > release,
>>> > > > as
>>> > > > > > it's best if its on developer machines for some time to let all
>>> > > quirks
>>> > > > > > surface. It's not so much IPv6 itself, but do we cause any
>>> > > regressions
>>> > > > on
>>> > > > > > IPv4?
>>> > > > > >
>>> > > > > > But: it can/should go into trunk and stabilize there
>>> > > > > >
>>> > > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
>>> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>>> > > > > >
>>> > > > > > > Hi Brahma,
>>> > > > > > >
>>> > > > > > > I have missed out mentioning about the IPV6 feature in the
>>> last
>>> > > mail,
>>> > > > > > > Support for IPV6 has been in development since 2015 and We
>>> have
>>> > > done
>>> > > > a
>>> > > > > > good
>>> > > > > > > amount of testing at our organisation, the feature is stable
>>> and
>>> > > used
>>> > > > > by
>>> > > > > > > our customers extensively in the last one year. I think it
>>> is a
>>> > > good
>>> > > > > time
>>> > > > > > > to add the IPV6 support to 3.3.1.
>>> > > > > > >
>>> > > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
>>> > > > > > >
>>> > > > > > > Thanks
>>> > > > > > > D M Murali Krishna Reddy
>>> > > > > > >
>>> > > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
>>> > > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>>> > > > > > >
>>> > > > > > > > Hi Brahma,
>>> > > > > > > >
>>> > > > > > > > Can we have this new feature "YARN Registry based AM
>>> discovery
>>> > > with
>>> > > > > > retry
>>> > > > > > > > and in-flight task persistent via JHS" in the upcoming
>>> 3.1.1
>>> > > > > release. I
>>> > > > > > > > have also attached a test-report in the below jira.
>>> > > > > > > >
>>> > > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > Thanks,
>>> > > > > > > > D M Murali Krishna Reddy
>>> > > > > > > >
>>> > > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
>>> > > > > > brahma@apache.org
>>> > > > > > > >
>>> > > > > > > > wrote:
>>> > > > > > > >
>>> > > > > > > >> Hi Bilwa,
>>> > > > > > > >>
>>> > > > > > > >> I have commented on the jira's you mentioned. Based on the
>>> > > > stability
>>> > > > > > we
>>> > > > > > > >> can
>>> > > > > > > >> plan this.But needs to be merged ASAP.
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <
>>> stbilwa@gmail.com>
>>> > > > wrote:
>>> > > > > > > >>
>>> > > > > > > >> > Hi Brahma,
>>> > > > > > > >> >
>>> > > > > > > >> > Can we have below features in 3.3.1 release? We have
>>> been
>>> > > using
>>> > > > > > these
>>> > > > > > > >> > features for a long time. They are stable and tested in
>>> > bigger
>>> > > > > > > clusters.
>>> > > > > > > >> >
>>> > > > > > > >> > 1. Container reuse -
>>> > > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
>>> > > > > > > >> > 2. Speculative attempts should not run on the same node
>>> -
>>> > > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
>>> > > > > > > >> >
>>> > > > > > > >> > Thanks,
>>> > > > > > > >> > Bilwa
>>> > > > > > > >> >
>>> > > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
>>> > > > > > brahma@apache.org
>>> > > > > > > >
>>> > > > > > > >> > wrote:
>>> > > > > > > >> >
>>> > > > > > > >> >> Sorry for the late reply..
>>> > > > > > > >> >>
>>> > > > > > > >> >> I will come up with a plan.. Please let me know if
>>> anybody
>>> > > has
>>> > > > > some
>>> > > > > > > >> >> features/improvements/bugs that need to be included.
>>> > > > > > > >> >>
>>> > > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
>>> > > > > sunilg@apache.org>
>>> > > > > > > >> wrote:
>>> > > > > > > >> >>
>>> > > > > > > >> >> > Hi Wei-Chiu,
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > Thanks
>>> > > > > > > >> >> > Sunil
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <
>>> stack@duboce.net>
>>> > > > > wrote:
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
>>> > > > > > > >> >> > <stevel@cloudera.com.invalid
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > wrote:
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > > Regarding blockers &c: how about we have a little
>>> > > > hackathon
>>> > > > > > > >> where we
>>> > > > > > > >> >> > try
>>> > > > > > > >> >> > > > and get things in. This means a promise of review
>>> > time
>>> > > > from
>>> > > > > > the
>>> > > > > > > >> >> people
>>> > > > > > > >> >> > > with
>>> > > > > > > >> >> > > > commit rights and other people who understand the
>>> > code
>>> > > > > > (Stack?)
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing,
>>> hackathon,
>>> > > > > > testing).
>>> > > > > > > >> >> > > Thanks,
>>> > > > > > > >> >> > > S
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > > > -steve
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
>>> > > > > > ayushtkn@gmail.com
>>> > > > > > > >
>>> > > > > > > >> >> wrote:
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > > > +1
>>> > > > > > > >> >> > > > > Just to mention we would need to release
>>> > > > > hadoop-thirdparty
>>> > > > > > > too
>>> > > > > > > >> >> > before.
>>> > > > > > > >> >> > > > > Presently we are using the snapshot version of
>>> it.
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > > -Ayush
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
>>> > > > > > > >> weichiu@apache.org
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > > > wrote:
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Hi all,
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago,
>>> and as
>>> > > of
>>> > > > > now
>>> > > > > > > >> we've
>>> > > > > > > >> >> > > > > accumulated
>>> > > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A
>>> number
>>> > > of
>>> > > > > > > >> >> downstreamers
>>> > > > > > > >> >> > > are
>>> > > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
>>> > guava
>>> > > > > > version
>>> > > > > > > >> >> > conflict
>>> > > > > > > >> >> > > > > issue.
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> >
>>> > > > > > > >> >>
>>> > > > > > > >>
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > We should start the release work for 3.3.1
>>> before
>>> > > the
>>> > > > > > diff
>>> > > > > > > >> >> becomes
>>> > > > > > > >> >> > > even
>>> > > > > > > >> >> > > > > > larger.
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > I believe there are currently only two real
>>> > > blockers
>>> > > > > > for a
>>> > > > > > > >> >> 3.3.1
>>> > > > > > > >> >> > > > (using
>>> > > > > > > >> >> > > > > > this filter
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> >
>>> > > > > > > >> >>
>>> > > > > > > >>
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
>>> > > > > > > >> >> > > > > > )
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > 1. HDFS-15566 <
>>> > > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
>>> > > > > > > >> >> > > > > > 2.
>>> > > > > > > >> >> > > > > > 1. HADOOP-17112 <
>>> > > > > > > >> >> > > >
>>> https://issues.apache.org/jira/browse/HADOOP-17112
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > 2.
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
>>> > 3.3.1
>>> > > > RM?
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Also, the HowToRelease wiki does not
>>> describe the
>>> > > ARM
>>> > > > > > build
>>> > > > > > > >> >> > process.
>>> > > > > > > >> >> > > > > That's
>>> > > > > > > >> >> > > > > > going to be important for future releases.
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >>
>>> > > > > >
>>> > ---------------------------------------------------------------------
>>> > > > > > > >> >> > > > > To unsubscribe, e-mail:
>>> > > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
>>> > > > > > > >> >> > > > > For additional commands, e-mail:
>>> > > > > > > >> hdfs-dev-help@hadoop.apache.org
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> >
>>> > > > > > > >> >>
>>> > > > > > > >> >>
>>> > > > > > > >> >> --
>>> > > > > > > >> >>
>>> > > > > > > >> >>
>>> > > > > > > >> >>
>>> > > > > > > >> >> --Brahma Reddy Battula
>>> > > > > > > >> >>
>>> > > > > > > >> >
>>> > > > > > > >>
>>> > > > > > > >> --
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >> --Brahma Reddy Battula
>>> > > > > > > >>
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > > --
>>> > > > -Vinay
>>> > > >
>>> > >
>>> >
>>>
>>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Back at this. The Hadoop thirdparty 1.1.0 is released.
We have one real blocker for the 3.3.1 release (HDFS-15790 Make
ProtobufRpcEngineProtos and ProtobufRpcEngineProtos2 Co-Exist
<https://issues.apache.org/jira/browse/HDFS-15790>)
I tried to rebase the PR but it didn't work like I wanted. Some help is
needed there. If either Vinay comes back to rebase the PR, or if someone
else can take over the PR, that would be great.
Meanwhile, I'll cut a branch for 3.3.1 later. If everything works out fine,
I should be able to make a RC0 by tonight (which doesn't contain
HDFS-15790) so you guys can quickly check out.
The first RC will probably not have the aarch64 binary. I don't have much
experience with that.
Cheers,
Wei-Chiu
On Thu, May 13, 2021 at 9:04 AM Wei-Chiu Chuang <we...@cloudera.com>
wrote:
> Hello it's me again.
>
> While working on the hadoop-thirdparty release, I would like to make
> progress on the Hadoop 3.3.1 release in parallel as well.
> Looking at the jira dashboard, I think the only real blocker is HDFS-15790
> https://github.com/apache/hadoop/pull/2767
>
> I'll try to review the remaining jiras and push out those that don't have
> much progress today.
>
> Thanks all!
> Weichiu
>
>
> On Tue, Apr 20, 2021 at 12:57 PM Wei-Chiu Chuang <we...@cloudera.com>
> wrote:
>
>> Billie and Viraj,
>>
>> As of today we are at 657 resolved jiras in 3.3.1
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20status%20in%20(Resolved)%20AND%20fixVersion%20in%20(3.3.1)
>> There are so many new features (
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.3+Release) I
>> counted 10, that it is barely a dot release.
>>
>> Please make every effort to backport the said jiras to branch-3.3. I'd
>> like to cut the branch sooner than later. I'd also call for more frequent
>> releases. At the current development pace, a dot release every quarter
>> would make sense IMO.
>>
>> Given the number of release blockers, I propose a tentative code freeze
>> date: end of next Friday. After that I'd like to cut the branch and make an
>> RC.
>>
>> Thoughts? I can't do this all alone. If you find things that are worth
>> stopping the release, please tag the jira with the label 'release-blocker'.
>>
>>
>>
>> On Tue, Apr 20, 2021 at 2:44 AM Billie Rinaldi <bi...@apache.org> wrote:
>>
>>> I was thinking of backporting HADOOP-16948 for 3.3.1.
>>>
>>> Billie
>>>
>>> On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
>>> <we...@cloudera.com.invalid> wrote:
>>>
>>> > Hello, reviving this thread.
>>> >
>>> > I created a dashboard for Hadoop 3.3.1 release.
>>> >
>>> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
>>> > Also a jira to track the release work: HADOOP-17647
>>> > <https://issues.apache.org/jira/browse/HADOOP-17647>
>>> >
>>> > We are currently at 5 release blockers and 3 critical issues for Hadoop
>>> > 3.3.1. I'll go through each of them and push out the ones that aren't
>>> > really blocking us.
>>> >
>>> > If you believe there are more features/bug fixes we should include in
>>> 3.3.1
>>> > (I spent the past few weeks backporting jiras but I'm sure I missed
>>> some)
>>> > please shout out.
>>> >
>>> > Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
>>> There
>>> > are a number of tasks to be done there too. Let's start another thread
>>> for
>>> > hadoop-thirdparty 1.1.0 release.
>>> >
>>> > On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <
>>> hemanthboyina224@gmail.com
>>> > >
>>> > wrote:
>>> >
>>> > > Hi Steve and Wei-Chiu
>>> > >
>>> > > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
>>> > trunk
>>> > > and tried to work out with IPV6, we have faced some issues and have
>>> made
>>> > > the required changes for ipv6 to work.After the changes were made we
>>> have
>>> > > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
>>> > > rigorously.Its been quite a some time these changes were deployed in
>>> > > production cluster and have been in use for extensive purpose.
>>> > >
>>> > > I think it's good time to add this feature.
>>> > >
>>> > > Thanks
>>> > > Hemanth Boyina
>>> > >
>>> > >
>>> > >
>>> > > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
>>> > wrote:
>>> > >
>>> > > > Hi David,
>>> > > >
>>> > > > >> Still hoping for help here:
>>> > > >
>>> > > > >> https://issues.apache.org/jira/browse/HDFS-15790
>>> > > >
>>> > > > I will raise a PR for the said solution soon (in a day or two).
>>> > > >
>>> > > > -Vinay
>>> > > >
>>> > > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
>>> > > >
>>> > > > > Hello,
>>> > > > >
>>> > > > > Still hoping for help here:
>>> > > > >
>>> > > > > https://issues.apache.org/jira/browse/HDFS-15790
>>> > > > >
>>> > > > > Looks like it has been worked on, not sure how to best move it
>>> > forward.
>>> > > > >
>>> > > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
>>> > > > <stevel@cloudera.com.invalid
>>> > > > > >
>>> > > > > wrote:
>>> > > > >
>>> > > > > > I'm going to argue its too late to do IPv6 support close to a
>>> > > release,
>>> > > > as
>>> > > > > > it's best if its on developer machines for some time to let all
>>> > > quirks
>>> > > > > > surface. It's not so much IPv6 itself, but do we cause any
>>> > > regressions
>>> > > > on
>>> > > > > > IPv4?
>>> > > > > >
>>> > > > > > But: it can/should go into trunk and stabilize there
>>> > > > > >
>>> > > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
>>> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>>> > > > > >
>>> > > > > > > Hi Brahma,
>>> > > > > > >
>>> > > > > > > I have missed out mentioning about the IPV6 feature in the
>>> last
>>> > > mail,
>>> > > > > > > Support for IPV6 has been in development since 2015 and We
>>> have
>>> > > done
>>> > > > a
>>> > > > > > good
>>> > > > > > > amount of testing at our organisation, the feature is stable
>>> and
>>> > > used
>>> > > > > by
>>> > > > > > > our customers extensively in the last one year. I think it
>>> is a
>>> > > good
>>> > > > > time
>>> > > > > > > to add the IPV6 support to 3.3.1.
>>> > > > > > >
>>> > > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
>>> > > > > > >
>>> > > > > > > Thanks
>>> > > > > > > D M Murali Krishna Reddy
>>> > > > > > >
>>> > > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
>>> > > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>>> > > > > > >
>>> > > > > > > > Hi Brahma,
>>> > > > > > > >
>>> > > > > > > > Can we have this new feature "YARN Registry based AM
>>> discovery
>>> > > with
>>> > > > > > retry
>>> > > > > > > > and in-flight task persistent via JHS" in the upcoming
>>> 3.1.1
>>> > > > > release. I
>>> > > > > > > > have also attached a test-report in the below jira.
>>> > > > > > > >
>>> > > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > Thanks,
>>> > > > > > > > D M Murali Krishna Reddy
>>> > > > > > > >
>>> > > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
>>> > > > > > brahma@apache.org
>>> > > > > > > >
>>> > > > > > > > wrote:
>>> > > > > > > >
>>> > > > > > > >> Hi Bilwa,
>>> > > > > > > >>
>>> > > > > > > >> I have commented on the jira's you mentioned. Based on the
>>> > > > stability
>>> > > > > > we
>>> > > > > > > >> can
>>> > > > > > > >> plan this.But needs to be merged ASAP.
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <
>>> stbilwa@gmail.com>
>>> > > > wrote:
>>> > > > > > > >>
>>> > > > > > > >> > Hi Brahma,
>>> > > > > > > >> >
>>> > > > > > > >> > Can we have below features in 3.3.1 release? We have
>>> been
>>> > > using
>>> > > > > > these
>>> > > > > > > >> > features for a long time. They are stable and tested in
>>> > bigger
>>> > > > > > > clusters.
>>> > > > > > > >> >
>>> > > > > > > >> > 1. Container reuse -
>>> > > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
>>> > > > > > > >> > 2. Speculative attempts should not run on the same node
>>> -
>>> > > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
>>> > > > > > > >> >
>>> > > > > > > >> > Thanks,
>>> > > > > > > >> > Bilwa
>>> > > > > > > >> >
>>> > > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
>>> > > > > > brahma@apache.org
>>> > > > > > > >
>>> > > > > > > >> > wrote:
>>> > > > > > > >> >
>>> > > > > > > >> >> Sorry for the late reply..
>>> > > > > > > >> >>
>>> > > > > > > >> >> I will come up with a plan.. Please let me know if
>>> anybody
>>> > > has
>>> > > > > some
>>> > > > > > > >> >> features/improvements/bugs that need to be included.
>>> > > > > > > >> >>
>>> > > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
>>> > > > > sunilg@apache.org>
>>> > > > > > > >> wrote:
>>> > > > > > > >> >>
>>> > > > > > > >> >> > Hi Wei-Chiu,
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > Thanks
>>> > > > > > > >> >> > Sunil
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <
>>> stack@duboce.net>
>>> > > > > wrote:
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
>>> > > > > > > >> >> > <stevel@cloudera.com.invalid
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > wrote:
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > > Regarding blockers &c: how about we have a little
>>> > > > hackathon
>>> > > > > > > >> where we
>>> > > > > > > >> >> > try
>>> > > > > > > >> >> > > > and get things in. This means a promise of review
>>> > time
>>> > > > from
>>> > > > > > the
>>> > > > > > > >> >> people
>>> > > > > > > >> >> > > with
>>> > > > > > > >> >> > > > commit rights and other people who understand the
>>> > code
>>> > > > > > (Stack?)
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing,
>>> hackathon,
>>> > > > > > testing).
>>> > > > > > > >> >> > > Thanks,
>>> > > > > > > >> >> > > S
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> > > > -steve
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
>>> > > > > > ayushtkn@gmail.com
>>> > > > > > > >
>>> > > > > > > >> >> wrote:
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > > > > +1
>>> > > > > > > >> >> > > > > Just to mention we would need to release
>>> > > > > hadoop-thirdparty
>>> > > > > > > too
>>> > > > > > > >> >> > before.
>>> > > > > > > >> >> > > > > Presently we are using the snapshot version of
>>> it.
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > > -Ayush
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
>>> > > > > > > >> weichiu@apache.org
>>> > > > > > > >> >> >
>>> > > > > > > >> >> > > > wrote:
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Hi all,
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago,
>>> and as
>>> > > of
>>> > > > > now
>>> > > > > > > >> we've
>>> > > > > > > >> >> > > > > accumulated
>>> > > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A
>>> number
>>> > > of
>>> > > > > > > >> >> downstreamers
>>> > > > > > > >> >> > > are
>>> > > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
>>> > guava
>>> > > > > > version
>>> > > > > > > >> >> > conflict
>>> > > > > > > >> >> > > > > issue.
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> >
>>> > > > > > > >> >>
>>> > > > > > > >>
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > We should start the release work for 3.3.1
>>> before
>>> > > the
>>> > > > > > diff
>>> > > > > > > >> >> becomes
>>> > > > > > > >> >> > > even
>>> > > > > > > >> >> > > > > > larger.
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > I believe there are currently only two real
>>> > > blockers
>>> > > > > > for a
>>> > > > > > > >> >> 3.3.1
>>> > > > > > > >> >> > > > (using
>>> > > > > > > >> >> > > > > > this filter
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> >
>>> > > > > > > >> >>
>>> > > > > > > >>
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
>>> > > > > > > >> >> > > > > > )
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > 1. HDFS-15566 <
>>> > > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
>>> > > > > > > >> >> > > > > > 2.
>>> > > > > > > >> >> > > > > > 1. HADOOP-17112 <
>>> > > > > > > >> >> > > >
>>> https://issues.apache.org/jira/browse/HADOOP-17112
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > 2.
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
>>> > 3.3.1
>>> > > > RM?
>>> > > > > > > >> >> > > > > >
>>> > > > > > > >> >> > > > > > Also, the HowToRelease wiki does not
>>> describe the
>>> > > ARM
>>> > > > > > build
>>> > > > > > > >> >> > process.
>>> > > > > > > >> >> > > > > That's
>>> > > > > > > >> >> > > > > > going to be important for future releases.
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >>
>>> > > > > >
>>> > ---------------------------------------------------------------------
>>> > > > > > > >> >> > > > > To unsubscribe, e-mail:
>>> > > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
>>> > > > > > > >> >> > > > > For additional commands, e-mail:
>>> > > > > > > >> hdfs-dev-help@hadoop.apache.org
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > > >
>>> > > > > > > >> >> > > >
>>> > > > > > > >> >> > >
>>> > > > > > > >> >> >
>>> > > > > > > >> >>
>>> > > > > > > >> >>
>>> > > > > > > >> >> --
>>> > > > > > > >> >>
>>> > > > > > > >> >>
>>> > > > > > > >> >>
>>> > > > > > > >> >> --Brahma Reddy Battula
>>> > > > > > > >> >>
>>> > > > > > > >> >
>>> > > > > > > >>
>>> > > > > > > >> --
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >>
>>> > > > > > > >> --Brahma Reddy Battula
>>> > > > > > > >>
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > > --
>>> > > > -Vinay
>>> > > >
>>> > >
>>> >
>>>
>>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Hello it's me again.
While working on the hadoop-thirdparty release, I would like to make
progress on the Hadoop 3.3.1 release in parallel as well.
Looking at the jira dashboard, I think the only real blocker is HDFS-15790
https://github.com/apache/hadoop/pull/2767
I'll try to review the remaining jiras and push out those that don't have
much progress today.
Thanks all!
Weichiu
On Tue, Apr 20, 2021 at 12:57 PM Wei-Chiu Chuang <we...@cloudera.com>
wrote:
> Billie and Viraj,
>
> As of today we are at 657 resolved jiras in 3.3.1
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20status%20in%20(Resolved)%20AND%20fixVersion%20in%20(3.3.1)
> There are so many new features (
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.3+Release) I
> counted 10, that it is barely a dot release.
>
> Please make every effort to backport the said jiras to branch-3.3. I'd
> like to cut the branch sooner than later. I'd also call for more frequent
> releases. At the current development pace, a dot release every quarter
> would make sense IMO.
>
> Given the number of release blockers, I propose a tentative code freeze
> date: end of next Friday. After that I'd like to cut the branch and make an
> RC.
>
> Thoughts? I can't do this all alone. If you find things that are worth
> stopping the release, please tag the jira with the label 'release-blocker'.
>
>
>
> On Tue, Apr 20, 2021 at 2:44 AM Billie Rinaldi <bi...@apache.org> wrote:
>
>> I was thinking of backporting HADOOP-16948 for 3.3.1.
>>
>> Billie
>>
>> On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
>> <we...@cloudera.com.invalid> wrote:
>>
>> > Hello, reviving this thread.
>> >
>> > I created a dashboard for Hadoop 3.3.1 release.
>> >
>> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
>> > Also a jira to track the release work: HADOOP-17647
>> > <https://issues.apache.org/jira/browse/HADOOP-17647>
>> >
>> > We are currently at 5 release blockers and 3 critical issues for Hadoop
>> > 3.3.1. I'll go through each of them and push out the ones that aren't
>> > really blocking us.
>> >
>> > If you believe there are more features/bug fixes we should include in
>> 3.3.1
>> > (I spent the past few weeks backporting jiras but I'm sure I missed
>> some)
>> > please shout out.
>> >
>> > Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
>> There
>> > are a number of tasks to be done there too. Let's start another thread
>> for
>> > hadoop-thirdparty 1.1.0 release.
>> >
>> > On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <
>> hemanthboyina224@gmail.com
>> > >
>> > wrote:
>> >
>> > > Hi Steve and Wei-Chiu
>> > >
>> > > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
>> > trunk
>> > > and tried to work out with IPV6, we have faced some issues and have
>> made
>> > > the required changes for ipv6 to work.After the changes were made we
>> have
>> > > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
>> > > rigorously.Its been quite a some time these changes were deployed in
>> > > production cluster and have been in use for extensive purpose.
>> > >
>> > > I think it's good time to add this feature.
>> > >
>> > > Thanks
>> > > Hemanth Boyina
>> > >
>> > >
>> > >
>> > > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
>> > wrote:
>> > >
>> > > > Hi David,
>> > > >
>> > > > >> Still hoping for help here:
>> > > >
>> > > > >> https://issues.apache.org/jira/browse/HDFS-15790
>> > > >
>> > > > I will raise a PR for the said solution soon (in a day or two).
>> > > >
>> > > > -Vinay
>> > > >
>> > > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
>> > > >
>> > > > > Hello,
>> > > > >
>> > > > > Still hoping for help here:
>> > > > >
>> > > > > https://issues.apache.org/jira/browse/HDFS-15790
>> > > > >
>> > > > > Looks like it has been worked on, not sure how to best move it
>> > forward.
>> > > > >
>> > > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
>> > > > <stevel@cloudera.com.invalid
>> > > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > I'm going to argue its too late to do IPv6 support close to a
>> > > release,
>> > > > as
>> > > > > > it's best if its on developer machines for some time to let all
>> > > quirks
>> > > > > > surface. It's not so much IPv6 itself, but do we cause any
>> > > regressions
>> > > > on
>> > > > > > IPv4?
>> > > > > >
>> > > > > > But: it can/should go into trunk and stabilize there
>> > > > > >
>> > > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
>> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>> > > > > >
>> > > > > > > Hi Brahma,
>> > > > > > >
>> > > > > > > I have missed out mentioning about the IPV6 feature in the
>> last
>> > > mail,
>> > > > > > > Support for IPV6 has been in development since 2015 and We
>> have
>> > > done
>> > > > a
>> > > > > > good
>> > > > > > > amount of testing at our organisation, the feature is stable
>> and
>> > > used
>> > > > > by
>> > > > > > > our customers extensively in the last one year. I think it is
>> a
>> > > good
>> > > > > time
>> > > > > > > to add the IPV6 support to 3.3.1.
>> > > > > > >
>> > > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
>> > > > > > >
>> > > > > > > Thanks
>> > > > > > > D M Murali Krishna Reddy
>> > > > > > >
>> > > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
>> > > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>> > > > > > >
>> > > > > > > > Hi Brahma,
>> > > > > > > >
>> > > > > > > > Can we have this new feature "YARN Registry based AM
>> discovery
>> > > with
>> > > > > > retry
>> > > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
>> > > > > release. I
>> > > > > > > > have also attached a test-report in the below jira.
>> > > > > > > >
>> > > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > > D M Murali Krishna Reddy
>> > > > > > > >
>> > > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
>> > > > > > brahma@apache.org
>> > > > > > > >
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > >> Hi Bilwa,
>> > > > > > > >>
>> > > > > > > >> I have commented on the jira's you mentioned. Based on the
>> > > > stability
>> > > > > > we
>> > > > > > > >> can
>> > > > > > > >> plan this.But needs to be merged ASAP.
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <
>> stbilwa@gmail.com>
>> > > > wrote:
>> > > > > > > >>
>> > > > > > > >> > Hi Brahma,
>> > > > > > > >> >
>> > > > > > > >> > Can we have below features in 3.3.1 release? We have been
>> > > using
>> > > > > > these
>> > > > > > > >> > features for a long time. They are stable and tested in
>> > bigger
>> > > > > > > clusters.
>> > > > > > > >> >
>> > > > > > > >> > 1. Container reuse -
>> > > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
>> > > > > > > >> > 2. Speculative attempts should not run on the same node -
>> > > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
>> > > > > > > >> >
>> > > > > > > >> > Thanks,
>> > > > > > > >> > Bilwa
>> > > > > > > >> >
>> > > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
>> > > > > > brahma@apache.org
>> > > > > > > >
>> > > > > > > >> > wrote:
>> > > > > > > >> >
>> > > > > > > >> >> Sorry for the late reply..
>> > > > > > > >> >>
>> > > > > > > >> >> I will come up with a plan.. Please let me know if
>> anybody
>> > > has
>> > > > > some
>> > > > > > > >> >> features/improvements/bugs that need to be included.
>> > > > > > > >> >>
>> > > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
>> > > > > sunilg@apache.org>
>> > > > > > > >> wrote:
>> > > > > > > >> >>
>> > > > > > > >> >> > Hi Wei-Chiu,
>> > > > > > > >> >> >
>> > > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
>> > > > > > > >> >> >
>> > > > > > > >> >> > Thanks
>> > > > > > > >> >> > Sunil
>> > > > > > > >> >> >
>> > > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <
>> stack@duboce.net>
>> > > > > wrote:
>> > > > > > > >> >> >
>> > > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
>> > > > > > > >> >> > <stevel@cloudera.com.invalid
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > wrote:
>> > > > > > > >> >> > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > > Regarding blockers &c: how about we have a little
>> > > > hackathon
>> > > > > > > >> where we
>> > > > > > > >> >> > try
>> > > > > > > >> >> > > > and get things in. This means a promise of review
>> > time
>> > > > from
>> > > > > > the
>> > > > > > > >> >> people
>> > > > > > > >> >> > > with
>> > > > > > > >> >> > > > commit rights and other people who understand the
>> > code
>> > > > > > (Stack?)
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing,
>> hackathon,
>> > > > > > testing).
>> > > > > > > >> >> > > Thanks,
>> > > > > > > >> >> > > S
>> > > > > > > >> >> > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > > > -steve
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
>> > > > > > ayushtkn@gmail.com
>> > > > > > > >
>> > > > > > > >> >> wrote:
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > > > +1
>> > > > > > > >> >> > > > > Just to mention we would need to release
>> > > > > hadoop-thirdparty
>> > > > > > > too
>> > > > > > > >> >> > before.
>> > > > > > > >> >> > > > > Presently we are using the snapshot version of
>> it.
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > > -Ayush
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
>> > > > > > > >> weichiu@apache.org
>> > > > > > > >> >> >
>> > > > > > > >> >> > > > wrote:
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Hi all,
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago,
>> and as
>> > > of
>> > > > > now
>> > > > > > > >> we've
>> > > > > > > >> >> > > > > accumulated
>> > > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A
>> number
>> > > of
>> > > > > > > >> >> downstreamers
>> > > > > > > >> >> > > are
>> > > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
>> > guava
>> > > > > > version
>> > > > > > > >> >> > conflict
>> > > > > > > >> >> > > > > issue.
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> >
>> > > > > > > >> >>
>> > > > > > > >>
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > We should start the release work for 3.3.1
>> before
>> > > the
>> > > > > > diff
>> > > > > > > >> >> becomes
>> > > > > > > >> >> > > even
>> > > > > > > >> >> > > > > > larger.
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > I believe there are currently only two real
>> > > blockers
>> > > > > > for a
>> > > > > > > >> >> 3.3.1
>> > > > > > > >> >> > > > (using
>> > > > > > > >> >> > > > > > this filter
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> >
>> > > > > > > >> >>
>> > > > > > > >>
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
>> > > > > > > >> >> > > > > > )
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > 1. HDFS-15566 <
>> > > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
>> > > > > > > >> >> > > > > > 2.
>> > > > > > > >> >> > > > > > 1. HADOOP-17112 <
>> > > > > > > >> >> > > >
>> https://issues.apache.org/jira/browse/HADOOP-17112
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > 2.
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
>> > 3.3.1
>> > > > RM?
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe
>> the
>> > > ARM
>> > > > > > build
>> > > > > > > >> >> > process.
>> > > > > > > >> >> > > > > That's
>> > > > > > > >> >> > > > > > going to be important for future releases.
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >>
>> > > > > >
>> > ---------------------------------------------------------------------
>> > > > > > > >> >> > > > > To unsubscribe, e-mail:
>> > > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > > > > >> >> > > > > For additional commands, e-mail:
>> > > > > > > >> hdfs-dev-help@hadoop.apache.org
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> >
>> > > > > > > >> >>
>> > > > > > > >> >>
>> > > > > > > >> >> --
>> > > > > > > >> >>
>> > > > > > > >> >>
>> > > > > > > >> >>
>> > > > > > > >> >> --Brahma Reddy Battula
>> > > > > > > >> >>
>> > > > > > > >> >
>> > > > > > > >>
>> > > > > > > >> --
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >> --Brahma Reddy Battula
>> > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > > --
>> > > > -Vinay
>> > > >
>> > >
>> >
>>
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Hello it's me again.
While working on the hadoop-thirdparty release, I would like to make
progress on the Hadoop 3.3.1 release in parallel as well.
Looking at the jira dashboard, I think the only real blocker is HDFS-15790
https://github.com/apache/hadoop/pull/2767
I'll try to review the remaining jiras and push out those that don't have
much progress today.
Thanks all!
Weichiu
On Tue, Apr 20, 2021 at 12:57 PM Wei-Chiu Chuang <we...@cloudera.com>
wrote:
> Billie and Viraj,
>
> As of today we are at 657 resolved jiras in 3.3.1
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20status%20in%20(Resolved)%20AND%20fixVersion%20in%20(3.3.1)
> There are so many new features (
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.3+Release) I
> counted 10, that it is barely a dot release.
>
> Please make every effort to backport the said jiras to branch-3.3. I'd
> like to cut the branch sooner than later. I'd also call for more frequent
> releases. At the current development pace, a dot release every quarter
> would make sense IMO.
>
> Given the number of release blockers, I propose a tentative code freeze
> date: end of next Friday. After that I'd like to cut the branch and make an
> RC.
>
> Thoughts? I can't do this all alone. If you find things that are worth
> stopping the release, please tag the jira with the label 'release-blocker'.
>
>
>
> On Tue, Apr 20, 2021 at 2:44 AM Billie Rinaldi <bi...@apache.org> wrote:
>
>> I was thinking of backporting HADOOP-16948 for 3.3.1.
>>
>> Billie
>>
>> On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
>> <we...@cloudera.com.invalid> wrote:
>>
>> > Hello, reviving this thread.
>> >
>> > I created a dashboard for Hadoop 3.3.1 release.
>> >
>> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
>> > Also a jira to track the release work: HADOOP-17647
>> > <https://issues.apache.org/jira/browse/HADOOP-17647>
>> >
>> > We are currently at 5 release blockers and 3 critical issues for Hadoop
>> > 3.3.1. I'll go through each of them and push out the ones that aren't
>> > really blocking us.
>> >
>> > If you believe there are more features/bug fixes we should include in
>> 3.3.1
>> > (I spent the past few weeks backporting jiras but I'm sure I missed
>> some)
>> > please shout out.
>> >
>> > Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
>> There
>> > are a number of tasks to be done there too. Let's start another thread
>> for
>> > hadoop-thirdparty 1.1.0 release.
>> >
>> > On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <
>> hemanthboyina224@gmail.com
>> > >
>> > wrote:
>> >
>> > > Hi Steve and Wei-Chiu
>> > >
>> > > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
>> > trunk
>> > > and tried to work out with IPV6, we have faced some issues and have
>> made
>> > > the required changes for ipv6 to work.After the changes were made we
>> have
>> > > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
>> > > rigorously.Its been quite a some time these changes were deployed in
>> > > production cluster and have been in use for extensive purpose.
>> > >
>> > > I think it's good time to add this feature.
>> > >
>> > > Thanks
>> > > Hemanth Boyina
>> > >
>> > >
>> > >
>> > > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
>> > wrote:
>> > >
>> > > > Hi David,
>> > > >
>> > > > >> Still hoping for help here:
>> > > >
>> > > > >> https://issues.apache.org/jira/browse/HDFS-15790
>> > > >
>> > > > I will raise a PR for the said solution soon (in a day or two).
>> > > >
>> > > > -Vinay
>> > > >
>> > > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
>> > > >
>> > > > > Hello,
>> > > > >
>> > > > > Still hoping for help here:
>> > > > >
>> > > > > https://issues.apache.org/jira/browse/HDFS-15790
>> > > > >
>> > > > > Looks like it has been worked on, not sure how to best move it
>> > forward.
>> > > > >
>> > > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
>> > > > <stevel@cloudera.com.invalid
>> > > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > I'm going to argue its too late to do IPv6 support close to a
>> > > release,
>> > > > as
>> > > > > > it's best if its on developer machines for some time to let all
>> > > quirks
>> > > > > > surface. It's not so much IPv6 itself, but do we cause any
>> > > regressions
>> > > > on
>> > > > > > IPv4?
>> > > > > >
>> > > > > > But: it can/should go into trunk and stabilize there
>> > > > > >
>> > > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
>> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>> > > > > >
>> > > > > > > Hi Brahma,
>> > > > > > >
>> > > > > > > I have missed out mentioning about the IPV6 feature in the
>> last
>> > > mail,
>> > > > > > > Support for IPV6 has been in development since 2015 and We
>> have
>> > > done
>> > > > a
>> > > > > > good
>> > > > > > > amount of testing at our organisation, the feature is stable
>> and
>> > > used
>> > > > > by
>> > > > > > > our customers extensively in the last one year. I think it is
>> a
>> > > good
>> > > > > time
>> > > > > > > to add the IPV6 support to 3.3.1.
>> > > > > > >
>> > > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
>> > > > > > >
>> > > > > > > Thanks
>> > > > > > > D M Murali Krishna Reddy
>> > > > > > >
>> > > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
>> > > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>> > > > > > >
>> > > > > > > > Hi Brahma,
>> > > > > > > >
>> > > > > > > > Can we have this new feature "YARN Registry based AM
>> discovery
>> > > with
>> > > > > > retry
>> > > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
>> > > > > release. I
>> > > > > > > > have also attached a test-report in the below jira.
>> > > > > > > >
>> > > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > > D M Murali Krishna Reddy
>> > > > > > > >
>> > > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
>> > > > > > brahma@apache.org
>> > > > > > > >
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > >> Hi Bilwa,
>> > > > > > > >>
>> > > > > > > >> I have commented on the jira's you mentioned. Based on the
>> > > > stability
>> > > > > > we
>> > > > > > > >> can
>> > > > > > > >> plan this.But needs to be merged ASAP.
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <
>> stbilwa@gmail.com>
>> > > > wrote:
>> > > > > > > >>
>> > > > > > > >> > Hi Brahma,
>> > > > > > > >> >
>> > > > > > > >> > Can we have below features in 3.3.1 release? We have been
>> > > using
>> > > > > > these
>> > > > > > > >> > features for a long time. They are stable and tested in
>> > bigger
>> > > > > > > clusters.
>> > > > > > > >> >
>> > > > > > > >> > 1. Container reuse -
>> > > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
>> > > > > > > >> > 2. Speculative attempts should not run on the same node -
>> > > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
>> > > > > > > >> >
>> > > > > > > >> > Thanks,
>> > > > > > > >> > Bilwa
>> > > > > > > >> >
>> > > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
>> > > > > > brahma@apache.org
>> > > > > > > >
>> > > > > > > >> > wrote:
>> > > > > > > >> >
>> > > > > > > >> >> Sorry for the late reply..
>> > > > > > > >> >>
>> > > > > > > >> >> I will come up with a plan.. Please let me know if
>> anybody
>> > > has
>> > > > > some
>> > > > > > > >> >> features/improvements/bugs that need to be included.
>> > > > > > > >> >>
>> > > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
>> > > > > sunilg@apache.org>
>> > > > > > > >> wrote:
>> > > > > > > >> >>
>> > > > > > > >> >> > Hi Wei-Chiu,
>> > > > > > > >> >> >
>> > > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
>> > > > > > > >> >> >
>> > > > > > > >> >> > Thanks
>> > > > > > > >> >> > Sunil
>> > > > > > > >> >> >
>> > > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <
>> stack@duboce.net>
>> > > > > wrote:
>> > > > > > > >> >> >
>> > > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
>> > > > > > > >> >> > <stevel@cloudera.com.invalid
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > wrote:
>> > > > > > > >> >> > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > > Regarding blockers &c: how about we have a little
>> > > > hackathon
>> > > > > > > >> where we
>> > > > > > > >> >> > try
>> > > > > > > >> >> > > > and get things in. This means a promise of review
>> > time
>> > > > from
>> > > > > > the
>> > > > > > > >> >> people
>> > > > > > > >> >> > > with
>> > > > > > > >> >> > > > commit rights and other people who understand the
>> > code
>> > > > > > (Stack?)
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing,
>> hackathon,
>> > > > > > testing).
>> > > > > > > >> >> > > Thanks,
>> > > > > > > >> >> > > S
>> > > > > > > >> >> > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > > > -steve
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
>> > > > > > ayushtkn@gmail.com
>> > > > > > > >
>> > > > > > > >> >> wrote:
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > > > +1
>> > > > > > > >> >> > > > > Just to mention we would need to release
>> > > > > hadoop-thirdparty
>> > > > > > > too
>> > > > > > > >> >> > before.
>> > > > > > > >> >> > > > > Presently we are using the snapshot version of
>> it.
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > > -Ayush
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
>> > > > > > > >> weichiu@apache.org
>> > > > > > > >> >> >
>> > > > > > > >> >> > > > wrote:
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Hi all,
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago,
>> and as
>> > > of
>> > > > > now
>> > > > > > > >> we've
>> > > > > > > >> >> > > > > accumulated
>> > > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A
>> number
>> > > of
>> > > > > > > >> >> downstreamers
>> > > > > > > >> >> > > are
>> > > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
>> > guava
>> > > > > > version
>> > > > > > > >> >> > conflict
>> > > > > > > >> >> > > > > issue.
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> >
>> > > > > > > >> >>
>> > > > > > > >>
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > We should start the release work for 3.3.1
>> before
>> > > the
>> > > > > > diff
>> > > > > > > >> >> becomes
>> > > > > > > >> >> > > even
>> > > > > > > >> >> > > > > > larger.
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > I believe there are currently only two real
>> > > blockers
>> > > > > > for a
>> > > > > > > >> >> 3.3.1
>> > > > > > > >> >> > > > (using
>> > > > > > > >> >> > > > > > this filter
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> >
>> > > > > > > >> >>
>> > > > > > > >>
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
>> > > > > > > >> >> > > > > > )
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > 1. HDFS-15566 <
>> > > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
>> > > > > > > >> >> > > > > > 2.
>> > > > > > > >> >> > > > > > 1. HADOOP-17112 <
>> > > > > > > >> >> > > >
>> https://issues.apache.org/jira/browse/HADOOP-17112
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > 2.
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
>> > 3.3.1
>> > > > RM?
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe
>> the
>> > > ARM
>> > > > > > build
>> > > > > > > >> >> > process.
>> > > > > > > >> >> > > > > That's
>> > > > > > > >> >> > > > > > going to be important for future releases.
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >>
>> > > > > >
>> > ---------------------------------------------------------------------
>> > > > > > > >> >> > > > > To unsubscribe, e-mail:
>> > > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > > > > >> >> > > > > For additional commands, e-mail:
>> > > > > > > >> hdfs-dev-help@hadoop.apache.org
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> >
>> > > > > > > >> >>
>> > > > > > > >> >>
>> > > > > > > >> >> --
>> > > > > > > >> >>
>> > > > > > > >> >>
>> > > > > > > >> >>
>> > > > > > > >> >> --Brahma Reddy Battula
>> > > > > > > >> >>
>> > > > > > > >> >
>> > > > > > > >>
>> > > > > > > >> --
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >> --Brahma Reddy Battula
>> > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > > --
>> > > > -Vinay
>> > > >
>> > >
>> >
>>
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Hello it's me again.
While working on the hadoop-thirdparty release, I would like to make
progress on the Hadoop 3.3.1 release in parallel as well.
Looking at the jira dashboard, I think the only real blocker is HDFS-15790
https://github.com/apache/hadoop/pull/2767
I'll try to review the remaining jiras and push out those that don't have
much progress today.
Thanks all!
Weichiu
On Tue, Apr 20, 2021 at 12:57 PM Wei-Chiu Chuang <we...@cloudera.com>
wrote:
> Billie and Viraj,
>
> As of today we are at 657 resolved jiras in 3.3.1
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20status%20in%20(Resolved)%20AND%20fixVersion%20in%20(3.3.1)
> There are so many new features (
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.3+Release) I
> counted 10, that it is barely a dot release.
>
> Please make every effort to backport the said jiras to branch-3.3. I'd
> like to cut the branch sooner than later. I'd also call for more frequent
> releases. At the current development pace, a dot release every quarter
> would make sense IMO.
>
> Given the number of release blockers, I propose a tentative code freeze
> date: end of next Friday. After that I'd like to cut the branch and make an
> RC.
>
> Thoughts? I can't do this all alone. If you find things that are worth
> stopping the release, please tag the jira with the label 'release-blocker'.
>
>
>
> On Tue, Apr 20, 2021 at 2:44 AM Billie Rinaldi <bi...@apache.org> wrote:
>
>> I was thinking of backporting HADOOP-16948 for 3.3.1.
>>
>> Billie
>>
>> On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
>> <we...@cloudera.com.invalid> wrote:
>>
>> > Hello, reviving this thread.
>> >
>> > I created a dashboard for Hadoop 3.3.1 release.
>> >
>> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
>> > Also a jira to track the release work: HADOOP-17647
>> > <https://issues.apache.org/jira/browse/HADOOP-17647>
>> >
>> > We are currently at 5 release blockers and 3 critical issues for Hadoop
>> > 3.3.1. I'll go through each of them and push out the ones that aren't
>> > really blocking us.
>> >
>> > If you believe there are more features/bug fixes we should include in
>> 3.3.1
>> > (I spent the past few weeks backporting jiras but I'm sure I missed
>> some)
>> > please shout out.
>> >
>> > Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
>> There
>> > are a number of tasks to be done there too. Let's start another thread
>> for
>> > hadoop-thirdparty 1.1.0 release.
>> >
>> > On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <
>> hemanthboyina224@gmail.com
>> > >
>> > wrote:
>> >
>> > > Hi Steve and Wei-Chiu
>> > >
>> > > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
>> > trunk
>> > > and tried to work out with IPV6, we have faced some issues and have
>> made
>> > > the required changes for ipv6 to work.After the changes were made we
>> have
>> > > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
>> > > rigorously.Its been quite a some time these changes were deployed in
>> > > production cluster and have been in use for extensive purpose.
>> > >
>> > > I think it's good time to add this feature.
>> > >
>> > > Thanks
>> > > Hemanth Boyina
>> > >
>> > >
>> > >
>> > > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
>> > wrote:
>> > >
>> > > > Hi David,
>> > > >
>> > > > >> Still hoping for help here:
>> > > >
>> > > > >> https://issues.apache.org/jira/browse/HDFS-15790
>> > > >
>> > > > I will raise a PR for the said solution soon (in a day or two).
>> > > >
>> > > > -Vinay
>> > > >
>> > > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
>> > > >
>> > > > > Hello,
>> > > > >
>> > > > > Still hoping for help here:
>> > > > >
>> > > > > https://issues.apache.org/jira/browse/HDFS-15790
>> > > > >
>> > > > > Looks like it has been worked on, not sure how to best move it
>> > forward.
>> > > > >
>> > > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
>> > > > <stevel@cloudera.com.invalid
>> > > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > I'm going to argue its too late to do IPv6 support close to a
>> > > release,
>> > > > as
>> > > > > > it's best if its on developer machines for some time to let all
>> > > quirks
>> > > > > > surface. It's not so much IPv6 itself, but do we cause any
>> > > regressions
>> > > > on
>> > > > > > IPv4?
>> > > > > >
>> > > > > > But: it can/should go into trunk and stabilize there
>> > > > > >
>> > > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
>> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>> > > > > >
>> > > > > > > Hi Brahma,
>> > > > > > >
>> > > > > > > I have missed out mentioning about the IPV6 feature in the
>> last
>> > > mail,
>> > > > > > > Support for IPV6 has been in development since 2015 and We
>> have
>> > > done
>> > > > a
>> > > > > > good
>> > > > > > > amount of testing at our organisation, the feature is stable
>> and
>> > > used
>> > > > > by
>> > > > > > > our customers extensively in the last one year. I think it is
>> a
>> > > good
>> > > > > time
>> > > > > > > to add the IPV6 support to 3.3.1.
>> > > > > > >
>> > > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
>> > > > > > >
>> > > > > > > Thanks
>> > > > > > > D M Murali Krishna Reddy
>> > > > > > >
>> > > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
>> > > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>> > > > > > >
>> > > > > > > > Hi Brahma,
>> > > > > > > >
>> > > > > > > > Can we have this new feature "YARN Registry based AM
>> discovery
>> > > with
>> > > > > > retry
>> > > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
>> > > > > release. I
>> > > > > > > > have also attached a test-report in the below jira.
>> > > > > > > >
>> > > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > > D M Murali Krishna Reddy
>> > > > > > > >
>> > > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
>> > > > > > brahma@apache.org
>> > > > > > > >
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > >> Hi Bilwa,
>> > > > > > > >>
>> > > > > > > >> I have commented on the jira's you mentioned. Based on the
>> > > > stability
>> > > > > > we
>> > > > > > > >> can
>> > > > > > > >> plan this.But needs to be merged ASAP.
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <
>> stbilwa@gmail.com>
>> > > > wrote:
>> > > > > > > >>
>> > > > > > > >> > Hi Brahma,
>> > > > > > > >> >
>> > > > > > > >> > Can we have below features in 3.3.1 release? We have been
>> > > using
>> > > > > > these
>> > > > > > > >> > features for a long time. They are stable and tested in
>> > bigger
>> > > > > > > clusters.
>> > > > > > > >> >
>> > > > > > > >> > 1. Container reuse -
>> > > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
>> > > > > > > >> > 2. Speculative attempts should not run on the same node -
>> > > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
>> > > > > > > >> >
>> > > > > > > >> > Thanks,
>> > > > > > > >> > Bilwa
>> > > > > > > >> >
>> > > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
>> > > > > > brahma@apache.org
>> > > > > > > >
>> > > > > > > >> > wrote:
>> > > > > > > >> >
>> > > > > > > >> >> Sorry for the late reply..
>> > > > > > > >> >>
>> > > > > > > >> >> I will come up with a plan.. Please let me know if
>> anybody
>> > > has
>> > > > > some
>> > > > > > > >> >> features/improvements/bugs that need to be included.
>> > > > > > > >> >>
>> > > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
>> > > > > sunilg@apache.org>
>> > > > > > > >> wrote:
>> > > > > > > >> >>
>> > > > > > > >> >> > Hi Wei-Chiu,
>> > > > > > > >> >> >
>> > > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
>> > > > > > > >> >> >
>> > > > > > > >> >> > Thanks
>> > > > > > > >> >> > Sunil
>> > > > > > > >> >> >
>> > > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <
>> stack@duboce.net>
>> > > > > wrote:
>> > > > > > > >> >> >
>> > > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
>> > > > > > > >> >> > <stevel@cloudera.com.invalid
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > wrote:
>> > > > > > > >> >> > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > > Regarding blockers &c: how about we have a little
>> > > > hackathon
>> > > > > > > >> where we
>> > > > > > > >> >> > try
>> > > > > > > >> >> > > > and get things in. This means a promise of review
>> > time
>> > > > from
>> > > > > > the
>> > > > > > > >> >> people
>> > > > > > > >> >> > > with
>> > > > > > > >> >> > > > commit rights and other people who understand the
>> > code
>> > > > > > (Stack?)
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing,
>> hackathon,
>> > > > > > testing).
>> > > > > > > >> >> > > Thanks,
>> > > > > > > >> >> > > S
>> > > > > > > >> >> > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > > > -steve
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
>> > > > > > ayushtkn@gmail.com
>> > > > > > > >
>> > > > > > > >> >> wrote:
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > > > +1
>> > > > > > > >> >> > > > > Just to mention we would need to release
>> > > > > hadoop-thirdparty
>> > > > > > > too
>> > > > > > > >> >> > before.
>> > > > > > > >> >> > > > > Presently we are using the snapshot version of
>> it.
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > > -Ayush
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
>> > > > > > > >> weichiu@apache.org
>> > > > > > > >> >> >
>> > > > > > > >> >> > > > wrote:
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Hi all,
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago,
>> and as
>> > > of
>> > > > > now
>> > > > > > > >> we've
>> > > > > > > >> >> > > > > accumulated
>> > > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A
>> number
>> > > of
>> > > > > > > >> >> downstreamers
>> > > > > > > >> >> > > are
>> > > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
>> > guava
>> > > > > > version
>> > > > > > > >> >> > conflict
>> > > > > > > >> >> > > > > issue.
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> >
>> > > > > > > >> >>
>> > > > > > > >>
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > We should start the release work for 3.3.1
>> before
>> > > the
>> > > > > > diff
>> > > > > > > >> >> becomes
>> > > > > > > >> >> > > even
>> > > > > > > >> >> > > > > > larger.
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > I believe there are currently only two real
>> > > blockers
>> > > > > > for a
>> > > > > > > >> >> 3.3.1
>> > > > > > > >> >> > > > (using
>> > > > > > > >> >> > > > > > this filter
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> >
>> > > > > > > >> >>
>> > > > > > > >>
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
>> > > > > > > >> >> > > > > > )
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > 1. HDFS-15566 <
>> > > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
>> > > > > > > >> >> > > > > > 2.
>> > > > > > > >> >> > > > > > 1. HADOOP-17112 <
>> > > > > > > >> >> > > >
>> https://issues.apache.org/jira/browse/HADOOP-17112
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > 2.
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
>> > 3.3.1
>> > > > RM?
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe
>> the
>> > > ARM
>> > > > > > build
>> > > > > > > >> >> > process.
>> > > > > > > >> >> > > > > That's
>> > > > > > > >> >> > > > > > going to be important for future releases.
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >>
>> > > > > >
>> > ---------------------------------------------------------------------
>> > > > > > > >> >> > > > > To unsubscribe, e-mail:
>> > > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > > > > >> >> > > > > For additional commands, e-mail:
>> > > > > > > >> hdfs-dev-help@hadoop.apache.org
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> >
>> > > > > > > >> >>
>> > > > > > > >> >>
>> > > > > > > >> >> --
>> > > > > > > >> >>
>> > > > > > > >> >>
>> > > > > > > >> >>
>> > > > > > > >> >> --Brahma Reddy Battula
>> > > > > > > >> >>
>> > > > > > > >> >
>> > > > > > > >>
>> > > > > > > >> --
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >> --Brahma Reddy Battula
>> > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > > --
>> > > > -Vinay
>> > > >
>> > >
>> >
>>
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Hello it's me again.
While working on the hadoop-thirdparty release, I would like to make
progress on the Hadoop 3.3.1 release in parallel as well.
Looking at the jira dashboard, I think the only real blocker is HDFS-15790
https://github.com/apache/hadoop/pull/2767
I'll try to review the remaining jiras and push out those that don't have
much progress today.
Thanks all!
Weichiu
On Tue, Apr 20, 2021 at 12:57 PM Wei-Chiu Chuang <we...@cloudera.com>
wrote:
> Billie and Viraj,
>
> As of today we are at 657 resolved jiras in 3.3.1
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20status%20in%20(Resolved)%20AND%20fixVersion%20in%20(3.3.1)
> There are so many new features (
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.3+Release) I
> counted 10, that it is barely a dot release.
>
> Please make every effort to backport the said jiras to branch-3.3. I'd
> like to cut the branch sooner than later. I'd also call for more frequent
> releases. At the current development pace, a dot release every quarter
> would make sense IMO.
>
> Given the number of release blockers, I propose a tentative code freeze
> date: end of next Friday. After that I'd like to cut the branch and make an
> RC.
>
> Thoughts? I can't do this all alone. If you find things that are worth
> stopping the release, please tag the jira with the label 'release-blocker'.
>
>
>
> On Tue, Apr 20, 2021 at 2:44 AM Billie Rinaldi <bi...@apache.org> wrote:
>
>> I was thinking of backporting HADOOP-16948 for 3.3.1.
>>
>> Billie
>>
>> On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
>> <we...@cloudera.com.invalid> wrote:
>>
>> > Hello, reviving this thread.
>> >
>> > I created a dashboard for Hadoop 3.3.1 release.
>> >
>> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
>> > Also a jira to track the release work: HADOOP-17647
>> > <https://issues.apache.org/jira/browse/HADOOP-17647>
>> >
>> > We are currently at 5 release blockers and 3 critical issues for Hadoop
>> > 3.3.1. I'll go through each of them and push out the ones that aren't
>> > really blocking us.
>> >
>> > If you believe there are more features/bug fixes we should include in
>> 3.3.1
>> > (I spent the past few weeks backporting jiras but I'm sure I missed
>> some)
>> > please shout out.
>> >
>> > Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
>> There
>> > are a number of tasks to be done there too. Let's start another thread
>> for
>> > hadoop-thirdparty 1.1.0 release.
>> >
>> > On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <
>> hemanthboyina224@gmail.com
>> > >
>> > wrote:
>> >
>> > > Hi Steve and Wei-Chiu
>> > >
>> > > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
>> > trunk
>> > > and tried to work out with IPV6, we have faced some issues and have
>> made
>> > > the required changes for ipv6 to work.After the changes were made we
>> have
>> > > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
>> > > rigorously.Its been quite a some time these changes were deployed in
>> > > production cluster and have been in use for extensive purpose.
>> > >
>> > > I think it's good time to add this feature.
>> > >
>> > > Thanks
>> > > Hemanth Boyina
>> > >
>> > >
>> > >
>> > > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
>> > wrote:
>> > >
>> > > > Hi David,
>> > > >
>> > > > >> Still hoping for help here:
>> > > >
>> > > > >> https://issues.apache.org/jira/browse/HDFS-15790
>> > > >
>> > > > I will raise a PR for the said solution soon (in a day or two).
>> > > >
>> > > > -Vinay
>> > > >
>> > > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
>> > > >
>> > > > > Hello,
>> > > > >
>> > > > > Still hoping for help here:
>> > > > >
>> > > > > https://issues.apache.org/jira/browse/HDFS-15790
>> > > > >
>> > > > > Looks like it has been worked on, not sure how to best move it
>> > forward.
>> > > > >
>> > > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
>> > > > <stevel@cloudera.com.invalid
>> > > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > I'm going to argue its too late to do IPv6 support close to a
>> > > release,
>> > > > as
>> > > > > > it's best if its on developer machines for some time to let all
>> > > quirks
>> > > > > > surface. It's not so much IPv6 itself, but do we cause any
>> > > regressions
>> > > > on
>> > > > > > IPv4?
>> > > > > >
>> > > > > > But: it can/should go into trunk and stabilize there
>> > > > > >
>> > > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
>> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>> > > > > >
>> > > > > > > Hi Brahma,
>> > > > > > >
>> > > > > > > I have missed out mentioning about the IPV6 feature in the
>> last
>> > > mail,
>> > > > > > > Support for IPV6 has been in development since 2015 and We
>> have
>> > > done
>> > > > a
>> > > > > > good
>> > > > > > > amount of testing at our organisation, the feature is stable
>> and
>> > > used
>> > > > > by
>> > > > > > > our customers extensively in the last one year. I think it is
>> a
>> > > good
>> > > > > time
>> > > > > > > to add the IPV6 support to 3.3.1.
>> > > > > > >
>> > > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
>> > > > > > >
>> > > > > > > Thanks
>> > > > > > > D M Murali Krishna Reddy
>> > > > > > >
>> > > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
>> > > > > > > muralikrishna.dmmkr@gmail.com> wrote:
>> > > > > > >
>> > > > > > > > Hi Brahma,
>> > > > > > > >
>> > > > > > > > Can we have this new feature "YARN Registry based AM
>> discovery
>> > > with
>> > > > > > retry
>> > > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
>> > > > > release. I
>> > > > > > > > have also attached a test-report in the below jira.
>> > > > > > > >
>> > > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > > D M Murali Krishna Reddy
>> > > > > > > >
>> > > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
>> > > > > > brahma@apache.org
>> > > > > > > >
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > >> Hi Bilwa,
>> > > > > > > >>
>> > > > > > > >> I have commented on the jira's you mentioned. Based on the
>> > > > stability
>> > > > > > we
>> > > > > > > >> can
>> > > > > > > >> plan this.But needs to be merged ASAP.
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <
>> stbilwa@gmail.com>
>> > > > wrote:
>> > > > > > > >>
>> > > > > > > >> > Hi Brahma,
>> > > > > > > >> >
>> > > > > > > >> > Can we have below features in 3.3.1 release? We have been
>> > > using
>> > > > > > these
>> > > > > > > >> > features for a long time. They are stable and tested in
>> > bigger
>> > > > > > > clusters.
>> > > > > > > >> >
>> > > > > > > >> > 1. Container reuse -
>> > > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
>> > > > > > > >> > 2. Speculative attempts should not run on the same node -
>> > > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
>> > > > > > > >> >
>> > > > > > > >> > Thanks,
>> > > > > > > >> > Bilwa
>> > > > > > > >> >
>> > > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
>> > > > > > brahma@apache.org
>> > > > > > > >
>> > > > > > > >> > wrote:
>> > > > > > > >> >
>> > > > > > > >> >> Sorry for the late reply..
>> > > > > > > >> >>
>> > > > > > > >> >> I will come up with a plan.. Please let me know if
>> anybody
>> > > has
>> > > > > some
>> > > > > > > >> >> features/improvements/bugs that need to be included.
>> > > > > > > >> >>
>> > > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
>> > > > > sunilg@apache.org>
>> > > > > > > >> wrote:
>> > > > > > > >> >>
>> > > > > > > >> >> > Hi Wei-Chiu,
>> > > > > > > >> >> >
>> > > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
>> > > > > > > >> >> >
>> > > > > > > >> >> > Thanks
>> > > > > > > >> >> > Sunil
>> > > > > > > >> >> >
>> > > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <
>> stack@duboce.net>
>> > > > > wrote:
>> > > > > > > >> >> >
>> > > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
>> > > > > > > >> >> > <stevel@cloudera.com.invalid
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > wrote:
>> > > > > > > >> >> > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > > Regarding blockers &c: how about we have a little
>> > > > hackathon
>> > > > > > > >> where we
>> > > > > > > >> >> > try
>> > > > > > > >> >> > > > and get things in. This means a promise of review
>> > time
>> > > > from
>> > > > > > the
>> > > > > > > >> >> people
>> > > > > > > >> >> > > with
>> > > > > > > >> >> > > > commit rights and other people who understand the
>> > code
>> > > > > > (Stack?)
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing,
>> hackathon,
>> > > > > > testing).
>> > > > > > > >> >> > > Thanks,
>> > > > > > > >> >> > > S
>> > > > > > > >> >> > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> > > > -steve
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
>> > > > > > ayushtkn@gmail.com
>> > > > > > > >
>> > > > > > > >> >> wrote:
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > > > > +1
>> > > > > > > >> >> > > > > Just to mention we would need to release
>> > > > > hadoop-thirdparty
>> > > > > > > too
>> > > > > > > >> >> > before.
>> > > > > > > >> >> > > > > Presently we are using the snapshot version of
>> it.
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > > -Ayush
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
>> > > > > > > >> weichiu@apache.org
>> > > > > > > >> >> >
>> > > > > > > >> >> > > > wrote:
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Hi all,
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago,
>> and as
>> > > of
>> > > > > now
>> > > > > > > >> we've
>> > > > > > > >> >> > > > > accumulated
>> > > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A
>> number
>> > > of
>> > > > > > > >> >> downstreamers
>> > > > > > > >> >> > > are
>> > > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
>> > guava
>> > > > > > version
>> > > > > > > >> >> > conflict
>> > > > > > > >> >> > > > > issue.
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> >
>> > > > > > > >> >>
>> > > > > > > >>
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > We should start the release work for 3.3.1
>> before
>> > > the
>> > > > > > diff
>> > > > > > > >> >> becomes
>> > > > > > > >> >> > > even
>> > > > > > > >> >> > > > > > larger.
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > I believe there are currently only two real
>> > > blockers
>> > > > > > for a
>> > > > > > > >> >> 3.3.1
>> > > > > > > >> >> > > > (using
>> > > > > > > >> >> > > > > > this filter
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> >
>> > > > > > > >> >>
>> > > > > > > >>
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
>> > > > > > > >> >> > > > > > )
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > 1. HDFS-15566 <
>> > > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
>> > > > > > > >> >> > > > > > 2.
>> > > > > > > >> >> > > > > > 1. HADOOP-17112 <
>> > > > > > > >> >> > > >
>> https://issues.apache.org/jira/browse/HADOOP-17112
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > 2.
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
>> > 3.3.1
>> > > > RM?
>> > > > > > > >> >> > > > > >
>> > > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe
>> the
>> > > ARM
>> > > > > > build
>> > > > > > > >> >> > process.
>> > > > > > > >> >> > > > > That's
>> > > > > > > >> >> > > > > > going to be important for future releases.
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >>
>> > > > > >
>> > ---------------------------------------------------------------------
>> > > > > > > >> >> > > > > To unsubscribe, e-mail:
>> > > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > > > > >> >> > > > > For additional commands, e-mail:
>> > > > > > > >> hdfs-dev-help@hadoop.apache.org
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > > >
>> > > > > > > >> >> > > >
>> > > > > > > >> >> > >
>> > > > > > > >> >> >
>> > > > > > > >> >>
>> > > > > > > >> >>
>> > > > > > > >> >> --
>> > > > > > > >> >>
>> > > > > > > >> >>
>> > > > > > > >> >>
>> > > > > > > >> >> --Brahma Reddy Battula
>> > > > > > > >> >>
>> > > > > > > >> >
>> > > > > > > >>
>> > > > > > > >> --
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >>
>> > > > > > > >> --Brahma Reddy Battula
>> > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > > --
>> > > > -Vinay
>> > > >
>> > >
>> >
>>
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Billie and Viraj,
As of today we are at 657 resolved jiras in 3.3.1
https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20status%20in%20(Resolved)%20AND%20fixVersion%20in%20(3.3.1)
There are so many new features (
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.3+Release) I
counted 10, that it is barely a dot release.
Please make every effort to backport the said jiras to branch-3.3. I'd like
to cut the branch sooner than later. I'd also call for more frequent
releases. At the current development pace, a dot release every quarter
would make sense IMO.
Given the number of release blockers, I propose a tentative code freeze
date: end of next Friday. After that I'd like to cut the branch and make an
RC.
Thoughts? I can't do this all alone. If you find things that are worth
stopping the release, please tag the jira with the label 'release-blocker'.
On Tue, Apr 20, 2021 at 2:44 AM Billie Rinaldi <bi...@apache.org> wrote:
> I was thinking of backporting HADOOP-16948 for 3.3.1.
>
> Billie
>
> On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
> <we...@cloudera.com.invalid> wrote:
>
> > Hello, reviving this thread.
> >
> > I created a dashboard for Hadoop 3.3.1 release.
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
> > Also a jira to track the release work: HADOOP-17647
> > <https://issues.apache.org/jira/browse/HADOOP-17647>
> >
> > We are currently at 5 release blockers and 3 critical issues for Hadoop
> > 3.3.1. I'll go through each of them and push out the ones that aren't
> > really blocking us.
> >
> > If you believe there are more features/bug fixes we should include in
> 3.3.1
> > (I spent the past few weeks backporting jiras but I'm sure I missed some)
> > please shout out.
> >
> > Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
> There
> > are a number of tasks to be done there too. Let's start another thread
> for
> > hadoop-thirdparty 1.1.0 release.
> >
> > On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <
> hemanthboyina224@gmail.com
> > >
> > wrote:
> >
> > > Hi Steve and Wei-Chiu
> > >
> > > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
> > trunk
> > > and tried to work out with IPV6, we have faced some issues and have
> made
> > > the required changes for ipv6 to work.After the changes were made we
> have
> > > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> > > rigorously.Its been quite a some time these changes were deployed in
> > > production cluster and have been in use for extensive purpose.
> > >
> > > I think it's good time to add this feature.
> > >
> > > Thanks
> > > Hemanth Boyina
> > >
> > >
> > >
> > > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
> > wrote:
> > >
> > > > Hi David,
> > > >
> > > > >> Still hoping for help here:
> > > >
> > > > >> https://issues.apache.org/jira/browse/HDFS-15790
> > > >
> > > > I will raise a PR for the said solution soon (in a day or two).
> > > >
> > > > -Vinay
> > > >
> > > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > Still hoping for help here:
> > > > >
> > > > > https://issues.apache.org/jira/browse/HDFS-15790
> > > > >
> > > > > Looks like it has been worked on, not sure how to best move it
> > forward.
> > > > >
> > > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > > > <stevel@cloudera.com.invalid
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > I'm going to argue its too late to do IPv6 support close to a
> > > release,
> > > > as
> > > > > > it's best if its on developer machines for some time to let all
> > > quirks
> > > > > > surface. It's not so much IPv6 itself, but do we cause any
> > > regressions
> > > > on
> > > > > > IPv4?
> > > > > >
> > > > > > But: it can/should go into trunk and stabilize there
> > > > > >
> > > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > >
> > > > > > > Hi Brahma,
> > > > > > >
> > > > > > > I have missed out mentioning about the IPV6 feature in the last
> > > mail,
> > > > > > > Support for IPV6 has been in development since 2015 and We have
> > > done
> > > > a
> > > > > > good
> > > > > > > amount of testing at our organisation, the feature is stable
> and
> > > used
> > > > > by
> > > > > > > our customers extensively in the last one year. I think it is a
> > > good
> > > > > time
> > > > > > > to add the IPV6 support to 3.3.1.
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > > > >
> > > > > > > Thanks
> > > > > > > D M Murali Krishna Reddy
> > > > > > >
> > > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > > >
> > > > > > > > Hi Brahma,
> > > > > > > >
> > > > > > > > Can we have this new feature "YARN Registry based AM
> discovery
> > > with
> > > > > > retry
> > > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > > > release. I
> > > > > > > > have also attached a test-report in the below jira.
> > > > > > > >
> > > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > D M Murali Krishna Reddy
> > > > > > > >
> > > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > > > brahma@apache.org
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Hi Bilwa,
> > > > > > > >>
> > > > > > > >> I have commented on the jira's you mentioned. Based on the
> > > > stability
> > > > > > we
> > > > > > > >> can
> > > > > > > >> plan this.But needs to be merged ASAP.
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <stbilwa@gmail.com
> >
> > > > wrote:
> > > > > > > >>
> > > > > > > >> > Hi Brahma,
> > > > > > > >> >
> > > > > > > >> > Can we have below features in 3.3.1 release? We have been
> > > using
> > > > > > these
> > > > > > > >> > features for a long time. They are stable and tested in
> > bigger
> > > > > > > clusters.
> > > > > > > >> >
> > > > > > > >> > 1. Container reuse -
> > > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > > > >> >
> > > > > > > >> > Thanks,
> > > > > > > >> > Bilwa
> > > > > > > >> >
> > > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > > > brahma@apache.org
> > > > > > > >
> > > > > > > >> > wrote:
> > > > > > > >> >
> > > > > > > >> >> Sorry for the late reply..
> > > > > > > >> >>
> > > > > > > >> >> I will come up with a plan.. Please let me know if
> anybody
> > > has
> > > > > some
> > > > > > > >> >> features/improvements/bugs that need to be included.
> > > > > > > >> >>
> > > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > > > sunilg@apache.org>
> > > > > > > >> wrote:
> > > > > > > >> >>
> > > > > > > >> >> > Hi Wei-Chiu,
> > > > > > > >> >> >
> > > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > > > >> >> >
> > > > > > > >> >> > Thanks
> > > > > > > >> >> > Sunil
> > > > > > > >> >> >
> > > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <
> stack@duboce.net>
> > > > > wrote:
> > > > > > > >> >> >
> > > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > > > >> >> > > >
> > > > > > > >> >> > > wrote:
> > > > > > > >> >> > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > > > hackathon
> > > > > > > >> where we
> > > > > > > >> >> > try
> > > > > > > >> >> > > > and get things in. This means a promise of review
> > time
> > > > from
> > > > > > the
> > > > > > > >> >> people
> > > > > > > >> >> > > with
> > > > > > > >> >> > > > commit rights and other people who understand the
> > code
> > > > > > (Stack?)
> > > > > > > >> >> > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing,
> hackathon,
> > > > > > testing).
> > > > > > > >> >> > > Thanks,
> > > > > > > >> >> > > S
> > > > > > > >> >> > >
> > > > > > > >> >> > >
> > > > > > > >> >> > >
> > > > > > > >> >> > >
> > > > > > > >> >> > > > -steve
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > > > ayushtkn@gmail.com
> > > > > > > >
> > > > > > > >> >> wrote:
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > > +1
> > > > > > > >> >> > > > > Just to mention we would need to release
> > > > > hadoop-thirdparty
> > > > > > > too
> > > > > > > >> >> > before.
> > > > > > > >> >> > > > > Presently we are using the snapshot version of
> it.
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > > -Ayush
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > > > >> weichiu@apache.org
> > > > > > > >> >> >
> > > > > > > >> >> > > > wrote:
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Hi all,
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and
> as
> > > of
> > > > > now
> > > > > > > >> we've
> > > > > > > >> >> > > > > accumulated
> > > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A
> number
> > > of
> > > > > > > >> >> downstreamers
> > > > > > > >> >> > > are
> > > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
> > guava
> > > > > > version
> > > > > > > >> >> > conflict
> > > > > > > >> >> > > > > issue.
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> >
> > > > > > > >> >>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > We should start the release work for 3.3.1
> before
> > > the
> > > > > > diff
> > > > > > > >> >> becomes
> > > > > > > >> >> > > even
> > > > > > > >> >> > > > > > larger.
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > I believe there are currently only two real
> > > blockers
> > > > > > for a
> > > > > > > >> >> 3.3.1
> > > > > > > >> >> > > > (using
> > > > > > > >> >> > > > > > this filter
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> >
> > > > > > > >> >>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > > > >> >> > > > > > )
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > > > >> >> > > > > > 2.
> > > > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > 2.
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
> > 3.3.1
> > > > RM?
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe
> the
> > > ARM
> > > > > > build
> > > > > > > >> >> > process.
> > > > > > > >> >> > > > > That's
> > > > > > > >> >> > > > > > going to be important for future releases.
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >>
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > > > >> >> > > > > For additional commands, e-mail:
> > > > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> >
> > > > > > > >> >>
> > > > > > > >> >>
> > > > > > > >> >> --
> > > > > > > >> >>
> > > > > > > >> >>
> > > > > > > >> >>
> > > > > > > >> >> --Brahma Reddy Battula
> > > > > > > >> >>
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >> --
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> --Brahma Reddy Battula
> > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > --
> > > > -Vinay
> > > >
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Billie and Viraj,
As of today we are at 657 resolved jiras in 3.3.1
https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20status%20in%20(Resolved)%20AND%20fixVersion%20in%20(3.3.1)
There are so many new features (
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.3+Release) I
counted 10, that it is barely a dot release.
Please make every effort to backport the said jiras to branch-3.3. I'd like
to cut the branch sooner than later. I'd also call for more frequent
releases. At the current development pace, a dot release every quarter
would make sense IMO.
Given the number of release blockers, I propose a tentative code freeze
date: end of next Friday. After that I'd like to cut the branch and make an
RC.
Thoughts? I can't do this all alone. If you find things that are worth
stopping the release, please tag the jira with the label 'release-blocker'.
On Tue, Apr 20, 2021 at 2:44 AM Billie Rinaldi <bi...@apache.org> wrote:
> I was thinking of backporting HADOOP-16948 for 3.3.1.
>
> Billie
>
> On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
> <we...@cloudera.com.invalid> wrote:
>
> > Hello, reviving this thread.
> >
> > I created a dashboard for Hadoop 3.3.1 release.
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
> > Also a jira to track the release work: HADOOP-17647
> > <https://issues.apache.org/jira/browse/HADOOP-17647>
> >
> > We are currently at 5 release blockers and 3 critical issues for Hadoop
> > 3.3.1. I'll go through each of them and push out the ones that aren't
> > really blocking us.
> >
> > If you believe there are more features/bug fixes we should include in
> 3.3.1
> > (I spent the past few weeks backporting jiras but I'm sure I missed some)
> > please shout out.
> >
> > Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
> There
> > are a number of tasks to be done there too. Let's start another thread
> for
> > hadoop-thirdparty 1.1.0 release.
> >
> > On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <
> hemanthboyina224@gmail.com
> > >
> > wrote:
> >
> > > Hi Steve and Wei-Chiu
> > >
> > > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
> > trunk
> > > and tried to work out with IPV6, we have faced some issues and have
> made
> > > the required changes for ipv6 to work.After the changes were made we
> have
> > > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> > > rigorously.Its been quite a some time these changes were deployed in
> > > production cluster and have been in use for extensive purpose.
> > >
> > > I think it's good time to add this feature.
> > >
> > > Thanks
> > > Hemanth Boyina
> > >
> > >
> > >
> > > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
> > wrote:
> > >
> > > > Hi David,
> > > >
> > > > >> Still hoping for help here:
> > > >
> > > > >> https://issues.apache.org/jira/browse/HDFS-15790
> > > >
> > > > I will raise a PR for the said solution soon (in a day or two).
> > > >
> > > > -Vinay
> > > >
> > > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > Still hoping for help here:
> > > > >
> > > > > https://issues.apache.org/jira/browse/HDFS-15790
> > > > >
> > > > > Looks like it has been worked on, not sure how to best move it
> > forward.
> > > > >
> > > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > > > <stevel@cloudera.com.invalid
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > I'm going to argue its too late to do IPv6 support close to a
> > > release,
> > > > as
> > > > > > it's best if its on developer machines for some time to let all
> > > quirks
> > > > > > surface. It's not so much IPv6 itself, but do we cause any
> > > regressions
> > > > on
> > > > > > IPv4?
> > > > > >
> > > > > > But: it can/should go into trunk and stabilize there
> > > > > >
> > > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > >
> > > > > > > Hi Brahma,
> > > > > > >
> > > > > > > I have missed out mentioning about the IPV6 feature in the last
> > > mail,
> > > > > > > Support for IPV6 has been in development since 2015 and We have
> > > done
> > > > a
> > > > > > good
> > > > > > > amount of testing at our organisation, the feature is stable
> and
> > > used
> > > > > by
> > > > > > > our customers extensively in the last one year. I think it is a
> > > good
> > > > > time
> > > > > > > to add the IPV6 support to 3.3.1.
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > > > >
> > > > > > > Thanks
> > > > > > > D M Murali Krishna Reddy
> > > > > > >
> > > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > > >
> > > > > > > > Hi Brahma,
> > > > > > > >
> > > > > > > > Can we have this new feature "YARN Registry based AM
> discovery
> > > with
> > > > > > retry
> > > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > > > release. I
> > > > > > > > have also attached a test-report in the below jira.
> > > > > > > >
> > > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > D M Murali Krishna Reddy
> > > > > > > >
> > > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > > > brahma@apache.org
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Hi Bilwa,
> > > > > > > >>
> > > > > > > >> I have commented on the jira's you mentioned. Based on the
> > > > stability
> > > > > > we
> > > > > > > >> can
> > > > > > > >> plan this.But needs to be merged ASAP.
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <stbilwa@gmail.com
> >
> > > > wrote:
> > > > > > > >>
> > > > > > > >> > Hi Brahma,
> > > > > > > >> >
> > > > > > > >> > Can we have below features in 3.3.1 release? We have been
> > > using
> > > > > > these
> > > > > > > >> > features for a long time. They are stable and tested in
> > bigger
> > > > > > > clusters.
> > > > > > > >> >
> > > > > > > >> > 1. Container reuse -
> > > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > > > >> >
> > > > > > > >> > Thanks,
> > > > > > > >> > Bilwa
> > > > > > > >> >
> > > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > > > brahma@apache.org
> > > > > > > >
> > > > > > > >> > wrote:
> > > > > > > >> >
> > > > > > > >> >> Sorry for the late reply..
> > > > > > > >> >>
> > > > > > > >> >> I will come up with a plan.. Please let me know if
> anybody
> > > has
> > > > > some
> > > > > > > >> >> features/improvements/bugs that need to be included.
> > > > > > > >> >>
> > > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > > > sunilg@apache.org>
> > > > > > > >> wrote:
> > > > > > > >> >>
> > > > > > > >> >> > Hi Wei-Chiu,
> > > > > > > >> >> >
> > > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > > > >> >> >
> > > > > > > >> >> > Thanks
> > > > > > > >> >> > Sunil
> > > > > > > >> >> >
> > > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <
> stack@duboce.net>
> > > > > wrote:
> > > > > > > >> >> >
> > > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > > > >> >> > > >
> > > > > > > >> >> > > wrote:
> > > > > > > >> >> > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > > > hackathon
> > > > > > > >> where we
> > > > > > > >> >> > try
> > > > > > > >> >> > > > and get things in. This means a promise of review
> > time
> > > > from
> > > > > > the
> > > > > > > >> >> people
> > > > > > > >> >> > > with
> > > > > > > >> >> > > > commit rights and other people who understand the
> > code
> > > > > > (Stack?)
> > > > > > > >> >> > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing,
> hackathon,
> > > > > > testing).
> > > > > > > >> >> > > Thanks,
> > > > > > > >> >> > > S
> > > > > > > >> >> > >
> > > > > > > >> >> > >
> > > > > > > >> >> > >
> > > > > > > >> >> > >
> > > > > > > >> >> > > > -steve
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > > > ayushtkn@gmail.com
> > > > > > > >
> > > > > > > >> >> wrote:
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > > +1
> > > > > > > >> >> > > > > Just to mention we would need to release
> > > > > hadoop-thirdparty
> > > > > > > too
> > > > > > > >> >> > before.
> > > > > > > >> >> > > > > Presently we are using the snapshot version of
> it.
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > > -Ayush
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > > > >> weichiu@apache.org
> > > > > > > >> >> >
> > > > > > > >> >> > > > wrote:
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Hi all,
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and
> as
> > > of
> > > > > now
> > > > > > > >> we've
> > > > > > > >> >> > > > > accumulated
> > > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A
> number
> > > of
> > > > > > > >> >> downstreamers
> > > > > > > >> >> > > are
> > > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
> > guava
> > > > > > version
> > > > > > > >> >> > conflict
> > > > > > > >> >> > > > > issue.
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> >
> > > > > > > >> >>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > We should start the release work for 3.3.1
> before
> > > the
> > > > > > diff
> > > > > > > >> >> becomes
> > > > > > > >> >> > > even
> > > > > > > >> >> > > > > > larger.
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > I believe there are currently only two real
> > > blockers
> > > > > > for a
> > > > > > > >> >> 3.3.1
> > > > > > > >> >> > > > (using
> > > > > > > >> >> > > > > > this filter
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> >
> > > > > > > >> >>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > > > >> >> > > > > > )
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > > > >> >> > > > > > 2.
> > > > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > 2.
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
> > 3.3.1
> > > > RM?
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe
> the
> > > ARM
> > > > > > build
> > > > > > > >> >> > process.
> > > > > > > >> >> > > > > That's
> > > > > > > >> >> > > > > > going to be important for future releases.
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >>
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > > > >> >> > > > > For additional commands, e-mail:
> > > > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> >
> > > > > > > >> >>
> > > > > > > >> >>
> > > > > > > >> >> --
> > > > > > > >> >>
> > > > > > > >> >>
> > > > > > > >> >>
> > > > > > > >> >> --Brahma Reddy Battula
> > > > > > > >> >>
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >> --
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> --Brahma Reddy Battula
> > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > --
> > > > -Vinay
> > > >
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Billie and Viraj,
As of today we are at 657 resolved jiras in 3.3.1
https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20status%20in%20(Resolved)%20AND%20fixVersion%20in%20(3.3.1)
There are so many new features (
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.3+Release) I
counted 10, that it is barely a dot release.
Please make every effort to backport the said jiras to branch-3.3. I'd like
to cut the branch sooner than later. I'd also call for more frequent
releases. At the current development pace, a dot release every quarter
would make sense IMO.
Given the number of release blockers, I propose a tentative code freeze
date: end of next Friday. After that I'd like to cut the branch and make an
RC.
Thoughts? I can't do this all alone. If you find things that are worth
stopping the release, please tag the jira with the label 'release-blocker'.
On Tue, Apr 20, 2021 at 2:44 AM Billie Rinaldi <bi...@apache.org> wrote:
> I was thinking of backporting HADOOP-16948 for 3.3.1.
>
> Billie
>
> On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
> <we...@cloudera.com.invalid> wrote:
>
> > Hello, reviving this thread.
> >
> > I created a dashboard for Hadoop 3.3.1 release.
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
> > Also a jira to track the release work: HADOOP-17647
> > <https://issues.apache.org/jira/browse/HADOOP-17647>
> >
> > We are currently at 5 release blockers and 3 critical issues for Hadoop
> > 3.3.1. I'll go through each of them and push out the ones that aren't
> > really blocking us.
> >
> > If you believe there are more features/bug fixes we should include in
> 3.3.1
> > (I spent the past few weeks backporting jiras but I'm sure I missed some)
> > please shout out.
> >
> > Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
> There
> > are a number of tasks to be done there too. Let's start another thread
> for
> > hadoop-thirdparty 1.1.0 release.
> >
> > On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <
> hemanthboyina224@gmail.com
> > >
> > wrote:
> >
> > > Hi Steve and Wei-Chiu
> > >
> > > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
> > trunk
> > > and tried to work out with IPV6, we have faced some issues and have
> made
> > > the required changes for ipv6 to work.After the changes were made we
> have
> > > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> > > rigorously.Its been quite a some time these changes were deployed in
> > > production cluster and have been in use for extensive purpose.
> > >
> > > I think it's good time to add this feature.
> > >
> > > Thanks
> > > Hemanth Boyina
> > >
> > >
> > >
> > > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
> > wrote:
> > >
> > > > Hi David,
> > > >
> > > > >> Still hoping for help here:
> > > >
> > > > >> https://issues.apache.org/jira/browse/HDFS-15790
> > > >
> > > > I will raise a PR for the said solution soon (in a day or two).
> > > >
> > > > -Vinay
> > > >
> > > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > Still hoping for help here:
> > > > >
> > > > > https://issues.apache.org/jira/browse/HDFS-15790
> > > > >
> > > > > Looks like it has been worked on, not sure how to best move it
> > forward.
> > > > >
> > > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > > > <stevel@cloudera.com.invalid
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > I'm going to argue its too late to do IPv6 support close to a
> > > release,
> > > > as
> > > > > > it's best if its on developer machines for some time to let all
> > > quirks
> > > > > > surface. It's not so much IPv6 itself, but do we cause any
> > > regressions
> > > > on
> > > > > > IPv4?
> > > > > >
> > > > > > But: it can/should go into trunk and stabilize there
> > > > > >
> > > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > >
> > > > > > > Hi Brahma,
> > > > > > >
> > > > > > > I have missed out mentioning about the IPV6 feature in the last
> > > mail,
> > > > > > > Support for IPV6 has been in development since 2015 and We have
> > > done
> > > > a
> > > > > > good
> > > > > > > amount of testing at our organisation, the feature is stable
> and
> > > used
> > > > > by
> > > > > > > our customers extensively in the last one year. I think it is a
> > > good
> > > > > time
> > > > > > > to add the IPV6 support to 3.3.1.
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > > > >
> > > > > > > Thanks
> > > > > > > D M Murali Krishna Reddy
> > > > > > >
> > > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > > >
> > > > > > > > Hi Brahma,
> > > > > > > >
> > > > > > > > Can we have this new feature "YARN Registry based AM
> discovery
> > > with
> > > > > > retry
> > > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > > > release. I
> > > > > > > > have also attached a test-report in the below jira.
> > > > > > > >
> > > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > D M Murali Krishna Reddy
> > > > > > > >
> > > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > > > brahma@apache.org
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Hi Bilwa,
> > > > > > > >>
> > > > > > > >> I have commented on the jira's you mentioned. Based on the
> > > > stability
> > > > > > we
> > > > > > > >> can
> > > > > > > >> plan this.But needs to be merged ASAP.
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <stbilwa@gmail.com
> >
> > > > wrote:
> > > > > > > >>
> > > > > > > >> > Hi Brahma,
> > > > > > > >> >
> > > > > > > >> > Can we have below features in 3.3.1 release? We have been
> > > using
> > > > > > these
> > > > > > > >> > features for a long time. They are stable and tested in
> > bigger
> > > > > > > clusters.
> > > > > > > >> >
> > > > > > > >> > 1. Container reuse -
> > > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > > > >> >
> > > > > > > >> > Thanks,
> > > > > > > >> > Bilwa
> > > > > > > >> >
> > > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > > > brahma@apache.org
> > > > > > > >
> > > > > > > >> > wrote:
> > > > > > > >> >
> > > > > > > >> >> Sorry for the late reply..
> > > > > > > >> >>
> > > > > > > >> >> I will come up with a plan.. Please let me know if
> anybody
> > > has
> > > > > some
> > > > > > > >> >> features/improvements/bugs that need to be included.
> > > > > > > >> >>
> > > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > > > sunilg@apache.org>
> > > > > > > >> wrote:
> > > > > > > >> >>
> > > > > > > >> >> > Hi Wei-Chiu,
> > > > > > > >> >> >
> > > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > > > >> >> >
> > > > > > > >> >> > Thanks
> > > > > > > >> >> > Sunil
> > > > > > > >> >> >
> > > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <
> stack@duboce.net>
> > > > > wrote:
> > > > > > > >> >> >
> > > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > > > >> >> > > >
> > > > > > > >> >> > > wrote:
> > > > > > > >> >> > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > > > hackathon
> > > > > > > >> where we
> > > > > > > >> >> > try
> > > > > > > >> >> > > > and get things in. This means a promise of review
> > time
> > > > from
> > > > > > the
> > > > > > > >> >> people
> > > > > > > >> >> > > with
> > > > > > > >> >> > > > commit rights and other people who understand the
> > code
> > > > > > (Stack?)
> > > > > > > >> >> > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing,
> hackathon,
> > > > > > testing).
> > > > > > > >> >> > > Thanks,
> > > > > > > >> >> > > S
> > > > > > > >> >> > >
> > > > > > > >> >> > >
> > > > > > > >> >> > >
> > > > > > > >> >> > >
> > > > > > > >> >> > > > -steve
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > > > ayushtkn@gmail.com
> > > > > > > >
> > > > > > > >> >> wrote:
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > > +1
> > > > > > > >> >> > > > > Just to mention we would need to release
> > > > > hadoop-thirdparty
> > > > > > > too
> > > > > > > >> >> > before.
> > > > > > > >> >> > > > > Presently we are using the snapshot version of
> it.
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > > -Ayush
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > > > >> weichiu@apache.org
> > > > > > > >> >> >
> > > > > > > >> >> > > > wrote:
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Hi all,
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and
> as
> > > of
> > > > > now
> > > > > > > >> we've
> > > > > > > >> >> > > > > accumulated
> > > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A
> number
> > > of
> > > > > > > >> >> downstreamers
> > > > > > > >> >> > > are
> > > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
> > guava
> > > > > > version
> > > > > > > >> >> > conflict
> > > > > > > >> >> > > > > issue.
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> >
> > > > > > > >> >>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > We should start the release work for 3.3.1
> before
> > > the
> > > > > > diff
> > > > > > > >> >> becomes
> > > > > > > >> >> > > even
> > > > > > > >> >> > > > > > larger.
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > I believe there are currently only two real
> > > blockers
> > > > > > for a
> > > > > > > >> >> 3.3.1
> > > > > > > >> >> > > > (using
> > > > > > > >> >> > > > > > this filter
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> >
> > > > > > > >> >>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > > > >> >> > > > > > )
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > > > >> >> > > > > > 2.
> > > > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > 2.
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
> > 3.3.1
> > > > RM?
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe
> the
> > > ARM
> > > > > > build
> > > > > > > >> >> > process.
> > > > > > > >> >> > > > > That's
> > > > > > > >> >> > > > > > going to be important for future releases.
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >>
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > > > >> >> > > > > For additional commands, e-mail:
> > > > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> >
> > > > > > > >> >>
> > > > > > > >> >>
> > > > > > > >> >> --
> > > > > > > >> >>
> > > > > > > >> >>
> > > > > > > >> >>
> > > > > > > >> >> --Brahma Reddy Battula
> > > > > > > >> >>
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >> --
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> --Brahma Reddy Battula
> > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > --
> > > > -Vinay
> > > >
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Billie and Viraj,
As of today we are at 657 resolved jiras in 3.3.1
https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20status%20in%20(Resolved)%20AND%20fixVersion%20in%20(3.3.1)
There are so many new features (
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.3+Release) I
counted 10, that it is barely a dot release.
Please make every effort to backport the said jiras to branch-3.3. I'd like
to cut the branch sooner than later. I'd also call for more frequent
releases. At the current development pace, a dot release every quarter
would make sense IMO.
Given the number of release blockers, I propose a tentative code freeze
date: end of next Friday. After that I'd like to cut the branch and make an
RC.
Thoughts? I can't do this all alone. If you find things that are worth
stopping the release, please tag the jira with the label 'release-blocker'.
On Tue, Apr 20, 2021 at 2:44 AM Billie Rinaldi <bi...@apache.org> wrote:
> I was thinking of backporting HADOOP-16948 for 3.3.1.
>
> Billie
>
> On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
> <we...@cloudera.com.invalid> wrote:
>
> > Hello, reviving this thread.
> >
> > I created a dashboard for Hadoop 3.3.1 release.
> >
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
> > Also a jira to track the release work: HADOOP-17647
> > <https://issues.apache.org/jira/browse/HADOOP-17647>
> >
> > We are currently at 5 release blockers and 3 critical issues for Hadoop
> > 3.3.1. I'll go through each of them and push out the ones that aren't
> > really blocking us.
> >
> > If you believe there are more features/bug fixes we should include in
> 3.3.1
> > (I spent the past few weeks backporting jiras but I'm sure I missed some)
> > please shout out.
> >
> > Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
> There
> > are a number of tasks to be done there too. Let's start another thread
> for
> > hadoop-thirdparty 1.1.0 release.
> >
> > On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <
> hemanthboyina224@gmail.com
> > >
> > wrote:
> >
> > > Hi Steve and Wei-Chiu
> > >
> > > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
> > trunk
> > > and tried to work out with IPV6, we have faced some issues and have
> made
> > > the required changes for ipv6 to work.After the changes were made we
> have
> > > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> > > rigorously.Its been quite a some time these changes were deployed in
> > > production cluster and have been in use for extensive purpose.
> > >
> > > I think it's good time to add this feature.
> > >
> > > Thanks
> > > Hemanth Boyina
> > >
> > >
> > >
> > > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
> > wrote:
> > >
> > > > Hi David,
> > > >
> > > > >> Still hoping for help here:
> > > >
> > > > >> https://issues.apache.org/jira/browse/HDFS-15790
> > > >
> > > > I will raise a PR for the said solution soon (in a day or two).
> > > >
> > > > -Vinay
> > > >
> > > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > Still hoping for help here:
> > > > >
> > > > > https://issues.apache.org/jira/browse/HDFS-15790
> > > > >
> > > > > Looks like it has been worked on, not sure how to best move it
> > forward.
> > > > >
> > > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > > > <stevel@cloudera.com.invalid
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > I'm going to argue its too late to do IPv6 support close to a
> > > release,
> > > > as
> > > > > > it's best if its on developer machines for some time to let all
> > > quirks
> > > > > > surface. It's not so much IPv6 itself, but do we cause any
> > > regressions
> > > > on
> > > > > > IPv4?
> > > > > >
> > > > > > But: it can/should go into trunk and stabilize there
> > > > > >
> > > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > >
> > > > > > > Hi Brahma,
> > > > > > >
> > > > > > > I have missed out mentioning about the IPV6 feature in the last
> > > mail,
> > > > > > > Support for IPV6 has been in development since 2015 and We have
> > > done
> > > > a
> > > > > > good
> > > > > > > amount of testing at our organisation, the feature is stable
> and
> > > used
> > > > > by
> > > > > > > our customers extensively in the last one year. I think it is a
> > > good
> > > > > time
> > > > > > > to add the IPV6 support to 3.3.1.
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > > > >
> > > > > > > Thanks
> > > > > > > D M Murali Krishna Reddy
> > > > > > >
> > > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > > >
> > > > > > > > Hi Brahma,
> > > > > > > >
> > > > > > > > Can we have this new feature "YARN Registry based AM
> discovery
> > > with
> > > > > > retry
> > > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > > > release. I
> > > > > > > > have also attached a test-report in the below jira.
> > > > > > > >
> > > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > D M Murali Krishna Reddy
> > > > > > > >
> > > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > > > brahma@apache.org
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Hi Bilwa,
> > > > > > > >>
> > > > > > > >> I have commented on the jira's you mentioned. Based on the
> > > > stability
> > > > > > we
> > > > > > > >> can
> > > > > > > >> plan this.But needs to be merged ASAP.
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <stbilwa@gmail.com
> >
> > > > wrote:
> > > > > > > >>
> > > > > > > >> > Hi Brahma,
> > > > > > > >> >
> > > > > > > >> > Can we have below features in 3.3.1 release? We have been
> > > using
> > > > > > these
> > > > > > > >> > features for a long time. They are stable and tested in
> > bigger
> > > > > > > clusters.
> > > > > > > >> >
> > > > > > > >> > 1. Container reuse -
> > > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > > > >> >
> > > > > > > >> > Thanks,
> > > > > > > >> > Bilwa
> > > > > > > >> >
> > > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > > > brahma@apache.org
> > > > > > > >
> > > > > > > >> > wrote:
> > > > > > > >> >
> > > > > > > >> >> Sorry for the late reply..
> > > > > > > >> >>
> > > > > > > >> >> I will come up with a plan.. Please let me know if
> anybody
> > > has
> > > > > some
> > > > > > > >> >> features/improvements/bugs that need to be included.
> > > > > > > >> >>
> > > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > > > sunilg@apache.org>
> > > > > > > >> wrote:
> > > > > > > >> >>
> > > > > > > >> >> > Hi Wei-Chiu,
> > > > > > > >> >> >
> > > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > > > >> >> >
> > > > > > > >> >> > Thanks
> > > > > > > >> >> > Sunil
> > > > > > > >> >> >
> > > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <
> stack@duboce.net>
> > > > > wrote:
> > > > > > > >> >> >
> > > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > > > >> >> > > >
> > > > > > > >> >> > > wrote:
> > > > > > > >> >> > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > > > hackathon
> > > > > > > >> where we
> > > > > > > >> >> > try
> > > > > > > >> >> > > > and get things in. This means a promise of review
> > time
> > > > from
> > > > > > the
> > > > > > > >> >> people
> > > > > > > >> >> > > with
> > > > > > > >> >> > > > commit rights and other people who understand the
> > code
> > > > > > (Stack?)
> > > > > > > >> >> > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing,
> hackathon,
> > > > > > testing).
> > > > > > > >> >> > > Thanks,
> > > > > > > >> >> > > S
> > > > > > > >> >> > >
> > > > > > > >> >> > >
> > > > > > > >> >> > >
> > > > > > > >> >> > >
> > > > > > > >> >> > > > -steve
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > > > ayushtkn@gmail.com
> > > > > > > >
> > > > > > > >> >> wrote:
> > > > > > > >> >> > > >
> > > > > > > >> >> > > > > +1
> > > > > > > >> >> > > > > Just to mention we would need to release
> > > > > hadoop-thirdparty
> > > > > > > too
> > > > > > > >> >> > before.
> > > > > > > >> >> > > > > Presently we are using the snapshot version of
> it.
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > > -Ayush
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > > > >> weichiu@apache.org
> > > > > > > >> >> >
> > > > > > > >> >> > > > wrote:
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Hi all,
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and
> as
> > > of
> > > > > now
> > > > > > > >> we've
> > > > > > > >> >> > > > > accumulated
> > > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A
> number
> > > of
> > > > > > > >> >> downstreamers
> > > > > > > >> >> > > are
> > > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
> > guava
> > > > > > version
> > > > > > > >> >> > conflict
> > > > > > > >> >> > > > > issue.
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> >
> > > > > > > >> >>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > We should start the release work for 3.3.1
> before
> > > the
> > > > > > diff
> > > > > > > >> >> becomes
> > > > > > > >> >> > > even
> > > > > > > >> >> > > > > > larger.
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > I believe there are currently only two real
> > > blockers
> > > > > > for a
> > > > > > > >> >> 3.3.1
> > > > > > > >> >> > > > (using
> > > > > > > >> >> > > > > > this filter
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> >
> > > > > > > >> >>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > > > >> >> > > > > > )
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > > > >> >> > > > > > 2.
> > > > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > 2.
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
> > 3.3.1
> > > > RM?
> > > > > > > >> >> > > > > >
> > > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe
> the
> > > ARM
> > > > > > build
> > > > > > > >> >> > process.
> > > > > > > >> >> > > > > That's
> > > > > > > >> >> > > > > > going to be important for future releases.
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >>
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > > > >> >> > > > > For additional commands, e-mail:
> > > > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > > >
> > > > > > > >> >> > > >
> > > > > > > >> >> > >
> > > > > > > >> >> >
> > > > > > > >> >>
> > > > > > > >> >>
> > > > > > > >> >> --
> > > > > > > >> >>
> > > > > > > >> >>
> > > > > > > >> >>
> > > > > > > >> >> --Brahma Reddy Battula
> > > > > > > >> >>
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >> --
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> --Brahma Reddy Battula
> > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > --
> > > > -Vinay
> > > >
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Billie Rinaldi <bi...@apache.org>.
I was thinking of backporting HADOOP-16948 for 3.3.1.
Billie
On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
<we...@cloudera.com.invalid> wrote:
> Hello, reviving this thread.
>
> I created a dashboard for Hadoop 3.3.1 release.
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
> Also a jira to track the release work: HADOOP-17647
> <https://issues.apache.org/jira/browse/HADOOP-17647>
>
> We are currently at 5 release blockers and 3 critical issues for Hadoop
> 3.3.1. I'll go through each of them and push out the ones that aren't
> really blocking us.
>
> If you believe there are more features/bug fixes we should include in 3.3.1
> (I spent the past few weeks backporting jiras but I'm sure I missed some)
> please shout out.
>
> Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too. There
> are a number of tasks to be done there too. Let's start another thread for
> hadoop-thirdparty 1.1.0 release.
>
> On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <hemanthboyina224@gmail.com
> >
> wrote:
>
> > Hi Steve and Wei-Chiu
> >
> > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
> trunk
> > and tried to work out with IPV6, we have faced some issues and have made
> > the required changes for ipv6 to work.After the changes were made we have
> > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> > rigorously.Its been quite a some time these changes were deployed in
> > production cluster and have been in use for extensive purpose.
> >
> > I think it's good time to add this feature.
> >
> > Thanks
> > Hemanth Boyina
> >
> >
> >
> > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
> wrote:
> >
> > > Hi David,
> > >
> > > >> Still hoping for help here:
> > >
> > > >> https://issues.apache.org/jira/browse/HDFS-15790
> > >
> > > I will raise a PR for the said solution soon (in a day or two).
> > >
> > > -Vinay
> > >
> > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > Still hoping for help here:
> > > >
> > > > https://issues.apache.org/jira/browse/HDFS-15790
> > > >
> > > > Looks like it has been worked on, not sure how to best move it
> forward.
> > > >
> > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > > <stevel@cloudera.com.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > I'm going to argue its too late to do IPv6 support close to a
> > release,
> > > as
> > > > > it's best if its on developer machines for some time to let all
> > quirks
> > > > > surface. It's not so much IPv6 itself, but do we cause any
> > regressions
> > > on
> > > > > IPv4?
> > > > >
> > > > > But: it can/should go into trunk and stabilize there
> > > > >
> > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > >
> > > > > > Hi Brahma,
> > > > > >
> > > > > > I have missed out mentioning about the IPV6 feature in the last
> > mail,
> > > > > > Support for IPV6 has been in development since 2015 and We have
> > done
> > > a
> > > > > good
> > > > > > amount of testing at our organisation, the feature is stable and
> > used
> > > > by
> > > > > > our customers extensively in the last one year. I think it is a
> > good
> > > > time
> > > > > > to add the IPV6 support to 3.3.1.
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > > >
> > > > > > Thanks
> > > > > > D M Murali Krishna Reddy
> > > > > >
> > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > >
> > > > > > > Hi Brahma,
> > > > > > >
> > > > > > > Can we have this new feature "YARN Registry based AM discovery
> > with
> > > > > retry
> > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > > release. I
> > > > > > > have also attached a test-report in the below jira.
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > D M Murali Krishna Reddy
> > > > > > >
> > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Bilwa,
> > > > > > >>
> > > > > > >> I have commented on the jira's you mentioned. Based on the
> > > stability
> > > > > we
> > > > > > >> can
> > > > > > >> plan this.But needs to be merged ASAP.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> > > wrote:
> > > > > > >>
> > > > > > >> > Hi Brahma,
> > > > > > >> >
> > > > > > >> > Can we have below features in 3.3.1 release? We have been
> > using
> > > > > these
> > > > > > >> > features for a long time. They are stable and tested in
> bigger
> > > > > > clusters.
> > > > > > >> >
> > > > > > >> > 1. Container reuse -
> > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > > >> >
> > > > > > >> > Thanks,
> > > > > > >> > Bilwa
> > > > > > >> >
> > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> >> Sorry for the late reply..
> > > > > > >> >>
> > > > > > >> >> I will come up with a plan.. Please let me know if anybody
> > has
> > > > some
> > > > > > >> >> features/improvements/bugs that need to be included.
> > > > > > >> >>
> > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > > sunilg@apache.org>
> > > > > > >> wrote:
> > > > > > >> >>
> > > > > > >> >> > Hi Wei-Chiu,
> > > > > > >> >> >
> > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > > >> >> >
> > > > > > >> >> > Thanks
> > > > > > >> >> > Sunil
> > > > > > >> >> >
> > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > > > wrote:
> > > > > > >> >> >
> > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > > >> >> > > >
> > > > > > >> >> > > wrote:
> > > > > > >> >> > >
> > > > > > >> >> > > >
> > > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > > hackathon
> > > > > > >> where we
> > > > > > >> >> > try
> > > > > > >> >> > > > and get things in. This means a promise of review
> time
> > > from
> > > > > the
> > > > > > >> >> people
> > > > > > >> >> > > with
> > > > > > >> >> > > > commit rights and other people who understand the
> code
> > > > > (Stack?)
> > > > > > >> >> > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > > > testing).
> > > > > > >> >> > > Thanks,
> > > > > > >> >> > > S
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > > > -steve
> > > > > > >> >> > > >
> > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > > ayushtkn@gmail.com
> > > > > > >
> > > > > > >> >> wrote:
> > > > > > >> >> > > >
> > > > > > >> >> > > > > +1
> > > > > > >> >> > > > > Just to mention we would need to release
> > > > hadoop-thirdparty
> > > > > > too
> > > > > > >> >> > before.
> > > > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > -Ayush
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > > >> weichiu@apache.org
> > > > > > >> >> >
> > > > > > >> >> > > > wrote:
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hi all,
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as
> > of
> > > > now
> > > > > > >> we've
> > > > > > >> >> > > > > accumulated
> > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number
> > of
> > > > > > >> >> downstreamers
> > > > > > >> >> > > are
> > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
> guava
> > > > > version
> > > > > > >> >> > conflict
> > > > > > >> >> > > > > issue.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > We should start the release work for 3.3.1 before
> > the
> > > > > diff
> > > > > > >> >> becomes
> > > > > > >> >> > > even
> > > > > > >> >> > > > > > larger.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > I believe there are currently only two real
> > blockers
> > > > > for a
> > > > > > >> >> 3.3.1
> > > > > > >> >> > > > (using
> > > > > > >> >> > > > > > this filter
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > > >> >> > > > > > )
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
> 3.3.1
> > > RM?
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the
> > ARM
> > > > > build
> > > > > > >> >> > process.
> > > > > > >> >> > > > > That's
> > > > > > >> >> > > > > > going to be important for future releases.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >>
> > > > >
> ---------------------------------------------------------------------
> > > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > > >> >> > > > > For additional commands, e-mail:
> > > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --Brahma Reddy Battula
> > > > > > >> >>
> > > > > > >> >
> > > > > > >>
> > > > > > >> --
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> --Brahma Reddy Battula
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > --
> > > -Vinay
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Billie Rinaldi <bi...@apache.org>.
I was thinking of backporting HADOOP-16948 for 3.3.1.
Billie
On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
<we...@cloudera.com.invalid> wrote:
> Hello, reviving this thread.
>
> I created a dashboard for Hadoop 3.3.1 release.
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
> Also a jira to track the release work: HADOOP-17647
> <https://issues.apache.org/jira/browse/HADOOP-17647>
>
> We are currently at 5 release blockers and 3 critical issues for Hadoop
> 3.3.1. I'll go through each of them and push out the ones that aren't
> really blocking us.
>
> If you believe there are more features/bug fixes we should include in 3.3.1
> (I spent the past few weeks backporting jiras but I'm sure I missed some)
> please shout out.
>
> Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too. There
> are a number of tasks to be done there too. Let's start another thread for
> hadoop-thirdparty 1.1.0 release.
>
> On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <hemanthboyina224@gmail.com
> >
> wrote:
>
> > Hi Steve and Wei-Chiu
> >
> > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
> trunk
> > and tried to work out with IPV6, we have faced some issues and have made
> > the required changes for ipv6 to work.After the changes were made we have
> > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> > rigorously.Its been quite a some time these changes were deployed in
> > production cluster and have been in use for extensive purpose.
> >
> > I think it's good time to add this feature.
> >
> > Thanks
> > Hemanth Boyina
> >
> >
> >
> > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
> wrote:
> >
> > > Hi David,
> > >
> > > >> Still hoping for help here:
> > >
> > > >> https://issues.apache.org/jira/browse/HDFS-15790
> > >
> > > I will raise a PR for the said solution soon (in a day or two).
> > >
> > > -Vinay
> > >
> > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > Still hoping for help here:
> > > >
> > > > https://issues.apache.org/jira/browse/HDFS-15790
> > > >
> > > > Looks like it has been worked on, not sure how to best move it
> forward.
> > > >
> > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > > <stevel@cloudera.com.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > I'm going to argue its too late to do IPv6 support close to a
> > release,
> > > as
> > > > > it's best if its on developer machines for some time to let all
> > quirks
> > > > > surface. It's not so much IPv6 itself, but do we cause any
> > regressions
> > > on
> > > > > IPv4?
> > > > >
> > > > > But: it can/should go into trunk and stabilize there
> > > > >
> > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > >
> > > > > > Hi Brahma,
> > > > > >
> > > > > > I have missed out mentioning about the IPV6 feature in the last
> > mail,
> > > > > > Support for IPV6 has been in development since 2015 and We have
> > done
> > > a
> > > > > good
> > > > > > amount of testing at our organisation, the feature is stable and
> > used
> > > > by
> > > > > > our customers extensively in the last one year. I think it is a
> > good
> > > > time
> > > > > > to add the IPV6 support to 3.3.1.
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > > >
> > > > > > Thanks
> > > > > > D M Murali Krishna Reddy
> > > > > >
> > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > >
> > > > > > > Hi Brahma,
> > > > > > >
> > > > > > > Can we have this new feature "YARN Registry based AM discovery
> > with
> > > > > retry
> > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > > release. I
> > > > > > > have also attached a test-report in the below jira.
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > D M Murali Krishna Reddy
> > > > > > >
> > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Bilwa,
> > > > > > >>
> > > > > > >> I have commented on the jira's you mentioned. Based on the
> > > stability
> > > > > we
> > > > > > >> can
> > > > > > >> plan this.But needs to be merged ASAP.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> > > wrote:
> > > > > > >>
> > > > > > >> > Hi Brahma,
> > > > > > >> >
> > > > > > >> > Can we have below features in 3.3.1 release? We have been
> > using
> > > > > these
> > > > > > >> > features for a long time. They are stable and tested in
> bigger
> > > > > > clusters.
> > > > > > >> >
> > > > > > >> > 1. Container reuse -
> > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > > >> >
> > > > > > >> > Thanks,
> > > > > > >> > Bilwa
> > > > > > >> >
> > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> >> Sorry for the late reply..
> > > > > > >> >>
> > > > > > >> >> I will come up with a plan.. Please let me know if anybody
> > has
> > > > some
> > > > > > >> >> features/improvements/bugs that need to be included.
> > > > > > >> >>
> > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > > sunilg@apache.org>
> > > > > > >> wrote:
> > > > > > >> >>
> > > > > > >> >> > Hi Wei-Chiu,
> > > > > > >> >> >
> > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > > >> >> >
> > > > > > >> >> > Thanks
> > > > > > >> >> > Sunil
> > > > > > >> >> >
> > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > > > wrote:
> > > > > > >> >> >
> > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > > >> >> > > >
> > > > > > >> >> > > wrote:
> > > > > > >> >> > >
> > > > > > >> >> > > >
> > > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > > hackathon
> > > > > > >> where we
> > > > > > >> >> > try
> > > > > > >> >> > > > and get things in. This means a promise of review
> time
> > > from
> > > > > the
> > > > > > >> >> people
> > > > > > >> >> > > with
> > > > > > >> >> > > > commit rights and other people who understand the
> code
> > > > > (Stack?)
> > > > > > >> >> > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > > > testing).
> > > > > > >> >> > > Thanks,
> > > > > > >> >> > > S
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > > > -steve
> > > > > > >> >> > > >
> > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > > ayushtkn@gmail.com
> > > > > > >
> > > > > > >> >> wrote:
> > > > > > >> >> > > >
> > > > > > >> >> > > > > +1
> > > > > > >> >> > > > > Just to mention we would need to release
> > > > hadoop-thirdparty
> > > > > > too
> > > > > > >> >> > before.
> > > > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > -Ayush
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > > >> weichiu@apache.org
> > > > > > >> >> >
> > > > > > >> >> > > > wrote:
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hi all,
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as
> > of
> > > > now
> > > > > > >> we've
> > > > > > >> >> > > > > accumulated
> > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number
> > of
> > > > > > >> >> downstreamers
> > > > > > >> >> > > are
> > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
> guava
> > > > > version
> > > > > > >> >> > conflict
> > > > > > >> >> > > > > issue.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > We should start the release work for 3.3.1 before
> > the
> > > > > diff
> > > > > > >> >> becomes
> > > > > > >> >> > > even
> > > > > > >> >> > > > > > larger.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > I believe there are currently only two real
> > blockers
> > > > > for a
> > > > > > >> >> 3.3.1
> > > > > > >> >> > > > (using
> > > > > > >> >> > > > > > this filter
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > > >> >> > > > > > )
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
> 3.3.1
> > > RM?
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the
> > ARM
> > > > > build
> > > > > > >> >> > process.
> > > > > > >> >> > > > > That's
> > > > > > >> >> > > > > > going to be important for future releases.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >>
> > > > >
> ---------------------------------------------------------------------
> > > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > > >> >> > > > > For additional commands, e-mail:
> > > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --Brahma Reddy Battula
> > > > > > >> >>
> > > > > > >> >
> > > > > > >>
> > > > > > >> --
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> --Brahma Reddy Battula
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > --
> > > -Vinay
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Viraj Jasani <vj...@apache.org>.
Thanks Wei-Chiu.
> If you believe there are more features/bug fixes we should include in
3.3.1
(I spent the past few weeks backporting jiras but I'm sure I missed some)
please shout out.
Although not marked critical, I believe HDFS-15982 could get in 3.3.1 if
you are fine. Thanks to Bhavik for filing this.
> Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
I agree we should work on this to upgrade some dependencies from known CVEs
(e.g guava). However, does this mean Hadoop 3.3.1 has dependency
on hadoop-thirdparty 1.1.0? (as we would like to upgrade guava and other
shaded thirdparty dependencies for 3.3.1 as well)
On Mon, Apr 19, 2021 at 11:04 AM Wei-Chiu Chuang
<we...@cloudera.com.invalid> wrote:
> Hello, reviving this thread.
>
> I created a dashboard for Hadoop 3.3.1 release.
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
> Also a jira to track the release work: HADOOP-17647
> <https://issues.apache.org/jira/browse/HADOOP-17647>
>
> We are currently at 5 release blockers and 3 critical issues for Hadoop
> 3.3.1. I'll go through each of them and push out the ones that aren't
> really blocking us.
>
> If you believe there are more features/bug fixes we should include in 3.3.1
> (I spent the past few weeks backporting jiras but I'm sure I missed some)
> please shout out.
>
> Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too. There
> are a number of tasks to be done there too. Let's start another thread for
> hadoop-thirdparty 1.1.0 release.
>
> On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <hemanthboyina224@gmail.com
> >
> wrote:
>
> > Hi Steve and Wei-Chiu
> >
> > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
> trunk
> > and tried to work out with IPV6, we have faced some issues and have made
> > the required changes for ipv6 to work.After the changes were made we have
> > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> > rigorously.Its been quite a some time these changes were deployed in
> > production cluster and have been in use for extensive purpose.
> >
> > I think it's good time to add this feature.
> >
> > Thanks
> > Hemanth Boyina
> >
> >
> >
> > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
> wrote:
> >
> > > Hi David,
> > >
> > > >> Still hoping for help here:
> > >
> > > >> https://issues.apache.org/jira/browse/HDFS-15790
> > >
> > > I will raise a PR for the said solution soon (in a day or two).
> > >
> > > -Vinay
> > >
> > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > Still hoping for help here:
> > > >
> > > > https://issues.apache.org/jira/browse/HDFS-15790
> > > >
> > > > Looks like it has been worked on, not sure how to best move it
> forward.
> > > >
> > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > > <stevel@cloudera.com.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > I'm going to argue its too late to do IPv6 support close to a
> > release,
> > > as
> > > > > it's best if its on developer machines for some time to let all
> > quirks
> > > > > surface. It's not so much IPv6 itself, but do we cause any
> > regressions
> > > on
> > > > > IPv4?
> > > > >
> > > > > But: it can/should go into trunk and stabilize there
> > > > >
> > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > >
> > > > > > Hi Brahma,
> > > > > >
> > > > > > I have missed out mentioning about the IPV6 feature in the last
> > mail,
> > > > > > Support for IPV6 has been in development since 2015 and We have
> > done
> > > a
> > > > > good
> > > > > > amount of testing at our organisation, the feature is stable and
> > used
> > > > by
> > > > > > our customers extensively in the last one year. I think it is a
> > good
> > > > time
> > > > > > to add the IPV6 support to 3.3.1.
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > > >
> > > > > > Thanks
> > > > > > D M Murali Krishna Reddy
> > > > > >
> > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > >
> > > > > > > Hi Brahma,
> > > > > > >
> > > > > > > Can we have this new feature "YARN Registry based AM discovery
> > with
> > > > > retry
> > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > > release. I
> > > > > > > have also attached a test-report in the below jira.
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > D M Murali Krishna Reddy
> > > > > > >
> > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Bilwa,
> > > > > > >>
> > > > > > >> I have commented on the jira's you mentioned. Based on the
> > > stability
> > > > > we
> > > > > > >> can
> > > > > > >> plan this.But needs to be merged ASAP.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> > > wrote:
> > > > > > >>
> > > > > > >> > Hi Brahma,
> > > > > > >> >
> > > > > > >> > Can we have below features in 3.3.1 release? We have been
> > using
> > > > > these
> > > > > > >> > features for a long time. They are stable and tested in
> bigger
> > > > > > clusters.
> > > > > > >> >
> > > > > > >> > 1. Container reuse -
> > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > > >> >
> > > > > > >> > Thanks,
> > > > > > >> > Bilwa
> > > > > > >> >
> > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> >> Sorry for the late reply..
> > > > > > >> >>
> > > > > > >> >> I will come up with a plan.. Please let me know if anybody
> > has
> > > > some
> > > > > > >> >> features/improvements/bugs that need to be included.
> > > > > > >> >>
> > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > > sunilg@apache.org>
> > > > > > >> wrote:
> > > > > > >> >>
> > > > > > >> >> > Hi Wei-Chiu,
> > > > > > >> >> >
> > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > > >> >> >
> > > > > > >> >> > Thanks
> > > > > > >> >> > Sunil
> > > > > > >> >> >
> > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > > > wrote:
> > > > > > >> >> >
> > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > > >> >> > > >
> > > > > > >> >> > > wrote:
> > > > > > >> >> > >
> > > > > > >> >> > > >
> > > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > > hackathon
> > > > > > >> where we
> > > > > > >> >> > try
> > > > > > >> >> > > > and get things in. This means a promise of review
> time
> > > from
> > > > > the
> > > > > > >> >> people
> > > > > > >> >> > > with
> > > > > > >> >> > > > commit rights and other people who understand the
> code
> > > > > (Stack?)
> > > > > > >> >> > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > > > testing).
> > > > > > >> >> > > Thanks,
> > > > > > >> >> > > S
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > > > -steve
> > > > > > >> >> > > >
> > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > > ayushtkn@gmail.com
> > > > > > >
> > > > > > >> >> wrote:
> > > > > > >> >> > > >
> > > > > > >> >> > > > > +1
> > > > > > >> >> > > > > Just to mention we would need to release
> > > > hadoop-thirdparty
> > > > > > too
> > > > > > >> >> > before.
> > > > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > -Ayush
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > > >> weichiu@apache.org
> > > > > > >> >> >
> > > > > > >> >> > > > wrote:
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hi all,
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as
> > of
> > > > now
> > > > > > >> we've
> > > > > > >> >> > > > > accumulated
> > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number
> > of
> > > > > > >> >> downstreamers
> > > > > > >> >> > > are
> > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
> guava
> > > > > version
> > > > > > >> >> > conflict
> > > > > > >> >> > > > > issue.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > We should start the release work for 3.3.1 before
> > the
> > > > > diff
> > > > > > >> >> becomes
> > > > > > >> >> > > even
> > > > > > >> >> > > > > > larger.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > I believe there are currently only two real
> > blockers
> > > > > for a
> > > > > > >> >> 3.3.1
> > > > > > >> >> > > > (using
> > > > > > >> >> > > > > > this filter
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > > >> >> > > > > > )
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
> 3.3.1
> > > RM?
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the
> > ARM
> > > > > build
> > > > > > >> >> > process.
> > > > > > >> >> > > > > That's
> > > > > > >> >> > > > > > going to be important for future releases.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >>
> > > > >
> ---------------------------------------------------------------------
> > > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > > >> >> > > > > For additional commands, e-mail:
> > > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --Brahma Reddy Battula
> > > > > > >> >>
> > > > > > >> >
> > > > > > >>
> > > > > > >> --
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> --Brahma Reddy Battula
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > --
> > > -Vinay
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Viraj Jasani <vj...@apache.org>.
Thanks Wei-Chiu.
> If you believe there are more features/bug fixes we should include in
3.3.1
(I spent the past few weeks backporting jiras but I'm sure I missed some)
please shout out.
Although not marked critical, I believe HDFS-15982 could get in 3.3.1 if
you are fine. Thanks to Bhavik for filing this.
> Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
I agree we should work on this to upgrade some dependencies from known CVEs
(e.g guava). However, does this mean Hadoop 3.3.1 has dependency
on hadoop-thirdparty 1.1.0? (as we would like to upgrade guava and other
shaded thirdparty dependencies for 3.3.1 as well)
On Mon, Apr 19, 2021 at 11:04 AM Wei-Chiu Chuang
<we...@cloudera.com.invalid> wrote:
> Hello, reviving this thread.
>
> I created a dashboard for Hadoop 3.3.1 release.
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
> Also a jira to track the release work: HADOOP-17647
> <https://issues.apache.org/jira/browse/HADOOP-17647>
>
> We are currently at 5 release blockers and 3 critical issues for Hadoop
> 3.3.1. I'll go through each of them and push out the ones that aren't
> really blocking us.
>
> If you believe there are more features/bug fixes we should include in 3.3.1
> (I spent the past few weeks backporting jiras but I'm sure I missed some)
> please shout out.
>
> Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too. There
> are a number of tasks to be done there too. Let's start another thread for
> hadoop-thirdparty 1.1.0 release.
>
> On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <hemanthboyina224@gmail.com
> >
> wrote:
>
> > Hi Steve and Wei-Chiu
> >
> > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
> trunk
> > and tried to work out with IPV6, we have faced some issues and have made
> > the required changes for ipv6 to work.After the changes were made we have
> > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> > rigorously.Its been quite a some time these changes were deployed in
> > production cluster and have been in use for extensive purpose.
> >
> > I think it's good time to add this feature.
> >
> > Thanks
> > Hemanth Boyina
> >
> >
> >
> > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
> wrote:
> >
> > > Hi David,
> > >
> > > >> Still hoping for help here:
> > >
> > > >> https://issues.apache.org/jira/browse/HDFS-15790
> > >
> > > I will raise a PR for the said solution soon (in a day or two).
> > >
> > > -Vinay
> > >
> > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > Still hoping for help here:
> > > >
> > > > https://issues.apache.org/jira/browse/HDFS-15790
> > > >
> > > > Looks like it has been worked on, not sure how to best move it
> forward.
> > > >
> > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > > <stevel@cloudera.com.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > I'm going to argue its too late to do IPv6 support close to a
> > release,
> > > as
> > > > > it's best if its on developer machines for some time to let all
> > quirks
> > > > > surface. It's not so much IPv6 itself, but do we cause any
> > regressions
> > > on
> > > > > IPv4?
> > > > >
> > > > > But: it can/should go into trunk and stabilize there
> > > > >
> > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > >
> > > > > > Hi Brahma,
> > > > > >
> > > > > > I have missed out mentioning about the IPV6 feature in the last
> > mail,
> > > > > > Support for IPV6 has been in development since 2015 and We have
> > done
> > > a
> > > > > good
> > > > > > amount of testing at our organisation, the feature is stable and
> > used
> > > > by
> > > > > > our customers extensively in the last one year. I think it is a
> > good
> > > > time
> > > > > > to add the IPV6 support to 3.3.1.
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > > >
> > > > > > Thanks
> > > > > > D M Murali Krishna Reddy
> > > > > >
> > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > >
> > > > > > > Hi Brahma,
> > > > > > >
> > > > > > > Can we have this new feature "YARN Registry based AM discovery
> > with
> > > > > retry
> > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > > release. I
> > > > > > > have also attached a test-report in the below jira.
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > D M Murali Krishna Reddy
> > > > > > >
> > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Bilwa,
> > > > > > >>
> > > > > > >> I have commented on the jira's you mentioned. Based on the
> > > stability
> > > > > we
> > > > > > >> can
> > > > > > >> plan this.But needs to be merged ASAP.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> > > wrote:
> > > > > > >>
> > > > > > >> > Hi Brahma,
> > > > > > >> >
> > > > > > >> > Can we have below features in 3.3.1 release? We have been
> > using
> > > > > these
> > > > > > >> > features for a long time. They are stable and tested in
> bigger
> > > > > > clusters.
> > > > > > >> >
> > > > > > >> > 1. Container reuse -
> > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > > >> >
> > > > > > >> > Thanks,
> > > > > > >> > Bilwa
> > > > > > >> >
> > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> >> Sorry for the late reply..
> > > > > > >> >>
> > > > > > >> >> I will come up with a plan.. Please let me know if anybody
> > has
> > > > some
> > > > > > >> >> features/improvements/bugs that need to be included.
> > > > > > >> >>
> > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > > sunilg@apache.org>
> > > > > > >> wrote:
> > > > > > >> >>
> > > > > > >> >> > Hi Wei-Chiu,
> > > > > > >> >> >
> > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > > >> >> >
> > > > > > >> >> > Thanks
> > > > > > >> >> > Sunil
> > > > > > >> >> >
> > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > > > wrote:
> > > > > > >> >> >
> > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > > >> >> > > >
> > > > > > >> >> > > wrote:
> > > > > > >> >> > >
> > > > > > >> >> > > >
> > > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > > hackathon
> > > > > > >> where we
> > > > > > >> >> > try
> > > > > > >> >> > > > and get things in. This means a promise of review
> time
> > > from
> > > > > the
> > > > > > >> >> people
> > > > > > >> >> > > with
> > > > > > >> >> > > > commit rights and other people who understand the
> code
> > > > > (Stack?)
> > > > > > >> >> > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > > > testing).
> > > > > > >> >> > > Thanks,
> > > > > > >> >> > > S
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > > > -steve
> > > > > > >> >> > > >
> > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > > ayushtkn@gmail.com
> > > > > > >
> > > > > > >> >> wrote:
> > > > > > >> >> > > >
> > > > > > >> >> > > > > +1
> > > > > > >> >> > > > > Just to mention we would need to release
> > > > hadoop-thirdparty
> > > > > > too
> > > > > > >> >> > before.
> > > > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > -Ayush
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > > >> weichiu@apache.org
> > > > > > >> >> >
> > > > > > >> >> > > > wrote:
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hi all,
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as
> > of
> > > > now
> > > > > > >> we've
> > > > > > >> >> > > > > accumulated
> > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number
> > of
> > > > > > >> >> downstreamers
> > > > > > >> >> > > are
> > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
> guava
> > > > > version
> > > > > > >> >> > conflict
> > > > > > >> >> > > > > issue.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > We should start the release work for 3.3.1 before
> > the
> > > > > diff
> > > > > > >> >> becomes
> > > > > > >> >> > > even
> > > > > > >> >> > > > > > larger.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > I believe there are currently only two real
> > blockers
> > > > > for a
> > > > > > >> >> 3.3.1
> > > > > > >> >> > > > (using
> > > > > > >> >> > > > > > this filter
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > > >> >> > > > > > )
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
> 3.3.1
> > > RM?
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the
> > ARM
> > > > > build
> > > > > > >> >> > process.
> > > > > > >> >> > > > > That's
> > > > > > >> >> > > > > > going to be important for future releases.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >>
> > > > >
> ---------------------------------------------------------------------
> > > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > > >> >> > > > > For additional commands, e-mail:
> > > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --Brahma Reddy Battula
> > > > > > >> >>
> > > > > > >> >
> > > > > > >>
> > > > > > >> --
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> --Brahma Reddy Battula
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > --
> > > -Vinay
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Billie Rinaldi <bi...@apache.org>.
I was thinking of backporting HADOOP-16948 for 3.3.1.
Billie
On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
<we...@cloudera.com.invalid> wrote:
> Hello, reviving this thread.
>
> I created a dashboard for Hadoop 3.3.1 release.
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
> Also a jira to track the release work: HADOOP-17647
> <https://issues.apache.org/jira/browse/HADOOP-17647>
>
> We are currently at 5 release blockers and 3 critical issues for Hadoop
> 3.3.1. I'll go through each of them and push out the ones that aren't
> really blocking us.
>
> If you believe there are more features/bug fixes we should include in 3.3.1
> (I spent the past few weeks backporting jiras but I'm sure I missed some)
> please shout out.
>
> Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too. There
> are a number of tasks to be done there too. Let's start another thread for
> hadoop-thirdparty 1.1.0 release.
>
> On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <hemanthboyina224@gmail.com
> >
> wrote:
>
> > Hi Steve and Wei-Chiu
> >
> > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
> trunk
> > and tried to work out with IPV6, we have faced some issues and have made
> > the required changes for ipv6 to work.After the changes were made we have
> > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> > rigorously.Its been quite a some time these changes were deployed in
> > production cluster and have been in use for extensive purpose.
> >
> > I think it's good time to add this feature.
> >
> > Thanks
> > Hemanth Boyina
> >
> >
> >
> > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
> wrote:
> >
> > > Hi David,
> > >
> > > >> Still hoping for help here:
> > >
> > > >> https://issues.apache.org/jira/browse/HDFS-15790
> > >
> > > I will raise a PR for the said solution soon (in a day or two).
> > >
> > > -Vinay
> > >
> > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > Still hoping for help here:
> > > >
> > > > https://issues.apache.org/jira/browse/HDFS-15790
> > > >
> > > > Looks like it has been worked on, not sure how to best move it
> forward.
> > > >
> > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > > <stevel@cloudera.com.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > I'm going to argue its too late to do IPv6 support close to a
> > release,
> > > as
> > > > > it's best if its on developer machines for some time to let all
> > quirks
> > > > > surface. It's not so much IPv6 itself, but do we cause any
> > regressions
> > > on
> > > > > IPv4?
> > > > >
> > > > > But: it can/should go into trunk and stabilize there
> > > > >
> > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > >
> > > > > > Hi Brahma,
> > > > > >
> > > > > > I have missed out mentioning about the IPV6 feature in the last
> > mail,
> > > > > > Support for IPV6 has been in development since 2015 and We have
> > done
> > > a
> > > > > good
> > > > > > amount of testing at our organisation, the feature is stable and
> > used
> > > > by
> > > > > > our customers extensively in the last one year. I think it is a
> > good
> > > > time
> > > > > > to add the IPV6 support to 3.3.1.
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > > >
> > > > > > Thanks
> > > > > > D M Murali Krishna Reddy
> > > > > >
> > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > >
> > > > > > > Hi Brahma,
> > > > > > >
> > > > > > > Can we have this new feature "YARN Registry based AM discovery
> > with
> > > > > retry
> > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > > release. I
> > > > > > > have also attached a test-report in the below jira.
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > D M Murali Krishna Reddy
> > > > > > >
> > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Bilwa,
> > > > > > >>
> > > > > > >> I have commented on the jira's you mentioned. Based on the
> > > stability
> > > > > we
> > > > > > >> can
> > > > > > >> plan this.But needs to be merged ASAP.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> > > wrote:
> > > > > > >>
> > > > > > >> > Hi Brahma,
> > > > > > >> >
> > > > > > >> > Can we have below features in 3.3.1 release? We have been
> > using
> > > > > these
> > > > > > >> > features for a long time. They are stable and tested in
> bigger
> > > > > > clusters.
> > > > > > >> >
> > > > > > >> > 1. Container reuse -
> > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > > >> >
> > > > > > >> > Thanks,
> > > > > > >> > Bilwa
> > > > > > >> >
> > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> >> Sorry for the late reply..
> > > > > > >> >>
> > > > > > >> >> I will come up with a plan.. Please let me know if anybody
> > has
> > > > some
> > > > > > >> >> features/improvements/bugs that need to be included.
> > > > > > >> >>
> > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > > sunilg@apache.org>
> > > > > > >> wrote:
> > > > > > >> >>
> > > > > > >> >> > Hi Wei-Chiu,
> > > > > > >> >> >
> > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > > >> >> >
> > > > > > >> >> > Thanks
> > > > > > >> >> > Sunil
> > > > > > >> >> >
> > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > > > wrote:
> > > > > > >> >> >
> > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > > >> >> > > >
> > > > > > >> >> > > wrote:
> > > > > > >> >> > >
> > > > > > >> >> > > >
> > > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > > hackathon
> > > > > > >> where we
> > > > > > >> >> > try
> > > > > > >> >> > > > and get things in. This means a promise of review
> time
> > > from
> > > > > the
> > > > > > >> >> people
> > > > > > >> >> > > with
> > > > > > >> >> > > > commit rights and other people who understand the
> code
> > > > > (Stack?)
> > > > > > >> >> > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > > > testing).
> > > > > > >> >> > > Thanks,
> > > > > > >> >> > > S
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > > > -steve
> > > > > > >> >> > > >
> > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > > ayushtkn@gmail.com
> > > > > > >
> > > > > > >> >> wrote:
> > > > > > >> >> > > >
> > > > > > >> >> > > > > +1
> > > > > > >> >> > > > > Just to mention we would need to release
> > > > hadoop-thirdparty
> > > > > > too
> > > > > > >> >> > before.
> > > > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > -Ayush
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > > >> weichiu@apache.org
> > > > > > >> >> >
> > > > > > >> >> > > > wrote:
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hi all,
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as
> > of
> > > > now
> > > > > > >> we've
> > > > > > >> >> > > > > accumulated
> > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number
> > of
> > > > > > >> >> downstreamers
> > > > > > >> >> > > are
> > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
> guava
> > > > > version
> > > > > > >> >> > conflict
> > > > > > >> >> > > > > issue.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > We should start the release work for 3.3.1 before
> > the
> > > > > diff
> > > > > > >> >> becomes
> > > > > > >> >> > > even
> > > > > > >> >> > > > > > larger.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > I believe there are currently only two real
> > blockers
> > > > > for a
> > > > > > >> >> 3.3.1
> > > > > > >> >> > > > (using
> > > > > > >> >> > > > > > this filter
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > > >> >> > > > > > )
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
> 3.3.1
> > > RM?
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the
> > ARM
> > > > > build
> > > > > > >> >> > process.
> > > > > > >> >> > > > > That's
> > > > > > >> >> > > > > > going to be important for future releases.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >>
> > > > >
> ---------------------------------------------------------------------
> > > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > > >> >> > > > > For additional commands, e-mail:
> > > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --Brahma Reddy Battula
> > > > > > >> >>
> > > > > > >> >
> > > > > > >>
> > > > > > >> --
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> --Brahma Reddy Battula
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > --
> > > -Vinay
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Viraj Jasani <vj...@apache.org>.
Thanks Wei-Chiu.
> If you believe there are more features/bug fixes we should include in
3.3.1
(I spent the past few weeks backporting jiras but I'm sure I missed some)
please shout out.
Although not marked critical, I believe HDFS-15982 could get in 3.3.1 if
you are fine. Thanks to Bhavik for filing this.
> Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
I agree we should work on this to upgrade some dependencies from known CVEs
(e.g guava). However, does this mean Hadoop 3.3.1 has dependency
on hadoop-thirdparty 1.1.0? (as we would like to upgrade guava and other
shaded thirdparty dependencies for 3.3.1 as well)
On Mon, Apr 19, 2021 at 11:04 AM Wei-Chiu Chuang
<we...@cloudera.com.invalid> wrote:
> Hello, reviving this thread.
>
> I created a dashboard for Hadoop 3.3.1 release.
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
> Also a jira to track the release work: HADOOP-17647
> <https://issues.apache.org/jira/browse/HADOOP-17647>
>
> We are currently at 5 release blockers and 3 critical issues for Hadoop
> 3.3.1. I'll go through each of them and push out the ones that aren't
> really blocking us.
>
> If you believe there are more features/bug fixes we should include in 3.3.1
> (I spent the past few weeks backporting jiras but I'm sure I missed some)
> please shout out.
>
> Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too. There
> are a number of tasks to be done there too. Let's start another thread for
> hadoop-thirdparty 1.1.0 release.
>
> On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <hemanthboyina224@gmail.com
> >
> wrote:
>
> > Hi Steve and Wei-Chiu
> >
> > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
> trunk
> > and tried to work out with IPV6, we have faced some issues and have made
> > the required changes for ipv6 to work.After the changes were made we have
> > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> > rigorously.Its been quite a some time these changes were deployed in
> > production cluster and have been in use for extensive purpose.
> >
> > I think it's good time to add this feature.
> >
> > Thanks
> > Hemanth Boyina
> >
> >
> >
> > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
> wrote:
> >
> > > Hi David,
> > >
> > > >> Still hoping for help here:
> > >
> > > >> https://issues.apache.org/jira/browse/HDFS-15790
> > >
> > > I will raise a PR for the said solution soon (in a day or two).
> > >
> > > -Vinay
> > >
> > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > Still hoping for help here:
> > > >
> > > > https://issues.apache.org/jira/browse/HDFS-15790
> > > >
> > > > Looks like it has been worked on, not sure how to best move it
> forward.
> > > >
> > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > > <stevel@cloudera.com.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > I'm going to argue its too late to do IPv6 support close to a
> > release,
> > > as
> > > > > it's best if its on developer machines for some time to let all
> > quirks
> > > > > surface. It's not so much IPv6 itself, but do we cause any
> > regressions
> > > on
> > > > > IPv4?
> > > > >
> > > > > But: it can/should go into trunk and stabilize there
> > > > >
> > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > >
> > > > > > Hi Brahma,
> > > > > >
> > > > > > I have missed out mentioning about the IPV6 feature in the last
> > mail,
> > > > > > Support for IPV6 has been in development since 2015 and We have
> > done
> > > a
> > > > > good
> > > > > > amount of testing at our organisation, the feature is stable and
> > used
> > > > by
> > > > > > our customers extensively in the last one year. I think it is a
> > good
> > > > time
> > > > > > to add the IPV6 support to 3.3.1.
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > > >
> > > > > > Thanks
> > > > > > D M Murali Krishna Reddy
> > > > > >
> > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > >
> > > > > > > Hi Brahma,
> > > > > > >
> > > > > > > Can we have this new feature "YARN Registry based AM discovery
> > with
> > > > > retry
> > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > > release. I
> > > > > > > have also attached a test-report in the below jira.
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > D M Murali Krishna Reddy
> > > > > > >
> > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Bilwa,
> > > > > > >>
> > > > > > >> I have commented on the jira's you mentioned. Based on the
> > > stability
> > > > > we
> > > > > > >> can
> > > > > > >> plan this.But needs to be merged ASAP.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> > > wrote:
> > > > > > >>
> > > > > > >> > Hi Brahma,
> > > > > > >> >
> > > > > > >> > Can we have below features in 3.3.1 release? We have been
> > using
> > > > > these
> > > > > > >> > features for a long time. They are stable and tested in
> bigger
> > > > > > clusters.
> > > > > > >> >
> > > > > > >> > 1. Container reuse -
> > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > > >> >
> > > > > > >> > Thanks,
> > > > > > >> > Bilwa
> > > > > > >> >
> > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> >> Sorry for the late reply..
> > > > > > >> >>
> > > > > > >> >> I will come up with a plan.. Please let me know if anybody
> > has
> > > > some
> > > > > > >> >> features/improvements/bugs that need to be included.
> > > > > > >> >>
> > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > > sunilg@apache.org>
> > > > > > >> wrote:
> > > > > > >> >>
> > > > > > >> >> > Hi Wei-Chiu,
> > > > > > >> >> >
> > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > > >> >> >
> > > > > > >> >> > Thanks
> > > > > > >> >> > Sunil
> > > > > > >> >> >
> > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > > > wrote:
> > > > > > >> >> >
> > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > > >> >> > > >
> > > > > > >> >> > > wrote:
> > > > > > >> >> > >
> > > > > > >> >> > > >
> > > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > > hackathon
> > > > > > >> where we
> > > > > > >> >> > try
> > > > > > >> >> > > > and get things in. This means a promise of review
> time
> > > from
> > > > > the
> > > > > > >> >> people
> > > > > > >> >> > > with
> > > > > > >> >> > > > commit rights and other people who understand the
> code
> > > > > (Stack?)
> > > > > > >> >> > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > > > testing).
> > > > > > >> >> > > Thanks,
> > > > > > >> >> > > S
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > > > -steve
> > > > > > >> >> > > >
> > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > > ayushtkn@gmail.com
> > > > > > >
> > > > > > >> >> wrote:
> > > > > > >> >> > > >
> > > > > > >> >> > > > > +1
> > > > > > >> >> > > > > Just to mention we would need to release
> > > > hadoop-thirdparty
> > > > > > too
> > > > > > >> >> > before.
> > > > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > -Ayush
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > > >> weichiu@apache.org
> > > > > > >> >> >
> > > > > > >> >> > > > wrote:
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hi all,
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as
> > of
> > > > now
> > > > > > >> we've
> > > > > > >> >> > > > > accumulated
> > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number
> > of
> > > > > > >> >> downstreamers
> > > > > > >> >> > > are
> > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
> guava
> > > > > version
> > > > > > >> >> > conflict
> > > > > > >> >> > > > > issue.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > We should start the release work for 3.3.1 before
> > the
> > > > > diff
> > > > > > >> >> becomes
> > > > > > >> >> > > even
> > > > > > >> >> > > > > > larger.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > I believe there are currently only two real
> > blockers
> > > > > for a
> > > > > > >> >> 3.3.1
> > > > > > >> >> > > > (using
> > > > > > >> >> > > > > > this filter
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > > >> >> > > > > > )
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
> 3.3.1
> > > RM?
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the
> > ARM
> > > > > build
> > > > > > >> >> > process.
> > > > > > >> >> > > > > That's
> > > > > > >> >> > > > > > going to be important for future releases.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >>
> > > > >
> ---------------------------------------------------------------------
> > > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > > >> >> > > > > For additional commands, e-mail:
> > > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --Brahma Reddy Battula
> > > > > > >> >>
> > > > > > >> >
> > > > > > >>
> > > > > > >> --
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> --Brahma Reddy Battula
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > --
> > > -Vinay
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Billie Rinaldi <bi...@apache.org>.
I was thinking of backporting HADOOP-16948 for 3.3.1.
Billie
On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang
<we...@cloudera.com.invalid> wrote:
> Hello, reviving this thread.
>
> I created a dashboard for Hadoop 3.3.1 release.
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
> Also a jira to track the release work: HADOOP-17647
> <https://issues.apache.org/jira/browse/HADOOP-17647>
>
> We are currently at 5 release blockers and 3 critical issues for Hadoop
> 3.3.1. I'll go through each of them and push out the ones that aren't
> really blocking us.
>
> If you believe there are more features/bug fixes we should include in 3.3.1
> (I spent the past few weeks backporting jiras but I'm sure I missed some)
> please shout out.
>
> Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too. There
> are a number of tasks to be done there too. Let's start another thread for
> hadoop-thirdparty 1.1.0 release.
>
> On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <hemanthboyina224@gmail.com
> >
> wrote:
>
> > Hi Steve and Wei-Chiu
> >
> > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
> trunk
> > and tried to work out with IPV6, we have faced some issues and have made
> > the required changes for ipv6 to work.After the changes were made we have
> > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> > rigorously.Its been quite a some time these changes were deployed in
> > production cluster and have been in use for extensive purpose.
> >
> > I think it's good time to add this feature.
> >
> > Thanks
> > Hemanth Boyina
> >
> >
> >
> > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
> wrote:
> >
> > > Hi David,
> > >
> > > >> Still hoping for help here:
> > >
> > > >> https://issues.apache.org/jira/browse/HDFS-15790
> > >
> > > I will raise a PR for the said solution soon (in a day or two).
> > >
> > > -Vinay
> > >
> > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > Still hoping for help here:
> > > >
> > > > https://issues.apache.org/jira/browse/HDFS-15790
> > > >
> > > > Looks like it has been worked on, not sure how to best move it
> forward.
> > > >
> > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > > <stevel@cloudera.com.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > I'm going to argue its too late to do IPv6 support close to a
> > release,
> > > as
> > > > > it's best if its on developer machines for some time to let all
> > quirks
> > > > > surface. It's not so much IPv6 itself, but do we cause any
> > regressions
> > > on
> > > > > IPv4?
> > > > >
> > > > > But: it can/should go into trunk and stabilize there
> > > > >
> > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > >
> > > > > > Hi Brahma,
> > > > > >
> > > > > > I have missed out mentioning about the IPV6 feature in the last
> > mail,
> > > > > > Support for IPV6 has been in development since 2015 and We have
> > done
> > > a
> > > > > good
> > > > > > amount of testing at our organisation, the feature is stable and
> > used
> > > > by
> > > > > > our customers extensively in the last one year. I think it is a
> > good
> > > > time
> > > > > > to add the IPV6 support to 3.3.1.
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > > >
> > > > > > Thanks
> > > > > > D M Murali Krishna Reddy
> > > > > >
> > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > >
> > > > > > > Hi Brahma,
> > > > > > >
> > > > > > > Can we have this new feature "YARN Registry based AM discovery
> > with
> > > > > retry
> > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > > release. I
> > > > > > > have also attached a test-report in the below jira.
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > D M Murali Krishna Reddy
> > > > > > >
> > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Bilwa,
> > > > > > >>
> > > > > > >> I have commented on the jira's you mentioned. Based on the
> > > stability
> > > > > we
> > > > > > >> can
> > > > > > >> plan this.But needs to be merged ASAP.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> > > wrote:
> > > > > > >>
> > > > > > >> > Hi Brahma,
> > > > > > >> >
> > > > > > >> > Can we have below features in 3.3.1 release? We have been
> > using
> > > > > these
> > > > > > >> > features for a long time. They are stable and tested in
> bigger
> > > > > > clusters.
> > > > > > >> >
> > > > > > >> > 1. Container reuse -
> > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > > >> >
> > > > > > >> > Thanks,
> > > > > > >> > Bilwa
> > > > > > >> >
> > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> >> Sorry for the late reply..
> > > > > > >> >>
> > > > > > >> >> I will come up with a plan.. Please let me know if anybody
> > has
> > > > some
> > > > > > >> >> features/improvements/bugs that need to be included.
> > > > > > >> >>
> > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > > sunilg@apache.org>
> > > > > > >> wrote:
> > > > > > >> >>
> > > > > > >> >> > Hi Wei-Chiu,
> > > > > > >> >> >
> > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > > >> >> >
> > > > > > >> >> > Thanks
> > > > > > >> >> > Sunil
> > > > > > >> >> >
> > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > > > wrote:
> > > > > > >> >> >
> > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > > >> >> > > >
> > > > > > >> >> > > wrote:
> > > > > > >> >> > >
> > > > > > >> >> > > >
> > > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > > hackathon
> > > > > > >> where we
> > > > > > >> >> > try
> > > > > > >> >> > > > and get things in. This means a promise of review
> time
> > > from
> > > > > the
> > > > > > >> >> people
> > > > > > >> >> > > with
> > > > > > >> >> > > > commit rights and other people who understand the
> code
> > > > > (Stack?)
> > > > > > >> >> > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > > > testing).
> > > > > > >> >> > > Thanks,
> > > > > > >> >> > > S
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > > > -steve
> > > > > > >> >> > > >
> > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > > ayushtkn@gmail.com
> > > > > > >
> > > > > > >> >> wrote:
> > > > > > >> >> > > >
> > > > > > >> >> > > > > +1
> > > > > > >> >> > > > > Just to mention we would need to release
> > > > hadoop-thirdparty
> > > > > > too
> > > > > > >> >> > before.
> > > > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > -Ayush
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > > >> weichiu@apache.org
> > > > > > >> >> >
> > > > > > >> >> > > > wrote:
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hi all,
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as
> > of
> > > > now
> > > > > > >> we've
> > > > > > >> >> > > > > accumulated
> > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number
> > of
> > > > > > >> >> downstreamers
> > > > > > >> >> > > are
> > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
> guava
> > > > > version
> > > > > > >> >> > conflict
> > > > > > >> >> > > > > issue.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > We should start the release work for 3.3.1 before
> > the
> > > > > diff
> > > > > > >> >> becomes
> > > > > > >> >> > > even
> > > > > > >> >> > > > > > larger.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > I believe there are currently only two real
> > blockers
> > > > > for a
> > > > > > >> >> 3.3.1
> > > > > > >> >> > > > (using
> > > > > > >> >> > > > > > this filter
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > > >> >> > > > > > )
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
> 3.3.1
> > > RM?
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the
> > ARM
> > > > > build
> > > > > > >> >> > process.
> > > > > > >> >> > > > > That's
> > > > > > >> >> > > > > > going to be important for future releases.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >>
> > > > >
> ---------------------------------------------------------------------
> > > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > > >> >> > > > > For additional commands, e-mail:
> > > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --Brahma Reddy Battula
> > > > > > >> >>
> > > > > > >> >
> > > > > > >>
> > > > > > >> --
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> --Brahma Reddy Battula
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > --
> > > -Vinay
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Viraj Jasani <vj...@apache.org>.
Thanks Wei-Chiu.
> If you believe there are more features/bug fixes we should include in
3.3.1
(I spent the past few weeks backporting jiras but I'm sure I missed some)
please shout out.
Although not marked critical, I believe HDFS-15982 could get in 3.3.1 if
you are fine. Thanks to Bhavik for filing this.
> Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too.
I agree we should work on this to upgrade some dependencies from known CVEs
(e.g guava). However, does this mean Hadoop 3.3.1 has dependency
on hadoop-thirdparty 1.1.0? (as we would like to upgrade guava and other
shaded thirdparty dependencies for 3.3.1 as well)
On Mon, Apr 19, 2021 at 11:04 AM Wei-Chiu Chuang
<we...@cloudera.com.invalid> wrote:
> Hello, reviving this thread.
>
> I created a dashboard for Hadoop 3.3.1 release.
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
> Also a jira to track the release work: HADOOP-17647
> <https://issues.apache.org/jira/browse/HADOOP-17647>
>
> We are currently at 5 release blockers and 3 critical issues for Hadoop
> 3.3.1. I'll go through each of them and push out the ones that aren't
> really blocking us.
>
> If you believe there are more features/bug fixes we should include in 3.3.1
> (I spent the past few weeks backporting jiras but I'm sure I missed some)
> please shout out.
>
> Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too. There
> are a number of tasks to be done there too. Let's start another thread for
> hadoop-thirdparty 1.1.0 release.
>
> On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <hemanthboyina224@gmail.com
> >
> wrote:
>
> > Hi Steve and Wei-Chiu
> >
> > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to
> trunk
> > and tried to work out with IPV6, we have faced some issues and have made
> > the required changes for ipv6 to work.After the changes were made we have
> > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> > rigorously.Its been quite a some time these changes were deployed in
> > production cluster and have been in use for extensive purpose.
> >
> > I think it's good time to add this feature.
> >
> > Thanks
> > Hemanth Boyina
> >
> >
> >
> > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org>
> wrote:
> >
> > > Hi David,
> > >
> > > >> Still hoping for help here:
> > >
> > > >> https://issues.apache.org/jira/browse/HDFS-15790
> > >
> > > I will raise a PR for the said solution soon (in a day or two).
> > >
> > > -Vinay
> > >
> > > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > Still hoping for help here:
> > > >
> > > > https://issues.apache.org/jira/browse/HDFS-15790
> > > >
> > > > Looks like it has been worked on, not sure how to best move it
> forward.
> > > >
> > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > > <stevel@cloudera.com.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > I'm going to argue its too late to do IPv6 support close to a
> > release,
> > > as
> > > > > it's best if its on developer machines for some time to let all
> > quirks
> > > > > surface. It's not so much IPv6 itself, but do we cause any
> > regressions
> > > on
> > > > > IPv4?
> > > > >
> > > > > But: it can/should go into trunk and stabilize there
> > > > >
> > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > >
> > > > > > Hi Brahma,
> > > > > >
> > > > > > I have missed out mentioning about the IPV6 feature in the last
> > mail,
> > > > > > Support for IPV6 has been in development since 2015 and We have
> > done
> > > a
> > > > > good
> > > > > > amount of testing at our organisation, the feature is stable and
> > used
> > > > by
> > > > > > our customers extensively in the last one year. I think it is a
> > good
> > > > time
> > > > > > to add the IPV6 support to 3.3.1.
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > > >
> > > > > > Thanks
> > > > > > D M Murali Krishna Reddy
> > > > > >
> > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > > >
> > > > > > > Hi Brahma,
> > > > > > >
> > > > > > > Can we have this new feature "YARN Registry based AM discovery
> > with
> > > > > retry
> > > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > > release. I
> > > > > > > have also attached a test-report in the below jira.
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > D M Murali Krishna Reddy
> > > > > > >
> > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Bilwa,
> > > > > > >>
> > > > > > >> I have commented on the jira's you mentioned. Based on the
> > > stability
> > > > > we
> > > > > > >> can
> > > > > > >> plan this.But needs to be merged ASAP.
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> > > wrote:
> > > > > > >>
> > > > > > >> > Hi Brahma,
> > > > > > >> >
> > > > > > >> > Can we have below features in 3.3.1 release? We have been
> > using
> > > > > these
> > > > > > >> > features for a long time. They are stable and tested in
> bigger
> > > > > > clusters.
> > > > > > >> >
> > > > > > >> > 1. Container reuse -
> > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > > >> >
> > > > > > >> > Thanks,
> > > > > > >> > Bilwa
> > > > > > >> >
> > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > > brahma@apache.org
> > > > > > >
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> >> Sorry for the late reply..
> > > > > > >> >>
> > > > > > >> >> I will come up with a plan.. Please let me know if anybody
> > has
> > > > some
> > > > > > >> >> features/improvements/bugs that need to be included.
> > > > > > >> >>
> > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > > sunilg@apache.org>
> > > > > > >> wrote:
> > > > > > >> >>
> > > > > > >> >> > Hi Wei-Chiu,
> > > > > > >> >> >
> > > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > > >> >> >
> > > > > > >> >> > Thanks
> > > > > > >> >> > Sunil
> > > > > > >> >> >
> > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > > > wrote:
> > > > > > >> >> >
> > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > > >> >> > > >
> > > > > > >> >> > > wrote:
> > > > > > >> >> > >
> > > > > > >> >> > > >
> > > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > > hackathon
> > > > > > >> where we
> > > > > > >> >> > try
> > > > > > >> >> > > > and get things in. This means a promise of review
> time
> > > from
> > > > > the
> > > > > > >> >> people
> > > > > > >> >> > > with
> > > > > > >> >> > > > commit rights and other people who understand the
> code
> > > > > (Stack?)
> > > > > > >> >> > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > > > testing).
> > > > > > >> >> > > Thanks,
> > > > > > >> >> > > S
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > >
> > > > > > >> >> > > > -steve
> > > > > > >> >> > > >
> > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > > ayushtkn@gmail.com
> > > > > > >
> > > > > > >> >> wrote:
> > > > > > >> >> > > >
> > > > > > >> >> > > > > +1
> > > > > > >> >> > > > > Just to mention we would need to release
> > > > hadoop-thirdparty
> > > > > > too
> > > > > > >> >> > before.
> > > > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > -Ayush
> > > > > > >> >> > > > >
> > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > > >> weichiu@apache.org
> > > > > > >> >> >
> > > > > > >> >> > > > wrote:
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hi all,
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as
> > of
> > > > now
> > > > > > >> we've
> > > > > > >> >> > > > > accumulated
> > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number
> > of
> > > > > > >> >> downstreamers
> > > > > > >> >> > > are
> > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the
> guava
> > > > > version
> > > > > > >> >> > conflict
> > > > > > >> >> > > > > issue.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > We should start the release work for 3.3.1 before
> > the
> > > > > diff
> > > > > > >> >> becomes
> > > > > > >> >> > > even
> > > > > > >> >> > > > > > larger.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > I believe there are currently only two real
> > blockers
> > > > > for a
> > > > > > >> >> 3.3.1
> > > > > > >> >> > > > (using
> > > > > > >> >> > > > > > this filter
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > > >> >> > > > > > )
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > 2.
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Is there anyone who would volunteer to be the
> 3.3.1
> > > RM?
> > > > > > >> >> > > > > >
> > > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the
> > ARM
> > > > > build
> > > > > > >> >> > process.
> > > > > > >> >> > > > > That's
> > > > > > >> >> > > > > > going to be important for future releases.
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >>
> > > > >
> ---------------------------------------------------------------------
> > > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > > >> >> > > > > For additional commands, e-mail:
> > > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > > >> >> > > > >
> > > > > > >> >> > > > >
> > > > > > >> >> > > >
> > > > > > >> >> > >
> > > > > > >> >> >
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >>
> > > > > > >> >> --Brahma Reddy Battula
> > > > > > >> >>
> > > > > > >> >
> > > > > > >>
> > > > > > >> --
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> --Brahma Reddy Battula
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > --
> > > -Vinay
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Hello, reviving this thread.
I created a dashboard for Hadoop 3.3.1 release.
https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
Also a jira to track the release work: HADOOP-17647
<https://issues.apache.org/jira/browse/HADOOP-17647>
We are currently at 5 release blockers and 3 critical issues for Hadoop
3.3.1. I'll go through each of them and push out the ones that aren't
really blocking us.
If you believe there are more features/bug fixes we should include in 3.3.1
(I spent the past few weeks backporting jiras but I'm sure I missed some)
please shout out.
Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too. There
are a number of tasks to be done there too. Let's start another thread for
hadoop-thirdparty 1.1.0 release.
On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <he...@gmail.com>
wrote:
> Hi Steve and Wei-Chiu
>
> Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to trunk
> and tried to work out with IPV6, we have faced some issues and have made
> the required changes for ipv6 to work.After the changes were made we have
> tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> rigorously.Its been quite a some time these changes were deployed in
> production cluster and have been in use for extensive purpose.
>
> I think it's good time to add this feature.
>
> Thanks
> Hemanth Boyina
>
>
>
> On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org> wrote:
>
> > Hi David,
> >
> > >> Still hoping for help here:
> >
> > >> https://issues.apache.org/jira/browse/HDFS-15790
> >
> > I will raise a PR for the said solution soon (in a day or two).
> >
> > -Vinay
> >
> > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > Still hoping for help here:
> > >
> > > https://issues.apache.org/jira/browse/HDFS-15790
> > >
> > > Looks like it has been worked on, not sure how to best move it forward.
> > >
> > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > <stevel@cloudera.com.invalid
> > > >
> > > wrote:
> > >
> > > > I'm going to argue its too late to do IPv6 support close to a
> release,
> > as
> > > > it's best if its on developer machines for some time to let all
> quirks
> > > > surface. It's not so much IPv6 itself, but do we cause any
> regressions
> > on
> > > > IPv4?
> > > >
> > > > But: it can/should go into trunk and stabilize there
> > > >
> > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > >
> > > > > Hi Brahma,
> > > > >
> > > > > I have missed out mentioning about the IPV6 feature in the last
> mail,
> > > > > Support for IPV6 has been in development since 2015 and We have
> done
> > a
> > > > good
> > > > > amount of testing at our organisation, the feature is stable and
> used
> > > by
> > > > > our customers extensively in the last one year. I think it is a
> good
> > > time
> > > > > to add the IPV6 support to 3.3.1.
> > > > >
> > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > >
> > > > > Thanks
> > > > > D M Murali Krishna Reddy
> > > > >
> > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > >
> > > > > > Hi Brahma,
> > > > > >
> > > > > > Can we have this new feature "YARN Registry based AM discovery
> with
> > > > retry
> > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > release. I
> > > > > > have also attached a test-report in the below jira.
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > D M Murali Krishna Reddy
> > > > > >
> > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > brahma@apache.org
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Bilwa,
> > > > > >>
> > > > > >> I have commented on the jira's you mentioned. Based on the
> > stability
> > > > we
> > > > > >> can
> > > > > >> plan this.But needs to be merged ASAP.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> > wrote:
> > > > > >>
> > > > > >> > Hi Brahma,
> > > > > >> >
> > > > > >> > Can we have below features in 3.3.1 release? We have been
> using
> > > > these
> > > > > >> > features for a long time. They are stable and tested in bigger
> > > > > clusters.
> > > > > >> >
> > > > > >> > 1. Container reuse -
> > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > >> >
> > > > > >> > Thanks,
> > > > > >> > Bilwa
> > > > > >> >
> > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > brahma@apache.org
> > > > > >
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> >> Sorry for the late reply..
> > > > > >> >>
> > > > > >> >> I will come up with a plan.. Please let me know if anybody
> has
> > > some
> > > > > >> >> features/improvements/bugs that need to be included.
> > > > > >> >>
> > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > sunilg@apache.org>
> > > > > >> wrote:
> > > > > >> >>
> > > > > >> >> > Hi Wei-Chiu,
> > > > > >> >> >
> > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > >> >> >
> > > > > >> >> > Thanks
> > > > > >> >> > Sunil
> > > > > >> >> >
> > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > > wrote:
> > > > > >> >> >
> > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > >> >> > > >
> > > > > >> >> > > wrote:
> > > > > >> >> > >
> > > > > >> >> > > >
> > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > hackathon
> > > > > >> where we
> > > > > >> >> > try
> > > > > >> >> > > > and get things in. This means a promise of review time
> > from
> > > > the
> > > > > >> >> people
> > > > > >> >> > > with
> > > > > >> >> > > > commit rights and other people who understand the code
> > > > (Stack?)
> > > > > >> >> > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > > testing).
> > > > > >> >> > > Thanks,
> > > > > >> >> > > S
> > > > > >> >> > >
> > > > > >> >> > >
> > > > > >> >> > >
> > > > > >> >> > >
> > > > > >> >> > > > -steve
> > > > > >> >> > > >
> > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > ayushtkn@gmail.com
> > > > > >
> > > > > >> >> wrote:
> > > > > >> >> > > >
> > > > > >> >> > > > > +1
> > > > > >> >> > > > > Just to mention we would need to release
> > > hadoop-thirdparty
> > > > > too
> > > > > >> >> > before.
> > > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > > >> >> > > > >
> > > > > >> >> > > > > -Ayush
> > > > > >> >> > > > >
> > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > >> weichiu@apache.org
> > > > > >> >> >
> > > > > >> >> > > > wrote:
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Hi all,
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as
> of
> > > now
> > > > > >> we've
> > > > > >> >> > > > > accumulated
> > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number
> of
> > > > > >> >> downstreamers
> > > > > >> >> > > are
> > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> > > > version
> > > > > >> >> > conflict
> > > > > >> >> > > > > issue.
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >> >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > We should start the release work for 3.3.1 before
> the
> > > > diff
> > > > > >> >> becomes
> > > > > >> >> > > even
> > > > > >> >> > > > > > larger.
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > I believe there are currently only two real
> blockers
> > > > for a
> > > > > >> >> 3.3.1
> > > > > >> >> > > > (using
> > > > > >> >> > > > > > this filter
> > > > > >> >> > > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >> >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > >> >> > > > > > )
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > >> >> > > > > > 2.
> > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > 2.
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1
> > RM?
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the
> ARM
> > > > build
> > > > > >> >> > process.
> > > > > >> >> > > > > That's
> > > > > >> >> > > > > > going to be important for future releases.
> > > > > >> >> > > > >
> > > > > >> >> > > > >
> > > > > >> >>
> > > > ---------------------------------------------------------------------
> > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > >> >> > > > > For additional commands, e-mail:
> > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > >> >> > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >> >>
> > > > > >> >>
> > > > > >> >> --
> > > > > >> >>
> > > > > >> >>
> > > > > >> >>
> > > > > >> >> --Brahma Reddy Battula
> > > > > >> >>
> > > > > >> >
> > > > > >>
> > > > > >> --
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> --Brahma Reddy Battula
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> > --
> > -Vinay
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Hello, reviving this thread.
I created a dashboard for Hadoop 3.3.1 release.
https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
Also a jira to track the release work: HADOOP-17647
<https://issues.apache.org/jira/browse/HADOOP-17647>
We are currently at 5 release blockers and 3 critical issues for Hadoop
3.3.1. I'll go through each of them and push out the ones that aren't
really blocking us.
If you believe there are more features/bug fixes we should include in 3.3.1
(I spent the past few weeks backporting jiras but I'm sure I missed some)
please shout out.
Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too. There
are a number of tasks to be done there too. Let's start another thread for
hadoop-thirdparty 1.1.0 release.
On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <he...@gmail.com>
wrote:
> Hi Steve and Wei-Chiu
>
> Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to trunk
> and tried to work out with IPV6, we have faced some issues and have made
> the required changes for ipv6 to work.After the changes were made we have
> tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> rigorously.Its been quite a some time these changes were deployed in
> production cluster and have been in use for extensive purpose.
>
> I think it's good time to add this feature.
>
> Thanks
> Hemanth Boyina
>
>
>
> On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org> wrote:
>
> > Hi David,
> >
> > >> Still hoping for help here:
> >
> > >> https://issues.apache.org/jira/browse/HDFS-15790
> >
> > I will raise a PR for the said solution soon (in a day or two).
> >
> > -Vinay
> >
> > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > Still hoping for help here:
> > >
> > > https://issues.apache.org/jira/browse/HDFS-15790
> > >
> > > Looks like it has been worked on, not sure how to best move it forward.
> > >
> > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > <stevel@cloudera.com.invalid
> > > >
> > > wrote:
> > >
> > > > I'm going to argue its too late to do IPv6 support close to a
> release,
> > as
> > > > it's best if its on developer machines for some time to let all
> quirks
> > > > surface. It's not so much IPv6 itself, but do we cause any
> regressions
> > on
> > > > IPv4?
> > > >
> > > > But: it can/should go into trunk and stabilize there
> > > >
> > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > >
> > > > > Hi Brahma,
> > > > >
> > > > > I have missed out mentioning about the IPV6 feature in the last
> mail,
> > > > > Support for IPV6 has been in development since 2015 and We have
> done
> > a
> > > > good
> > > > > amount of testing at our organisation, the feature is stable and
> used
> > > by
> > > > > our customers extensively in the last one year. I think it is a
> good
> > > time
> > > > > to add the IPV6 support to 3.3.1.
> > > > >
> > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > >
> > > > > Thanks
> > > > > D M Murali Krishna Reddy
> > > > >
> > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > >
> > > > > > Hi Brahma,
> > > > > >
> > > > > > Can we have this new feature "YARN Registry based AM discovery
> with
> > > > retry
> > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > release. I
> > > > > > have also attached a test-report in the below jira.
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > D M Murali Krishna Reddy
> > > > > >
> > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > brahma@apache.org
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Bilwa,
> > > > > >>
> > > > > >> I have commented on the jira's you mentioned. Based on the
> > stability
> > > > we
> > > > > >> can
> > > > > >> plan this.But needs to be merged ASAP.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> > wrote:
> > > > > >>
> > > > > >> > Hi Brahma,
> > > > > >> >
> > > > > >> > Can we have below features in 3.3.1 release? We have been
> using
> > > > these
> > > > > >> > features for a long time. They are stable and tested in bigger
> > > > > clusters.
> > > > > >> >
> > > > > >> > 1. Container reuse -
> > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > >> >
> > > > > >> > Thanks,
> > > > > >> > Bilwa
> > > > > >> >
> > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > brahma@apache.org
> > > > > >
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> >> Sorry for the late reply..
> > > > > >> >>
> > > > > >> >> I will come up with a plan.. Please let me know if anybody
> has
> > > some
> > > > > >> >> features/improvements/bugs that need to be included.
> > > > > >> >>
> > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > sunilg@apache.org>
> > > > > >> wrote:
> > > > > >> >>
> > > > > >> >> > Hi Wei-Chiu,
> > > > > >> >> >
> > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > >> >> >
> > > > > >> >> > Thanks
> > > > > >> >> > Sunil
> > > > > >> >> >
> > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > > wrote:
> > > > > >> >> >
> > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > >> >> > > >
> > > > > >> >> > > wrote:
> > > > > >> >> > >
> > > > > >> >> > > >
> > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > hackathon
> > > > > >> where we
> > > > > >> >> > try
> > > > > >> >> > > > and get things in. This means a promise of review time
> > from
> > > > the
> > > > > >> >> people
> > > > > >> >> > > with
> > > > > >> >> > > > commit rights and other people who understand the code
> > > > (Stack?)
> > > > > >> >> > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > > testing).
> > > > > >> >> > > Thanks,
> > > > > >> >> > > S
> > > > > >> >> > >
> > > > > >> >> > >
> > > > > >> >> > >
> > > > > >> >> > >
> > > > > >> >> > > > -steve
> > > > > >> >> > > >
> > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > ayushtkn@gmail.com
> > > > > >
> > > > > >> >> wrote:
> > > > > >> >> > > >
> > > > > >> >> > > > > +1
> > > > > >> >> > > > > Just to mention we would need to release
> > > hadoop-thirdparty
> > > > > too
> > > > > >> >> > before.
> > > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > > >> >> > > > >
> > > > > >> >> > > > > -Ayush
> > > > > >> >> > > > >
> > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > >> weichiu@apache.org
> > > > > >> >> >
> > > > > >> >> > > > wrote:
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Hi all,
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as
> of
> > > now
> > > > > >> we've
> > > > > >> >> > > > > accumulated
> > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number
> of
> > > > > >> >> downstreamers
> > > > > >> >> > > are
> > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> > > > version
> > > > > >> >> > conflict
> > > > > >> >> > > > > issue.
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >> >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > We should start the release work for 3.3.1 before
> the
> > > > diff
> > > > > >> >> becomes
> > > > > >> >> > > even
> > > > > >> >> > > > > > larger.
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > I believe there are currently only two real
> blockers
> > > > for a
> > > > > >> >> 3.3.1
> > > > > >> >> > > > (using
> > > > > >> >> > > > > > this filter
> > > > > >> >> > > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >> >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > >> >> > > > > > )
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > >> >> > > > > > 2.
> > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > 2.
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1
> > RM?
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the
> ARM
> > > > build
> > > > > >> >> > process.
> > > > > >> >> > > > > That's
> > > > > >> >> > > > > > going to be important for future releases.
> > > > > >> >> > > > >
> > > > > >> >> > > > >
> > > > > >> >>
> > > > ---------------------------------------------------------------------
> > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > >> >> > > > > For additional commands, e-mail:
> > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > >> >> > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >> >>
> > > > > >> >>
> > > > > >> >> --
> > > > > >> >>
> > > > > >> >>
> > > > > >> >>
> > > > > >> >> --Brahma Reddy Battula
> > > > > >> >>
> > > > > >> >
> > > > > >>
> > > > > >> --
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> --Brahma Reddy Battula
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> > --
> > -Vinay
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Hello, reviving this thread.
I created a dashboard for Hadoop 3.3.1 release.
https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
Also a jira to track the release work: HADOOP-17647
<https://issues.apache.org/jira/browse/HADOOP-17647>
We are currently at 5 release blockers and 3 critical issues for Hadoop
3.3.1. I'll go through each of them and push out the ones that aren't
really blocking us.
If you believe there are more features/bug fixes we should include in 3.3.1
(I spent the past few weeks backporting jiras but I'm sure I missed some)
please shout out.
Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too. There
are a number of tasks to be done there too. Let's start another thread for
hadoop-thirdparty 1.1.0 release.
On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <he...@gmail.com>
wrote:
> Hi Steve and Wei-Chiu
>
> Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to trunk
> and tried to work out with IPV6, we have faced some issues and have made
> the required changes for ipv6 to work.After the changes were made we have
> tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> rigorously.Its been quite a some time these changes were deployed in
> production cluster and have been in use for extensive purpose.
>
> I think it's good time to add this feature.
>
> Thanks
> Hemanth Boyina
>
>
>
> On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org> wrote:
>
> > Hi David,
> >
> > >> Still hoping for help here:
> >
> > >> https://issues.apache.org/jira/browse/HDFS-15790
> >
> > I will raise a PR for the said solution soon (in a day or two).
> >
> > -Vinay
> >
> > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > Still hoping for help here:
> > >
> > > https://issues.apache.org/jira/browse/HDFS-15790
> > >
> > > Looks like it has been worked on, not sure how to best move it forward.
> > >
> > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > <stevel@cloudera.com.invalid
> > > >
> > > wrote:
> > >
> > > > I'm going to argue its too late to do IPv6 support close to a
> release,
> > as
> > > > it's best if its on developer machines for some time to let all
> quirks
> > > > surface. It's not so much IPv6 itself, but do we cause any
> regressions
> > on
> > > > IPv4?
> > > >
> > > > But: it can/should go into trunk and stabilize there
> > > >
> > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > >
> > > > > Hi Brahma,
> > > > >
> > > > > I have missed out mentioning about the IPV6 feature in the last
> mail,
> > > > > Support for IPV6 has been in development since 2015 and We have
> done
> > a
> > > > good
> > > > > amount of testing at our organisation, the feature is stable and
> used
> > > by
> > > > > our customers extensively in the last one year. I think it is a
> good
> > > time
> > > > > to add the IPV6 support to 3.3.1.
> > > > >
> > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > >
> > > > > Thanks
> > > > > D M Murali Krishna Reddy
> > > > >
> > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > >
> > > > > > Hi Brahma,
> > > > > >
> > > > > > Can we have this new feature "YARN Registry based AM discovery
> with
> > > > retry
> > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > release. I
> > > > > > have also attached a test-report in the below jira.
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > D M Murali Krishna Reddy
> > > > > >
> > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > brahma@apache.org
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Bilwa,
> > > > > >>
> > > > > >> I have commented on the jira's you mentioned. Based on the
> > stability
> > > > we
> > > > > >> can
> > > > > >> plan this.But needs to be merged ASAP.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> > wrote:
> > > > > >>
> > > > > >> > Hi Brahma,
> > > > > >> >
> > > > > >> > Can we have below features in 3.3.1 release? We have been
> using
> > > > these
> > > > > >> > features for a long time. They are stable and tested in bigger
> > > > > clusters.
> > > > > >> >
> > > > > >> > 1. Container reuse -
> > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > >> >
> > > > > >> > Thanks,
> > > > > >> > Bilwa
> > > > > >> >
> > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > brahma@apache.org
> > > > > >
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> >> Sorry for the late reply..
> > > > > >> >>
> > > > > >> >> I will come up with a plan.. Please let me know if anybody
> has
> > > some
> > > > > >> >> features/improvements/bugs that need to be included.
> > > > > >> >>
> > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > sunilg@apache.org>
> > > > > >> wrote:
> > > > > >> >>
> > > > > >> >> > Hi Wei-Chiu,
> > > > > >> >> >
> > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > >> >> >
> > > > > >> >> > Thanks
> > > > > >> >> > Sunil
> > > > > >> >> >
> > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > > wrote:
> > > > > >> >> >
> > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > >> >> > > >
> > > > > >> >> > > wrote:
> > > > > >> >> > >
> > > > > >> >> > > >
> > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > hackathon
> > > > > >> where we
> > > > > >> >> > try
> > > > > >> >> > > > and get things in. This means a promise of review time
> > from
> > > > the
> > > > > >> >> people
> > > > > >> >> > > with
> > > > > >> >> > > > commit rights and other people who understand the code
> > > > (Stack?)
> > > > > >> >> > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > > testing).
> > > > > >> >> > > Thanks,
> > > > > >> >> > > S
> > > > > >> >> > >
> > > > > >> >> > >
> > > > > >> >> > >
> > > > > >> >> > >
> > > > > >> >> > > > -steve
> > > > > >> >> > > >
> > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > ayushtkn@gmail.com
> > > > > >
> > > > > >> >> wrote:
> > > > > >> >> > > >
> > > > > >> >> > > > > +1
> > > > > >> >> > > > > Just to mention we would need to release
> > > hadoop-thirdparty
> > > > > too
> > > > > >> >> > before.
> > > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > > >> >> > > > >
> > > > > >> >> > > > > -Ayush
> > > > > >> >> > > > >
> > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > >> weichiu@apache.org
> > > > > >> >> >
> > > > > >> >> > > > wrote:
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Hi all,
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as
> of
> > > now
> > > > > >> we've
> > > > > >> >> > > > > accumulated
> > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number
> of
> > > > > >> >> downstreamers
> > > > > >> >> > > are
> > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> > > > version
> > > > > >> >> > conflict
> > > > > >> >> > > > > issue.
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >> >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > We should start the release work for 3.3.1 before
> the
> > > > diff
> > > > > >> >> becomes
> > > > > >> >> > > even
> > > > > >> >> > > > > > larger.
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > I believe there are currently only two real
> blockers
> > > > for a
> > > > > >> >> 3.3.1
> > > > > >> >> > > > (using
> > > > > >> >> > > > > > this filter
> > > > > >> >> > > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >> >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > >> >> > > > > > )
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > >> >> > > > > > 2.
> > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > 2.
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1
> > RM?
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the
> ARM
> > > > build
> > > > > >> >> > process.
> > > > > >> >> > > > > That's
> > > > > >> >> > > > > > going to be important for future releases.
> > > > > >> >> > > > >
> > > > > >> >> > > > >
> > > > > >> >>
> > > > ---------------------------------------------------------------------
> > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > >> >> > > > > For additional commands, e-mail:
> > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > >> >> > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >> >>
> > > > > >> >>
> > > > > >> >> --
> > > > > >> >>
> > > > > >> >>
> > > > > >> >>
> > > > > >> >> --Brahma Reddy Battula
> > > > > >> >>
> > > > > >> >
> > > > > >>
> > > > > >> --
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> --Brahma Reddy Battula
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> > --
> > -Vinay
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Hello, reviving this thread.
I created a dashboard for Hadoop 3.3.1 release.
https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122
Also a jira to track the release work: HADOOP-17647
<https://issues.apache.org/jira/browse/HADOOP-17647>
We are currently at 5 release blockers and 3 critical issues for Hadoop
3.3.1. I'll go through each of them and push out the ones that aren't
really blocking us.
If you believe there are more features/bug fixes we should include in 3.3.1
(I spent the past few weeks backporting jiras but I'm sure I missed some)
please shout out.
Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too. There
are a number of tasks to be done there too. Let's start another thread for
hadoop-thirdparty 1.1.0 release.
On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina <he...@gmail.com>
wrote:
> Hi Steve and Wei-Chiu
>
> Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to trunk
> and tried to work out with IPV6, we have faced some issues and have made
> the required changes for ipv6 to work.After the changes were made we have
> tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
> rigorously.Its been quite a some time these changes were deployed in
> production cluster and have been in use for extensive purpose.
>
> I think it's good time to add this feature.
>
> Thanks
> Hemanth Boyina
>
>
>
> On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org> wrote:
>
> > Hi David,
> >
> > >> Still hoping for help here:
> >
> > >> https://issues.apache.org/jira/browse/HDFS-15790
> >
> > I will raise a PR for the said solution soon (in a day or two).
> >
> > -Vinay
> >
> > On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > Still hoping for help here:
> > >
> > > https://issues.apache.org/jira/browse/HDFS-15790
> > >
> > > Looks like it has been worked on, not sure how to best move it forward.
> > >
> > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> > <stevel@cloudera.com.invalid
> > > >
> > > wrote:
> > >
> > > > I'm going to argue its too late to do IPv6 support close to a
> release,
> > as
> > > > it's best if its on developer machines for some time to let all
> quirks
> > > > surface. It's not so much IPv6 itself, but do we cause any
> regressions
> > on
> > > > IPv4?
> > > >
> > > > But: it can/should go into trunk and stabilize there
> > > >
> > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > >
> > > > > Hi Brahma,
> > > > >
> > > > > I have missed out mentioning about the IPV6 feature in the last
> mail,
> > > > > Support for IPV6 has been in development since 2015 and We have
> done
> > a
> > > > good
> > > > > amount of testing at our organisation, the feature is stable and
> used
> > > by
> > > > > our customers extensively in the last one year. I think it is a
> good
> > > time
> > > > > to add the IPV6 support to 3.3.1.
> > > > >
> > > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > > >
> > > > > Thanks
> > > > > D M Murali Krishna Reddy
> > > > >
> > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > > >
> > > > > > Hi Brahma,
> > > > > >
> > > > > > Can we have this new feature "YARN Registry based AM discovery
> with
> > > > retry
> > > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > > release. I
> > > > > > have also attached a test-report in the below jira.
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > D M Murali Krishna Reddy
> > > > > >
> > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > > brahma@apache.org
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Bilwa,
> > > > > >>
> > > > > >> I have commented on the jira's you mentioned. Based on the
> > stability
> > > > we
> > > > > >> can
> > > > > >> plan this.But needs to be merged ASAP.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> > wrote:
> > > > > >>
> > > > > >> > Hi Brahma,
> > > > > >> >
> > > > > >> > Can we have below features in 3.3.1 release? We have been
> using
> > > > these
> > > > > >> > features for a long time. They are stable and tested in bigger
> > > > > clusters.
> > > > > >> >
> > > > > >> > 1. Container reuse -
> > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > > >> > 2. Speculative attempts should not run on the same node -
> > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > > >> >
> > > > > >> > Thanks,
> > > > > >> > Bilwa
> > > > > >> >
> > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > > brahma@apache.org
> > > > > >
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> >> Sorry for the late reply..
> > > > > >> >>
> > > > > >> >> I will come up with a plan.. Please let me know if anybody
> has
> > > some
> > > > > >> >> features/improvements/bugs that need to be included.
> > > > > >> >>
> > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > > sunilg@apache.org>
> > > > > >> wrote:
> > > > > >> >>
> > > > > >> >> > Hi Wei-Chiu,
> > > > > >> >> >
> > > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > > >> >> >
> > > > > >> >> > Thanks
> > > > > >> >> > Sunil
> > > > > >> >> >
> > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > > wrote:
> > > > > >> >> >
> > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > > >> >> > <stevel@cloudera.com.invalid
> > > > > >> >> > > >
> > > > > >> >> > > wrote:
> > > > > >> >> > >
> > > > > >> >> > > >
> > > > > >> >> > > > Regarding blockers &c: how about we have a little
> > hackathon
> > > > > >> where we
> > > > > >> >> > try
> > > > > >> >> > > > and get things in. This means a promise of review time
> > from
> > > > the
> > > > > >> >> people
> > > > > >> >> > > with
> > > > > >> >> > > > commit rights and other people who understand the code
> > > > (Stack?)
> > > > > >> >> > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > > testing).
> > > > > >> >> > > Thanks,
> > > > > >> >> > > S
> > > > > >> >> > >
> > > > > >> >> > >
> > > > > >> >> > >
> > > > > >> >> > >
> > > > > >> >> > > > -steve
> > > > > >> >> > > >
> > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > > ayushtkn@gmail.com
> > > > > >
> > > > > >> >> wrote:
> > > > > >> >> > > >
> > > > > >> >> > > > > +1
> > > > > >> >> > > > > Just to mention we would need to release
> > > hadoop-thirdparty
> > > > > too
> > > > > >> >> > before.
> > > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > > >> >> > > > >
> > > > > >> >> > > > > -Ayush
> > > > > >> >> > > > >
> > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > > >> weichiu@apache.org
> > > > > >> >> >
> > > > > >> >> > > > wrote:
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Hi all,
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as
> of
> > > now
> > > > > >> we've
> > > > > >> >> > > > > accumulated
> > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number
> of
> > > > > >> >> downstreamers
> > > > > >> >> > > are
> > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> > > > version
> > > > > >> >> > conflict
> > > > > >> >> > > > > issue.
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >> >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > We should start the release work for 3.3.1 before
> the
> > > > diff
> > > > > >> >> becomes
> > > > > >> >> > > even
> > > > > >> >> > > > > > larger.
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > I believe there are currently only two real
> blockers
> > > > for a
> > > > > >> >> 3.3.1
> > > > > >> >> > > > (using
> > > > > >> >> > > > > > this filter
> > > > > >> >> > > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >> >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > >> >> > > > > > )
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > 1. HDFS-15566 <
> > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > >> >> > > > > > 2.
> > > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > 2.
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1
> > RM?
> > > > > >> >> > > > > >
> > > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the
> ARM
> > > > build
> > > > > >> >> > process.
> > > > > >> >> > > > > That's
> > > > > >> >> > > > > > going to be important for future releases.
> > > > > >> >> > > > >
> > > > > >> >> > > > >
> > > > > >> >>
> > > > ---------------------------------------------------------------------
> > > > > >> >> > > > > To unsubscribe, e-mail:
> > > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > >> >> > > > > For additional commands, e-mail:
> > > > > >> hdfs-dev-help@hadoop.apache.org
> > > > > >> >> > > > >
> > > > > >> >> > > > >
> > > > > >> >> > > >
> > > > > >> >> > >
> > > > > >> >> >
> > > > > >> >>
> > > > > >> >>
> > > > > >> >> --
> > > > > >> >>
> > > > > >> >>
> > > > > >> >>
> > > > > >> >> --Brahma Reddy Battula
> > > > > >> >>
> > > > > >> >
> > > > > >>
> > > > > >> --
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> --Brahma Reddy Battula
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> > --
> > -Vinay
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by hemanth boyina <he...@gmail.com>.
Hi Steve and Wei-Chiu
Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to trunk
and tried to work out with IPV6, we have faced some issues and have made
the required changes for ipv6 to work.After the changes were made we have
tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
rigorously.Its been quite a some time these changes were deployed in
production cluster and have been in use for extensive purpose.
I think it's good time to add this feature.
Thanks
Hemanth Boyina
On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org> wrote:
> Hi David,
>
> >> Still hoping for help here:
>
> >> https://issues.apache.org/jira/browse/HDFS-15790
>
> I will raise a PR for the said solution soon (in a day or two).
>
> -Vinay
>
> On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
>
> > Hello,
> >
> > Still hoping for help here:
> >
> > https://issues.apache.org/jira/browse/HDFS-15790
> >
> > Looks like it has been worked on, not sure how to best move it forward.
> >
> > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > > I'm going to argue its too late to do IPv6 support close to a release,
> as
> > > it's best if its on developer machines for some time to let all quirks
> > > surface. It's not so much IPv6 itself, but do we cause any regressions
> on
> > > IPv4?
> > >
> > > But: it can/should go into trunk and stabilize there
> > >
> > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > muralikrishna.dmmkr@gmail.com> wrote:
> > >
> > > > Hi Brahma,
> > > >
> > > > I have missed out mentioning about the IPV6 feature in the last mail,
> > > > Support for IPV6 has been in development since 2015 and We have done
> a
> > > good
> > > > amount of testing at our organisation, the feature is stable and used
> > by
> > > > our customers extensively in the last one year. I think it is a good
> > time
> > > > to add the IPV6 support to 3.3.1.
> > > >
> > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > >
> > > > Thanks
> > > > D M Murali Krishna Reddy
> > > >
> > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > >
> > > > > Hi Brahma,
> > > > >
> > > > > Can we have this new feature "YARN Registry based AM discovery with
> > > retry
> > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > release. I
> > > > > have also attached a test-report in the below jira.
> > > > >
> > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > >
> > > > >
> > > > > Thanks,
> > > > > D M Murali Krishna Reddy
> > > > >
> > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > brahma@apache.org
> > > > >
> > > > > wrote:
> > > > >
> > > > >> Hi Bilwa,
> > > > >>
> > > > >> I have commented on the jira's you mentioned. Based on the
> stability
> > > we
> > > > >> can
> > > > >> plan this.But needs to be merged ASAP.
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> wrote:
> > > > >>
> > > > >> > Hi Brahma,
> > > > >> >
> > > > >> > Can we have below features in 3.3.1 release? We have been using
> > > these
> > > > >> > features for a long time. They are stable and tested in bigger
> > > > clusters.
> > > > >> >
> > > > >> > 1. Container reuse -
> > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > >> > 2. Speculative attempts should not run on the same node -
> > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > >> >
> > > > >> > Thanks,
> > > > >> > Bilwa
> > > > >> >
> > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > brahma@apache.org
> > > > >
> > > > >> > wrote:
> > > > >> >
> > > > >> >> Sorry for the late reply..
> > > > >> >>
> > > > >> >> I will come up with a plan.. Please let me know if anybody has
> > some
> > > > >> >> features/improvements/bugs that need to be included.
> > > > >> >>
> > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > sunilg@apache.org>
> > > > >> wrote:
> > > > >> >>
> > > > >> >> > Hi Wei-Chiu,
> > > > >> >> >
> > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > >> >> >
> > > > >> >> > Thanks
> > > > >> >> > Sunil
> > > > >> >> >
> > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > wrote:
> > > > >> >> >
> > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > >> >> > <stevel@cloudera.com.invalid
> > > > >> >> > > >
> > > > >> >> > > wrote:
> > > > >> >> > >
> > > > >> >> > > >
> > > > >> >> > > > Regarding blockers &c: how about we have a little
> hackathon
> > > > >> where we
> > > > >> >> > try
> > > > >> >> > > > and get things in. This means a promise of review time
> from
> > > the
> > > > >> >> people
> > > > >> >> > > with
> > > > >> >> > > > commit rights and other people who understand the code
> > > (Stack?)
> > > > >> >> > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > testing).
> > > > >> >> > > Thanks,
> > > > >> >> > > S
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > > > -steve
> > > > >> >> > > >
> > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > ayushtkn@gmail.com
> > > > >
> > > > >> >> wrote:
> > > > >> >> > > >
> > > > >> >> > > > > +1
> > > > >> >> > > > > Just to mention we would need to release
> > hadoop-thirdparty
> > > > too
> > > > >> >> > before.
> > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > >> >> > > > >
> > > > >> >> > > > > -Ayush
> > > > >> >> > > > >
> > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > >> weichiu@apache.org
> > > > >> >> >
> > > > >> >> > > > wrote:
> > > > >> >> > > > > >
> > > > >> >> > > > > > Hi all,
> > > > >> >> > > > > >
> > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of
> > now
> > > > >> we've
> > > > >> >> > > > > accumulated
> > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > > > >> >> downstreamers
> > > > >> >> > > are
> > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> > > version
> > > > >> >> > conflict
> > > > >> >> > > > > issue.
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > >> >> > > > > >
> > > > >> >> > > > > > We should start the release work for 3.3.1 before the
> > > diff
> > > > >> >> becomes
> > > > >> >> > > even
> > > > >> >> > > > > > larger.
> > > > >> >> > > > > >
> > > > >> >> > > > > > I believe there are currently only two real blockers
> > > for a
> > > > >> >> 3.3.1
> > > > >> >> > > > (using
> > > > >> >> > > > > > this filter
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > >> >> > > > > > )
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > > > 1. HDFS-15566 <
> > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > >> >> > > > > > 2.
> > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > >> >> > > > > >
> > > > >> >> > > > > > 2.
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1
> RM?
> > > > >> >> > > > > >
> > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> > > build
> > > > >> >> > process.
> > > > >> >> > > > > That's
> > > > >> >> > > > > > going to be important for future releases.
> > > > >> >> > > > >
> > > > >> >> > > > >
> > > > >> >>
> > > ---------------------------------------------------------------------
> > > > >> >> > > > > To unsubscribe, e-mail:
> > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > >> >> > > > > For additional commands, e-mail:
> > > > >> hdfs-dev-help@hadoop.apache.org
> > > > >> >> > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >> >>
> > > > >> >> --
> > > > >> >>
> > > > >> >>
> > > > >> >>
> > > > >> >> --Brahma Reddy Battula
> > > > >> >>
> > > > >> >
> > > > >>
> > > > >> --
> > > > >>
> > > > >>
> > > > >>
> > > > >> --Brahma Reddy Battula
> > > > >>
> > > > >
> > > >
> > >
> >
> --
> -Vinay
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by hemanth boyina <he...@gmail.com>.
Hi Steve and Wei-Chiu
Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to trunk
and tried to work out with IPV6, we have faced some issues and have made
the required changes for ipv6 to work.After the changes were made we have
tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
rigorously.Its been quite a some time these changes were deployed in
production cluster and have been in use for extensive purpose.
I think it's good time to add this feature.
Thanks
Hemanth Boyina
On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org> wrote:
> Hi David,
>
> >> Still hoping for help here:
>
> >> https://issues.apache.org/jira/browse/HDFS-15790
>
> I will raise a PR for the said solution soon (in a day or two).
>
> -Vinay
>
> On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
>
> > Hello,
> >
> > Still hoping for help here:
> >
> > https://issues.apache.org/jira/browse/HDFS-15790
> >
> > Looks like it has been worked on, not sure how to best move it forward.
> >
> > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > > I'm going to argue its too late to do IPv6 support close to a release,
> as
> > > it's best if its on developer machines for some time to let all quirks
> > > surface. It's not so much IPv6 itself, but do we cause any regressions
> on
> > > IPv4?
> > >
> > > But: it can/should go into trunk and stabilize there
> > >
> > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > muralikrishna.dmmkr@gmail.com> wrote:
> > >
> > > > Hi Brahma,
> > > >
> > > > I have missed out mentioning about the IPV6 feature in the last mail,
> > > > Support for IPV6 has been in development since 2015 and We have done
> a
> > > good
> > > > amount of testing at our organisation, the feature is stable and used
> > by
> > > > our customers extensively in the last one year. I think it is a good
> > time
> > > > to add the IPV6 support to 3.3.1.
> > > >
> > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > >
> > > > Thanks
> > > > D M Murali Krishna Reddy
> > > >
> > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > >
> > > > > Hi Brahma,
> > > > >
> > > > > Can we have this new feature "YARN Registry based AM discovery with
> > > retry
> > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > release. I
> > > > > have also attached a test-report in the below jira.
> > > > >
> > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > >
> > > > >
> > > > > Thanks,
> > > > > D M Murali Krishna Reddy
> > > > >
> > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > brahma@apache.org
> > > > >
> > > > > wrote:
> > > > >
> > > > >> Hi Bilwa,
> > > > >>
> > > > >> I have commented on the jira's you mentioned. Based on the
> stability
> > > we
> > > > >> can
> > > > >> plan this.But needs to be merged ASAP.
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> wrote:
> > > > >>
> > > > >> > Hi Brahma,
> > > > >> >
> > > > >> > Can we have below features in 3.3.1 release? We have been using
> > > these
> > > > >> > features for a long time. They are stable and tested in bigger
> > > > clusters.
> > > > >> >
> > > > >> > 1. Container reuse -
> > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > >> > 2. Speculative attempts should not run on the same node -
> > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > >> >
> > > > >> > Thanks,
> > > > >> > Bilwa
> > > > >> >
> > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > brahma@apache.org
> > > > >
> > > > >> > wrote:
> > > > >> >
> > > > >> >> Sorry for the late reply..
> > > > >> >>
> > > > >> >> I will come up with a plan.. Please let me know if anybody has
> > some
> > > > >> >> features/improvements/bugs that need to be included.
> > > > >> >>
> > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > sunilg@apache.org>
> > > > >> wrote:
> > > > >> >>
> > > > >> >> > Hi Wei-Chiu,
> > > > >> >> >
> > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > >> >> >
> > > > >> >> > Thanks
> > > > >> >> > Sunil
> > > > >> >> >
> > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > wrote:
> > > > >> >> >
> > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > >> >> > <stevel@cloudera.com.invalid
> > > > >> >> > > >
> > > > >> >> > > wrote:
> > > > >> >> > >
> > > > >> >> > > >
> > > > >> >> > > > Regarding blockers &c: how about we have a little
> hackathon
> > > > >> where we
> > > > >> >> > try
> > > > >> >> > > > and get things in. This means a promise of review time
> from
> > > the
> > > > >> >> people
> > > > >> >> > > with
> > > > >> >> > > > commit rights and other people who understand the code
> > > (Stack?)
> > > > >> >> > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > testing).
> > > > >> >> > > Thanks,
> > > > >> >> > > S
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > > > -steve
> > > > >> >> > > >
> > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > ayushtkn@gmail.com
> > > > >
> > > > >> >> wrote:
> > > > >> >> > > >
> > > > >> >> > > > > +1
> > > > >> >> > > > > Just to mention we would need to release
> > hadoop-thirdparty
> > > > too
> > > > >> >> > before.
> > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > >> >> > > > >
> > > > >> >> > > > > -Ayush
> > > > >> >> > > > >
> > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > >> weichiu@apache.org
> > > > >> >> >
> > > > >> >> > > > wrote:
> > > > >> >> > > > > >
> > > > >> >> > > > > > Hi all,
> > > > >> >> > > > > >
> > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of
> > now
> > > > >> we've
> > > > >> >> > > > > accumulated
> > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > > > >> >> downstreamers
> > > > >> >> > > are
> > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> > > version
> > > > >> >> > conflict
> > > > >> >> > > > > issue.
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > >> >> > > > > >
> > > > >> >> > > > > > We should start the release work for 3.3.1 before the
> > > diff
> > > > >> >> becomes
> > > > >> >> > > even
> > > > >> >> > > > > > larger.
> > > > >> >> > > > > >
> > > > >> >> > > > > > I believe there are currently only two real blockers
> > > for a
> > > > >> >> 3.3.1
> > > > >> >> > > > (using
> > > > >> >> > > > > > this filter
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > >> >> > > > > > )
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > > > 1. HDFS-15566 <
> > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > >> >> > > > > > 2.
> > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > >> >> > > > > >
> > > > >> >> > > > > > 2.
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1
> RM?
> > > > >> >> > > > > >
> > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> > > build
> > > > >> >> > process.
> > > > >> >> > > > > That's
> > > > >> >> > > > > > going to be important for future releases.
> > > > >> >> > > > >
> > > > >> >> > > > >
> > > > >> >>
> > > ---------------------------------------------------------------------
> > > > >> >> > > > > To unsubscribe, e-mail:
> > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > >> >> > > > > For additional commands, e-mail:
> > > > >> hdfs-dev-help@hadoop.apache.org
> > > > >> >> > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >> >>
> > > > >> >> --
> > > > >> >>
> > > > >> >>
> > > > >> >>
> > > > >> >> --Brahma Reddy Battula
> > > > >> >>
> > > > >> >
> > > > >>
> > > > >> --
> > > > >>
> > > > >>
> > > > >>
> > > > >> --Brahma Reddy Battula
> > > > >>
> > > > >
> > > >
> > >
> >
> --
> -Vinay
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by hemanth boyina <he...@gmail.com>.
Hi Steve and Wei-Chiu
Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to trunk
and tried to work out with IPV6, we have faced some issues and have made
the required changes for ipv6 to work.After the changes were made we have
tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
rigorously.Its been quite a some time these changes were deployed in
production cluster and have been in use for extensive purpose.
I think it's good time to add this feature.
Thanks
Hemanth Boyina
On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org> wrote:
> Hi David,
>
> >> Still hoping for help here:
>
> >> https://issues.apache.org/jira/browse/HDFS-15790
>
> I will raise a PR for the said solution soon (in a day or two).
>
> -Vinay
>
> On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
>
> > Hello,
> >
> > Still hoping for help here:
> >
> > https://issues.apache.org/jira/browse/HDFS-15790
> >
> > Looks like it has been worked on, not sure how to best move it forward.
> >
> > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > > I'm going to argue its too late to do IPv6 support close to a release,
> as
> > > it's best if its on developer machines for some time to let all quirks
> > > surface. It's not so much IPv6 itself, but do we cause any regressions
> on
> > > IPv4?
> > >
> > > But: it can/should go into trunk and stabilize there
> > >
> > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > muralikrishna.dmmkr@gmail.com> wrote:
> > >
> > > > Hi Brahma,
> > > >
> > > > I have missed out mentioning about the IPV6 feature in the last mail,
> > > > Support for IPV6 has been in development since 2015 and We have done
> a
> > > good
> > > > amount of testing at our organisation, the feature is stable and used
> > by
> > > > our customers extensively in the last one year. I think it is a good
> > time
> > > > to add the IPV6 support to 3.3.1.
> > > >
> > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > >
> > > > Thanks
> > > > D M Murali Krishna Reddy
> > > >
> > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > >
> > > > > Hi Brahma,
> > > > >
> > > > > Can we have this new feature "YARN Registry based AM discovery with
> > > retry
> > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > release. I
> > > > > have also attached a test-report in the below jira.
> > > > >
> > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > >
> > > > >
> > > > > Thanks,
> > > > > D M Murali Krishna Reddy
> > > > >
> > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > brahma@apache.org
> > > > >
> > > > > wrote:
> > > > >
> > > > >> Hi Bilwa,
> > > > >>
> > > > >> I have commented on the jira's you mentioned. Based on the
> stability
> > > we
> > > > >> can
> > > > >> plan this.But needs to be merged ASAP.
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> wrote:
> > > > >>
> > > > >> > Hi Brahma,
> > > > >> >
> > > > >> > Can we have below features in 3.3.1 release? We have been using
> > > these
> > > > >> > features for a long time. They are stable and tested in bigger
> > > > clusters.
> > > > >> >
> > > > >> > 1. Container reuse -
> > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > >> > 2. Speculative attempts should not run on the same node -
> > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > >> >
> > > > >> > Thanks,
> > > > >> > Bilwa
> > > > >> >
> > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > brahma@apache.org
> > > > >
> > > > >> > wrote:
> > > > >> >
> > > > >> >> Sorry for the late reply..
> > > > >> >>
> > > > >> >> I will come up with a plan.. Please let me know if anybody has
> > some
> > > > >> >> features/improvements/bugs that need to be included.
> > > > >> >>
> > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > sunilg@apache.org>
> > > > >> wrote:
> > > > >> >>
> > > > >> >> > Hi Wei-Chiu,
> > > > >> >> >
> > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > >> >> >
> > > > >> >> > Thanks
> > > > >> >> > Sunil
> > > > >> >> >
> > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > wrote:
> > > > >> >> >
> > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > >> >> > <stevel@cloudera.com.invalid
> > > > >> >> > > >
> > > > >> >> > > wrote:
> > > > >> >> > >
> > > > >> >> > > >
> > > > >> >> > > > Regarding blockers &c: how about we have a little
> hackathon
> > > > >> where we
> > > > >> >> > try
> > > > >> >> > > > and get things in. This means a promise of review time
> from
> > > the
> > > > >> >> people
> > > > >> >> > > with
> > > > >> >> > > > commit rights and other people who understand the code
> > > (Stack?)
> > > > >> >> > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > testing).
> > > > >> >> > > Thanks,
> > > > >> >> > > S
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > > > -steve
> > > > >> >> > > >
> > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > ayushtkn@gmail.com
> > > > >
> > > > >> >> wrote:
> > > > >> >> > > >
> > > > >> >> > > > > +1
> > > > >> >> > > > > Just to mention we would need to release
> > hadoop-thirdparty
> > > > too
> > > > >> >> > before.
> > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > >> >> > > > >
> > > > >> >> > > > > -Ayush
> > > > >> >> > > > >
> > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > >> weichiu@apache.org
> > > > >> >> >
> > > > >> >> > > > wrote:
> > > > >> >> > > > > >
> > > > >> >> > > > > > Hi all,
> > > > >> >> > > > > >
> > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of
> > now
> > > > >> we've
> > > > >> >> > > > > accumulated
> > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > > > >> >> downstreamers
> > > > >> >> > > are
> > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> > > version
> > > > >> >> > conflict
> > > > >> >> > > > > issue.
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > >> >> > > > > >
> > > > >> >> > > > > > We should start the release work for 3.3.1 before the
> > > diff
> > > > >> >> becomes
> > > > >> >> > > even
> > > > >> >> > > > > > larger.
> > > > >> >> > > > > >
> > > > >> >> > > > > > I believe there are currently only two real blockers
> > > for a
> > > > >> >> 3.3.1
> > > > >> >> > > > (using
> > > > >> >> > > > > > this filter
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > >> >> > > > > > )
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > > > 1. HDFS-15566 <
> > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > >> >> > > > > > 2.
> > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > >> >> > > > > >
> > > > >> >> > > > > > 2.
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1
> RM?
> > > > >> >> > > > > >
> > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> > > build
> > > > >> >> > process.
> > > > >> >> > > > > That's
> > > > >> >> > > > > > going to be important for future releases.
> > > > >> >> > > > >
> > > > >> >> > > > >
> > > > >> >>
> > > ---------------------------------------------------------------------
> > > > >> >> > > > > To unsubscribe, e-mail:
> > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > >> >> > > > > For additional commands, e-mail:
> > > > >> hdfs-dev-help@hadoop.apache.org
> > > > >> >> > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >> >>
> > > > >> >> --
> > > > >> >>
> > > > >> >>
> > > > >> >>
> > > > >> >> --Brahma Reddy Battula
> > > > >> >>
> > > > >> >
> > > > >>
> > > > >> --
> > > > >>
> > > > >>
> > > > >>
> > > > >> --Brahma Reddy Battula
> > > > >>
> > > > >
> > > >
> > >
> >
> --
> -Vinay
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by hemanth boyina <he...@gmail.com>.
Hi Steve and Wei-Chiu
Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to trunk
and tried to work out with IPV6, we have faced some issues and have made
the required changes for ipv6 to work.After the changes were made we have
tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested
rigorously.Its been quite a some time these changes were deployed in
production cluster and have been in use for extensive purpose.
I think it's good time to add this feature.
Thanks
Hemanth Boyina
On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vi...@apache.org> wrote:
> Hi David,
>
> >> Still hoping for help here:
>
> >> https://issues.apache.org/jira/browse/HDFS-15790
>
> I will raise a PR for the said solution soon (in a day or two).
>
> -Vinay
>
> On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
>
> > Hello,
> >
> > Still hoping for help here:
> >
> > https://issues.apache.org/jira/browse/HDFS-15790
> >
> > Looks like it has been worked on, not sure how to best move it forward.
> >
> > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran
> <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > > I'm going to argue its too late to do IPv6 support close to a release,
> as
> > > it's best if its on developer machines for some time to let all quirks
> > > surface. It's not so much IPv6 itself, but do we cause any regressions
> on
> > > IPv4?
> > >
> > > But: it can/should go into trunk and stabilize there
> > >
> > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > > muralikrishna.dmmkr@gmail.com> wrote:
> > >
> > > > Hi Brahma,
> > > >
> > > > I have missed out mentioning about the IPV6 feature in the last mail,
> > > > Support for IPV6 has been in development since 2015 and We have done
> a
> > > good
> > > > amount of testing at our organisation, the feature is stable and used
> > by
> > > > our customers extensively in the last one year. I think it is a good
> > time
> > > > to add the IPV6 support to 3.3.1.
> > > >
> > > > https://issues.apache.org/jira/browse/HADOOP-11890
> > > >
> > > > Thanks
> > > > D M Murali Krishna Reddy
> > > >
> > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > > muralikrishna.dmmkr@gmail.com> wrote:
> > > >
> > > > > Hi Brahma,
> > > > >
> > > > > Can we have this new feature "YARN Registry based AM discovery with
> > > retry
> > > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> > release. I
> > > > > have also attached a test-report in the below jira.
> > > > >
> > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > > >
> > > > >
> > > > > Thanks,
> > > > > D M Murali Krishna Reddy
> > > > >
> > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > > brahma@apache.org
> > > > >
> > > > > wrote:
> > > > >
> > > > >> Hi Bilwa,
> > > > >>
> > > > >> I have commented on the jira's you mentioned. Based on the
> stability
> > > we
> > > > >> can
> > > > >> plan this.But needs to be merged ASAP.
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com>
> wrote:
> > > > >>
> > > > >> > Hi Brahma,
> > > > >> >
> > > > >> > Can we have below features in 3.3.1 release? We have been using
> > > these
> > > > >> > features for a long time. They are stable and tested in bigger
> > > > clusters.
> > > > >> >
> > > > >> > 1. Container reuse -
> > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > > >> > 2. Speculative attempts should not run on the same node -
> > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > > >> >
> > > > >> > Thanks,
> > > > >> > Bilwa
> > > > >> >
> > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > > brahma@apache.org
> > > > >
> > > > >> > wrote:
> > > > >> >
> > > > >> >> Sorry for the late reply..
> > > > >> >>
> > > > >> >> I will come up with a plan.. Please let me know if anybody has
> > some
> > > > >> >> features/improvements/bugs that need to be included.
> > > > >> >>
> > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> > sunilg@apache.org>
> > > > >> wrote:
> > > > >> >>
> > > > >> >> > Hi Wei-Chiu,
> > > > >> >> >
> > > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > > >> >> >
> > > > >> >> > Thanks
> > > > >> >> > Sunil
> > > > >> >> >
> > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> > wrote:
> > > > >> >> >
> > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > > >> >> > <stevel@cloudera.com.invalid
> > > > >> >> > > >
> > > > >> >> > > wrote:
> > > > >> >> > >
> > > > >> >> > > >
> > > > >> >> > > > Regarding blockers &c: how about we have a little
> hackathon
> > > > >> where we
> > > > >> >> > try
> > > > >> >> > > > and get things in. This means a promise of review time
> from
> > > the
> > > > >> >> people
> > > > >> >> > > with
> > > > >> >> > > > commit rights and other people who understand the code
> > > (Stack?)
> > > > >> >> > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > > testing).
> > > > >> >> > > Thanks,
> > > > >> >> > > S
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > > > -steve
> > > > >> >> > > >
> > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > > ayushtkn@gmail.com
> > > > >
> > > > >> >> wrote:
> > > > >> >> > > >
> > > > >> >> > > > > +1
> > > > >> >> > > > > Just to mention we would need to release
> > hadoop-thirdparty
> > > > too
> > > > >> >> > before.
> > > > >> >> > > > > Presently we are using the snapshot version of it.
> > > > >> >> > > > >
> > > > >> >> > > > > -Ayush
> > > > >> >> > > > >
> > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > > >> weichiu@apache.org
> > > > >> >> >
> > > > >> >> > > > wrote:
> > > > >> >> > > > > >
> > > > >> >> > > > > > Hi all,
> > > > >> >> > > > > >
> > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of
> > now
> > > > >> we've
> > > > >> >> > > > > accumulated
> > > > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > > > >> >> downstreamers
> > > > >> >> > > are
> > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> > > version
> > > > >> >> > conflict
> > > > >> >> > > > > issue.
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > >> >> > > > > >
> > > > >> >> > > > > > We should start the release work for 3.3.1 before the
> > > diff
> > > > >> >> becomes
> > > > >> >> > > even
> > > > >> >> > > > > > larger.
> > > > >> >> > > > > >
> > > > >> >> > > > > > I believe there are currently only two real blockers
> > > for a
> > > > >> >> 3.3.1
> > > > >> >> > > > (using
> > > > >> >> > > > > > this filter
> > > > >> >> > > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > >> >> > > > > > )
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > > > 1. HDFS-15566 <
> > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > >> >> > > > > > 2.
> > > > >> >> > > > > > 1. HADOOP-17112 <
> > > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > >> >> > > > > >
> > > > >> >> > > > > > 2.
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > > >
> > > > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1
> RM?
> > > > >> >> > > > > >
> > > > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> > > build
> > > > >> >> > process.
> > > > >> >> > > > > That's
> > > > >> >> > > > > > going to be important for future releases.
> > > > >> >> > > > >
> > > > >> >> > > > >
> > > > >> >>
> > > ---------------------------------------------------------------------
> > > > >> >> > > > > To unsubscribe, e-mail:
> > > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > > >> >> > > > > For additional commands, e-mail:
> > > > >> hdfs-dev-help@hadoop.apache.org
> > > > >> >> > > > >
> > > > >> >> > > > >
> > > > >> >> > > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >> >>
> > > > >> >> --
> > > > >> >>
> > > > >> >>
> > > > >> >>
> > > > >> >> --Brahma Reddy Battula
> > > > >> >>
> > > > >> >
> > > > >>
> > > > >> --
> > > > >>
> > > > >>
> > > > >>
> > > > >> --Brahma Reddy Battula
> > > > >>
> > > > >
> > > >
> > >
> >
> --
> -Vinay
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Vinayakumar B <vi...@apache.org>.
Hi David,
>> Still hoping for help here:
>> https://issues.apache.org/jira/browse/HDFS-15790
I will raise a PR for the said solution soon (in a day or two).
-Vinay
On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> Hello,
>
> Still hoping for help here:
>
> https://issues.apache.org/jira/browse/HDFS-15790
>
> Looks like it has been worked on, not sure how to best move it forward.
>
> On Wed, Mar 10, 2021, 12:21 PM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > I'm going to argue its too late to do IPv6 support close to a release, as
> > it's best if its on developer machines for some time to let all quirks
> > surface. It's not so much IPv6 itself, but do we cause any regressions on
> > IPv4?
> >
> > But: it can/should go into trunk and stabilize there
> >
> > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > muralikrishna.dmmkr@gmail.com> wrote:
> >
> > > Hi Brahma,
> > >
> > > I have missed out mentioning about the IPV6 feature in the last mail,
> > > Support for IPV6 has been in development since 2015 and We have done a
> > good
> > > amount of testing at our organisation, the feature is stable and used
> by
> > > our customers extensively in the last one year. I think it is a good
> time
> > > to add the IPV6 support to 3.3.1.
> > >
> > > https://issues.apache.org/jira/browse/HADOOP-11890
> > >
> > > Thanks
> > > D M Murali Krishna Reddy
> > >
> > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > muralikrishna.dmmkr@gmail.com> wrote:
> > >
> > > > Hi Brahma,
> > > >
> > > > Can we have this new feature "YARN Registry based AM discovery with
> > retry
> > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> release. I
> > > > have also attached a test-report in the below jira.
> > > >
> > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > >
> > > >
> > > > Thanks,
> > > > D M Murali Krishna Reddy
> > > >
> > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > brahma@apache.org
> > > >
> > > > wrote:
> > > >
> > > >> Hi Bilwa,
> > > >>
> > > >> I have commented on the jira's you mentioned. Based on the stability
> > we
> > > >> can
> > > >> plan this.But needs to be merged ASAP.
> > > >>
> > > >>
> > > >>
> > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> > > >>
> > > >> > Hi Brahma,
> > > >> >
> > > >> > Can we have below features in 3.3.1 release? We have been using
> > these
> > > >> > features for a long time. They are stable and tested in bigger
> > > clusters.
> > > >> >
> > > >> > 1. Container reuse -
> > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > >> > 2. Speculative attempts should not run on the same node -
> > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > >> >
> > > >> > Thanks,
> > > >> > Bilwa
> > > >> >
> > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > brahma@apache.org
> > > >
> > > >> > wrote:
> > > >> >
> > > >> >> Sorry for the late reply..
> > > >> >>
> > > >> >> I will come up with a plan.. Please let me know if anybody has
> some
> > > >> >> features/improvements/bugs that need to be included.
> > > >> >>
> > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> sunilg@apache.org>
> > > >> wrote:
> > > >> >>
> > > >> >> > Hi Wei-Chiu,
> > > >> >> >
> > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > >> >> >
> > > >> >> > Thanks
> > > >> >> > Sunil
> > > >> >> >
> > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> wrote:
> > > >> >> >
> > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > >> >> > <stevel@cloudera.com.invalid
> > > >> >> > > >
> > > >> >> > > wrote:
> > > >> >> > >
> > > >> >> > > >
> > > >> >> > > > Regarding blockers &c: how about we have a little hackathon
> > > >> where we
> > > >> >> > try
> > > >> >> > > > and get things in. This means a promise of review time from
> > the
> > > >> >> people
> > > >> >> > > with
> > > >> >> > > > commit rights and other people who understand the code
> > (Stack?)
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > testing).
> > > >> >> > > Thanks,
> > > >> >> > > S
> > > >> >> > >
> > > >> >> > >
> > > >> >> > >
> > > >> >> > >
> > > >> >> > > > -steve
> > > >> >> > > >
> > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > ayushtkn@gmail.com
> > > >
> > > >> >> wrote:
> > > >> >> > > >
> > > >> >> > > > > +1
> > > >> >> > > > > Just to mention we would need to release
> hadoop-thirdparty
> > > too
> > > >> >> > before.
> > > >> >> > > > > Presently we are using the snapshot version of it.
> > > >> >> > > > >
> > > >> >> > > > > -Ayush
> > > >> >> > > > >
> > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > >> weichiu@apache.org
> > > >> >> >
> > > >> >> > > > wrote:
> > > >> >> > > > > >
> > > >> >> > > > > > Hi all,
> > > >> >> > > > > >
> > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of
> now
> > > >> we've
> > > >> >> > > > > accumulated
> > > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > > >> >> downstreamers
> > > >> >> > > are
> > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> > version
> > > >> >> > conflict
> > > >> >> > > > > issue.
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > >> >> > > > > >
> > > >> >> > > > > > We should start the release work for 3.3.1 before the
> > diff
> > > >> >> becomes
> > > >> >> > > even
> > > >> >> > > > > > larger.
> > > >> >> > > > > >
> > > >> >> > > > > > I believe there are currently only two real blockers
> > for a
> > > >> >> 3.3.1
> > > >> >> > > > (using
> > > >> >> > > > > > this filter
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > >> >> > > > > > )
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > > > 1. HDFS-15566 <
> > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > >> >> > > > > > 2.
> > > >> >> > > > > > 1. HADOOP-17112 <
> > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > >> >> > > > > >
> > > >> >> > > > > > 2.
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > > >> >> > > > > >
> > > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> > build
> > > >> >> > process.
> > > >> >> > > > > That's
> > > >> >> > > > > > going to be important for future releases.
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >>
> > ---------------------------------------------------------------------
> > > >> >> > > > > To unsubscribe, e-mail:
> > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > >> >> > > > > For additional commands, e-mail:
> > > >> hdfs-dev-help@hadoop.apache.org
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >> >>
> > > >> >> --
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >> --Brahma Reddy Battula
> > > >> >>
> > > >> >
> > > >>
> > > >> --
> > > >>
> > > >>
> > > >>
> > > >> --Brahma Reddy Battula
> > > >>
> > > >
> > >
> >
>
--
-Vinay
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Vinayakumar B <vi...@apache.org>.
Hi David,
>> Still hoping for help here:
>> https://issues.apache.org/jira/browse/HDFS-15790
I will raise a PR for the said solution soon (in a day or two).
-Vinay
On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> Hello,
>
> Still hoping for help here:
>
> https://issues.apache.org/jira/browse/HDFS-15790
>
> Looks like it has been worked on, not sure how to best move it forward.
>
> On Wed, Mar 10, 2021, 12:21 PM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > I'm going to argue its too late to do IPv6 support close to a release, as
> > it's best if its on developer machines for some time to let all quirks
> > surface. It's not so much IPv6 itself, but do we cause any regressions on
> > IPv4?
> >
> > But: it can/should go into trunk and stabilize there
> >
> > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > muralikrishna.dmmkr@gmail.com> wrote:
> >
> > > Hi Brahma,
> > >
> > > I have missed out mentioning about the IPV6 feature in the last mail,
> > > Support for IPV6 has been in development since 2015 and We have done a
> > good
> > > amount of testing at our organisation, the feature is stable and used
> by
> > > our customers extensively in the last one year. I think it is a good
> time
> > > to add the IPV6 support to 3.3.1.
> > >
> > > https://issues.apache.org/jira/browse/HADOOP-11890
> > >
> > > Thanks
> > > D M Murali Krishna Reddy
> > >
> > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > muralikrishna.dmmkr@gmail.com> wrote:
> > >
> > > > Hi Brahma,
> > > >
> > > > Can we have this new feature "YARN Registry based AM discovery with
> > retry
> > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> release. I
> > > > have also attached a test-report in the below jira.
> > > >
> > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > >
> > > >
> > > > Thanks,
> > > > D M Murali Krishna Reddy
> > > >
> > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > brahma@apache.org
> > > >
> > > > wrote:
> > > >
> > > >> Hi Bilwa,
> > > >>
> > > >> I have commented on the jira's you mentioned. Based on the stability
> > we
> > > >> can
> > > >> plan this.But needs to be merged ASAP.
> > > >>
> > > >>
> > > >>
> > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> > > >>
> > > >> > Hi Brahma,
> > > >> >
> > > >> > Can we have below features in 3.3.1 release? We have been using
> > these
> > > >> > features for a long time. They are stable and tested in bigger
> > > clusters.
> > > >> >
> > > >> > 1. Container reuse -
> > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > >> > 2. Speculative attempts should not run on the same node -
> > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > >> >
> > > >> > Thanks,
> > > >> > Bilwa
> > > >> >
> > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > brahma@apache.org
> > > >
> > > >> > wrote:
> > > >> >
> > > >> >> Sorry for the late reply..
> > > >> >>
> > > >> >> I will come up with a plan.. Please let me know if anybody has
> some
> > > >> >> features/improvements/bugs that need to be included.
> > > >> >>
> > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> sunilg@apache.org>
> > > >> wrote:
> > > >> >>
> > > >> >> > Hi Wei-Chiu,
> > > >> >> >
> > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > >> >> >
> > > >> >> > Thanks
> > > >> >> > Sunil
> > > >> >> >
> > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> wrote:
> > > >> >> >
> > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > >> >> > <stevel@cloudera.com.invalid
> > > >> >> > > >
> > > >> >> > > wrote:
> > > >> >> > >
> > > >> >> > > >
> > > >> >> > > > Regarding blockers &c: how about we have a little hackathon
> > > >> where we
> > > >> >> > try
> > > >> >> > > > and get things in. This means a promise of review time from
> > the
> > > >> >> people
> > > >> >> > > with
> > > >> >> > > > commit rights and other people who understand the code
> > (Stack?)
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > testing).
> > > >> >> > > Thanks,
> > > >> >> > > S
> > > >> >> > >
> > > >> >> > >
> > > >> >> > >
> > > >> >> > >
> > > >> >> > > > -steve
> > > >> >> > > >
> > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > ayushtkn@gmail.com
> > > >
> > > >> >> wrote:
> > > >> >> > > >
> > > >> >> > > > > +1
> > > >> >> > > > > Just to mention we would need to release
> hadoop-thirdparty
> > > too
> > > >> >> > before.
> > > >> >> > > > > Presently we are using the snapshot version of it.
> > > >> >> > > > >
> > > >> >> > > > > -Ayush
> > > >> >> > > > >
> > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > >> weichiu@apache.org
> > > >> >> >
> > > >> >> > > > wrote:
> > > >> >> > > > > >
> > > >> >> > > > > > Hi all,
> > > >> >> > > > > >
> > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of
> now
> > > >> we've
> > > >> >> > > > > accumulated
> > > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > > >> >> downstreamers
> > > >> >> > > are
> > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> > version
> > > >> >> > conflict
> > > >> >> > > > > issue.
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > >> >> > > > > >
> > > >> >> > > > > > We should start the release work for 3.3.1 before the
> > diff
> > > >> >> becomes
> > > >> >> > > even
> > > >> >> > > > > > larger.
> > > >> >> > > > > >
> > > >> >> > > > > > I believe there are currently only two real blockers
> > for a
> > > >> >> 3.3.1
> > > >> >> > > > (using
> > > >> >> > > > > > this filter
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > >> >> > > > > > )
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > > > 1. HDFS-15566 <
> > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > >> >> > > > > > 2.
> > > >> >> > > > > > 1. HADOOP-17112 <
> > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > >> >> > > > > >
> > > >> >> > > > > > 2.
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > > >> >> > > > > >
> > > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> > build
> > > >> >> > process.
> > > >> >> > > > > That's
> > > >> >> > > > > > going to be important for future releases.
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >>
> > ---------------------------------------------------------------------
> > > >> >> > > > > To unsubscribe, e-mail:
> > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > >> >> > > > > For additional commands, e-mail:
> > > >> hdfs-dev-help@hadoop.apache.org
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >> >>
> > > >> >> --
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >> --Brahma Reddy Battula
> > > >> >>
> > > >> >
> > > >>
> > > >> --
> > > >>
> > > >>
> > > >>
> > > >> --Brahma Reddy Battula
> > > >>
> > > >
> > >
> >
>
--
-Vinay
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Vinayakumar B <vi...@apache.org>.
Hi David,
>> Still hoping for help here:
>> https://issues.apache.org/jira/browse/HDFS-15790
I will raise a PR for the said solution soon (in a day or two).
-Vinay
On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> Hello,
>
> Still hoping for help here:
>
> https://issues.apache.org/jira/browse/HDFS-15790
>
> Looks like it has been worked on, not sure how to best move it forward.
>
> On Wed, Mar 10, 2021, 12:21 PM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > I'm going to argue its too late to do IPv6 support close to a release, as
> > it's best if its on developer machines for some time to let all quirks
> > surface. It's not so much IPv6 itself, but do we cause any regressions on
> > IPv4?
> >
> > But: it can/should go into trunk and stabilize there
> >
> > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > muralikrishna.dmmkr@gmail.com> wrote:
> >
> > > Hi Brahma,
> > >
> > > I have missed out mentioning about the IPV6 feature in the last mail,
> > > Support for IPV6 has been in development since 2015 and We have done a
> > good
> > > amount of testing at our organisation, the feature is stable and used
> by
> > > our customers extensively in the last one year. I think it is a good
> time
> > > to add the IPV6 support to 3.3.1.
> > >
> > > https://issues.apache.org/jira/browse/HADOOP-11890
> > >
> > > Thanks
> > > D M Murali Krishna Reddy
> > >
> > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > muralikrishna.dmmkr@gmail.com> wrote:
> > >
> > > > Hi Brahma,
> > > >
> > > > Can we have this new feature "YARN Registry based AM discovery with
> > retry
> > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> release. I
> > > > have also attached a test-report in the below jira.
> > > >
> > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > >
> > > >
> > > > Thanks,
> > > > D M Murali Krishna Reddy
> > > >
> > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > brahma@apache.org
> > > >
> > > > wrote:
> > > >
> > > >> Hi Bilwa,
> > > >>
> > > >> I have commented on the jira's you mentioned. Based on the stability
> > we
> > > >> can
> > > >> plan this.But needs to be merged ASAP.
> > > >>
> > > >>
> > > >>
> > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> > > >>
> > > >> > Hi Brahma,
> > > >> >
> > > >> > Can we have below features in 3.3.1 release? We have been using
> > these
> > > >> > features for a long time. They are stable and tested in bigger
> > > clusters.
> > > >> >
> > > >> > 1. Container reuse -
> > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > >> > 2. Speculative attempts should not run on the same node -
> > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > >> >
> > > >> > Thanks,
> > > >> > Bilwa
> > > >> >
> > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > brahma@apache.org
> > > >
> > > >> > wrote:
> > > >> >
> > > >> >> Sorry for the late reply..
> > > >> >>
> > > >> >> I will come up with a plan.. Please let me know if anybody has
> some
> > > >> >> features/improvements/bugs that need to be included.
> > > >> >>
> > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> sunilg@apache.org>
> > > >> wrote:
> > > >> >>
> > > >> >> > Hi Wei-Chiu,
> > > >> >> >
> > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > >> >> >
> > > >> >> > Thanks
> > > >> >> > Sunil
> > > >> >> >
> > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> wrote:
> > > >> >> >
> > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > >> >> > <stevel@cloudera.com.invalid
> > > >> >> > > >
> > > >> >> > > wrote:
> > > >> >> > >
> > > >> >> > > >
> > > >> >> > > > Regarding blockers &c: how about we have a little hackathon
> > > >> where we
> > > >> >> > try
> > > >> >> > > > and get things in. This means a promise of review time from
> > the
> > > >> >> people
> > > >> >> > > with
> > > >> >> > > > commit rights and other people who understand the code
> > (Stack?)
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > testing).
> > > >> >> > > Thanks,
> > > >> >> > > S
> > > >> >> > >
> > > >> >> > >
> > > >> >> > >
> > > >> >> > >
> > > >> >> > > > -steve
> > > >> >> > > >
> > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > ayushtkn@gmail.com
> > > >
> > > >> >> wrote:
> > > >> >> > > >
> > > >> >> > > > > +1
> > > >> >> > > > > Just to mention we would need to release
> hadoop-thirdparty
> > > too
> > > >> >> > before.
> > > >> >> > > > > Presently we are using the snapshot version of it.
> > > >> >> > > > >
> > > >> >> > > > > -Ayush
> > > >> >> > > > >
> > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > >> weichiu@apache.org
> > > >> >> >
> > > >> >> > > > wrote:
> > > >> >> > > > > >
> > > >> >> > > > > > Hi all,
> > > >> >> > > > > >
> > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of
> now
> > > >> we've
> > > >> >> > > > > accumulated
> > > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > > >> >> downstreamers
> > > >> >> > > are
> > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> > version
> > > >> >> > conflict
> > > >> >> > > > > issue.
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > >> >> > > > > >
> > > >> >> > > > > > We should start the release work for 3.3.1 before the
> > diff
> > > >> >> becomes
> > > >> >> > > even
> > > >> >> > > > > > larger.
> > > >> >> > > > > >
> > > >> >> > > > > > I believe there are currently only two real blockers
> > for a
> > > >> >> 3.3.1
> > > >> >> > > > (using
> > > >> >> > > > > > this filter
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > >> >> > > > > > )
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > > > 1. HDFS-15566 <
> > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > >> >> > > > > > 2.
> > > >> >> > > > > > 1. HADOOP-17112 <
> > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > >> >> > > > > >
> > > >> >> > > > > > 2.
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > > >> >> > > > > >
> > > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> > build
> > > >> >> > process.
> > > >> >> > > > > That's
> > > >> >> > > > > > going to be important for future releases.
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >>
> > ---------------------------------------------------------------------
> > > >> >> > > > > To unsubscribe, e-mail:
> > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > >> >> > > > > For additional commands, e-mail:
> > > >> hdfs-dev-help@hadoop.apache.org
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >> >>
> > > >> >> --
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >> --Brahma Reddy Battula
> > > >> >>
> > > >> >
> > > >>
> > > >> --
> > > >>
> > > >>
> > > >>
> > > >> --Brahma Reddy Battula
> > > >>
> > > >
> > >
> >
>
--
-Vinay
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Vinayakumar B <vi...@apache.org>.
Hi David,
>> Still hoping for help here:
>> https://issues.apache.org/jira/browse/HDFS-15790
I will raise a PR for the said solution soon (in a day or two).
-Vinay
On Thu, 11 Mar 2021 at 5:39 AM, David <da...@gmail.com> wrote:
> Hello,
>
> Still hoping for help here:
>
> https://issues.apache.org/jira/browse/HDFS-15790
>
> Looks like it has been worked on, not sure how to best move it forward.
>
> On Wed, Mar 10, 2021, 12:21 PM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > I'm going to argue its too late to do IPv6 support close to a release, as
> > it's best if its on developer machines for some time to let all quirks
> > surface. It's not so much IPv6 itself, but do we cause any regressions on
> > IPv4?
> >
> > But: it can/should go into trunk and stabilize there
> >
> > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> > muralikrishna.dmmkr@gmail.com> wrote:
> >
> > > Hi Brahma,
> > >
> > > I have missed out mentioning about the IPV6 feature in the last mail,
> > > Support for IPV6 has been in development since 2015 and We have done a
> > good
> > > amount of testing at our organisation, the feature is stable and used
> by
> > > our customers extensively in the last one year. I think it is a good
> time
> > > to add the IPV6 support to 3.3.1.
> > >
> > > https://issues.apache.org/jira/browse/HADOOP-11890
> > >
> > > Thanks
> > > D M Murali Krishna Reddy
> > >
> > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > > muralikrishna.dmmkr@gmail.com> wrote:
> > >
> > > > Hi Brahma,
> > > >
> > > > Can we have this new feature "YARN Registry based AM discovery with
> > retry
> > > > and in-flight task persistent via JHS" in the upcoming 3.1.1
> release. I
> > > > have also attached a test-report in the below jira.
> > > >
> > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > > >
> > > >
> > > > Thanks,
> > > > D M Murali Krishna Reddy
> > > >
> > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> > brahma@apache.org
> > > >
> > > > wrote:
> > > >
> > > >> Hi Bilwa,
> > > >>
> > > >> I have commented on the jira's you mentioned. Based on the stability
> > we
> > > >> can
> > > >> plan this.But needs to be merged ASAP.
> > > >>
> > > >>
> > > >>
> > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> > > >>
> > > >> > Hi Brahma,
> > > >> >
> > > >> > Can we have below features in 3.3.1 release? We have been using
> > these
> > > >> > features for a long time. They are stable and tested in bigger
> > > clusters.
> > > >> >
> > > >> > 1. Container reuse -
> > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > > >> > 2. Speculative attempts should not run on the same node -
> > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > > >> >
> > > >> > Thanks,
> > > >> > Bilwa
> > > >> >
> > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> > brahma@apache.org
> > > >
> > > >> > wrote:
> > > >> >
> > > >> >> Sorry for the late reply..
> > > >> >>
> > > >> >> I will come up with a plan.. Please let me know if anybody has
> some
> > > >> >> features/improvements/bugs that need to be included.
> > > >> >>
> > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <
> sunilg@apache.org>
> > > >> wrote:
> > > >> >>
> > > >> >> > Hi Wei-Chiu,
> > > >> >> >
> > > >> >> > What will be the next steps here for 3.3.1 planning?
> > > >> >> >
> > > >> >> > Thanks
> > > >> >> > Sunil
> > > >> >> >
> > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net>
> wrote:
> > > >> >> >
> > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > > >> >> > <stevel@cloudera.com.invalid
> > > >> >> > > >
> > > >> >> > > wrote:
> > > >> >> > >
> > > >> >> > > >
> > > >> >> > > > Regarding blockers &c: how about we have a little hackathon
> > > >> where we
> > > >> >> > try
> > > >> >> > > > and get things in. This means a promise of review time from
> > the
> > > >> >> people
> > > >> >> > > with
> > > >> >> > > > commit rights and other people who understand the code
> > (Stack?)
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> > testing).
> > > >> >> > > Thanks,
> > > >> >> > > S
> > > >> >> > >
> > > >> >> > >
> > > >> >> > >
> > > >> >> > >
> > > >> >> > > > -steve
> > > >> >> > > >
> > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> > ayushtkn@gmail.com
> > > >
> > > >> >> wrote:
> > > >> >> > > >
> > > >> >> > > > > +1
> > > >> >> > > > > Just to mention we would need to release
> hadoop-thirdparty
> > > too
> > > >> >> > before.
> > > >> >> > > > > Presently we are using the snapshot version of it.
> > > >> >> > > > >
> > > >> >> > > > > -Ayush
> > > >> >> > > > >
> > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > > >> weichiu@apache.org
> > > >> >> >
> > > >> >> > > > wrote:
> > > >> >> > > > > >
> > > >> >> > > > > > Hi all,
> > > >> >> > > > > >
> > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of
> now
> > > >> we've
> > > >> >> > > > > accumulated
> > > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > > >> >> downstreamers
> > > >> >> > > are
> > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> > version
> > > >> >> > conflict
> > > >> >> > > > > issue.
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > >> >> > > > > >
> > > >> >> > > > > > We should start the release work for 3.3.1 before the
> > diff
> > > >> >> becomes
> > > >> >> > > even
> > > >> >> > > > > > larger.
> > > >> >> > > > > >
> > > >> >> > > > > > I believe there are currently only two real blockers
> > for a
> > > >> >> 3.3.1
> > > >> >> > > > (using
> > > >> >> > > > > > this filter
> > > >> >> > > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > >> >> > > > > > )
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > > > 1. HDFS-15566 <
> > > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > > >> >> > > > > > 2.
> > > >> >> > > > > > 1. HADOOP-17112 <
> > > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > >> >> > > > > >
> > > >> >> > > > > > 2.
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > > >
> > > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > > >> >> > > > > >
> > > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> > build
> > > >> >> > process.
> > > >> >> > > > > That's
> > > >> >> > > > > > going to be important for future releases.
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >>
> > ---------------------------------------------------------------------
> > > >> >> > > > > To unsubscribe, e-mail:
> > > hdfs-dev-unsubscribe@hadoop.apache.org
> > > >> >> > > > > For additional commands, e-mail:
> > > >> hdfs-dev-help@hadoop.apache.org
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >> >>
> > > >> >> --
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >> --Brahma Reddy Battula
> > > >> >>
> > > >> >
> > > >>
> > > >> --
> > > >>
> > > >>
> > > >>
> > > >> --Brahma Reddy Battula
> > > >>
> > > >
> > >
> >
>
--
-Vinay
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by David <da...@gmail.com>.
Hello,
Still hoping for help here:
https://issues.apache.org/jira/browse/HDFS-15790
Looks like it has been worked on, not sure how to best move it forward.
On Wed, Mar 10, 2021, 12:21 PM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> I'm going to argue its too late to do IPv6 support close to a release, as
> it's best if its on developer machines for some time to let all quirks
> surface. It's not so much IPv6 itself, but do we cause any regressions on
> IPv4?
>
> But: it can/should go into trunk and stabilize there
>
> On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> muralikrishna.dmmkr@gmail.com> wrote:
>
> > Hi Brahma,
> >
> > I have missed out mentioning about the IPV6 feature in the last mail,
> > Support for IPV6 has been in development since 2015 and We have done a
> good
> > amount of testing at our organisation, the feature is stable and used by
> > our customers extensively in the last one year. I think it is a good time
> > to add the IPV6 support to 3.3.1.
> >
> > https://issues.apache.org/jira/browse/HADOOP-11890
> >
> > Thanks
> > D M Murali Krishna Reddy
> >
> > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > muralikrishna.dmmkr@gmail.com> wrote:
> >
> > > Hi Brahma,
> > >
> > > Can we have this new feature "YARN Registry based AM discovery with
> retry
> > > and in-flight task persistent via JHS" in the upcoming 3.1.1 release. I
> > > have also attached a test-report in the below jira.
> > >
> > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > >
> > >
> > > Thanks,
> > > D M Murali Krishna Reddy
> > >
> > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > > wrote:
> > >
> > >> Hi Bilwa,
> > >>
> > >> I have commented on the jira's you mentioned. Based on the stability
> we
> > >> can
> > >> plan this.But needs to be merged ASAP.
> > >>
> > >>
> > >>
> > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> > >>
> > >> > Hi Brahma,
> > >> >
> > >> > Can we have below features in 3.3.1 release? We have been using
> these
> > >> > features for a long time. They are stable and tested in bigger
> > clusters.
> > >> >
> > >> > 1. Container reuse -
> > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > >> > 2. Speculative attempts should not run on the same node -
> > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > >> >
> > >> > Thanks,
> > >> > Bilwa
> > >> >
> > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > >> > wrote:
> > >> >
> > >> >> Sorry for the late reply..
> > >> >>
> > >> >> I will come up with a plan.. Please let me know if anybody has some
> > >> >> features/improvements/bugs that need to be included.
> > >> >>
> > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org>
> > >> wrote:
> > >> >>
> > >> >> > Hi Wei-Chiu,
> > >> >> >
> > >> >> > What will be the next steps here for 3.3.1 planning?
> > >> >> >
> > >> >> > Thanks
> > >> >> > Sunil
> > >> >> >
> > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> > >> >> >
> > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > >> >> > <stevel@cloudera.com.invalid
> > >> >> > > >
> > >> >> > > wrote:
> > >> >> > >
> > >> >> > > >
> > >> >> > > > Regarding blockers &c: how about we have a little hackathon
> > >> where we
> > >> >> > try
> > >> >> > > > and get things in. This means a promise of review time from
> the
> > >> >> people
> > >> >> > > with
> > >> >> > > > commit rights and other people who understand the code
> (Stack?)
> > >> >> > > >
> > >> >> > > >
> > >> >> > >
> > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> testing).
> > >> >> > > Thanks,
> > >> >> > > S
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > > > -steve
> > >> >> > > >
> > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> ayushtkn@gmail.com
> > >
> > >> >> wrote:
> > >> >> > > >
> > >> >> > > > > +1
> > >> >> > > > > Just to mention we would need to release hadoop-thirdparty
> > too
> > >> >> > before.
> > >> >> > > > > Presently we are using the snapshot version of it.
> > >> >> > > > >
> > >> >> > > > > -Ayush
> > >> >> > > > >
> > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > >> weichiu@apache.org
> > >> >> >
> > >> >> > > > wrote:
> > >> >> > > > > >
> > >> >> > > > > > Hi all,
> > >> >> > > > > >
> > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now
> > >> we've
> > >> >> > > > > accumulated
> > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > >> >> downstreamers
> > >> >> > > are
> > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> version
> > >> >> > conflict
> > >> >> > > > > issue.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > >> >> > > > > >
> > >> >> > > > > > We should start the release work for 3.3.1 before the
> diff
> > >> >> becomes
> > >> >> > > even
> > >> >> > > > > > larger.
> > >> >> > > > > >
> > >> >> > > > > > I believe there are currently only two real blockers
> for a
> > >> >> 3.3.1
> > >> >> > > > (using
> > >> >> > > > > > this filter
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > >> >> > > > > > )
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > 1. HDFS-15566 <
> > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > >> >> > > > > > 2.
> > >> >> > > > > > 1. HADOOP-17112 <
> > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > >> >> > > > > >
> > >> >> > > > > > 2.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > >> >> > > > > >
> > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> build
> > >> >> > process.
> > >> >> > > > > That's
> > >> >> > > > > > going to be important for future releases.
> > >> >> > > > >
> > >> >> > > > >
> > >> >>
> ---------------------------------------------------------------------
> > >> >> > > > > To unsubscribe, e-mail:
> > hdfs-dev-unsubscribe@hadoop.apache.org
> > >> >> > > > > For additional commands, e-mail:
> > >> hdfs-dev-help@hadoop.apache.org
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >>
> > >> >> --
> > >> >>
> > >> >>
> > >> >>
> > >> >> --Brahma Reddy Battula
> > >> >>
> > >> >
> > >>
> > >> --
> > >>
> > >>
> > >>
> > >> --Brahma Reddy Battula
> > >>
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by David <da...@gmail.com>.
Hello,
Still hoping for help here:
https://issues.apache.org/jira/browse/HDFS-15790
Looks like it has been worked on, not sure how to best move it forward.
On Wed, Mar 10, 2021, 12:21 PM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> I'm going to argue its too late to do IPv6 support close to a release, as
> it's best if its on developer machines for some time to let all quirks
> surface. It's not so much IPv6 itself, but do we cause any regressions on
> IPv4?
>
> But: it can/should go into trunk and stabilize there
>
> On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> muralikrishna.dmmkr@gmail.com> wrote:
>
> > Hi Brahma,
> >
> > I have missed out mentioning about the IPV6 feature in the last mail,
> > Support for IPV6 has been in development since 2015 and We have done a
> good
> > amount of testing at our organisation, the feature is stable and used by
> > our customers extensively in the last one year. I think it is a good time
> > to add the IPV6 support to 3.3.1.
> >
> > https://issues.apache.org/jira/browse/HADOOP-11890
> >
> > Thanks
> > D M Murali Krishna Reddy
> >
> > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > muralikrishna.dmmkr@gmail.com> wrote:
> >
> > > Hi Brahma,
> > >
> > > Can we have this new feature "YARN Registry based AM discovery with
> retry
> > > and in-flight task persistent via JHS" in the upcoming 3.1.1 release. I
> > > have also attached a test-report in the below jira.
> > >
> > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > >
> > >
> > > Thanks,
> > > D M Murali Krishna Reddy
> > >
> > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > > wrote:
> > >
> > >> Hi Bilwa,
> > >>
> > >> I have commented on the jira's you mentioned. Based on the stability
> we
> > >> can
> > >> plan this.But needs to be merged ASAP.
> > >>
> > >>
> > >>
> > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> > >>
> > >> > Hi Brahma,
> > >> >
> > >> > Can we have below features in 3.3.1 release? We have been using
> these
> > >> > features for a long time. They are stable and tested in bigger
> > clusters.
> > >> >
> > >> > 1. Container reuse -
> > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > >> > 2. Speculative attempts should not run on the same node -
> > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > >> >
> > >> > Thanks,
> > >> > Bilwa
> > >> >
> > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > >> > wrote:
> > >> >
> > >> >> Sorry for the late reply..
> > >> >>
> > >> >> I will come up with a plan.. Please let me know if anybody has some
> > >> >> features/improvements/bugs that need to be included.
> > >> >>
> > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org>
> > >> wrote:
> > >> >>
> > >> >> > Hi Wei-Chiu,
> > >> >> >
> > >> >> > What will be the next steps here for 3.3.1 planning?
> > >> >> >
> > >> >> > Thanks
> > >> >> > Sunil
> > >> >> >
> > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> > >> >> >
> > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > >> >> > <stevel@cloudera.com.invalid
> > >> >> > > >
> > >> >> > > wrote:
> > >> >> > >
> > >> >> > > >
> > >> >> > > > Regarding blockers &c: how about we have a little hackathon
> > >> where we
> > >> >> > try
> > >> >> > > > and get things in. This means a promise of review time from
> the
> > >> >> people
> > >> >> > > with
> > >> >> > > > commit rights and other people who understand the code
> (Stack?)
> > >> >> > > >
> > >> >> > > >
> > >> >> > >
> > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> testing).
> > >> >> > > Thanks,
> > >> >> > > S
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > > > -steve
> > >> >> > > >
> > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> ayushtkn@gmail.com
> > >
> > >> >> wrote:
> > >> >> > > >
> > >> >> > > > > +1
> > >> >> > > > > Just to mention we would need to release hadoop-thirdparty
> > too
> > >> >> > before.
> > >> >> > > > > Presently we are using the snapshot version of it.
> > >> >> > > > >
> > >> >> > > > > -Ayush
> > >> >> > > > >
> > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > >> weichiu@apache.org
> > >> >> >
> > >> >> > > > wrote:
> > >> >> > > > > >
> > >> >> > > > > > Hi all,
> > >> >> > > > > >
> > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now
> > >> we've
> > >> >> > > > > accumulated
> > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > >> >> downstreamers
> > >> >> > > are
> > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> version
> > >> >> > conflict
> > >> >> > > > > issue.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > >> >> > > > > >
> > >> >> > > > > > We should start the release work for 3.3.1 before the
> diff
> > >> >> becomes
> > >> >> > > even
> > >> >> > > > > > larger.
> > >> >> > > > > >
> > >> >> > > > > > I believe there are currently only two real blockers
> for a
> > >> >> 3.3.1
> > >> >> > > > (using
> > >> >> > > > > > this filter
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > >> >> > > > > > )
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > 1. HDFS-15566 <
> > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > >> >> > > > > > 2.
> > >> >> > > > > > 1. HADOOP-17112 <
> > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > >> >> > > > > >
> > >> >> > > > > > 2.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > >> >> > > > > >
> > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> build
> > >> >> > process.
> > >> >> > > > > That's
> > >> >> > > > > > going to be important for future releases.
> > >> >> > > > >
> > >> >> > > > >
> > >> >>
> ---------------------------------------------------------------------
> > >> >> > > > > To unsubscribe, e-mail:
> > hdfs-dev-unsubscribe@hadoop.apache.org
> > >> >> > > > > For additional commands, e-mail:
> > >> hdfs-dev-help@hadoop.apache.org
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >>
> > >> >> --
> > >> >>
> > >> >>
> > >> >>
> > >> >> --Brahma Reddy Battula
> > >> >>
> > >> >
> > >>
> > >> --
> > >>
> > >>
> > >>
> > >> --Brahma Reddy Battula
> > >>
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by David <da...@gmail.com>.
Hello,
Still hoping for help here:
https://issues.apache.org/jira/browse/HDFS-15790
Looks like it has been worked on, not sure how to best move it forward.
On Wed, Mar 10, 2021, 12:21 PM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> I'm going to argue its too late to do IPv6 support close to a release, as
> it's best if its on developer machines for some time to let all quirks
> surface. It's not so much IPv6 itself, but do we cause any regressions on
> IPv4?
>
> But: it can/should go into trunk and stabilize there
>
> On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> muralikrishna.dmmkr@gmail.com> wrote:
>
> > Hi Brahma,
> >
> > I have missed out mentioning about the IPV6 feature in the last mail,
> > Support for IPV6 has been in development since 2015 and We have done a
> good
> > amount of testing at our organisation, the feature is stable and used by
> > our customers extensively in the last one year. I think it is a good time
> > to add the IPV6 support to 3.3.1.
> >
> > https://issues.apache.org/jira/browse/HADOOP-11890
> >
> > Thanks
> > D M Murali Krishna Reddy
> >
> > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > muralikrishna.dmmkr@gmail.com> wrote:
> >
> > > Hi Brahma,
> > >
> > > Can we have this new feature "YARN Registry based AM discovery with
> retry
> > > and in-flight task persistent via JHS" in the upcoming 3.1.1 release. I
> > > have also attached a test-report in the below jira.
> > >
> > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > >
> > >
> > > Thanks,
> > > D M Murali Krishna Reddy
> > >
> > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > > wrote:
> > >
> > >> Hi Bilwa,
> > >>
> > >> I have commented on the jira's you mentioned. Based on the stability
> we
> > >> can
> > >> plan this.But needs to be merged ASAP.
> > >>
> > >>
> > >>
> > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> > >>
> > >> > Hi Brahma,
> > >> >
> > >> > Can we have below features in 3.3.1 release? We have been using
> these
> > >> > features for a long time. They are stable and tested in bigger
> > clusters.
> > >> >
> > >> > 1. Container reuse -
> > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > >> > 2. Speculative attempts should not run on the same node -
> > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > >> >
> > >> > Thanks,
> > >> > Bilwa
> > >> >
> > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > >> > wrote:
> > >> >
> > >> >> Sorry for the late reply..
> > >> >>
> > >> >> I will come up with a plan.. Please let me know if anybody has some
> > >> >> features/improvements/bugs that need to be included.
> > >> >>
> > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org>
> > >> wrote:
> > >> >>
> > >> >> > Hi Wei-Chiu,
> > >> >> >
> > >> >> > What will be the next steps here for 3.3.1 planning?
> > >> >> >
> > >> >> > Thanks
> > >> >> > Sunil
> > >> >> >
> > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> > >> >> >
> > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > >> >> > <stevel@cloudera.com.invalid
> > >> >> > > >
> > >> >> > > wrote:
> > >> >> > >
> > >> >> > > >
> > >> >> > > > Regarding blockers &c: how about we have a little hackathon
> > >> where we
> > >> >> > try
> > >> >> > > > and get things in. This means a promise of review time from
> the
> > >> >> people
> > >> >> > > with
> > >> >> > > > commit rights and other people who understand the code
> (Stack?)
> > >> >> > > >
> > >> >> > > >
> > >> >> > >
> > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> testing).
> > >> >> > > Thanks,
> > >> >> > > S
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > > > -steve
> > >> >> > > >
> > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> ayushtkn@gmail.com
> > >
> > >> >> wrote:
> > >> >> > > >
> > >> >> > > > > +1
> > >> >> > > > > Just to mention we would need to release hadoop-thirdparty
> > too
> > >> >> > before.
> > >> >> > > > > Presently we are using the snapshot version of it.
> > >> >> > > > >
> > >> >> > > > > -Ayush
> > >> >> > > > >
> > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > >> weichiu@apache.org
> > >> >> >
> > >> >> > > > wrote:
> > >> >> > > > > >
> > >> >> > > > > > Hi all,
> > >> >> > > > > >
> > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now
> > >> we've
> > >> >> > > > > accumulated
> > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > >> >> downstreamers
> > >> >> > > are
> > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> version
> > >> >> > conflict
> > >> >> > > > > issue.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > >> >> > > > > >
> > >> >> > > > > > We should start the release work for 3.3.1 before the
> diff
> > >> >> becomes
> > >> >> > > even
> > >> >> > > > > > larger.
> > >> >> > > > > >
> > >> >> > > > > > I believe there are currently only two real blockers
> for a
> > >> >> 3.3.1
> > >> >> > > > (using
> > >> >> > > > > > this filter
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > >> >> > > > > > )
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > 1. HDFS-15566 <
> > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > >> >> > > > > > 2.
> > >> >> > > > > > 1. HADOOP-17112 <
> > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > >> >> > > > > >
> > >> >> > > > > > 2.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > >> >> > > > > >
> > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> build
> > >> >> > process.
> > >> >> > > > > That's
> > >> >> > > > > > going to be important for future releases.
> > >> >> > > > >
> > >> >> > > > >
> > >> >>
> ---------------------------------------------------------------------
> > >> >> > > > > To unsubscribe, e-mail:
> > hdfs-dev-unsubscribe@hadoop.apache.org
> > >> >> > > > > For additional commands, e-mail:
> > >> hdfs-dev-help@hadoop.apache.org
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >>
> > >> >> --
> > >> >>
> > >> >>
> > >> >>
> > >> >> --Brahma Reddy Battula
> > >> >>
> > >> >
> > >>
> > >> --
> > >>
> > >>
> > >>
> > >> --Brahma Reddy Battula
> > >>
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by David <da...@gmail.com>.
Hello,
Still hoping for help here:
https://issues.apache.org/jira/browse/HDFS-15790
Looks like it has been worked on, not sure how to best move it forward.
On Wed, Mar 10, 2021, 12:21 PM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> I'm going to argue its too late to do IPv6 support close to a release, as
> it's best if its on developer machines for some time to let all quirks
> surface. It's not so much IPv6 itself, but do we cause any regressions on
> IPv4?
>
> But: it can/should go into trunk and stabilize there
>
> On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> muralikrishna.dmmkr@gmail.com> wrote:
>
> > Hi Brahma,
> >
> > I have missed out mentioning about the IPV6 feature in the last mail,
> > Support for IPV6 has been in development since 2015 and We have done a
> good
> > amount of testing at our organisation, the feature is stable and used by
> > our customers extensively in the last one year. I think it is a good time
> > to add the IPV6 support to 3.3.1.
> >
> > https://issues.apache.org/jira/browse/HADOOP-11890
> >
> > Thanks
> > D M Murali Krishna Reddy
> >
> > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > muralikrishna.dmmkr@gmail.com> wrote:
> >
> > > Hi Brahma,
> > >
> > > Can we have this new feature "YARN Registry based AM discovery with
> retry
> > > and in-flight task persistent via JHS" in the upcoming 3.1.1 release. I
> > > have also attached a test-report in the below jira.
> > >
> > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > >
> > >
> > > Thanks,
> > > D M Murali Krishna Reddy
> > >
> > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > > wrote:
> > >
> > >> Hi Bilwa,
> > >>
> > >> I have commented on the jira's you mentioned. Based on the stability
> we
> > >> can
> > >> plan this.But needs to be merged ASAP.
> > >>
> > >>
> > >>
> > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> > >>
> > >> > Hi Brahma,
> > >> >
> > >> > Can we have below features in 3.3.1 release? We have been using
> these
> > >> > features for a long time. They are stable and tested in bigger
> > clusters.
> > >> >
> > >> > 1. Container reuse -
> > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > >> > 2. Speculative attempts should not run on the same node -
> > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > >> >
> > >> > Thanks,
> > >> > Bilwa
> > >> >
> > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > >> > wrote:
> > >> >
> > >> >> Sorry for the late reply..
> > >> >>
> > >> >> I will come up with a plan.. Please let me know if anybody has some
> > >> >> features/improvements/bugs that need to be included.
> > >> >>
> > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org>
> > >> wrote:
> > >> >>
> > >> >> > Hi Wei-Chiu,
> > >> >> >
> > >> >> > What will be the next steps here for 3.3.1 planning?
> > >> >> >
> > >> >> > Thanks
> > >> >> > Sunil
> > >> >> >
> > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> > >> >> >
> > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > >> >> > <stevel@cloudera.com.invalid
> > >> >> > > >
> > >> >> > > wrote:
> > >> >> > >
> > >> >> > > >
> > >> >> > > > Regarding blockers &c: how about we have a little hackathon
> > >> where we
> > >> >> > try
> > >> >> > > > and get things in. This means a promise of review time from
> the
> > >> >> people
> > >> >> > > with
> > >> >> > > > commit rights and other people who understand the code
> (Stack?)
> > >> >> > > >
> > >> >> > > >
> > >> >> > >
> > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> testing).
> > >> >> > > Thanks,
> > >> >> > > S
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > > > -steve
> > >> >> > > >
> > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> ayushtkn@gmail.com
> > >
> > >> >> wrote:
> > >> >> > > >
> > >> >> > > > > +1
> > >> >> > > > > Just to mention we would need to release hadoop-thirdparty
> > too
> > >> >> > before.
> > >> >> > > > > Presently we are using the snapshot version of it.
> > >> >> > > > >
> > >> >> > > > > -Ayush
> > >> >> > > > >
> > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > >> weichiu@apache.org
> > >> >> >
> > >> >> > > > wrote:
> > >> >> > > > > >
> > >> >> > > > > > Hi all,
> > >> >> > > > > >
> > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now
> > >> we've
> > >> >> > > > > accumulated
> > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > >> >> downstreamers
> > >> >> > > are
> > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> version
> > >> >> > conflict
> > >> >> > > > > issue.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > >> >> > > > > >
> > >> >> > > > > > We should start the release work for 3.3.1 before the
> diff
> > >> >> becomes
> > >> >> > > even
> > >> >> > > > > > larger.
> > >> >> > > > > >
> > >> >> > > > > > I believe there are currently only two real blockers
> for a
> > >> >> 3.3.1
> > >> >> > > > (using
> > >> >> > > > > > this filter
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > >> >> > > > > > )
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > 1. HDFS-15566 <
> > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > >> >> > > > > > 2.
> > >> >> > > > > > 1. HADOOP-17112 <
> > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > >> >> > > > > >
> > >> >> > > > > > 2.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > >> >> > > > > >
> > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> build
> > >> >> > process.
> > >> >> > > > > That's
> > >> >> > > > > > going to be important for future releases.
> > >> >> > > > >
> > >> >> > > > >
> > >> >>
> ---------------------------------------------------------------------
> > >> >> > > > > To unsubscribe, e-mail:
> > hdfs-dev-unsubscribe@hadoop.apache.org
> > >> >> > > > > For additional commands, e-mail:
> > >> hdfs-dev-help@hadoop.apache.org
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >>
> > >> >> --
> > >> >>
> > >> >>
> > >> >>
> > >> >> --Brahma Reddy Battula
> > >> >>
> > >> >
> > >>
> > >> --
> > >>
> > >>
> > >>
> > >> --Brahma Reddy Battula
> > >>
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Without looking into details, I would say that new features like IPv6
support should get into 3.4.0 (and hopefully spend a little time to
stabilize) before backporting to 3.3.x or lower branches.
On Thu, Mar 11, 2021 at 1:21 AM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> I'm going to argue its too late to do IPv6 support close to a release, as
> it's best if its on developer machines for some time to let all quirks
> surface. It's not so much IPv6 itself, but do we cause any regressions on
> IPv4?
>
> But: it can/should go into trunk and stabilize there
>
> On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> muralikrishna.dmmkr@gmail.com> wrote:
>
> > Hi Brahma,
> >
> > I have missed out mentioning about the IPV6 feature in the last mail,
> > Support for IPV6 has been in development since 2015 and We have done a
> good
> > amount of testing at our organisation, the feature is stable and used by
> > our customers extensively in the last one year. I think it is a good time
> > to add the IPV6 support to 3.3.1.
> >
> > https://issues.apache.org/jira/browse/HADOOP-11890
> >
> > Thanks
> > D M Murali Krishna Reddy
> >
> > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > muralikrishna.dmmkr@gmail.com> wrote:
> >
> > > Hi Brahma,
> > >
> > > Can we have this new feature "YARN Registry based AM discovery with
> retry
> > > and in-flight task persistent via JHS" in the upcoming 3.1.1 release. I
> > > have also attached a test-report in the below jira.
> > >
> > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > >
> > >
> > > Thanks,
> > > D M Murali Krishna Reddy
> > >
> > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > > wrote:
> > >
> > >> Hi Bilwa,
> > >>
> > >> I have commented on the jira's you mentioned. Based on the stability
> we
> > >> can
> > >> plan this.But needs to be merged ASAP.
> > >>
> > >>
> > >>
> > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> > >>
> > >> > Hi Brahma,
> > >> >
> > >> > Can we have below features in 3.3.1 release? We have been using
> these
> > >> > features for a long time. They are stable and tested in bigger
> > clusters.
> > >> >
> > >> > 1. Container reuse -
> > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > >> > 2. Speculative attempts should not run on the same node -
> > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > >> >
> > >> > Thanks,
> > >> > Bilwa
> > >> >
> > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > >> > wrote:
> > >> >
> > >> >> Sorry for the late reply..
> > >> >>
> > >> >> I will come up with a plan.. Please let me know if anybody has some
> > >> >> features/improvements/bugs that need to be included.
> > >> >>
> > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org>
> > >> wrote:
> > >> >>
> > >> >> > Hi Wei-Chiu,
> > >> >> >
> > >> >> > What will be the next steps here for 3.3.1 planning?
> > >> >> >
> > >> >> > Thanks
> > >> >> > Sunil
> > >> >> >
> > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> > >> >> >
> > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > >> >> > <stevel@cloudera.com.invalid
> > >> >> > > >
> > >> >> > > wrote:
> > >> >> > >
> > >> >> > > >
> > >> >> > > > Regarding blockers &c: how about we have a little hackathon
> > >> where we
> > >> >> > try
> > >> >> > > > and get things in. This means a promise of review time from
> the
> > >> >> people
> > >> >> > > with
> > >> >> > > > commit rights and other people who understand the code
> (Stack?)
> > >> >> > > >
> > >> >> > > >
> > >> >> > >
> > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> testing).
> > >> >> > > Thanks,
> > >> >> > > S
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > > > -steve
> > >> >> > > >
> > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> ayushtkn@gmail.com
> > >
> > >> >> wrote:
> > >> >> > > >
> > >> >> > > > > +1
> > >> >> > > > > Just to mention we would need to release hadoop-thirdparty
> > too
> > >> >> > before.
> > >> >> > > > > Presently we are using the snapshot version of it.
> > >> >> > > > >
> > >> >> > > > > -Ayush
> > >> >> > > > >
> > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > >> weichiu@apache.org
> > >> >> >
> > >> >> > > > wrote:
> > >> >> > > > > >
> > >> >> > > > > > Hi all,
> > >> >> > > > > >
> > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now
> > >> we've
> > >> >> > > > > accumulated
> > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > >> >> downstreamers
> > >> >> > > are
> > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> version
> > >> >> > conflict
> > >> >> > > > > issue.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > >> >> > > > > >
> > >> >> > > > > > We should start the release work for 3.3.1 before the
> diff
> > >> >> becomes
> > >> >> > > even
> > >> >> > > > > > larger.
> > >> >> > > > > >
> > >> >> > > > > > I believe there are currently only two real blockers
> for a
> > >> >> 3.3.1
> > >> >> > > > (using
> > >> >> > > > > > this filter
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > >> >> > > > > > )
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > 1. HDFS-15566 <
> > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > >> >> > > > > > 2.
> > >> >> > > > > > 1. HADOOP-17112 <
> > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > >> >> > > > > >
> > >> >> > > > > > 2.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > >> >> > > > > >
> > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> build
> > >> >> > process.
> > >> >> > > > > That's
> > >> >> > > > > > going to be important for future releases.
> > >> >> > > > >
> > >> >> > > > >
> > >> >>
> ---------------------------------------------------------------------
> > >> >> > > > > To unsubscribe, e-mail:
> > hdfs-dev-unsubscribe@hadoop.apache.org
> > >> >> > > > > For additional commands, e-mail:
> > >> hdfs-dev-help@hadoop.apache.org
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >>
> > >> >> --
> > >> >>
> > >> >>
> > >> >>
> > >> >> --Brahma Reddy Battula
> > >> >>
> > >> >
> > >>
> > >> --
> > >>
> > >>
> > >>
> > >> --Brahma Reddy Battula
> > >>
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Wei-Chiu Chuang <we...@cloudera.com.INVALID>.
Without looking into details, I would say that new features like IPv6
support should get into 3.4.0 (and hopefully spend a little time to
stabilize) before backporting to 3.3.x or lower branches.
On Thu, Mar 11, 2021 at 1:21 AM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> I'm going to argue its too late to do IPv6 support close to a release, as
> it's best if its on developer machines for some time to let all quirks
> surface. It's not so much IPv6 itself, but do we cause any regressions on
> IPv4?
>
> But: it can/should go into trunk and stabilize there
>
> On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
> muralikrishna.dmmkr@gmail.com> wrote:
>
> > Hi Brahma,
> >
> > I have missed out mentioning about the IPV6 feature in the last mail,
> > Support for IPV6 has been in development since 2015 and We have done a
> good
> > amount of testing at our organisation, the feature is stable and used by
> > our customers extensively in the last one year. I think it is a good time
> > to add the IPV6 support to 3.3.1.
> >
> > https://issues.apache.org/jira/browse/HADOOP-11890
> >
> > Thanks
> > D M Murali Krishna Reddy
> >
> > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> > muralikrishna.dmmkr@gmail.com> wrote:
> >
> > > Hi Brahma,
> > >
> > > Can we have this new feature "YARN Registry based AM discovery with
> retry
> > > and in-flight task persistent via JHS" in the upcoming 3.1.1 release. I
> > > have also attached a test-report in the below jira.
> > >
> > > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> > >
> > >
> > > Thanks,
> > > D M Murali Krishna Reddy
> > >
> > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > > wrote:
> > >
> > >> Hi Bilwa,
> > >>
> > >> I have commented on the jira's you mentioned. Based on the stability
> we
> > >> can
> > >> plan this.But needs to be merged ASAP.
> > >>
> > >>
> > >>
> > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> > >>
> > >> > Hi Brahma,
> > >> >
> > >> > Can we have below features in 3.3.1 release? We have been using
> these
> > >> > features for a long time. They are stable and tested in bigger
> > clusters.
> > >> >
> > >> > 1. Container reuse -
> > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > >> > 2. Speculative attempts should not run on the same node -
> > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> > >> >
> > >> > Thanks,
> > >> > Bilwa
> > >> >
> > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <
> brahma@apache.org
> > >
> > >> > wrote:
> > >> >
> > >> >> Sorry for the late reply..
> > >> >>
> > >> >> I will come up with a plan.. Please let me know if anybody has some
> > >> >> features/improvements/bugs that need to be included.
> > >> >>
> > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org>
> > >> wrote:
> > >> >>
> > >> >> > Hi Wei-Chiu,
> > >> >> >
> > >> >> > What will be the next steps here for 3.3.1 planning?
> > >> >> >
> > >> >> > Thanks
> > >> >> > Sunil
> > >> >> >
> > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> > >> >> >
> > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > >> >> > <stevel@cloudera.com.invalid
> > >> >> > > >
> > >> >> > > wrote:
> > >> >> > >
> > >> >> > > >
> > >> >> > > > Regarding blockers &c: how about we have a little hackathon
> > >> where we
> > >> >> > try
> > >> >> > > > and get things in. This means a promise of review time from
> the
> > >> >> people
> > >> >> > > with
> > >> >> > > > commit rights and other people who understand the code
> (Stack?)
> > >> >> > > >
> > >> >> > > >
> > >> >> > >
> > >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon,
> testing).
> > >> >> > > Thanks,
> > >> >> > > S
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > > > -steve
> > >> >> > > >
> > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <
> ayushtkn@gmail.com
> > >
> > >> >> wrote:
> > >> >> > > >
> > >> >> > > > > +1
> > >> >> > > > > Just to mention we would need to release hadoop-thirdparty
> > too
> > >> >> > before.
> > >> >> > > > > Presently we are using the snapshot version of it.
> > >> >> > > > >
> > >> >> > > > > -Ayush
> > >> >> > > > >
> > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> > >> weichiu@apache.org
> > >> >> >
> > >> >> > > > wrote:
> > >> >> > > > > >
> > >> >> > > > > > Hi all,
> > >> >> > > > > >
> > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now
> > >> we've
> > >> >> > > > > accumulated
> > >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> > >> >> downstreamers
> > >> >> > > are
> > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava
> version
> > >> >> > conflict
> > >> >> > > > > issue.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > >> >> > > > > >
> > >> >> > > > > > We should start the release work for 3.3.1 before the
> diff
> > >> >> becomes
> > >> >> > > even
> > >> >> > > > > > larger.
> > >> >> > > > > >
> > >> >> > > > > > I believe there are currently only two real blockers
> for a
> > >> >> 3.3.1
> > >> >> > > > (using
> > >> >> > > > > > this filter
> > >> >> > > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > >> >> > > > > > )
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > 1. HDFS-15566 <
> > >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> > >> >> > > > > > 2.
> > >> >> > > > > > 1. HADOOP-17112 <
> > >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > >> >> > > > > >
> > >> >> > > > > > 2.
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > >
> > >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > >> >> > > > > >
> > >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM
> build
> > >> >> > process.
> > >> >> > > > > That's
> > >> >> > > > > > going to be important for future releases.
> > >> >> > > > >
> > >> >> > > > >
> > >> >>
> ---------------------------------------------------------------------
> > >> >> > > > > To unsubscribe, e-mail:
> > hdfs-dev-unsubscribe@hadoop.apache.org
> > >> >> > > > > For additional commands, e-mail:
> > >> hdfs-dev-help@hadoop.apache.org
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >>
> > >> >> --
> > >> >>
> > >> >>
> > >> >>
> > >> >> --Brahma Reddy Battula
> > >> >>
> > >> >
> > >>
> > >> --
> > >>
> > >>
> > >>
> > >> --Brahma Reddy Battula
> > >>
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
I'm going to argue its too late to do IPv6 support close to a release, as
it's best if its on developer machines for some time to let all quirks
surface. It's not so much IPv6 itself, but do we cause any regressions on
IPv4?
But: it can/should go into trunk and stabilize there
On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
muralikrishna.dmmkr@gmail.com> wrote:
> Hi Brahma,
>
> I have missed out mentioning about the IPV6 feature in the last mail,
> Support for IPV6 has been in development since 2015 and We have done a good
> amount of testing at our organisation, the feature is stable and used by
> our customers extensively in the last one year. I think it is a good time
> to add the IPV6 support to 3.3.1.
>
> https://issues.apache.org/jira/browse/HADOOP-11890
>
> Thanks
> D M Murali Krishna Reddy
>
> On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> muralikrishna.dmmkr@gmail.com> wrote:
>
> > Hi Brahma,
> >
> > Can we have this new feature "YARN Registry based AM discovery with retry
> > and in-flight task persistent via JHS" in the upcoming 3.1.1 release. I
> > have also attached a test-report in the below jira.
> >
> > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> >
> >
> > Thanks,
> > D M Murali Krishna Reddy
> >
> > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <brahma@apache.org
> >
> > wrote:
> >
> >> Hi Bilwa,
> >>
> >> I have commented on the jira's you mentioned. Based on the stability we
> >> can
> >> plan this.But needs to be merged ASAP.
> >>
> >>
> >>
> >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> >>
> >> > Hi Brahma,
> >> >
> >> > Can we have below features in 3.3.1 release? We have been using these
> >> > features for a long time. They are stable and tested in bigger
> clusters.
> >> >
> >> > 1. Container reuse -
> >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> >> > 2. Speculative attempts should not run on the same node -
> >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> >> >
> >> > Thanks,
> >> > Bilwa
> >> >
> >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <brahma@apache.org
> >
> >> > wrote:
> >> >
> >> >> Sorry for the late reply..
> >> >>
> >> >> I will come up with a plan.. Please let me know if anybody has some
> >> >> features/improvements/bugs that need to be included.
> >> >>
> >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org>
> >> wrote:
> >> >>
> >> >> > Hi Wei-Chiu,
> >> >> >
> >> >> > What will be the next steps here for 3.3.1 planning?
> >> >> >
> >> >> > Thanks
> >> >> > Sunil
> >> >> >
> >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> >> >> >
> >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> >> >> > <stevel@cloudera.com.invalid
> >> >> > > >
> >> >> > > wrote:
> >> >> > >
> >> >> > > >
> >> >> > > > Regarding blockers &c: how about we have a little hackathon
> >> where we
> >> >> > try
> >> >> > > > and get things in. This means a promise of review time from the
> >> >> people
> >> >> > > with
> >> >> > > > commit rights and other people who understand the code (Stack?)
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
> >> >> > > Thanks,
> >> >> > > S
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > > > -steve
> >> >> > > >
> >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ayushtkn@gmail.com
> >
> >> >> wrote:
> >> >> > > >
> >> >> > > > > +1
> >> >> > > > > Just to mention we would need to release hadoop-thirdparty
> too
> >> >> > before.
> >> >> > > > > Presently we are using the snapshot version of it.
> >> >> > > > >
> >> >> > > > > -Ayush
> >> >> > > > >
> >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> >> weichiu@apache.org
> >> >> >
> >> >> > > > wrote:
> >> >> > > > > >
> >> >> > > > > > Hi all,
> >> >> > > > > >
> >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now
> >> we've
> >> >> > > > > accumulated
> >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> >> >> downstreamers
> >> >> > > are
> >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava version
> >> >> > conflict
> >> >> > > > > issue.
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> >> >> > > > > >
> >> >> > > > > > We should start the release work for 3.3.1 before the diff
> >> >> becomes
> >> >> > > even
> >> >> > > > > > larger.
> >> >> > > > > >
> >> >> > > > > > I believe there are currently only two real blockers for a
> >> >> 3.3.1
> >> >> > > > (using
> >> >> > > > > > this filter
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> >> >> > > > > > )
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > 1. HDFS-15566 <
> >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> >> >> > > > > > 2.
> >> >> > > > > > 1. HADOOP-17112 <
> >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> >> >> > > > > >
> >> >> > > > > > 2.
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> >> >> > > > > >
> >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM build
> >> >> > process.
> >> >> > > > > That's
> >> >> > > > > > going to be important for future releases.
> >> >> > > > >
> >> >> > > > >
> >> >> ---------------------------------------------------------------------
> >> >> > > > > To unsubscribe, e-mail:
> hdfs-dev-unsubscribe@hadoop.apache.org
> >> >> > > > > For additional commands, e-mail:
> >> hdfs-dev-help@hadoop.apache.org
> >> >> > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >> >>
> >> >> --
> >> >>
> >> >>
> >> >>
> >> >> --Brahma Reddy Battula
> >> >>
> >> >
> >>
> >> --
> >>
> >>
> >>
> >> --Brahma Reddy Battula
> >>
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
I'm going to argue its too late to do IPv6 support close to a release, as
it's best if its on developer machines for some time to let all quirks
surface. It's not so much IPv6 itself, but do we cause any regressions on
IPv4?
But: it can/should go into trunk and stabilize there
On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
muralikrishna.dmmkr@gmail.com> wrote:
> Hi Brahma,
>
> I have missed out mentioning about the IPV6 feature in the last mail,
> Support for IPV6 has been in development since 2015 and We have done a good
> amount of testing at our organisation, the feature is stable and used by
> our customers extensively in the last one year. I think it is a good time
> to add the IPV6 support to 3.3.1.
>
> https://issues.apache.org/jira/browse/HADOOP-11890
>
> Thanks
> D M Murali Krishna Reddy
>
> On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> muralikrishna.dmmkr@gmail.com> wrote:
>
> > Hi Brahma,
> >
> > Can we have this new feature "YARN Registry based AM discovery with retry
> > and in-flight task persistent via JHS" in the upcoming 3.1.1 release. I
> > have also attached a test-report in the below jira.
> >
> > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> >
> >
> > Thanks,
> > D M Murali Krishna Reddy
> >
> > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <brahma@apache.org
> >
> > wrote:
> >
> >> Hi Bilwa,
> >>
> >> I have commented on the jira's you mentioned. Based on the stability we
> >> can
> >> plan this.But needs to be merged ASAP.
> >>
> >>
> >>
> >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> >>
> >> > Hi Brahma,
> >> >
> >> > Can we have below features in 3.3.1 release? We have been using these
> >> > features for a long time. They are stable and tested in bigger
> clusters.
> >> >
> >> > 1. Container reuse -
> >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> >> > 2. Speculative attempts should not run on the same node -
> >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> >> >
> >> > Thanks,
> >> > Bilwa
> >> >
> >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <brahma@apache.org
> >
> >> > wrote:
> >> >
> >> >> Sorry for the late reply..
> >> >>
> >> >> I will come up with a plan.. Please let me know if anybody has some
> >> >> features/improvements/bugs that need to be included.
> >> >>
> >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org>
> >> wrote:
> >> >>
> >> >> > Hi Wei-Chiu,
> >> >> >
> >> >> > What will be the next steps here for 3.3.1 planning?
> >> >> >
> >> >> > Thanks
> >> >> > Sunil
> >> >> >
> >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> >> >> >
> >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> >> >> > <stevel@cloudera.com.invalid
> >> >> > > >
> >> >> > > wrote:
> >> >> > >
> >> >> > > >
> >> >> > > > Regarding blockers &c: how about we have a little hackathon
> >> where we
> >> >> > try
> >> >> > > > and get things in. This means a promise of review time from the
> >> >> people
> >> >> > > with
> >> >> > > > commit rights and other people who understand the code (Stack?)
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
> >> >> > > Thanks,
> >> >> > > S
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > > > -steve
> >> >> > > >
> >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ayushtkn@gmail.com
> >
> >> >> wrote:
> >> >> > > >
> >> >> > > > > +1
> >> >> > > > > Just to mention we would need to release hadoop-thirdparty
> too
> >> >> > before.
> >> >> > > > > Presently we are using the snapshot version of it.
> >> >> > > > >
> >> >> > > > > -Ayush
> >> >> > > > >
> >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> >> weichiu@apache.org
> >> >> >
> >> >> > > > wrote:
> >> >> > > > > >
> >> >> > > > > > Hi all,
> >> >> > > > > >
> >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now
> >> we've
> >> >> > > > > accumulated
> >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> >> >> downstreamers
> >> >> > > are
> >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava version
> >> >> > conflict
> >> >> > > > > issue.
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> >> >> > > > > >
> >> >> > > > > > We should start the release work for 3.3.1 before the diff
> >> >> becomes
> >> >> > > even
> >> >> > > > > > larger.
> >> >> > > > > >
> >> >> > > > > > I believe there are currently only two real blockers for a
> >> >> 3.3.1
> >> >> > > > (using
> >> >> > > > > > this filter
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> >> >> > > > > > )
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > 1. HDFS-15566 <
> >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> >> >> > > > > > 2.
> >> >> > > > > > 1. HADOOP-17112 <
> >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> >> >> > > > > >
> >> >> > > > > > 2.
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> >> >> > > > > >
> >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM build
> >> >> > process.
> >> >> > > > > That's
> >> >> > > > > > going to be important for future releases.
> >> >> > > > >
> >> >> > > > >
> >> >> ---------------------------------------------------------------------
> >> >> > > > > To unsubscribe, e-mail:
> hdfs-dev-unsubscribe@hadoop.apache.org
> >> >> > > > > For additional commands, e-mail:
> >> hdfs-dev-help@hadoop.apache.org
> >> >> > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >> >>
> >> >> --
> >> >>
> >> >>
> >> >>
> >> >> --Brahma Reddy Battula
> >> >>
> >> >
> >>
> >> --
> >>
> >>
> >>
> >> --Brahma Reddy Battula
> >>
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
I'm going to argue its too late to do IPv6 support close to a release, as
it's best if its on developer machines for some time to let all quirks
surface. It's not so much IPv6 itself, but do we cause any regressions on
IPv4?
But: it can/should go into trunk and stabilize there
On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
muralikrishna.dmmkr@gmail.com> wrote:
> Hi Brahma,
>
> I have missed out mentioning about the IPV6 feature in the last mail,
> Support for IPV6 has been in development since 2015 and We have done a good
> amount of testing at our organisation, the feature is stable and used by
> our customers extensively in the last one year. I think it is a good time
> to add the IPV6 support to 3.3.1.
>
> https://issues.apache.org/jira/browse/HADOOP-11890
>
> Thanks
> D M Murali Krishna Reddy
>
> On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> muralikrishna.dmmkr@gmail.com> wrote:
>
> > Hi Brahma,
> >
> > Can we have this new feature "YARN Registry based AM discovery with retry
> > and in-flight task persistent via JHS" in the upcoming 3.1.1 release. I
> > have also attached a test-report in the below jira.
> >
> > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> >
> >
> > Thanks,
> > D M Murali Krishna Reddy
> >
> > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <brahma@apache.org
> >
> > wrote:
> >
> >> Hi Bilwa,
> >>
> >> I have commented on the jira's you mentioned. Based on the stability we
> >> can
> >> plan this.But needs to be merged ASAP.
> >>
> >>
> >>
> >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> >>
> >> > Hi Brahma,
> >> >
> >> > Can we have below features in 3.3.1 release? We have been using these
> >> > features for a long time. They are stable and tested in bigger
> clusters.
> >> >
> >> > 1. Container reuse -
> >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> >> > 2. Speculative attempts should not run on the same node -
> >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> >> >
> >> > Thanks,
> >> > Bilwa
> >> >
> >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <brahma@apache.org
> >
> >> > wrote:
> >> >
> >> >> Sorry for the late reply..
> >> >>
> >> >> I will come up with a plan.. Please let me know if anybody has some
> >> >> features/improvements/bugs that need to be included.
> >> >>
> >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org>
> >> wrote:
> >> >>
> >> >> > Hi Wei-Chiu,
> >> >> >
> >> >> > What will be the next steps here for 3.3.1 planning?
> >> >> >
> >> >> > Thanks
> >> >> > Sunil
> >> >> >
> >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> >> >> >
> >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> >> >> > <stevel@cloudera.com.invalid
> >> >> > > >
> >> >> > > wrote:
> >> >> > >
> >> >> > > >
> >> >> > > > Regarding blockers &c: how about we have a little hackathon
> >> where we
> >> >> > try
> >> >> > > > and get things in. This means a promise of review time from the
> >> >> people
> >> >> > > with
> >> >> > > > commit rights and other people who understand the code (Stack?)
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
> >> >> > > Thanks,
> >> >> > > S
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > > > -steve
> >> >> > > >
> >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ayushtkn@gmail.com
> >
> >> >> wrote:
> >> >> > > >
> >> >> > > > > +1
> >> >> > > > > Just to mention we would need to release hadoop-thirdparty
> too
> >> >> > before.
> >> >> > > > > Presently we are using the snapshot version of it.
> >> >> > > > >
> >> >> > > > > -Ayush
> >> >> > > > >
> >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> >> weichiu@apache.org
> >> >> >
> >> >> > > > wrote:
> >> >> > > > > >
> >> >> > > > > > Hi all,
> >> >> > > > > >
> >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now
> >> we've
> >> >> > > > > accumulated
> >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> >> >> downstreamers
> >> >> > > are
> >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava version
> >> >> > conflict
> >> >> > > > > issue.
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> >> >> > > > > >
> >> >> > > > > > We should start the release work for 3.3.1 before the diff
> >> >> becomes
> >> >> > > even
> >> >> > > > > > larger.
> >> >> > > > > >
> >> >> > > > > > I believe there are currently only two real blockers for a
> >> >> 3.3.1
> >> >> > > > (using
> >> >> > > > > > this filter
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> >> >> > > > > > )
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > 1. HDFS-15566 <
> >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> >> >> > > > > > 2.
> >> >> > > > > > 1. HADOOP-17112 <
> >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> >> >> > > > > >
> >> >> > > > > > 2.
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> >> >> > > > > >
> >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM build
> >> >> > process.
> >> >> > > > > That's
> >> >> > > > > > going to be important for future releases.
> >> >> > > > >
> >> >> > > > >
> >> >> ---------------------------------------------------------------------
> >> >> > > > > To unsubscribe, e-mail:
> hdfs-dev-unsubscribe@hadoop.apache.org
> >> >> > > > > For additional commands, e-mail:
> >> hdfs-dev-help@hadoop.apache.org
> >> >> > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >> >>
> >> >> --
> >> >>
> >> >>
> >> >>
> >> >> --Brahma Reddy Battula
> >> >>
> >> >
> >>
> >> --
> >>
> >>
> >>
> >> --Brahma Reddy Battula
> >>
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
I'm going to argue its too late to do IPv6 support close to a release, as
it's best if its on developer machines for some time to let all quirks
surface. It's not so much IPv6 itself, but do we cause any regressions on
IPv4?
But: it can/should go into trunk and stabilize there
On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr <
muralikrishna.dmmkr@gmail.com> wrote:
> Hi Brahma,
>
> I have missed out mentioning about the IPV6 feature in the last mail,
> Support for IPV6 has been in development since 2015 and We have done a good
> amount of testing at our organisation, the feature is stable and used by
> our customers extensively in the last one year. I think it is a good time
> to add the IPV6 support to 3.3.1.
>
> https://issues.apache.org/jira/browse/HADOOP-11890
>
> Thanks
> D M Murali Krishna Reddy
>
> On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
> muralikrishna.dmmkr@gmail.com> wrote:
>
> > Hi Brahma,
> >
> > Can we have this new feature "YARN Registry based AM discovery with retry
> > and in-flight task persistent via JHS" in the upcoming 3.1.1 release. I
> > have also attached a test-report in the below jira.
> >
> > https://issues.apache.org/jira/browse/MAPREDUCE-6726
> >
> >
> > Thanks,
> > D M Murali Krishna Reddy
> >
> > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <brahma@apache.org
> >
> > wrote:
> >
> >> Hi Bilwa,
> >>
> >> I have commented on the jira's you mentioned. Based on the stability we
> >> can
> >> plan this.But needs to be merged ASAP.
> >>
> >>
> >>
> >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> >>
> >> > Hi Brahma,
> >> >
> >> > Can we have below features in 3.3.1 release? We have been using these
> >> > features for a long time. They are stable and tested in bigger
> clusters.
> >> >
> >> > 1. Container reuse -
> >> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> >> > 2. Speculative attempts should not run on the same node -
> >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> >> >
> >> > Thanks,
> >> > Bilwa
> >> >
> >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <brahma@apache.org
> >
> >> > wrote:
> >> >
> >> >> Sorry for the late reply..
> >> >>
> >> >> I will come up with a plan.. Please let me know if anybody has some
> >> >> features/improvements/bugs that need to be included.
> >> >>
> >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org>
> >> wrote:
> >> >>
> >> >> > Hi Wei-Chiu,
> >> >> >
> >> >> > What will be the next steps here for 3.3.1 planning?
> >> >> >
> >> >> > Thanks
> >> >> > Sunil
> >> >> >
> >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> >> >> >
> >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> >> >> > <stevel@cloudera.com.invalid
> >> >> > > >
> >> >> > > wrote:
> >> >> > >
> >> >> > > >
> >> >> > > > Regarding blockers &c: how about we have a little hackathon
> >> where we
> >> >> > try
> >> >> > > > and get things in. This means a promise of review time from the
> >> >> people
> >> >> > > with
> >> >> > > > commit rights and other people who understand the code (Stack?)
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
> >> >> > > Thanks,
> >> >> > > S
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > > > -steve
> >> >> > > >
> >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ayushtkn@gmail.com
> >
> >> >> wrote:
> >> >> > > >
> >> >> > > > > +1
> >> >> > > > > Just to mention we would need to release hadoop-thirdparty
> too
> >> >> > before.
> >> >> > > > > Presently we are using the snapshot version of it.
> >> >> > > > >
> >> >> > > > > -Ayush
> >> >> > > > >
> >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> >> weichiu@apache.org
> >> >> >
> >> >> > > > wrote:
> >> >> > > > > >
> >> >> > > > > > Hi all,
> >> >> > > > > >
> >> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now
> >> we've
> >> >> > > > > accumulated
> >> >> > > > > > more than 400 changes in the branch-3.3. A number of
> >> >> downstreamers
> >> >> > > are
> >> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava version
> >> >> > conflict
> >> >> > > > > issue.
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> >> >> > > > > >
> >> >> > > > > > We should start the release work for 3.3.1 before the diff
> >> >> becomes
> >> >> > > even
> >> >> > > > > > larger.
> >> >> > > > > >
> >> >> > > > > > I believe there are currently only two real blockers for a
> >> >> 3.3.1
> >> >> > > > (using
> >> >> > > > > > this filter
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> >> >> > > > > > )
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > 1. HDFS-15566 <
> >> >> https://issues.apache.org/jira/browse/HDFS-15566>
> >> >> > > > > > 2.
> >> >> > > > > > 1. HADOOP-17112 <
> >> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> >> >> > > > > >
> >> >> > > > > > 2.
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> >> >> > > > > >
> >> >> > > > > > Also, the HowToRelease wiki does not describe the ARM build
> >> >> > process.
> >> >> > > > > That's
> >> >> > > > > > going to be important for future releases.
> >> >> > > > >
> >> >> > > > >
> >> >> ---------------------------------------------------------------------
> >> >> > > > > To unsubscribe, e-mail:
> hdfs-dev-unsubscribe@hadoop.apache.org
> >> >> > > > > For additional commands, e-mail:
> >> hdfs-dev-help@hadoop.apache.org
> >> >> > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >> >>
> >> >> --
> >> >>
> >> >>
> >> >>
> >> >> --Brahma Reddy Battula
> >> >>
> >> >
> >>
> >> --
> >>
> >>
> >>
> >> --Brahma Reddy Battula
> >>
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Muralikrishna Dmmkr <mu...@gmail.com>.
Hi Brahma,
I have missed out mentioning about the IPV6 feature in the last mail,
Support for IPV6 has been in development since 2015 and We have done a good
amount of testing at our organisation, the feature is stable and used by
our customers extensively in the last one year. I think it is a good time
to add the IPV6 support to 3.3.1.
https://issues.apache.org/jira/browse/HADOOP-11890
Thanks
D M Murali Krishna Reddy
On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr <
muralikrishna.dmmkr@gmail.com> wrote:
> Hi Brahma,
>
> Can we have this new feature "YARN Registry based AM discovery with retry
> and in-flight task persistent via JHS" in the upcoming 3.1.1 release. I
> have also attached a test-report in the below jira.
>
> https://issues.apache.org/jira/browse/MAPREDUCE-6726
>
>
> Thanks,
> D M Murali Krishna Reddy
>
> On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <br...@apache.org>
> wrote:
>
>> Hi Bilwa,
>>
>> I have commented on the jira's you mentioned. Based on the stability we
>> can
>> plan this.But needs to be merged ASAP.
>>
>>
>>
>> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
>>
>> > Hi Brahma,
>> >
>> > Can we have below features in 3.3.1 release? We have been using these
>> > features for a long time. They are stable and tested in bigger clusters.
>> >
>> > 1. Container reuse -
>> https://issues.apache.org/jira/browse/MAPREDUCE-6749
>> > 2. Speculative attempts should not run on the same node -
>> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
>> >
>> > Thanks,
>> > Bilwa
>> >
>> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <br...@apache.org>
>> > wrote:
>> >
>> >> Sorry for the late reply..
>> >>
>> >> I will come up with a plan.. Please let me know if anybody has some
>> >> features/improvements/bugs that need to be included.
>> >>
>> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org>
>> wrote:
>> >>
>> >> > Hi Wei-Chiu,
>> >> >
>> >> > What will be the next steps here for 3.3.1 planning?
>> >> >
>> >> > Thanks
>> >> > Sunil
>> >> >
>> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
>> >> >
>> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
>> >> > <stevel@cloudera.com.invalid
>> >> > > >
>> >> > > wrote:
>> >> > >
>> >> > > >
>> >> > > > Regarding blockers &c: how about we have a little hackathon
>> where we
>> >> > try
>> >> > > > and get things in. This means a promise of review time from the
>> >> people
>> >> > > with
>> >> > > > commit rights and other people who understand the code (Stack?)
>> >> > > >
>> >> > > >
>> >> > >
>> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
>> >> > > Thanks,
>> >> > > S
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > > > -steve
>> >> > > >
>> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com>
>> >> wrote:
>> >> > > >
>> >> > > > > +1
>> >> > > > > Just to mention we would need to release hadoop-thirdparty too
>> >> > before.
>> >> > > > > Presently we are using the snapshot version of it.
>> >> > > > >
>> >> > > > > -Ayush
>> >> > > > >
>> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
>> weichiu@apache.org
>> >> >
>> >> > > > wrote:
>> >> > > > > >
>> >> > > > > > Hi all,
>> >> > > > > >
>> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now
>> we've
>> >> > > > > accumulated
>> >> > > > > > more than 400 changes in the branch-3.3. A number of
>> >> downstreamers
>> >> > > are
>> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava version
>> >> > conflict
>> >> > > > > issue.
>> >> > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>> >> > > > > >
>> >> > > > > > We should start the release work for 3.3.1 before the diff
>> >> becomes
>> >> > > even
>> >> > > > > > larger.
>> >> > > > > >
>> >> > > > > > I believe there are currently only two real blockers for a
>> >> 3.3.1
>> >> > > > (using
>> >> > > > > > this filter
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
>> >> > > > > > )
>> >> > > > > >
>> >> > > > > >
>> >> > > > > > 1. HDFS-15566 <
>> >> https://issues.apache.org/jira/browse/HDFS-15566>
>> >> > > > > > 2.
>> >> > > > > > 1. HADOOP-17112 <
>> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
>> >> > > > > >
>> >> > > > > > 2.
>> >> > > > > >
>> >> > > > > >
>> >> > > > > >
>> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
>> >> > > > > >
>> >> > > > > > Also, the HowToRelease wiki does not describe the ARM build
>> >> > process.
>> >> > > > > That's
>> >> > > > > > going to be important for future releases.
>> >> > > > >
>> >> > > > >
>> >> ---------------------------------------------------------------------
>> >> > > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> >> > > > > For additional commands, e-mail:
>> hdfs-dev-help@hadoop.apache.org
>> >> > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> >>
>> >> --
>> >>
>> >>
>> >>
>> >> --Brahma Reddy Battula
>> >>
>> >
>>
>> --
>>
>>
>>
>> --Brahma Reddy Battula
>>
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Muralikrishna Dmmkr <mu...@gmail.com>.
Hi Brahma,
Can we have this new feature "YARN Registry based AM discovery with retry
and in-flight task persistent via JHS" in the upcoming 3.1.1 release. I
have also attached a test-report in the below jira.
https://issues.apache.org/jira/browse/MAPREDUCE-6726
Thanks,
D M Murali Krishna Reddy
On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <br...@apache.org>
wrote:
> Hi Bilwa,
>
> I have commented on the jira's you mentioned. Based on the stability we can
> plan this.But needs to be merged ASAP.
>
>
>
> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
>
> > Hi Brahma,
> >
> > Can we have below features in 3.3.1 release? We have been using these
> > features for a long time. They are stable and tested in bigger clusters.
> >
> > 1. Container reuse -
> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > 2. Speculative attempts should not run on the same node -
> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> >
> > Thanks,
> > Bilwa
> >
> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <br...@apache.org>
> > wrote:
> >
> >> Sorry for the late reply..
> >>
> >> I will come up with a plan.. Please let me know if anybody has some
> >> features/improvements/bugs that need to be included.
> >>
> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org>
> wrote:
> >>
> >> > Hi Wei-Chiu,
> >> >
> >> > What will be the next steps here for 3.3.1 planning?
> >> >
> >> > Thanks
> >> > Sunil
> >> >
> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> >> >
> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> >> > <stevel@cloudera.com.invalid
> >> > > >
> >> > > wrote:
> >> > >
> >> > > >
> >> > > > Regarding blockers &c: how about we have a little hackathon where
> we
> >> > try
> >> > > > and get things in. This means a promise of review time from the
> >> people
> >> > > with
> >> > > > commit rights and other people who understand the code (Stack?)
> >> > > >
> >> > > >
> >> > >
> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
> >> > > Thanks,
> >> > > S
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > > -steve
> >> > > >
> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com>
> >> wrote:
> >> > > >
> >> > > > > +1
> >> > > > > Just to mention we would need to release hadoop-thirdparty too
> >> > before.
> >> > > > > Presently we are using the snapshot version of it.
> >> > > > >
> >> > > > > -Ayush
> >> > > > >
> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> weichiu@apache.org
> >> >
> >> > > > wrote:
> >> > > > > >
> >> > > > > > Hi all,
> >> > > > > >
> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now we've
> >> > > > > accumulated
> >> > > > > > more than 400 changes in the branch-3.3. A number of
> >> downstreamers
> >> > > are
> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava version
> >> > conflict
> >> > > > > issue.
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> >> > > > > >
> >> > > > > > We should start the release work for 3.3.1 before the diff
> >> becomes
> >> > > even
> >> > > > > > larger.
> >> > > > > >
> >> > > > > > I believe there are currently only two real blockers for a
> >> 3.3.1
> >> > > > (using
> >> > > > > > this filter
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> >> > > > > > )
> >> > > > > >
> >> > > > > >
> >> > > > > > 1. HDFS-15566 <
> >> https://issues.apache.org/jira/browse/HDFS-15566>
> >> > > > > > 2.
> >> > > > > > 1. HADOOP-17112 <
> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> >> > > > > >
> >> > > > > > 2.
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> >> > > > > >
> >> > > > > > Also, the HowToRelease wiki does not describe the ARM build
> >> > process.
> >> > > > > That's
> >> > > > > > going to be important for future releases.
> >> > > > >
> >> > > > >
> >> ---------------------------------------------------------------------
> >> > > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> > > > > For additional commands, e-mail:
> hdfs-dev-help@hadoop.apache.org
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
> >> --
> >>
> >>
> >>
> >> --Brahma Reddy Battula
> >>
> >
>
> --
>
>
>
> --Brahma Reddy Battula
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Muralikrishna Dmmkr <mu...@gmail.com>.
Hi Brahma,
Can we have this new feature "YARN Registry based AM discovery with retry
and in-flight task persistent via JHS" in the upcoming 3.1.1 release. I
have also attached a test-report in the below jira.
https://issues.apache.org/jira/browse/MAPREDUCE-6726
Thanks,
D M Murali Krishna Reddy
On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula <br...@apache.org>
wrote:
> Hi Bilwa,
>
> I have commented on the jira's you mentioned. Based on the stability we can
> plan this.But needs to be merged ASAP.
>
>
>
> On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
>
> > Hi Brahma,
> >
> > Can we have below features in 3.3.1 release? We have been using these
> > features for a long time. They are stable and tested in bigger clusters.
> >
> > 1. Container reuse -
> https://issues.apache.org/jira/browse/MAPREDUCE-6749
> > 2. Speculative attempts should not run on the same node -
> > https://issues.apache.org/jira/browse/MAPREDUCE-7169
> >
> > Thanks,
> > Bilwa
> >
> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <br...@apache.org>
> > wrote:
> >
> >> Sorry for the late reply..
> >>
> >> I will come up with a plan.. Please let me know if anybody has some
> >> features/improvements/bugs that need to be included.
> >>
> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org>
> wrote:
> >>
> >> > Hi Wei-Chiu,
> >> >
> >> > What will be the next steps here for 3.3.1 planning?
> >> >
> >> > Thanks
> >> > Sunil
> >> >
> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> >> >
> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> >> > <stevel@cloudera.com.invalid
> >> > > >
> >> > > wrote:
> >> > >
> >> > > >
> >> > > > Regarding blockers &c: how about we have a little hackathon where
> we
> >> > try
> >> > > > and get things in. This means a promise of review time from the
> >> people
> >> > > with
> >> > > > commit rights and other people who understand the code (Stack?)
> >> > > >
> >> > > >
> >> > >
> >> > > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
> >> > > Thanks,
> >> > > S
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > > -steve
> >> > > >
> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com>
> >> wrote:
> >> > > >
> >> > > > > +1
> >> > > > > Just to mention we would need to release hadoop-thirdparty too
> >> > before.
> >> > > > > Presently we are using the snapshot version of it.
> >> > > > >
> >> > > > > -Ayush
> >> > > > >
> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <
> weichiu@apache.org
> >> >
> >> > > > wrote:
> >> > > > > >
> >> > > > > > Hi all,
> >> > > > > >
> >> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now we've
> >> > > > > accumulated
> >> > > > > > more than 400 changes in the branch-3.3. A number of
> >> downstreamers
> >> > > are
> >> > > > > > eagerly waiting for 3.3.1 which addresses the guava version
> >> > conflict
> >> > > > > issue.
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> >> > > > > >
> >> > > > > > We should start the release work for 3.3.1 before the diff
> >> becomes
> >> > > even
> >> > > > > > larger.
> >> > > > > >
> >> > > > > > I believe there are currently only two real blockers for a
> >> 3.3.1
> >> > > > (using
> >> > > > > > this filter
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> >> > > > > > )
> >> > > > > >
> >> > > > > >
> >> > > > > > 1. HDFS-15566 <
> >> https://issues.apache.org/jira/browse/HDFS-15566>
> >> > > > > > 2.
> >> > > > > > 1. HADOOP-17112 <
> >> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> >> > > > > >
> >> > > > > > 2.
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> >> > > > > >
> >> > > > > > Also, the HowToRelease wiki does not describe the ARM build
> >> > process.
> >> > > > > That's
> >> > > > > > going to be important for future releases.
> >> > > > >
> >> > > > >
> >> ---------------------------------------------------------------------
> >> > > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> > > > > For additional commands, e-mail:
> hdfs-dev-help@hadoop.apache.org
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
> >> --
> >>
> >>
> >>
> >> --Brahma Reddy Battula
> >>
> >
>
> --
>
>
>
> --Brahma Reddy Battula
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Brahma Reddy Battula <br...@apache.org>.
Hi Bilwa,
I have commented on the jira's you mentioned. Based on the stability we can
plan this.But needs to be merged ASAP.
On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> Hi Brahma,
>
> Can we have below features in 3.3.1 release? We have been using these
> features for a long time. They are stable and tested in bigger clusters.
>
> 1. Container reuse - https://issues.apache.org/jira/browse/MAPREDUCE-6749
> 2. Speculative attempts should not run on the same node -
> https://issues.apache.org/jira/browse/MAPREDUCE-7169
>
> Thanks,
> Bilwa
>
> On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <br...@apache.org>
> wrote:
>
>> Sorry for the late reply..
>>
>> I will come up with a plan.. Please let me know if anybody has some
>> features/improvements/bugs that need to be included.
>>
>> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org> wrote:
>>
>> > Hi Wei-Chiu,
>> >
>> > What will be the next steps here for 3.3.1 planning?
>> >
>> > Thanks
>> > Sunil
>> >
>> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
>> >
>> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
>> > <stevel@cloudera.com.invalid
>> > > >
>> > > wrote:
>> > >
>> > > >
>> > > > Regarding blockers &c: how about we have a little hackathon where we
>> > try
>> > > > and get things in. This means a promise of review time from the
>> people
>> > > with
>> > > > commit rights and other people who understand the code (Stack?)
>> > > >
>> > > >
>> > >
>> > > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
>> > > Thanks,
>> > > S
>> > >
>> > >
>> > >
>> > >
>> > > > -steve
>> > > >
>> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com>
>> wrote:
>> > > >
>> > > > > +1
>> > > > > Just to mention we would need to release hadoop-thirdparty too
>> > before.
>> > > > > Presently we are using the snapshot version of it.
>> > > > >
>> > > > > -Ayush
>> > > > >
>> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <weichiu@apache.org
>> >
>> > > > wrote:
>> > > > > >
>> > > > > > Hi all,
>> > > > > >
>> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now we've
>> > > > > accumulated
>> > > > > > more than 400 changes in the branch-3.3. A number of
>> downstreamers
>> > > are
>> > > > > > eagerly waiting for 3.3.1 which addresses the guava version
>> > conflict
>> > > > > issue.
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>> > > > > >
>> > > > > > We should start the release work for 3.3.1 before the diff
>> becomes
>> > > even
>> > > > > > larger.
>> > > > > >
>> > > > > > I believe there are currently only two real blockers for a
>> 3.3.1
>> > > > (using
>> > > > > > this filter
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
>> > > > > > )
>> > > > > >
>> > > > > >
>> > > > > > 1. HDFS-15566 <
>> https://issues.apache.org/jira/browse/HDFS-15566>
>> > > > > > 2.
>> > > > > > 1. HADOOP-17112 <
>> > > > https://issues.apache.org/jira/browse/HADOOP-17112
>> > > > > >
>> > > > > > 2.
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
>> > > > > >
>> > > > > > Also, the HowToRelease wiki does not describe the ARM build
>> > process.
>> > > > > That's
>> > > > > > going to be important for future releases.
>> > > > >
>> > > > >
>> ---------------------------------------------------------------------
>> > > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>>
>> --
>>
>>
>>
>> --Brahma Reddy Battula
>>
>
--
--Brahma Reddy Battula
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Brahma Reddy Battula <br...@apache.org>.
Hi Bilwa,
I have commented on the jira's you mentioned. Based on the stability we can
plan this.But needs to be merged ASAP.
On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> Hi Brahma,
>
> Can we have below features in 3.3.1 release? We have been using these
> features for a long time. They are stable and tested in bigger clusters.
>
> 1. Container reuse - https://issues.apache.org/jira/browse/MAPREDUCE-6749
> 2. Speculative attempts should not run on the same node -
> https://issues.apache.org/jira/browse/MAPREDUCE-7169
>
> Thanks,
> Bilwa
>
> On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <br...@apache.org>
> wrote:
>
>> Sorry for the late reply..
>>
>> I will come up with a plan.. Please let me know if anybody has some
>> features/improvements/bugs that need to be included.
>>
>> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org> wrote:
>>
>> > Hi Wei-Chiu,
>> >
>> > What will be the next steps here for 3.3.1 planning?
>> >
>> > Thanks
>> > Sunil
>> >
>> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
>> >
>> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
>> > <stevel@cloudera.com.invalid
>> > > >
>> > > wrote:
>> > >
>> > > >
>> > > > Regarding blockers &c: how about we have a little hackathon where we
>> > try
>> > > > and get things in. This means a promise of review time from the
>> people
>> > > with
>> > > > commit rights and other people who understand the code (Stack?)
>> > > >
>> > > >
>> > >
>> > > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
>> > > Thanks,
>> > > S
>> > >
>> > >
>> > >
>> > >
>> > > > -steve
>> > > >
>> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com>
>> wrote:
>> > > >
>> > > > > +1
>> > > > > Just to mention we would need to release hadoop-thirdparty too
>> > before.
>> > > > > Presently we are using the snapshot version of it.
>> > > > >
>> > > > > -Ayush
>> > > > >
>> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <weichiu@apache.org
>> >
>> > > > wrote:
>> > > > > >
>> > > > > > Hi all,
>> > > > > >
>> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now we've
>> > > > > accumulated
>> > > > > > more than 400 changes in the branch-3.3. A number of
>> downstreamers
>> > > are
>> > > > > > eagerly waiting for 3.3.1 which addresses the guava version
>> > conflict
>> > > > > issue.
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>> > > > > >
>> > > > > > We should start the release work for 3.3.1 before the diff
>> becomes
>> > > even
>> > > > > > larger.
>> > > > > >
>> > > > > > I believe there are currently only two real blockers for a
>> 3.3.1
>> > > > (using
>> > > > > > this filter
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
>> > > > > > )
>> > > > > >
>> > > > > >
>> > > > > > 1. HDFS-15566 <
>> https://issues.apache.org/jira/browse/HDFS-15566>
>> > > > > > 2.
>> > > > > > 1. HADOOP-17112 <
>> > > > https://issues.apache.org/jira/browse/HADOOP-17112
>> > > > > >
>> > > > > > 2.
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
>> > > > > >
>> > > > > > Also, the HowToRelease wiki does not describe the ARM build
>> > process.
>> > > > > That's
>> > > > > > going to be important for future releases.
>> > > > >
>> > > > >
>> ---------------------------------------------------------------------
>> > > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>>
>> --
>>
>>
>>
>> --Brahma Reddy Battula
>>
>
--
--Brahma Reddy Battula
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Brahma Reddy Battula <br...@apache.org>.
Hi Bilwa,
I have commented on the jira's you mentioned. Based on the stability we can
plan this.But needs to be merged ASAP.
On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> Hi Brahma,
>
> Can we have below features in 3.3.1 release? We have been using these
> features for a long time. They are stable and tested in bigger clusters.
>
> 1. Container reuse - https://issues.apache.org/jira/browse/MAPREDUCE-6749
> 2. Speculative attempts should not run on the same node -
> https://issues.apache.org/jira/browse/MAPREDUCE-7169
>
> Thanks,
> Bilwa
>
> On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <br...@apache.org>
> wrote:
>
>> Sorry for the late reply..
>>
>> I will come up with a plan.. Please let me know if anybody has some
>> features/improvements/bugs that need to be included.
>>
>> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org> wrote:
>>
>> > Hi Wei-Chiu,
>> >
>> > What will be the next steps here for 3.3.1 planning?
>> >
>> > Thanks
>> > Sunil
>> >
>> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
>> >
>> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
>> > <stevel@cloudera.com.invalid
>> > > >
>> > > wrote:
>> > >
>> > > >
>> > > > Regarding blockers &c: how about we have a little hackathon where we
>> > try
>> > > > and get things in. This means a promise of review time from the
>> people
>> > > with
>> > > > commit rights and other people who understand the code (Stack?)
>> > > >
>> > > >
>> > >
>> > > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
>> > > Thanks,
>> > > S
>> > >
>> > >
>> > >
>> > >
>> > > > -steve
>> > > >
>> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com>
>> wrote:
>> > > >
>> > > > > +1
>> > > > > Just to mention we would need to release hadoop-thirdparty too
>> > before.
>> > > > > Presently we are using the snapshot version of it.
>> > > > >
>> > > > > -Ayush
>> > > > >
>> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <weichiu@apache.org
>> >
>> > > > wrote:
>> > > > > >
>> > > > > > Hi all,
>> > > > > >
>> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now we've
>> > > > > accumulated
>> > > > > > more than 400 changes in the branch-3.3. A number of
>> downstreamers
>> > > are
>> > > > > > eagerly waiting for 3.3.1 which addresses the guava version
>> > conflict
>> > > > > issue.
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>> > > > > >
>> > > > > > We should start the release work for 3.3.1 before the diff
>> becomes
>> > > even
>> > > > > > larger.
>> > > > > >
>> > > > > > I believe there are currently only two real blockers for a
>> 3.3.1
>> > > > (using
>> > > > > > this filter
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
>> > > > > > )
>> > > > > >
>> > > > > >
>> > > > > > 1. HDFS-15566 <
>> https://issues.apache.org/jira/browse/HDFS-15566>
>> > > > > > 2.
>> > > > > > 1. HADOOP-17112 <
>> > > > https://issues.apache.org/jira/browse/HADOOP-17112
>> > > > > >
>> > > > > > 2.
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
>> > > > > >
>> > > > > > Also, the HowToRelease wiki does not describe the ARM build
>> > process.
>> > > > > That's
>> > > > > > going to be important for future releases.
>> > > > >
>> > > > >
>> ---------------------------------------------------------------------
>> > > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>>
>> --
>>
>>
>>
>> --Brahma Reddy Battula
>>
>
--
--Brahma Reddy Battula
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Brahma Reddy Battula <br...@apache.org>.
Hi Bilwa,
I have commented on the jira's you mentioned. Based on the stability we can
plan this.But needs to be merged ASAP.
On Fri, Feb 19, 2021 at 5:20 PM bilwa st <st...@gmail.com> wrote:
> Hi Brahma,
>
> Can we have below features in 3.3.1 release? We have been using these
> features for a long time. They are stable and tested in bigger clusters.
>
> 1. Container reuse - https://issues.apache.org/jira/browse/MAPREDUCE-6749
> 2. Speculative attempts should not run on the same node -
> https://issues.apache.org/jira/browse/MAPREDUCE-7169
>
> Thanks,
> Bilwa
>
> On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <br...@apache.org>
> wrote:
>
>> Sorry for the late reply..
>>
>> I will come up with a plan.. Please let me know if anybody has some
>> features/improvements/bugs that need to be included.
>>
>> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org> wrote:
>>
>> > Hi Wei-Chiu,
>> >
>> > What will be the next steps here for 3.3.1 planning?
>> >
>> > Thanks
>> > Sunil
>> >
>> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
>> >
>> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
>> > <stevel@cloudera.com.invalid
>> > > >
>> > > wrote:
>> > >
>> > > >
>> > > > Regarding blockers &c: how about we have a little hackathon where we
>> > try
>> > > > and get things in. This means a promise of review time from the
>> people
>> > > with
>> > > > commit rights and other people who understand the code (Stack?)
>> > > >
>> > > >
>> > >
>> > > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
>> > > Thanks,
>> > > S
>> > >
>> > >
>> > >
>> > >
>> > > > -steve
>> > > >
>> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com>
>> wrote:
>> > > >
>> > > > > +1
>> > > > > Just to mention we would need to release hadoop-thirdparty too
>> > before.
>> > > > > Presently we are using the snapshot version of it.
>> > > > >
>> > > > > -Ayush
>> > > > >
>> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <weichiu@apache.org
>> >
>> > > > wrote:
>> > > > > >
>> > > > > > Hi all,
>> > > > > >
>> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now we've
>> > > > > accumulated
>> > > > > > more than 400 changes in the branch-3.3. A number of
>> downstreamers
>> > > are
>> > > > > > eagerly waiting for 3.3.1 which addresses the guava version
>> > conflict
>> > > > > issue.
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>> > > > > >
>> > > > > > We should start the release work for 3.3.1 before the diff
>> becomes
>> > > even
>> > > > > > larger.
>> > > > > >
>> > > > > > I believe there are currently only two real blockers for a
>> 3.3.1
>> > > > (using
>> > > > > > this filter
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
>> > > > > > )
>> > > > > >
>> > > > > >
>> > > > > > 1. HDFS-15566 <
>> https://issues.apache.org/jira/browse/HDFS-15566>
>> > > > > > 2.
>> > > > > > 1. HADOOP-17112 <
>> > > > https://issues.apache.org/jira/browse/HADOOP-17112
>> > > > > >
>> > > > > > 2.
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
>> > > > > >
>> > > > > > Also, the HowToRelease wiki does not describe the ARM build
>> > process.
>> > > > > That's
>> > > > > > going to be important for future releases.
>> > > > >
>> > > > >
>> ---------------------------------------------------------------------
>> > > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> > > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>>
>> --
>>
>>
>>
>> --Brahma Reddy Battula
>>
>
--
--Brahma Reddy Battula
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by bilwa st <st...@gmail.com>.
Hi Brahma,
Can we have below features in 3.3.1 release? We have been using these
features for a long time. They are stable and tested in bigger clusters.
1. Container reuse - https://issues.apache.org/jira/browse/MAPREDUCE-6749
2. Speculative attempts should not run on the same node -
https://issues.apache.org/jira/browse/MAPREDUCE-7169
Thanks,
Bilwa
On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula <br...@apache.org>
wrote:
> Sorry for the late reply..
>
> I will come up with a plan.. Please let me know if anybody has some
> features/improvements/bugs that need to be included.
>
> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org> wrote:
>
> > Hi Wei-Chiu,
> >
> > What will be the next steps here for 3.3.1 planning?
> >
> > Thanks
> > Sunil
> >
> > On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> >
> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> > <stevel@cloudera.com.invalid
> > > >
> > > wrote:
> > >
> > > >
> > > > Regarding blockers &c: how about we have a little hackathon where we
> > try
> > > > and get things in. This means a promise of review time from the
> people
> > > with
> > > > commit rights and other people who understand the code (Stack?)
> > > >
> > > >
> > >
> > > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
> > > Thanks,
> > > S
> > >
> > >
> > >
> > >
> > > > -steve
> > > >
> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com>
> wrote:
> > > >
> > > > > +1
> > > > > Just to mention we would need to release hadoop-thirdparty too
> > before.
> > > > > Presently we are using the snapshot version of it.
> > > > >
> > > > > -Ayush
> > > > >
> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org>
> > > > wrote:
> > > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > Hadoop 3.3.0 was released half a year ago, and as of now we've
> > > > > accumulated
> > > > > > more than 400 changes in the branch-3.3. A number of
> downstreamers
> > > are
> > > > > > eagerly waiting for 3.3.1 which addresses the guava version
> > conflict
> > > > > issue.
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > > >
> > > > > > We should start the release work for 3.3.1 before the diff
> becomes
> > > even
> > > > > > larger.
> > > > > >
> > > > > > I believe there are currently only two real blockers for a 3.3.1
> > > > (using
> > > > > > this filter
> > > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > > )
> > > > > >
> > > > > >
> > > > > > 1. HDFS-15566 <
> https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > > 2.
> > > > > > 1. HADOOP-17112 <
> > > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > > >
> > > > > > 2.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > > > > >
> > > > > > Also, the HowToRelease wiki does not describe the ARM build
> > process.
> > > > > That's
> > > > > > going to be important for future releases.
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> > > > >
> > > > >
> > > >
> > >
> >
>
>
> --
>
>
>
> --Brahma Reddy Battula
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Brahma Reddy Battula <br...@apache.org>.
Sorry for the late reply..
I will come up with a plan.. Please let me know if anybody has some
features/improvements/bugs that need to be included.
On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org> wrote:
> Hi Wei-Chiu,
>
> What will be the next steps here for 3.3.1 planning?
>
> Thanks
> Sunil
>
> On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
>
> > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > >
> > > Regarding blockers &c: how about we have a little hackathon where we
> try
> > > and get things in. This means a promise of review time from the people
> > with
> > > commit rights and other people who understand the code (Stack?)
> > >
> > >
> >
> > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
> > Thanks,
> > S
> >
> >
> >
> >
> > > -steve
> > >
> > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
> > >
> > > > +1
> > > > Just to mention we would need to release hadoop-thirdparty too
> before.
> > > > Presently we are using the snapshot version of it.
> > > >
> > > > -Ayush
> > > >
> > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org>
> > > wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > Hadoop 3.3.0 was released half a year ago, and as of now we've
> > > > accumulated
> > > > > more than 400 changes in the branch-3.3. A number of downstreamers
> > are
> > > > > eagerly waiting for 3.3.1 which addresses the guava version
> conflict
> > > > issue.
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > >
> > > > > We should start the release work for 3.3.1 before the diff becomes
> > even
> > > > > larger.
> > > > >
> > > > > I believe there are currently only two real blockers for a 3.3.1
> > > (using
> > > > > this filter
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > )
> > > > >
> > > > >
> > > > > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > 2.
> > > > > 1. HADOOP-17112 <
> > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > >
> > > > > 2.
> > > > >
> > > > >
> > > > >
> > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > > > >
> > > > > Also, the HowToRelease wiki does not describe the ARM build
> process.
> > > > That's
> > > > > going to be important for future releases.
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> > > >
> > > >
> > >
> >
>
--
--Brahma Reddy Battula
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Brahma Reddy Battula <br...@apache.org>.
Sorry for the late reply..
I will come up with a plan.. Please let me know if anybody has some
features/improvements/bugs that need to be included.
On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org> wrote:
> Hi Wei-Chiu,
>
> What will be the next steps here for 3.3.1 planning?
>
> Thanks
> Sunil
>
> On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
>
> > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > >
> > > Regarding blockers &c: how about we have a little hackathon where we
> try
> > > and get things in. This means a promise of review time from the people
> > with
> > > commit rights and other people who understand the code (Stack?)
> > >
> > >
> >
> > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
> > Thanks,
> > S
> >
> >
> >
> >
> > > -steve
> > >
> > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
> > >
> > > > +1
> > > > Just to mention we would need to release hadoop-thirdparty too
> before.
> > > > Presently we are using the snapshot version of it.
> > > >
> > > > -Ayush
> > > >
> > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org>
> > > wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > Hadoop 3.3.0 was released half a year ago, and as of now we've
> > > > accumulated
> > > > > more than 400 changes in the branch-3.3. A number of downstreamers
> > are
> > > > > eagerly waiting for 3.3.1 which addresses the guava version
> conflict
> > > > issue.
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > >
> > > > > We should start the release work for 3.3.1 before the diff becomes
> > even
> > > > > larger.
> > > > >
> > > > > I believe there are currently only two real blockers for a 3.3.1
> > > (using
> > > > > this filter
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > )
> > > > >
> > > > >
> > > > > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > 2.
> > > > > 1. HADOOP-17112 <
> > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > >
> > > > > 2.
> > > > >
> > > > >
> > > > >
> > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > > > >
> > > > > Also, the HowToRelease wiki does not describe the ARM build
> process.
> > > > That's
> > > > > going to be important for future releases.
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> > > >
> > > >
> > >
> >
>
--
--Brahma Reddy Battula
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Brahma Reddy Battula <br...@apache.org>.
Sorry for the late reply..
I will come up with a plan.. Please let me know if anybody has some
features/improvements/bugs that need to be included.
On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org> wrote:
> Hi Wei-Chiu,
>
> What will be the next steps here for 3.3.1 planning?
>
> Thanks
> Sunil
>
> On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
>
> > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > >
> > > Regarding blockers &c: how about we have a little hackathon where we
> try
> > > and get things in. This means a promise of review time from the people
> > with
> > > commit rights and other people who understand the code (Stack?)
> > >
> > >
> >
> > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
> > Thanks,
> > S
> >
> >
> >
> >
> > > -steve
> > >
> > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
> > >
> > > > +1
> > > > Just to mention we would need to release hadoop-thirdparty too
> before.
> > > > Presently we are using the snapshot version of it.
> > > >
> > > > -Ayush
> > > >
> > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org>
> > > wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > Hadoop 3.3.0 was released half a year ago, and as of now we've
> > > > accumulated
> > > > > more than 400 changes in the branch-3.3. A number of downstreamers
> > are
> > > > > eagerly waiting for 3.3.1 which addresses the guava version
> conflict
> > > > issue.
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > >
> > > > > We should start the release work for 3.3.1 before the diff becomes
> > even
> > > > > larger.
> > > > >
> > > > > I believe there are currently only two real blockers for a 3.3.1
> > > (using
> > > > > this filter
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > )
> > > > >
> > > > >
> > > > > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > 2.
> > > > > 1. HADOOP-17112 <
> > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > >
> > > > > 2.
> > > > >
> > > > >
> > > > >
> > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > > > >
> > > > > Also, the HowToRelease wiki does not describe the ARM build
> process.
> > > > That's
> > > > > going to be important for future releases.
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> > > >
> > > >
> > >
> >
>
--
--Brahma Reddy Battula
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Brahma Reddy Battula <br...@apache.org>.
Sorry for the late reply..
I will come up with a plan.. Please let me know if anybody has some
features/improvements/bugs that need to be included.
On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan <su...@apache.org> wrote:
> Hi Wei-Chiu,
>
> What will be the next steps here for 3.3.1 planning?
>
> Thanks
> Sunil
>
> On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
>
> > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran
> <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > >
> > > Regarding blockers &c: how about we have a little hackathon where we
> try
> > > and get things in. This means a promise of review time from the people
> > with
> > > commit rights and other people who understand the code (Stack?)
> > >
> > >
> >
> > I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
> > Thanks,
> > S
> >
> >
> >
> >
> > > -steve
> > >
> > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
> > >
> > > > +1
> > > > Just to mention we would need to release hadoop-thirdparty too
> before.
> > > > Presently we are using the snapshot version of it.
> > > >
> > > > -Ayush
> > > >
> > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org>
> > > wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > Hadoop 3.3.0 was released half a year ago, and as of now we've
> > > > accumulated
> > > > > more than 400 changes in the branch-3.3. A number of downstreamers
> > are
> > > > > eagerly waiting for 3.3.1 which addresses the guava version
> conflict
> > > > issue.
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > > >
> > > > > We should start the release work for 3.3.1 before the diff becomes
> > even
> > > > > larger.
> > > > >
> > > > > I believe there are currently only two real blockers for a 3.3.1
> > > (using
> > > > > this filter
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > > )
> > > > >
> > > > >
> > > > > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > > > > 2.
> > > > > 1. HADOOP-17112 <
> > > https://issues.apache.org/jira/browse/HADOOP-17112
> > > > >
> > > > > 2.
> > > > >
> > > > >
> > > > >
> > > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > > > >
> > > > > Also, the HowToRelease wiki does not describe the ARM build
> process.
> > > > That's
> > > > > going to be important for future releases.
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> > > >
> > > >
> > >
> >
>
--
--Brahma Reddy Battula
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Sunil Govindan <su...@apache.org>.
Hi Wei-Chiu,
What will be the next steps here for 3.3.1 planning?
Thanks
Sunil
On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> >
> > Regarding blockers &c: how about we have a little hackathon where we try
> > and get things in. This means a promise of review time from the people
> with
> > commit rights and other people who understand the code (Stack?)
> >
> >
>
> I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
> Thanks,
> S
>
>
>
>
> > -steve
> >
> > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
> >
> > > +1
> > > Just to mention we would need to release hadoop-thirdparty too before.
> > > Presently we are using the snapshot version of it.
> > >
> > > -Ayush
> > >
> > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org>
> > wrote:
> > > >
> > > > Hi all,
> > > >
> > > > Hadoop 3.3.0 was released half a year ago, and as of now we've
> > > accumulated
> > > > more than 400 changes in the branch-3.3. A number of downstreamers
> are
> > > > eagerly waiting for 3.3.1 which addresses the guava version conflict
> > > issue.
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > >
> > > > We should start the release work for 3.3.1 before the diff becomes
> even
> > > > larger.
> > > >
> > > > I believe there are currently only two real blockers for a 3.3.1
> > (using
> > > > this filter
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > )
> > > >
> > > >
> > > > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > > > 2.
> > > > 1. HADOOP-17112 <
> > https://issues.apache.org/jira/browse/HADOOP-17112
> > > >
> > > > 2.
> > > >
> > > >
> > > >
> > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > > >
> > > > Also, the HowToRelease wiki does not describe the ARM build process.
> > > That's
> > > > going to be important for future releases.
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Sunil Govindan <su...@apache.org>.
Hi Wei-Chiu,
What will be the next steps here for 3.3.1 planning?
Thanks
Sunil
On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> >
> > Regarding blockers &c: how about we have a little hackathon where we try
> > and get things in. This means a promise of review time from the people
> with
> > commit rights and other people who understand the code (Stack?)
> >
> >
>
> I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
> Thanks,
> S
>
>
>
>
> > -steve
> >
> > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
> >
> > > +1
> > > Just to mention we would need to release hadoop-thirdparty too before.
> > > Presently we are using the snapshot version of it.
> > >
> > > -Ayush
> > >
> > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org>
> > wrote:
> > > >
> > > > Hi all,
> > > >
> > > > Hadoop 3.3.0 was released half a year ago, and as of now we've
> > > accumulated
> > > > more than 400 changes in the branch-3.3. A number of downstreamers
> are
> > > > eagerly waiting for 3.3.1 which addresses the guava version conflict
> > > issue.
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > >
> > > > We should start the release work for 3.3.1 before the diff becomes
> even
> > > > larger.
> > > >
> > > > I believe there are currently only two real blockers for a 3.3.1
> > (using
> > > > this filter
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > )
> > > >
> > > >
> > > > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > > > 2.
> > > > 1. HADOOP-17112 <
> > https://issues.apache.org/jira/browse/HADOOP-17112
> > > >
> > > > 2.
> > > >
> > > >
> > > >
> > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > > >
> > > > Also, the HowToRelease wiki does not describe the ARM build process.
> > > That's
> > > > going to be important for future releases.
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Sunil Govindan <su...@apache.org>.
Hi Wei-Chiu,
What will be the next steps here for 3.3.1 planning?
Thanks
Sunil
On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> >
> > Regarding blockers &c: how about we have a little hackathon where we try
> > and get things in. This means a promise of review time from the people
> with
> > commit rights and other people who understand the code (Stack?)
> >
> >
>
> I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
> Thanks,
> S
>
>
>
>
> > -steve
> >
> > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
> >
> > > +1
> > > Just to mention we would need to release hadoop-thirdparty too before.
> > > Presently we are using the snapshot version of it.
> > >
> > > -Ayush
> > >
> > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org>
> > wrote:
> > > >
> > > > Hi all,
> > > >
> > > > Hadoop 3.3.0 was released half a year ago, and as of now we've
> > > accumulated
> > > > more than 400 changes in the branch-3.3. A number of downstreamers
> are
> > > > eagerly waiting for 3.3.1 which addresses the guava version conflict
> > > issue.
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > >
> > > > We should start the release work for 3.3.1 before the diff becomes
> even
> > > > larger.
> > > >
> > > > I believe there are currently only two real blockers for a 3.3.1
> > (using
> > > > this filter
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > )
> > > >
> > > >
> > > > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > > > 2.
> > > > 1. HADOOP-17112 <
> > https://issues.apache.org/jira/browse/HADOOP-17112
> > > >
> > > > 2.
> > > >
> > > >
> > > >
> > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > > >
> > > > Also, the HowToRelease wiki does not describe the ARM build process.
> > > That's
> > > > going to be important for future releases.
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Sunil Govindan <su...@apache.org>.
Hi Wei-Chiu,
What will be the next steps here for 3.3.1 planning?
Thanks
Sunil
On Mon, Feb 8, 2021 at 11:56 PM Stack <st...@duboce.net> wrote:
> On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> >
> > Regarding blockers &c: how about we have a little hackathon where we try
> > and get things in. This means a promise of review time from the people
> with
> > commit rights and other people who understand the code (Stack?)
> >
> >
>
> I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
> Thanks,
> S
>
>
>
>
> > -steve
> >
> > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
> >
> > > +1
> > > Just to mention we would need to release hadoop-thirdparty too before.
> > > Presently we are using the snapshot version of it.
> > >
> > > -Ayush
> > >
> > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org>
> > wrote:
> > > >
> > > > Hi all,
> > > >
> > > > Hadoop 3.3.0 was released half a year ago, and as of now we've
> > > accumulated
> > > > more than 400 changes in the branch-3.3. A number of downstreamers
> are
> > > > eagerly waiting for 3.3.1 which addresses the guava version conflict
> > > issue.
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > > >
> > > > We should start the release work for 3.3.1 before the diff becomes
> even
> > > > larger.
> > > >
> > > > I believe there are currently only two real blockers for a 3.3.1
> > (using
> > > > this filter
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > > )
> > > >
> > > >
> > > > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > > > 2.
> > > > 1. HADOOP-17112 <
> > https://issues.apache.org/jira/browse/HADOOP-17112
> > > >
> > > > 2.
> > > >
> > > >
> > > >
> > > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > > >
> > > > Also, the HowToRelease wiki does not describe the ARM build process.
> > > That's
> > > > going to be important for future releases.
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Stack <st...@duboce.net>.
On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran <st...@cloudera.com.invalid>
wrote:
>
> Regarding blockers &c: how about we have a little hackathon where we try
> and get things in. This means a promise of review time from the people with
> commit rights and other people who understand the code (Stack?)
>
>
I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
Thanks,
S
> -steve
>
> On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
>
> > +1
> > Just to mention we would need to release hadoop-thirdparty too before.
> > Presently we are using the snapshot version of it.
> >
> > -Ayush
> >
> > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org>
> wrote:
> > >
> > > Hi all,
> > >
> > > Hadoop 3.3.0 was released half a year ago, and as of now we've
> > accumulated
> > > more than 400 changes in the branch-3.3. A number of downstreamers are
> > > eagerly waiting for 3.3.1 which addresses the guava version conflict
> > issue.
> > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > >
> > > We should start the release work for 3.3.1 before the diff becomes even
> > > larger.
> > >
> > > I believe there are currently only two real blockers for a 3.3.1
> (using
> > > this filter
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > )
> > >
> > >
> > > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > > 2.
> > > 1. HADOOP-17112 <
> https://issues.apache.org/jira/browse/HADOOP-17112
> > >
> > > 2.
> > >
> > >
> > >
> > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > >
> > > Also, the HowToRelease wiki does not describe the ARM build process.
> > That's
> > > going to be important for future releases.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Stack <st...@duboce.net>.
On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran <st...@cloudera.com.invalid>
wrote:
>
> Regarding blockers &c: how about we have a little hackathon where we try
> and get things in. This means a promise of review time from the people with
> commit rights and other people who understand the code (Stack?)
>
>
I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
Thanks,
S
> -steve
>
> On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
>
> > +1
> > Just to mention we would need to release hadoop-thirdparty too before.
> > Presently we are using the snapshot version of it.
> >
> > -Ayush
> >
> > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org>
> wrote:
> > >
> > > Hi all,
> > >
> > > Hadoop 3.3.0 was released half a year ago, and as of now we've
> > accumulated
> > > more than 400 changes in the branch-3.3. A number of downstreamers are
> > > eagerly waiting for 3.3.1 which addresses the guava version conflict
> > issue.
> > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > >
> > > We should start the release work for 3.3.1 before the diff becomes even
> > > larger.
> > >
> > > I believe there are currently only two real blockers for a 3.3.1
> (using
> > > this filter
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > )
> > >
> > >
> > > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > > 2.
> > > 1. HADOOP-17112 <
> https://issues.apache.org/jira/browse/HADOOP-17112
> > >
> > > 2.
> > >
> > >
> > >
> > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > >
> > > Also, the HowToRelease wiki does not describe the ARM build process.
> > That's
> > > going to be important for future releases.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Stack <st...@duboce.net>.
On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran <st...@cloudera.com.invalid>
wrote:
>
> Regarding blockers &c: how about we have a little hackathon where we try
> and get things in. This means a promise of review time from the people with
> commit rights and other people who understand the code (Stack?)
>
>
I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
Thanks,
S
> -steve
>
> On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
>
> > +1
> > Just to mention we would need to release hadoop-thirdparty too before.
> > Presently we are using the snapshot version of it.
> >
> > -Ayush
> >
> > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org>
> wrote:
> > >
> > > Hi all,
> > >
> > > Hadoop 3.3.0 was released half a year ago, and as of now we've
> > accumulated
> > > more than 400 changes in the branch-3.3. A number of downstreamers are
> > > eagerly waiting for 3.3.1 which addresses the guava version conflict
> > issue.
> > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > >
> > > We should start the release work for 3.3.1 before the diff becomes even
> > > larger.
> > >
> > > I believe there are currently only two real blockers for a 3.3.1
> (using
> > > this filter
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > )
> > >
> > >
> > > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > > 2.
> > > 1. HADOOP-17112 <
> https://issues.apache.org/jira/browse/HADOOP-17112
> > >
> > > 2.
> > >
> > >
> > >
> > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > >
> > > Also, the HowToRelease wiki does not describe the ARM build process.
> > That's
> > > going to be important for future releases.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Stack <st...@duboce.net>.
On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran <st...@cloudera.com.invalid>
wrote:
>
> Regarding blockers &c: how about we have a little hackathon where we try
> and get things in. This means a promise of review time from the people with
> commit rights and other people who understand the code (Stack?)
>
>
I'm up for helping get 3.3.1 out (reviewing, hackathon, testing).
Thanks,
S
> -steve
>
> On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
>
> > +1
> > Just to mention we would need to release hadoop-thirdparty too before.
> > Presently we are using the snapshot version of it.
> >
> > -Ayush
> >
> > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org>
> wrote:
> > >
> > > Hi all,
> > >
> > > Hadoop 3.3.0 was released half a year ago, and as of now we've
> > accumulated
> > > more than 400 changes in the branch-3.3. A number of downstreamers are
> > > eagerly waiting for 3.3.1 which addresses the guava version conflict
> > issue.
> > >
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> > >
> > > We should start the release work for 3.3.1 before the diff becomes even
> > > larger.
> > >
> > > I believe there are currently only two real blockers for a 3.3.1
> (using
> > > this filter
> > >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > > )
> > >
> > >
> > > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > > 2.
> > > 1. HADOOP-17112 <
> https://issues.apache.org/jira/browse/HADOOP-17112
> > >
> > > 2.
> > >
> > >
> > >
> > > Is there anyone who would volunteer to be the 3.3.1 RM?
> > >
> > > Also, the HowToRelease wiki does not describe the ARM build process.
> > That's
> > > going to be important for future releases.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >
> >
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
yeah, it's time.I've been trying to get things lined up for this
object-store wise, a few things tagged as blockers where I really want them
in
Wei Chiu -if you want to be the release manager I'll share the workload,
Regarding blockers &c: how about we have a little hackathon where we try
and get things in. This means a promise of review time from the people with
commit rights and other people who understand the code (Stack?)
-steve
On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
> +1
> Just to mention we would need to release hadoop-thirdparty too before.
> Presently we are using the snapshot version of it.
>
> -Ayush
>
> > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org> wrote:
> >
> > Hi all,
> >
> > Hadoop 3.3.0 was released half a year ago, and as of now we've
> accumulated
> > more than 400 changes in the branch-3.3. A number of downstreamers are
> > eagerly waiting for 3.3.1 which addresses the guava version conflict
> issue.
> >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> >
> > We should start the release work for 3.3.1 before the diff becomes even
> > larger.
> >
> > I believe there are currently only two real blockers for a 3.3.1 (using
> > this filter
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > )
> >
> >
> > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > 2.
> > 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112
> >
> > 2.
> >
> >
> >
> > Is there anyone who would volunteer to be the 3.3.1 RM?
> >
> > Also, the HowToRelease wiki does not describe the ARM build process.
> That's
> > going to be important for future releases.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
yeah, it's time.I've been trying to get things lined up for this
object-store wise, a few things tagged as blockers where I really want them
in
Wei Chiu -if you want to be the release manager I'll share the workload,
Regarding blockers &c: how about we have a little hackathon where we try
and get things in. This means a promise of review time from the people with
commit rights and other people who understand the code (Stack?)
-steve
On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
> +1
> Just to mention we would need to release hadoop-thirdparty too before.
> Presently we are using the snapshot version of it.
>
> -Ayush
>
> > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org> wrote:
> >
> > Hi all,
> >
> > Hadoop 3.3.0 was released half a year ago, and as of now we've
> accumulated
> > more than 400 changes in the branch-3.3. A number of downstreamers are
> > eagerly waiting for 3.3.1 which addresses the guava version conflict
> issue.
> >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> >
> > We should start the release work for 3.3.1 before the diff becomes even
> > larger.
> >
> > I believe there are currently only two real blockers for a 3.3.1 (using
> > this filter
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > )
> >
> >
> > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > 2.
> > 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112
> >
> > 2.
> >
> >
> >
> > Is there anyone who would volunteer to be the 3.3.1 RM?
> >
> > Also, the HowToRelease wiki does not describe the ARM build process.
> That's
> > going to be important for future releases.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
yeah, it's time.I've been trying to get things lined up for this
object-store wise, a few things tagged as blockers where I really want them
in
Wei Chiu -if you want to be the release manager I'll share the workload,
Regarding blockers &c: how about we have a little hackathon where we try
and get things in. This means a promise of review time from the people with
commit rights and other people who understand the code (Stack?)
-steve
On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
> +1
> Just to mention we would need to release hadoop-thirdparty too before.
> Presently we are using the snapshot version of it.
>
> -Ayush
>
> > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org> wrote:
> >
> > Hi all,
> >
> > Hadoop 3.3.0 was released half a year ago, and as of now we've
> accumulated
> > more than 400 changes in the branch-3.3. A number of downstreamers are
> > eagerly waiting for 3.3.1 which addresses the guava version conflict
> issue.
> >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> >
> > We should start the release work for 3.3.1 before the diff becomes even
> > larger.
> >
> > I believe there are currently only two real blockers for a 3.3.1 (using
> > this filter
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > )
> >
> >
> > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > 2.
> > 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112
> >
> > 2.
> >
> >
> >
> > Is there anyone who would volunteer to be the 3.3.1 RM?
> >
> > Also, the HowToRelease wiki does not describe the ARM build process.
> That's
> > going to be important for future releases.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
yeah, it's time.I've been trying to get things lined up for this
object-store wise, a few things tagged as blockers where I really want them
in
Wei Chiu -if you want to be the release manager I'll share the workload,
Regarding blockers &c: how about we have a little hackathon where we try
and get things in. This means a promise of review time from the people with
commit rights and other people who understand the code (Stack?)
-steve
On Thu, 28 Jan 2021 at 06:48, Ayush Saxena <ay...@gmail.com> wrote:
> +1
> Just to mention we would need to release hadoop-thirdparty too before.
> Presently we are using the snapshot version of it.
>
> -Ayush
>
> > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org> wrote:
> >
> > Hi all,
> >
> > Hadoop 3.3.0 was released half a year ago, and as of now we've
> accumulated
> > more than 400 changes in the branch-3.3. A number of downstreamers are
> > eagerly waiting for 3.3.1 which addresses the guava version conflict
> issue.
> >
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
> >
> > We should start the release work for 3.3.1 before the diff becomes even
> > larger.
> >
> > I believe there are currently only two real blockers for a 3.3.1 (using
> > this filter
> >
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> > )
> >
> >
> > 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> > 2.
> > 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112
> >
> > 2.
> >
> >
> >
> > Is there anyone who would volunteer to be the 3.3.1 RM?
> >
> > Also, the HowToRelease wiki does not describe the ARM build process.
> That's
> > going to be important for future releases.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Ayush Saxena <ay...@gmail.com>.
+1
Just to mention we would need to release hadoop-thirdparty too before. Presently we are using the snapshot version of it.
-Ayush
> On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org> wrote:
>
> Hi all,
>
> Hadoop 3.3.0 was released half a year ago, and as of now we've accumulated
> more than 400 changes in the branch-3.3. A number of downstreamers are
> eagerly waiting for 3.3.1 which addresses the guava version conflict issue.
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>
> We should start the release work for 3.3.1 before the diff becomes even
> larger.
>
> I believe there are currently only two real blockers for a 3.3.1 (using
> this filter
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> )
>
>
> 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> 2.
> 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112>
> 2.
>
>
>
> Is there anyone who would volunteer to be the 3.3.1 RM?
>
> Also, the HowToRelease wiki does not describe the ARM build process. That's
> going to be important for future releases.
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by David <da...@gmail.com>.
Would be great for the Hive team to see this one as well:
https://issues.apache.org/jira/browse/HDFS-15790
On Wed, Jan 27, 2021 at 8:28 PM Wei-Chiu Chuang <we...@apache.org> wrote:
> Hi all,
>
> Hadoop 3.3.0 was released half a year ago, and as of now we've accumulated
> more than 400 changes in the branch-3.3. A number of downstreamers are
> eagerly waiting for 3.3.1 which addresses the guava version conflict issue.
>
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>
> We should start the release work for 3.3.1 before the diff becomes even
> larger.
>
> I believe there are currently only two real blockers for a 3.3.1 (using
> this filter
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> )
>
>
> 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> 2.
> 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112>
> 2.
>
>
>
> Is there anyone who would volunteer to be the 3.3.1 RM?
>
> Also, the HowToRelease wiki does not describe the ARM build process. That's
> going to be important for future releases.
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Stack <st...@duboce.net>.
Thanks for bringing up the topic Wei-Chiu. +1 on a 3.3.1 soon.
Was going to spend time testing....
Yours,
S
On Wed, Jan 27, 2021 at 5:28 PM Wei-Chiu Chuang <we...@apache.org> wrote:
> Hi all,
>
> Hadoop 3.3.0 was released half a year ago, and as of now we've accumulated
> more than 400 changes in the branch-3.3. A number of downstreamers are
> eagerly waiting for 3.3.1 which addresses the guava version conflict issue.
>
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>
> We should start the release work for 3.3.1 before the diff becomes even
> larger.
>
> I believe there are currently only two real blockers for a 3.3.1 (using
> this filter
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> )
>
>
> 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> 2.
> 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112>
> 2.
>
>
>
> Is there anyone who would volunteer to be the 3.3.1 RM?
>
> Also, the HowToRelease wiki does not describe the ARM build process. That's
> going to be important for future releases.
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by David <da...@gmail.com>.
Would be great for the Hive team to see this one as well:
https://issues.apache.org/jira/browse/HDFS-15790
On Wed, Jan 27, 2021 at 8:28 PM Wei-Chiu Chuang <we...@apache.org> wrote:
> Hi all,
>
> Hadoop 3.3.0 was released half a year ago, and as of now we've accumulated
> more than 400 changes in the branch-3.3. A number of downstreamers are
> eagerly waiting for 3.3.1 which addresses the guava version conflict issue.
>
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>
> We should start the release work for 3.3.1 before the diff becomes even
> larger.
>
> I believe there are currently only two real blockers for a 3.3.1 (using
> this filter
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> )
>
>
> 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> 2.
> 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112>
> 2.
>
>
>
> Is there anyone who would volunteer to be the 3.3.1 RM?
>
> Also, the HowToRelease wiki does not describe the ARM build process. That's
> going to be important for future releases.
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Stack <st...@duboce.net>.
Thanks for bringing up the topic Wei-Chiu. +1 on a 3.3.1 soon.
Was going to spend time testing....
Yours,
S
On Wed, Jan 27, 2021 at 5:28 PM Wei-Chiu Chuang <we...@apache.org> wrote:
> Hi all,
>
> Hadoop 3.3.0 was released half a year ago, and as of now we've accumulated
> more than 400 changes in the branch-3.3. A number of downstreamers are
> eagerly waiting for 3.3.1 which addresses the guava version conflict issue.
>
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>
> We should start the release work for 3.3.1 before the diff becomes even
> larger.
>
> I believe there are currently only two real blockers for a 3.3.1 (using
> this filter
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> )
>
>
> 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> 2.
> 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112>
> 2.
>
>
>
> Is there anyone who would volunteer to be the 3.3.1 RM?
>
> Also, the HowToRelease wiki does not describe the ARM build process. That's
> going to be important for future releases.
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Stack <st...@duboce.net>.
Thanks for bringing up the topic Wei-Chiu. +1 on a 3.3.1 soon.
Was going to spend time testing....
Yours,
S
On Wed, Jan 27, 2021 at 5:28 PM Wei-Chiu Chuang <we...@apache.org> wrote:
> Hi all,
>
> Hadoop 3.3.0 was released half a year ago, and as of now we've accumulated
> more than 400 changes in the branch-3.3. A number of downstreamers are
> eagerly waiting for 3.3.1 which addresses the guava version conflict issue.
>
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>
> We should start the release work for 3.3.1 before the diff becomes even
> larger.
>
> I believe there are currently only two real blockers for a 3.3.1 (using
> this filter
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> )
>
>
> 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> 2.
> 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112>
> 2.
>
>
>
> Is there anyone who would volunteer to be the 3.3.1 RM?
>
> Also, the HowToRelease wiki does not describe the ARM build process. That's
> going to be important for future releases.
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Stack <st...@duboce.net>.
Thanks for bringing up the topic Wei-Chiu. +1 on a 3.3.1 soon.
Was going to spend time testing....
Yours,
S
On Wed, Jan 27, 2021 at 5:28 PM Wei-Chiu Chuang <we...@apache.org> wrote:
> Hi all,
>
> Hadoop 3.3.0 was released half a year ago, and as of now we've accumulated
> more than 400 changes in the branch-3.3. A number of downstreamers are
> eagerly waiting for 3.3.1 which addresses the guava version conflict issue.
>
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>
> We should start the release work for 3.3.1 before the diff becomes even
> larger.
>
> I believe there are currently only two real blockers for a 3.3.1 (using
> this filter
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> )
>
>
> 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> 2.
> 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112>
> 2.
>
>
>
> Is there anyone who would volunteer to be the 3.3.1 RM?
>
> Also, the HowToRelease wiki does not describe the ARM build process. That's
> going to be important for future releases.
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by David <da...@gmail.com>.
Would be great for the Hive team to see this one as well:
https://issues.apache.org/jira/browse/HDFS-15790
On Wed, Jan 27, 2021 at 8:28 PM Wei-Chiu Chuang <we...@apache.org> wrote:
> Hi all,
>
> Hadoop 3.3.0 was released half a year ago, and as of now we've accumulated
> more than 400 changes in the branch-3.3. A number of downstreamers are
> eagerly waiting for 3.3.1 which addresses the guava version conflict issue.
>
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>
> We should start the release work for 3.3.1 before the diff becomes even
> larger.
>
> I believe there are currently only two real blockers for a 3.3.1 (using
> this filter
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> )
>
>
> 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> 2.
> 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112>
> 2.
>
>
>
> Is there anyone who would volunteer to be the 3.3.1 RM?
>
> Also, the HowToRelease wiki does not describe the ARM build process. That's
> going to be important for future releases.
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Ayush Saxena <ay...@gmail.com>.
+1
Just to mention we would need to release hadoop-thirdparty too before. Presently we are using the snapshot version of it.
-Ayush
> On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org> wrote:
>
> Hi all,
>
> Hadoop 3.3.0 was released half a year ago, and as of now we've accumulated
> more than 400 changes in the branch-3.3. A number of downstreamers are
> eagerly waiting for 3.3.1 which addresses the guava version conflict issue.
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>
> We should start the release work for 3.3.1 before the diff becomes even
> larger.
>
> I believe there are currently only two real blockers for a 3.3.1 (using
> this filter
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> )
>
>
> 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> 2.
> 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112>
> 2.
>
>
>
> Is there anyone who would volunteer to be the 3.3.1 RM?
>
> Also, the HowToRelease wiki does not describe the ARM build process. That's
> going to be important for future releases.
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by David <da...@gmail.com>.
Would be great for the Hive team to see this one as well:
https://issues.apache.org/jira/browse/HDFS-15790
On Wed, Jan 27, 2021 at 8:28 PM Wei-Chiu Chuang <we...@apache.org> wrote:
> Hi all,
>
> Hadoop 3.3.0 was released half a year ago, and as of now we've accumulated
> more than 400 changes in the branch-3.3. A number of downstreamers are
> eagerly waiting for 3.3.1 which addresses the guava version conflict issue.
>
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>
> We should start the release work for 3.3.1 before the diff becomes even
> larger.
>
> I believe there are currently only two real blockers for a 3.3.1 (using
> this filter
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> )
>
>
> 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> 2.
> 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112>
> 2.
>
>
>
> Is there anyone who would volunteer to be the 3.3.1 RM?
>
> Also, the HowToRelease wiki does not describe the ARM build process. That's
> going to be important for future releases.
>
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Ayush Saxena <ay...@gmail.com>.
+1
Just to mention we would need to release hadoop-thirdparty too before. Presently we are using the snapshot version of it.
-Ayush
> On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org> wrote:
>
> Hi all,
>
> Hadoop 3.3.0 was released half a year ago, and as of now we've accumulated
> more than 400 changes in the branch-3.3. A number of downstreamers are
> eagerly waiting for 3.3.1 which addresses the guava version conflict issue.
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>
> We should start the release work for 3.3.1 before the diff becomes even
> larger.
>
> I believe there are currently only two real blockers for a 3.3.1 (using
> this filter
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> )
>
>
> 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> 2.
> 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112>
> 2.
>
>
>
> Is there anyone who would volunteer to be the 3.3.1 RM?
>
> Also, the HowToRelease wiki does not describe the ARM build process. That's
> going to be important for future releases.
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [DISCUSS] Hadoop 3.3.1 release
Posted by Ayush Saxena <ay...@gmail.com>.
+1
Just to mention we would need to release hadoop-thirdparty too before. Presently we are using the snapshot version of it.
-Ayush
> On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang <we...@apache.org> wrote:
>
> Hi all,
>
> Hadoop 3.3.0 was released half a year ago, and as of now we've accumulated
> more than 400 changes in the branch-3.3. A number of downstreamers are
> eagerly waiting for 3.3.1 which addresses the guava version conflict issue.
>
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20
>
> We should start the release work for 3.3.1 before the diff becomes even
> larger.
>
> I believe there are currently only two real blockers for a 3.3.1 (using
> this filter
> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC
> )
>
>
> 1. HDFS-15566 <https://issues.apache.org/jira/browse/HDFS-15566>
> 2.
> 1. HADOOP-17112 <https://issues.apache.org/jira/browse/HADOOP-17112>
> 2.
>
>
>
> Is there anyone who would volunteer to be the 3.3.1 RM?
>
> Also, the HowToRelease wiki does not describe the ARM build process. That's
> going to be important for future releases.
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org