You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by slfan1989 <sl...@apache.org> on 2025/11/04 02:03:17 UTC
Re: [DISCUSS] Future Release and Support Strategy for Hadoop
Many thanks to Steve for the feedback. We still have a few upgrade details
to track to ensure a smooth Hadoop 3.5.0 release.
Best Regards,
Shilun Fan
On Thu, Oct 23, 2025 at 3:52 AM Steve Loughran <st...@cloudera.com> wrote:
>
> This is truly awesome. Great work by everyone involved!
>
> It does mean that
>
>
> 1. *anyone* backporting to branch-3.4 MUST do a yetus run to make sure
> things are still all good on java8.
> 2. People can start to learn and play with java17 language and API
> changes. I'd recommend starting with the tests there. And I have no idea
> what the changes are having been stuck in java8...which at least added
> closures.
>
> Again, well done for a massive effort
>
>
>
> On Fri, 17 Oct 2025 at 01:09, slfan1989 <sl...@apache.org> wrote:
>
>> The *trunk branch* has now been fully upgraded to *JDK 17 and JDK 21*. I
>> have updated *hadoop-qbt-trunk-java17-linux-x86_64* to *push build
>> results daily*, and we will continue to monitor the outcomes over the
>> next few days.
>>
>> The following issues and follow-up actions have been identified:
>>
>> 1.
>>
>> After upgrading the *SpotBugs plugin to version 4.9.7*, the
>> verification rules have become stricter, resulting in new SpotBugs
>> warnings. We plan to start refactoring and fixing these issues *early
>> next week*.
>> 2.
>>
>> *Some modules are reporting Javadoc errors*, which will also be
>> addressed *early next week*.
>> 3.
>>
>> The *mvn site build is currently failing*. We will submit a *separate
>> PR* to resolve this issue. *Hualong Zhang* has already completed *JDiff
>> and hadoop-annotation compatibility updates for JDK 17*. Once his PR
>> is merged, the mvn site build is expected to return to normal.
>>
>> Overall, the mainline compilation upgrade has been successfully
>> completed, and the next focus will be on improving code quality and
>> stabilizing the build pipeline.
>>
>> Best Regards, Shilun Fan
>>
>>
>>>
>>>
Re: [DISCUSS] Future Release and Support Strategy for Hadoop
Posted by Chris Nauroth <cn...@apache.org>.
Hi everyone,
This is indeed great news, and thank you to everyone!
I have one more GCS patch in mind for 3.5.0. I'd like to move hadoop-gcp
from hadoop-tools to hadoop-cloud-storage. I think we confused ourselves a
bit in some of the early discussions and made a mistake picking
hadoop-tools. It's important to make the move to hadoop-cloud-storage
before the 3.5.0 release so that we don't get stuck with compatibility
issues trying to make the move later. I'll aim to get a patch together in
the next few days.
Aside from that, it seems that JDiff is the only other open item. That's
getting close to a +1 here:
https://github.com/apache/hadoop/pull/8038
I had volunteered to be release manager for 3.5.0. Is it safe to say that
after inclusion of these 2 remaining items, I can kick off the release
process? Please let me know if there are any other pending items.
Chris Nauroth
On Mon, Nov 3, 2025 at 6:03 PM slfan1989 <sl...@apache.org> wrote:
> Many thanks to Steve for the feedback. We still have a few upgrade details
> to track to ensure a smooth Hadoop 3.5.0 release.
>
> Best Regards,
> Shilun Fan
>
> On Thu, Oct 23, 2025 at 3:52 AM Steve Loughran <st...@cloudera.com>
> wrote:
>
> >
> > This is truly awesome. Great work by everyone involved!
> >
> > It does mean that
> >
> >
> > 1. *anyone* backporting to branch-3.4 MUST do a yetus run to make sure
> > things are still all good on java8.
> > 2. People can start to learn and play with java17 language and API
> > changes. I'd recommend starting with the tests there. And I have no
> idea
> > what the changes are having been stuck in java8...which at least added
> > closures.
> >
> > Again, well done for a massive effort
> >
> >
> >
> > On Fri, 17 Oct 2025 at 01:09, slfan1989 <sl...@apache.org> wrote:
> >
> >> The *trunk branch* has now been fully upgraded to *JDK 17 and JDK 21*. I
> >> have updated *hadoop-qbt-trunk-java17-linux-x86_64* to *push build
> >> results daily*, and we will continue to monitor the outcomes over the
> >> next few days.
> >>
> >> The following issues and follow-up actions have been identified:
> >>
> >> 1.
> >>
> >> After upgrading the *SpotBugs plugin to version 4.9.7*, the
> >> verification rules have become stricter, resulting in new SpotBugs
> >> warnings. We plan to start refactoring and fixing these issues *early
> >> next week*.
> >> 2.
> >>
> >> *Some modules are reporting Javadoc errors*, which will also be
> >> addressed *early next week*.
> >> 3.
> >>
> >> The *mvn site build is currently failing*. We will submit a *separate
> >> PR* to resolve this issue. *Hualong Zhang* has already completed
> *JDiff
> >> and hadoop-annotation compatibility updates for JDK 17*. Once his PR
> >> is merged, the mvn site build is expected to return to normal.
> >>
> >> Overall, the mainline compilation upgrade has been successfully
> >> completed, and the next focus will be on improving code quality and
> >> stabilizing the build pipeline.
> >>
> >> Best Regards, Shilun Fan
> >>
> >>
> >>>
> >>>
>
Re: [DISCUSS] Future Release and Support Strategy for Hadoop
Posted by Chris Nauroth <cn...@apache.org>.
Hi everyone,
This is indeed great news, and thank you to everyone!
I have one more GCS patch in mind for 3.5.0. I'd like to move hadoop-gcp
from hadoop-tools to hadoop-cloud-storage. I think we confused ourselves a
bit in some of the early discussions and made a mistake picking
hadoop-tools. It's important to make the move to hadoop-cloud-storage
before the 3.5.0 release so that we don't get stuck with compatibility
issues trying to make the move later. I'll aim to get a patch together in
the next few days.
Aside from that, it seems that JDiff is the only other open item. That's
getting close to a +1 here:
https://github.com/apache/hadoop/pull/8038
I had volunteered to be release manager for 3.5.0. Is it safe to say that
after inclusion of these 2 remaining items, I can kick off the release
process? Please let me know if there are any other pending items.
Chris Nauroth
On Mon, Nov 3, 2025 at 6:03 PM slfan1989 <sl...@apache.org> wrote:
> Many thanks to Steve for the feedback. We still have a few upgrade details
> to track to ensure a smooth Hadoop 3.5.0 release.
>
> Best Regards,
> Shilun Fan
>
> On Thu, Oct 23, 2025 at 3:52 AM Steve Loughran <st...@cloudera.com>
> wrote:
>
> >
> > This is truly awesome. Great work by everyone involved!
> >
> > It does mean that
> >
> >
> > 1. *anyone* backporting to branch-3.4 MUST do a yetus run to make sure
> > things are still all good on java8.
> > 2. People can start to learn and play with java17 language and API
> > changes. I'd recommend starting with the tests there. And I have no
> idea
> > what the changes are having been stuck in java8...which at least added
> > closures.
> >
> > Again, well done for a massive effort
> >
> >
> >
> > On Fri, 17 Oct 2025 at 01:09, slfan1989 <sl...@apache.org> wrote:
> >
> >> The *trunk branch* has now been fully upgraded to *JDK 17 and JDK 21*. I
> >> have updated *hadoop-qbt-trunk-java17-linux-x86_64* to *push build
> >> results daily*, and we will continue to monitor the outcomes over the
> >> next few days.
> >>
> >> The following issues and follow-up actions have been identified:
> >>
> >> 1.
> >>
> >> After upgrading the *SpotBugs plugin to version 4.9.7*, the
> >> verification rules have become stricter, resulting in new SpotBugs
> >> warnings. We plan to start refactoring and fixing these issues *early
> >> next week*.
> >> 2.
> >>
> >> *Some modules are reporting Javadoc errors*, which will also be
> >> addressed *early next week*.
> >> 3.
> >>
> >> The *mvn site build is currently failing*. We will submit a *separate
> >> PR* to resolve this issue. *Hualong Zhang* has already completed
> *JDiff
> >> and hadoop-annotation compatibility updates for JDK 17*. Once his PR
> >> is merged, the mvn site build is expected to return to normal.
> >>
> >> Overall, the mainline compilation upgrade has been successfully
> >> completed, and the next focus will be on improving code quality and
> >> stabilizing the build pipeline.
> >>
> >> Best Regards, Shilun Fan
> >>
> >>
> >>>
> >>>
>
Re: [DISCUSS] Future Release and Support Strategy for Hadoop
Posted by Chris Nauroth <cn...@apache.org>.
Hi everyone,
This is indeed great news, and thank you to everyone!
I have one more GCS patch in mind for 3.5.0. I'd like to move hadoop-gcp
from hadoop-tools to hadoop-cloud-storage. I think we confused ourselves a
bit in some of the early discussions and made a mistake picking
hadoop-tools. It's important to make the move to hadoop-cloud-storage
before the 3.5.0 release so that we don't get stuck with compatibility
issues trying to make the move later. I'll aim to get a patch together in
the next few days.
Aside from that, it seems that JDiff is the only other open item. That's
getting close to a +1 here:
https://github.com/apache/hadoop/pull/8038
I had volunteered to be release manager for 3.5.0. Is it safe to say that
after inclusion of these 2 remaining items, I can kick off the release
process? Please let me know if there are any other pending items.
Chris Nauroth
On Mon, Nov 3, 2025 at 6:03 PM slfan1989 <sl...@apache.org> wrote:
> Many thanks to Steve for the feedback. We still have a few upgrade details
> to track to ensure a smooth Hadoop 3.5.0 release.
>
> Best Regards,
> Shilun Fan
>
> On Thu, Oct 23, 2025 at 3:52 AM Steve Loughran <st...@cloudera.com>
> wrote:
>
> >
> > This is truly awesome. Great work by everyone involved!
> >
> > It does mean that
> >
> >
> > 1. *anyone* backporting to branch-3.4 MUST do a yetus run to make sure
> > things are still all good on java8.
> > 2. People can start to learn and play with java17 language and API
> > changes. I'd recommend starting with the tests there. And I have no
> idea
> > what the changes are having been stuck in java8...which at least added
> > closures.
> >
> > Again, well done for a massive effort
> >
> >
> >
> > On Fri, 17 Oct 2025 at 01:09, slfan1989 <sl...@apache.org> wrote:
> >
> >> The *trunk branch* has now been fully upgraded to *JDK 17 and JDK 21*. I
> >> have updated *hadoop-qbt-trunk-java17-linux-x86_64* to *push build
> >> results daily*, and we will continue to monitor the outcomes over the
> >> next few days.
> >>
> >> The following issues and follow-up actions have been identified:
> >>
> >> 1.
> >>
> >> After upgrading the *SpotBugs plugin to version 4.9.7*, the
> >> verification rules have become stricter, resulting in new SpotBugs
> >> warnings. We plan to start refactoring and fixing these issues *early
> >> next week*.
> >> 2.
> >>
> >> *Some modules are reporting Javadoc errors*, which will also be
> >> addressed *early next week*.
> >> 3.
> >>
> >> The *mvn site build is currently failing*. We will submit a *separate
> >> PR* to resolve this issue. *Hualong Zhang* has already completed
> *JDiff
> >> and hadoop-annotation compatibility updates for JDK 17*. Once his PR
> >> is merged, the mvn site build is expected to return to normal.
> >>
> >> Overall, the mainline compilation upgrade has been successfully
> >> completed, and the next focus will be on improving code quality and
> >> stabilizing the build pipeline.
> >>
> >> Best Regards, Shilun Fan
> >>
> >>
> >>>
> >>>
>
Re: [DISCUSS] Future Release and Support Strategy for Hadoop
Posted by Chris Nauroth <cn...@apache.org>.
Hi everyone,
This is indeed great news, and thank you to everyone!
I have one more GCS patch in mind for 3.5.0. I'd like to move hadoop-gcp
from hadoop-tools to hadoop-cloud-storage. I think we confused ourselves a
bit in some of the early discussions and made a mistake picking
hadoop-tools. It's important to make the move to hadoop-cloud-storage
before the 3.5.0 release so that we don't get stuck with compatibility
issues trying to make the move later. I'll aim to get a patch together in
the next few days.
Aside from that, it seems that JDiff is the only other open item. That's
getting close to a +1 here:
https://github.com/apache/hadoop/pull/8038
I had volunteered to be release manager for 3.5.0. Is it safe to say that
after inclusion of these 2 remaining items, I can kick off the release
process? Please let me know if there are any other pending items.
Chris Nauroth
On Mon, Nov 3, 2025 at 6:03 PM slfan1989 <sl...@apache.org> wrote:
> Many thanks to Steve for the feedback. We still have a few upgrade details
> to track to ensure a smooth Hadoop 3.5.0 release.
>
> Best Regards,
> Shilun Fan
>
> On Thu, Oct 23, 2025 at 3:52 AM Steve Loughran <st...@cloudera.com>
> wrote:
>
> >
> > This is truly awesome. Great work by everyone involved!
> >
> > It does mean that
> >
> >
> > 1. *anyone* backporting to branch-3.4 MUST do a yetus run to make sure
> > things are still all good on java8.
> > 2. People can start to learn and play with java17 language and API
> > changes. I'd recommend starting with the tests there. And I have no
> idea
> > what the changes are having been stuck in java8...which at least added
> > closures.
> >
> > Again, well done for a massive effort
> >
> >
> >
> > On Fri, 17 Oct 2025 at 01:09, slfan1989 <sl...@apache.org> wrote:
> >
> >> The *trunk branch* has now been fully upgraded to *JDK 17 and JDK 21*. I
> >> have updated *hadoop-qbt-trunk-java17-linux-x86_64* to *push build
> >> results daily*, and we will continue to monitor the outcomes over the
> >> next few days.
> >>
> >> The following issues and follow-up actions have been identified:
> >>
> >> 1.
> >>
> >> After upgrading the *SpotBugs plugin to version 4.9.7*, the
> >> verification rules have become stricter, resulting in new SpotBugs
> >> warnings. We plan to start refactoring and fixing these issues *early
> >> next week*.
> >> 2.
> >>
> >> *Some modules are reporting Javadoc errors*, which will also be
> >> addressed *early next week*.
> >> 3.
> >>
> >> The *mvn site build is currently failing*. We will submit a *separate
> >> PR* to resolve this issue. *Hualong Zhang* has already completed
> *JDiff
> >> and hadoop-annotation compatibility updates for JDK 17*. Once his PR
> >> is merged, the mvn site build is expected to return to normal.
> >>
> >> Overall, the mainline compilation upgrade has been successfully
> >> completed, and the next focus will be on improving code quality and
> >> stabilizing the build pipeline.
> >>
> >> Best Regards, Shilun Fan
> >>
> >>
> >>>
> >>>
>