You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Masatake Iwasaki <iw...@oss.nttdata.com> on 2022/08/01 03:03:57 UTC
Re: [VOTE] Release Apache Hadoop 3.3.4
+1(binding)
* skimmed the contents of site documentation.
* built the source tarball on Rocky Linux 8 (x86_64) by OpenJDK 8 with `-Pnative`.
* launched pseudo distributed cluster including kms and httpfs with Kerberos and SSL enabled.
* created encryption zone, put and read files via httpfs.
* ran example MR wordcount over encryption zone.
* launched 3-node docker cluster with NN-HA and RM-HA enabled and ran some example MR jobs.
* built HBase 2.4.13, Hive 3.1.3 and Spark 3.2.1 against Hadoop 3.3.4 RC1
on CentOS 7 (x86_64) by using Bigtop and ran smoke-tests.
https://github.com/apache/bigtop/pull/964
* HBsae needed to fix relocation rules due to dependency on okhttp added by HDFS-16453.
* built Spark 3.3.0 using the staging repository.
Thanks,
Masatake Iwasaki
On 2022/07/30 3:47, Steve Loughran wrote:
> I have put together a release candidate (RC1) for Hadoop 3.3.4
>
> The RC is available at:
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/
>
> The git tag is release-3.3.4-RC1, commit a585a73c3e0
>
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1358/
>
> You can find my public key at:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Change log
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/CHANGELOG.md
>
> Release notes
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/RELEASENOTES.md
>
> There's a very small number of changes, primarily critical code/packaging
> issues and security fixes.
>
> See the release notes for details.
>
> Please try the release and vote. The vote will run for 5 days.
>
> steve
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.3.4
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
no, not set up for an arm build and i was trying to get this out in a a
hurry. i'll will have to build a new ec2 image for the docker runs there
On Mon, 8 Aug 2022 at 02:41, Cheng Pan <ch...@apache.org> wrote:
> Thanks Steve for this release, I respect the Hadoop community voting
> result, and I will keep working on the OkHttp/Koltin issue for the
> next release.
> Another question is I don't see the ARM tarball on [1], have you published
> it?
>
> [1] https://dist.apache.org/repos/dist/release/hadoop/common/hadoop-3.3.4/
>
> Thanks,
> Cheng Pan
>
> On Thu, Aug 4, 2022 at 2:20 AM Steve Loughran <st...@cloudera.com> wrote:
> >
> > i see you've got a PR up to remove okhttp entirely. hdfs people will
> need to review it, but if you can get it into branch-3.3 it'll be in the
> forthcoming feature release.
> https://issues.apache.org/jira/browse/HDFS-16714
> >
> > On Mon, 1 Aug 2022 at 10:23, Steve Loughran <st...@cloudera.com> wrote:
> >>
> >> i was a bit worried about it too, but apparently its a very small jar
> >>
> >> problem is, we already use okhttp in places, and that update was needed
> >>
> >> On Mon, 1 Aug 2022 at 05:36, Cheng Pan <ch...@apache.org> wrote:
> >>>
> >>> As a downstream consumer, I'm concerning that HDFS-16453 introduced
> Kotlin dependencies in hadoop-hdfs-client, as Masatake Iwasaki said,
> >>>
> >>> > HBsae needed to fix relocation rules due to dependency on okhttp
> added by HDFS-16453.
> >>>
> >>> Given hadoop-common already has apache http client dependencies, why
> should introduce okhttp?
> >>>
> >>> Thanks,
> >>> Cheng Pan
> >>>
> >>> On 2022/08/01 03:03:57 Masatake Iwasaki wrote:
> >>> > +1(binding)
> >>> >
> >>> > * skimmed the contents of site documentation.
> >>> >
> >>> > * built the source tarball on Rocky Linux 8 (x86_64) by OpenJDK 8
> with `-Pnative`.
> >>> >
> >>> > * launched pseudo distributed cluster including kms and httpfs with
> Kerberos and SSL enabled.
> >>> > * created encryption zone, put and read files via httpfs.
> >>> > * ran example MR wordcount over encryption zone.
> >>> >
> >>> > * launched 3-node docker cluster with NN-HA and RM-HA enabled and
> ran some example MR jobs.
> >>> >
> >>> > * built HBase 2.4.13, Hive 3.1.3 and Spark 3.2.1 against Hadoop
> 3.3.4 RC1
> >>> > on CentOS 7 (x86_64) by using Bigtop and ran smoke-tests.
> >>> > https://github.com/apache/bigtop/pull/964
> >>> >
> >>> > * HBsae needed to fix relocation rules due to dependency on
> okhttp added by HDFS-16453.
> >>> >
> >>> > * built Spark 3.3.0 using the staging repository.
> >>> >
> >>> > Thanks,
> >>> > Masatake Iwasaki
> >>> >
> >>> > On 2022/07/30 3:47, Steve Loughran wrote:
> >>> > > I have put together a release candidate (RC1) for Hadoop 3.3.4
> >>> > >
> >>> > > The RC is available at:
> >>> > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/
> >>> > >
> >>> > > The git tag is release-3.3.4-RC1, commit a585a73c3e0
> >>> > >
> >>> > > The maven artifacts are staged at
> >>> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1358/
> >>> > >
> >>> > > You can find my public key at:
> >>> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >>> > >
> >>> > > Change log
> >>> > >
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/CHANGELOG.md
> >>> > >
> >>> > > Release notes
> >>> > >
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/RELEASENOTES.md
> >>> > >
> >>> > > There's a very small number of changes, primarily critical
> code/packaging
> >>> > > issues and security fixes.
> >>> > >
> >>> > > See the release notes for details.
> >>> > >
> >>> > > Please try the release and vote. The vote will run for 5 days.
> >>> > >
> >>> > > steve
> >>> > >
> >>> >
> >>> > ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >>> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>> >
> >>> >
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>>
>
Re: [VOTE] Release Apache Hadoop 3.3.4
Posted by Cheng Pan <ch...@apache.org>.
Thanks Steve for this release, I respect the Hadoop community voting
result, and I will keep working on the OkHttp/Koltin issue for the
next release.
Another question is I don't see the ARM tarball on [1], have you published it?
[1] https://dist.apache.org/repos/dist/release/hadoop/common/hadoop-3.3.4/
Thanks,
Cheng Pan
On Thu, Aug 4, 2022 at 2:20 AM Steve Loughran <st...@cloudera.com> wrote:
>
> i see you've got a PR up to remove okhttp entirely. hdfs people will need to review it, but if you can get it into branch-3.3 it'll be in the forthcoming feature release. https://issues.apache.org/jira/browse/HDFS-16714
>
> On Mon, 1 Aug 2022 at 10:23, Steve Loughran <st...@cloudera.com> wrote:
>>
>> i was a bit worried about it too, but apparently its a very small jar
>>
>> problem is, we already use okhttp in places, and that update was needed
>>
>> On Mon, 1 Aug 2022 at 05:36, Cheng Pan <ch...@apache.org> wrote:
>>>
>>> As a downstream consumer, I'm concerning that HDFS-16453 introduced Kotlin dependencies in hadoop-hdfs-client, as Masatake Iwasaki said,
>>>
>>> > HBsae needed to fix relocation rules due to dependency on okhttp added by HDFS-16453.
>>>
>>> Given hadoop-common already has apache http client dependencies, why should introduce okhttp?
>>>
>>> Thanks,
>>> Cheng Pan
>>>
>>> On 2022/08/01 03:03:57 Masatake Iwasaki wrote:
>>> > +1(binding)
>>> >
>>> > * skimmed the contents of site documentation.
>>> >
>>> > * built the source tarball on Rocky Linux 8 (x86_64) by OpenJDK 8 with `-Pnative`.
>>> >
>>> > * launched pseudo distributed cluster including kms and httpfs with Kerberos and SSL enabled.
>>> > * created encryption zone, put and read files via httpfs.
>>> > * ran example MR wordcount over encryption zone.
>>> >
>>> > * launched 3-node docker cluster with NN-HA and RM-HA enabled and ran some example MR jobs.
>>> >
>>> > * built HBase 2.4.13, Hive 3.1.3 and Spark 3.2.1 against Hadoop 3.3.4 RC1
>>> > on CentOS 7 (x86_64) by using Bigtop and ran smoke-tests.
>>> > https://github.com/apache/bigtop/pull/964
>>> >
>>> > * HBsae needed to fix relocation rules due to dependency on okhttp added by HDFS-16453.
>>> >
>>> > * built Spark 3.3.0 using the staging repository.
>>> >
>>> > Thanks,
>>> > Masatake Iwasaki
>>> >
>>> > On 2022/07/30 3:47, Steve Loughran wrote:
>>> > > I have put together a release candidate (RC1) for Hadoop 3.3.4
>>> > >
>>> > > The RC is available at:
>>> > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/
>>> > >
>>> > > The git tag is release-3.3.4-RC1, commit a585a73c3e0
>>> > >
>>> > > The maven artifacts are staged at
>>> > > https://repository.apache.org/content/repositories/orgapachehadoop-1358/
>>> > >
>>> > > You can find my public key at:
>>> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>> > >
>>> > > Change log
>>> > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/CHANGELOG.md
>>> > >
>>> > > Release notes
>>> > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/RELEASENOTES.md
>>> > >
>>> > > There's a very small number of changes, primarily critical code/packaging
>>> > > issues and security fixes.
>>> > >
>>> > > See the release notes for details.
>>> > >
>>> > > Please try the release and vote. The vote will run for 5 days.
>>> > >
>>> > > steve
>>> > >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.3.4
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
i see you've got a PR up to remove okhttp entirely. hdfs people will need
to review it, but if you can get it into branch-3.3 it'll be in the
forthcoming feature release.
https://issues.apache.org/jira/browse/HDFS-16714
On Mon, 1 Aug 2022 at 10:23, Steve Loughran <st...@cloudera.com> wrote:
> i was a bit worried about it too, but apparently its a very small jar
>
> problem is, we already use okhttp in places, and that update was needed
>
> On Mon, 1 Aug 2022 at 05:36, Cheng Pan <ch...@apache.org> wrote:
>
>> As a downstream consumer, I'm concerning that HDFS-16453 introduced
>> Kotlin dependencies in hadoop-hdfs-client, as Masatake Iwasaki said,
>>
>> > HBsae needed to fix relocation rules due to dependency on okhttp added
>> by HDFS-16453.
>>
>> Given hadoop-common already has apache http client dependencies, why
>> should introduce okhttp?
>>
>> Thanks,
>> Cheng Pan
>>
>> On 2022/08/01 03:03:57 Masatake Iwasaki wrote:
>> > +1(binding)
>> >
>> > * skimmed the contents of site documentation.
>> >
>> > * built the source tarball on Rocky Linux 8 (x86_64) by OpenJDK 8 with
>> `-Pnative`.
>> >
>> > * launched pseudo distributed cluster including kms and httpfs with
>> Kerberos and SSL enabled.
>> > * created encryption zone, put and read files via httpfs.
>> > * ran example MR wordcount over encryption zone.
>> >
>> > * launched 3-node docker cluster with NN-HA and RM-HA enabled and ran
>> some example MR jobs.
>> >
>> > * built HBase 2.4.13, Hive 3.1.3 and Spark 3.2.1 against Hadoop 3.3.4
>> RC1
>> > on CentOS 7 (x86_64) by using Bigtop and ran smoke-tests.
>> > https://github.com/apache/bigtop/pull/964
>> >
>> > * HBsae needed to fix relocation rules due to dependency on okhttp
>> added by HDFS-16453.
>> >
>> > * built Spark 3.3.0 using the staging repository.
>> >
>> > Thanks,
>> > Masatake Iwasaki
>> >
>> > On 2022/07/30 3:47, Steve Loughran wrote:
>> > > I have put together a release candidate (RC1) for Hadoop 3.3.4
>> > >
>> > > The RC is available at:
>> > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/
>> > >
>> > > The git tag is release-3.3.4-RC1, commit a585a73c3e0
>> > >
>> > > The maven artifacts are staged at
>> > >
>> https://repository.apache.org/content/repositories/orgapachehadoop-1358/
>> > >
>> > > You can find my public key at:
>> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> > >
>> > > Change log
>> > >
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/CHANGELOG.md
>> > >
>> > > Release notes
>> > >
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/RELEASENOTES.md
>> > >
>> > > There's a very small number of changes, primarily critical
>> code/packaging
>> > > issues and security fixes.
>> > >
>> > > See the release notes for details.
>> > >
>> > > Please try the release and vote. The vote will run for 5 days.
>> > >
>> > > steve
>> > >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>
>>
Re: [VOTE] Release Apache Hadoop 3.3.4
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
i was a bit worried about it too, but apparently its a very small jar
problem is, we already use okhttp in places, and that update was needed
On Mon, 1 Aug 2022 at 05:36, Cheng Pan <ch...@apache.org> wrote:
> As a downstream consumer, I'm concerning that HDFS-16453 introduced Kotlin
> dependencies in hadoop-hdfs-client, as Masatake Iwasaki said,
>
> > HBsae needed to fix relocation rules due to dependency on okhttp added
> by HDFS-16453.
>
> Given hadoop-common already has apache http client dependencies, why
> should introduce okhttp?
>
> Thanks,
> Cheng Pan
>
> On 2022/08/01 03:03:57 Masatake Iwasaki wrote:
> > +1(binding)
> >
> > * skimmed the contents of site documentation.
> >
> > * built the source tarball on Rocky Linux 8 (x86_64) by OpenJDK 8 with
> `-Pnative`.
> >
> > * launched pseudo distributed cluster including kms and httpfs with
> Kerberos and SSL enabled.
> > * created encryption zone, put and read files via httpfs.
> > * ran example MR wordcount over encryption zone.
> >
> > * launched 3-node docker cluster with NN-HA and RM-HA enabled and ran
> some example MR jobs.
> >
> > * built HBase 2.4.13, Hive 3.1.3 and Spark 3.2.1 against Hadoop 3.3.4 RC1
> > on CentOS 7 (x86_64) by using Bigtop and ran smoke-tests.
> > https://github.com/apache/bigtop/pull/964
> >
> > * HBsae needed to fix relocation rules due to dependency on okhttp
> added by HDFS-16453.
> >
> > * built Spark 3.3.0 using the staging repository.
> >
> > Thanks,
> > Masatake Iwasaki
> >
> > On 2022/07/30 3:47, Steve Loughran wrote:
> > > I have put together a release candidate (RC1) for Hadoop 3.3.4
> > >
> > > The RC is available at:
> > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/
> > >
> > > The git tag is release-3.3.4-RC1, commit a585a73c3e0
> > >
> > > The maven artifacts are staged at
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1358/
> > >
> > > You can find my public key at:
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > >
> > > Change log
> > >
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/CHANGELOG.md
> > >
> > > Release notes
> > >
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/RELEASENOTES.md
> > >
> > > There's a very small number of changes, primarily critical
> code/packaging
> > > issues and security fixes.
> > >
> > > See the release notes for details.
> > >
> > > Please try the release and vote. The vote will run for 5 days.
> > >
> > > steve
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
Re: [VOTE] Release Apache Hadoop 3.3.4
Posted by Cheng Pan <ch...@apache.org>.
As a downstream consumer, I'm concerning that HDFS-16453 introduced Kotlin dependencies in hadoop-hdfs-client, as Masatake Iwasaki said,
> HBsae needed to fix relocation rules due to dependency on okhttp added by HDFS-16453.
Given hadoop-common already has apache http client dependencies, why should introduce okhttp?
Thanks,
Cheng Pan
On 2022/08/01 03:03:57 Masatake Iwasaki wrote:
> +1(binding)
>
> * skimmed the contents of site documentation.
>
> * built the source tarball on Rocky Linux 8 (x86_64) by OpenJDK 8 with `-Pnative`.
>
> * launched pseudo distributed cluster including kms and httpfs with Kerberos and SSL enabled.
> * created encryption zone, put and read files via httpfs.
> * ran example MR wordcount over encryption zone.
>
> * launched 3-node docker cluster with NN-HA and RM-HA enabled and ran some example MR jobs.
>
> * built HBase 2.4.13, Hive 3.1.3 and Spark 3.2.1 against Hadoop 3.3.4 RC1
> on CentOS 7 (x86_64) by using Bigtop and ran smoke-tests.
> https://github.com/apache/bigtop/pull/964
>
> * HBsae needed to fix relocation rules due to dependency on okhttp added by HDFS-16453.
>
> * built Spark 3.3.0 using the staging repository.
>
> Thanks,
> Masatake Iwasaki
>
> On 2022/07/30 3:47, Steve Loughran wrote:
> > I have put together a release candidate (RC1) for Hadoop 3.3.4
> >
> > The RC is available at:
> > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/
> >
> > The git tag is release-3.3.4-RC1, commit a585a73c3e0
> >
> > The maven artifacts are staged at
> > https://repository.apache.org/content/repositories/orgapachehadoop-1358/
> >
> > You can find my public key at:
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> > Change log
> > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/CHANGELOG.md
> >
> > Release notes
> > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.4-RC1/RELEASENOTES.md
> >
> > There's a very small number of changes, primarily critical code/packaging
> > issues and security fixes.
> >
> > See the release notes for details.
> >
> > Please try the release and vote. The vote will run for 5 days.
> >
> > steve
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org