You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kudu.apache.org by Hao Hao <ha...@cloudera.com.INVALID> on 2020/04/16 22:28:00 UTC
[VOTE] Apache Kudu 1.12.0-RC1
Hello Kudu devs!
The Apache Kudu team is happy to announce the first release candidate for
Apache Kudu 1.12.0.
Apache Kudu 1.12.0 is a minor release that offers many improvements and
fixes since the prior release.
This is a source-only release. The artifacts have been staged here:
https://dist.apache.org/repos/dist/dev/kudu/1.12.0-RC1/
Java convenience binaries in the form of a Maven repository are staged here:
https://repository.apache.org/content/repositories/orgapachekudu-1059/
Linux and macOS kudu-binary JAR artifacts are staged here correspondingly:
https://repository.apache.org/content/repositories/orgapachekudu-1060/
https://repository.apache.org/content/repositories/orgapachekudu-1061/
It is tagged in Git as 1.12.0-RC1 and the corresponding hash is the
following:
https://gitbox.apache.org/repos/asf?p=kudu.git;a=commit;h=ae919d457cda36d854f81732b34af2384441d436
The WIP release notes can be found here:
https://gerrit.cloudera.org/c/15685/
The KEYS file to verify the artifact signatures can be found here:
https://dist.apache.org/repos/dist/release/kudu/KEYS
I'd suggest going through the README and the release notes, building Kudu,
and
running the unit tests. Testing out the Maven repo would also be
appreciated.
Also, it's worth running Kudu Java tests against kudu-binary JAR artifact
as described in the commit message here:
https://gitbox.apache.org/repos/asf?p=kudu.git;a=commit;h=8a6faaa93f3e206ac75e8087731daccaf7ab646a
The vote will run until Tuesday Apr 21st 18:00:00 PST 2020. This is a bit
over the suggested 72 hours due to the weekend.
Thank you,
Hao
Re: [VOTE] Apache Kudu 1.12.0-RC1
Posted by Hao Hao <ha...@cloudera.com.INVALID>.
Thanks a lot for all of your votes and response! I agree to do a new RC as
I also encountered the same
error when testing the jars by using the java-example as what Grant
reported on macos High Sierra.
I will cut RC2 with the fix (when available) and the various test fix
patches mentioned in the thread.
Best,
Hao
On Tue, Apr 21, 2020 at 5:51 PM Alexey Serbin <as...@cloudera.com.invalid>
wrote:
> +0
>
> Checked out from the git repo at branch 1.12.x tagged 1.12.0-RC1, compiled
> on macOS HighSierra (10.13), ran tests.
>
> The following tests FAILED:
> 12 - client-test.0 (Failed)
> 47 - types-test (Failed)
> 77 - fs_manager-test (Failed)
> 145 - master_authz-itest.0 (Failed)
> 146 - master_authz-itest.1 (Failed)
> 147 - master_authz-itest.2 (Failed)
> 148 - master_authz-itest.3 (Failed)
> 149 - master_authz-itest.4 (Failed)
> 150 - master_authz-itest.5 (Failed)
> 151 - master_authz-itest.6 (Failed)
> 152 - master_authz-itest.7 (Failed)
> 154 - master_failover-itest.0 (Failed)
> 155 - master_failover-itest.1 (Failed)
> 156 - master_failover-itest.2 (Failed)
> 181 - security-itest (Failed)
> 218 - sentry_authz_provider-test.0 (Failed)
> 219 - sentry_authz_provider-test.1 (Failed)
> 220 - sentry_authz_provider-test.2 (Failed)
> 221 - sentry_authz_provider-test.3 (Failed)
> 222 - sentry_authz_provider-test.4 (Failed)
> 223 - sentry_authz_provider-test.5 (Failed)
> 224 - sentry_authz_provider-test.6 (Failed)
> 225 - sentry_authz_provider-test.7 (Failed)
> 255 - sentry_client-test (Failed)
> 258 - webserver-test (Failed)
> 270 - compaction-test (Failed)
> 323 - kudu-tool-test.0 (Failed)
> 343 - tablet_server-test.2 (Failed)
> 371 - file_cache-stress-test (Failed)
> 423 - trace-test (Failed)
>
> Most of the failures were due to HMS/Sentry issues and that's nothing new,
> some tests we fixed just today (2020-04-21). Not a big deal since macOS is
> a development platform for Kudu.
>
> Checked out from the git repo at branch 1.12.x tagged 1.12.0-RC1, compiled
> on CentOS6.6, ran tests.
> The following tests failed:
>
> master_authz-itest.0
> master_authz-itest.1
> master_authz-itest.2
> master_authz-itest.3
> master_authz-itest.4
> master_authz-itest.5
> master_authz-itest.6
> master_authz-itest.7
> security-itest.txt
>
> The failures in master_authz-itest are due to failure to start Ranger and
> looks like that's test-only issue related to my environment. As for the
> failure in security-itest.txt, I didn't get to the bottom of that yet but
> it seems a test-only issue as well:
>
> src/kudu/integration-tests/security-itest.cc:432: Failure
> Value of: s.ToString()
>
> Expected: has substring "server requires authentication, but client does
> not have Kerberos credentials available"
> Actual: "OK"
>
> Given that we have a few patches with various fixes coming in after tagging
> RC1, it would not hurt cutting RC2 to incorporate those, but since those
> fixes are test-onty, going with RC1 might be an option as well.
>
>
> Thanks,
>
> Alexey
>
>
> On Tue, Apr 21, 2020 at 4:00 PM Attila Bukor <ab...@apache.org> wrote:
>
> > +1, but if this vote fails (deadline is close and we don't have 3 +1s
> > yet), I
> > think it would be best to backport this change and the release notes
> > before we
> > cut RC2.
> >
> > Grant, did you mean to vote on the release or only share your testing
> > results?
> >
> > Attila
> >
> > On Mon, Apr 20, 2020 at 12:36:15PM -0500, Grant Henke wrote:
> > > I forgot to mention that this patch was required to run some of the
> tests
> > > that use min-ranger in Docker:
> > > https://gerrit.cloudera.org/#/c/15756/
> > >
> > > It shouldn't block the release, but we can backport it to branch-1.12.x
> > to
> > > facilitate future testing.
> > >
> > > On Mon, Apr 20, 2020 at 12:22 PM Grant Henke <gr...@apache.org>
> > wrote:
> > >
> > > > I ran the following to test various OS versions in Docker:
> > > >
> > > > *# Build all the images*
> > > >
> > > > *export
> > > > BASES="centos:7,centos:8,ubuntu:xenial,ubuntu:bionic,debian:stretch"*
> > > > *export TARGETS="build"*
> > > > *./docker/docker-build.sh*
> > > >
> > > > *# For each image, run the tests*
> > > >
> > > > *docker run -it --rm apache/kudu:build-latest-<os> /bin/bash*
> > > > *mkdir build/debug*
> > > > *cd build/debug*
> > > > *export NO_REBUILD_THIRDPARTY=1*
> > > > *../../thirdparty/installed/common/bin/cmake \*
> > > > * -DCMAKE_BUILD_TYPE=debug ../..*
> > > > *make -j6*
> > > > *ctest -j6*
> > > > *cd /kudu/java/*
> > > > *./gradlew test*
> > > > *cd /kudu/python*
> > > > *export KUDU_HOME="/kudu"*
> > > > *python setup.py build_ext*
> > > > *python setup.py test*
> > > >
> > > > Below are the flaky and failing tests I observed:
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *centos:7 saw flakes in
> > > > TestKuduTable.testFormatRangePartitionsStringColumcentos:8 saw flakes
> > in
> > > > auto_rebalancer-test mini_ranger-test
> > > > DefaultSourceTest.testInsertionMultipleubuntu:xenial saw failures in
> > > > client-stress-test (Failed) memory_gc-itest (Failed)
> > > > raft_consensus-itest.2 (Failed)ubuntu:xenial saw flakes in
> > > > auto_rebalancer-testubuntu:bionic saw flakes in
> > > > mini_ranger-testdebian:stretch saw flakes in kudu-tool-test.1*
> > > >
> > > >
> > > > * ITClientStress.testManyShortClientsGeneratingScanTokens
> > > > TestMiniKuduCluster.testHiveMetastoreIntegration
> > > > TestMiniKuduCluster.testKerberos*
> > > >
> > > >
> > > > I seem to remember that those tests that are falling in xenial have
> > always
> > > > been an issue in
> > > > Docker and are likely a Docker image/setup issue. I don't think they
> > are a
> > > > concern for this
> > > > release, but I will investigate further.
> > > >
> > > > I tested the jars and artifacts by using the java-example on both Mac
> > and
> > > > Linux.
> > > > I updated the pom to use the staged repositories and version 1.12.0
> and
> > > > ran `mvn test`:
> > > >
> https://github.com/apache/kudu/tree/master/examples/java/java-example
> > > >
> > > > Things worked as expected on Linux, but on a Mac I found an issue
> with
> > > > libc++ linking. I haven't dug into the issue yet.
> > > > I am on Catalina, perhaps someone with an older MacOS version can
> test
> > too?
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *[main] INFO
> > > > org.apache.kudu.shaded.com.google.gradle.osdetector.OsDetector -
> > > > os.detected.classifier=osx-x86_64[main] INFO
> > > > org.apache.kudu.test.cluster.KuduBinaryLocator - Using Kudu binary
> jar
> > > > directory:
> > > >
> >
> /var/folders/3y/nrs6__c10n107f357n7yx2100000gp/T/kudu-binary-jar6249991106299083358[main]
> > > > INFO org.apache.kudu.test.cluster.MiniKuduCluster - Starting process:
> > > >
> >
> [/var/folders/3y/nrs6__c10n107f357n7yx2100000gp/T/kudu-binary-jar6249991106299083358/kudu-binary-1.12.0-osx-x86_64/bin/kudu,
> > > > test, mini_cluster, --serialization=pb][cluster stderr printer] INFO
> > > > org.apache.kudu.test.cluster.MiniKuduCluster - dyld: Library not
> > loaded:
> > > > @rpath/libc++abi.1.dylib[cluster stderr printer] INFO
> > > > org.apache.kudu.test.cluster.MiniKuduCluster - Referenced from:
> > > >
> >
> /private/var/folders/3y/nrs6__c10n107f357n7yx2100000gp/T/kudu-binary-jar6249991106299083358/kudu-binary-1.12.0-osx-x86_64/lib/libglog.0.dylib[cluster
> > > > stderr printer] INFO org.apache.kudu.test.cluster.MiniKuduCluster -
> > > > Reason: image not found[main] ERROR
> > org.apache.kudu.test.junit.RetryRule -
> > > > org.apache.kudu.examples.ExampleTest.testCreateExampleTable: failed
> > attempt
> > > > 1java.io.EOFException*
> > > >
> > > >
> > > > On Thu, Apr 16, 2020 at 5:37 PM Hao Hao <hao.hao@cloudera.com.invalid
> >
> > > > wrote:
> > > >
> > > >> Hello Kudu devs!
> > > >>
> > > >> The Apache Kudu team is happy to announce the first release
> candidate
> > for
> > > >> Apache Kudu 1.12.0.
> > > >>
> > > >> Apache Kudu 1.12.0 is a minor release that offers many improvements
> > and
> > > >> fixes since the prior release.
> > > >>
> > > >> This is a source-only release. The artifacts have been staged here:
> > > >> https://dist.apache.org/repos/dist/dev/kudu/1.12.0-RC1/
> > > >>
> > > >> Java convenience binaries in the form of a Maven repository are
> staged
> > > >> here:
> > > >>
> > https://repository.apache.org/content/repositories/orgapachekudu-1059/
> > > >>
> > > >> Linux and macOS kudu-binary JAR artifacts are staged here
> > correspondingly:
> > > >>
> > https://repository.apache.org/content/repositories/orgapachekudu-1060/
> > > >>
> > https://repository.apache.org/content/repositories/orgapachekudu-1061/
> > > >>
> > > >> It is tagged in Git as 1.12.0-RC1 and the corresponding hash is the
> > > >> following:
> > > >>
> > > >>
> >
> https://gitbox.apache.org/repos/asf?p=kudu.git;a=commit;h=ae919d457cda36d854f81732b34af2384441d436
> > > >>
> > > >> The WIP release notes can be found here:
> > > >> https://gerrit.cloudera.org/c/15685/
> > > >>
> > > >> The KEYS file to verify the artifact signatures can be found here:
> > > >> https://dist.apache.org/repos/dist/release/kudu/KEYS
> > > >>
> > > >> I'd suggest going through the README and the release notes, building
> > Kudu,
> > > >> and
> > > >> running the unit tests. Testing out the Maven repo would also be
> > > >> appreciated.
> > > >> Also, it's worth running Kudu Java tests against kudu-binary JAR
> > artifact
> > > >> as described in the commit message here:
> > > >>
> > > >>
> > > >>
> >
> https://gitbox.apache.org/repos/asf?p=kudu.git;a=commit;h=8a6faaa93f3e206ac75e8087731daccaf7ab646a
> > > >>
> > > >> The vote will run until Tuesday Apr 21st 18:00:00 PST 2020. This is
> a
> > bit
> > > >> over the suggested 72 hours due to the weekend.
> > > >>
> > > >> Thank you,
> > > >> Hao
> > > >>
> > > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
Re: [VOTE] Apache Kudu 1.12.0-RC1
Posted by Alexey Serbin <as...@cloudera.com.INVALID>.
+0
Checked out from the git repo at branch 1.12.x tagged 1.12.0-RC1, compiled
on macOS HighSierra (10.13), ran tests.
The following tests FAILED:
12 - client-test.0 (Failed)
47 - types-test (Failed)
77 - fs_manager-test (Failed)
145 - master_authz-itest.0 (Failed)
146 - master_authz-itest.1 (Failed)
147 - master_authz-itest.2 (Failed)
148 - master_authz-itest.3 (Failed)
149 - master_authz-itest.4 (Failed)
150 - master_authz-itest.5 (Failed)
151 - master_authz-itest.6 (Failed)
152 - master_authz-itest.7 (Failed)
154 - master_failover-itest.0 (Failed)
155 - master_failover-itest.1 (Failed)
156 - master_failover-itest.2 (Failed)
181 - security-itest (Failed)
218 - sentry_authz_provider-test.0 (Failed)
219 - sentry_authz_provider-test.1 (Failed)
220 - sentry_authz_provider-test.2 (Failed)
221 - sentry_authz_provider-test.3 (Failed)
222 - sentry_authz_provider-test.4 (Failed)
223 - sentry_authz_provider-test.5 (Failed)
224 - sentry_authz_provider-test.6 (Failed)
225 - sentry_authz_provider-test.7 (Failed)
255 - sentry_client-test (Failed)
258 - webserver-test (Failed)
270 - compaction-test (Failed)
323 - kudu-tool-test.0 (Failed)
343 - tablet_server-test.2 (Failed)
371 - file_cache-stress-test (Failed)
423 - trace-test (Failed)
Most of the failures were due to HMS/Sentry issues and that's nothing new,
some tests we fixed just today (2020-04-21). Not a big deal since macOS is
a development platform for Kudu.
Checked out from the git repo at branch 1.12.x tagged 1.12.0-RC1, compiled
on CentOS6.6, ran tests.
The following tests failed:
master_authz-itest.0
master_authz-itest.1
master_authz-itest.2
master_authz-itest.3
master_authz-itest.4
master_authz-itest.5
master_authz-itest.6
master_authz-itest.7
security-itest.txt
The failures in master_authz-itest are due to failure to start Ranger and
looks like that's test-only issue related to my environment. As for the
failure in security-itest.txt, I didn't get to the bottom of that yet but
it seems a test-only issue as well:
src/kudu/integration-tests/security-itest.cc:432: Failure
Value of: s.ToString()
Expected: has substring "server requires authentication, but client does
not have Kerberos credentials available"
Actual: "OK"
Given that we have a few patches with various fixes coming in after tagging
RC1, it would not hurt cutting RC2 to incorporate those, but since those
fixes are test-onty, going with RC1 might be an option as well.
Thanks,
Alexey
On Tue, Apr 21, 2020 at 4:00 PM Attila Bukor <ab...@apache.org> wrote:
> +1, but if this vote fails (deadline is close and we don't have 3 +1s
> yet), I
> think it would be best to backport this change and the release notes
> before we
> cut RC2.
>
> Grant, did you mean to vote on the release or only share your testing
> results?
>
> Attila
>
> On Mon, Apr 20, 2020 at 12:36:15PM -0500, Grant Henke wrote:
> > I forgot to mention that this patch was required to run some of the tests
> > that use min-ranger in Docker:
> > https://gerrit.cloudera.org/#/c/15756/
> >
> > It shouldn't block the release, but we can backport it to branch-1.12.x
> to
> > facilitate future testing.
> >
> > On Mon, Apr 20, 2020 at 12:22 PM Grant Henke <gr...@apache.org>
> wrote:
> >
> > > I ran the following to test various OS versions in Docker:
> > >
> > > *# Build all the images*
> > >
> > > *export
> > > BASES="centos:7,centos:8,ubuntu:xenial,ubuntu:bionic,debian:stretch"*
> > > *export TARGETS="build"*
> > > *./docker/docker-build.sh*
> > >
> > > *# For each image, run the tests*
> > >
> > > *docker run -it --rm apache/kudu:build-latest-<os> /bin/bash*
> > > *mkdir build/debug*
> > > *cd build/debug*
> > > *export NO_REBUILD_THIRDPARTY=1*
> > > *../../thirdparty/installed/common/bin/cmake \*
> > > * -DCMAKE_BUILD_TYPE=debug ../..*
> > > *make -j6*
> > > *ctest -j6*
> > > *cd /kudu/java/*
> > > *./gradlew test*
> > > *cd /kudu/python*
> > > *export KUDU_HOME="/kudu"*
> > > *python setup.py build_ext*
> > > *python setup.py test*
> > >
> > > Below are the flaky and failing tests I observed:
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > *centos:7 saw flakes in
> > > TestKuduTable.testFormatRangePartitionsStringColumcentos:8 saw flakes
> in
> > > auto_rebalancer-test mini_ranger-test
> > > DefaultSourceTest.testInsertionMultipleubuntu:xenial saw failures in
> > > client-stress-test (Failed) memory_gc-itest (Failed)
> > > raft_consensus-itest.2 (Failed)ubuntu:xenial saw flakes in
> > > auto_rebalancer-testubuntu:bionic saw flakes in
> > > mini_ranger-testdebian:stretch saw flakes in kudu-tool-test.1*
> > >
> > >
> > > * ITClientStress.testManyShortClientsGeneratingScanTokens
> > > TestMiniKuduCluster.testHiveMetastoreIntegration
> > > TestMiniKuduCluster.testKerberos*
> > >
> > >
> > > I seem to remember that those tests that are falling in xenial have
> always
> > > been an issue in
> > > Docker and are likely a Docker image/setup issue. I don't think they
> are a
> > > concern for this
> > > release, but I will investigate further.
> > >
> > > I tested the jars and artifacts by using the java-example on both Mac
> and
> > > Linux.
> > > I updated the pom to use the staged repositories and version 1.12.0 and
> > > ran `mvn test`:
> > > https://github.com/apache/kudu/tree/master/examples/java/java-example
> > >
> > > Things worked as expected on Linux, but on a Mac I found an issue with
> > > libc++ linking. I haven't dug into the issue yet.
> > > I am on Catalina, perhaps someone with an older MacOS version can test
> too?
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > *[main] INFO
> > > org.apache.kudu.shaded.com.google.gradle.osdetector.OsDetector -
> > > os.detected.classifier=osx-x86_64[main] INFO
> > > org.apache.kudu.test.cluster.KuduBinaryLocator - Using Kudu binary jar
> > > directory:
> > >
> /var/folders/3y/nrs6__c10n107f357n7yx2100000gp/T/kudu-binary-jar6249991106299083358[main]
> > > INFO org.apache.kudu.test.cluster.MiniKuduCluster - Starting process:
> > >
> [/var/folders/3y/nrs6__c10n107f357n7yx2100000gp/T/kudu-binary-jar6249991106299083358/kudu-binary-1.12.0-osx-x86_64/bin/kudu,
> > > test, mini_cluster, --serialization=pb][cluster stderr printer] INFO
> > > org.apache.kudu.test.cluster.MiniKuduCluster - dyld: Library not
> loaded:
> > > @rpath/libc++abi.1.dylib[cluster stderr printer] INFO
> > > org.apache.kudu.test.cluster.MiniKuduCluster - Referenced from:
> > >
> /private/var/folders/3y/nrs6__c10n107f357n7yx2100000gp/T/kudu-binary-jar6249991106299083358/kudu-binary-1.12.0-osx-x86_64/lib/libglog.0.dylib[cluster
> > > stderr printer] INFO org.apache.kudu.test.cluster.MiniKuduCluster -
> > > Reason: image not found[main] ERROR
> org.apache.kudu.test.junit.RetryRule -
> > > org.apache.kudu.examples.ExampleTest.testCreateExampleTable: failed
> attempt
> > > 1java.io.EOFException*
> > >
> > >
> > > On Thu, Apr 16, 2020 at 5:37 PM Hao Hao <ha...@cloudera.com.invalid>
> > > wrote:
> > >
> > >> Hello Kudu devs!
> > >>
> > >> The Apache Kudu team is happy to announce the first release candidate
> for
> > >> Apache Kudu 1.12.0.
> > >>
> > >> Apache Kudu 1.12.0 is a minor release that offers many improvements
> and
> > >> fixes since the prior release.
> > >>
> > >> This is a source-only release. The artifacts have been staged here:
> > >> https://dist.apache.org/repos/dist/dev/kudu/1.12.0-RC1/
> > >>
> > >> Java convenience binaries in the form of a Maven repository are staged
> > >> here:
> > >>
> https://repository.apache.org/content/repositories/orgapachekudu-1059/
> > >>
> > >> Linux and macOS kudu-binary JAR artifacts are staged here
> correspondingly:
> > >>
> https://repository.apache.org/content/repositories/orgapachekudu-1060/
> > >>
> https://repository.apache.org/content/repositories/orgapachekudu-1061/
> > >>
> > >> It is tagged in Git as 1.12.0-RC1 and the corresponding hash is the
> > >> following:
> > >>
> > >>
> https://gitbox.apache.org/repos/asf?p=kudu.git;a=commit;h=ae919d457cda36d854f81732b34af2384441d436
> > >>
> > >> The WIP release notes can be found here:
> > >> https://gerrit.cloudera.org/c/15685/
> > >>
> > >> The KEYS file to verify the artifact signatures can be found here:
> > >> https://dist.apache.org/repos/dist/release/kudu/KEYS
> > >>
> > >> I'd suggest going through the README and the release notes, building
> Kudu,
> > >> and
> > >> running the unit tests. Testing out the Maven repo would also be
> > >> appreciated.
> > >> Also, it's worth running Kudu Java tests against kudu-binary JAR
> artifact
> > >> as described in the commit message here:
> > >>
> > >>
> > >>
> https://gitbox.apache.org/repos/asf?p=kudu.git;a=commit;h=8a6faaa93f3e206ac75e8087731daccaf7ab646a
> > >>
> > >> The vote will run until Tuesday Apr 21st 18:00:00 PST 2020. This is a
> bit
> > >> over the suggested 72 hours due to the weekend.
> > >>
> > >> Thank you,
> > >> Hao
> > >>
> > >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [VOTE] Apache Kudu 1.12.0-RC1
Posted by Attila Bukor <ab...@apache.org>.
+1, but if this vote fails (deadline is close and we don't have 3 +1s yet), I
think it would be best to backport this change and the release notes before we
cut RC2.
Grant, did you mean to vote on the release or only share your testing results?
Attila
On Mon, Apr 20, 2020 at 12:36:15PM -0500, Grant Henke wrote:
> I forgot to mention that this patch was required to run some of the tests
> that use min-ranger in Docker:
> https://gerrit.cloudera.org/#/c/15756/
>
> It shouldn't block the release, but we can backport it to branch-1.12.x to
> facilitate future testing.
>
> On Mon, Apr 20, 2020 at 12:22 PM Grant Henke <gr...@apache.org> wrote:
>
> > I ran the following to test various OS versions in Docker:
> >
> > *# Build all the images*
> >
> > *export
> > BASES="centos:7,centos:8,ubuntu:xenial,ubuntu:bionic,debian:stretch"*
> > *export TARGETS="build"*
> > *./docker/docker-build.sh*
> >
> > *# For each image, run the tests*
> >
> > *docker run -it --rm apache/kudu:build-latest-<os> /bin/bash*
> > *mkdir build/debug*
> > *cd build/debug*
> > *export NO_REBUILD_THIRDPARTY=1*
> > *../../thirdparty/installed/common/bin/cmake \*
> > * -DCMAKE_BUILD_TYPE=debug ../..*
> > *make -j6*
> > *ctest -j6*
> > *cd /kudu/java/*
> > *./gradlew test*
> > *cd /kudu/python*
> > *export KUDU_HOME="/kudu"*
> > *python setup.py build_ext*
> > *python setup.py test*
> >
> > Below are the flaky and failing tests I observed:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *centos:7 saw flakes in
> > TestKuduTable.testFormatRangePartitionsStringColumcentos:8 saw flakes in
> > auto_rebalancer-test mini_ranger-test
> > DefaultSourceTest.testInsertionMultipleubuntu:xenial saw failures in
> > client-stress-test (Failed) memory_gc-itest (Failed)
> > raft_consensus-itest.2 (Failed)ubuntu:xenial saw flakes in
> > auto_rebalancer-testubuntu:bionic saw flakes in
> > mini_ranger-testdebian:stretch saw flakes in kudu-tool-test.1*
> >
> >
> > * ITClientStress.testManyShortClientsGeneratingScanTokens
> > TestMiniKuduCluster.testHiveMetastoreIntegration
> > TestMiniKuduCluster.testKerberos*
> >
> >
> > I seem to remember that those tests that are falling in xenial have always
> > been an issue in
> > Docker and are likely a Docker image/setup issue. I don't think they are a
> > concern for this
> > release, but I will investigate further.
> >
> > I tested the jars and artifacts by using the java-example on both Mac and
> > Linux.
> > I updated the pom to use the staged repositories and version 1.12.0 and
> > ran `mvn test`:
> > https://github.com/apache/kudu/tree/master/examples/java/java-example
> >
> > Things worked as expected on Linux, but on a Mac I found an issue with
> > libc++ linking. I haven't dug into the issue yet.
> > I am on Catalina, perhaps someone with an older MacOS version can test too?
> >
> >
> >
> >
> >
> >
> >
> >
> > *[main] INFO
> > org.apache.kudu.shaded.com.google.gradle.osdetector.OsDetector -
> > os.detected.classifier=osx-x86_64[main] INFO
> > org.apache.kudu.test.cluster.KuduBinaryLocator - Using Kudu binary jar
> > directory:
> > /var/folders/3y/nrs6__c10n107f357n7yx2100000gp/T/kudu-binary-jar6249991106299083358[main]
> > INFO org.apache.kudu.test.cluster.MiniKuduCluster - Starting process:
> > [/var/folders/3y/nrs6__c10n107f357n7yx2100000gp/T/kudu-binary-jar6249991106299083358/kudu-binary-1.12.0-osx-x86_64/bin/kudu,
> > test, mini_cluster, --serialization=pb][cluster stderr printer] INFO
> > org.apache.kudu.test.cluster.MiniKuduCluster - dyld: Library not loaded:
> > @rpath/libc++abi.1.dylib[cluster stderr printer] INFO
> > org.apache.kudu.test.cluster.MiniKuduCluster - Referenced from:
> > /private/var/folders/3y/nrs6__c10n107f357n7yx2100000gp/T/kudu-binary-jar6249991106299083358/kudu-binary-1.12.0-osx-x86_64/lib/libglog.0.dylib[cluster
> > stderr printer] INFO org.apache.kudu.test.cluster.MiniKuduCluster -
> > Reason: image not found[main] ERROR org.apache.kudu.test.junit.RetryRule -
> > org.apache.kudu.examples.ExampleTest.testCreateExampleTable: failed attempt
> > 1java.io.EOFException*
> >
> >
> > On Thu, Apr 16, 2020 at 5:37 PM Hao Hao <ha...@cloudera.com.invalid>
> > wrote:
> >
> >> Hello Kudu devs!
> >>
> >> The Apache Kudu team is happy to announce the first release candidate for
> >> Apache Kudu 1.12.0.
> >>
> >> Apache Kudu 1.12.0 is a minor release that offers many improvements and
> >> fixes since the prior release.
> >>
> >> This is a source-only release. The artifacts have been staged here:
> >> https://dist.apache.org/repos/dist/dev/kudu/1.12.0-RC1/
> >>
> >> Java convenience binaries in the form of a Maven repository are staged
> >> here:
> >> https://repository.apache.org/content/repositories/orgapachekudu-1059/
> >>
> >> Linux and macOS kudu-binary JAR artifacts are staged here correspondingly:
> >> https://repository.apache.org/content/repositories/orgapachekudu-1060/
> >> https://repository.apache.org/content/repositories/orgapachekudu-1061/
> >>
> >> It is tagged in Git as 1.12.0-RC1 and the corresponding hash is the
> >> following:
> >>
> >> https://gitbox.apache.org/repos/asf?p=kudu.git;a=commit;h=ae919d457cda36d854f81732b34af2384441d436
> >>
> >> The WIP release notes can be found here:
> >> https://gerrit.cloudera.org/c/15685/
> >>
> >> The KEYS file to verify the artifact signatures can be found here:
> >> https://dist.apache.org/repos/dist/release/kudu/KEYS
> >>
> >> I'd suggest going through the README and the release notes, building Kudu,
> >> and
> >> running the unit tests. Testing out the Maven repo would also be
> >> appreciated.
> >> Also, it's worth running Kudu Java tests against kudu-binary JAR artifact
> >> as described in the commit message here:
> >>
> >>
> >> https://gitbox.apache.org/repos/asf?p=kudu.git;a=commit;h=8a6faaa93f3e206ac75e8087731daccaf7ab646a
> >>
> >> The vote will run until Tuesday Apr 21st 18:00:00 PST 2020. This is a bit
> >> over the suggested 72 hours due to the weekend.
> >>
> >> Thank you,
> >> Hao
> >>
> >
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [VOTE] Apache Kudu 1.12.0-RC1
Posted by Grant Henke <gh...@cloudera.com.INVALID>.
I forgot to mention that this patch was required to run some of the tests
that use min-ranger in Docker:
https://gerrit.cloudera.org/#/c/15756/
It shouldn't block the release, but we can backport it to branch-1.12.x to
facilitate future testing.
On Mon, Apr 20, 2020 at 12:22 PM Grant Henke <gr...@apache.org> wrote:
> I ran the following to test various OS versions in Docker:
>
> *# Build all the images*
>
> *export
> BASES="centos:7,centos:8,ubuntu:xenial,ubuntu:bionic,debian:stretch"*
> *export TARGETS="build"*
> *./docker/docker-build.sh*
>
> *# For each image, run the tests*
>
> *docker run -it --rm apache/kudu:build-latest-<os> /bin/bash*
> *mkdir build/debug*
> *cd build/debug*
> *export NO_REBUILD_THIRDPARTY=1*
> *../../thirdparty/installed/common/bin/cmake \*
> * -DCMAKE_BUILD_TYPE=debug ../..*
> *make -j6*
> *ctest -j6*
> *cd /kudu/java/*
> *./gradlew test*
> *cd /kudu/python*
> *export KUDU_HOME="/kudu"*
> *python setup.py build_ext*
> *python setup.py test*
>
> Below are the flaky and failing tests I observed:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *centos:7 saw flakes in
> TestKuduTable.testFormatRangePartitionsStringColumcentos:8 saw flakes in
> auto_rebalancer-test mini_ranger-test
> DefaultSourceTest.testInsertionMultipleubuntu:xenial saw failures in
> client-stress-test (Failed) memory_gc-itest (Failed)
> raft_consensus-itest.2 (Failed)ubuntu:xenial saw flakes in
> auto_rebalancer-testubuntu:bionic saw flakes in
> mini_ranger-testdebian:stretch saw flakes in kudu-tool-test.1*
>
>
> * ITClientStress.testManyShortClientsGeneratingScanTokens
> TestMiniKuduCluster.testHiveMetastoreIntegration
> TestMiniKuduCluster.testKerberos*
>
>
> I seem to remember that those tests that are falling in xenial have always
> been an issue in
> Docker and are likely a Docker image/setup issue. I don't think they are a
> concern for this
> release, but I will investigate further.
>
> I tested the jars and artifacts by using the java-example on both Mac and
> Linux.
> I updated the pom to use the staged repositories and version 1.12.0 and
> ran `mvn test`:
> https://github.com/apache/kudu/tree/master/examples/java/java-example
>
> Things worked as expected on Linux, but on a Mac I found an issue with
> libc++ linking. I haven't dug into the issue yet.
> I am on Catalina, perhaps someone with an older MacOS version can test too?
>
>
>
>
>
>
>
>
> *[main] INFO
> org.apache.kudu.shaded.com.google.gradle.osdetector.OsDetector -
> os.detected.classifier=osx-x86_64[main] INFO
> org.apache.kudu.test.cluster.KuduBinaryLocator - Using Kudu binary jar
> directory:
> /var/folders/3y/nrs6__c10n107f357n7yx2100000gp/T/kudu-binary-jar6249991106299083358[main]
> INFO org.apache.kudu.test.cluster.MiniKuduCluster - Starting process:
> [/var/folders/3y/nrs6__c10n107f357n7yx2100000gp/T/kudu-binary-jar6249991106299083358/kudu-binary-1.12.0-osx-x86_64/bin/kudu,
> test, mini_cluster, --serialization=pb][cluster stderr printer] INFO
> org.apache.kudu.test.cluster.MiniKuduCluster - dyld: Library not loaded:
> @rpath/libc++abi.1.dylib[cluster stderr printer] INFO
> org.apache.kudu.test.cluster.MiniKuduCluster - Referenced from:
> /private/var/folders/3y/nrs6__c10n107f357n7yx2100000gp/T/kudu-binary-jar6249991106299083358/kudu-binary-1.12.0-osx-x86_64/lib/libglog.0.dylib[cluster
> stderr printer] INFO org.apache.kudu.test.cluster.MiniKuduCluster -
> Reason: image not found[main] ERROR org.apache.kudu.test.junit.RetryRule -
> org.apache.kudu.examples.ExampleTest.testCreateExampleTable: failed attempt
> 1java.io.EOFException*
>
>
> On Thu, Apr 16, 2020 at 5:37 PM Hao Hao <ha...@cloudera.com.invalid>
> wrote:
>
>> Hello Kudu devs!
>>
>> The Apache Kudu team is happy to announce the first release candidate for
>> Apache Kudu 1.12.0.
>>
>> Apache Kudu 1.12.0 is a minor release that offers many improvements and
>> fixes since the prior release.
>>
>> This is a source-only release. The artifacts have been staged here:
>> https://dist.apache.org/repos/dist/dev/kudu/1.12.0-RC1/
>>
>> Java convenience binaries in the form of a Maven repository are staged
>> here:
>> https://repository.apache.org/content/repositories/orgapachekudu-1059/
>>
>> Linux and macOS kudu-binary JAR artifacts are staged here correspondingly:
>> https://repository.apache.org/content/repositories/orgapachekudu-1060/
>> https://repository.apache.org/content/repositories/orgapachekudu-1061/
>>
>> It is tagged in Git as 1.12.0-RC1 and the corresponding hash is the
>> following:
>>
>> https://gitbox.apache.org/repos/asf?p=kudu.git;a=commit;h=ae919d457cda36d854f81732b34af2384441d436
>>
>> The WIP release notes can be found here:
>> https://gerrit.cloudera.org/c/15685/
>>
>> The KEYS file to verify the artifact signatures can be found here:
>> https://dist.apache.org/repos/dist/release/kudu/KEYS
>>
>> I'd suggest going through the README and the release notes, building Kudu,
>> and
>> running the unit tests. Testing out the Maven repo would also be
>> appreciated.
>> Also, it's worth running Kudu Java tests against kudu-binary JAR artifact
>> as described in the commit message here:
>>
>>
>> https://gitbox.apache.org/repos/asf?p=kudu.git;a=commit;h=8a6faaa93f3e206ac75e8087731daccaf7ab646a
>>
>> The vote will run until Tuesday Apr 21st 18:00:00 PST 2020. This is a bit
>> over the suggested 72 hours due to the weekend.
>>
>> Thank you,
>> Hao
>>
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [VOTE] Apache Kudu 1.12.0-RC1
Posted by Grant Henke <gr...@apache.org>.
I ran the following to test various OS versions in Docker:
*# Build all the images*
*export
BASES="centos:7,centos:8,ubuntu:xenial,ubuntu:bionic,debian:stretch"*
*export TARGETS="build"*
*./docker/docker-build.sh*
*# For each image, run the tests*
*docker run -it --rm apache/kudu:build-latest-<os> /bin/bash*
*mkdir build/debug*
*cd build/debug*
*export NO_REBUILD_THIRDPARTY=1*
*../../thirdparty/installed/common/bin/cmake \*
* -DCMAKE_BUILD_TYPE=debug ../..*
*make -j6*
*ctest -j6*
*cd /kudu/java/*
*./gradlew test*
*cd /kudu/python*
*export KUDU_HOME="/kudu"*
*python setup.py build_ext*
*python setup.py test*
Below are the flaky and failing tests I observed:
*centos:7 saw flakes in
TestKuduTable.testFormatRangePartitionsStringColumcentos:8 saw flakes in
auto_rebalancer-test mini_ranger-test
DefaultSourceTest.testInsertionMultipleubuntu:xenial saw failures in
client-stress-test (Failed) memory_gc-itest (Failed)
raft_consensus-itest.2 (Failed)ubuntu:xenial saw flakes in
auto_rebalancer-testubuntu:bionic saw flakes in
mini_ranger-testdebian:stretch saw flakes in kudu-tool-test.1*
* ITClientStress.testManyShortClientsGeneratingScanTokens
TestMiniKuduCluster.testHiveMetastoreIntegration
TestMiniKuduCluster.testKerberos*
I seem to remember that those tests that are falling in xenial have always
been an issue in
Docker and are likely a Docker image/setup issue. I don't think they are a
concern for this
release, but I will investigate further.
I tested the jars and artifacts by using the java-example on both Mac and
Linux.
I updated the pom to use the staged repositories and version 1.12.0 and ran
`mvn test`:
https://github.com/apache/kudu/tree/master/examples/java/java-example
Things worked as expected on Linux, but on a Mac I found an issue with
libc++ linking. I haven't dug into the issue yet.
I am on Catalina, perhaps someone with an older MacOS version can test too?
*[main] INFO org.apache.kudu.shaded.com.google.gradle.osdetector.OsDetector
- os.detected.classifier=osx-x86_64[main] INFO
org.apache.kudu.test.cluster.KuduBinaryLocator - Using Kudu binary jar
directory:
/var/folders/3y/nrs6__c10n107f357n7yx2100000gp/T/kudu-binary-jar6249991106299083358[main]
INFO org.apache.kudu.test.cluster.MiniKuduCluster - Starting process:
[/var/folders/3y/nrs6__c10n107f357n7yx2100000gp/T/kudu-binary-jar6249991106299083358/kudu-binary-1.12.0-osx-x86_64/bin/kudu,
test, mini_cluster, --serialization=pb][cluster stderr printer] INFO
org.apache.kudu.test.cluster.MiniKuduCluster - dyld: Library not loaded:
@rpath/libc++abi.1.dylib[cluster stderr printer] INFO
org.apache.kudu.test.cluster.MiniKuduCluster - Referenced from:
/private/var/folders/3y/nrs6__c10n107f357n7yx2100000gp/T/kudu-binary-jar6249991106299083358/kudu-binary-1.12.0-osx-x86_64/lib/libglog.0.dylib[cluster
stderr printer] INFO org.apache.kudu.test.cluster.MiniKuduCluster -
Reason: image not found[main] ERROR org.apache.kudu.test.junit.RetryRule -
org.apache.kudu.examples.ExampleTest.testCreateExampleTable: failed attempt
1java.io.EOFException*
On Thu, Apr 16, 2020 at 5:37 PM Hao Hao <ha...@cloudera.com.invalid>
wrote:
> Hello Kudu devs!
>
> The Apache Kudu team is happy to announce the first release candidate for
> Apache Kudu 1.12.0.
>
> Apache Kudu 1.12.0 is a minor release that offers many improvements and
> fixes since the prior release.
>
> This is a source-only release. The artifacts have been staged here:
> https://dist.apache.org/repos/dist/dev/kudu/1.12.0-RC1/
>
> Java convenience binaries in the form of a Maven repository are staged
> here:
> https://repository.apache.org/content/repositories/orgapachekudu-1059/
>
> Linux and macOS kudu-binary JAR artifacts are staged here correspondingly:
> https://repository.apache.org/content/repositories/orgapachekudu-1060/
> https://repository.apache.org/content/repositories/orgapachekudu-1061/
>
> It is tagged in Git as 1.12.0-RC1 and the corresponding hash is the
> following:
>
> https://gitbox.apache.org/repos/asf?p=kudu.git;a=commit;h=ae919d457cda36d854f81732b34af2384441d436
>
> The WIP release notes can be found here:
> https://gerrit.cloudera.org/c/15685/
>
> The KEYS file to verify the artifact signatures can be found here:
> https://dist.apache.org/repos/dist/release/kudu/KEYS
>
> I'd suggest going through the README and the release notes, building Kudu,
> and
> running the unit tests. Testing out the Maven repo would also be
> appreciated.
> Also, it's worth running Kudu Java tests against kudu-binary JAR artifact
> as described in the commit message here:
>
>
> https://gitbox.apache.org/repos/asf?p=kudu.git;a=commit;h=8a6faaa93f3e206ac75e8087731daccaf7ab646a
>
> The vote will run until Tuesday Apr 21st 18:00:00 PST 2020. This is a bit
> over the suggested 72 hours due to the weekend.
>
> Thank you,
> Hao
>