You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/02/22 00:44:04 UTC
[GitHub] [hbase] ndimiduk opened a new pull request #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
ndimiduk opened a new pull request #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195
This PR provides changes necessary to run JDK11 checks during nightlies. It depends on HBASE-23767/#1183.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1195:
HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
busbey commented on a change in pull request #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r384754162
##########
File path: dev-support/docker/Dockerfile
##########
@@ -14,47 +14,162 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-# Dockerfile for installing the necessary dependencies for building Hadoop.
-# See BUILDING.txt.
-
-FROM maven:3.5-jdk-8
-
-RUN apt-get -q update && apt-get -q install --no-install-recommends -y \
- git \
- bats \
- findbugs \
- libperl-critic-perl \
- pylint \
- python-dateutil \
- rsync \
- make \
- gcc \
- libc6-dev \
- ruby \
- ruby-dev \
- wget \
- && \
- gem install --no-document rake rubocop ruby-lint
-
-ENV FINDBUGS_HOME /usr
-
-####
-# Install shellcheck
-###
-RUN mkdir -p /opt/shellcheck && \
- curl -L -s -S \
- https://storage.googleapis.com/shellcheck/shellcheck-stable.linux.x86_64.tar.xz \
- -o /opt/shellcheck.tar.xz && \
- tar xJf /opt/shellcheck.tar.xz --strip-components 1 -C /opt/shellcheck && \
- ln -s /opt/shellcheck/shellcheck /usr/bin/shellcheck && \
- rm -f /opt/shellcheck.tar.xz
+#
+# Dockerfile used as the build and test environment for Yetus.
+#
+# Built in multiple stages so as to avoid re-downloading large binaries when
+# tweaking unrelated aspects of the image.
-###
-# Avoid out of memory errors in builds
-###
-ENV MAVEN_OPTS -Xmx3g
+# start with a minimal image into which we can download remote tarballs
+FROM ubuntu:18.04 AS BASE_IMAGE
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
+# hadolint ignore=DL3009
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq update && \
+ DEBIAN_FRONTEND=noninteractive apt-get -qq install --no-install-recommends -y \
+ ca-certificates=20180409 \
+ curl=7.58.0-2ubuntu3.8 \
+ locales=2.27-3ubuntu1
+
+RUN locale-gen en_US.UTF-8
+ENV LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8
+
+##
+# download sundry dependencies
+#
+
+FROM BASE_IMAGE AS FINDBUGS_DOWNLOAD_IMAGE
Review comment:
TODO HBASE-23077 / HBASE-22383
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-594677969
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/9/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-590711650
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 8s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 8s | master passed |
| +1 :green_heart: | compile | 4m 3s | master passed with JDK v2020-01-14 |
| +1 :green_heart: | compile | 3m 22s | master passed with JDK v1.8.0_232 |
| +1 :green_heart: | shadedjars | 6m 10s | branch has no errors when building our shaded downstream artifacts. |
| -1 :x: | javadoc | 0m 20s | root in master failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 5m 28s | master passed with JDK v1.8.0_232 |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 8m 3s | the patch passed |
| +1 :green_heart: | compile | 6m 31s | the patch passed with JDK v2020-01-14 |
| +1 :green_heart: | javac | 6m 31s | the patch passed |
| +1 :green_heart: | compile | 5m 2s | the patch passed with JDK v1.8.0_232 |
| +1 :green_heart: | javac | 5m 2s | the patch passed |
| +1 :green_heart: | hadolint | 0m 2s | The patch generated 0 new + 0 unchanged - 4 fixed = 0 total (was 4) |
| +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. |
| +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 4s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedjars | 8m 9s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 26m 54s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | javadoc | 0m 34s | root in the patch failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 6m 11s | the patch passed with JDK v1.8.0_232 |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 145m 22s | root in the patch passed with JDK v1.8.0_232. |
| +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. |
| | | 245m 44s | |
| Reason | Tests |
|-------:|:------|
| JDK v2020-01-14 Failed junit tests | hadoop.hbase.util.TestFutureUtils |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | dupname asflicense hadolint shellcheck shelldocs javac javadoc unit shadedjars hadoopcheck xml compile |
| uname | Linux 110c6694b488 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1195/out/precommit/personality/provided.sh |
| git revision | master / 3e21dc73de |
| Default Java | 1.8.0_232 |
| Multi-JDK versions | /usr/lib/jvm/jdk-11.0.6+10:2020-01-14 /usr/lib/jvm/jdk8u232-b09:1.8.0_232 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/4/artifact/out/branch-javadoc-root-jdk2020-01-14.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/4/artifact/out/patch-javadoc-root-jdk2020-01-14.txt |
| JDK v1.8.0_232 Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/4/testReport/ |
| Max. process+thread count | 6890 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/4/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 hadolint=1.17.5-0-g443423c |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1195:
HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
busbey commented on a change in pull request #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r384867009
##########
File path: dev-support/Jenkinsfile_GitHub
##########
@@ -144,8 +143,9 @@ pipeline {
YETUS_ARGS+=("--docker")
YETUS_ARGS+=("--dockerfile=${DOCKERFILE}")
YETUS_ARGS+=("--mvn-custom-repos")
- YETUS_ARGS+=("--multijdkdirs=/usr/lib/jvm/java-8-openjdk-amd64")
- YETUS_ARGS+=("--findbugs-home=/usr")
+ YETUS_ARGS+=("--java-home=/usr/lib/jvm/java-8")
+ YETUS_ARGS+=("--multijdkdirs=/usr/lib/jvm/java-8,/usr/lib/jvm/java-11")
+ YETUS_ARGS+=("--multijdktests=compile,unit")
Review comment:
Hurm. The test report still looks like a mixed bag. The test failures table shows both jdks, but the test vote table only has jdk8. The footer entry with the link to unit test console output also only has jdk8.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-590653670
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/4/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-591655797
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/8/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on issue #1195: HBASE-23876 [WIP] Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
busbey commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-591628804
if I test this locally with YETUS-943 in place I should get correct jdk handling right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-594816198
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/10/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-595379517
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/11/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-595379402
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/11/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-594677972
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/9/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-594698740
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 16s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | hadolint | 0m 1s | The patch generated 0 new + 0 unchanged - 4 fixed = 0 total (was 4) |
| +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | hadoopcheck | 18m 26s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. |
| | | 28m 9s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/9/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | dupname asflicense hadolint shellcheck shelldocs hadoopcheck xml |
| uname | Linux 9bb93a7adc17 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 6d9802fc2e |
| Max. process+thread count | 63 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/9/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 hadolint=1.17.5-0-g443423c |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1195:
HBASE-23876 Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
busbey commented on a change in pull request #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r387869807
##########
File path: dev-support/Jenkinsfile
##########
@@ -49,6 +50,8 @@ pipeline {
// These tests currently have known failures. Once they burn down to 0, remove from here so that new problems will cause a failure.
TESTS_FILTER = 'cc,checkstyle,javac,javadoc,pylint,shellcheck,whitespace,perlcritic,ruby-lint,rubocop,mvnsite'
EXCLUDE_TESTS_URL = "${JENKINS_URL}/job/HBase-Find-Flaky-Tests/job/${BRANCH_NAME}/lastSuccessfulBuild/artifact/excludes"
+ SHALLOW_CHECKS = 'all,-unit,-findbugs' // run by the 'yetus general check'
Review comment:
so long as the general check is jdk8 then doing it in general is good. IIRC when we were sharing this jenkinsfile with branch-1 it was in jdk8-hadoop2 because that was jdk8 on both sets of branches.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ndimiduk commented on a change in pull request #1195:
HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on a change in pull request #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r385253307
##########
File path: dev-support/Jenkinsfile_GitHub
##########
@@ -144,8 +143,9 @@ pipeline {
YETUS_ARGS+=("--docker")
YETUS_ARGS+=("--dockerfile=${DOCKERFILE}")
YETUS_ARGS+=("--mvn-custom-repos")
- YETUS_ARGS+=("--multijdkdirs=/usr/lib/jvm/java-8-openjdk-amd64")
- YETUS_ARGS+=("--findbugs-home=/usr")
+ YETUS_ARGS+=("--java-home=/usr/lib/jvm/java-8")
+ YETUS_ARGS+=("--multijdkdirs=/usr/lib/jvm/java-8,/usr/lib/jvm/java-11")
+ YETUS_ARGS+=("--multijdktests=compile,unit")
Review comment:
Okay. Let me break the JDK11 check out of a multijdk block and into separate stages. Do you prefer parallel or serial stages?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1195:
HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
busbey commented on a change in pull request #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r384754422
##########
File path: dev-support/docker/Dockerfile
##########
@@ -14,47 +14,162 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-# Dockerfile for installing the necessary dependencies for building Hadoop.
-# See BUILDING.txt.
-
-FROM maven:3.5-jdk-8
-
-RUN apt-get -q update && apt-get -q install --no-install-recommends -y \
- git \
- bats \
- findbugs \
- libperl-critic-perl \
- pylint \
- python-dateutil \
- rsync \
- make \
- gcc \
- libc6-dev \
- ruby \
- ruby-dev \
- wget \
- && \
- gem install --no-document rake rubocop ruby-lint
-
-ENV FINDBUGS_HOME /usr
-
-####
-# Install shellcheck
-###
-RUN mkdir -p /opt/shellcheck && \
- curl -L -s -S \
- https://storage.googleapis.com/shellcheck/shellcheck-stable.linux.x86_64.tar.xz \
- -o /opt/shellcheck.tar.xz && \
- tar xJf /opt/shellcheck.tar.xz --strip-components 1 -C /opt/shellcheck && \
- ln -s /opt/shellcheck/shellcheck /usr/bin/shellcheck && \
- rm -f /opt/shellcheck.tar.xz
+#
+# Dockerfile used as the build and test environment for Yetus.
+#
+# Built in multiple stages so as to avoid re-downloading large binaries when
+# tweaking unrelated aspects of the image.
-###
-# Avoid out of memory errors in builds
-###
-ENV MAVEN_OPTS -Xmx3g
+# start with a minimal image into which we can download remote tarballs
+FROM ubuntu:18.04 AS BASE_IMAGE
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
+# hadolint ignore=DL3009
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq update && \
+ DEBIAN_FRONTEND=noninteractive apt-get -qq install --no-install-recommends -y \
+ ca-certificates=20180409 \
+ curl=7.58.0-2ubuntu3.8 \
+ locales=2.27-3ubuntu1
+
+RUN locale-gen en_US.UTF-8
+ENV LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8
+
+##
+# download sundry dependencies
+#
+
+FROM BASE_IMAGE AS FINDBUGS_DOWNLOAD_IMAGE
+ENV FINDBUGS_VERSION '3.0.1'
+ENV FINDBUGS_URL "https://downloads.sourceforge.net/project/findbugs/findbugs/${FINDBUGS_VERSION}/findbugs-${FINDBUGS_VERSION}.tar.gz"
+ENV FINDBUGS_SHA256 'e80e0da0c213a27504ef3188ef25f107651700ffc66433eac6a7454bbe336419'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/findbugs.tar.gz "${FINDBUGS_URL}" && \
Review comment:
personal nit: long curl options much easier to maintain since most of our contributor don't have curl short flags memorized.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-591654946
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 15s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 13s | master passed |
| +1 :green_heart: | compile | 4m 2s | master passed with JDK v2020-01-14 |
| +1 :green_heart: | compile | 3m 33s | master passed with JDK v1.8.0_232 |
| +1 :green_heart: | shadedjars | 6m 10s | branch has no errors when building our shaded downstream artifacts. |
| -1 :x: | javadoc | 0m 21s | root in master failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 7m 39s | master passed with JDK v1.8.0_232 |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 8m 48s | the patch passed |
| +1 :green_heart: | compile | 6m 55s | the patch passed with JDK v2020-01-14 |
| +1 :green_heart: | javac | 6m 55s | the patch passed |
| +1 :green_heart: | compile | 5m 25s | the patch passed with JDK v1.8.0_232 |
| +1 :green_heart: | javac | 5m 25s | the patch passed |
| +1 :green_heart: | hadolint | 0m 2s | The patch generated 0 new + 0 unchanged - 4 fixed = 0 total (was 4) |
| +1 :green_heart: | shellcheck | 0m 4s | There were no new shellcheck issues. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 4s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedjars | 8m 27s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 26m 7s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | javadoc | 0m 20s | root in the patch failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 2m 56s | the patch passed with JDK v1.8.0_232 |
||| _ Other Tests _ |
| -1 :x: | unit | 151m 49s | root in the patch failed with JDK v1.8.0_232. |
| +1 :green_heart: | asflicense | 0m 33s | The patch does not generate ASF License warnings. |
| | | 252m 22s | |
| Reason | Tests |
|-------:|:------|
| JDK v2020-01-14 Failed junit tests | hadoop.hbase.util.TestFutureUtils |
| JDK v1.8.0_232 Failed junit tests | hadoop.hbase.master.procedure.TestDisableTableProcedure |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/7/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | dupname asflicense hadolint shellcheck shelldocs javac javadoc unit shadedjars hadoopcheck xml compile |
| uname | Linux 5c224c39db61 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1195/out/precommit/personality/provided.sh |
| git revision | master / 3c3aae9a2f |
| Default Java | 1.8.0_232 |
| Multi-JDK versions | /usr/lib/jvm/jdk-11.0.6+10:2020-01-14 /usr/lib/jvm/jdk8u232-b09:1.8.0_232 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/7/artifact/out/branch-javadoc-root-jdk2020-01-14.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/7/artifact/out/patch-javadoc-root-jdk2020-01-14.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/7/artifact/out/patch-unit-root-jdk1.8.0_232.txt |
| JDK v1.8.0_232 Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/7/testReport/ |
| Max. process+thread count | 6707 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/7/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 hadolint=1.17.5-0-g443423c |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-595381220
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 47s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | hadolint | 0m 2s | The patch generated 0 new + 0 unchanged - 4 fixed = 0 total (was 4) |
| +1 :green_heart: | shellcheck | 0m 3s | There were no new shellcheck issues. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 23s | The patch does not generate ASF License warnings. |
| | | 4m 50s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/11/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | dupname asflicense hadolint shellcheck shelldocs |
| uname | Linux 6e9b5efb5c34 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / aff0ff5d97 |
| Max. process+thread count | 51 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/11/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 hadolint=1.17.5-0-g443423c |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1195:
HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
busbey commented on a change in pull request #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r384753107
##########
File path: dev-support/docker/Dockerfile
##########
@@ -14,47 +14,162 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-# Dockerfile for installing the necessary dependencies for building Hadoop.
-# See BUILDING.txt.
-
-FROM maven:3.5-jdk-8
-
-RUN apt-get -q update && apt-get -q install --no-install-recommends -y \
- git \
- bats \
- findbugs \
- libperl-critic-perl \
- pylint \
- python-dateutil \
- rsync \
- make \
- gcc \
- libc6-dev \
- ruby \
- ruby-dev \
- wget \
- && \
- gem install --no-document rake rubocop ruby-lint
-
-ENV FINDBUGS_HOME /usr
-
-####
-# Install shellcheck
-###
-RUN mkdir -p /opt/shellcheck && \
- curl -L -s -S \
- https://storage.googleapis.com/shellcheck/shellcheck-stable.linux.x86_64.tar.xz \
- -o /opt/shellcheck.tar.xz && \
- tar xJf /opt/shellcheck.tar.xz --strip-components 1 -C /opt/shellcheck && \
- ln -s /opt/shellcheck/shellcheck /usr/bin/shellcheck && \
- rm -f /opt/shellcheck.tar.xz
+#
+# Dockerfile used as the build and test environment for Yetus.
Review comment:
nit: s/Yetus/HBase or HBase's use of Apache Yetus
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ndimiduk commented on a change in pull request #1195:
HBASE-23876 Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on a change in pull request #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r387843422
##########
File path: dev-support/Jenkinsfile
##########
@@ -49,6 +50,8 @@ pipeline {
// These tests currently have known failures. Once they burn down to 0, remove from here so that new problems will cause a failure.
TESTS_FILTER = 'cc,checkstyle,javac,javadoc,pylint,shellcheck,whitespace,perlcritic,ruby-lint,rubocop,mvnsite'
EXCLUDE_TESTS_URL = "${JENKINS_URL}/job/HBase-Find-Flaky-Tests/job/${BRANCH_NAME}/lastSuccessfulBuild/artifact/excludes"
+ SHALLOW_CHECKS = 'all,-unit,-findbugs' // run by the 'yetus general check'
+ DEEP_CHECKS = 'maven,mvninstall,compile,javac,unit,htmlout' // run by 'yetus jdkX HadoopY checks'
Review comment:
You're the expert in this area, I'll do as you suggest. That one is just called `shadedjars` ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ndimiduk commented on issue #1195: HBASE-23876 [WIP] Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-589896310
I pushed a [feature branch](https://github.com/apache/hbase/tree/HBASE-23876/jdk11-nightly-master) that should be picked up by the nightlies job.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1195:
HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
busbey commented on a change in pull request #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r384755666
##########
File path: dev-support/docker/Dockerfile
##########
@@ -14,47 +14,162 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-# Dockerfile for installing the necessary dependencies for building Hadoop.
-# See BUILDING.txt.
-
-FROM maven:3.5-jdk-8
-
-RUN apt-get -q update && apt-get -q install --no-install-recommends -y \
- git \
- bats \
- findbugs \
- libperl-critic-perl \
- pylint \
- python-dateutil \
- rsync \
- make \
- gcc \
- libc6-dev \
- ruby \
- ruby-dev \
- wget \
- && \
- gem install --no-document rake rubocop ruby-lint
-
-ENV FINDBUGS_HOME /usr
-
-####
-# Install shellcheck
-###
-RUN mkdir -p /opt/shellcheck && \
- curl -L -s -S \
- https://storage.googleapis.com/shellcheck/shellcheck-stable.linux.x86_64.tar.xz \
- -o /opt/shellcheck.tar.xz && \
- tar xJf /opt/shellcheck.tar.xz --strip-components 1 -C /opt/shellcheck && \
- ln -s /opt/shellcheck/shellcheck /usr/bin/shellcheck && \
- rm -f /opt/shellcheck.tar.xz
+#
+# Dockerfile used as the build and test environment for Yetus.
+#
+# Built in multiple stages so as to avoid re-downloading large binaries when
+# tweaking unrelated aspects of the image.
-###
-# Avoid out of memory errors in builds
-###
-ENV MAVEN_OPTS -Xmx3g
+# start with a minimal image into which we can download remote tarballs
+FROM ubuntu:18.04 AS BASE_IMAGE
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
+# hadolint ignore=DL3009
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq update && \
+ DEBIAN_FRONTEND=noninteractive apt-get -qq install --no-install-recommends -y \
+ ca-certificates=20180409 \
+ curl=7.58.0-2ubuntu3.8 \
+ locales=2.27-3ubuntu1
+
+RUN locale-gen en_US.UTF-8
+ENV LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8
+
+##
+# download sundry dependencies
+#
+
+FROM BASE_IMAGE AS FINDBUGS_DOWNLOAD_IMAGE
+ENV FINDBUGS_VERSION '3.0.1'
+ENV FINDBUGS_URL "https://downloads.sourceforge.net/project/findbugs/findbugs/${FINDBUGS_VERSION}/findbugs-${FINDBUGS_VERSION}.tar.gz"
+ENV FINDBUGS_SHA256 'e80e0da0c213a27504ef3188ef25f107651700ffc66433eac6a7454bbe336419'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/findbugs.tar.gz "${FINDBUGS_URL}" && \
+ echo "${FINDBUGS_SHA256} */tmp/findbugs.tar.gz" | sha256sum -c -
+
+FROM BASE_IMAGE AS HADOLINT_DOWNLOAD_IMAGE
+ENV HADOLINT_VERSION '1.17.5'
+ENV HADOLINT_URL "https://github.com/hadolint/hadolint/releases/download/v${HADOLINT_VERSION}/hadolint-Linux-x86_64"
+ENV HADOLINT_SHA256 '20dd38bc0602040f19268adc14c3d1aae11af27b463af43f3122076baf827a35'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/hadolint "${HADOLINT_URL}" && \
+ echo "${HADOLINT_SHA256} */tmp/hadolint" | sha256sum -c -
+
+FROM BASE_IMAGE AS MAVEN_DOWNLOAD_IMAGE
+ENV MAVEN_VERSION='3.5.4'
+ENV MAVEN_URL "https://archive.apache.org/dist/maven/maven-3/${MAVEN_VERSION}/binaries/apache-maven-${MAVEN_VERSION}-bin.tar.gz"
+ENV MAVEN_SHA256 'ce50b1c91364cb77efe3776f756a6d92b76d9038b0a0782f7d53acf1e997a14d'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/maven.tar.gz "${MAVEN_URL}" && \
+ echo "${MAVEN_SHA256} */tmp/maven.tar.gz" | sha256sum -c -
+
+FROM BASE_IMAGE AS OPENJDK8_DOWNLOAD_IMAGE
+ENV OPENJDK8_URL 'https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u232-b09/OpenJDK8U-jdk_x64_linux_hotspot_8u232b09.tar.gz'
+ENV OPENJDK8_SHA256 '7b7884f2eb2ba2d47f4c0bf3bb1a2a95b73a3a7734bd47ebf9798483a7bcc423'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/adoptopenjdk8.tar.gz "${OPENJDK8_URL}" && \
+ echo "${OPENJDK8_SHA256} */tmp/adoptopenjdk8.tar.gz" | sha256sum -c -
+
+FROM BASE_IMAGE AS OPENJDK11_DOWNLOAD_IMAGE
+ENV OPENJDK11_URL 'https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.6%2B10/OpenJDK11U-jdk_x64_linux_hotspot_11.0.6_10.tar.gz'
+ENV OPENJDK11_SHA256 '330d19a2eaa07ed02757d7a785a77bab49f5ee710ea03b4ee2fa220ddd0feffc'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/adoptopenjdk11.tar.gz "${OPENJDK11_URL}" && \
+ echo "${OPENJDK11_SHA256} */tmp/adoptopenjdk11.tar.gz" | sha256sum -c -
+
+##
+# build the final image
+#
+
+FROM BASE_IMAGE
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
+##
+# install dependencies from system packages.
+# be careful not to install any system packages (i.e., findbugs) that will
+# pull in the default-jre.
+#
+
+# bring the base image into conformance with the expectations imposed by
+# Yetus and our personality file of what a build environment looks like.
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq install --no-install-recommends -y \
+ bash=4.4.18-2ubuntu1.2 \
+ build-essential=12.4ubuntu1 \
+ curl=7.58.0-2ubuntu3.8 \
+ diffutils=1:3.6-1 \
+ git=1:2.17.1-1ubuntu0.5 \
+ rsync=3.1.2-2.1ubuntu1 \
+ tar=1.29b-2ubuntu0.1 \
+ wget=1.19.4-1ubuntu2.2
+
+# install the dependencies required in order to enable the sundry precommit
+# checks/features provided by Yetus plugins.
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq install --no-install-recommends -y \
+ bats=0.4.0-1.1 \
+ libperl-critic-perl=1.130-1 \
+ python3=3.6.7-1~18.04 \
+ python3-pip=9.0.1-2.3~ubuntu1.18.04.1 \
+ python3-setuptools=39.0.1-2 \
+ ruby=1:2.5.1 \
+ ruby-dev=1:2.5.1 \
+ shellcheck=0.4.6-1 \
+ && \
+ apt-get clean && \
+ rm -rf /var/lib/apt/lists/*
+
+RUN python3 -mpip install --upgrade pip && \
+ python3 -mpip install pylint==2.4.4
+
+RUN gem install --no-document \
+ rake:13.0.1 \
+ rubocop:0.80.0 \
+ ruby-lint:2.3.1
+
+# hadolint ignore=DL3010
+COPY --from=FINDBUGS_DOWNLOAD_IMAGE /tmp/findbugs.tar.gz /tmp/findbugs.tar.gz
+RUN tar xzf /tmp/findbugs.tar.gz -C /opt && \
+ ln -s "/opt/$(dirname "$(tar -tf /tmp/findbugs.tar.gz | head -n1)")" /opt/findbugs && \
+ rm /tmp/findbugs.tar.gz
+
+COPY --from=HADOLINT_DOWNLOAD_IMAGE /tmp/hadolint /tmp/hadolint
+RUN mv /tmp/hadolint /usr/local/bin && \
+ chmod a+x /usr/local/bin/hadolint
+
+# hadolint ignore=DL3010
+COPY --from=MAVEN_DOWNLOAD_IMAGE /tmp/maven.tar.gz /tmp/maven.tar.gz
+RUN tar xzf /tmp/maven.tar.gz -C /opt && \
+ ln -s "/opt/$(dirname "$(tar -tf /tmp/maven.tar.gz | head -n1)")" /opt/maven && \
+ rm /tmp/maven.tar.gz
+
+##
+# ensure JVMs are available under `/usr/lib/jvm` and prefix each installation
+# as `java-` so as to conform with Yetus's assumptions.
+#
+
Review comment:
what happened to jdk7?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-594816218
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/10/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-591257167
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 7m 28s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +0 :ok: | shelldocs | 0m 1s | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 6s | master passed |
| +1 :green_heart: | compile | 4m 5s | master passed with JDK v2020-01-14 |
| +1 :green_heart: | compile | 3m 21s | master passed with JDK v1.8.0_232 |
| +1 :green_heart: | shadedjars | 5m 13s | branch has no errors when building our shaded downstream artifacts. |
| -1 :x: | javadoc | 0m 20s | root in master failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 2m 53s | master passed with JDK v1.8.0_232 |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 29s | the patch passed |
| +1 :green_heart: | compile | 4m 16s | the patch passed with JDK v2020-01-14 |
| +1 :green_heart: | javac | 4m 16s | the patch passed |
| +1 :green_heart: | compile | 3m 43s | the patch passed with JDK v1.8.0_232 |
| +1 :green_heart: | javac | 3m 43s | the patch passed |
| +1 :green_heart: | hadolint | 0m 2s | The patch generated 0 new + 0 unchanged - 4 fixed = 0 total (was 4) |
| +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedjars | 5m 22s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 18m 54s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | javadoc | 0m 19s | root in the patch failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 3m 13s | the patch passed with JDK v1.8.0_232 |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 155m 56s | root in the patch passed with JDK v1.8.0_232. |
| +1 :green_heart: | asflicense | 0m 33s | The patch does not generate ASF License warnings. |
| | | 237m 24s | |
| Reason | Tests |
|-------:|:------|
| JDK v2020-01-14 Failed junit tests | hadoop.hbase.util.TestFutureUtils |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | dupname asflicense hadolint shellcheck shelldocs javac javadoc unit shadedjars hadoopcheck xml compile |
| uname | Linux 5329e638c2d9 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1195/out/precommit/personality/provided.sh |
| git revision | master / ecbed33092 |
| Default Java | 1.8.0_232 |
| Multi-JDK versions | /usr/lib/jvm/jdk-11.0.6+10:2020-01-14 /usr/lib/jvm/jdk8u232-b09:1.8.0_232 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/6/artifact/out/branch-javadoc-root-jdk2020-01-14.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/6/artifact/out/patch-javadoc-root-jdk2020-01-14.txt |
| JDK v1.8.0_232 Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/6/testReport/ |
| Max. process+thread count | 6586 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/6/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 hadolint=1.17.5-0-g443423c |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ndimiduk merged pull request #1195: HBASE-23876 Add JDK11
compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
ndimiduk merged pull request #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ndimiduk commented on a change in pull request #1195:
HBASE-23876 Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on a change in pull request #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r387844028
##########
File path: dev-support/Jenkinsfile
##########
@@ -49,6 +50,8 @@ pipeline {
// These tests currently have known failures. Once they burn down to 0, remove from here so that new problems will cause a failure.
TESTS_FILTER = 'cc,checkstyle,javac,javadoc,pylint,shellcheck,whitespace,perlcritic,ruby-lint,rubocop,mvnsite'
EXCLUDE_TESTS_URL = "${JENKINS_URL}/job/HBase-Find-Flaky-Tests/job/${BRANCH_NAME}/lastSuccessfulBuild/artifact/excludes"
+ SHALLOW_CHECKS = 'all,-unit,-findbugs' // run by the 'yetus general check'
Review comment:
Actually, why do we skip `findbugs` entirely? It should be done either as a shallow check or as a deep check, yes?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-594714385
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 41s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 8m 57s | master passed |
| +1 :green_heart: | compile | 4m 22s | master passed |
| +1 :green_heart: | shadedjars | 5m 37s | branch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 20s | root in master failed. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 7m 0s | the patch passed |
| +1 :green_heart: | compile | 4m 25s | the patch passed |
| +1 :green_heart: | javac | 4m 25s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 46s | patch has no errors when building our shaded downstream artifacts. |
| -0 :warning: | javadoc | 0m 24s | root in the patch failed. |
||| _ Other Tests _ |
| -1 :x: | unit | 2m 8s | root in the patch failed. |
| | | 45m 51s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/9/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 13a5646ded16 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 6d9802fc2e |
| Default Java | 2020-01-14 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/9/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/9/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/9/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/9/testReport/ |
| Max. process+thread count | 364 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/9/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ndimiduk commented on a change in pull request #1195:
HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on a change in pull request #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r384848332
##########
File path: dev-support/Jenkinsfile_GitHub
##########
@@ -144,8 +143,9 @@ pipeline {
YETUS_ARGS+=("--docker")
YETUS_ARGS+=("--dockerfile=${DOCKERFILE}")
YETUS_ARGS+=("--mvn-custom-repos")
- YETUS_ARGS+=("--multijdkdirs=/usr/lib/jvm/java-8-openjdk-amd64")
- YETUS_ARGS+=("--findbugs-home=/usr")
+ YETUS_ARGS+=("--java-home=/usr/lib/jvm/java-8")
+ YETUS_ARGS+=("--multijdkdirs=/usr/lib/jvm/java-8,/usr/lib/jvm/java-11")
+ YETUS_ARGS+=("--multijdktests=compile,unit")
Review comment:
I'm not sure what to tell you. This looks like explicit support to me, at least for running the unit tests with all supplied JDKs.
https://github.com/apache/yetus/blob/138de2232a351e9d9f4c3d751e7d25e52804dfe0/precommit/src/main/shell/test-patch.sh#L2103-L2105
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-594814867
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 42s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 3s | master passed |
| +1 :green_heart: | compile | 3m 22s | master passed |
| +1 :green_heart: | shadedjars | 5m 15s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 2m 47s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 39s | the patch passed |
| +1 :green_heart: | compile | 3m 22s | the patch passed |
| +1 :green_heart: | javac | 3m 22s | the patch passed |
| +1 :green_heart: | shadedjars | 5m 13s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 2m 46s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 139m 58s | root in the patch passed. |
| | | 183m 17s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/9/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 7efcbc4a23b5 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 6d9802fc2e |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/9/testReport/ |
| Max. process+thread count | 7075 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/9/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on issue #1195: HBASE-23876 [WIP] Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
busbey commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-594683331
> @ndimiduk / @busbey what would be the best way to test the Jenkins file changes over here, with a local docker image built out of your changes here / #1183 to verify the output that the yetus is able to generate?
sorry, missed this.
yes I believe that's correct. or take a look at the feature branch Nick pushed on asf jenkins to see how things look up there.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1195:
HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
busbey commented on a change in pull request #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r385297704
##########
File path: dev-support/Jenkinsfile_GitHub
##########
@@ -144,8 +143,9 @@ pipeline {
YETUS_ARGS+=("--docker")
YETUS_ARGS+=("--dockerfile=${DOCKERFILE}")
YETUS_ARGS+=("--mvn-custom-repos")
- YETUS_ARGS+=("--multijdkdirs=/usr/lib/jvm/java-8-openjdk-amd64")
- YETUS_ARGS+=("--findbugs-home=/usr")
+ YETUS_ARGS+=("--java-home=/usr/lib/jvm/java-8")
+ YETUS_ARGS+=("--multijdkdirs=/usr/lib/jvm/java-8,/usr/lib/jvm/java-11")
+ YETUS_ARGS+=("--multijdktests=compile,unit")
Review comment:
I personally prefer parallel. especially once we get the github integration in place to list the tests and check them off as we go it'll be nice.
however, if we do serial I believe jenkins will optimize executor reuse. so that might give a better experience (both us being nice to others and minimizing total time to be "done") prior to that incremental progress reporting.
do whichever you like.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-594818539
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 4m 23s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
||| _ Patch Compile Tests _ |
||| _ Other Tests _ |
| | | 5m 35s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/10/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | |
| uname | Linux ada10149c97e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / b6eefcaeb7 |
| Max. process+thread count | 46 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/10/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-591119825
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/5/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ndimiduk commented on a change in pull request #1195:
HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on a change in pull request #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r385449600
##########
File path: dev-support/Jenkinsfile_GitHub
##########
@@ -144,8 +143,9 @@ pipeline {
YETUS_ARGS+=("--docker")
YETUS_ARGS+=("--dockerfile=${DOCKERFILE}")
YETUS_ARGS+=("--mvn-custom-repos")
- YETUS_ARGS+=("--multijdkdirs=/usr/lib/jvm/java-8-openjdk-amd64")
- YETUS_ARGS+=("--findbugs-home=/usr")
+ YETUS_ARGS+=("--java-home=/usr/lib/jvm/java-8")
+ YETUS_ARGS+=("--multijdkdirs=/usr/lib/jvm/java-8,/usr/lib/jvm/java-11")
+ YETUS_ARGS+=("--multijdktests=compile,unit")
Review comment:
> Hurm. The test report still looks like a mixed bag. The test failures table shows both jdks, but the test vote table only has jdk8. The footer entry with the link to unit test console output also only has jdk8.
You're right. Have a look at the table printed over on https://github.com/apache/hbase/pull/1214#issuecomment-591757259. This should be filed as a bug/feature in Yetus?
> do whichever you like.
I've started plumbing in for parallel steps, since that's what nightly does and I can follow along. This is more involved than I originally imagined. I'm not sure how reporting of each check's success will work out -- right now we have a single summary written by Yetus, after this I expect we'll have 3 summaries (general, jdk8, jdk11), each written back independently. I'm inclined to not bother also collecting and writing a summary and the reports, given that we want to invest in HBASE-23902.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1195:
HBASE-23876 Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
busbey commented on a change in pull request #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r387871050
##########
File path: dev-support/Jenkinsfile
##########
@@ -49,6 +50,8 @@ pipeline {
// These tests currently have known failures. Once they burn down to 0, remove from here so that new problems will cause a failure.
TESTS_FILTER = 'cc,checkstyle,javac,javadoc,pylint,shellcheck,whitespace,perlcritic,ruby-lint,rubocop,mvnsite'
EXCLUDE_TESTS_URL = "${JENKINS_URL}/job/HBase-Find-Flaky-Tests/job/${BRANCH_NAME}/lastSuccessfulBuild/artifact/excludes"
+ SHALLOW_CHECKS = 'all,-unit,-findbugs' // run by the 'yetus general check'
+ DEEP_CHECKS = 'maven,mvninstall,compile,javac,unit,htmlout' // run by 'yetus jdkX HadoopY checks'
Review comment:
yes shadedjars is correct.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-590592673
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/3/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ndimiduk commented on a change in pull request #1195:
HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on a change in pull request #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r384849238
##########
File path: dev-support/docker/Dockerfile
##########
@@ -14,47 +14,162 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-# Dockerfile for installing the necessary dependencies for building Hadoop.
-# See BUILDING.txt.
-
-FROM maven:3.5-jdk-8
-
-RUN apt-get -q update && apt-get -q install --no-install-recommends -y \
- git \
- bats \
- findbugs \
- libperl-critic-perl \
- pylint \
- python-dateutil \
- rsync \
- make \
- gcc \
- libc6-dev \
- ruby \
- ruby-dev \
- wget \
- && \
- gem install --no-document rake rubocop ruby-lint
-
-ENV FINDBUGS_HOME /usr
-
-####
-# Install shellcheck
-###
-RUN mkdir -p /opt/shellcheck && \
- curl -L -s -S \
- https://storage.googleapis.com/shellcheck/shellcheck-stable.linux.x86_64.tar.xz \
- -o /opt/shellcheck.tar.xz && \
- tar xJf /opt/shellcheck.tar.xz --strip-components 1 -C /opt/shellcheck && \
- ln -s /opt/shellcheck/shellcheck /usr/bin/shellcheck && \
- rm -f /opt/shellcheck.tar.xz
+#
+# Dockerfile used as the build and test environment for Yetus.
+#
+# Built in multiple stages so as to avoid re-downloading large binaries when
+# tweaking unrelated aspects of the image.
-###
-# Avoid out of memory errors in builds
-###
-ENV MAVEN_OPTS -Xmx3g
+# start with a minimal image into which we can download remote tarballs
+FROM ubuntu:18.04 AS BASE_IMAGE
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
+# hadolint ignore=DL3009
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq update && \
+ DEBIAN_FRONTEND=noninteractive apt-get -qq install --no-install-recommends -y \
+ ca-certificates=20180409 \
+ curl=7.58.0-2ubuntu3.8 \
+ locales=2.27-3ubuntu1
+
+RUN locale-gen en_US.UTF-8
+ENV LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8
+
+##
+# download sundry dependencies
+#
+
+FROM BASE_IMAGE AS FINDBUGS_DOWNLOAD_IMAGE
+ENV FINDBUGS_VERSION '3.0.1'
+ENV FINDBUGS_URL "https://downloads.sourceforge.net/project/findbugs/findbugs/${FINDBUGS_VERSION}/findbugs-${FINDBUGS_VERSION}.tar.gz"
+ENV FINDBUGS_SHA256 'e80e0da0c213a27504ef3188ef25f107651700ffc66433eac6a7454bbe336419'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/findbugs.tar.gz "${FINDBUGS_URL}" && \
+ echo "${FINDBUGS_SHA256} */tmp/findbugs.tar.gz" | sha256sum -c -
+
+FROM BASE_IMAGE AS HADOLINT_DOWNLOAD_IMAGE
+ENV HADOLINT_VERSION '1.17.5'
+ENV HADOLINT_URL "https://github.com/hadolint/hadolint/releases/download/v${HADOLINT_VERSION}/hadolint-Linux-x86_64"
+ENV HADOLINT_SHA256 '20dd38bc0602040f19268adc14c3d1aae11af27b463af43f3122076baf827a35'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/hadolint "${HADOLINT_URL}" && \
+ echo "${HADOLINT_SHA256} */tmp/hadolint" | sha256sum -c -
+
+FROM BASE_IMAGE AS MAVEN_DOWNLOAD_IMAGE
+ENV MAVEN_VERSION='3.5.4'
+ENV MAVEN_URL "https://archive.apache.org/dist/maven/maven-3/${MAVEN_VERSION}/binaries/apache-maven-${MAVEN_VERSION}-bin.tar.gz"
+ENV MAVEN_SHA256 'ce50b1c91364cb77efe3776f756a6d92b76d9038b0a0782f7d53acf1e997a14d'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/maven.tar.gz "${MAVEN_URL}" && \
+ echo "${MAVEN_SHA256} */tmp/maven.tar.gz" | sha256sum -c -
+
+FROM BASE_IMAGE AS OPENJDK8_DOWNLOAD_IMAGE
+ENV OPENJDK8_URL 'https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u232-b09/OpenJDK8U-jdk_x64_linux_hotspot_8u232b09.tar.gz'
+ENV OPENJDK8_SHA256 '7b7884f2eb2ba2d47f4c0bf3bb1a2a95b73a3a7734bd47ebf9798483a7bcc423'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/adoptopenjdk8.tar.gz "${OPENJDK8_URL}" && \
+ echo "${OPENJDK8_SHA256} */tmp/adoptopenjdk8.tar.gz" | sha256sum -c -
+
+FROM BASE_IMAGE AS OPENJDK11_DOWNLOAD_IMAGE
+ENV OPENJDK11_URL 'https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.6%2B10/OpenJDK11U-jdk_x64_linux_hotspot_11.0.6_10.tar.gz'
+ENV OPENJDK11_SHA256 '330d19a2eaa07ed02757d7a785a77bab49f5ee710ea03b4ee2fa220ddd0feffc'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/adoptopenjdk11.tar.gz "${OPENJDK11_URL}" && \
+ echo "${OPENJDK11_SHA256} */tmp/adoptopenjdk11.tar.gz" | sha256sum -c -
+
+##
+# build the final image
+#
+
+FROM BASE_IMAGE
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
+##
+# install dependencies from system packages.
+# be careful not to install any system packages (i.e., findbugs) that will
+# pull in the default-jre.
+#
+
+# bring the base image into conformance with the expectations imposed by
+# Yetus and our personality file of what a build environment looks like.
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq install --no-install-recommends -y \
+ bash=4.4.18-2ubuntu1.2 \
+ build-essential=12.4ubuntu1 \
+ curl=7.58.0-2ubuntu3.8 \
+ diffutils=1:3.6-1 \
+ git=1:2.17.1-1ubuntu0.5 \
+ rsync=3.1.2-2.1ubuntu1 \
+ tar=1.29b-2ubuntu0.1 \
+ wget=1.19.4-1ubuntu2.2
+
+# install the dependencies required in order to enable the sundry precommit
+# checks/features provided by Yetus plugins.
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq install --no-install-recommends -y \
+ bats=0.4.0-1.1 \
+ libperl-critic-perl=1.130-1 \
+ python3=3.6.7-1~18.04 \
+ python3-pip=9.0.1-2.3~ubuntu1.18.04.1 \
+ python3-setuptools=39.0.1-2 \
+ ruby=1:2.5.1 \
+ ruby-dev=1:2.5.1 \
+ shellcheck=0.4.6-1 \
+ && \
+ apt-get clean && \
+ rm -rf /var/lib/apt/lists/*
+
+RUN python3 -mpip install --upgrade pip && \
+ python3 -mpip install pylint==2.4.4
+
+RUN gem install --no-document \
+ rake:13.0.1 \
+ rubocop:0.80.0 \
+ ruby-lint:2.3.1
+
+# hadolint ignore=DL3010
+COPY --from=FINDBUGS_DOWNLOAD_IMAGE /tmp/findbugs.tar.gz /tmp/findbugs.tar.gz
+RUN tar xzf /tmp/findbugs.tar.gz -C /opt && \
+ ln -s "/opt/$(dirname "$(tar -tf /tmp/findbugs.tar.gz | head -n1)")" /opt/findbugs && \
+ rm /tmp/findbugs.tar.gz
+
+COPY --from=HADOLINT_DOWNLOAD_IMAGE /tmp/hadolint /tmp/hadolint
+RUN mv /tmp/hadolint /usr/local/bin && \
+ chmod a+x /usr/local/bin/hadolint
+
+# hadolint ignore=DL3010
+COPY --from=MAVEN_DOWNLOAD_IMAGE /tmp/maven.tar.gz /tmp/maven.tar.gz
+RUN tar xzf /tmp/maven.tar.gz -C /opt && \
+ ln -s "/opt/$(dirname "$(tar -tf /tmp/maven.tar.gz | head -n1)")" /opt/maven && \
+ rm /tmp/maven.tar.gz
+
+##
+# ensure JVMs are available under `/usr/lib/jvm` and prefix each installation
+# as `java-` so as to conform with Yetus's assumptions.
+#
+
Review comment:
I dropped it following our discussion last friday, and realizing that the migration to all branches building for the same Dockerfile will be a larger effort. It can be restored if feel strongly, but I don't think it provides any benefit at this time.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-591542643
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/7/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-594818608
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 4m 32s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
||| _ Patch Compile Tests _ |
||| _ Other Tests _ |
| | | 5m 42s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/10/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | |
| uname | Linux 083b0a025567 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / b6eefcaeb7 |
| Max. process+thread count | 54 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/10/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-595379385
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/11/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1195:
HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
busbey commented on a change in pull request #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r384752663
##########
File path: dev-support/Jenkinsfile_GitHub
##########
@@ -144,8 +143,9 @@ pipeline {
YETUS_ARGS+=("--docker")
YETUS_ARGS+=("--dockerfile=${DOCKERFILE}")
YETUS_ARGS+=("--mvn-custom-repos")
- YETUS_ARGS+=("--multijdkdirs=/usr/lib/jvm/java-8-openjdk-amd64")
- YETUS_ARGS+=("--findbugs-home=/usr")
+ YETUS_ARGS+=("--java-home=/usr/lib/jvm/java-8")
+ YETUS_ARGS+=("--multijdkdirs=/usr/lib/jvm/java-8,/usr/lib/jvm/java-11")
+ YETUS_ARGS+=("--multijdktests=compile,unit")
Review comment:
huh. I thought multijdk handling of unit tests didn't work. ref: https://s.apache.org/fqjcn
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-590653193
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 12s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 8s | master passed |
| +1 :green_heart: | compile | 4m 3s | master passed with JDK v2020-01-14 |
| +1 :green_heart: | compile | 3m 19s | master passed with JDK v1.8.0_232 |
| +1 :green_heart: | shadedjars | 5m 8s | branch has no errors when building our shaded downstream artifacts. |
| -1 :x: | javadoc | 0m 20s | root in master failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 2m 49s | master passed with JDK v1.8.0_232 |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 31s | the patch passed |
| +1 :green_heart: | compile | 4m 1s | the patch passed with JDK v2020-01-14 |
| +1 :green_heart: | javac | 4m 1s | the patch passed |
| +1 :green_heart: | compile | 3m 22s | the patch passed with JDK v1.8.0_232 |
| +1 :green_heart: | javac | 3m 22s | the patch passed |
| +1 :green_heart: | hadolint | 0m 2s | The patch generated 0 new + 0 unchanged - 4 fixed = 0 total (was 4) |
| +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedjars | 5m 5s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 17m 41s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | javadoc | 0m 18s | root in the patch failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 2m 51s | the patch passed with JDK v1.8.0_232 |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 146m 3s | root in the patch passed with JDK v1.8.0_232. |
| +1 :green_heart: | asflicense | 0m 31s | The patch does not generate ASF License warnings. |
| | | 222m 13s | |
| Reason | Tests |
|-------:|:------|
| JDK v2020-01-14 Failed junit tests | hadoop.hbase.util.TestFutureUtils |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | dupname asflicense hadolint shellcheck shelldocs javac javadoc unit shadedjars hadoopcheck xml compile |
| uname | Linux aa09dec11b52 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1195/out/precommit/personality/provided.sh |
| git revision | master / 3e21dc73de |
| Default Java | 1.8.0_232 |
| Multi-JDK versions | /usr/lib/jvm/jdk-11.0.6+10:2020-01-14 /usr/lib/jvm/jdk8u232-b09:1.8.0_232 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/3/artifact/out/branch-javadoc-root-jdk2020-01-14.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/3/artifact/out/patch-javadoc-root-jdk2020-01-14.txt |
| JDK v1.8.0_232 Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/3/testReport/ |
| Max. process+thread count | 6851 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/3/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 hadolint=1.17.5-0-g443423c |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1195:
HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
busbey commented on a change in pull request #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r387821101
##########
File path: dev-support/Jenkinsfile
##########
@@ -49,6 +50,8 @@ pipeline {
// These tests currently have known failures. Once they burn down to 0, remove from here so that new problems will cause a failure.
TESTS_FILTER = 'cc,checkstyle,javac,javadoc,pylint,shellcheck,whitespace,perlcritic,ruby-lint,rubocop,mvnsite'
EXCLUDE_TESTS_URL = "${JENKINS_URL}/job/HBase-Find-Flaky-Tests/job/${BRANCH_NAME}/lastSuccessfulBuild/artifact/excludes"
+ SHALLOW_CHECKS = 'all,-unit,-findbugs' // run by the 'yetus general check'
+ DEEP_CHECKS = 'maven,mvninstall,compile,javac,unit,htmlout' // run by 'yetus jdkX HadoopY checks'
Review comment:
I believe we produce different shaded jars with different hadoop versions. is it worth pulling that check over into the hadoop specific builds? it shouldn't vary by jdk version.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ndimiduk commented on a change in pull request #1195:
HBASE-23876 Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on a change in pull request #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r388467877
##########
File path: dev-support/Jenkinsfile
##########
@@ -49,6 +50,8 @@ pipeline {
// These tests currently have known failures. Once they burn down to 0, remove from here so that new problems will cause a failure.
TESTS_FILTER = 'cc,checkstyle,javac,javadoc,pylint,shellcheck,whitespace,perlcritic,ruby-lint,rubocop,mvnsite'
EXCLUDE_TESTS_URL = "${JENKINS_URL}/job/HBase-Find-Flaky-Tests/job/${BRANCH_NAME}/lastSuccessfulBuild/artifact/excludes"
+ SHALLOW_CHECKS = 'all,-unit,-findbugs' // run by the 'yetus general check'
Review comment:
Oh, I see. Thanks for the pointer.
Reading through the nightly code again, I'm back to believing that I'll break branch-1. Let me add back JDK7 to the docker file as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-594818618
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 4m 3s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | hadolint | 0m 2s | The patch generated 0 new + 0 unchanged - 4 fixed = 0 total (was 4) |
| +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. |
| | | 5m 43s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/10/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | dupname asflicense hadolint shellcheck shelldocs |
| uname | Linux d915ab04e35f 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / b6eefcaeb7 |
| Max. process+thread count | 47 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/10/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 hadolint=1.17.5-0-g443423c |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-591685727
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 12s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 15s | master passed |
| +1 :green_heart: | compile | 4m 34s | master passed with JDK v2020-01-14 |
| +1 :green_heart: | compile | 3m 44s | master passed with JDK v1.8.0_232 |
| +1 :green_heart: | shadedjars | 5m 25s | branch has no errors when building our shaded downstream artifacts. |
| -1 :x: | javadoc | 0m 21s | root in master failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 2m 58s | master passed with JDK v1.8.0_232 |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 38s | the patch passed |
| +1 :green_heart: | compile | 4m 21s | the patch passed with JDK v2020-01-14 |
| +1 :green_heart: | javac | 4m 21s | the patch passed |
| +1 :green_heart: | compile | 3m 23s | the patch passed with JDK v1.8.0_232 |
| +1 :green_heart: | javac | 3m 23s | the patch passed |
| +1 :green_heart: | hadolint | 0m 2s | The patch generated 0 new + 0 unchanged - 4 fixed = 0 total (was 4) |
| +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedjars | 5m 7s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 18m 24s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | javadoc | 0m 19s | root in the patch failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 2m 56s | the patch passed with JDK v1.8.0_232 |
||| _ Other Tests _ |
| -1 :x: | unit | 4m 41s | root in the patch failed with JDK v1.8.0_232. |
| +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. |
| | | 78m 48s | |
| Reason | Tests |
|-------:|:------|
| JDK v2020-01-14 Failed junit tests | hadoop.hbase.util.TestFutureUtils |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/8/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | dupname asflicense hadolint shellcheck shelldocs javac javadoc unit shadedjars hadoopcheck xml compile |
| uname | Linux 2e0b70ca6585 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1195/out/precommit/personality/provided.sh |
| git revision | master / 3c3aae9a2f |
| Default Java | 1.8.0_232 |
| Multi-JDK versions | /usr/lib/jvm/jdk-11.0.6+10:2020-01-14 /usr/lib/jvm/jdk8u232-b09:1.8.0_232 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/8/artifact/out/branch-javadoc-root-jdk2020-01-14.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/8/artifact/out/patch-javadoc-root-jdk2020-01-14.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/8/artifact/out/patch-unit-root-jdk1.8.0_232.txt |
| JDK v1.8.0_232 Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/8/testReport/ |
| Max. process+thread count | 582 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/8/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 hadolint=1.17.5-0-g443423c |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1195:
HBASE-23876 Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
busbey commented on a change in pull request #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r387869420
##########
File path: dev-support/Jenkinsfile
##########
@@ -49,6 +50,8 @@ pipeline {
// These tests currently have known failures. Once they burn down to 0, remove from here so that new problems will cause a failure.
TESTS_FILTER = 'cc,checkstyle,javac,javadoc,pylint,shellcheck,whitespace,perlcritic,ruby-lint,rubocop,mvnsite'
EXCLUDE_TESTS_URL = "${JENKINS_URL}/job/HBase-Find-Flaky-Tests/job/${BRANCH_NAME}/lastSuccessfulBuild/artifact/excludes"
+ SHALLOW_CHECKS = 'all,-unit,-findbugs' // run by the 'yetus general check'
Review comment:
we used to do findbugs just during one of the hadoop checks I think. line 345 prior to this changest.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-591199260
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 6s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 1s | master passed |
| +1 :green_heart: | compile | 4m 4s | master passed with JDK v2020-01-14 |
| +1 :green_heart: | compile | 3m 25s | master passed with JDK v1.8.0_232 |
| +1 :green_heart: | shadedjars | 5m 11s | branch has no errors when building our shaded downstream artifacts. |
| -1 :x: | javadoc | 0m 20s | root in master failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 2m 49s | master passed with JDK v1.8.0_232 |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 42s | the patch passed |
| +1 :green_heart: | compile | 4m 1s | the patch passed with JDK v2020-01-14 |
| +1 :green_heart: | javac | 4m 1s | the patch passed |
| +1 :green_heart: | compile | 3m 19s | the patch passed with JDK v1.8.0_232 |
| +1 :green_heart: | javac | 3m 19s | the patch passed |
| +1 :green_heart: | hadolint | 0m 1s | The patch generated 0 new + 0 unchanged - 4 fixed = 0 total (was 4) |
| +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedjars | 5m 4s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 17m 42s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | javadoc | 0m 18s | root in the patch failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 2m 50s | the patch passed with JDK v1.8.0_232 |
||| _ Other Tests _ |
| -1 :x: | unit | 116m 52s | root in the patch failed with JDK v1.8.0_232. |
| +1 :green_heart: | asflicense | 0m 44s | The patch does not generate ASF License warnings. |
| | | 188m 31s | |
| Reason | Tests |
|-------:|:------|
| JDK v2020-01-14 Failed junit tests | hadoop.hbase.util.TestFutureUtils |
| JDK v1.8.0_232 Failed junit tests | hadoop.hbase.replication.regionserver.TestWALEntryStream |
| | hadoop.hbase.io.hfile.bucket.TestBucketCache |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | dupname asflicense hadolint shellcheck shelldocs javac javadoc unit shadedjars hadoopcheck xml compile |
| uname | Linux 08b53edc990e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1195/out/precommit/personality/provided.sh |
| git revision | master / 9f223c2236 |
| Default Java | 1.8.0_232 |
| Multi-JDK versions | /usr/lib/jvm/jdk-11.0.6+10:2020-01-14 /usr/lib/jvm/jdk8u232-b09:1.8.0_232 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/5/artifact/out/branch-javadoc-root-jdk2020-01-14.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/5/artifact/out/patch-javadoc-root-jdk2020-01-14.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/5/artifact/out/patch-unit-root-jdk1.8.0_232.txt |
| JDK v1.8.0_232 Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/5/testReport/ |
| Max. process+thread count | 5304 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/5/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 hadolint=1.17.5-0-g443423c |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ndimiduk commented on issue #1195: HBASE-23876 [WIP] Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-589895771
@busbey comments like https://github.com/apache/hbase/blob/b53e9a325cffee0a2dba53de5afd5e63084c334e/dev-support/Jenkinsfile#L198-L200
and this bit of logic https://github.com/apache/hbase/blob/b53e9a325cffee0a2dba53de5afd5e63084c334e/dev-support/Jenkinsfile#L281-L294
(notice the scm checkout is performed into the dir `component`, but the yetus_nightly script is invoked from `BASEDIR`)
makes me think that nightlies are not so closely tied to their branch-local Jenkinsfiles as I originally understood.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] jatsakthi commented on issue #1195: HBASE-23876 [WIP] Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
jatsakthi commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-592280841
@ndimiduk / @busbey what would be the best way to test the Jenkins file changes over here, with a local docker image built out of your changes here / #1183 to verify the output that the yetus is able to generate?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-595380905
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 3m 12s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
||| _ Patch Compile Tests _ |
||| _ Other Tests _ |
| | | 4m 17s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/11/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | |
| uname | Linux c14b7c27aaf6 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / aff0ff5d97 |
| Max. process+thread count | 52 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/11/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] busbey commented on a change in pull request #1195:
HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
busbey commented on a change in pull request #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r384865833
##########
File path: dev-support/docker/Dockerfile
##########
@@ -14,47 +14,162 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-# Dockerfile for installing the necessary dependencies for building Hadoop.
-# See BUILDING.txt.
-
-FROM maven:3.5-jdk-8
-
-RUN apt-get -q update && apt-get -q install --no-install-recommends -y \
- git \
- bats \
- findbugs \
- libperl-critic-perl \
- pylint \
- python-dateutil \
- rsync \
- make \
- gcc \
- libc6-dev \
- ruby \
- ruby-dev \
- wget \
- && \
- gem install --no-document rake rubocop ruby-lint
-
-ENV FINDBUGS_HOME /usr
-
-####
-# Install shellcheck
-###
-RUN mkdir -p /opt/shellcheck && \
- curl -L -s -S \
- https://storage.googleapis.com/shellcheck/shellcheck-stable.linux.x86_64.tar.xz \
- -o /opt/shellcheck.tar.xz && \
- tar xJf /opt/shellcheck.tar.xz --strip-components 1 -C /opt/shellcheck && \
- ln -s /opt/shellcheck/shellcheck /usr/bin/shellcheck && \
- rm -f /opt/shellcheck.tar.xz
+#
+# Dockerfile used as the build and test environment for Yetus.
+#
+# Built in multiple stages so as to avoid re-downloading large binaries when
+# tweaking unrelated aspects of the image.
-###
-# Avoid out of memory errors in builds
-###
-ENV MAVEN_OPTS -Xmx3g
+# start with a minimal image into which we can download remote tarballs
+FROM ubuntu:18.04 AS BASE_IMAGE
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
+# hadolint ignore=DL3009
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq update && \
+ DEBIAN_FRONTEND=noninteractive apt-get -qq install --no-install-recommends -y \
+ ca-certificates=20180409 \
+ curl=7.58.0-2ubuntu3.8 \
+ locales=2.27-3ubuntu1
+
+RUN locale-gen en_US.UTF-8
+ENV LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8
+
+##
+# download sundry dependencies
+#
+
+FROM BASE_IMAGE AS FINDBUGS_DOWNLOAD_IMAGE
+ENV FINDBUGS_VERSION '3.0.1'
+ENV FINDBUGS_URL "https://downloads.sourceforge.net/project/findbugs/findbugs/${FINDBUGS_VERSION}/findbugs-${FINDBUGS_VERSION}.tar.gz"
+ENV FINDBUGS_SHA256 'e80e0da0c213a27504ef3188ef25f107651700ffc66433eac6a7454bbe336419'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/findbugs.tar.gz "${FINDBUGS_URL}" && \
+ echo "${FINDBUGS_SHA256} */tmp/findbugs.tar.gz" | sha256sum -c -
+
+FROM BASE_IMAGE AS HADOLINT_DOWNLOAD_IMAGE
+ENV HADOLINT_VERSION '1.17.5'
+ENV HADOLINT_URL "https://github.com/hadolint/hadolint/releases/download/v${HADOLINT_VERSION}/hadolint-Linux-x86_64"
+ENV HADOLINT_SHA256 '20dd38bc0602040f19268adc14c3d1aae11af27b463af43f3122076baf827a35'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/hadolint "${HADOLINT_URL}" && \
+ echo "${HADOLINT_SHA256} */tmp/hadolint" | sha256sum -c -
+
+FROM BASE_IMAGE AS MAVEN_DOWNLOAD_IMAGE
+ENV MAVEN_VERSION='3.5.4'
+ENV MAVEN_URL "https://archive.apache.org/dist/maven/maven-3/${MAVEN_VERSION}/binaries/apache-maven-${MAVEN_VERSION}-bin.tar.gz"
+ENV MAVEN_SHA256 'ce50b1c91364cb77efe3776f756a6d92b76d9038b0a0782f7d53acf1e997a14d'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/maven.tar.gz "${MAVEN_URL}" && \
+ echo "${MAVEN_SHA256} */tmp/maven.tar.gz" | sha256sum -c -
+
+FROM BASE_IMAGE AS OPENJDK8_DOWNLOAD_IMAGE
+ENV OPENJDK8_URL 'https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u232-b09/OpenJDK8U-jdk_x64_linux_hotspot_8u232b09.tar.gz'
+ENV OPENJDK8_SHA256 '7b7884f2eb2ba2d47f4c0bf3bb1a2a95b73a3a7734bd47ebf9798483a7bcc423'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/adoptopenjdk8.tar.gz "${OPENJDK8_URL}" && \
+ echo "${OPENJDK8_SHA256} */tmp/adoptopenjdk8.tar.gz" | sha256sum -c -
+
+FROM BASE_IMAGE AS OPENJDK11_DOWNLOAD_IMAGE
+ENV OPENJDK11_URL 'https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.6%2B10/OpenJDK11U-jdk_x64_linux_hotspot_11.0.6_10.tar.gz'
+ENV OPENJDK11_SHA256 '330d19a2eaa07ed02757d7a785a77bab49f5ee710ea03b4ee2fa220ddd0feffc'
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+RUN curl -LfsSo /tmp/adoptopenjdk11.tar.gz "${OPENJDK11_URL}" && \
+ echo "${OPENJDK11_SHA256} */tmp/adoptopenjdk11.tar.gz" | sha256sum -c -
+
+##
+# build the final image
+#
+
+FROM BASE_IMAGE
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
+##
+# install dependencies from system packages.
+# be careful not to install any system packages (i.e., findbugs) that will
+# pull in the default-jre.
+#
+
+# bring the base image into conformance with the expectations imposed by
+# Yetus and our personality file of what a build environment looks like.
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq install --no-install-recommends -y \
+ bash=4.4.18-2ubuntu1.2 \
+ build-essential=12.4ubuntu1 \
+ curl=7.58.0-2ubuntu3.8 \
+ diffutils=1:3.6-1 \
+ git=1:2.17.1-1ubuntu0.5 \
+ rsync=3.1.2-2.1ubuntu1 \
+ tar=1.29b-2ubuntu0.1 \
+ wget=1.19.4-1ubuntu2.2
+
+# install the dependencies required in order to enable the sundry precommit
+# checks/features provided by Yetus plugins.
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq install --no-install-recommends -y \
+ bats=0.4.0-1.1 \
+ libperl-critic-perl=1.130-1 \
+ python3=3.6.7-1~18.04 \
+ python3-pip=9.0.1-2.3~ubuntu1.18.04.1 \
+ python3-setuptools=39.0.1-2 \
+ ruby=1:2.5.1 \
+ ruby-dev=1:2.5.1 \
+ shellcheck=0.4.6-1 \
+ && \
+ apt-get clean && \
+ rm -rf /var/lib/apt/lists/*
+
+RUN python3 -mpip install --upgrade pip && \
+ python3 -mpip install pylint==2.4.4
+
+RUN gem install --no-document \
+ rake:13.0.1 \
+ rubocop:0.80.0 \
+ ruby-lint:2.3.1
+
+# hadolint ignore=DL3010
+COPY --from=FINDBUGS_DOWNLOAD_IMAGE /tmp/findbugs.tar.gz /tmp/findbugs.tar.gz
+RUN tar xzf /tmp/findbugs.tar.gz -C /opt && \
+ ln -s "/opt/$(dirname "$(tar -tf /tmp/findbugs.tar.gz | head -n1)")" /opt/findbugs && \
+ rm /tmp/findbugs.tar.gz
+
+COPY --from=HADOLINT_DOWNLOAD_IMAGE /tmp/hadolint /tmp/hadolint
+RUN mv /tmp/hadolint /usr/local/bin && \
+ chmod a+x /usr/local/bin/hadolint
+
+# hadolint ignore=DL3010
+COPY --from=MAVEN_DOWNLOAD_IMAGE /tmp/maven.tar.gz /tmp/maven.tar.gz
+RUN tar xzf /tmp/maven.tar.gz -C /opt && \
+ ln -s "/opt/$(dirname "$(tar -tf /tmp/maven.tar.gz | head -n1)")" /opt/maven && \
+ rm /tmp/maven.tar.gz
+
+##
+# ensure JVMs are available under `/usr/lib/jvm` and prefix each installation
+# as `java-` so as to conform with Yetus's assumptions.
+#
+
Review comment:
Ah. That makes sense. Fine by me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-589918372
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 20s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +0 :ok: | shelldocs | 0m 1s | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 2s | master passed |
| +1 :green_heart: | compile | 4m 1s | master passed with JDK v2020-01-14 |
| +1 :green_heart: | compile | 3m 20s | master passed with JDK v1.8.0_232 |
| +1 :green_heart: | shadedjars | 5m 11s | branch has no errors when building our shaded downstream artifacts. |
| -1 :x: | javadoc | 0m 48s | root in master failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 5m 37s | master passed with JDK v1.8.0_232 |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 20s | the patch passed |
| +1 :green_heart: | compile | 6m 2s | the patch passed with JDK v2020-01-14 |
| +1 :green_heart: | javac | 6m 2s | the patch passed |
| +1 :green_heart: | compile | 3m 34s | the patch passed with JDK v1.8.0_232 |
| +1 :green_heart: | javac | 3m 35s | the patch passed |
| +1 :green_heart: | hadolint | 0m 1s | The patch generated 0 new + 0 unchanged - 4 fixed = 0 total (was 4) |
| +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedjars | 5m 6s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 17m 43s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | javadoc | 0m 19s | root in the patch failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 2m 50s | the patch passed with JDK v1.8.0_232 |
||| _ Other Tests _ |
| -1 :x: | unit | 156m 38s | root in the patch failed with JDK v1.8.0_232. |
| +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. |
| | | 239m 25s | |
| Reason | Tests |
|-------:|:------|
| JDK v2020-01-14 Failed junit tests | hadoop.hbase.util.TestFutureUtils |
| JDK v1.8.0_232 Failed junit tests | hadoop.hbase.rsgroup.TestRSGroupsKillRS |
| | hadoop.hbase.rsgroup.TestRSGroupsBasics |
| | hadoop.hbase.rsgroup.TestEnableRSGroups |
| | hadoop.hbase.master.procedure.TestSCPWithReplicasWithRSGroup |
| | hadoop.hbase.rsgroup.TestRSGroupsOfflineMode |
| | hadoop.hbase.rsgroup.TestRSGroupsWithACL |
| | hadoop.hbase.rsgroup.TestRSGroupsAdmin1 |
| | hadoop.hbase.rsgroup.TestRSGroupMajorCompactionTTL |
| | hadoop.hbase.rsgroup.TestRSGroupsBalance |
| | hadoop.hbase.coprocessor.TestCoprocessorServiceBackwardCompatibility |
| | hadoop.hbase.coprocessor.TestBatchCoprocessorEndpoint |
| | hadoop.hbase.coprocessor.TestCoprocessorTableEndpoint |
| | hadoop.hbase.coprocessor.TestRowProcessorEndpoint |
| | hadoop.hbase.coprocessor.TestClassLoading |
| | hadoop.hbase.regionserver.TestServerCustomProtocol |
| | hadoop.hbase.coprocessor.TestImportExport |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | dupname asflicense hadolint shellcheck shelldocs javac javadoc unit shadedjars hadoopcheck xml compile |
| uname | Linux 857c24b655d9 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1195/out/precommit/personality/provided.sh |
| git revision | master / 400b7ce954 |
| Default Java | 1.8.0_232 |
| Multi-JDK versions | /usr/lib/jvm/jdk-11.0.6+10:2020-01-14 /usr/lib/jvm/jdk8u232-b09:1.8.0_232 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/1/artifact/out/branch-javadoc-root-jdk2020-01-14.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/1/artifact/out/patch-javadoc-root-jdk2020-01-14.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/1/artifact/out/patch-unit-root-jdk1.8.0_232.txt |
| JDK v1.8.0_232 Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/1/testReport/ |
| Max. process+thread count | 6381 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/1/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 hadolint=1.17.5-0-g443423c |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-594677964
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/9/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-595380223
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 37s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --findbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
||| _ Patch Compile Tests _ |
||| _ Other Tests _ |
| | | 2m 43s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/11/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | |
| uname | Linux a8ad6875fa65 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / aff0ff5d97 |
| Max. process+thread count | 46 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/11/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-590503411
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/2/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ndimiduk commented on issue #1195: HBASE-23876 [WIP] Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-591173981
Okay, looks like it's running. At least I'm getting as far in Jenkins as I get trying to build and test locally. Check out:
https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-589896037
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/1/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-590560101
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 3m 58s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 37s | master passed |
| +1 :green_heart: | compile | 3m 38s | master passed with JDK v2020-01-14 |
| +1 :green_heart: | compile | 3m 0s | master passed with JDK v1.8.0_232 |
| +1 :green_heart: | shadedjars | 4m 41s | branch has no errors when building our shaded downstream artifacts. |
| -1 :x: | javadoc | 0m 21s | root in master failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 2m 46s | master passed with JDK v1.8.0_232 |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 7s | the patch passed |
| +1 :green_heart: | compile | 3m 36s | the patch passed with JDK v2020-01-14 |
| +1 :green_heart: | javac | 3m 36s | the patch passed |
| +1 :green_heart: | compile | 3m 2s | the patch passed with JDK v1.8.0_232 |
| +1 :green_heart: | javac | 3m 2s | the patch passed |
| +1 :green_heart: | hadolint | 0m 2s | The patch generated 0 new + 0 unchanged - 4 fixed = 0 total (was 4) |
| +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedjars | 4m 40s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | hadoopcheck | 15m 59s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. |
| -1 :x: | javadoc | 0m 20s | root in the patch failed with JDK v2020-01-14. |
| +1 :green_heart: | javadoc | 2m 47s | the patch passed with JDK v1.8.0_232 |
||| _ Other Tests _ |
| -1 :x: | unit | 66m 12s | root in the patch failed with JDK v1.8.0_232. |
| +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. |
| | | 134m 32s | |
| Reason | Tests |
|-------:|:------|
| JDK v2020-01-14 Failed junit tests | hadoop.hbase.util.TestFutureUtils |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/1195 |
| Optional Tests | dupname asflicense hadolint shellcheck shelldocs javac javadoc unit shadedjars hadoopcheck xml compile |
| uname | Linux 5a0703ef1b8c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1195/out/precommit/personality/provided.sh |
| git revision | master / 3e21dc73de |
| Default Java | 1.8.0_232 |
| Multi-JDK versions | /usr/lib/jvm/jdk-11.0.6+10:2020-01-14 /usr/lib/jvm/jdk8u232-b09:1.8.0_232 |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/2/artifact/out/branch-javadoc-root-jdk2020-01-14.txt |
| javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/2/artifact/out/patch-javadoc-root-jdk2020-01-14.txt |
| unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/2/artifact/out/patch-unit-root-jdk1.8.0_232.txt |
| JDK v1.8.0_232 Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/2/testReport/ |
| Max. process+thread count | 8803 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/2/console |
| versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 hadolint=1.17.5-0-g443423c |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 [WIP]
Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-591199907
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/6/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ndimiduk commented on a change in pull request #1195:
HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
ndimiduk commented on a change in pull request #1195: HBASE-23876 [WIP] Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#discussion_r384848856
##########
File path: dev-support/docker/Dockerfile
##########
@@ -14,47 +14,162 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-# Dockerfile for installing the necessary dependencies for building Hadoop.
-# See BUILDING.txt.
-
-FROM maven:3.5-jdk-8
-
-RUN apt-get -q update && apt-get -q install --no-install-recommends -y \
- git \
- bats \
- findbugs \
- libperl-critic-perl \
- pylint \
- python-dateutil \
- rsync \
- make \
- gcc \
- libc6-dev \
- ruby \
- ruby-dev \
- wget \
- && \
- gem install --no-document rake rubocop ruby-lint
-
-ENV FINDBUGS_HOME /usr
-
-####
-# Install shellcheck
-###
-RUN mkdir -p /opt/shellcheck && \
- curl -L -s -S \
- https://storage.googleapis.com/shellcheck/shellcheck-stable.linux.x86_64.tar.xz \
- -o /opt/shellcheck.tar.xz && \
- tar xJf /opt/shellcheck.tar.xz --strip-components 1 -C /opt/shellcheck && \
- ln -s /opt/shellcheck/shellcheck /usr/bin/shellcheck && \
- rm -f /opt/shellcheck.tar.xz
+#
+# Dockerfile used as the build and test environment for Yetus.
+#
+# Built in multiple stages so as to avoid re-downloading large binaries when
+# tweaking unrelated aspects of the image.
-###
-# Avoid out of memory errors in builds
-###
-ENV MAVEN_OPTS -Xmx3g
+# start with a minimal image into which we can download remote tarballs
+FROM ubuntu:18.04 AS BASE_IMAGE
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
+# hadolint ignore=DL3009
+RUN DEBIAN_FRONTEND=noninteractive apt-get -qq update && \
+ DEBIAN_FRONTEND=noninteractive apt-get -qq install --no-install-recommends -y \
+ ca-certificates=20180409 \
+ curl=7.58.0-2ubuntu3.8 \
+ locales=2.27-3ubuntu1
+
+RUN locale-gen en_US.UTF-8
+ENV LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8
+
+##
+# download sundry dependencies
+#
+
+FROM BASE_IMAGE AS FINDBUGS_DOWNLOAD_IMAGE
Review comment:
*nod*
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1195: HBASE-23876 Add
JDK11 compilation and unit test support to nightly job
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1195: HBASE-23876 Add JDK11 compilation and unit test support to nightly job
URL: https://github.com/apache/hbase/pull/1195#issuecomment-594816216
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1195/10/console in case of problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services