You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2021/08/26 21:24:24 UTC
[GitHub] [hbase] taklwu opened a new pull request #3634: HBASE-26168 Backport HBASE-25811 "The client integration test is fail…
taklwu opened a new pull request #3634:
URL: https://github.com/apache/hbase/pull/3634
…ing after HBASE-22120 merged" into branch-2
move opentelemetry jars to client-facing-thirdparty
add opentelemetry jars when init map reduce job dependencies
original commit 8d2a0efb7a9c17e58892369b281b4af49a5f0d7e
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] taklwu merged pull request #3634: HBASE-26168 Backport HBASE-25811 "The client integration test is fail…
Posted by GitBox <gi...@apache.org>.
taklwu merged pull request #3634:
URL: https://github.com/apache/hbase/pull/3634
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3634: HBASE-26168 Backport HBASE-25811 "The client integration test is fail…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3634:
URL: https://github.com/apache/hbase/pull/3634#issuecomment-906779221
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 48s | Docker mode activated. |
| -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ HBASE-25853 Compile Tests _ |
| +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 5m 51s | HBASE-25853 passed |
| +1 :green_heart: | compile | 1m 4s | HBASE-25853 passed |
| +1 :green_heart: | shadedjars | 10m 23s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 44s | HBASE-25853 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 5m 18s | the patch passed |
| +1 :green_heart: | compile | 1m 11s | the patch passed |
| +1 :green_heart: | javac | 1m 11s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 46s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 40s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 33s | hbase-mapreduce in the patch passed. |
| +1 :green_heart: | unit | 0m 27s | hbase-assembly in the patch passed. |
| | | 49m 13s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3634 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 0e7b949ce26a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-25853 / d807632686 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/testReport/ |
| Max. process+thread count | 4837 (vs. ulimit of 12500) |
| modules | C: hbase-mapreduce hbase-assembly U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] taklwu commented on pull request #3634: HBASE-26168 Backport HBASE-25811 "The client integration test is fail…
Posted by GitBox <gi...@apache.org>.
taklwu commented on pull request #3634:
URL: https://github.com/apache/hbase/pull/3634#issuecomment-906777428
this commit is required for the completion of HBASE-25853, otherwise the import TSV nightly job with shaded client artifact fail with NoClassDefFoundError exception. and I tested the nightly tests locally with the following steps (with outputs)
```
# first maven full build
HBASE_SOURCE=/Users/taklwu/workspace/hbase2
cd $HBASE_SOURCE
mvn clean install -DskipTests -Prelease && mvn install -DskipTests site assembly:single -Prelease
# unzip the assembly and hbase-client
cd ~/hbase-nightly
install_artifact=$(ls -1 ${HBASE_SOURCE}/hbase-assembly/target/hbase-*-bin.tar.gz | sort | head -n 1)
mkdir hbase-install
tar --strip-component=1 -xzf "${install_artifact}" -C "hbase-install"
client_artifact=$(ls -1 ${HBASE_SOURCE}/hbase-assembly/target/hbase-*-bin.tar.gz | sort | tail -n 1)
mkdir hbase-client
tar --strip-component=1 -xzf "${client_artifact}" -C "hbase-client"
# run the nightly
rm -rf output-integration/hadoop-2/*
bash $HBASE_SOURCE/dev-support/hbase_nightly_pseudo-distributed-test.sh --single-process --working-dir output-integration/hadoop-2 --hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop hadoop-2/share/hadoop/yarn/timelineservice hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.10.1-tests.jar hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.10.1-tests.jar hadoop-2/bin/mapred | tee ~/hbase-nightly/sanity-check.log
Using HBase client-side artifact
You'll find logs and temp files in /Users/taklwu/hbase-nightly/output-integration/hadoop-2
Hadoop version information:
Hadoop 2.10.1
Subversion https://github.com/apache/hadoop -r 1827467c9a56f133025f28557bfc2c562d78e816
Compiled by centos on 2020-09-14T13:17Z
Compiled with protoc 2.5.0
From source with checksum 3114edef868f1f3824e7d0f68be03650
This command was run using /Users/taklwu/hbase-nightly/hadoop-2/share/hadoop/common/hadoop-common-2.10.1.jar
JAVA_HOME: /Users/taklwu/.sdkman/candidates/java/8.292.10.1-amzn
HBase version information:
HBase 2.5.0-SNAPSHOT
Source code repository git://21992.local/Users/taklwu/workspace/hbase2 revision=723831a9de938faf7c2a1b2e4047b68ed281af52
Compiled by taklwu on Thu Aug 26 14:29:11 PDT 2021
From source with checksum c90bbbd1a0293de7e3621195fad2d2e6ade1a2b82fdcb05ee7690e3fb8b338a743d7428dee3ef18b9c6b6d7de1ea979bc8073f5b9bb686b24fc48aa4e8b48e90
Writing out configuration for HBase.
Starting up Hadoop
waiting for Hadoop to finish starting up.
waiting for Hadoop to finish starting up.
waiting for Hadoop to finish starting up.
Listing HDFS contents
Starting up HBase
running master, logging to /Users/taklwu/hbase-nightly/hbase-install/bin/../logs/hbase-taklwu-master-21992.local.out
retry waiting for hbase to come up.
Setting up table 'test:example' with 1,000 regions
writing out example TSV to example.tsv
uploading example.tsv to HDFS
2021-08-26 14:59:41,249 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2021-08-26 14:59:42,671 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Importing TSV via shaded client artifact for HBase - MapReduce integration.
Verifying row count from import.
Hadoop client jars not given; getting them from 'hadoop classpath' for the example.
Building shaded client example.
Running shaded client example. It'll fetch the set of regions, round-trip them to a file in HDFS, then write them one-per-row into the test table.
Checking on results of example program.
2021-08-26 15:02:29,669 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Verifying row count from example.
Found 1051 rows, which is enough to cover 48 for import, 1000 example's use of user table regions, 2 for example's use of meta/namespace regions, and 1 for example's count record
Shutting down HBase
stopping hbase............
Shutdown: listing HDFS contents
Shutting down Hadoop
# check the command exit code and output of mr-importtsv.err
% echo $?
0
% cat output-integration/hadoop-2/mr-importtsv.err
...
2021-08-26 15:00:00,482 INFO [main] mapreduce.Job: Job job_local361623923_0001 completed successfully
2021-08-26 15:00:00,487 INFO [main] mapreduce.Job: Counters: 21
File System Counters
FILE: Number of bytes read=47342109
FILE: Number of bytes written=48290866
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=756
HDFS: Number of bytes written=0
HDFS: Number of read operations=3
HDFS: Number of large read operations=0
HDFS: Number of write operations=0
Map-Reduce Framework
Map input records=48
Map output records=48
Input split bytes=119
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=0
Total committed heap usage (bytes)=556793856
ImportTsv
Bad Lines=0
File Input Format Counters
Bytes Read=756
File Output Format Counters
Bytes Written=0
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3634: HBASE-26168 Backport HBASE-25811 "The client integration test is fail…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3634:
URL: https://github.com/apache/hbase/pull/3634#issuecomment-906770784
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ HBASE-25853 Compile Tests _ |
| +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 3m 42s | HBASE-25853 passed |
| +1 :green_heart: | compile | 1m 14s | HBASE-25853 passed |
| +1 :green_heart: | checkstyle | 0m 33s | HBASE-25853 passed |
| +1 :green_heart: | spotbugs | 0m 47s | HBASE-25853 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 21s | the patch passed |
| +1 :green_heart: | compile | 1m 13s | the patch passed |
| +1 :green_heart: | javac | 1m 13s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 30s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 3s | The patch has no ill-formed XML file. |
| +1 :green_heart: | hadoopcheck | 11m 59s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. |
| +1 :green_heart: | spotbugs | 0m 55s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. |
| | | 33m 28s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3634 |
| Optional Tests | dupname asflicense javac xml spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 5d75ec136fa9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-25853 / d807632686 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Max. process+thread count | 96 (vs. ulimit of 12500) |
| modules | C: hbase-mapreduce hbase-assembly U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3634: HBASE-26168 Backport HBASE-25811 "The client integration test is fail…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3634:
URL: https://github.com/apache/hbase/pull/3634#issuecomment-906781068
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 40s | Docker mode activated. |
| -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ HBASE-25853 Compile Tests _ |
| +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 4m 50s | HBASE-25853 passed |
| +1 :green_heart: | compile | 0m 59s | HBASE-25853 passed |
| +1 :green_heart: | shadedjars | 8m 27s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 40s | HBASE-25853 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 44s | the patch passed |
| +1 :green_heart: | compile | 1m 2s | the patch passed |
| +1 :green_heart: | javac | 1m 2s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 40s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 38s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 18m 39s | hbase-mapreduce in the patch passed. |
| +1 :green_heart: | unit | 0m 30s | hbase-assembly in the patch passed. |
| | | 53m 10s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3634 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux b3f478963855 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-25853 / d807632686 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/testReport/ |
| Max. process+thread count | 2616 (vs. ulimit of 12500) |
| modules | C: hbase-mapreduce hbase-assembly U: . |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org