You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "NihalJain (via GitHub)" <gi...@apache.org> on 2023/02/27 18:02:12 UTC
[GitHub] [hbase-connectors] NihalJain opened a new pull request, #110: HBASE-27639 Support hbase-connectors compilation with HBase 2.5.3, Ha…
NihalJain opened a new pull request, #110:
URL: https://github.com/apache/hbase-connectors/pull/110
…doop 3.2.4 and Spark 3.2.3
* Added mockito-all as otherwise we get java.lang.NoClassDefFoundError: org/mockito/stubbing/Answer at org.apache.hadoop.hdfs.MiniDFSCluster.isNameNodeUp() for hadoop 3.2.4
* Exlclude hadoop-client-api and hadoop-client-runtime coming from Spark 3.2, else minicluster.start() fails
* Exclude lower versioned paranamer coming from avro otherwise tests fail with java.lang.ArrayIndexOutOfBoundsException
* Added spark.hadoopRDD.ignoreEmptySplits for test due to behaviour change in spark 3.2, where the below conf is true by default. We will get empty table as result (for small sized tables) for HBase version not having HBASE-26340
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] Apache-HBase commented on pull request #110: HBASE-27639 Support hbase-connectors compilation with HBase 2.5.3, Ha…
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #110:
URL: https://github.com/apache/hbase-connectors/pull/110#issuecomment-1446846343
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 42s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | spotbugs | 0m 0s | spotbugs executables are not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 1m 28s | master passed |
| +1 :green_heart: | compile | 0m 51s | master passed |
| +1 :green_heart: | javadoc | 0m 51s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 48s | the patch passed |
| +1 :green_heart: | compile | 0m 53s | the patch passed |
| +1 :green_heart: | javac | 0m 53s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. |
| +1 :green_heart: | javadoc | 0m 52s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 8m 20s | root in the patch passed. |
| | | 16m 12s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-110/1/artifact/yetus-precommit-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase-connectors/pull/110 |
| Optional Tests | dupname javac javadoc unit xml compile spotbugs findbugs |
| uname | Linux 3917dba2d347 5.4.0-137-generic #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 2023 x86_64 GNU/Linux |
| Build tool | hb_maven |
| Personality | dev-support/jenkins/hbase-personality.sh |
| git revision | master / 672f3de |
| Default Java | Oracle Corporation-1.8.0_282-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-110/1/testReport/ |
| Max. process+thread count | 944 (vs. ulimit of 12500) |
| modules | C: spark/hbase-spark spark/hbase-spark-it . U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-110/1/console |
| versions | git=2.20.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] NihalJain commented on a diff in pull request #110: HBASE-27639 Support hbase-connectors compilation with HBase 2.5.3, Ha…
Posted by "NihalJain (via GitHub)" <gi...@apache.org>.
NihalJain commented on code in PR #110:
URL: https://github.com/apache/hbase-connectors/pull/110#discussion_r1119222569
##########
spark/hbase-spark/src/test/java/org/apache/hadoop/hbase/spark/TestJavaHBaseContext.java:
##########
@@ -69,21 +70,29 @@ public class TestJavaHBaseContext implements Serializable {
public static final HBaseClassTestRule TIMEOUT =
HBaseClassTestRule.forClass(TestJavaHBaseContext.class);
- private static transient JavaSparkContext JSC;
- private static HBaseTestingUtility TEST_UTIL;
Review Comment:
Changes in field scope at L73, 74 and 75 are not needed as we are not using them in extended class. Will fix in next commit, as part of any other review comments, if received, on the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] Apache-HBase commented on pull request #110: HBASE-27639 Support hbase-connectors compilation with HBase 2.5.3, Ha…
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #110:
URL: https://github.com/apache/hbase-connectors/pull/110#issuecomment-1447705503
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 45s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +0 :ok: | spotbugs | 0m 0s | spotbugs executables are not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ master Compile Tests _ |
| +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 1m 15s | master passed |
| +1 :green_heart: | compile | 0m 51s | master passed |
| +1 :green_heart: | javadoc | 0m 49s | master passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 46s | the patch passed |
| +1 :green_heart: | compile | 0m 51s | the patch passed |
| +1 :green_heart: | javac | 0m 51s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. |
| +1 :green_heart: | javadoc | 0m 48s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 8m 25s | root in the patch passed. |
| | | 15m 50s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-110/2/artifact/yetus-precommit-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase-connectors/pull/110 |
| Optional Tests | dupname javac javadoc unit xml compile spotbugs findbugs |
| uname | Linux 961d9ca8ae86 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 GNU/Linux |
| Build tool | hb_maven |
| Personality | dev-support/jenkins/hbase-personality.sh |
| git revision | master / 672f3de |
| Default Java | Oracle Corporation-1.8.0_282-b08 |
| Test Results | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-110/2/testReport/ |
| Max. process+thread count | 944 (vs. ulimit of 12500) |
| modules | C: spark/hbase-spark spark/hbase-spark-it . U: . |
| Console output | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-110/2/console |
| versions | git=2.20.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] NihalJain commented on a diff in pull request #110: HBASE-27639 Support hbase-connectors compilation with HBase 2.5.3, Ha…
Posted by "NihalJain (via GitHub)" <gi...@apache.org>.
NihalJain commented on code in PR #110:
URL: https://github.com/apache/hbase-connectors/pull/110#discussion_r1119110569
##########
spark/hbase-spark/src/test/java/org/apache/hadoop/hbase/spark/TestJavaHBaseContext.java:
##########
@@ -517,6 +527,7 @@ private void populateTableWithMockData(Configuration conf, TableName tableName)
puts.add(put);
}
table.put(puts);
+ admin.flush(tableName);
Review Comment:
Flushing here as otherwise HBASE-26340 does not take effect
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] petersomogyi merged pull request #110: HBASE-27639 Support hbase-connectors compilation with HBase 2.5.3, Ha…
Posted by "petersomogyi (via GitHub)" <gi...@apache.org>.
petersomogyi merged PR #110:
URL: https://github.com/apache/hbase-connectors/pull/110
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] NihalJain commented on pull request #110: HBASE-27639 Support hbase-connectors compilation with HBase 2.5.3, Ha…
Posted by "NihalJain (via GitHub)" <gi...@apache.org>.
NihalJain commented on PR #110:
URL: https://github.com/apache/hbase-connectors/pull/110#issuecomment-1446802618
This patch fixes all compilation and test issues for hbase, spark and hadoop through specified versions
Ran following build combinations locally and ensured all SUCCESS:
- mvn clean install
- mvn -Dspark.version=3.1.2 -Dscala.binary.version=2.12 -Dscala.version=2.12.15 -Dhadoop-three.version=3.2.4 -Dhbase.version=2.5.3 clean install
- mvn -Dspark.version=3.2.3 -Dscala.binary.version=2.12 -Dscala.version=2.12.15 -Dhadoop-three.version=3.2.4 -Dhbase.version=2.5.3 clean install
- mvn -Dspark.version=3.2.3 -Dscala.binary.version=2.12 -Dscala.version=2.12.15 -Dhadoop-three.version=3.2.2 -Dhbase.version=2.5.3 clean install
- mvn -Dspark.version=3.2.0 -Dscala.binary.version=2.12 -Dscala.version=2.12.15 -Dhadoop-three.version=3.2.2 -Dhbase.version=2.4.9 clean install
- mvn clean install -Dhbase.version=2.4.16 -Dhadoop-three.version=3.2.4
- mvn clean install -Dhbase.version=2.4.16 -Dhadoop.profile=2.0
- mvn clean install -Dhbase.version=2.3.7 -Dhadoop-three.version=3.2.4
- mvn clean install -Dhbase.version=2.3.7 -Dhadoop.profile=2.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] stoty commented on pull request #110: HBASE-27639 Support hbase-connectors compilation with HBase 2.5.3, Ha…
Posted by "stoty (via GitHub)" <gi...@apache.org>.
stoty commented on PR #110:
URL: https://github.com/apache/hbase-connectors/pull/110#issuecomment-1449523669
+1 LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] NihalJain commented on a diff in pull request #110: HBASE-27639 Support hbase-connectors compilation with HBase 2.5.3, Ha…
Posted by "NihalJain (via GitHub)" <gi...@apache.org>.
NihalJain commented on code in PR #110:
URL: https://github.com/apache/hbase-connectors/pull/110#discussion_r1119222569
##########
spark/hbase-spark/src/test/java/org/apache/hadoop/hbase/spark/TestJavaHBaseContext.java:
##########
@@ -69,21 +70,29 @@ public class TestJavaHBaseContext implements Serializable {
public static final HBaseClassTestRule TIMEOUT =
HBaseClassTestRule.forClass(TestJavaHBaseContext.class);
- private static transient JavaSparkContext JSC;
- private static HBaseTestingUtility TEST_UTIL;
Review Comment:
Changes in fiele scope at L73, 74 and 75 are not needed as we are not using them in extended class. Will fix in next commit, as part of any other review comments, if received, on the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org