You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2022/11/19 04:18:03 UTC
[GitHub] [hbase-connectors] ILuffZhe opened a new pull request, #106: HBASE-27488 [hbase-connectors] Duplicate result when searching HBase by Spark
ILuffZhe opened a new pull request, #106:
URL: https://github.com/apache/hbase-connectors/pull/106
…by Spark
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] Reidddddd commented on a diff in pull request #106: HBASE-27488 [hbase-connectors] Duplicate result when searching HBase by Spark
Posted by "Reidddddd (via GitHub)" <gi...@apache.org>.
Reidddddd commented on code in PR #106:
URL: https://github.com/apache/hbase-connectors/pull/106#discussion_r1281559293
##########
spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:
##########
@@ -110,16 +110,20 @@ class HBaseTableScanRDD(relation: HBaseRelation,
hbaseContext: HBaseContext): Iterator[Result] = {
g.grouped(relation.bulkGetSize).flatMap{ x =>
val gets = new ArrayList[Get](x.size)
+ val pointsSet = new mutable.HashSet[String]()
Review Comment:
the parameter name sounds confused..
could it be `rowkeySet`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] Reidddddd commented on pull request #106: HBASE-27488 [hbase-connectors] Duplicate result when searching HBase by Spark
Posted by "Reidddddd (via GitHub)" <gi...@apache.org>.
Reidddddd commented on PR #106:
URL: https://github.com/apache/hbase-connectors/pull/106#issuecomment-1661775447
waiting for QA
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] ILuffZhe commented on a diff in pull request #106: HBASE-27488 [hbase-connectors] Duplicate result when searching HBase by Spark
Posted by "ILuffZhe (via GitHub)" <gi...@apache.org>.
ILuffZhe commented on code in PR #106:
URL: https://github.com/apache/hbase-connectors/pull/106#discussion_r1281581225
##########
spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:
##########
@@ -110,16 +110,20 @@ class HBaseTableScanRDD(relation: HBaseRelation,
hbaseContext: HBaseContext): Iterator[Result] = {
g.grouped(relation.bulkGetSize).flatMap{ x =>
val gets = new ArrayList[Get](x.size)
+ val pointsSet = new mutable.HashSet[String]()
Review Comment:
Make sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] ILuffZhe commented on pull request #106: HBASE-27488 [hbase-connectors] Duplicate result when searching HBase by Spark
Posted by "ILuffZhe (via GitHub)" <gi...@apache.org>.
ILuffZhe commented on PR #106:
URL: https://github.com/apache/hbase-connectors/pull/106#issuecomment-1661988061
@Reidddddd In this test case, we have duplicate rowkey filter, so the expression should be `KEY_FIELD == 0 OR KEY_FIELD == 1`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] Reidddddd merged pull request #106: HBASE-27488 [hbase-connectors] Duplicate result when searching HBase by Spark
Posted by "Reidddddd (via GitHub)" <gi...@apache.org>.
Reidddddd merged PR #106:
URL: https://github.com/apache/hbase-connectors/pull/106
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] Reidddddd commented on pull request #106: HBASE-27488 [hbase-connectors] Duplicate result when searching HBase by Spark
Posted by "Reidddddd (via GitHub)" <gi...@apache.org>.
Reidddddd commented on PR #106:
URL: https://github.com/apache/hbase-connectors/pull/106#issuecomment-1661857034
Reason | Tests
-- | --
Failed junit tests | hadoop.hbase.spark.DefaultSourceSuite
@ILuffZhe please help check if it is related
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] ILuffZhe commented on pull request #106: HBASE-27488 [hbase-connectors] Duplicate result when searching HBase by Spark
Posted by "ILuffZhe (via GitHub)" <gi...@apache.org>.
ILuffZhe commented on PR #106:
URL: https://github.com/apache/hbase-connectors/pull/106#issuecomment-1661862192
> Reason Tests
> Failed junit tests hadoop.hbase.spark.DefaultSourceSuite
> @ILuffZhe please help check if it is related
Sure, I'm already on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] Apache-HBase commented on pull request #106: HBASE-27488 [hbase-connectors] Duplicate result when searching HBase by Spark
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on PR #106:
URL: https://github.com/apache/hbase-connectors/pull/106#issuecomment-1320800563
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 22s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 26s | master passed |
| +1 :green_heart: | compile | 0m 40s | master passed |
| +1 :green_heart: | scaladoc | 0m 49s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | the patch passed |
| +1 :green_heart: | compile | 0m 39s | the patch passed |
| +1 :green_heart: | scalac | 0m 39s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | scaladoc | 0m 49s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 8m 6s | hbase-spark in the patch passed. |
| | | 16m 24s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-106/1/artifact/yetus-precommit-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase-connectors/pull/106 |
| Optional Tests | dupname scalac scaladoc unit compile |
| uname | Linux 362823b827c4 5.4.0-131-generic #147-Ubuntu SMP Fri Oct 14 17:07:22 UTC 2022 x86_64 GNU/Linux |
| Build tool | hb_maven |
| Personality | dev-support/jenkins/hbase-personality.sh |
| git revision | master / 496cd58 |
| Test Results | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-106/1/testReport/ |
| Max. process+thread count | 918 (vs. ulimit of 12500) |
| modules | C: spark/hbase-spark U: spark/hbase-spark |
| Console output | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-106/1/console |
| versions | git=2.20.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] Apache-HBase commented on pull request #106: HBASE-27488 [hbase-connectors] Duplicate result when searching HBase by Spark
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #106:
URL: https://github.com/apache/hbase-connectors/pull/106#issuecomment-1661837608
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 40s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 57s | master passed |
| +1 :green_heart: | compile | 0m 58s | master passed |
| +1 :green_heart: | scaladoc | 1m 17s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 22s | the patch passed |
| +1 :green_heart: | compile | 1m 18s | the patch passed |
| +1 :green_heart: | scalac | 1m 18s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | scaladoc | 1m 35s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 6m 8s | hbase-spark in the patch failed. |
| | | 16m 34s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hbase.spark.DefaultSourceSuite |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-106/3/artifact/yetus-precommit-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase-connectors/pull/106 |
| Optional Tests | dupname scalac scaladoc unit compile |
| uname | Linux 0ed15b02c145 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64 GNU/Linux |
| Build tool | hb_maven |
| Personality | dev-support/jenkins/hbase-personality.sh |
| git revision | master / 5d56e16 |
| unit | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-106/3/artifact/yetus-precommit-check/output/patch-unit-spark_hbase-spark.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-106/3/testReport/ |
| Max. process+thread count | 955 (vs. ulimit of 12500) |
| modules | C: spark/hbase-spark U: spark/hbase-spark |
| Console output | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-106/3/console |
| versions | git=2.20.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] Apache-HBase commented on pull request #106: HBASE-27488 [hbase-connectors] Duplicate result when searching HBase by Spark
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #106:
URL: https://github.com/apache/hbase-connectors/pull/106#issuecomment-1662031215
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 43s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 58s | master passed |
| +1 :green_heart: | compile | 1m 9s | master passed |
| +1 :green_heart: | scaladoc | 1m 20s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 21s | the patch passed |
| +1 :green_heart: | compile | 1m 4s | the patch passed |
| +1 :green_heart: | scalac | 1m 4s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | scaladoc | 1m 22s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 14s | hbase-spark in the patch passed. |
| | | 20m 18s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-106/4/artifact/yetus-precommit-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase-connectors/pull/106 |
| Optional Tests | dupname scalac scaladoc unit compile |
| uname | Linux 0eae2080e5be 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64 GNU/Linux |
| Build tool | hb_maven |
| Personality | dev-support/jenkins/hbase-personality.sh |
| git revision | master / 5d56e16 |
| Test Results | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-106/4/testReport/ |
| Max. process+thread count | 959 (vs. ulimit of 12500) |
| modules | C: spark/hbase-spark U: spark/hbase-spark |
| Console output | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-106/4/console |
| versions | git=2.20.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] Apache-HBase commented on pull request #106: HBASE-27488 [hbase-connectors] Duplicate result when searching HBase by Spark
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #106:
URL: https://github.com/apache/hbase-connectors/pull/106#issuecomment-1661811441
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 55s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 52s | master passed |
| +1 :green_heart: | compile | 0m 59s | master passed |
| +1 :green_heart: | scaladoc | 1m 13s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 12s | the patch passed |
| +1 :green_heart: | compile | 1m 3s | the patch passed |
| +1 :green_heart: | scalac | 1m 3s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | scaladoc | 1m 29s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 6m 31s | hbase-spark in the patch failed. |
| | | 17m 20s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hbase.spark.DefaultSourceSuite |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-106/2/artifact/yetus-precommit-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase-connectors/pull/106 |
| Optional Tests | dupname scalac scaladoc unit compile |
| uname | Linux 5a8ddaf30629 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64 GNU/Linux |
| Build tool | hb_maven |
| Personality | dev-support/jenkins/hbase-personality.sh |
| git revision | master / 5d56e16 |
| unit | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-106/2/artifact/yetus-precommit-check/output/patch-unit-spark_hbase-spark.txt |
| Test Results | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-106/2/testReport/ |
| Max. process+thread count | 953 (vs. ulimit of 12500) |
| modules | C: spark/hbase-spark U: spark/hbase-spark |
| Console output | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-106/2/console |
| versions | git=2.20.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org