You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "ss77892 (via GitHub)" <gi...@apache.org> on 2023/02/09 18:52:00 UTC
[GitHub] [hbase-connectors] ss77892 opened a new pull request, #108: HBASE-27630: hbase-spark bulkload stage directory limited to hdfs only
ss77892 opened a new pull request, #108:
URL: https://github.com/apache/hbase-connectors/pull/108
The reason for the problem was the usage of the default FileSystem instances for creating and using staging stuff. Now we use the staging path to obtain the FileSystem instance, so it should work correctly for any different from the hdfs schema.
The test uses a local filesystem and doesn't load files to the table due to limitations:
1. HBase minicluster uses hdfs only, and there is no way to specify a custom hbase.rootdir for that. So the load would not work because the staging dir is on a different filesystem
2. I tried to get hadoop-aws mock implementation working for an external filesystem, but that would require making the tests for this module published first
Both those two bullets can be addressed in separate jiras if required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] petersomogyi merged pull request #108: HBASE-27630: hbase-spark bulkload stage directory limited to hdfs only
Posted by "petersomogyi (via GitHub)" <gi...@apache.org>.
petersomogyi merged PR #108:
URL: https://github.com/apache/hbase-connectors/pull/108
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] Apache-HBase commented on pull request #108: HBASE-27630: hbase-spark bulkload stage directory limited to hdfs only
Posted by "Apache-HBase (via GitHub)" <gi...@apache.org>.
Apache-HBase commented on PR #108:
URL: https://github.com/apache/hbase-connectors/pull/108#issuecomment-1424695830
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 42s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 36s | master passed |
| +1 :green_heart: | compile | 0m 38s | master passed |
| +1 :green_heart: | scaladoc | 0m 48s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 48s | the patch passed |
| +1 :green_heart: | compile | 0m 39s | the patch passed |
| +1 :green_heart: | scalac | 0m 39s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | scaladoc | 0m 47s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 7m 35s | hbase-spark in the patch passed. |
| | | 14m 16s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-108/1/artifact/yetus-precommit-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase-connectors/pull/108 |
| Optional Tests | dupname scalac scaladoc unit compile |
| uname | Linux 1c4312f80fa8 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 GNU/Linux |
| Build tool | hb_maven |
| Personality | dev-support/jenkins/hbase-personality.sh |
| git revision | master / c221896 |
| Test Results | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-108/1/testReport/ |
| Max. process+thread count | 945 (vs. ulimit of 12500) |
| modules | C: spark/hbase-spark U: spark/hbase-spark |
| Console output | https://ci-hbase.apache.org/job/HBase-Connectors-PreCommit/job/PR-108/1/console |
| versions | git=2.20.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase-connectors] stoty commented on pull request #108: HBASE-27630: hbase-spark bulkload stage directory limited to hdfs only
Posted by "stoty (via GitHub)" <gi...@apache.org>.
stoty commented on PR #108:
URL: https://github.com/apache/hbase-connectors/pull/108#issuecomment-1429254712
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org