You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2021/08/13 03:06:44 UTC
[GitHub] [hbase] bitterfox opened a new pull request #3582: HBASE-26196 Support configuration override for remote cluster of HFileOutputFormat locality sensitive
bitterfox opened a new pull request #3582:
URL: https://github.com/apache/hbase/pull/3582
https://issues.apache.org/jira/browse/HBASE-26196
Usecase:
```
TableName tableName = TableName.valueOf("table");
Configuration conf = job.getConfiguration();
// assume cluster B is secure
Configuration clusterB = HBaseConfiguration.createClusterConf(
conf, "hbase-b-1.example.com,hbase-b-2.example.com,hbase-b-3.example.com:2181:/hbase-b");
clusterB.set(User.HBASE_SECURITY_CONF_KEY, "kerberos");
// configure others too like MASTER_KRB_PRINCIPAL, REGIONSERVER_KRB_PRINCIPAL and so on
// hack. TableMapReduceUtil.initCredentialsForCluster create UserProvider using job's configuration
// Thus if User.HBASE_SECURITY_CONF_KEY is simple, token isn't obtained
conf.set(User.HBASE_SECURITY_CONF_KEY, "kerberos");
TableMapReduceUtil.initCredentialsForCluster(job, clusterB);
// assume cluster A isn't secure
conf.set(HConstants.ZOOKEEPER_QUORUM, "hbase-a-1.example.com,hbase-a-2.example.com,hbase-a-3.example.com");
conf.setInt(HConstants.ZOOKEEPER_CLIENT_PORT, 2181);
conf.set(HConstants.ZOOKEEPER_ZNODE_PARENT, "/hbase-a");
conf.set(User.HBASE_SECURITY_CONF_KEY, "simple");
TableMapReduceUtil.initTableMapperJob(tableName, scanner, Mapper.class, ImmutableBytesWritable.class, Put.class, job);
try (Connection con = ConnectionFactory.createConnection(clusterB);
Table table = con.getTable(tableName);
RegionLocator regionLocator = con.getRegionLocator(tableName)) {
HFileOutputFormat2.configureIncrementalLoad(job, table, regionLocator);
conf.set(HFileOutputFormat2.REMOTE_CLUSTER_CONF_PREFIX + User.HBASE_SECURITY_CONF_KEY, "kerberos");
}
return job.waitForCompletion(true) ? 0 : 1;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3582: HBASE-26196 Support configuration override for remote cluster of HFileOutputFormat locality sensitive
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3582:
URL: https://github.com/apache/hbase/pull/3582#issuecomment-898172275
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 5s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 4s | master passed |
| +1 :green_heart: | compile | 0m 30s | master passed |
| +1 :green_heart: | shadedjars | 9m 5s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 22s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 47s | the patch passed |
| +1 :green_heart: | compile | 0m 28s | the patch passed |
| +1 :green_heart: | javac | 0m 28s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 11s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 20s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 13m 56s | hbase-mapreduce in the patch passed. |
| | | 45m 51s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3582 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux f0878c386829 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 11222fc4df |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/testReport/ |
| Max. process+thread count | 2901 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3582: HBASE-26196 Support configuration override for remote cluster of HFileOutputFormat locality sensitive
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3582:
URL: https://github.com/apache/hbase/pull/3582#issuecomment-898170415
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 26s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 2s | master passed |
| +1 :green_heart: | compile | 0m 27s | master passed |
| +1 :green_heart: | shadedjars | 8m 9s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 21s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 43s | the patch passed |
| +1 :green_heart: | compile | 0m 26s | the patch passed |
| +1 :green_heart: | javac | 0m 26s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 12s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 19s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 11m 7s | hbase-mapreduce in the patch passed. |
| | | 38m 22s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3582 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 51fec86558d9 4.15.0-151-generic #157-Ubuntu SMP Fri Jul 9 23:07:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 11222fc4df |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/testReport/ |
| Max. process+thread count | 4025 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] saintstack merged pull request #3582: HBASE-26196 Support configuration override for remote cluster of HFileOutputFormat locality sensitive
Posted by GitBox <gi...@apache.org>.
saintstack merged pull request #3582:
URL: https://github.com/apache/hbase/pull/3582
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3582: HBASE-26196 Support configuration override for remote cluster of HFileOutputFormat locality sensitive
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3582:
URL: https://github.com/apache/hbase/pull/3582#issuecomment-898171397
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 2s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 24s | master passed |
| +1 :green_heart: | compile | 0m 47s | master passed |
| +1 :green_heart: | checkstyle | 0m 20s | master passed |
| +1 :green_heart: | spotbugs | 0m 47s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 4s | the patch passed |
| +1 :green_heart: | compile | 0m 44s | the patch passed |
| -0 :warning: | javac | 0m 44s | hbase-mapreduce generated 1 new + 197 unchanged - 1 fixed = 198 total (was 198) |
| -0 :warning: | checkstyle | 0m 18s | hbase-mapreduce: The patch generated 1 new + 40 unchanged - 0 fixed = 41 total (was 40) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 20m 28s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 0m 55s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 42m 17s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3582 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c2a6b551c3f3 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 11222fc4df |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-mapreduce.txt |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-mapreduce.txt |
| Max. process+thread count | 86 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3582/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org