You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2021/08/17 08:52:18 UTC
[GitHub] [hbase] bitterfox opened a new pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
bitterfox opened a new pull request #3591:
URL: https://github.com/apache/hbase/pull/3591
https://issues.apache.org/jira/browse/HBASE-26204
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] bitterfox commented on a change in pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
bitterfox commented on a change in pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#discussion_r691988146
##########
File path: hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationSecureClusterCredentials.java
##########
@@ -0,0 +1,166 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.replication;
+
+import static org.junit.Assert.assertEquals;
+import java.io.File;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.function.Supplier;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtil;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.coprocessor.CoprocessorHost;
+import org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication;
+import org.apache.hadoop.hbase.security.HBaseKerberosUtils;
+import org.apache.hadoop.hbase.security.access.AccessController;
+import org.apache.hadoop.hbase.security.access.SecureTestUtil;
+import org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier;
+import org.apache.hadoop.hbase.security.token.TokenProvider;
+import org.apache.hadoop.hbase.security.visibility.VisibilityTestUtil;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.ReplicationTests;
+import org.apache.hadoop.hbase.zookeeper.ZKClusterId;
+import org.apache.hadoop.hbase.zookeeper.ZKConfig;
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.mapreduce.Job;
+import org.apache.hadoop.minikdc.MiniKdc;
+import org.apache.hadoop.security.Credentials;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.hadoop.security.token.Token;
+import org.apache.hadoop.security.token.TokenIdentifier;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameter;
+import org.junit.runners.Parameterized.Parameters;
+
+@Category({ ReplicationTests.class, LargeTests.class })
+@RunWith(Parameterized.class)
+public class TestVerifyReplicationSecureClusterCredentials {
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+ HBaseClassTestRule.forClass(TestVerifyReplicationSecureClusterCredentials.class);
+
+ private static MiniKdc KDC;
+ private static final HBaseTestingUtil UTIL1 = new HBaseTestingUtil();
Review comment:
Well, both are a secure cluster as `setupCluster` method apply secure configuration before start cluster.
I'd like to add test case mixing secure cluster and non-secure cluster, but it seems like it's difficult due to some static fields:
https://github.com/apache/hbase/pull/3591#issuecomment-900802340
https://github.com/apache/hbase/pull/3591#pullrequestreview-733731305
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] bitterfox commented on a change in pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
bitterfox commented on a change in pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#discussion_r691980547
##########
File path: hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
##########
@@ -755,7 +793,8 @@ public int run(String[] args) throws Exception {
* @throws Exception When running the job fails.
*/
public static void main(String[] args) throws Exception {
- int res = ToolRunner.run(HBaseConfiguration.create(), new VerifyReplication(), args);
- System.exit(res);
+// int res = ToolRunner.run(HBaseConfiguration.create(), new VerifyReplication(), args);
Review comment:
Oops, embarrassed. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] bitterfox edited a comment on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
bitterfox edited a comment on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-900194011
Thank you for your comments.
> Can we add explanation on how to use this in the printUsage method?
I see. I'll check the current usage first.
> What if the remote peer isn't kerberised? I guess initCredentialsForCluster call would throw an exception?
I think there are several cases as you mentioned, but in general, initCredentialsForCluster shouldn't throw exceptions and do nothing for non-kerberos cluster configuration(connection).
https://github.com/apache/hbase/blob/721cb96f8c543ed4f8cb2ba3a1196f98fa8bd26a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java#L589
I know initCredentialsForCluster is odd as another issue.
https://issues.apache.org/jira/browse/HBASE-26205
https://github.com/apache/hbase/pull/3592
Current initCredentialsForCluster won't work for the following case and I'd like to fix it in HBASE-26205 (#3592) as indivisual change (because initCredentialsForCluster is common feature)
- cluster A is non-secure(hbase.security.authentication=simple in job conf), cluster B is secure(hbase.security.authentication=kerberos in cluster conf)
- cluster A is secure(hbase.security.authentication=kerberos in job conf), cluster B is non-secure(hbase.security.authentication=simple in cluster conf)
> Since we are making VerifyReplication a general comparison tool (not replication specific anymore), shouldn't we cover all possible scenarios (unsecure/secure, secure/secure, secure/unsecure, unsecure/unsecure)?
Yes, it could handle either combination with configuration for the VerifyReplication execution.
For example, these is some examples of the execution for the combinations.
(Please make sure, #3592 is necessary for mixed env)
```
# non-secure vs secure (as well as for opposite)
hbase VerifyReplication \
-D hbase.security.authentication=simple \
-D hbase.zookeeper.quorum=non-secure-hbase-a-1.example.com,non-secure-hbase-a-2.example.com,non-secure-hbase-a-3.example.com \
-D hbase.zookeeper.property.clientPort=2181 \
-D zookeeper.znode.parent=/hbase-a
-D verifyrep.peer.hbase.security.authentication=kerberos \
-D verifyrep.peer.hbase.regionserver.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
-D verifyrep.peer.hbase.master.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
... \
secure-hbase-b-1.example.com,secure-hbase-b-2.example.com,secure-hbase-b-3.example.com:2181:/hbase-b \
table
# secure vs secure (use different principal for cluster daemons)
hbase VerifyReplication \
-D hbase.security.authentication=kerberos \
-D hbase.zookeeper.quorum=secure-hbase-a-1.example.com,secure-hbase-a-2.example.com,secure-hbase-a-3.example.com \
-D hbase.zookeeper.property.clientPort=2181 \
-D zookeeper.znode.parent=/hbase-a
-D hbase.regionserver.kerberos.principal=secure-hbase-a/_HOST@EXAMPLE.COM \
-D hbase.master.kerberos.principal=secure-hbase-a/_HOST@EXAMPLE.COM \
-D verifyrep.peer.hbase.regionserver.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
-D verifyrep.peer.hbase.master.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
... \
secure-hbase-b-1.example.com,secure-hbase-b-2.example.com,secure-hbase-b-3.example.com:2181:/hbase-b \
table
```
> Any chance to add UTs?
I'll try it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] shahrs87 commented on a change in pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
shahrs87 commented on a change in pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#discussion_r691241668
##########
File path: hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
##########
@@ -736,6 +740,40 @@ private static void printUsage(final String errorMsg) {
System.err.println(" $ hbase " +
"org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication" +
" --starttime=1265875194289 --endtime=1265878794289 5 TestTable ");
+ System.err.println();
+ System.err.println(" To verify the data in TestTable between the cluster runs VerifyReplication and cluster-b");
+ System.err.println(" Assume quorum address for cluster-b is cluster-b-1.example.com,cluster-b-2.example.com,cluster-b-3.example.com:2181:/cluster-b");
+ System.err.println(
+ " $ hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication \\\n" +
+ " cluster-b-1.example.com,cluster-b-2.example.com,cluster-b-3.example.com:2181:/cluster-b \\\n" +
+ " TestTable");
+ System.err.println();
+ System.err.println(" To verify the data in TestTable between the secured cluster runs VerifyReplication and insecure cluster-b");
+ System.err.println(
+ " $ hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication \\\n" +
+ " -D verifyrep.peer.hbase.security.authentication=simple \\\n" +
Review comment:
Do we expect a space between D and verifyrep.peer.hbase.security.authentication ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-901857835
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 26s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 16s | master passed |
| +1 :green_heart: | compile | 0m 29s | master passed |
| +1 :green_heart: | shadedjars | 8m 4s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 21s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 19s | the patch passed |
| +1 :green_heart: | compile | 0m 29s | the patch passed |
| +1 :green_heart: | javac | 0m 29s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 9s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 22s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 44s | hbase-mapreduce in the patch passed. |
| | | 38m 42s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 529bae00b77e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / c8d9d4df80 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/6/testReport/ |
| Max. process+thread count | 3736 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/6/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-900146546
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 48s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 18s | master passed |
| +1 :green_heart: | compile | 0m 48s | master passed |
| +1 :green_heart: | checkstyle | 0m 20s | master passed |
| +1 :green_heart: | spotbugs | 0m 47s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 5s | the patch passed |
| +1 :green_heart: | compile | 0m 47s | the patch passed |
| +1 :green_heart: | javac | 0m 47s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 17s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 20m 28s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 0m 55s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. |
| | | 44m 12s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 55405b473c48 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 721cb96f8c |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Max. process+thread count | 86 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-901053285
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 26s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 32s | master passed |
| +1 :green_heart: | compile | 0m 28s | master passed |
| +1 :green_heart: | shadedjars | 8m 12s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 24s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 16s | the patch passed |
| +1 :green_heart: | compile | 0m 29s | the patch passed |
| +1 :green_heart: | javac | 0m 29s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 12s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 21s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 57s | hbase-mapreduce in the patch passed. |
| | | 38m 37s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 681aaaf7b99d 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 33e15a261d |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/3/testReport/ |
| Max. process+thread count | 4660 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/3/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-900812334
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 27s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 15s | master passed |
| +1 :green_heart: | compile | 0m 30s | master passed |
| +1 :green_heart: | shadedjars | 8m 12s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 21s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 19s | the patch passed |
| +1 :green_heart: | compile | 0m 31s | the patch passed |
| +1 :green_heart: | javac | 0m 31s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 11s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 20s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 46s | hbase-mapreduce in the patch passed. |
| | | 39m 11s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 92870a2e54d0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 721cb96f8c |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/2/testReport/ |
| Max. process+thread count | 4005 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-901827365
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 29s | master passed |
| +1 :green_heart: | compile | 0m 33s | master passed |
| +1 :green_heart: | shadedjars | 9m 9s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 24s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 28s | the patch passed |
| +1 :green_heart: | compile | 0m 29s | the patch passed |
| +1 :green_heart: | javac | 0m 29s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 26s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 22s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 50s | hbase-mapreduce in the patch passed. |
| | | 42m 9s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 059391e16cff 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / c8d9d4df80 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/5/testReport/ |
| Max. process+thread count | 3906 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/5/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] shahrs87 commented on a change in pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
shahrs87 commented on a change in pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#discussion_r691242020
##########
File path: hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
##########
@@ -755,7 +793,8 @@ public int run(String[] args) throws Exception {
* @throws Exception When running the job fails.
*/
public static void main(String[] args) throws Exception {
- int res = ToolRunner.run(HBaseConfiguration.create(), new VerifyReplication(), args);
- System.exit(res);
+// int res = ToolRunner.run(HBaseConfiguration.create(), new VerifyReplication(), args);
Review comment:
Forgot to uncomment ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] shahrs87 commented on a change in pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
shahrs87 commented on a change in pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#discussion_r691241668
##########
File path: hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
##########
@@ -736,6 +740,40 @@ private static void printUsage(final String errorMsg) {
System.err.println(" $ hbase " +
"org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication" +
" --starttime=1265875194289 --endtime=1265878794289 5 TestTable ");
+ System.err.println();
+ System.err.println(" To verify the data in TestTable between the cluster runs VerifyReplication and cluster-b");
+ System.err.println(" Assume quorum address for cluster-b is cluster-b-1.example.com,cluster-b-2.example.com,cluster-b-3.example.com:2181:/cluster-b");
+ System.err.println(
+ " $ hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication \\\n" +
+ " cluster-b-1.example.com,cluster-b-2.example.com,cluster-b-3.example.com:2181:/cluster-b \\\n" +
+ " TestTable");
+ System.err.println();
+ System.err.println(" To verify the data in TestTable between the secured cluster runs VerifyReplication and insecure cluster-b");
+ System.err.println(
+ " $ hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication \\\n" +
+ " -D verifyrep.peer.hbase.security.authentication=simple \\\n" +
Review comment:
Do we expect a space between D and verifyrep.peer.hbase.security.authentication ?
##########
File path: hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
##########
@@ -755,7 +793,8 @@ public int run(String[] args) throws Exception {
* @throws Exception When running the job fails.
*/
public static void main(String[] args) throws Exception {
- int res = ToolRunner.run(HBaseConfiguration.create(), new VerifyReplication(), args);
- System.exit(res);
+// int res = ToolRunner.run(HBaseConfiguration.create(), new VerifyReplication(), args);
Review comment:
Forgot to uncomment ?
##########
File path: hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationSecureClusterCredentials.java
##########
@@ -0,0 +1,166 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.replication;
+
+import static org.junit.Assert.assertEquals;
+import java.io.File;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.function.Supplier;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtil;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.coprocessor.CoprocessorHost;
+import org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication;
+import org.apache.hadoop.hbase.security.HBaseKerberosUtils;
+import org.apache.hadoop.hbase.security.access.AccessController;
+import org.apache.hadoop.hbase.security.access.SecureTestUtil;
+import org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier;
+import org.apache.hadoop.hbase.security.token.TokenProvider;
+import org.apache.hadoop.hbase.security.visibility.VisibilityTestUtil;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.ReplicationTests;
+import org.apache.hadoop.hbase.zookeeper.ZKClusterId;
+import org.apache.hadoop.hbase.zookeeper.ZKConfig;
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.mapreduce.Job;
+import org.apache.hadoop.minikdc.MiniKdc;
+import org.apache.hadoop.security.Credentials;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.hadoop.security.token.Token;
+import org.apache.hadoop.security.token.TokenIdentifier;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameter;
+import org.junit.runners.Parameterized.Parameters;
+
+@Category({ ReplicationTests.class, LargeTests.class })
+@RunWith(Parameterized.class)
+public class TestVerifyReplicationSecureClusterCredentials {
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+ HBaseClassTestRule.forClass(TestVerifyReplicationSecureClusterCredentials.class);
+
+ private static MiniKdc KDC;
+ private static final HBaseTestingUtil UTIL1 = new HBaseTestingUtil();
Review comment:
nit: add a comment that UTIL1 is secure cluster and UTIL2 is non-secure cluster.
##########
File path: hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
##########
@@ -736,6 +740,40 @@ private static void printUsage(final String errorMsg) {
System.err.println(" $ hbase " +
"org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication" +
" --starttime=1265875194289 --endtime=1265878794289 5 TestTable ");
+ System.err.println();
+ System.err.println(" To verify the data in TestTable between the cluster runs VerifyReplication and cluster-b");
Review comment:
nice work to add examples on how to run VerifyReplication between non secure and secure environment. It is very helpful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-901109624
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] bitterfox edited a comment on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
bitterfox edited a comment on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-900194011
Thank you for your comments.
> Can we add explanation on how to use this in the printUsage method?
I see. I'll check the current usage first.
> What if the remote peer isn't kerberised? I guess initCredentialsForCluster call would throw an exception?
I think there are several cases as you mentioned, but in general, initCredentialsForCluster shouldn't throw exceptions and do nothing for non-kerberos cluster configuration(connection).
https://github.com/apache/hbase/blob/721cb96f8c543ed4f8cb2ba3a1196f98fa8bd26a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java#L589
I know initCredentialsForCluster might throw some exception if job conf has hbase.security.authentication=simple/kerberos and cluster conf has hbase.security.authentication=kerberos/simple as another issue.
https://issues.apache.org/jira/browse/HBASE-26205
https://github.com/apache/hbase/pull/3592
Current initCredentialsForCluster won't work for the following case and I'd like to fix it in HBASE-26205 (#3592) as indivisual change (because initCredentialsForCluster is common feature)
- cluster A is non-secure(hbase.security.authentication=simple in job conf), cluster B is secure(hbase.security.authentication=kerberos in cluster conf)
- cluster A is secure(hbase.security.authentication=kerberos in job conf), cluster B is non-secure(hbase.security.authentication=simple in cluster conf)
> Since we are making VerifyReplication a general comparison tool (not replication specific anymore), shouldn't we cover all possible scenarios (unsecure/secure, secure/secure, secure/unsecure, unsecure/unsecure)?
Yes, it could handle either combination with configuration for the VerifyReplication execution.
For example, these is some examples of the execution for the combinations.
(Please make sure, #3592 is necessary for mixed env)
```
# non-secure vs secure (as well as for opposite)
hbase VerifyReplication \
-D hbase.security.authentication=simple \
-D hbase.zookeeper.quorum=non-secure-hbase-a-1.example.com,non-secure-hbase-a-2.example.com,non-secure-hbase-a-3.example.com \
-D hbase.zookeeper.property.clientPort=2181 \
-D zookeeper.znode.parent=/hbase-a
-D verifyrep.peer.hbase.security.authentication=kerberos \
-D verifyrep.peer.hbase.regionserver.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
-D verifyrep.peer.hbase.master.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
... \
secure-hbase-b-1.example.com,secure-hbase-b-2.example.com,secure-hbase-b-3.example.com:2181:/hbase-b \
table
# secure vs secure (use different principal for cluster daemons)
hbase VerifyReplication \
-D hbase.security.authentication=kerberos \
-D hbase.zookeeper.quorum=secure-hbase-a-1.example.com,secure-hbase-a-2.example.com,secure-hbase-a-3.example.com \
-D hbase.zookeeper.property.clientPort=2181 \
-D zookeeper.znode.parent=/hbase-a
-D hbase.regionserver.kerberos.principal=secure-hbase-a/_HOST@EXAMPLE.COM \
-D hbase.master.kerberos.principal=secure-hbase-a/_HOST@EXAMPLE.COM \
-D verifyrep.peer.hbase.regionserver.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
-D verifyrep.peer.hbase.master.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
... \
secure-hbase-b-1.example.com,secure-hbase-b-2.example.com,secure-hbase-b-3.example.com:2181:/hbase-b \
table
```
> Any chance to add UTs?
I'll try it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-901055519
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 7s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 13s | master passed |
| +1 :green_heart: | compile | 0m 46s | master passed |
| +1 :green_heart: | checkstyle | 0m 18s | master passed |
| +1 :green_heart: | spotbugs | 0m 45s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 3s | the patch passed |
| +1 :green_heart: | compile | 0m 46s | the patch passed |
| +1 :green_heart: | javac | 0m 46s | the patch passed |
| -0 :warning: | checkstyle | 0m 18s | hbase-mapreduce: The patch generated 14 new + 8 unchanged - 0 fixed = 22 total (was 8) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | hadoopcheck | 20m 22s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 0m 55s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. |
| | | 42m 14s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | dupname asflicense javac hadoopcheck xml compile spotbugs hbaseanti checkstyle |
| uname | Linux f1c962637406 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 33e15a261d |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-mapreduce.txt |
| Max. process+thread count | 85 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/3/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] bitterfox commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
bitterfox commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-900802340
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a change in pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on a change in pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#discussion_r691917237
##########
File path: hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
##########
@@ -736,6 +740,40 @@ private static void printUsage(final String errorMsg) {
System.err.println(" $ hbase " +
"org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication" +
" --starttime=1265875194289 --endtime=1265878794289 5 TestTable ");
+ System.err.println();
+ System.err.println(" To verify the data in TestTable between the cluster runs VerifyReplication and cluster-b");
Review comment:
Thanks for putting up these examples!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] bitterfox edited a comment on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
bitterfox edited a comment on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-900194011
Thank you for your comments.
> Can we add explanation on how to use this in the printUsage method?
I see. I'll check the current usage first.
> What if the remote peer isn't kerberised? I guess initCredentialsForCluster call would throw an exception?
I think there are several cases as you mentioned, but in general, initCredentialsForCluster shouldn't throw exceptions and do nothing for non-kerberos cluster configuration(connection).
https://github.com/apache/hbase/blob/721cb96f8c543ed4f8cb2ba3a1196f98fa8bd26a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java#L589
I know initCredentialsForCluster might throw some exception if job conf has hbase.security.authentication=simple/kerberos and cluster conf has hbase.security.authentication=kerberos/simple as another issue.
https://issues.apache.org/jira/browse/HBASE-26205
https://github.com/apache/hbase/pull/3592
Current initCredentialsForCluster won't work for the following case and I'd like to fix it in HBASE-26205 (#3592) as indivisual change (because initCredentialsForCluster is common feature)
- cluster A is non-secure(hbase.security.authentication=simple in job conf), cluster B is secure(hbase.security.authentication=kerberos in cluster conf)
- cluster A is secure(hbase.security.authentication=kerberos in job conf), cluster B is non-secure(hbase.security.authentication=simple in cluster conf)
> Since we are making VerifyReplication a general comparison tool (not replication specific anymore), shouldn't we cover all possible scenarios (unsecure/secure, secure/secure, secure/unsecure, unsecure/unsecure)?
Yes, it could handle any combination with configuration for the VerifyReplication execution.
For example, these is some examples of the execution for the combinations.
(Please make sure, #3592 is necessary for mixed env)
```
# non-secure vs secure (as well as for opposite)
hbase VerifyReplication \
-D hbase.security.authentication=simple \
-D hbase.zookeeper.quorum=non-secure-hbase-a-1.example.com,non-secure-hbase-a-2.example.com,non-secure-hbase-a-3.example.com \
-D hbase.zookeeper.property.clientPort=2181 \
-D zookeeper.znode.parent=/hbase-a
-D verifyrep.peer.hbase.security.authentication=kerberos \
-D verifyrep.peer.hbase.regionserver.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
-D verifyrep.peer.hbase.master.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
... \
secure-hbase-b-1.example.com,secure-hbase-b-2.example.com,secure-hbase-b-3.example.com:2181:/hbase-b \
table
# secure vs secure (use different principal for cluster daemons)
hbase VerifyReplication \
-D hbase.security.authentication=kerberos \
-D hbase.zookeeper.quorum=secure-hbase-a-1.example.com,secure-hbase-a-2.example.com,secure-hbase-a-3.example.com \
-D hbase.zookeeper.property.clientPort=2181 \
-D zookeeper.znode.parent=/hbase-a
-D hbase.regionserver.kerberos.principal=secure-hbase-a/_HOST@EXAMPLE.COM \
-D hbase.master.kerberos.principal=secure-hbase-a/_HOST@EXAMPLE.COM \
-D verifyrep.peer.hbase.regionserver.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
-D verifyrep.peer.hbase.master.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
... \
secure-hbase-b-1.example.com,secure-hbase-b-2.example.com,secure-hbase-b-3.example.com:2181:/hbase-b \
table
```
> Any chance to add UTs?
I'll try it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] bitterfox commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
bitterfox commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-900802340
>>Any chance to add UTs?
>I'll try it.
Added test case for both is secure.
Unfortunately, I couldn't add a test case for non-secure and secure because it's difficult to start non-secure and secure cluster at the same time in a single process.
User (UserGroupInformation) uses static heavily and it can be either SIMPLE auth or Kerberos auth, so either cluster confuse and won't work properly.
If you know how to boot both of non-secure and secure cluster at the same time, please tell me so that I could add test case for it...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-901859757
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 38s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 17s | master passed |
| +1 :green_heart: | compile | 0m 29s | master passed |
| +1 :green_heart: | shadedjars | 9m 30s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 19s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 19s | the patch passed |
| +1 :green_heart: | compile | 0m 30s | the patch passed |
| +1 :green_heart: | javac | 0m 30s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 29s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 19s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 51s | hbase-mapreduce in the patch passed. |
| | | 42m 4s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 5971a084e02f 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / c8d9d4df80 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/6/testReport/ |
| Max. process+thread count | 3866 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/6/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-900143856
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 17s | master passed |
| +1 :green_heart: | compile | 0m 27s | master passed |
| +1 :green_heart: | shadedjars | 8m 47s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 21s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 52s | the patch passed |
| +1 :green_heart: | compile | 0m 27s | the patch passed |
| +1 :green_heart: | javac | 0m 27s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 48s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 18s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 50s | hbase-mapreduce in the patch passed. |
| | | 39m 56s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux d5197bb01cc9 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 721cb96f8c |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/1/testReport/ |
| Max. process+thread count | 4302 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-900814706
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 20s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 32s | master passed |
| +1 :green_heart: | compile | 0m 25s | master passed |
| +1 :green_heart: | shadedjars | 9m 13s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 22s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 8s | the patch passed |
| +1 :green_heart: | compile | 0m 30s | the patch passed |
| +1 :green_heart: | javac | 0m 30s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 9s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 19s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 14m 25s | hbase-mapreduce in the patch passed. |
| | | 45m 40s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 63411d865f68 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 721cb96f8c |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/2/testReport/ |
| Max. process+thread count | 3030 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] bitterfox commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
bitterfox commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-901032200
Added examples
```
Examples:
To verify the data replicated from TestTable for a 1 hour window with peer #5
$ hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication --starttime=1265875194289 --endtime=1265878794289 5 TestTable
To verify the data in TestTable between the cluster runs VerifyReplication and cluster-b
Assume quorum address for cluster-b is cluster-b-1.example.com,cluster-b-2.example.com,cluster-b-3.example.com:2181:/cluster-b
$ hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication \
cluster-b-1.example.com,cluster-b-2.example.com,cluster-b-3.example.com:2181:/cluster-b \
TestTable
To verify the data in TestTable between the secured cluster runs VerifyReplication and insecure cluster-b
$ hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication \
-D verifyrep.peer.hbase.security.authentication=simple \
cluster-b-1.example.com,cluster-b-2.example.com,cluster-b-3.example.com:2181:/cluster-b \
TestTable
To verify the data in TestTable between the secured cluster runs VerifyReplication and secured cluster-b
Assume cluster-b uses different kerberos principal, cluster-b/_HOST@E, for master and regionserver kerberos principal from another cluster
$ hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication \
-D verifyrep.peer.hbase.regionserver.kerberos.principal=cluster-b/_HOST@EXAMPLE.COM \
-D verifyrep.peer.hbase.master.kerberos.principal=cluster-b/_HOST@EXAMPLE.COM \
cluster-b-1.example.com,cluster-b-2.example.com,cluster-b-3.example.com:2181:/cluster-b \
TestTable
To verify the data in TestTable between the insecure cluster runs VerifyReplication and secured cluster-b
$ hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication \
-D verifyrep.peer.hbase.security.authentication=kerberos \
-D verifyrep.peer.hbase.regionserver.kerberos.principal=cluster-b/_HOST@EXAMPLE.COM \
-D verifyrep.peer.hbase.master.kerberos.principal=cluster-b/_HOST@EXAMPLE.COM \
cluster-b-1.example.com,cluster-b-2.example.com,cluster-b-3.example.com:2181:/cluster-b \
TestTable
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-901108581
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 53s | master passed |
| +1 :green_heart: | compile | 0m 25s | master passed |
| +1 :green_heart: | shadedjars | 8m 10s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 19s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 50s | the patch passed |
| +1 :green_heart: | compile | 0m 26s | the patch passed |
| +1 :green_heart: | javac | 0m 26s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 7s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 18s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 12m 50s | hbase-mapreduce in the patch passed. |
| | | 40m 6s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 01478cde48e3 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 33e15a261d |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/4/testReport/ |
| Max. process+thread count | 3598 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/4/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-901833543
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 44s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 8s | master passed |
| +1 :green_heart: | compile | 1m 1s | master passed |
| +1 :green_heart: | checkstyle | 0m 23s | master passed |
| +1 :green_heart: | spotbugs | 1m 0s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 1s | the patch passed |
| +1 :green_heart: | compile | 0m 51s | the patch passed |
| +1 :green_heart: | javac | 0m 51s | the patch passed |
| -0 :warning: | checkstyle | 0m 22s | hbase-mapreduce: The patch generated 10 new + 8 unchanged - 0 fixed = 18 total (was 8) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | hadoopcheck | 24m 39s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 1m 9s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 51m 49s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/5/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | dupname asflicense javac hadoopcheck xml compile spotbugs hbaseanti checkstyle |
| uname | Linux 5f5f5217e9aa 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / c8d9d4df80 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-mapreduce.txt |
| Max. process+thread count | 86 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/5/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-901824866
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 29s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 58s | master passed |
| +1 :green_heart: | compile | 0m 26s | master passed |
| +1 :green_heart: | shadedjars | 8m 15s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 21s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 43s | the patch passed |
| +1 :green_heart: | compile | 0m 28s | the patch passed |
| +1 :green_heart: | javac | 0m 28s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 19s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 19s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 44s | hbase-mapreduce in the patch passed. |
| | | 38m 21s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux a730e57f3fce 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / c8d9d4df80 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/5/testReport/ |
| Max. process+thread count | 4394 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/5/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-901859535
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 11s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 3s | master passed |
| +1 :green_heart: | compile | 0m 46s | master passed |
| +1 :green_heart: | checkstyle | 0m 17s | master passed |
| +1 :green_heart: | spotbugs | 0m 42s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 3s | the patch passed |
| +1 :green_heart: | compile | 0m 45s | the patch passed |
| +1 :green_heart: | javac | 0m 45s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 16s | hbase-mapreduce: The patch generated 0 new + 3 unchanged - 5 fixed = 3 total (was 8) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | hadoopcheck | 20m 14s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 0m 55s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. |
| | | 41m 51s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/6/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | dupname asflicense javac hadoopcheck xml compile spotbugs hbaseanti checkstyle |
| uname | Linux 39f87e629686 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / c8d9d4df80 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Max. process+thread count | 86 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/6/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-900813537
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 9s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 28s | master passed |
| +1 :green_heart: | compile | 0m 49s | master passed |
| +1 :green_heart: | checkstyle | 0m 19s | master passed |
| +1 :green_heart: | spotbugs | 0m 47s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 2s | the patch passed |
| +1 :green_heart: | compile | 0m 45s | the patch passed |
| +1 :green_heart: | javac | 0m 45s | the patch passed |
| -0 :warning: | checkstyle | 0m 17s | hbase-mapreduce: The patch generated 3 new + 8 unchanged - 0 fixed = 11 total (was 8) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. |
| +1 :green_heart: | hadoopcheck | 20m 16s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. |
| +1 :green_heart: | spotbugs | 0m 55s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. |
| | | 42m 37s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | dupname asflicense javac hadoopcheck xml compile spotbugs hbaseanti checkstyle |
| uname | Linux e7510041c45b 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 721cb96f8c |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-mapreduce.txt |
| Max. process+thread count | 85 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/2/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] bitterfox commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
bitterfox commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-900194011
Thank you for your comments.
> Can we add explanation on how to use this in the printUsage method?
I see. I'll check the current usage first.
> What if the remote peer isn't kerberised? I guess initCredentialsForCluster call would throw an exception?
I think there are several cases as you mentioned, but in general, initCredentialsForCluster shouldn't throw exceptions for non-kerberos cluster configuration(connection).
I know initCredentialsForCluster is odd as another issue.
https://issues.apache.org/jira/browse/HBASE-26205
https://github.com/apache/hbase/pull/3592
Current initCredentialsForCluster won't work for the following case and I'd like to fix it in HBASE-26205 (#3592) as indivisual change (because initCredentialsForCluster is common feature)
- cluster A is non-secure(hbase.security.authentication=simple in job conf), cluster B is secure(hbase.security.authentication=kerberos in cluster conf)
- cluster A is secure(hbase.security.authentication=kerberos in job conf), cluster B is non-secure(hbase.security.authentication=simple in cluster conf)
> Since we are making VerifyReplication a general comparison tool (not replication specific anymore), shouldn't we cover all possible scenarios (unsecure/secure, secure/secure, secure/unsecure, unsecure/unsecure)?
Yes, it could handle either combination with configuration for the VerifyReplication execution.
For example, these is some examples of the execution for the combinations.
(Please make sure, #3592 is necessary for mixed env)
```
# non-secure vs secure (as well as for opposite)
hbase VerifyReplication \
-D hbase.security.authentication=simple \
-D hbase.zookeeper.quorum=non-secure-hbase-a-1.example.com,non-secure-hbase-a-2.example.com,non-secure-hbase-a-3.example.com \
-D hbase.zookeeper.property.clientPort=2181 \
-D zookeeper.znode.parent=/hbase-a
-D verifyrep.peer.hbase.security.authentication=kerberos \
-D verifyrep.peer.hbase.regionserver.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
-D verifyrep.peer.hbase.master.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
... \
secure-hbase-b-1.example.com,secure-hbase-b-2.example.com,secure-hbase-b-3.example.com:2181:/hbase-b \
table
# secure vs secure (use different principal for cluster daemons)
hbase VerifyReplication \
-D hbase.security.authentication=kerberos \
-D hbase.zookeeper.quorum=secure-hbase-a-1.example.com,secure-hbase-a-2.example.com,secure-hbase-a-3.example.com \
-D hbase.zookeeper.property.clientPort=2181 \
-D zookeeper.znode.parent=/hbase-a
-D hbase.regionserver.kerberos.principal=secure-hbase-a/_HOST@EXAMPLE.COM \
-D hbase.master.kerberos.principal=secure-hbase-a/_HOST@EXAMPLE.COM \
-D verifyrep.peer.hbase.regionserver.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
-D verifyrep.peer.hbase.master.kerberos.principal=secure-hbase-b/_HOST@EXAMPLE.COM \
... \
secure-hbase-b-1.example.com,secure-hbase-b-2.example.com,secure-hbase-b-3.example.com:2181:/hbase-b \
table
```
> Any chance to add UTs?
I'll try it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] bitterfox commented on a change in pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
bitterfox commented on a change in pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#discussion_r691985755
##########
File path: hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
##########
@@ -736,6 +740,40 @@ private static void printUsage(final String errorMsg) {
System.err.println(" $ hbase " +
"org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication" +
" --starttime=1265875194289 --endtime=1265878794289 5 TestTable ");
+ System.err.println();
+ System.err.println(" To verify the data in TestTable between the cluster runs VerifyReplication and cluster-b");
+ System.err.println(" Assume quorum address for cluster-b is cluster-b-1.example.com,cluster-b-2.example.com,cluster-b-3.example.com:2181:/cluster-b");
+ System.err.println(
+ " $ hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication \\\n" +
+ " cluster-b-1.example.com,cluster-b-2.example.com,cluster-b-3.example.com:2181:/cluster-b \\\n" +
+ " TestTable");
+ System.err.println();
+ System.err.println(" To verify the data in TestTable between the secured cluster runs VerifyReplication and insecure cluster-b");
+ System.err.println(
+ " $ hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication \\\n" +
+ " -D verifyrep.peer.hbase.security.authentication=simple \\\n" +
Review comment:
Yes, this is a part of the generic option for the Hadoop tool and isn't option for Java command itself
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/CommandsManual.html#Generic_Options
ToolRunner internally process this sort of common options in GenericOptionsParser
https://github.com/apache/hadoop/blob/07627ef19e2bf4c87f12b53e508edf8fee05856a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/GenericOptionsParser.java#L50-L112
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] wchevreuil merged pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
wchevreuil merged pull request #3591:
URL: https://github.com/apache/hbase/pull/3591
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] shahrs87 commented on a change in pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
shahrs87 commented on a change in pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#discussion_r691245709
##########
File path: hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java
##########
@@ -736,6 +740,40 @@ private static void printUsage(final String errorMsg) {
System.err.println(" $ hbase " +
"org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication" +
" --starttime=1265875194289 --endtime=1265878794289 5 TestTable ");
+ System.err.println();
+ System.err.println(" To verify the data in TestTable between the cluster runs VerifyReplication and cluster-b");
Review comment:
nice work to add examples on how to run VerifyReplication between non secure and secure environment. It is very helpful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] shahrs87 commented on a change in pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
shahrs87 commented on a change in pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#discussion_r691243088
##########
File path: hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationSecureClusterCredentials.java
##########
@@ -0,0 +1,166 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.replication;
+
+import static org.junit.Assert.assertEquals;
+import java.io.File;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.function.Supplier;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtil;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.coprocessor.CoprocessorHost;
+import org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication;
+import org.apache.hadoop.hbase.security.HBaseKerberosUtils;
+import org.apache.hadoop.hbase.security.access.AccessController;
+import org.apache.hadoop.hbase.security.access.SecureTestUtil;
+import org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier;
+import org.apache.hadoop.hbase.security.token.TokenProvider;
+import org.apache.hadoop.hbase.security.visibility.VisibilityTestUtil;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.ReplicationTests;
+import org.apache.hadoop.hbase.zookeeper.ZKClusterId;
+import org.apache.hadoop.hbase.zookeeper.ZKConfig;
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.mapreduce.Job;
+import org.apache.hadoop.minikdc.MiniKdc;
+import org.apache.hadoop.security.Credentials;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.hadoop.security.token.Token;
+import org.apache.hadoop.security.token.TokenIdentifier;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameter;
+import org.junit.runners.Parameterized.Parameters;
+
+@Category({ ReplicationTests.class, LargeTests.class })
+@RunWith(Parameterized.class)
+public class TestVerifyReplicationSecureClusterCredentials {
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+ HBaseClassTestRule.forClass(TestVerifyReplicationSecureClusterCredentials.class);
+
+ private static MiniKdc KDC;
+ private static final HBaseTestingUtil UTIL1 = new HBaseTestingUtil();
Review comment:
nit: add a comment that UTIL1 is secure cluster and UTIL2 is non-secure cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-901057199
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 6s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 20s | master passed |
| +1 :green_heart: | compile | 0m 26s | master passed |
| +1 :green_heart: | shadedjars | 8m 59s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 19s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 12s | the patch passed |
| +1 :green_heart: | compile | 0m 27s | the patch passed |
| +1 :green_heart: | javac | 0m 27s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 56s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 17s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 14m 23s | hbase-mapreduce in the patch passed. |
| | | 44m 47s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux f0a068d03794 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 33e15a261d |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/3/testReport/ |
| Max. process+thread count | 2777 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/3/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-900143708
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 28s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 34s | master passed |
| +1 :green_heart: | compile | 0m 30s | master passed |
| +1 :green_heart: | shadedjars | 8m 20s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 24s | master passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 20s | the patch passed |
| +1 :green_heart: | compile | 0m 32s | the patch passed |
| +1 :green_heart: | javac | 0m 32s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 12s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 21s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 42s | hbase-mapreduce in the patch passed. |
| | | 39m 44s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3591 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 1a1d21d5b503 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 721cb96f8c |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/1/testReport/ |
| Max. process+thread count | 3934 (vs. ulimit of 30000) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3591/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3591: HBASE-26204 Obtain credential for VerifyReplication with peerQuorumAddress
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3591:
URL: https://github.com/apache/hbase/pull/3591#issuecomment-900812334
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org