You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by GitBox <gi...@apache.org> on 2021/08/04 00:26:43 UTC
[GitHub] [phoenix] ramatronics opened a new pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
ramatronics opened a new pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280
HBase now supports a zookeeper-less connection strategy using a Master Registry implementation. For this to work through Phoenix, we need to set a list <host:port>s of the HMaster quorum
```xml
<property>
<name>hbase.masters</name>
<value>master1:16000,master2:16001,master3:16000</value>
</property>
```
To support opting into this from a Phoenix connection URL, we can introduce a "connector type" as follows:
```
jdbc:phoenix+zk:hostname1,2,3...:<properties>
jdbc:phoenix+hrpc:hostname1,2,3...:<properties>
```
Above are examples of opting into hrpc/zk registry implementations of HBase.
If no connector is specified, the Phoenix driver will default to a Zookeeper-based connection as it's always been doing prior to this. If an invalid connector type is specified, we throw a `MalformedUrlException`, similarly if there's a `+` char but the connector type is empty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r685690091
##########
File path: phoenix-core/src/test/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriverTest.java
##########
@@ -153,7 +152,56 @@ public void testNegativeGetConnectionInfo() throws SQLException {
}
}
}
-
+
+ @Test
+ public void testConnectorBootstrap() throws SQLException {
+ // HRPC
+ ConnectionInfo c1 = ConnectionInfo.create("jdbc:phoenix+hrpc:hostname1,hostname2,hostname3:90210:user/principal:/user.keytab;test=false");
+ ReadOnlyProps rop1 = c1.asProps();
+ assertEquals("hostname1:90210,hostname2:90210,hostname3:90210", rop1.get("hbase.masters"));
+ assertTrue(c1.isHRPCBootstrap());
+ assertFalse(c1.isZkBootstrap());
+
+
+ ConnectionInfo c2 = ConnectionInfo.create("jdbc:phoenix+hrpc:hostname1,hostname2,hostname3:user/principal:/user.keytab;test=false");
+ ReadOnlyProps rop2 = c2.asProps();
+ assertEquals("hostname1:60010,hostname2:60010,hostname3:60010", rop2.get("hbase.masters"));
+ assertTrue(c2.isHRPCBootstrap());
+ assertFalse(c2.isZkBootstrap());
+
+
+ // ZK
+ String[] jdbcUrls = new String[]{
+ "jdbc:phoenix+zk:hostname1,hostname2,hostname3:2181:user/principal:/user.keytab;test=false",
Review comment:
Maybe add a test case for ZK quorums with mismatched ports ?
i.e. hostname1:11100,hostname2:11020,hostname3:1130
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -522,15 +562,37 @@ public ConnectionInfo(ConnectionInfo other) {
public ReadOnlyProps asProps() {
Map<String, String> connectionProps = Maps.newHashMapWithExpectedSize(3);
- if (getZookeeperQuorum() != null) {
- connectionProps.put(QueryServices.ZOOKEEPER_QUORUM_ATTRIB, getZookeeperQuorum());
- }
- if (getPort() != null) {
- connectionProps.put(QueryServices.ZOOKEEPER_PORT_ATTRIB, getPort().toString());
+ if (this.isHRPCBootstrap() && getZookeeperQuorum() != null) {
Review comment:
rename ZooKoeperQuorum variable and accessors to simply Quorum ?
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/util/PhoenixRuntime.java
##########
@@ -221,6 +224,14 @@
REQUEST_METRIC_ATTRIB,
};
+ /**
+ * Constants to help determine what to bootstrap the HBase connection with.
+ */
+ public final static String BOOTSTRAP_ZK = "zk";
+ public final static String BOOTSTRAP_HRPC = "hrpc";
+ public final static String BOOTSTRAP_HRPC_DEFAULT_HMASTER_PORT = "60010";
Review comment:
According to the https://hbase.apache.org/book.html, the default port is 16010 now.
However, I think that just using the **hbase.master.info.port** property value, and letting HBase figure out the default port would be the best option.
I think that the current code won't pick up the default port from hbase-site.xml if you specify the masters, but not the ports.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r698988086
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -33,6 +33,10 @@
import javax.annotation.concurrent.Immutable;
+import org.apache.phoenix.jdbc.bootstrapz.HBaseRegistryBootstrap;
Review comment:
can we have just org.apache.phoenix.jdbc.bootstrap. ?
We generally don't use plurals in package names
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/util/PhoenixRuntime.java
##########
@@ -19,6 +19,7 @@
import static org.apache.phoenix.thirdparty.com.google.common.base.Preconditions.checkNotNull;
import static org.apache.phoenix.thirdparty.com.google.common.base.Preconditions.checkArgument;
+import static org.apache.phoenix.thirdparty.com.google.common.collect.Sets.newHashSet;
Review comment:
seems to be unused imports.
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -199,15 +201,20 @@ public void close() throws SQLException {
private static final Object KERBEROS_LOGIN_LOCK = new Object();
private static final char WINDOWS_SEPARATOR_CHAR = '\\';
private static final String REALM_EQUIVALENCY_WARNING_MSG = "Provided principal does not contan a realm and the default realm cannot be determined. Ignoring realm equivalency check.";
+
private static SQLException getMalFormedUrlException(String url) {
return new SQLExceptionInfo.Builder(SQLExceptionCode.MALFORMED_CONNECTION_URL)
.setMessage(url).build().buildException();
}
-
- public String getZookeeperConnectionString() {
- return getZookeeperQuorum() + ":" + getPort();
- }
-
+
+ private static SQLException getMalFormedUrlException(String message, String url) {
+ return getMalFormedUrlException(message + "\n" + url);
+ }
+
+ public String getZookeeperConnectionString() {
Review comment:
This is used by Tephra only, which won't be able to make sense of a HRpc quorum.
Better add a check here, and fail early if the boostrap is not ZK.
(or maybe figure out the cluster ZK quorum somehow, it is possible)
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/bootstrapz/HRpcHBaseRegistryBootstrap.java
##########
@@ -0,0 +1,44 @@
+package org.apache.phoenix.jdbc.bootstrapz;
Review comment:
I think that moving parameter parsing to the bootstrap module, and having an API like:
```
BootstrapRegistry bs = BootstrapFactory.fromURL(String URL);
Map connpProps = bs.generateConnectionProps();
bs.toString()
bs.getZookeperQuorum() (which fails for HRpc)
bs.equals(), bs.hashMap()
```
would be a better encapsulation, and simplify code.
WDYT @joshelser (as this was originally your suggestion) ?
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -604,12 +645,15 @@ public boolean equals(Object obj) {
if (keytab == null) {
if (other.keytab != null) return false;
} else if (!keytab.equals(other.keytab)) return false;
+ if (bootstrap == null) {
+ if (other.bootstrap != null) return false;
+ } else if (!bootstrap.equals(other.bootstrap)) return false;
return true;
}
@Override
public String toString() {
- return zookeeperQuorum + (port == null ? "" : ":" + port)
+ return quorum + (port == null ? "" : ":" + port)
Review comment:
This loses the bootstrap type info.
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -225,16 +232,45 @@ private static boolean isMultiPortUrl(String portStr) {
}
return false;
}
-
+
public static ConnectionInfo create(String url) throws SQLException {
url = url == null ? "" : url;
if (url.isEmpty() || url.equalsIgnoreCase("jdbc:phoenix:")
|| url.equalsIgnoreCase("jdbc:phoenix")) {
return defaultConnectionInfo(url);
}
+
+ // ex: jdbc:phoenix+hrpc:hostname1....
url = url.startsWith(PhoenixRuntime.JDBC_PROTOCOL)
? url.substring(PhoenixRuntime.JDBC_PROTOCOL.length())
: PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR + url;
+
+ HBaseRegistryBootstrapType bootstrapAsEnum = null;
+
+ // ex: +hrpc:hostname1....
+ if (url.startsWith(String.valueOf(PhoenixRuntime.JDBC_PROTOCOL_CONNECTOR_PREFIX))) {
+ int offset = url.indexOf(PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR);
+ final String firstToken = url.substring(0, offset);
+
+ // ex: +hrpc
+ final String bootstrapString = firstToken
+ .replace(String.valueOf(PhoenixRuntime.JDBC_PROTOCOL_CONNECTOR_PREFIX), "")
+ .toUpperCase();
+
+ if (Strings.isNullOrEmpty(bootstrapString)) {
+ throw getMalFormedUrlException(url);
Review comment:
IUC this will not allow URLs without bootstraps specified.
We should support bootstrap-less URLs, and default to ZK for backwards compatibility.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-895572778
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 22s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 21m 37s | master passed |
| +0 | hbaserecompile | 28m 38s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 2s | master passed |
| +1 :green_heart: | checkstyle | 1m 15s | master passed |
| +1 :green_heart: | javadoc | 0m 52s | master passed |
| +0 :ok: | spotbugs | 3m 7s | phoenix-core in master has 973 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 31s | root in the patch failed. |
| +0 | hbaserecompile | 9m 58s | HBase recompiled. |
| -1 :x: | compile | 0m 20s | phoenix-core in the patch failed. |
| -1 :x: | javac | 0m 20s | phoenix-core in the patch failed. |
| -1 :x: | checkstyle | 1m 16s | phoenix-core: The patch generated 24 new + 737 unchanged - 15 fixed = 761 total (was 752) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 50s | the patch passed |
| -1 :x: | spotbugs | 0m 19s | phoenix-core in the patch failed. |
||| _ Other Tests _ |
| -1 :x: | unit | 0m 19s | phoenix-core in the patch failed. |
| +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. |
| | | 52m 34s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/1280 |
| JIRA Issue | PHOENIX-6523 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile |
| uname | Linux 823e64d1ba6c 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / c4d297d |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| mvninstall | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/2/artifact/yetus-general-check/output/patch-mvninstall-root.txt |
| compile | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/2/artifact/yetus-general-check/output/patch-compile-phoenix-core.txt |
| javac | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/2/artifact/yetus-general-check/output/patch-compile-phoenix-core.txt |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/2/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| spotbugs | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/2/artifact/yetus-general-check/output/patch-spotbugs-phoenix-core.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/2/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/2/testReport/ |
| Max. process+thread count | 96 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/2/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] yanxinyi commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
yanxinyi commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r707497846
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/bootstrap/HBaseRegistryBootstrapFactory.java
##########
@@ -0,0 +1,13 @@
+package org.apache.phoenix.jdbc.bootstrap;
Review comment:
same as above and rest of new files, plz add apache license header
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-897591209
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 45s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 24m 27s | master passed |
| +0 | hbaserecompile | 36m 48s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 7s | master passed |
| +1 :green_heart: | checkstyle | 1m 11s | master passed |
| +1 :green_heart: | javadoc | 0m 54s | master passed |
| +0 :ok: | spotbugs | 3m 30s | phoenix-core in master has 973 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 16m 41s | the patch passed |
| +0 | hbaserecompile | 33m 6s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 3s | the patch passed |
| +1 :green_heart: | javac | 1m 3s | the patch passed |
| -1 :x: | checkstyle | 1m 13s | phoenix-core: The patch generated 29 new + 725 unchanged - 27 fixed = 754 total (was 752) |
| -1 :x: | whitespace | 0m 0s | The patch 2 line(s) with tabs. |
| +1 :green_heart: | javadoc | 0m 52s | the patch passed |
| +1 :green_heart: | spotbugs | 3m 31s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 116m 43s | phoenix-core in the patch failed. |
| +1 :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. |
| | | 206m 49s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/5/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/1280 |
| JIRA Issue | PHOENIX-6523 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile |
| uname | Linux 119635c75cbf 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / c4d297d |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/5/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| whitespace | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/5/artifact/yetus-general-check/output/whitespace-tabs.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/5/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/5/testReport/ |
| Max. process+thread count | 12703 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/5/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ramatronics commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
ramatronics commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r699852938
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -225,16 +232,45 @@ private static boolean isMultiPortUrl(String portStr) {
}
return false;
}
-
+
public static ConnectionInfo create(String url) throws SQLException {
url = url == null ? "" : url;
if (url.isEmpty() || url.equalsIgnoreCase("jdbc:phoenix:")
|| url.equalsIgnoreCase("jdbc:phoenix")) {
return defaultConnectionInfo(url);
}
+
+ // ex: jdbc:phoenix+hrpc:hostname1....
url = url.startsWith(PhoenixRuntime.JDBC_PROTOCOL)
? url.substring(PhoenixRuntime.JDBC_PROTOCOL.length())
: PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR + url;
+
+ HBaseRegistryBootstrapType bootstrapAsEnum = null;
+
+ // ex: +hrpc:hostname1....
+ if (url.startsWith(String.valueOf(PhoenixRuntime.JDBC_PROTOCOL_CONNECTOR_PREFIX))) {
+ int offset = url.indexOf(PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR);
+ final String firstToken = url.substring(0, offset);
+
+ // ex: +hrpc
+ final String bootstrapString = firstToken
+ .replace(String.valueOf(PhoenixRuntime.JDBC_PROTOCOL_CONNECTOR_PREFIX), "")
+ .toUpperCase();
+
+ if (Strings.isNullOrEmpty(bootstrapString)) {
+ throw getMalFormedUrlException(url);
Review comment:
@stoty are you saying a URL like `+:hostname1....` is a real possibility? I think that's a malformed string. If it was bootstrap-less it would be `jdbc:phoenix:hostname1`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r698988086
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -33,6 +33,10 @@
import javax.annotation.concurrent.Immutable;
+import org.apache.phoenix.jdbc.bootstrapz.HBaseRegistryBootstrap;
Review comment:
can we have just org.apache.phoenix.jdbc.bootstrap. ?
We generally don't use plurals in package names
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/util/PhoenixRuntime.java
##########
@@ -19,6 +19,7 @@
import static org.apache.phoenix.thirdparty.com.google.common.base.Preconditions.checkNotNull;
import static org.apache.phoenix.thirdparty.com.google.common.base.Preconditions.checkArgument;
+import static org.apache.phoenix.thirdparty.com.google.common.collect.Sets.newHashSet;
Review comment:
seems to be unused imports.
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -199,15 +201,20 @@ public void close() throws SQLException {
private static final Object KERBEROS_LOGIN_LOCK = new Object();
private static final char WINDOWS_SEPARATOR_CHAR = '\\';
private static final String REALM_EQUIVALENCY_WARNING_MSG = "Provided principal does not contan a realm and the default realm cannot be determined. Ignoring realm equivalency check.";
+
private static SQLException getMalFormedUrlException(String url) {
return new SQLExceptionInfo.Builder(SQLExceptionCode.MALFORMED_CONNECTION_URL)
.setMessage(url).build().buildException();
}
-
- public String getZookeeperConnectionString() {
- return getZookeeperQuorum() + ":" + getPort();
- }
-
+
+ private static SQLException getMalFormedUrlException(String message, String url) {
+ return getMalFormedUrlException(message + "\n" + url);
+ }
+
+ public String getZookeeperConnectionString() {
Review comment:
This is used by Tephra only, which won't be able to make sense of a HRpc quorum.
Better add a check here, and fail early if the boostrap is not ZK.
(or maybe figure out the cluster ZK quorum somehow, it is possible)
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/bootstrapz/HRpcHBaseRegistryBootstrap.java
##########
@@ -0,0 +1,44 @@
+package org.apache.phoenix.jdbc.bootstrapz;
Review comment:
I think that moving parameter parsing to the bootstrap module, and having an API like:
```
BootstrapRegistry bs = BootstrapFactory.fromURL(String URL);
Map connpProps = bs.generateConnectionProps();
bs.toString()
bs.getZookeperQuorum() (which fails for HRpc)
bs.equals(), bs.hashMap()
```
would be a better encapsulation, and simplify code.
WDYT @joshelser (as this was originally your suggestion) ?
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -604,12 +645,15 @@ public boolean equals(Object obj) {
if (keytab == null) {
if (other.keytab != null) return false;
} else if (!keytab.equals(other.keytab)) return false;
+ if (bootstrap == null) {
+ if (other.bootstrap != null) return false;
+ } else if (!bootstrap.equals(other.bootstrap)) return false;
return true;
}
@Override
public String toString() {
- return zookeeperQuorum + (port == null ? "" : ":" + port)
+ return quorum + (port == null ? "" : ":" + port)
Review comment:
This loses the bootstrap type info.
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -225,16 +232,45 @@ private static boolean isMultiPortUrl(String portStr) {
}
return false;
}
-
+
public static ConnectionInfo create(String url) throws SQLException {
url = url == null ? "" : url;
if (url.isEmpty() || url.equalsIgnoreCase("jdbc:phoenix:")
|| url.equalsIgnoreCase("jdbc:phoenix")) {
return defaultConnectionInfo(url);
}
+
+ // ex: jdbc:phoenix+hrpc:hostname1....
url = url.startsWith(PhoenixRuntime.JDBC_PROTOCOL)
? url.substring(PhoenixRuntime.JDBC_PROTOCOL.length())
: PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR + url;
+
+ HBaseRegistryBootstrapType bootstrapAsEnum = null;
+
+ // ex: +hrpc:hostname1....
+ if (url.startsWith(String.valueOf(PhoenixRuntime.JDBC_PROTOCOL_CONNECTOR_PREFIX))) {
+ int offset = url.indexOf(PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR);
+ final String firstToken = url.substring(0, offset);
+
+ // ex: +hrpc
+ final String bootstrapString = firstToken
+ .replace(String.valueOf(PhoenixRuntime.JDBC_PROTOCOL_CONNECTOR_PREFIX), "")
+ .toUpperCase();
+
+ if (Strings.isNullOrEmpty(bootstrapString)) {
+ throw getMalFormedUrlException(url);
Review comment:
IUC this will not allow URLs without bootstraps specified.
We should support bootstrap-less URLs, and default to ZK for backwards compatibility.
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -199,15 +201,20 @@ public void close() throws SQLException {
private static final Object KERBEROS_LOGIN_LOCK = new Object();
private static final char WINDOWS_SEPARATOR_CHAR = '\\';
private static final String REALM_EQUIVALENCY_WARNING_MSG = "Provided principal does not contan a realm and the default realm cannot be determined. Ignoring realm equivalency check.";
+
private static SQLException getMalFormedUrlException(String url) {
return new SQLExceptionInfo.Builder(SQLExceptionCode.MALFORMED_CONNECTION_URL)
.setMessage(url).build().buildException();
}
-
- public String getZookeeperConnectionString() {
- return getZookeeperQuorum() + ":" + getPort();
- }
-
+
+ private static SQLException getMalFormedUrlException(String message, String url) {
+ return getMalFormedUrlException(message + "\n" + url);
+ }
+
+ public String getZookeeperConnectionString() {
Review comment:
This is used by Tephra only, which won't be able to make sense of a HRpc quorum.
Better add a check here, and fail early if the boostrap is not ZK.
(or maybe figure out the cluster ZK quorum somehow, if it is possible)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-908914864
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 54s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 3s | master passed |
| +0 | hbaserecompile | 39m 23s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 5s | master passed |
| +1 :green_heart: | checkstyle | 1m 15s | master passed |
| +1 :green_heart: | javadoc | 0m 54s | master passed |
| +0 :ok: | spotbugs | 3m 22s | phoenix-core in master has 972 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 16m 49s | the patch passed |
| +0 | hbaserecompile | 29m 49s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 7s | the patch passed |
| +1 :green_heart: | javac | 1m 7s | the patch passed |
| -1 :x: | checkstyle | 1m 16s | phoenix-core: The patch generated 35 new + 735 unchanged - 17 fixed = 770 total (was 752) |
| -1 :x: | whitespace | 0m 0s | The patch 2 line(s) with tabs. |
| +1 :green_heart: | javadoc | 0m 51s | the patch passed |
| +1 :green_heart: | spotbugs | 3m 32s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 116m 15s | phoenix-core in the patch failed. |
| -1 :x: | asflicense | 0m 34s | The patch generated 5 ASF License warnings. |
| | | 205m 54s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | phoenix.end2end.UpsertSelectIT |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/8/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/1280 |
| JIRA Issue | PHOENIX-6523 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile |
| uname | Linux 43818037d335 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / 9dfb423 |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/8/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| whitespace | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/8/artifact/yetus-general-check/output/whitespace-tabs.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/8/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/8/testReport/ |
| asflicense | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/8/artifact/yetus-general-check/output/patch-asflicense-problems.txt |
| Max. process+thread count | 11682 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/8/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ramatronics commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
ramatronics commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-960105984
I moved Registry context to a separate package. I think it covers @joshelser's ask of starting to clean up the `PhoenixEmbeddedDriver`. It's an in-between to where we want to be ideally (@stoty as you pointed [out here](https://github.com/apache/phoenix/pull/1280#discussion_r698995791)) vs what we can do now realistically.
Created a class called `EmbeddedDriverContext` to pass between the EmbeddedDriver and the Registry classes. I tagged in comments with the 🥴 emoji places where the context is passed back and forth (unnecessarily perhaps) - so let me know if we should make any changes there.
I think an integration test would really help here so I'm going to get started on that.
CC: @dbwong @yanxinyi @virajjasani @apurtell
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-922900664
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 5m 53s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 21m 22s | master passed |
| +0 | hbaserecompile | 28m 3s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 5s | master passed |
| +1 :green_heart: | checkstyle | 1m 15s | master passed |
| +1 :green_heart: | javadoc | 0m 51s | master passed |
| +0 :ok: | spotbugs | 3m 6s | phoenix-core in master has 972 extant spotbugs warnings. |
| -0 :warning: | patch | 3m 15s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 13m 25s | the patch passed |
| +0 | hbaserecompile | 23m 12s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 5s | the patch passed |
| +1 :green_heart: | javac | 1m 5s | the patch passed |
| -1 :x: | checkstyle | 1m 15s | phoenix-core: The patch generated 48 new + 734 unchanged - 18 fixed = 782 total (was 752) |
| -1 :x: | whitespace | 0m 0s | The patch 4 line(s) with tabs. |
| +1 :green_heart: | javadoc | 0m 52s | the patch passed |
| -1 :x: | spotbugs | 3m 21s | phoenix-core generated 1 new + 972 unchanged - 0 fixed = 973 total (was 972) |
||| _ Other Tests _ |
| -1 :x: | unit | 161m 45s | phoenix-core in the patch failed. |
| -1 :x: | asflicense | 0m 16s | The patch generated 7 ASF License warnings. |
| | | 230m 39s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:phoenix-core |
| | Class org.apache.phoenix.jdbc.MalformedUrlException is not derived from an Exception, even though it is named as such At MalformedUrlException.java:from an Exception, even though it is named as such At MalformedUrlException.java:[lines 8-15] |
| Failed junit tests | phoenix.mapreduce.util.PhoenixConfigurationUtilTest |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/10/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/1280 |
| JIRA Issue | PHOENIX-6523 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile |
| uname | Linux 03f2c05daf9d 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / f606289 |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/10/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| whitespace | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/10/artifact/yetus-general-check/output/whitespace-tabs.txt |
| spotbugs | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/10/artifact/yetus-general-check/output/new-spotbugs-phoenix-core.html |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/10/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/10/testReport/ |
| asflicense | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/10/artifact/yetus-general-check/output/patch-asflicense-problems.txt |
| Max. process+thread count | 486 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/10/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-895722503
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 4s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 24m 13s | master passed |
| +0 | hbaserecompile | 31m 12s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 5s | master passed |
| +1 :green_heart: | checkstyle | 1m 13s | master passed |
| +1 :green_heart: | javadoc | 0m 51s | master passed |
| +0 :ok: | spotbugs | 3m 26s | phoenix-core in master has 973 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 16m 53s | the patch passed |
| +0 | hbaserecompile | 27m 22s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 5s | the patch passed |
| +1 :green_heart: | javac | 1m 5s | the patch passed |
| -1 :x: | checkstyle | 1m 14s | phoenix-core: The patch generated 33 new + 727 unchanged - 25 fixed = 760 total (was 752) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 51s | the patch passed |
| +1 :green_heart: | spotbugs | 3m 38s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 117m 50s | phoenix-core in the patch failed. |
| +1 :green_heart: | asflicense | 0m 34s | The patch does not generate ASF License warnings. |
| | | 190m 58s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/1280 |
| JIRA Issue | PHOENIX-6523 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile |
| uname | Linux 5da2aa6dfc67 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / c4d297d |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/3/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/3/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/3/testReport/ |
| Max. process+thread count | 11607 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/3/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ramatronics commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
ramatronics commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-906106591
@stoty the links to checkstyle/whitespaces are missing
Also @joshelser I split and moved the parsing logic into classes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] bharathv commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
bharathv commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r690587438
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -517,20 +570,46 @@ public ConnectionInfo(String zookeeperQuorum, Integer port, String rootNode) {
* @param other The instance to copy
*/
public ConnectionInfo(ConnectionInfo other) {
- this(other.zookeeperQuorum, other.port, other.rootNode, other.principal, other.keytab);
+ this(other.quorum, other.port, other.rootNode, other.principal, other.keytab);
}
public ReadOnlyProps asProps() {
Map<String, String> connectionProps = Maps.newHashMapWithExpectedSize(3);
- if (getZookeeperQuorum() != null) {
- connectionProps.put(QueryServices.ZOOKEEPER_QUORUM_ATTRIB, getZookeeperQuorum());
- }
- if (getPort() != null) {
- connectionProps.put(QueryServices.ZOOKEEPER_PORT_ATTRIB, getPort().toString());
+ if (this.isHRPCBootstrap() && getQuorum() != null) {
+ final String[] masters = getQuorum().split(",");
Review comment:
+1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-916878888
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 5m 56s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 24m 2s | master passed |
| +0 | hbaserecompile | 31m 7s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 5s | master passed |
| +1 :green_heart: | checkstyle | 1m 14s | master passed |
| +1 :green_heart: | javadoc | 0m 52s | master passed |
| +0 :ok: | spotbugs | 3m 23s | phoenix-core in master has 972 extant spotbugs warnings. |
| -0 :warning: | patch | 3m 32s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 16m 44s | the patch passed |
| +0 | hbaserecompile | 27m 5s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 4s | the patch passed |
| +1 :green_heart: | javac | 1m 4s | the patch passed |
| -1 :x: | checkstyle | 1m 14s | phoenix-core: The patch generated 36 new + 735 unchanged - 17 fixed = 771 total (was 752) |
| -1 :x: | whitespace | 0m 0s | The patch 4 line(s) with tabs. |
| +1 :green_heart: | javadoc | 0m 51s | the patch passed |
| +1 :green_heart: | spotbugs | 3m 30s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 161m 56s | phoenix-core in the patch failed. |
| -1 :x: | asflicense | 0m 14s | The patch generated 5 ASF License warnings. |
| | | 237m 53s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | phoenix.mapreduce.util.PhoenixConfigurationUtilTest |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/9/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/1280 |
| JIRA Issue | PHOENIX-6523 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile |
| uname | Linux 8979c3c1e2f9 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / 19b2260 |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/9/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| whitespace | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/9/artifact/yetus-general-check/output/whitespace-tabs.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/9/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/9/testReport/ |
| asflicense | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/9/artifact/yetus-general-check/output/patch-asflicense-problems.txt |
| Max. process+thread count | 432 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/9/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ramatronics commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
ramatronics commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-960105984
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r699970369
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -225,16 +232,45 @@ private static boolean isMultiPortUrl(String portStr) {
}
return false;
}
-
+
public static ConnectionInfo create(String url) throws SQLException {
url = url == null ? "" : url;
if (url.isEmpty() || url.equalsIgnoreCase("jdbc:phoenix:")
|| url.equalsIgnoreCase("jdbc:phoenix")) {
return defaultConnectionInfo(url);
}
+
+ // ex: jdbc:phoenix+hrpc:hostname1....
url = url.startsWith(PhoenixRuntime.JDBC_PROTOCOL)
? url.substring(PhoenixRuntime.JDBC_PROTOCOL.length())
: PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR + url;
+
+ HBaseRegistryBootstrapType bootstrapAsEnum = null;
+
+ // ex: +hrpc:hostname1....
+ if (url.startsWith(String.valueOf(PhoenixRuntime.JDBC_PROTOCOL_CONNECTOR_PREFIX))) {
+ int offset = url.indexOf(PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR);
+ final String firstToken = url.substring(0, offset);
+
+ // ex: +hrpc
+ final String bootstrapString = firstToken
+ .replace(String.valueOf(PhoenixRuntime.JDBC_PROTOCOL_CONNECTOR_PREFIX), "")
+ .toUpperCase();
+
+ if (Strings.isNullOrEmpty(bootstrapString)) {
+ throw getMalFormedUrlException(url);
Review comment:
Right, I didn't notice this block is only for URLS that have the "+" character.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] yanxinyi commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
yanxinyi commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r712650650
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -33,6 +33,7 @@
import javax.annotation.concurrent.Immutable;
+import org.apache.phoenix.jdbc.bootstrap.*;
Review comment:
plz avoid to used import *
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] yanxinyi commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
yanxinyi commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r707499222
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/util/PhoenixRuntime.java
##########
@@ -56,6 +56,7 @@
import org.apache.phoenix.thirdparty.org.apache.commons.cli.ParseException;
import org.apache.commons.lang3.StringEscapeUtils;
import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HBaseConfiguration;
Review comment:
unused import?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-908914864
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 6m 54s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 3s | master passed |
| +0 | hbaserecompile | 39m 23s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 5s | master passed |
| +1 :green_heart: | checkstyle | 1m 15s | master passed |
| +1 :green_heart: | javadoc | 0m 54s | master passed |
| +0 :ok: | spotbugs | 3m 22s | phoenix-core in master has 972 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 16m 49s | the patch passed |
| +0 | hbaserecompile | 29m 49s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 7s | the patch passed |
| +1 :green_heart: | javac | 1m 7s | the patch passed |
| -1 :x: | checkstyle | 1m 16s | phoenix-core: The patch generated 35 new + 735 unchanged - 17 fixed = 770 total (was 752) |
| -1 :x: | whitespace | 0m 0s | The patch 2 line(s) with tabs. |
| +1 :green_heart: | javadoc | 0m 51s | the patch passed |
| +1 :green_heart: | spotbugs | 3m 32s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 116m 15s | phoenix-core in the patch failed. |
| -1 :x: | asflicense | 0m 34s | The patch generated 5 ASF License warnings. |
| | | 205m 54s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | phoenix.end2end.UpsertSelectIT |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/8/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/1280 |
| JIRA Issue | PHOENIX-6523 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile |
| uname | Linux 43818037d335 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / 9dfb423 |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/8/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| whitespace | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/8/artifact/yetus-general-check/output/whitespace-tabs.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/8/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/8/testReport/ |
| asflicense | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/8/artifact/yetus-general-check/output/patch-asflicense-problems.txt |
| Max. process+thread count | 11682 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/8/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r690053593
##########
File path: phoenix-core/src/test/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriverTest.java
##########
@@ -153,7 +152,56 @@ public void testNegativeGetConnectionInfo() throws SQLException {
}
}
}
-
+
+ @Test
+ public void testConnectorBootstrap() throws SQLException {
+ // HRPC
+ ConnectionInfo c1 = ConnectionInfo.create("jdbc:phoenix+hrpc:hostname1,hostname2,hostname3:90210:user/principal:/user.keytab;test=false");
+ ReadOnlyProps rop1 = c1.asProps();
+ assertEquals("hostname1:90210,hostname2:90210,hostname3:90210", rop1.get("hbase.masters"));
+ assertTrue(c1.isHRPCBootstrap());
+ assertFalse(c1.isZkBootstrap());
+
+
+ ConnectionInfo c2 = ConnectionInfo.create("jdbc:phoenix+hrpc:hostname1,hostname2,hostname3:user/principal:/user.keytab;test=false");
+ ReadOnlyProps rop2 = c2.asProps();
+ assertEquals("hostname1:60010,hostname2:60010,hostname3:60010", rop2.get("hbase.masters"));
+ assertTrue(c2.isHRPCBootstrap());
+ assertFalse(c2.isZkBootstrap());
+
+
+ // ZK
+ String[] jdbcUrls = new String[]{
+ "jdbc:phoenix+zk:hostname1,hostname2,hostname3:2181:user/principal:/user.keytab;test=false",
Review comment:
Good point on the delimiter problem.
The HBase feature is described in HBASE-12706. I don't think we should invent a new format if we cannot support the HBase native format.
It can always be set in hbase-site.xml.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-901690571
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 3s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 24m 13s | master passed |
| +0 | hbaserecompile | 31m 58s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 5s | master passed |
| +1 :green_heart: | checkstyle | 1m 16s | master passed |
| +1 :green_heart: | javadoc | 0m 52s | master passed |
| +0 :ok: | spotbugs | 3m 20s | phoenix-core in master has 972 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 16m 46s | the patch passed |
| +0 | hbaserecompile | 27m 37s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 6s | the patch passed |
| +1 :green_heart: | javac | 1m 6s | the patch passed |
| -1 :x: | checkstyle | 1m 14s | phoenix-core: The patch generated 35 new + 735 unchanged - 17 fixed = 770 total (was 752) |
| -1 :x: | whitespace | 0m 0s | The patch 2 line(s) with tabs. |
| +1 :green_heart: | javadoc | 0m 53s | the patch passed |
| +1 :green_heart: | spotbugs | 3m 33s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 118m 2s | phoenix-core in the patch passed. |
| -1 :x: | asflicense | 0m 14s | The patch generated 5 ASF License warnings. |
| | | 190m 7s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/7/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/1280 |
| JIRA Issue | PHOENIX-6523 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile |
| uname | Linux ba856428ad27 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / bfc4621 |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/7/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| whitespace | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/7/artifact/yetus-general-check/output/whitespace-tabs.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/7/testReport/ |
| asflicense | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/7/artifact/yetus-general-check/output/patch-asflicense-problems.txt |
| Max. process+thread count | 10727 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/7/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-897461798
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 1s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 21m 57s | master passed |
| +0 | hbaserecompile | 28m 49s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 2s | master passed |
| +1 :green_heart: | checkstyle | 1m 14s | master passed |
| +1 :green_heart: | javadoc | 0m 51s | master passed |
| +0 :ok: | spotbugs | 3m 7s | phoenix-core in master has 973 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 13m 30s | the patch passed |
| +0 | hbaserecompile | 25m 52s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 3s | the patch passed |
| +1 :green_heart: | javac | 1m 3s | the patch passed |
| -1 :x: | checkstyle | 1m 14s | phoenix-core: The patch generated 20 new + 735 unchanged - 17 fixed = 755 total (was 752) |
| -1 :x: | whitespace | 0m 0s | The patch 2 line(s) with tabs. |
| +1 :green_heart: | javadoc | 0m 49s | the patch passed |
| +1 :green_heart: | spotbugs | 3m 15s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 119m 13s | phoenix-core in the patch passed. |
| +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. |
| | | 186m 0s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/1280 |
| JIRA Issue | PHOENIX-6523 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile |
| uname | Linux edf46d24c8ec 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / c4d297d |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/4/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| whitespace | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/4/artifact/yetus-general-check/output/whitespace-tabs.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/4/testReport/ |
| Max. process+thread count | 7332 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/4/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ramatronics commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
ramatronics commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r706014653
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/bootstrapz/HRpcHBaseRegistryBootstrap.java
##########
@@ -0,0 +1,44 @@
+package org.apache.phoenix.jdbc.bootstrapz;
Review comment:
It gets a little tricky because the URL is mutated accordingly to tokenize later if there's a +; and we'd be moving that logic into the `fromURL` method. Also we'd have to move the `MalformedUrlException` class into its own package to avoid a circular dependency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ramatronics commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
ramatronics commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r712001257
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/bootstrapz/HRpcHBaseRegistryBootstrap.java
##########
@@ -0,0 +1,44 @@
+package org.apache.phoenix.jdbc.bootstrapz;
Review comment:
@stoty check out the updates. I don't think it makes sense to persist the URL in the bootstrap. There's no reason for the BootstrapRegistry object to keep the context of the URL... so I ended up creating a factory method that parses the URL, and spits out a BootstrapRegistry along with the remaining URL in a tuple format.
Let me know if that looks good I'll get started on the integration test with the mini cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r707162085
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/bootstrapz/HRpcHBaseRegistryBootstrap.java
##########
@@ -0,0 +1,44 @@
+package org.apache.phoenix.jdbc.bootstrapz;
Review comment:
Moving MalformedUrlException to a seprate file is not a problem.
Also the I think that the URL without the boostrap info could be returned as
```
BootstrapRegistry bs = BootstrapFactory.fromURL(String URL);
Map connpProps = bs.generateConnectionProps();
url = bs.getRemainingURL();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ramatronics commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
ramatronics commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r715281233
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/bootstrapz/HRpcHBaseRegistryBootstrap.java
##########
@@ -0,0 +1,44 @@
+package org.apache.phoenix.jdbc.bootstrapz;
Review comment:
> The current stateless solution is not worth it.
> you should encapsulate your code and data in a class.
I guess I'm failing to see the opportunity here for encapsulating the code/data. My original intent was to abstract the interface for `generateConnectionProps` and the logic each registry has for it. Doesn't necessarily mean we have to feed it some data to abstract as well. Sure maybe down the road, it may make sense to move more data from `PhoenixEmbeddedDriver` into these registry classes as per need... but I don't think any of these classes require context from the URL after being created for the current use case.
> relies on the fact the state for the ZK and HRpp connections are named similarly
How? Was hoping to rely on the fact that unique ZK and HRpc identifiers can be parsed into a discrete set of enums. They essentially provide translation of the string value from the conn string to an enum which we then parse into the actual class of the registry. We can eliminate the enum and just add straight string comparisons, but regardless we need to check the protocol string specified in the connection URL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] yanxinyi commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
yanxinyi commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r707497381
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/bootstrap/HBaseRegistryBootstrap.java
##########
@@ -0,0 +1,32 @@
+package org.apache.phoenix.jdbc.bootstrap;
Review comment:
nit: apache license header
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] yanxinyi commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
yanxinyi commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r707498966
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/query/QueryServices.java
##########
@@ -56,6 +56,7 @@
"phoenix.query.server.orderBy.spooling.enabled";
public static final String HBASE_CLIENT_KEYTAB = "hbase.myclient.keytab";
public static final String HBASE_CLIENT_PRINCIPAL = "hbase.myclient.principal";
+
Review comment:
nit: remove empty line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] dbwong commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
dbwong commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r685659569
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -44,14 +30,22 @@
import org.apache.phoenix.query.ConnectionQueryServices;
import org.apache.phoenix.query.HBaseFactoryProvider;
import org.apache.phoenix.query.QueryServices;
+import org.apache.phoenix.thirdparty.com.google.common.collect.ImmutableMap;
+import org.apache.phoenix.thirdparty.com.google.common.collect.Maps;
import org.apache.phoenix.util.PhoenixRuntime;
import org.apache.phoenix.util.PropertiesUtil;
import org.apache.phoenix.util.ReadOnlyProps;
import org.apache.phoenix.util.SQLCloseable;
import org.slf4j.LoggerFactory;
-import org.apache.phoenix.thirdparty.com.google.common.collect.ImmutableMap;
-import org.apache.phoenix.thirdparty.com.google.common.collect.Maps;
+import javax.annotation.concurrent.Immutable;
+import java.io.IOException;
+import java.sql.*;
Review comment:
We do not use import * in phoenix code guidelines, you can use our eclipse template or similar.
##########
File path: phoenix-core/src/test/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriverTest.java
##########
@@ -18,21 +18,20 @@
package org.apache.phoenix.jdbc;
-import static org.junit.Assert.assertEquals;
-import static org.junit.Assert.assertFalse;
-import static org.junit.Assert.assertTrue;
-
-import java.sql.Driver;
-import java.sql.DriverManager;
-import java.sql.SQLException;
-
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HConstants;
import org.apache.phoenix.exception.SQLExceptionCode;
import org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.ConnectionInfo;
import org.apache.phoenix.query.HBaseFactoryProvider;
+import org.apache.phoenix.util.ReadOnlyProps;
import org.junit.Test;
+import java.sql.Driver;
+import java.sql.DriverManager;
+import java.sql.SQLException;
+
+import static org.junit.Assert.*;
Review comment:
* import again
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/query/QueryServices.java
##########
@@ -56,6 +56,9 @@
"phoenix.query.server.orderBy.spooling.enabled";
public static final String HBASE_CLIENT_KEYTAB = "hbase.myclient.keytab";
public static final String HBASE_CLIENT_PRINCIPAL = "hbase.myclient.principal";
+
+ public static final String HBASE_MASTERS = "hbase.masters";
Review comment:
Is this defined already in a publicly facing hbase module? If so can we directly include it via hbase compat or similar?
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -481,16 +514,20 @@ private Configuration getConfiguration(ReadOnlyProps props, Properties info, Str
}
return config;
}
-
+
private final Integer port;
private final String rootNode;
private final String zookeeperQuorum;
private final boolean isConnectionless;
private final String principal;
private final String keytab;
private final User user;
-
+
public ConnectionInfo(String zookeeperQuorum, Integer port, String rootNode, String principal, String keytab) {
+ this(zookeeperQuorum, port, rootNode, principal, keytab, null);
+ }
+
+ public ConnectionInfo(String zookeeperQuorum, Integer port, String rootNode, String principal, String keytab, String bootstrap) {
Review comment:
Can we consider a parsed Enum or was there a reason you used a String for bootstrap?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ramatronics commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
ramatronics commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-960105984
I moved Registry context to a separate package. I think it covers @joshelser's ask of starting to clean up the `PhoenixEmbeddedDriver`. It's an in-between to where we want to be ideally (@stoty as you pointed [out here](https://github.com/apache/phoenix/pull/1280#discussion_r698995791)) vs what we can do now realistically.
Created a class called `EmbeddedDriverContext` to pass between the EmbeddedDriver and the Registry classes. I tagged in comments with the 🥴 emoji places where the context is passed back and forth (unnecessarily perhaps) - so let me know if we should make any changes there.
I think an integration test would really help here so I'm going to get started on that.
CC: @dbwong @yanxinyi @virajjasani @apurtell
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-918006113
Looking at the code, I'd expect PhoenixEmbeddedDriver.ConnectionInfo.normalize() to fail for Hrpc connections.
Could you add an Integration Test to check that the Hrpc connections work (i.e. starting a minicluster and connecting to it with HRPC) ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] joshelser commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
joshelser commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-898789496
```
jdbc:phoenix+zk:hostname1,2,3...:<properties>
jdbc:phoenix+hrpc:hostname1,2,3...:<properties>
```
What is "hrpc" supposed to stand for in this case? I would think "masters" or "registry" might be more clear?
I think the problem here is that @bharathv built the "default" implementation to be pluggable. While this is nice, do we have any expectations to support any possible mechanism? It would be helpful (for me, a reviewer) to understand what is in scope and what's out of scope. I'm assuming you don't want to try to support any registry, but it might be a good idea to keep that in mind so as to not code ourselves into a corner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] joshelser commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
joshelser commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r688845312
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -225,16 +233,44 @@ private static boolean isMultiPortUrl(String portStr) {
}
return false;
}
-
+
public static ConnectionInfo create(String url) throws SQLException {
url = url == null ? "" : url;
if (url.isEmpty() || url.equalsIgnoreCase("jdbc:phoenix:")
|| url.equalsIgnoreCase("jdbc:phoenix")) {
return defaultConnectionInfo(url);
}
+
+ // ex: jdbc:phoenix+hrpc:hostname1....
url = url.startsWith(PhoenixRuntime.JDBC_PROTOCOL)
? url.substring(PhoenixRuntime.JDBC_PROTOCOL.length())
: PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR + url;
+
+ PhoenixRuntime.HBaseBootstrap bootstrapAsEnum = null;
+
+ // ex: +hrpc:hostname1....
+ if (url.startsWith(String.valueOf(PhoenixRuntime.JDBC_PROTOCOL_CONNECTOR_PREFIX))) {
+ final String firstToken = url.split(String.valueOf(PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR))[0];
Review comment:
Suggest not using `String.split(String)` as this ends up being a regex operation.
Please use `int offset = url.indexOf(PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR)` and `url.subString(0, offset)` to extract the first piece (or use a tokenizer like elsewhere in this file).
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -517,20 +570,46 @@ public ConnectionInfo(String zookeeperQuorum, Integer port, String rootNode) {
* @param other The instance to copy
*/
public ConnectionInfo(ConnectionInfo other) {
- this(other.zookeeperQuorum, other.port, other.rootNode, other.principal, other.keytab);
+ this(other.quorum, other.port, other.rootNode, other.principal, other.keytab);
}
public ReadOnlyProps asProps() {
Map<String, String> connectionProps = Maps.newHashMapWithExpectedSize(3);
- if (getZookeeperQuorum() != null) {
- connectionProps.put(QueryServices.ZOOKEEPER_QUORUM_ATTRIB, getZookeeperQuorum());
- }
- if (getPort() != null) {
- connectionProps.put(QueryServices.ZOOKEEPER_PORT_ATTRIB, getPort().toString());
+ if (this.isHRPCBootstrap() && getQuorum() != null) {
+ final String[] masters = getQuorum().split(",");
Review comment:
While the enum suggestion is cleaner, I think pushing this "bootstrapping" into separate classes for each implementation is a pretty standard object-oriented pattern to follow. (e.g. make an interface for PhoenixRuntimeBootstrap with two implementations: one for ZK and one for MasterRegistry).
PhoenixEmbeddedDriver.java is a big mess already and I think pushing even more logic into it will make it worse.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ramatronics commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
ramatronics commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-901534582
> to understand what is in scope and what's out of scope
The scope right now is to support HBase registry _if specified_ or else always default to ZK. I think the most important thing here is to ensure there's no regression to existing connections. This HRpc thing should be treated as an opt-in, else we should preserve the same behaviour as before.
> I'm assuming you don't want to try to support any registry, but it might be a good idea to keep that in mind so as to not code ourselves into a corner.
I think using some OOP to clean the classes up as you suggested is a good foundation to set for other registries down the road
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ramatronics commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
ramatronics commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r687415822
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -481,16 +514,20 @@ private Configuration getConfiguration(ReadOnlyProps props, Properties info, Str
}
return config;
}
-
+
private final Integer port;
private final String rootNode;
private final String zookeeperQuorum;
private final boolean isConnectionless;
private final String principal;
private final String keytab;
private final User user;
-
+
public ConnectionInfo(String zookeeperQuorum, Integer port, String rootNode, String principal, String keytab) {
+ this(zookeeperQuorum, port, rootNode, principal, keytab, null);
+ }
+
+ public ConnectionInfo(String zookeeperQuorum, Integer port, String rootNode, String principal, String keytab, String bootstrap) {
Review comment:
Been writing a lot of Python lately. _whats an enum_?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r712236140
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/bootstrapz/HRpcHBaseRegistryBootstrap.java
##########
@@ -0,0 +1,44 @@
+package org.apache.phoenix.jdbc.bootstrapz;
Review comment:
I don't want to hold up the patch, but the current solution feels like the worst of both worlds, as it adds a lots of complexity without removing any.
Classic OO principles say that you should encapsulate your code and data in a class.
The current stateless solution is not worth it. It relies on the fact the state for the ZK and HRpp connections are named similarly, and can be stored in the same variables, but is a bad abstraction.
IMO either just fold back the code into the main class, and save the hassle of using registries, or use stateful registry classes with proper data encapsulation as I suggested above.
@joshelser @yanxinyi what's your take on this ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r699970369
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -225,16 +232,45 @@ private static boolean isMultiPortUrl(String portStr) {
}
return false;
}
-
+
public static ConnectionInfo create(String url) throws SQLException {
url = url == null ? "" : url;
if (url.isEmpty() || url.equalsIgnoreCase("jdbc:phoenix:")
|| url.equalsIgnoreCase("jdbc:phoenix")) {
return defaultConnectionInfo(url);
}
+
+ // ex: jdbc:phoenix+hrpc:hostname1....
url = url.startsWith(PhoenixRuntime.JDBC_PROTOCOL)
? url.substring(PhoenixRuntime.JDBC_PROTOCOL.length())
: PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR + url;
+
+ HBaseRegistryBootstrapType bootstrapAsEnum = null;
+
+ // ex: +hrpc:hostname1....
+ if (url.startsWith(String.valueOf(PhoenixRuntime.JDBC_PROTOCOL_CONNECTOR_PREFIX))) {
+ int offset = url.indexOf(PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR);
+ final String firstToken = url.substring(0, offset);
+
+ // ex: +hrpc
+ final String bootstrapString = firstToken
+ .replace(String.valueOf(PhoenixRuntime.JDBC_PROTOCOL_CONNECTOR_PREFIX), "")
+ .toUpperCase();
+
+ if (Strings.isNullOrEmpty(bootstrapString)) {
+ throw getMalFormedUrlException(url);
Review comment:
Right, I didn't notice this block is only for URLS that have the "+" character.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] bharathv commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
bharathv commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r690587438
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -517,20 +570,46 @@ public ConnectionInfo(String zookeeperQuorum, Integer port, String rootNode) {
* @param other The instance to copy
*/
public ConnectionInfo(ConnectionInfo other) {
- this(other.zookeeperQuorum, other.port, other.rootNode, other.principal, other.keytab);
+ this(other.quorum, other.port, other.rootNode, other.principal, other.keytab);
}
public ReadOnlyProps asProps() {
Map<String, String> connectionProps = Maps.newHashMapWithExpectedSize(3);
- if (getZookeeperQuorum() != null) {
- connectionProps.put(QueryServices.ZOOKEEPER_QUORUM_ATTRIB, getZookeeperQuorum());
- }
- if (getPort() != null) {
- connectionProps.put(QueryServices.ZOOKEEPER_PORT_ATTRIB, getPort().toString());
+ if (this.isHRPCBootstrap() && getQuorum() != null) {
+ final String[] masters = getQuorum().split(",");
Review comment:
+1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ramatronics commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
ramatronics commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r699852938
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -225,16 +232,45 @@ private static boolean isMultiPortUrl(String portStr) {
}
return false;
}
-
+
public static ConnectionInfo create(String url) throws SQLException {
url = url == null ? "" : url;
if (url.isEmpty() || url.equalsIgnoreCase("jdbc:phoenix:")
|| url.equalsIgnoreCase("jdbc:phoenix")) {
return defaultConnectionInfo(url);
}
+
+ // ex: jdbc:phoenix+hrpc:hostname1....
url = url.startsWith(PhoenixRuntime.JDBC_PROTOCOL)
? url.substring(PhoenixRuntime.JDBC_PROTOCOL.length())
: PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR + url;
+
+ HBaseRegistryBootstrapType bootstrapAsEnum = null;
+
+ // ex: +hrpc:hostname1....
+ if (url.startsWith(String.valueOf(PhoenixRuntime.JDBC_PROTOCOL_CONNECTOR_PREFIX))) {
+ int offset = url.indexOf(PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR);
+ final String firstToken = url.substring(0, offset);
+
+ // ex: +hrpc
+ final String bootstrapString = firstToken
+ .replace(String.valueOf(PhoenixRuntime.JDBC_PROTOCOL_CONNECTOR_PREFIX), "")
+ .toUpperCase();
+
+ if (Strings.isNullOrEmpty(bootstrapString)) {
+ throw getMalFormedUrlException(url);
Review comment:
@stoty are you saying a URL like `+:hostname1....` is a real possibility? I think that's a malformed string. If it was bootstrap-less it would be `jdbc:phoenix:hostname1`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-901603823
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 5m 38s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 24m 19s | master passed |
| +0 | hbaserecompile | 31m 26s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 8s | master passed |
| +1 :green_heart: | checkstyle | 1m 15s | master passed |
| +1 :green_heart: | javadoc | 0m 52s | master passed |
| +0 :ok: | spotbugs | 3m 20s | phoenix-core in master has 972 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 16m 36s | the patch passed |
| +0 | hbaserecompile | 27m 14s | HBase recompiled. |
| +1 :green_heart: | compile | 1m 7s | the patch passed |
| +1 :green_heart: | javac | 1m 7s | the patch passed |
| -1 :x: | checkstyle | 1m 20s | phoenix-core: The patch generated 36 new + 735 unchanged - 17 fixed = 771 total (was 752) |
| -1 :x: | whitespace | 0m 0s | The patch 2 line(s) with tabs. |
| +1 :green_heart: | javadoc | 0m 57s | the patch passed |
| +1 :green_heart: | spotbugs | 4m 11s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 122m 8s | phoenix-core in the patch failed. |
| -1 :x: | asflicense | 0m 34s | The patch generated 5 ASF License warnings. |
| | | 200m 52s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/6/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/1280 |
| JIRA Issue | PHOENIX-6523 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile |
| uname | Linux edb51fb2019d 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | master / bfc4621 |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/6/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| whitespace | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/6/artifact/yetus-general-check/output/whitespace-tabs.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/6/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/6/testReport/ |
| asflicense | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/6/artifact/yetus-general-check/output/patch-asflicense-problems.txt |
| Max. process+thread count | 11315 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1280/6/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ramatronics commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
ramatronics commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-906109289
Jenkins test this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r698994781
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java
##########
@@ -199,15 +201,20 @@ public void close() throws SQLException {
private static final Object KERBEROS_LOGIN_LOCK = new Object();
private static final char WINDOWS_SEPARATOR_CHAR = '\\';
private static final String REALM_EQUIVALENCY_WARNING_MSG = "Provided principal does not contan a realm and the default realm cannot be determined. Ignoring realm equivalency check.";
+
private static SQLException getMalFormedUrlException(String url) {
return new SQLExceptionInfo.Builder(SQLExceptionCode.MALFORMED_CONNECTION_URL)
.setMessage(url).build().buildException();
}
-
- public String getZookeeperConnectionString() {
- return getZookeeperQuorum() + ":" + getPort();
- }
-
+
+ private static SQLException getMalFormedUrlException(String message, String url) {
+ return getMalFormedUrlException(message + "\n" + url);
+ }
+
+ public String getZookeeperConnectionString() {
Review comment:
This is used by Tephra only, which won't be able to make sense of a HRpc quorum.
Better add a check here, and fail early if the boostrap is not ZK.
(or maybe figure out the cluster ZK quorum somehow, if it is possible)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ramatronics commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
ramatronics commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r687411337
##########
File path: phoenix-core/src/test/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriverTest.java
##########
@@ -153,7 +152,56 @@ public void testNegativeGetConnectionInfo() throws SQLException {
}
}
}
-
+
+ @Test
+ public void testConnectorBootstrap() throws SQLException {
+ // HRPC
+ ConnectionInfo c1 = ConnectionInfo.create("jdbc:phoenix+hrpc:hostname1,hostname2,hostname3:90210:user/principal:/user.keytab;test=false");
+ ReadOnlyProps rop1 = c1.asProps();
+ assertEquals("hostname1:90210,hostname2:90210,hostname3:90210", rop1.get("hbase.masters"));
+ assertTrue(c1.isHRPCBootstrap());
+ assertFalse(c1.isZkBootstrap());
+
+
+ ConnectionInfo c2 = ConnectionInfo.create("jdbc:phoenix+hrpc:hostname1,hostname2,hostname3:user/principal:/user.keytab;test=false");
+ ReadOnlyProps rop2 = c2.asProps();
+ assertEquals("hostname1:60010,hostname2:60010,hostname3:60010", rop2.get("hbase.masters"));
+ assertTrue(c2.isHRPCBootstrap());
+ assertFalse(c2.isZkBootstrap());
+
+
+ // ZK
+ String[] jdbcUrls = new String[]{
+ "jdbc:phoenix+zk:hostname1,hostname2,hostname3:2181:user/principal:/user.keytab;test=false",
Review comment:
Doesn't look like there's current support for ZK quorums with mismatched ports... also probably because in the example string you provided, the same delimiter to tokenize (`:`) is the same one used to distinguish hostname from the port in each pair.
If we do want to support it, would something like this work?
```
jdbc:phoenix+zk:hostname1,hostname2,hostname3:2181,2182,2183:user/principal:/user.keytab;test=false
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ramatronics commented on a change in pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
ramatronics commented on a change in pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#discussion_r715281233
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/jdbc/bootstrapz/HRpcHBaseRegistryBootstrap.java
##########
@@ -0,0 +1,44 @@
+package org.apache.phoenix.jdbc.bootstrapz;
Review comment:
> The current stateless solution is not worth it.
> you should encapsulate your code and data in a class.
I guess I'm failing to see the opportunity here for encapsulating the code/data. My original intent was to abstract the interface for `generateConnectionProps` and the logic each registry has for it. Doesn't necessarily mean we have to feed it some data to abstract as well. Sure maybe down the road, it may make sense to move more data from `PhoenixEmbeddedDriver` into these registry classes as per need... but I don't think any of these classes require context from the URL after being created for the current use case.
> relies on the fact the state for the ZK and HRpp connections are named similarly
How? Was hoping to rely on the fact that unique ZK and HRpc identifiers can be parsed into a discrete set of enums.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #1280: PHOENIX-6523: Support for HRpc connection protocol through JDBC URL
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #1280:
URL: https://github.com/apache/phoenix/pull/1280#issuecomment-906108752
@ramatronics the build artifacts are cleaned up aggressively in a fews days, as the jenkins hosts have little storage.
You need to trigger a rebuild (or just run those checks locally).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org