You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by GitBox <gi...@apache.org> on 2020/02/27 00:11:41 UTC
[GitHub] [helix] narendly opened a new pull request #819: Make
DedicatedZkClient use HttpRoutingDataReader for routing data
narendly opened a new pull request #819: Make DedicatedZkClient use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819
### Issues
- [x] My PR addresses the following Helix issues and references them in the PR description:
Fixes #818
### Description
- [x] Here are some details about my PR, including screenshots of any UI changes:
We want all implementations of RealmAwareZkClient to do a one-time query to Metadata Store Directory Service for routing data and cache it in memory. In order to accomplish that, we have introduced HttpRoutingDataReader, which is a Singleton class that makes a REST call to read routing data and caches it in memory. This diff updates the initialization logic in DedicatedZkClient accordingly.
Changelist:
1. Update DedicatedZkClient initialization logic
2. Fix tests
### Tests
- [x] The following tests are written for this issue:
TestDedicatedZkClientFactory
- [ ] The following is the result of the "mvn test" command on the appropriate module:
(Copy & paste the result of "mvn test")
### Commits
- [x] My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Code Quality
- [x] My diff has been formatted using helix-style.xml (helix-style-intellij.xml if IntelliJ IDE is used)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm commented on a change in pull request #819: Make
RealmAwareZkClient implementatiosn use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
pkuwm commented on a change in pull request #819: Make RealmAwareZkClient implementatiosn use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r386807578
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/impl/client/FederatedZkClient.java
##########
@@ -80,19 +83,17 @@
private PathBasedZkSerializer _pathBasedZkSerializer;
// TODO: support capacity of ZkClient number in one FederatedZkClient and do garbage collection.
- public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig,
- MetadataStoreRoutingData metadataStoreRoutingData) {
- if (metadataStoreRoutingData == null) {
- throw new IllegalArgumentException("MetadataStoreRoutingData cannot be null!");
- }
+ public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig)
+ throws IOException, InvalidRoutingDataException {
Review comment:
I don't think it is a good idea to declare exception in a constructor which I would try to avoid. For this case, I would catch them and throw an `IllegalStateException`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] narendly commented on a change in pull request #819: Make
RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
narendly commented on a change in pull request #819: Make RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r387486514
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/impl/client/DedicatedZkClient.java
##########
@@ -56,32 +59,42 @@
private final ZkClient _rawZkClient;
private final MetadataStoreRoutingData _metadataStoreRoutingData;
private final String _zkRealmShardingKey;
- private final String _zkRealmAddress;
- // TODO: Remove MetadataStoreRoutingData from constructor
+ /**
+ * DedicatedZkClient connects to a single ZK realm and supports full ZkClient functionalities
+ * such as CRUD, change callback, and ephemeral operations for a single ZkRealmShardingKey.
+ * @param connectionConfig
+ * @param clientConfig
+ * @throws IOException
+ * @throws InvalidRoutingDataException
+ */
public DedicatedZkClient(RealmAwareZkClient.RealmAwareZkConnectionConfig connectionConfig,
- RealmAwareZkClient.RealmAwareZkClientConfig clientConfig,
- MetadataStoreRoutingData metadataStoreRoutingData) {
-
+ RealmAwareZkClient.RealmAwareZkClientConfig clientConfig)
+ throws IOException, InvalidRoutingDataException {
if (connectionConfig == null) {
throw new IllegalArgumentException("RealmAwareZkConnectionConfig cannot be null!");
}
- _zkRealmShardingKey = connectionConfig.getZkRealmShardingKey();
+ if (clientConfig == null) {
+ throw new IllegalArgumentException("RealmAwareZkConnectionConfig cannot be null!");
Review comment:
Good catch. Fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] kaisun2000 commented on a change in pull request #819: Make
DedicatedZkClient use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
kaisun2000 commented on a change in pull request #819: Make DedicatedZkClient use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r385445067
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/api/factory/RealmAwareZkClientFactory.java
##########
@@ -31,25 +34,21 @@
* Build a RealmAwareZkClient using specified connection config and client config.
* @param connectionConfig
* @param clientConfig
- * @param metadataStoreRoutingData
* @return HelixZkClient
Review comment:
Document when/why IOExeption would be thrown and why InvalidRoutingDataException would be thrown
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] kaisun2000 commented on a change in pull request #819: Make
DedicatedZkClient use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
kaisun2000 commented on a change in pull request #819: Make DedicatedZkClient use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r385454995
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/util/HttpRoutingDataReader.java
##########
@@ -102,7 +102,8 @@ public static MetadataStoreRoutingData getMetadataStoreRoutingData()
* @throws IOException
*/
private static String getAllRoutingData() throws IOException {
- // Note that MSDS_ENDPOINT should provide high-availability - it risks becoming a single point of failure if it's backed by a single IP address/host
+ // Note that MSDS_ENDPOINT should provide high-availability - it risks becoming a single point
+ // of failure if it's backed by a single IP address/host
Review comment:
A little off topic. Do we have a issue to track this one? If we need to enhance this part, this is the number one issue in my opinion.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] narendly commented on a change in pull request #819: Make
RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
narendly commented on a change in pull request #819: Make RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r387489615
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/impl/client/FederatedZkClient.java
##########
@@ -80,19 +83,17 @@
private PathBasedZkSerializer _pathBasedZkSerializer;
// TODO: support capacity of ZkClient number in one FederatedZkClient and do garbage collection.
- public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig,
- MetadataStoreRoutingData metadataStoreRoutingData) {
- if (metadataStoreRoutingData == null) {
- throw new IllegalArgumentException("MetadataStoreRoutingData cannot be null!");
- }
+ public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig)
+ throws IOException, InvalidRoutingDataException {
Review comment:
Here's one example where using checked exceptions is beneficial.
1) You could imagine a scenario where in the future we might have different sources from which we could obtain routing data. There is MSDS-1, MSDS-2, and MSDS-3 with different endpoints. In this case, we could add logic for trying options 1, 2, and 3 in order in case we run into IOException. Now, this logic won't apply to InvalidRoutingDataException because that implies that a data source is available but the actual routing data is invalid. This way, we could handle different exceptions differently.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] narendly commented on a change in pull request #819: Make
RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
narendly commented on a change in pull request #819: Make RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r387492149
##########
File path: zookeeper-api/src/test/java/org/apache/helix/zookeeper/constant/TestConstants.java
##########
@@ -0,0 +1,45 @@
+package org.apache.helix.zookeeper.constant;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.util.Collection;
+import java.util.Map;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+
+
+/**
+ * Constants to be used for testing.
+ */
+public class TestConstants {
+ // ZK hostname prefix and port to be used throughout the zookeeper-api module
+ public static final String ZK_PREFIX = "localhost:";
+ public static final int ZK_START_PORT = 2127;
+
+ // Based on the ZK hostname constants, construct a set of fake routing data mappings
+ public static Map<String, Collection<String>> FAKE_ROUTING_DATA = ImmutableMap
Review comment:
Added final!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm commented on a change in pull request #819: Make
RealmAwareZkClient implementatiosn use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
pkuwm commented on a change in pull request #819: Make RealmAwareZkClient implementatiosn use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r386807578
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/impl/client/FederatedZkClient.java
##########
@@ -80,19 +83,17 @@
private PathBasedZkSerializer _pathBasedZkSerializer;
// TODO: support capacity of ZkClient number in one FederatedZkClient and do garbage collection.
- public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig,
- MetadataStoreRoutingData metadataStoreRoutingData) {
- if (metadataStoreRoutingData == null) {
- throw new IllegalArgumentException("MetadataStoreRoutingData cannot be null!");
- }
+ public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig)
+ throws IOException, InvalidRoutingDataException {
Review comment:
It may not be a good idea to declare exception in a constructor which I would try to avoid. For this case, I suggest catching them and throwing an `IllegalStateException`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm commented on a change in pull request #819: Make
RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
pkuwm commented on a change in pull request #819: Make RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r387442475
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/impl/client/DedicatedZkClient.java
##########
@@ -457,17 +470,17 @@ public PathBasedZkSerializer getZkSerializer() {
* @return
*/
private void checkIfPathContainsShardingKey(String path) {
- // TODO: replace with the singleton MetadataStoreRoutingData
try {
- String zkRealmForPath = _metadataStoreRoutingData.getMetadataStoreRealm(path);
- if (!_zkRealmAddress.equals(zkRealmForPath)) {
- throw new IllegalArgumentException("Given path: " + path + "'s ZK realm: " + zkRealmForPath
- + " does not match the ZK realm: " + _zkRealmAddress + " and sharding key: "
- + _zkRealmShardingKey + " for this DedicatedZkClient!");
+ String targetShardingKey = _metadataStoreRoutingData.getShardingKeyInPath(path);
+ if (!_zkRealmShardingKey.equals(targetShardingKey)) {
Review comment:
Nit, Shall we move this condition check out of try...catch block? I understand it not may be unrelated to your change. But since you already touch it and change the name :)
One more thing: This could throw NPE. `_zkRealmShardingKey` could be null if it is not set in connectionConfig when the user expects it to be a multi-realm mode by default.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm commented on a change in pull request #819: Make
RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
pkuwm commented on a change in pull request #819: Make RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r387186961
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/impl/client/FederatedZkClient.java
##########
@@ -80,19 +83,17 @@
private PathBasedZkSerializer _pathBasedZkSerializer;
// TODO: support capacity of ZkClient number in one FederatedZkClient and do garbage collection.
- public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig,
- MetadataStoreRoutingData metadataStoreRoutingData) {
- if (metadataStoreRoutingData == null) {
- throw new IllegalArgumentException("MetadataStoreRoutingData cannot be null!");
- }
+ public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig)
+ throws IOException, InvalidRoutingDataException {
Review comment:
I understand it is OK to throw exceptions in a constructor. In this case, if we declare exceptions in the signature, would it make it inconvenient to initialize/build a FederatedZkClient with try...catch? Personally, I would not like that. If the routing data does not exist or IO exception, it means routing data is not available for this FederatedZkClient construction. A runtime exception is good enough: `IllegalStateException("Routing data is not available: " + e.getMessage())`. I wouldn't bother try...catch to construct a FederatedZkClient.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] narendly commented on a change in pull request #819: Make
RealmAwareZkClient implementatiosn use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
narendly commented on a change in pull request #819: Make RealmAwareZkClient implementatiosn use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r386847876
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/impl/client/FederatedZkClient.java
##########
@@ -80,19 +83,17 @@
private PathBasedZkSerializer _pathBasedZkSerializer;
// TODO: support capacity of ZkClient number in one FederatedZkClient and do garbage collection.
- public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig,
- MetadataStoreRoutingData metadataStoreRoutingData) {
- if (metadataStoreRoutingData == null) {
- throw new IllegalArgumentException("MetadataStoreRoutingData cannot be null!");
- }
+ public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig)
+ throws IOException, InvalidRoutingDataException {
Review comment:
Throwing exceptions in a constructor is not bad practice. In fact, it is the only way for a constructor to indicate that there is a problem; e.g. that the parameters/states are invalid.
This is a behavior agreed upon with @kaisun2000, which reminds me to add the Javadoc. IOException in the case of HTTP read failure, InvalidRoutingDataException in the case of an invalid routing data input. Users should be able to tell what's going on, and we let them distinguish failure cases by which type of exception we throw.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm commented on a change in pull request #819: Make
RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
pkuwm commented on a change in pull request #819: Make RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r387445415
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/impl/client/FederatedZkClient.java
##########
@@ -80,19 +83,27 @@
private PathBasedZkSerializer _pathBasedZkSerializer;
// TODO: support capacity of ZkClient number in one FederatedZkClient and do garbage collection.
- public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig,
- MetadataStoreRoutingData metadataStoreRoutingData) {
- if (metadataStoreRoutingData == null) {
- throw new IllegalArgumentException("MetadataStoreRoutingData cannot be null!");
+ public FederatedZkClient(RealmAwareZkClient.RealmAwareZkConnectionConfig connectionConfig,
+ RealmAwareZkClient.RealmAwareZkClientConfig clientConfig)
+ throws IOException, InvalidRoutingDataException {
+ if (connectionConfig == null) {
+ throw new IllegalArgumentException("RealmAwareZkConnectionConfig cannot be null!");
}
if (clientConfig == null) {
- throw new IllegalArgumentException("Client config cannot be null!");
+ throw new IllegalArgumentException("RealmAwareZkConnectionConfig cannot be null!");
Review comment:
Client config in message.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] narendly merged pull request #819: Make RealmAwareZkClient
implementations use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
narendly merged pull request #819: Make RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] narendly commented on a change in pull request #819: Make
RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
narendly commented on a change in pull request #819: Make RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r387489773
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/impl/client/FederatedZkClient.java
##########
@@ -80,19 +83,27 @@
private PathBasedZkSerializer _pathBasedZkSerializer;
// TODO: support capacity of ZkClient number in one FederatedZkClient and do garbage collection.
- public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig,
- MetadataStoreRoutingData metadataStoreRoutingData) {
- if (metadataStoreRoutingData == null) {
- throw new IllegalArgumentException("MetadataStoreRoutingData cannot be null!");
+ public FederatedZkClient(RealmAwareZkClient.RealmAwareZkConnectionConfig connectionConfig,
+ RealmAwareZkClient.RealmAwareZkClientConfig clientConfig)
+ throws IOException, InvalidRoutingDataException {
+ if (connectionConfig == null) {
+ throw new IllegalArgumentException("RealmAwareZkConnectionConfig cannot be null!");
}
if (clientConfig == null) {
- throw new IllegalArgumentException("Client config cannot be null!");
+ throw new IllegalArgumentException("RealmAwareZkConnectionConfig cannot be null!");
Review comment:
Fixed. thank you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm commented on a change in pull request #819: Make
RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
pkuwm commented on a change in pull request #819: Make RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r387445166
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/impl/client/FederatedZkClient.java
##########
@@ -80,19 +83,17 @@
private PathBasedZkSerializer _pathBasedZkSerializer;
// TODO: support capacity of ZkClient number in one FederatedZkClient and do garbage collection.
- public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig,
- MetadataStoreRoutingData metadataStoreRoutingData) {
- if (metadataStoreRoutingData == null) {
- throw new IllegalArgumentException("MetadataStoreRoutingData cannot be null!");
- }
+ public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig)
+ throws IOException, InvalidRoutingDataException {
Review comment:
Like I said, personally, hiding failure type in this case is more user-friendly than floating these exceptions up. And I also understand and respect your opinion.
I am not sure how Helix Java APIs could handle these 2 exceptions differently when constructing this FederatedZkClient. Regarding a few other users, we could get opinions from them. I will be OK if we can get options from other users/committers and they think floating exceptions are better :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] narendly commented on a change in pull request #819: Make
DedicatedZkClient use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
narendly commented on a change in pull request #819: Make DedicatedZkClient use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r385472899
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/impl/factory/SharedZkClientFactory.java
##########
@@ -45,21 +45,12 @@ protected SharedZkClientFactory() {
@Override
public RealmAwareZkClient buildZkClient(
RealmAwareZkClient.RealmAwareZkConnectionConfig connectionConfig,
- RealmAwareZkClient.RealmAwareZkClientConfig clientConfig,
- MetadataStoreRoutingData metadataStoreRoutingData) {
+ RealmAwareZkClient.RealmAwareZkClientConfig clientConfig) {
Review comment:
No worries.
I won't merge this in until other ZkClients have been merged in. I might include other ZkClients in the scope of this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] narendly commented on a change in pull request #819: Make
DedicatedZkClient use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
narendly commented on a change in pull request #819: Make DedicatedZkClient use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r385472743
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/api/factory/RealmAwareZkClientFactory.java
##########
@@ -31,25 +34,21 @@
* Build a RealmAwareZkClient using specified connection config and client config.
* @param connectionConfig
* @param clientConfig
- * @param metadataStoreRoutingData
* @return HelixZkClient
Review comment:
That is correct. I will update the JavaDoc :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] narendly commented on a change in pull request #819: Make
RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
narendly commented on a change in pull request #819: Make RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r387219455
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/impl/client/FederatedZkClient.java
##########
@@ -80,19 +83,17 @@
private PathBasedZkSerializer _pathBasedZkSerializer;
// TODO: support capacity of ZkClient number in one FederatedZkClient and do garbage collection.
- public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig,
- MetadataStoreRoutingData metadataStoreRoutingData) {
- if (metadataStoreRoutingData == null) {
- throw new IllegalArgumentException("MetadataStoreRoutingData cannot be null!");
- }
+ public FederatedZkClient(RealmAwareZkClient.RealmAwareZkClientConfig clientConfig)
+ throws IOException, InvalidRoutingDataException {
Review comment:
- FederatedZkClient is still a low-level construct
- This means that the users of this component will mostly be Helix Java APIs, and probably a few users who wish to use directly.
- In that case, I think we still need to float these exceptions up and handle each case at the Helix Java API level. That way, we make it 1) user-friendly and 2) we still achieve the low-level granularity with FederatedZkClient that users of this class would want instead of hiding failure types behind an IllegalStateException.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] narendly commented on issue #819: Make RealmAwareZkClient
implementations use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
narendly commented on issue #819: Make RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#issuecomment-594378577
This PR is ready to be merged, approved by @pkuwm
We want all implementations of RealmAwareZkClient to do a one-time query to Metadata Store Directory Service for routing data and cache it in memory. In order to accomplish that, we have introduced HttpRoutingDataReader, which is a Singleton class that makes a REST call to read routing data and caches it in memory. This diff updates the initialization logic in RealmAwareZkClients accordingly.
Changelist:
Update all RealmAwareZkClient initialization logic
Fix tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] kaisun2000 commented on issue #819: Make DedicatedZkClient
use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
kaisun2000 commented on issue #819: Make DedicatedZkClient use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#issuecomment-592255363
Overall looks good to me, just some nits.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] kaisun2000 commented on a change in pull request #819: Make
DedicatedZkClient use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
kaisun2000 commented on a change in pull request #819: Make DedicatedZkClient use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r385454565
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/impl/factory/SharedZkClientFactory.java
##########
@@ -45,21 +45,12 @@ protected SharedZkClientFactory() {
@Override
public RealmAwareZkClient buildZkClient(
RealmAwareZkClient.RealmAwareZkConnectionConfig connectionConfig,
- RealmAwareZkClient.RealmAwareZkClientConfig clientConfig,
- MetadataStoreRoutingData metadataStoreRoutingData) {
+ RealmAwareZkClient.RealmAwareZkClientConfig clientConfig) {
Review comment:
This would be a conflict. Now my change is in, we need a rebase for this one.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm commented on a change in pull request #819: Make
RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
pkuwm commented on a change in pull request #819: Make RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r387446002
##########
File path: zookeeper-api/src/test/java/org/apache/helix/zookeeper/constant/TestConstants.java
##########
@@ -0,0 +1,45 @@
+package org.apache.helix.zookeeper.constant;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.util.Collection;
+import java.util.Map;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+
+
+/**
+ * Constants to be used for testing.
+ */
+public class TestConstants {
+ // ZK hostname prefix and port to be used throughout the zookeeper-api module
+ public static final String ZK_PREFIX = "localhost:";
+ public static final int ZK_START_PORT = 2127;
+
+ // Based on the ZK hostname constants, construct a set of fake routing data mappings
+ public static Map<String, Collection<String>> FAKE_ROUTING_DATA = ImmutableMap
Review comment:
static final?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] kaisun2000 commented on a change in pull request #819: Make
DedicatedZkClient use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
kaisun2000 commented on a change in pull request #819: Make DedicatedZkClient use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r385445067
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/api/factory/RealmAwareZkClientFactory.java
##########
@@ -31,25 +34,21 @@
* Build a RealmAwareZkClient using specified connection config and client config.
* @param connectionConfig
* @param clientConfig
- * @param metadataStoreRoutingData
* @return HelixZkClient
Review comment:
Document when/why IOExeption would be thrown and why InvalidRoutingDataException would be thrown.
Seem from HttpRoutingDataReader code, if we can't get data/parse data from http respone, it would be IO exception, if constructing the trie based on parse http reponse failed, it is InvalidRoutingDataException.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] narendly commented on a change in pull request #819: Make
DedicatedZkClient use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
narendly commented on a change in pull request #819: Make DedicatedZkClient use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r385473057
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/util/HttpRoutingDataReader.java
##########
@@ -102,7 +102,8 @@ public static MetadataStoreRoutingData getMetadataStoreRoutingData()
* @throws IOException
*/
private static String getAllRoutingData() throws IOException {
- // Note that MSDS_ENDPOINT should provide high-availability - it risks becoming a single point of failure if it's backed by a single IP address/host
+ // Note that MSDS_ENDPOINT should provide high-availability - it risks becoming a single point
+ // of failure if it's backed by a single IP address/host
Review comment:
We do not - could you please create an issue for it? Not an immediate blocker.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] narendly commented on a change in pull request #819: Make
RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
narendly commented on a change in pull request #819: Make RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r387488129
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/impl/client/DedicatedZkClient.java
##########
@@ -457,17 +470,17 @@ public PathBasedZkSerializer getZkSerializer() {
* @return
*/
private void checkIfPathContainsShardingKey(String path) {
- // TODO: replace with the singleton MetadataStoreRoutingData
try {
- String zkRealmForPath = _metadataStoreRoutingData.getMetadataStoreRealm(path);
- if (!_zkRealmAddress.equals(zkRealmForPath)) {
- throw new IllegalArgumentException("Given path: " + path + "'s ZK realm: " + zkRealmForPath
- + " does not match the ZK realm: " + _zkRealmAddress + " and sharding key: "
- + _zkRealmShardingKey + " for this DedicatedZkClient!");
+ String targetShardingKey = _metadataStoreRoutingData.getShardingKeyInPath(path);
+ if (!_zkRealmShardingKey.equals(targetShardingKey)) {
Review comment:
I am glad that you brought this up. In fact, DedicatedZkClient and SharedZkClient are single-realm APIs, so we need to check in the constructor that Zk Sharding key is not null or empty.
I will add this check. Then we don't need to worry about NPEs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] kaisun2000 commented on a change in pull request #819: Make
DedicatedZkClient use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
kaisun2000 commented on a change in pull request #819: Make DedicatedZkClient use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r385454995
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/util/HttpRoutingDataReader.java
##########
@@ -102,7 +102,8 @@ public static MetadataStoreRoutingData getMetadataStoreRoutingData()
* @throws IOException
*/
private static String getAllRoutingData() throws IOException {
- // Note that MSDS_ENDPOINT should provide high-availability - it risks becoming a single point of failure if it's backed by a single IP address/host
+ // Note that MSDS_ENDPOINT should provide high-availability - it risks becoming a single point
+ // of failure if it's backed by a single IP address/host
Review comment:
A little of topic. Do we have a issue to track this one? If we need to enhance this part, this is the number one issue in my opinion.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm commented on a change in pull request #819: Make
RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
Posted by GitBox <gi...@apache.org>.
pkuwm commented on a change in pull request #819: Make RealmAwareZkClient implementations use HttpRoutingDataReader for routing data
URL: https://github.com/apache/helix/pull/819#discussion_r387441969
##########
File path: zookeeper-api/src/main/java/org/apache/helix/zookeeper/impl/client/DedicatedZkClient.java
##########
@@ -56,32 +59,42 @@
private final ZkClient _rawZkClient;
private final MetadataStoreRoutingData _metadataStoreRoutingData;
private final String _zkRealmShardingKey;
- private final String _zkRealmAddress;
- // TODO: Remove MetadataStoreRoutingData from constructor
+ /**
+ * DedicatedZkClient connects to a single ZK realm and supports full ZkClient functionalities
+ * such as CRUD, change callback, and ephemeral operations for a single ZkRealmShardingKey.
+ * @param connectionConfig
+ * @param clientConfig
+ * @throws IOException
+ * @throws InvalidRoutingDataException
+ */
public DedicatedZkClient(RealmAwareZkClient.RealmAwareZkConnectionConfig connectionConfig,
- RealmAwareZkClient.RealmAwareZkClientConfig clientConfig,
- MetadataStoreRoutingData metadataStoreRoutingData) {
-
+ RealmAwareZkClient.RealmAwareZkClientConfig clientConfig)
+ throws IOException, InvalidRoutingDataException {
if (connectionConfig == null) {
throw new IllegalArgumentException("RealmAwareZkConnectionConfig cannot be null!");
}
- _zkRealmShardingKey = connectionConfig.getZkRealmShardingKey();
+ if (clientConfig == null) {
+ throw new IllegalArgumentException("RealmAwareZkConnectionConfig cannot be null!");
Review comment:
Typo in exception message. Should be RealmAwareZkClientConfig
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org