You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by "xyuanlu (via GitHub)" <gi...@apache.org> on 2024/07/19 06:45:45 UTC
[PR] Gateway service - service structure dummy class [helix]
xyuanlu opened a new pull request, #2840:
URL: https://github.com/apache/helix/pull/2840
### Issues
- [ X My PR addresses the following Helix issues and references them in the PR description:
resolves https://github.com/apache/helix/issues/2821
### Description
- [X] Here are some details about my PR, including screenshots of any UI changes:
This change continuous add gateway service.
### Tests
- [X] The following tests are written for this issue:
NA
- The following is the result of the "mvn test" command on the appropriate module:
(If CI test fails due to known issue, please specify the issue and test PR locally. Then copy & paste the result of "mvn test" to here.)
### Changes that Break Backward Compatibility (Optional)
- My PR contains changes that break backward compatibility or previous assumptions for certain methods or API. They include:
(Consider including all behavior changes for public methods or API. Also include these changes in merge description so that other developers are aware of these changes. This allows them to make relevant code changes in feature branches accounting for the new method/API behavior.)
### Documentation (Optional)
- In case of new functionality, my PR adds documentation in the following wiki page:
(Link the GitHub wiki you added)
### Commits
- My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Code Quality
- My diff has been formatted using helix-style.xml
(helix-style-intellij.xml if IntelliJ IDE is used)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
Re: [PR] Gateway service - service structure dummy class [helix]
Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on PR #2840:
URL: https://github.com/apache/helix/pull/2840#issuecomment-2244035610
This PR is ready to be merged, approved by @junkaixue .
Commit message: [Gateway service - service structure dummy class]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
Re: [PR] Gateway service - service structure dummy class [helix]
Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu merged PR #2840:
URL: https://github.com/apache/helix/pull/2840
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
Re: [PR] Gateway service - service structure dummy class [helix]
Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on code in PR #2840:
URL: https://github.com/apache/helix/pull/2840#discussion_r1687196832
##########
helix-gateway/src/main/java/org/apache/helix/gateway/grpcservice/HelixGatewayServiceService.java:
##########
@@ -16,6 +27,13 @@ public StreamObserver<proto.org.apache.helix.gateway.HelixGatewayServiceOuterCla
public void onNext(ShardStateMessage request) {
// called when a client sends a message
//....
+ String instanceName = request.getInstanceName();
+ if (_observerMap.containsValue(instanceName)) {
+ // this is from an existing client
+ } else {
+ updateObserver(instanceName, responseObserver);
+ // update state map
+ }
Review Comment:
TFTR updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
Re: [PR] Gateway service - service structure dummy class [helix]
Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on code in PR #2840:
URL: https://github.com/apache/helix/pull/2840#discussion_r1687275472
##########
helix-gateway/src/main/java/org/apache/helix/gateway/service/HelixGatewayServiceProcessor.java:
##########
@@ -0,0 +1,11 @@
+package org.apache.helix.gateway.service;
+
+/**
+ * Translate from/to GRPC function call to Helix Gateway Service event.
+ */
+public interface HelixGatewayServiceProcessor {
+
+ public boolean sendStateTransitionMessage(String instanceName);
+
+ public void pushEventToManager(GatewayServiceManager.GateWayServiceEvent event);
Review Comment:
TFTR. Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
Re: [PR] Gateway service - service structure dummy class [helix]
Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on code in PR #2840:
URL: https://github.com/apache/helix/pull/2840#discussion_r1687132135
##########
helix-gateway/src/main/java/org/apache/helix/gateway/util/StateTransitionMessageTranslator.java:
##########
@@ -0,0 +1,8 @@
+package org.apache.helix.gateway.util;
+
+public class StateTransitionMessageTranslator {
+
+ public static void translateSTMsgToProto() {
Review Comment:
Per offline sync, Updated the naming.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
Re: [PR] Gateway service - service structure dummy class [helix]
Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on code in PR #2840:
URL: https://github.com/apache/helix/pull/2840#discussion_r1684837817
##########
helix-gateway/src/main/java/org/apache/helix/gateway/util/StateTransitionMessageTranslator.java:
##########
@@ -0,0 +1,8 @@
+package org.apache.helix.gateway.util;
+
+public class StateTransitionMessageTranslator {
+
+ public static void translateSTMsgToProto() {
Review Comment:
I am thinking it should a util class with abstract functions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
Re: [PR] Gateway service - service structure dummy class [helix]
Posted by "junkaixue (via GitHub)" <gi...@apache.org>.
junkaixue commented on code in PR #2840:
URL: https://github.com/apache/helix/pull/2840#discussion_r1687268505
##########
helix-gateway/src/main/java/org/apache/helix/gateway/service/HelixGatewayServiceProcessor.java:
##########
@@ -0,0 +1,11 @@
+package org.apache.helix.gateway.service;
+
+/**
+ * Translate from/to GRPC function call to Helix Gateway Service event.
+ */
+public interface HelixGatewayServiceProcessor {
+
+ public boolean sendStateTransitionMessage(String instanceName);
+
+ public void pushEventToManager(GatewayServiceManager.GateWayServiceEvent event);
Review Comment:
Let's be consistent with naming?
send or push.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
Re: [PR] Gateway service - service structure dummy class [helix]
Posted by "junkaixue (via GitHub)" <gi...@apache.org>.
junkaixue commented on code in PR #2840:
URL: https://github.com/apache/helix/pull/2840#discussion_r1687146859
##########
helix-gateway/src/main/java/org/apache/helix/gateway/service/GatewayServiceManager.java:
##########
@@ -0,0 +1,65 @@
+package org.apache.helix.gateway.service;
+
+import java.util.Map;
+
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import org.apache.helix.gateway.grpcservice.HelixGatewayServiceService;
+
+
+/**
+ * A top layer class that send/receive messages from Grpc end point, and dispatch them to corrsponding gateway services.
+ * 1. get event from Grpc service
+ * 2. Maintain a gateway service registry, one gateway service maps to one Helix cluster
+ * 3. On init connect, create the participant manager
+ * 4. For ST reply message, update the tracker
+ */
+
+public class GatewayServiceManager {
+
+ HelixGatewayServiceService _helixGatewayServiceService;
+
+ HelixGatewayServiceProcessor _helixGatewayServiceProcessor;
+
+ Map<String, HelixGatewayService> _helixGatewayServiceMap;
+
+ // TODO: add thread pool for init
+ // single thread tp for update
+
+ public enum EventType {
+ INIT, // init connection to gateway service
+ UPDATE, // update state transition result
+ SHUTDOWN // shutdown connection to gateway service.
Review Comment:
how about use:
connect,
disconnect,
update.
Because sometimes lost connection cannot say it as shutdown but disconnect.
##########
helix-gateway/src/main/java/org/apache/helix/gateway/util/StateTransitionMessageTranslateUtil.java:
##########
@@ -0,0 +1,8 @@
+package org.apache.helix.gateway.util;
+
+public class StateTransitionMessageTranslateUtil {
+
+ public static void translateSTMsgToProto() {
Review Comment:
Should return proto object?
##########
helix-gateway/src/main/java/org/apache/helix/gateway/util/StateTransitionMessageTranslateUtil.java:
##########
@@ -0,0 +1,8 @@
+package org.apache.helix.gateway.util;
+
+public class StateTransitionMessageTranslateUtil {
Review Comment:
make it final
##########
helix-gateway/src/main/java/org/apache/helix/gateway/grpcservice/HelixGatewayServiceService.java:
##########
@@ -16,6 +27,13 @@ public StreamObserver<proto.org.apache.helix.gateway.HelixGatewayServiceOuterCla
public void onNext(ShardStateMessage request) {
// called when a client sends a message
//....
+ String instanceName = request.getInstanceName();
+ if (_observerMap.containsValue(instanceName)) {
+ // this is from an existing client
+ } else {
+ updateObserver(instanceName, responseObserver);
+ // update state map
+ }
Review Comment:
Let's do this:
if (! contains) {
update
}
do regular thing.
##########
helix-gateway/src/main/java/org/apache/helix/gateway/util/StateTransitionMessageTranslateUtil.java:
##########
@@ -0,0 +1,8 @@
+package org.apache.helix.gateway.util;
+
+public class StateTransitionMessageTranslateUtil {
+
+ public static void translateSTMsgToProto() {
Review Comment:
What about proto to ST?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
Re: [PR] Gateway service - service structure dummy class [helix]
Posted by "junkaixue (via GitHub)" <gi...@apache.org>.
junkaixue commented on code in PR #2840:
URL: https://github.com/apache/helix/pull/2840#discussion_r1684703183
##########
helix-gateway/src/main/proto/HelixGatewayService.proto:
##########
@@ -26,26 +26,25 @@ message SingleShardTransitionStatus {
optional string currentState = 3; // If it failed, what is the current state it should reported as.
}
-// resource has list of replica
-
message SingleResourceState {
- string resource = 1;
- repeated SingleShardState SingleReplicaState= 2;
+ string resource = 1; // name of the resource
+ repeated SingleShardState SingleReplicaState = 2; // State of each shard
Review Comment:
shard not replica
##########
helix-gateway/src/main/proto/HelixGatewayService.proto:
##########
@@ -26,26 +26,25 @@ message SingleShardTransitionStatus {
optional string currentState = 3; // If it failed, what is the current state it should reported as.
}
-// resource has list of replica
-
message SingleResourceState {
- string resource = 1;
- repeated SingleShardState SingleReplicaState= 2;
+ string resource = 1; // name of the resource
+ repeated SingleShardState SingleReplicaState = 2; // State of each shard
}
message SingleShardState {
- string shardaName = 1;
- string currentState = 2;
+ string shardaName = 1; // Name of the shard
Review Comment:
typo "sharda"
##########
helix-gateway/src/main/java/org/apache/helix/gateway/util/StateTransitionMessageTranslator.java:
##########
@@ -0,0 +1,8 @@
+package org.apache.helix.gateway.util;
+
+public class StateTransitionMessageTranslator {
+
+ public static void translateSTMsgToProto() {
Review Comment:
I would assume this should be interface instead of concrete class.
interface I am expecting was:
fromHelixToProto
fromProtoToHelix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
Re: [PR] Gateway service - service structure dummy class [helix]
Posted by "xyuanlu (via GitHub)" <gi...@apache.org>.
xyuanlu commented on code in PR #2840:
URL: https://github.com/apache/helix/pull/2840#discussion_r1684792125
##########
helix-gateway/src/main/proto/HelixGatewayService.proto:
##########
@@ -26,26 +26,25 @@ message SingleShardTransitionStatus {
optional string currentState = 3; // If it failed, what is the current state it should reported as.
}
-// resource has list of replica
-
message SingleResourceState {
- string resource = 1;
- repeated SingleShardState SingleReplicaState= 2;
+ string resource = 1; // name of the resource
+ repeated SingleShardState SingleReplicaState = 2; // State of each shard
Review Comment:
TFTR. Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org