You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hugegraph.apache.org by GitBox <gi...@apache.org> on 2022/05/15 14:06:07 UTC
[GitHub] [incubator-hugegraph] Linary opened a new pull request, #1527: Improve some action for install snapshot and add peer
Linary opened a new pull request, #1527:
URL: https://github.com/apache/incubator-hugegraph/pull/1527
1. Fix bug that relative path need normalize when load snapshot
2. Use zip decompresser since decompress tar file may lead dead loop
3. Move raft.endpoint and raft.group_peers into rest-server.properties
4. Pass raft.endpoint and raft.group_peers from server to core
5. Let RestServer start before GremlinServer
6. Use read-index to ensure the raft log caught up for new node
7. Let add_peer and remove_peer API work in async job
8. Let truncateBackend don't truncate system store
Change-Id: I7de3d98cb51c3b7c32a4f19a0d99ed622f78d331
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hugegraph] javeme merged pull request #1527: Improve some action for install snapshot and add peer
Posted by GitBox <gi...@apache.org>.
javeme merged PR #1527:
URL: https://github.com/apache/incubator-hugegraph/pull/1527
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hugegraph] Linary commented on pull request #1527: Improve some action for install snapshot and add peer
Posted by GitBox <gi...@apache.org>.
Linary commented on PR #1527:
URL: https://github.com/apache/incubator-hugegraph/pull/1527#issuecomment-1127361414
+2, Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hugegraph] javeme commented on pull request #1527: Improve some action for install snapshot and add peer
Posted by GitBox <gi...@apache.org>.
javeme commented on PR #1527:
URL: https://github.com/apache/incubator-hugegraph/pull/1527#issuecomment-1127187402
ci error:
```java
2022-05-15 19:04:18 [main] [INFO] c.b.h.b.s.BackendProviderFactory - Opening backend store 'rocksdb' in raft mode for graph 'hugegraph'
2022-05-15 19:04:18 [main] [INFO] c.b.h.b.s.r.RaftBackendStoreProvider - Init raft backend system store
Error: -15 19:04:18 [main] [ERROR] c.b.h.d.HugeGraphServer - HugeRestServer start error:
java.lang.RuntimeException: GraphFactory could not instantiate this Graph implementation [class com.baidu.hugegraph.auth.HugeFactoryAuthProxy]
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:84) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:72) ~[gremlin-core-3.5.1.jar:3.5.1]
at com.baidu.hugegraph.auth.StandardAuthenticator.setup(StandardAuthenticator.java:121) ~[hugegraph-api-0.13.0.jar:0.69.0.0]
at com.baidu.hugegraph.auth.HugeAuthenticator.loadAuthenticator(HugeAuthenticator.java:148) ~[hugegraph-api-0.13.0.jar:0.69.0.0]
at com.baidu.hugegraph.core.GraphManager.<init>(GraphManager.java:96) ~[hugegraph-api-0.13.0.jar:0.69.0.0]
at com.baidu.hugegraph.server.ApplicationConfig$GraphManagerFactory$1.onEvent(ApplicationConfig.java:112) ~[hugegraph-api-0.13.0.jar:0.69.0.0]
at org.glassfish.jersey.server.internal.monitoring.CompositeApplicationEventListener.onEvent(CompositeApplicationEventListener.java:49) ~[jersey-server-3.0.3.jar:?]
at org.glassfish.jersey.server.internal.monitoring.MonitoringContainerListener.onStartup(MonitoringContainerListener.java:56) ~[jersey-server-3.0.3.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.onStartup(ApplicationHandler.java:711) ~[jersey-server-3.0.3.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.start(GrizzlyHttpContainer.java:330) ~[jersey-container-grizzly2-http-3.0.3.jar:?]
at org.glassfish.grizzly.http.server.HttpHandlerChain.start(HttpHandlerChain.java:376) ~[grizzly-http-server-3.0.1.jar:3.0.1]
at org.glassfish.grizzly.http.server.HttpServer.setupHttpHandler(HttpServer.java:268) ~[grizzly-http-server-3.0.1.jar:3.0.1]
at org.glassfish.grizzly.http.server.HttpServer.start(HttpServer.java:245) ~[grizzly-http-server-3.0.1.jar:3.0.1]
at com.baidu.hugegraph.server.RestServer.start(RestServer.java:68) ~[hugegraph-api-0.13.0.jar:0.69.0.0]
at com.baidu.hugegraph.server.RestServer.start(RestServer.java:175) ~[hugegraph-api-0.13.0.jar:0.69.0.0]
at com.baidu.hugegraph.dist.HugeRestServer.start(HugeRestServer.java:34) ~[hugegraph-dist-0.13.0.jar:?]
at com.baidu.hugegraph.dist.HugeGraphServer.<init>(HugeGraphServer.java:62) ~[hugegraph-dist-0.13.0.jar:?]
at com.baidu.hugegraph.dist.HugeGraphServer.main(HugeGraphServer.java:121) ~[hugegraph-dist-0.13.0.jar:?]
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_332]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_332]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_332]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_332]
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:80) ~[gremlin-core-3.5.1.jar:3.5.1]
... 17 more
Caused by: java.lang.IllegalStateException: Please ensure init raft context
at com.google.common.base.Preconditions.checkState(Preconditions.java:531) ~[guava-25.1-jre.jar:?]
at com.baidu.hugegraph.util.E.checkState(E.java:68) ~[hugegraph-common-2.1.2.jar:2.1.2.0]
at com.baidu.hugegraph.backend.store.raft.RaftBackendStoreProvider.context(RaftBackendStoreProvider.java:74) ~[hugegraph-core-0.13.0.jar:0.13.0.0]
at com.baidu.hugegraph.backend.store.raft.RaftBackendStoreProvider.loadSystemStore(RaftBackendStoreProvider.java:143) ~[hugegraph-core-0.13.0.jar:0.13.0.0]
at com.baidu.hugegraph.StandardHugeGraph.loadSystemStore(StandardHugeGraph.java:466) ~[hugegraph-core-0.13.0.jar:0.13.0.0]
at com.baidu.hugegraph.StandardHugeGraph.access$1300(StandardHugeGraph.java:113) ~[hugegraph-core-0.13.0.jar:0.13.0.0]
at com.baidu.hugegraph.StandardHugeGraph$StandardHugeGraphParams.loadSystemStore(StandardHugeGraph.java:1185) ~[hugegraph-core-0.13.0.jar:0.13.0.0]
at com.baidu.hugegraph.task.ServerInfoManager.listenChanges(ServerInfoManager.java:109) ~[hugegraph-core-0.13.0.jar:0.13.0.0]
at com.baidu.hugegraph.task.ServerInfoManager.<init>(ServerInfoManager.java:85) ~[hugegraph-core-0.13.0.jar:0.13.0.0]
at com.baidu.hugegraph.task.StandardTaskScheduler.<init>(StandardTaskScheduler.java:106) ~[hugegraph-core-0.13.0.jar:0.13.0.0]
at com.baidu.hugegraph.task.TaskManager.addScheduler(TaskManager.java:92) ~[hugegraph-core-0.13.0.jar:0.13.0.0]
at com.baidu.hugegraph.StandardHugeGraph.<init>(StandardHugeGraph.java:218) ~[hugegraph-core-0.13.0.jar:0.13.0.0]
at com.baidu.hugegraph.HugeFactory.open(HugeFactory.java:91) ~[hugegraph-core-0.13.0.jar:0.13.0.0]
at com.baidu.hugegraph.HugeFactory.open(HugeFactory.java:69) ~[hugegraph-core-0.13.0.jar:0.13.0.0]
at com.baidu.hugegraph.auth.HugeFactoryAuthProxy.open(HugeFactoryAuthProxy.java:83) ~[hugegraph-api-0.13.0.jar:0.69.0.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_332]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_332]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_332]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_332]
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:80) ~[gremlin-core-3.5.1.jar:3.5.1]
... 17 more
The operation timed out(60s) when attempting to connect to http://127.0.0.1:8082/graphs
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hugegraph] javeme commented on a diff in pull request #1527: Improve some action for install snapshot and add peer
Posted by GitBox <gi...@apache.org>.
javeme commented on code in PR #1527:
URL: https://github.com/apache/incubator-hugegraph/pull/1527#discussion_r873147362
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/config/CoreOptions.java:
##########
@@ -252,10 +228,19 @@ public static synchronized CoreOptions instance() {
public static final ConfigOption<Integer> RAFT_RPC_TIMEOUT =
new ConfigOption<>(
"raft.rpc_timeout",
- "The rpc timeout for jraft rpc.",
+ "The general rpc timeout for jraft rpc.",
positiveInt(),
// jraft default value is 5000(ms)
- 60000
+ 60 * 1000
+ );
+
+ public static final ConfigOption<Integer> RAFT_INSTALL_SNAPSHOT_TIMEOUT =
+ new ConfigOption<>(
+ "raft.install_snapshot_rpc_timeout",
+ "The install snapshot rpc timeout for jraft rpc.",
+ positiveInt(),
+ // jraft default value is 5 * 60 * 1000
Review Comment:
5 minutes?
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/rpc/RpcForwarder.java:
##########
@@ -19,7 +19,7 @@
package com.baidu.hugegraph.backend.store.raft.rpc;
-import static com.baidu.hugegraph.backend.store.raft.RaftSharedContext.WAIT_RPC_TIMEOUT;
+import static com.baidu.hugegraph.backend.store.raft.RaftContext.WAIT_RPC_TIMEOUT;
Review Comment:
avoid static import
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftNode.java:
##########
@@ -98,39 +104,52 @@ public void shutdown() {
this.node.shutdown();
}
- public void snapshot() {
- if (!this.context.useSnapshot()) {
- return;
- }
+ public RaftClosure<?> snapshot() {
RaftClosure<?> future = new RaftClosure<>();
try {
this.node().snapshot(future);
- future.waitFinished();
+ return future;
} catch (Throwable e) {
throw new BackendException("Failed to create snapshot", e);
}
}
- private Node initRaftNode() throws IOException {
+ private RaftGroupService initRaftNode() throws IOException {
NodeOptions nodeOptions = this.context.nodeOptions();
nodeOptions.setFsm(this.stateMachine);
- // TODO: When support sharding, groupId needs to be bound to shard Id
+ /*
+ * TODO: the groupId is same as graph name now, when support sharding,
+ * groupId needs to be bound to shard Id
+ */
String groupId = this.context.group();
PeerId endpoint = this.context.endpoint();
/*
* Start raft node with shared rpc server:
- * return new RaftGroupService(groupId, endpoint, nodeOptions,
- * this.context.rpcServer(), true)
- * .start(false)
*/
- return RaftServiceFactory.createAndInitRaftNode(groupId, endpoint,
- nodeOptions);
+ RpcServer rpcServer = this.context.rpcServer();
+ LOG.info("The raft endpoint '{}', initial group peers [{}]",
+ endpoint, nodeOptions.getInitialConf());
+ // Shared rpc server
+ return new RaftGroupService(groupId, endpoint, nodeOptions,
+ rpcServer, true);
+ }
+
+ public void close() {
+ if (this.raftGroupService != null) {
+ this.raftGroupService.shutdown();
+ try {
+ this.raftGroupService.join();
+ } catch (final InterruptedException e) {
+ throw new RaftException("Interrupted while shutdown " +
+ "raftGroupService");
Review Comment:
align
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftContext.java:
##########
@@ -389,7 +392,6 @@ private void registerRpcRequestProcessors() {
this.rpcServer.registerProcessor(new SetLeaderProcessor(this));
this.rpcServer.registerProcessor(new ListPeersProcessor(this));
}
-
Review Comment:
ditto
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftContext.java:
##########
@@ -104,26 +106,41 @@ public final class RaftSharedContext {
private RaftGroupManager raftGroupManager;
private RpcForwarder rpcForwarder;
- public RaftSharedContext(HugeGraphParams params) {
+ public RaftContext(HugeGraphParams params, RpcServer rpcServer,
+ PeerId endpoint) {
this.params = params;
- HugeConfig config = this.config();
+ this.rpcServer = rpcServer;
+ this.endpoint = endpoint;
+ HugeConfig config = params.configuration();
this.schemaStoreName = config.get(CoreOptions.STORE_SCHEMA);
this.graphStoreName = config.get(CoreOptions.STORE_GRAPH);
this.systemStoreName = config.get(CoreOptions.STORE_SYSTEM);
this.stores = new RaftBackendStore[StoreType.ALL.getNumber()];
- this.rpcServer = this.initAndStartRpcServer();
+
+ /*
+ * TODO Depending on the name of the config item for server options,
+ * need to get through ServerConfig and CoreConfig
Review Comment:
raft.group_peers is transfered from ServerConfig instead of CoreConfig, since it's shared by all graphs.
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftContext.java:
##########
@@ -104,26 +106,41 @@ public final class RaftSharedContext {
private RaftGroupManager raftGroupManager;
private RpcForwarder rpcForwarder;
- public RaftSharedContext(HugeGraphParams params) {
+ public RaftContext(HugeGraphParams params, RpcServer rpcServer,
+ PeerId endpoint) {
this.params = params;
- HugeConfig config = this.config();
+ this.rpcServer = rpcServer;
+ this.endpoint = endpoint;
+ HugeConfig config = params.configuration();
this.schemaStoreName = config.get(CoreOptions.STORE_SCHEMA);
this.graphStoreName = config.get(CoreOptions.STORE_GRAPH);
this.systemStoreName = config.get(CoreOptions.STORE_SYSTEM);
this.stores = new RaftBackendStore[StoreType.ALL.getNumber()];
- this.rpcServer = this.initAndStartRpcServer();
+
+ /*
+ * TODO Depending on the name of the config item for server options,
+ * need to get through ServerConfig and CoreConfig
+ */
+ // this.endpoint = new PeerId();
+ // String endpointStr = config.getString("raft.endpoint");
+ // if (!this.endpoint.parse(endpointStr)) {
+ // throw new HugeException("Failed to parse endpoint %s", endpointStr);
+ // }
+ this.groupPeers = new Configuration();
+ String groupPeersStr = config.getString("raft.group_peers");
+ if (!this.groupPeers.parse(groupPeersStr)) {
+ throw new HugeException("Failed to parse group peers %s",
+ groupPeersStr);
+ }
+
if (config.get(CoreOptions.RAFT_SAFE_READ)) {
int threads = config.get(CoreOptions.RAFT_READ_INDEX_THREADS);
this.readIndexExecutor = this.createReadIndexExecutor(threads);
} else {
this.readIndexExecutor = null;
}
- if (config.get(CoreOptions.RAFT_USE_SNAPSHOT)) {
- this.snapshotExecutor = this.createSnapshotExecutor(4);
- } else {
- this.snapshotExecutor = null;
- }
+ this.snapshotExecutor = this.createSnapshotExecutor(4);
Review Comment:
define a const var
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftNode.java:
##########
@@ -45,18 +48,21 @@ public final class RaftNode {
private static final Logger LOG = Log.logger(RaftNode.class);
- private final RaftSharedContext context;
+ private final RaftContext context;
+ private final RaftGroupService raftGroupService;
private final Node node;
private final StoreStateMachine stateMachine;
private final AtomicReference<LeaderInfo> leaderInfo;
private final AtomicBoolean started;
private final AtomicInteger busyCounter;
- public RaftNode(RaftSharedContext context) {
+ public RaftNode(RaftContext context) {
this.context = context;
this.stateMachine = new StoreStateMachine(context);
try {
- this.node = this.initRaftNode();
+ this.raftGroupService = this.initRaftNode();
+ // Start node
+ this.node = this.raftGroupService.start(false);
Review Comment:
do need to hold raftGroupService?
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftNode.java:
##########
@@ -98,39 +104,52 @@ public void shutdown() {
this.node.shutdown();
}
- public void snapshot() {
- if (!this.context.useSnapshot()) {
- return;
- }
+ public RaftClosure<?> snapshot() {
RaftClosure<?> future = new RaftClosure<>();
try {
this.node().snapshot(future);
- future.waitFinished();
+ return future;
} catch (Throwable e) {
throw new BackendException("Failed to create snapshot", e);
}
}
- private Node initRaftNode() throws IOException {
+ private RaftGroupService initRaftNode() throws IOException {
NodeOptions nodeOptions = this.context.nodeOptions();
nodeOptions.setFsm(this.stateMachine);
- // TODO: When support sharding, groupId needs to be bound to shard Id
+ /*
+ * TODO: the groupId is same as graph name now, when support sharding,
+ * groupId needs to be bound to shard Id
+ */
String groupId = this.context.group();
PeerId endpoint = this.context.endpoint();
/*
* Start raft node with shared rpc server:
- * return new RaftGroupService(groupId, endpoint, nodeOptions,
- * this.context.rpcServer(), true)
- * .start(false)
*/
- return RaftServiceFactory.createAndInitRaftNode(groupId, endpoint,
- nodeOptions);
+ RpcServer rpcServer = this.context.rpcServer();
+ LOG.info("The raft endpoint '{}', initial group peers [{}]",
+ endpoint, nodeOptions.getInitialConf());
+ // Shared rpc server
+ return new RaftGroupService(groupId, endpoint, nodeOptions,
+ rpcServer, true);
Review Comment:
can share rpc-server by RaftServiceFactory.createAndInitRaftNode()
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftContext.java:
##########
@@ -353,14 +352,17 @@ public ExecutorService backendExecutor() {
return this.backendExecutor;
}
+ public ExecutorService readIndexExecutor() {
+ return this.readIndexExecutor;
+ }
+
public GraphMode graphMode() {
return this.params.mode();
}
private HugeConfig config() {
return this.params.configuration();
}
-
Review Comment:
keep blank line
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftBackendStoreProvider.java:
##########
@@ -181,10 +213,15 @@ public void clear() {
}
@Override
- public void truncate() {
+ public void truncate(HugeGraph graph) {
Review Comment:
pass systemStoreName?
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/config/CoreOptions.java:
##########
@@ -252,10 +228,19 @@ public static synchronized CoreOptions instance() {
public static final ConfigOption<Integer> RAFT_RPC_TIMEOUT =
new ConfigOption<>(
"raft.rpc_timeout",
- "The rpc timeout for jraft rpc.",
+ "The general rpc timeout for jraft rpc.",
Review Comment:
in millisecond
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftContext.java:
##########
@@ -375,6 +377,7 @@ private RpcServer initAndStartRpcServer() {
NodeManager.getInstance().addAddress(endpoint.getEndpoint());
RpcServer rpcServer = RaftRpcServerFactory.createAndStartRaftRpcServer(
endpoint.getEndpoint());
+ LOG.info("RPC server is started successfully");
Review Comment:
replaced by https://github.com/apache/incubator-hugegraph/pull/1527/files#diff-7a0b04965824fe2f392d66bc9cb0addb6b059d81142c3117f8ff6c390b5e4e03R294
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftNode.java:
##########
@@ -98,39 +104,52 @@ public void shutdown() {
this.node.shutdown();
}
- public void snapshot() {
- if (!this.context.useSnapshot()) {
- return;
- }
+ public RaftClosure<?> snapshot() {
RaftClosure<?> future = new RaftClosure<>();
try {
this.node().snapshot(future);
- future.waitFinished();
+ return future;
} catch (Throwable e) {
throw new BackendException("Failed to create snapshot", e);
}
}
- private Node initRaftNode() throws IOException {
+ private RaftGroupService initRaftNode() throws IOException {
NodeOptions nodeOptions = this.context.nodeOptions();
nodeOptions.setFsm(this.stateMachine);
- // TODO: When support sharding, groupId needs to be bound to shard Id
+ /*
+ * TODO: the groupId is same as graph name now, when support sharding,
+ * groupId needs to be bound to shard Id
+ */
String groupId = this.context.group();
PeerId endpoint = this.context.endpoint();
/*
* Start raft node with shared rpc server:
- * return new RaftGroupService(groupId, endpoint, nodeOptions,
- * this.context.rpcServer(), true)
- * .start(false)
*/
- return RaftServiceFactory.createAndInitRaftNode(groupId, endpoint,
- nodeOptions);
+ RpcServer rpcServer = this.context.rpcServer();
+ LOG.info("The raft endpoint '{}', initial group peers [{}]",
+ endpoint, nodeOptions.getInitialConf());
+ // Shared rpc server
+ return new RaftGroupService(groupId, endpoint, nodeOptions,
+ rpcServer, true);
+ }
+
+ public void close() {
+ if (this.raftGroupService != null) {
+ this.raftGroupService.shutdown();
+ try {
+ this.raftGroupService.join();
Review Comment:
don't need to shutdown raftGroupService when raftNode closed?
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftBackendStoreProvider.java:
##########
@@ -45,22 +50,50 @@ public class RaftBackendStoreProvider implements BackendStoreProvider {
private static final Logger LOG = Log.logger(RaftBackendStoreProvider.class);
private final BackendStoreProvider provider;
- private final RaftSharedContext context;
private RaftBackendStore schemaStore;
private RaftBackendStore graphStore;
private RaftBackendStore systemStore;
+ private RaftContext context;
- public RaftBackendStoreProvider(BackendStoreProvider provider,
- HugeGraphParams params) {
+ public RaftBackendStoreProvider(BackendStoreProvider provider) {
this.provider = provider;
- this.context = new RaftSharedContext(params);
this.schemaStore = null;
this.graphStore = null;
this.systemStore = null;
+ this.context = null;
+ }
+
+ public void initRaftContext(HugeGraphParams params, RpcServer rpcServer) {
+ HugeConfig config = params.configuration();
+ Integer lowWaterMark = config.get(
+ CoreOptions.RAFT_RPC_BUF_LOW_WATER_MARK);
+ System.setProperty("bolt.channel_write_buf_low_water_mark",
Review Comment:
reuse from RaftContext.initAndStartRpcServer()?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hugegraph] Linary closed pull request #1527: Improve some action for install snapshot and add peer
Posted by GitBox <gi...@apache.org>.
Linary closed pull request #1527: Improve some action for install snapshot and add peer
URL: https://github.com/apache/incubator-hugegraph/pull/1527
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hugegraph] wangyao2016 commented on pull request #1527: Improve some action for install snapshot and add peer
Posted by GitBox <gi...@apache.org>.
wangyao2016 commented on PR #1527:
URL: https://github.com/apache/incubator-hugegraph/pull/1527#issuecomment-1128720092
With this PR, I cannot start hugegraph successful. Err:
`2022-05-17 18:37:28 [main] [INFO] c.b.h.s.RestServer - RestServer starting...
2022-05-17 18:37:30 [main] [INFO] c.b.h.u.ConfigUtil - Scanning option 'graphs' directory './conf/graphs'
2022-05-17 18:37:30 [main] [INFO] c.b.h.b.c.Cache - Init RamCache for 'users-hugegraph' with capacity 10240
2022-05-17 18:37:30 [main] [INFO] c.b.h.b.c.Cache - Init RamCache for 'users_pwd-hugegraph' with capacity 10240
2022-05-17 18:37:30 [main] [INFO] c.b.h.b.c.Cache - Init RamCache for 'token-hugegraph' with capacity 10240
2022-05-17 18:37:30 [main] [INFO] c.b.h.s.RestServer - Graph 'hugegraph' was successfully configured via './conf/graphs/hugegraph.properties'
2022-05-17 18:37:30 [main] [INFO] c.b.h.r.RpcServer - RpcServer config is empty, skip starting RpcServer
2022-05-17 18:37:30 [main] [ERROR] c.b.h.d.HugeGraphServer - HugeRestServer start error:
java.lang.NullPointerException: null
at com.baidu.hugegraph.testutil.Whitebox.getInternalState(Whitebox.java:69) ~[hugegraph-common-2.1.2.jar:2.1.2.0]
at com.baidu.hugegraph.core.GraphManager.remotingRpcServer(GraphManager.java:301) ~[hugegraph-api-0.13.0.jar:0.69.0.0]
at com.baidu.hugegraph.core.GraphManager.waitGraphsReady(GraphManager.java:373) ~[hugegraph-api-0.13.0.jar:0.69.0.0]
at com.baidu.hugegraph.core.GraphManager.<init>(GraphManager.java:112) ~[hugegraph-api-0.13.0.jar:0.69.0.0]
at com.baidu.hugegraph.server.ApplicationConfig$GraphManagerFactory$1.onEvent(ApplicationConfig.java:130) ~[hugegraph-api-0.13.0.jar:0.69.0.0]
at org.glassfish.jersey.server.internal.monitoring.CompositeApplicationEventListener.onEvent(CompositeApplicationEventListener.java:49) ~[jersey-server-3.0.3.jar:?]
at org.glassfish.jersey.server.internal.monitoring.MonitoringContainerListener.onStartup(MonitoringContainerListener.java:56) ~[jersey-server-3.0.3.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.onStartup(ApplicationHandler.java:711) ~[jersey-server-3.0.3.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.start(GrizzlyHttpContainer.java:330) ~[jersey-container-grizzly2-http-3.0.3.jar:?]
at org.glassfish.grizzly.http.server.HttpHandlerChain.start(HttpHandlerChain.java:376) ~[grizzly-http-server-3.0.1.jar:3.0.1]
at org.glassfish.grizzly.http.server.HttpServer.setupHttpHandler(HttpServer.java:268) ~[grizzly-http-server-3.0.1.jar:3.0.1]
at org.glassfish.grizzly.http.server.HttpServer.start(HttpServer.java:245) ~[grizzly-http-server-3.0.1.jar:3.0.1]
at com.baidu.hugegraph.server.RestServer.start(RestServer.java:73) ~[hugegraph-api-0.13.0.jar:0.69.0.0]
at com.baidu.hugegraph.server.RestServer.start(RestServer.java:180) ~[hugegraph-api-0.13.0.jar:0.69.0.0]
at com.baidu.hugegraph.dist.HugeRestServer.start(HugeRestServer.java:34) ~[hugegraph-dist-0.13.0.jar:?]
at com.baidu.hugegraph.dist.HugeGraphServer.<init>(HugeGraphServer.java:62) ~[hugegraph-dist-0.13.0.jar:?]
at com.baidu.hugegraph.dist.HugeGraphServer.main(HugeGraphServer.java:121) ~[hugegraph-dist-0.13.0.jar:?]`
I found [serverConfig.getServer()](https://github.com/apache/incubator-hugegraph/commit/8609968e6648d69189b6ea6d37b68b81e245e4dc#diff-7a0b04965824fe2f392d66bc9cb0addb6b059d81142c3117f8ff6c390b5e4e03R299) was null. Actually ServerConfig is `[protocol=bolt, port=8091, host=127.0.0.1]`
Can you take a look?Thanks. @Linary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hugegraph] Linary commented on a diff in pull request #1527: Improve some action for install snapshot and add peer
Posted by GitBox <gi...@apache.org>.
Linary commented on code in PR #1527:
URL: https://github.com/apache/incubator-hugegraph/pull/1527#discussion_r873133967
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftBackendStoreProvider.java:
##########
@@ -219,8 +260,16 @@ public void createSnapshot() {
StoreCommand command = new StoreCommand(StoreType.GRAPH,
StoreAction.SNAPSHOT, null);
RaftStoreClosure closure = new RaftStoreClosure(command);
- this.context.node().submitAndWait(command, closure);
- LOG.debug("Graph '{}' has writed snapshot", this.graph());
+ RaftClosure<?> future = this.context.node().submitAndWait(command,
+ closure);
+ if (future != null) {
Review Comment:
Maybe not. Maybe it's defensive programming
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hugegraph] javeme commented on a diff in pull request #1527: Improve some action for install snapshot and add peer
Posted by GitBox <gi...@apache.org>.
javeme commented on code in PR #1527:
URL: https://github.com/apache/incubator-hugegraph/pull/1527#discussion_r857337462
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftContext.java:
##########
@@ -104,26 +106,38 @@ public final class RaftSharedContext {
private RaftGroupManager raftGroupManager;
private RpcForwarder rpcForwarder;
- public RaftSharedContext(HugeGraphParams params) {
+ public RaftContext(HugeGraphParams params, RpcServer rpcServer,
+ PeerId endpoint) {
this.params = params;
- HugeConfig config = this.config();
+ this.rpcServer = rpcServer;
+ this.endpoint = endpoint;
+ HugeConfig config = params.configuration();
this.schemaStoreName = config.get(CoreOptions.STORE_SCHEMA);
this.graphStoreName = config.get(CoreOptions.STORE_GRAPH);
this.systemStoreName = config.get(CoreOptions.STORE_SYSTEM);
this.stores = new RaftBackendStore[StoreType.ALL.getNumber()];
- this.rpcServer = this.initAndStartRpcServer();
+
+// // TODO: 依赖了ServerOptions的配置项名,需要打通ServerConfig和CoreConfig
Review Comment:
translate to English?
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftNode.java:
##########
@@ -98,39 +104,52 @@ public void shutdown() {
this.node.shutdown();
}
- public void snapshot() {
- if (!this.context.useSnapshot()) {
- return;
- }
+ public RaftClosure<?> snapshot() {
RaftClosure<?> future = new RaftClosure<>();
try {
this.node().snapshot(future);
- future.waitFinished();
+ return future;
} catch (Throwable e) {
throw new BackendException("Failed to create snapshot", e);
}
}
- private Node initRaftNode() throws IOException {
+ private RaftGroupService initRaftNode() throws IOException {
NodeOptions nodeOptions = this.context.nodeOptions();
nodeOptions.setFsm(this.stateMachine);
- // TODO: When support sharding, groupId needs to be bound to shard Id
+ /*
+ * TODO: the groupId is same as graph name now, when support sharding,
+ * groupId needs to be bound to shard Id
+ */
String groupId = this.context.group();
PeerId endpoint = this.context.endpoint();
/*
* Start raft node with shared rpc server:
- * return new RaftGroupService(groupId, endpoint, nodeOptions,
- * this.context.rpcServer(), true)
- * .start(false)
*/
- return RaftServiceFactory.createAndInitRaftNode(groupId, endpoint,
- nodeOptions);
+ RpcServer rpcServer = this.context.rpcServer();
+ LOG.info("The raft endpoint '{}', initial group peers [{}]",
+ endpoint, nodeOptions.getInitialConf());
+ // Shared rpc server
+ return new RaftGroupService(groupId, endpoint, nodeOptions,
+ rpcServer, true);
+ }
+
+ public void close() {
+ if (this.raftGroupService != null) {
+ this.raftGroupService.shutdown();
+ try {
+ this.raftGroupService.join();
+ } catch (final InterruptedException e) {
+ throw new RaftException("Interrupted while shutdown " +
+ "raftGroupService");
+ }
+ }
}
private void submitCommand(StoreCommand command, RaftStoreClosure closure) {
// Wait leader elected
LeaderInfo leaderInfo = this.waitLeaderElected(
- RaftSharedContext.NO_TIMEOUT);
+ RaftContext.WAIT_LEADER_TIMEOUT);
Review Comment:
can align with "this"?
##########
hugegraph-api/src/main/java/com/baidu/hugegraph/core/GraphManager.java:
##########
@@ -257,6 +286,45 @@ public void close() {
this.unlistenChanges();
}
+ private void loadGraph(HugeConfig serverConfig, String name, String path) {
+ HugeConfig config = new HugeConfig(path);
+ String raftGroupPeers = serverConfig.get(ServerOptions.RAFT_GROUP_PEERS);
+ config.addProperty(ServerOptions.RAFT_GROUP_PEERS.name(), raftGroupPeers);
+
+ final Graph graph = GraphFactory.open(config);
+ this.graphs.put(name, graph);
+ LOG.info("Graph '{}' was successfully configured via '{}'", name, path);
+
+ if (this.requireAuthentication() &&
+ !(graph instanceof HugeGraphAuthProxy)) {
+ LOG.warn("You may need to support access control for '{}' with {}",
+ path, HugeFactoryAuthProxy.GRAPH_FACTORY);
+ }
+ }
+
+// private com.alipay.sofa.jraft.rpc.RpcServer startRaftRpcServer(HugeConfig
Review Comment:
can delete unused code
##########
hugegraph-api/src/main/java/com/baidu/hugegraph/core/GraphManager.java:
##########
@@ -100,30 +106,53 @@ public GraphManager(HugeConfig conf, EventHub hub) {
this.loadGraphs(ConfigUtil.scanGraphsDir(this.graphsDir));
// this.installLicense(conf, "");
// Raft will load snapshot firstly then launch election and replay log
- this.waitGraphsStarted();
- this.checkBackendVersionOrExit(conf);
this.startRpcServer();
+
+ initAllSystemSchema();
Review Comment:
this.initAllSystemSchema()
##########
hugegraph-api/src/main/java/com/baidu/hugegraph/core/GraphManager.java:
##########
@@ -257,6 +286,45 @@ public void close() {
this.unlistenChanges();
}
+ private void loadGraph(HugeConfig serverConfig, String name, String path) {
Review Comment:
can move to line 401 to diff them
##########
hugegraph-api/src/main/java/com/baidu/hugegraph/core/GraphManager.java:
##########
@@ -100,30 +106,53 @@ public GraphManager(HugeConfig conf, EventHub hub) {
this.loadGraphs(ConfigUtil.scanGraphsDir(this.graphsDir));
// this.installLicense(conf, "");
// Raft will load snapshot firstly then launch election and replay log
- this.waitGraphsStarted();
- this.checkBackendVersionOrExit(conf);
this.startRpcServer();
+
+ initAllSystemSchema();
+
+ ServerConfig serverConfig = Whitebox.getInternalState(this.rpcServer,
+ "serverConfig");
+ com.alipay.remoting.rpc.RpcServer remotingRpcServer;
+ remotingRpcServer = Whitebox.getInternalState(serverConfig.getServer(),
+ "remotingServer");
Review Comment:
add a method remotingRpcServer()
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftBackendStoreProvider.java:
##########
@@ -45,22 +51,51 @@ public class RaftBackendStoreProvider implements BackendStoreProvider {
private static final Logger LOG = Log.logger(RaftBackendStoreProvider.class);
private final BackendStoreProvider provider;
- private final RaftSharedContext context;
private RaftBackendStore schemaStore;
private RaftBackendStore graphStore;
private RaftBackendStore systemStore;
+ private RaftContext context;
- public RaftBackendStoreProvider(BackendStoreProvider provider,
- HugeGraphParams params) {
+ public RaftBackendStoreProvider(BackendStoreProvider provider) {
this.provider = provider;
- this.context = new RaftSharedContext(params);
this.schemaStore = null;
this.graphStore = null;
this.systemStore = null;
+ this.context = null;
+ }
+
+ public void initRaftContext(HugeGraphParams params, RpcServer rpcServer) {
+ HugeConfig config = params.configuration();
+ Integer lowWaterMark = config.get(
+ CoreOptions.RAFT_RPC_BUF_LOW_WATER_MARK);
+ System.setProperty("bolt.channel_write_buf_low_water_mark",
+ String.valueOf(lowWaterMark));
+ Integer highWaterMark = config.get(
+ CoreOptions.RAFT_RPC_BUF_HIGH_WATER_MARK);
+ System.setProperty("bolt.channel_write_buf_high_water_mark",
+ String.valueOf(highWaterMark));
+
+// PeerId endpoint = new PeerId();
Review Comment:
add "TODO: xx" comment
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftBackendStoreProvider.java:
##########
@@ -45,22 +51,51 @@ public class RaftBackendStoreProvider implements BackendStoreProvider {
private static final Logger LOG = Log.logger(RaftBackendStoreProvider.class);
private final BackendStoreProvider provider;
- private final RaftSharedContext context;
private RaftBackendStore schemaStore;
private RaftBackendStore graphStore;
private RaftBackendStore systemStore;
+ private RaftContext context;
- public RaftBackendStoreProvider(BackendStoreProvider provider,
- HugeGraphParams params) {
+ public RaftBackendStoreProvider(BackendStoreProvider provider) {
this.provider = provider;
- this.context = new RaftSharedContext(params);
this.schemaStore = null;
this.graphStore = null;
this.systemStore = null;
+ this.context = null;
+ }
+
+ public void initRaftContext(HugeGraphParams params, RpcServer rpcServer) {
+ HugeConfig config = params.configuration();
+ Integer lowWaterMark = config.get(
+ CoreOptions.RAFT_RPC_BUF_LOW_WATER_MARK);
+ System.setProperty("bolt.channel_write_buf_low_water_mark",
+ String.valueOf(lowWaterMark));
+ Integer highWaterMark = config.get(
+ CoreOptions.RAFT_RPC_BUF_HIGH_WATER_MARK);
+ System.setProperty("bolt.channel_write_buf_high_water_mark",
+ String.valueOf(highWaterMark));
+
+// PeerId endpoint = new PeerId();
+// String endpointStr = config.get(ServerOptions.RAFT_ENDPOINT);
+// if (!endpoint.parse(endpointStr)) {
+// throw new HugeException("Failed to parse endpoint %s", endpointStr);
+// }
+
+ // Reference from RaftRpcServerFactory.createAndStartRaftRpcServer
+ com.alipay.sofa.jraft.rpc.RpcServer raftRpcServer =
+ new BoltRpcServer(rpcServer);
+ RaftRpcServerFactory.addRaftRequestProcessors(raftRpcServer);
+ raftRpcServer.init(null);
+
+ PeerId endpoint = new PeerId(rpcServer.ip(), rpcServer.port());
+ this.context = new RaftContext(params, raftRpcServer, endpoint);
+
+ //
Review Comment:
unused?
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftBackendStoreProvider.java:
##########
@@ -197,7 +236,9 @@ public void initSystemInfo(HugeGraph graph) {
BackendStoreSystemInfo info = graph.backendStoreSystemInfo();
info.init();
- this.notifyAndWaitEvent(Events.STORE_INITED);
+ // 创建系统schema,保存到pool里面
Review Comment:
translate to English
##########
hugegraph-api/src/main/java/com/baidu/hugegraph/core/GraphManager.java:
##########
@@ -100,30 +106,53 @@ public GraphManager(HugeConfig conf, EventHub hub) {
this.loadGraphs(ConfigUtil.scanGraphsDir(this.graphsDir));
// this.installLicense(conf, "");
// Raft will load snapshot firstly then launch election and replay log
- this.waitGraphsStarted();
- this.checkBackendVersionOrExit(conf);
this.startRpcServer();
+
+ initAllSystemSchema();
+
+ ServerConfig serverConfig = Whitebox.getInternalState(this.rpcServer,
+ "serverConfig");
+ com.alipay.remoting.rpc.RpcServer remotingRpcServer;
+ remotingRpcServer = Whitebox.getInternalState(serverConfig.getServer(),
+ "remotingServer");
+ this.waitGraphsReady(remotingRpcServer);
+
+ this.checkBackendVersionOrExit(conf);
+
this.serverStarted(conf);
this.addMetrics(conf);
}
- public void loadGraphs(final Map<String, String> graphConfs) {
+ public {
+ //
+ register(-1, ~task, xx, xx, xx,);
+
+
+
+
+
+
Review Comment:
need to clean code?
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftBackendStoreProvider.java:
##########
@@ -197,7 +236,9 @@ public void initSystemInfo(HugeGraph graph) {
BackendStoreSystemInfo info = graph.backendStoreSystemInfo();
info.init();
- this.notifyAndWaitEvent(Events.STORE_INITED);
+ // 创建系统schema,保存到pool里面
+ init();
Review Comment:
this.init()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hugegraph] javeme commented on a diff in pull request #1527: Improve some action for install snapshot and add peer
Posted by GitBox <gi...@apache.org>.
javeme commented on code in PR #1527:
URL: https://github.com/apache/incubator-hugegraph/pull/1527#discussion_r857365568
##########
hugegraph-core/src/main/java/com/baidu/hugegraph/backend/store/raft/RaftBackendStoreProvider.java:
##########
@@ -219,8 +260,16 @@ public void createSnapshot() {
StoreCommand command = new StoreCommand(StoreType.GRAPH,
StoreAction.SNAPSHOT, null);
RaftStoreClosure closure = new RaftStoreClosure(command);
- this.context.node().submitAndWait(command, closure);
- LOG.debug("Graph '{}' has writed snapshot", this.graph());
+ RaftClosure<?> future = this.context.node().submitAndWait(command,
+ closure);
+ if (future != null) {
Review Comment:
when the future may be null?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org