You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2022/04/05 11:02:38 UTC
[GitHub] [iotdb] OneSizeFitsQuorum opened a new pull request, #5417: [IOTDB-2832] Initialize the consensus layer of the Datanode
OneSizeFitsQuorum opened a new pull request, #5417:
URL: https://github.com/apache/iotdb/pull/5417
This PR consists of the following four tasks :
- Factory classes `IConsensusFactory` for consensus layer initialization are defined to facilitate integration of other modules
- Added parameters related to the consensus layer for configNode and Datanode configuration files
- Switch the ratis implementation of configNode whose default consensus protocol is copy 1 to ensure recovery after restart.
- Set the default consensus protocol of ConfigNode to ratis which replicaNum is 1 to ensure crash recovery.
- The consensus layer singleton of Datanode is added, and the StateMachine is registered in the corresponding engine when the consensus layer initializes it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] wangchao316 commented on a diff in pull request #5417: [IOTDB-2832] Initialize the consensus layer of the Datanode
Posted by GitBox <gi...@apache.org>.
wangchao316 commented on code in PR #5417:
URL: https://github.com/apache/iotdb/pull/5417#discussion_r845676501
##########
server/src/main/java/org/apache/iotdb/db/engine/StorageEngine.java:
##########
@@ -1083,6 +1087,14 @@ protected void getSeriesSchemas(InsertPlan insertPlan, VirtualStorageGroupProces
}
}
+ public DataRegion createDataRegion(DataRegionId regionId, String sg, long ttl) {
+ return null;
Review Comment:
null? why?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin commented on a diff in pull request #5417: [IOTDB-2832] Initialize the consensus layer of the Datanode
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on code in PR #5417:
URL: https://github.com/apache/iotdb/pull/5417#discussion_r843414475
##########
server/src/main/java/org/apache/iotdb/db/consensus/ConsensusImpl.java:
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.consensus;
+
+import org.apache.iotdb.commons.cluster.Endpoint;
+import org.apache.iotdb.consensus.IConsensus;
+import org.apache.iotdb.consensus.IConsensusFactory;
+import org.apache.iotdb.db.conf.IoTDBConfig;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
+import org.apache.iotdb.db.engine.StorageEngine;
+import org.apache.iotdb.db.metadata.schemaregion.SchemaEngine;
+
+import java.io.File;
+
+public class ConsensusImpl {
Review Comment:
add doc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] sonarcloud[bot] commented on pull request #5417: [IOTDB-2832] Initialize the consensus layer of the Datanode
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #5417:
URL: https://github.com/apache/iotdb/pull/5417#issuecomment-1092430848
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_incubator-iotdb&pullRequest=5417)
[](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=CODE_SMELL) [38 Code Smells](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=5417&metric=new_coverage&view=list) [32.2% Coverage](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=5417&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=5417&metric=new_duplicated_lines_density&view=list) [1.3% Duplication](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=5417&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] OneSizeFitsQuorum commented on a diff in pull request #5417: [IOTDB-2832] Initialize the consensus layer of the Datanode
Posted by GitBox <gi...@apache.org>.
OneSizeFitsQuorum commented on code in PR #5417:
URL: https://github.com/apache/iotdb/pull/5417#discussion_r844555468
##########
server/src/main/java/org/apache/iotdb/db/metadata/schemaregion/SchemaEngine.java:
##########
@@ -60,6 +64,10 @@ public SchemaRegion getSchemaRegion(ConsensusGroupId schemaRegionId) {
return schemaRegionMap.get(schemaRegionId);
}
+ public SchemaRegionStateMachine getOrCreateDataRegionStateMachine(ConsensusGroupId gid) {
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin commented on a diff in pull request #5417: [IOTDB-2832] Initialize the consensus layer of the Datanode
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on code in PR #5417:
URL: https://github.com/apache/iotdb/pull/5417#discussion_r842834529
##########
server/src/main/java/org/apache/iotdb/db/metadata/schemaregion/SchemaEngine.java:
##########
@@ -33,6 +34,9 @@
private Map<ConsensusGroupId, SchemaRegion> schemaRegionMap;
+ private final Map<ConsensusGroupId, SchemaRegionStateMachine> stateMachineMap =
Review Comment:
We need to discuss how to manage the key of SchemaRegion in SchemaEngine.
##########
server/src/main/java/org/apache/iotdb/db/consensus/ConsensusImpl.java:
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.consensus;
+
+import org.apache.iotdb.commons.cluster.Endpoint;
+import org.apache.iotdb.consensus.IConsensus;
+import org.apache.iotdb.consensus.IConsensusFactory;
+import org.apache.iotdb.db.conf.IoTDBConfig;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
+import org.apache.iotdb.db.engine.StorageEngine;
+import org.apache.iotdb.db.metadata.schemaregion.SchemaEngine;
+
+import java.io.File;
+
+public class ConsensusImpl {
Review Comment:
Where is this class used?
##########
consensus/src/main/java/org/apache/iotdb/consensus/IConsensusFactory.java:
##########
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.consensus;
+
+import org.apache.iotdb.commons.cluster.Endpoint;
+import org.apache.iotdb.consensus.statemachine.IStateMachine;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.lang.reflect.Constructor;
+import java.lang.reflect.InvocationTargetException;
+import java.util.Optional;
+
+public class IConsensusFactory {
Review Comment:
```suggestion
public class ConsensusFactory {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] OneSizeFitsQuorum commented on a diff in pull request #5417: [IOTDB-2832] Initialize the consensus layer of the Datanode
Posted by GitBox <gi...@apache.org>.
OneSizeFitsQuorum commented on code in PR #5417:
URL: https://github.com/apache/iotdb/pull/5417#discussion_r843415329
##########
consensus/src/main/java/org/apache/iotdb/consensus/IConsensusFactory.java:
##########
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.consensus;
+
+import org.apache.iotdb.commons.cluster.Endpoint;
+import org.apache.iotdb.consensus.statemachine.IStateMachine;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.lang.reflect.Constructor;
+import java.lang.reflect.InvocationTargetException;
+import java.util.Optional;
+
+public class IConsensusFactory {
Review Comment:
Fixed
##########
server/src/main/java/org/apache/iotdb/db/consensus/ConsensusImpl.java:
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.consensus;
+
+import org.apache.iotdb.commons.cluster.Endpoint;
+import org.apache.iotdb.consensus.IConsensus;
+import org.apache.iotdb.consensus.IConsensusFactory;
+import org.apache.iotdb.db.conf.IoTDBConfig;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
+import org.apache.iotdb.db.engine.StorageEngine;
+import org.apache.iotdb.db.metadata.schemaregion.SchemaEngine;
+
+import java.io.File;
+
+public class ConsensusImpl {
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yifuzhou commented on a diff in pull request #5417: [IOTDB-2832] Initialize the consensus layer of the Datanode
Posted by GitBox <gi...@apache.org>.
yifuzhou commented on code in PR #5417:
URL: https://github.com/apache/iotdb/pull/5417#discussion_r843590729
##########
server/src/main/java/org/apache/iotdb/db/metadata/schemaregion/SchemaEngine.java:
##########
@@ -60,6 +64,10 @@ public SchemaRegion getSchemaRegion(ConsensusGroupId schemaRegionId) {
return schemaRegionMap.get(schemaRegionId);
}
+ public SchemaRegionStateMachine getOrCreateDataRegionStateMachine(ConsensusGroupId gid) {
Review Comment:
should be getOrCreateSchemaRegionStatemachine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin merged pull request #5417: [IOTDB-2832] Initialize the consensus layer of the Datanode
Posted by GitBox <gi...@apache.org>.
qiaojialin merged PR #5417:
URL: https://github.com/apache/iotdb/pull/5417
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] sonarcloud[bot] commented on pull request #5417: [IOTDB-2832] Initialize the consensus layer of the Datanode
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #5417:
URL: https://github.com/apache/iotdb/pull/5417#issuecomment-1092782033
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache_incubator-iotdb&pullRequest=5417)
[](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=CODE_SMELL) [37 Code Smells](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=5417&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=5417&metric=new_coverage&view=list) [31.5% Coverage](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=5417&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=5417&metric=new_duplicated_lines_density&view=list) [1.3% Duplication](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=5417&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] OneSizeFitsQuorum commented on a diff in pull request #5417: [IOTDB-2832] Initialize the consensus layer of the Datanode
Posted by GitBox <gi...@apache.org>.
OneSizeFitsQuorum commented on code in PR #5417:
URL: https://github.com/apache/iotdb/pull/5417#discussion_r845690064
##########
server/src/main/java/org/apache/iotdb/db/engine/StorageEngine.java:
##########
@@ -1083,6 +1087,14 @@ protected void getSeriesSchemas(InsertPlan insertPlan, VirtualStorageGroupProces
}
}
+ public DataRegion createDataRegion(DataRegionId regionId, String sg, long ttl) {
+ return null;
Review Comment:
StorageEngine will implement this function in a later PR, and I've added comments and TODO
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org