You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2022/03/26 07:01:37 UTC
[GitHub] [iotdb] MarcosZyk opened a new pull request #5349: Add SchemaRegion Interfaces to SchemaEngine
MarcosZyk opened a new pull request #5349:
URL: https://github.com/apache/iotdb/pull/5349
## Description
### Add SchemaRegion Management to metadata module
1. Add SchemaRegion Management to SchemaEngine for new cluster development
2. In current standalone mode, or in v0.14, each storage group will only have one schemaRegion
### Future work
I'tried to implement multi schemaRegions for one storage group, which is quite similar with the schemaRegion management and metadata operation implement in cluster mode. Everything worked fine except one feature, Template, which is too complex to be handled, which hasn't been design in new cluster.
The template can be set to any Internal node under sg in mtree, and this is the main difficulty. In current schema partition design, an Internal node may exists in several schemaRegions of one storage group, since the schema is partitioned by device hash.
I've designed and implement a version but it isn't worth enough to be mocked or applied on standalone mode. Hope there will be a nice solution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin merged pull request #5349: Add SchemaRegion Interfaces to SchemaEngine
Posted by GitBox <gi...@apache.org>.
qiaojialin merged pull request #5349:
URL: https://github.com/apache/iotdb/pull/5349
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin commented on a change in pull request #5349: Add SchemaRegion Interfaces to SchemaEngine
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #5349:
URL: https://github.com/apache/iotdb/pull/5349#discussion_r835895068
##########
File path: cluster/src/test/java/org/apache/iotdb/cluster/partition/SchemaEngineWhiteBox.java
##########
@@ -35,7 +35,7 @@ public static SchemaEngine newSchemaEngine(String logFilePath) {
SchemaEngine schemaEngine = constructor.newInstance();
new File(logFilePath).getParentFile().mkdirs();
Whitebox.setInternalState(schemaEngine, "logFilePath", logFilePath);
- schemaEngine.initForMultiSchemaEngineTest();
+ // schemaEngine.initForMultiSchemaEngineTest();
Review comment:
remove?
##########
File path: node-commons/src/main/java/org/apache/iotdb/commons/partition/SchemaRegionId.java
##########
@@ -18,14 +18,33 @@
*/
package org.apache.iotdb.commons.partition;
+import java.util.Objects;
+
public class SchemaRegionId {
private int schemaRegionId;
+ public SchemaRegionId(int schemaRegionId) {
+ this.schemaRegionId = schemaRegionId;
+ }
+
public int getSchemaRegionId() {
return schemaRegionId;
}
public void setSchemaRegionId(int schemaRegionId) {
this.schemaRegionId = schemaRegionId;
}
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) return true;
+ if (o == null || getClass() != o.getClass()) return false;
Review comment:
add {}
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/schemaregion/SchemaRegionManager.java
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.metadata.schemaregion;
+
+import org.apache.iotdb.commons.partition.SchemaRegionId;
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.metadata.mnode.IStorageGroupMNode;
+import org.apache.iotdb.db.metadata.path.PartialPath;
+
+import java.util.Collection;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+// manage all the schemaRegion in this dataNode
+public class SchemaRegionManager {
Review comment:
```suggestion
public class SchemaEngine {
```
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/localconfig/LocalSchemaConfigManager.java
##########
@@ -0,0 +1,763 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.metadata.localconfig;
+
+import org.apache.iotdb.commons.concurrent.IoTDBThreadPoolFactory;
+import org.apache.iotdb.commons.conf.IoTDBConstant;
+import org.apache.iotdb.commons.partition.SchemaRegionId;
+import org.apache.iotdb.db.conf.IoTDBConfig;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
+import org.apache.iotdb.db.engine.fileSystem.SystemFileFactory;
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.exception.metadata.PathNotExistException;
+import org.apache.iotdb.db.exception.metadata.StorageGroupAlreadySetException;
+import org.apache.iotdb.db.exception.metadata.StorageGroupNotSetException;
+import org.apache.iotdb.db.exception.metadata.UndefinedTemplateException;
+import org.apache.iotdb.db.metadata.SchemaEngine;
+import org.apache.iotdb.db.metadata.mnode.IStorageGroupMNode;
+import org.apache.iotdb.db.metadata.path.PartialPath;
+import org.apache.iotdb.db.metadata.rescon.TimeseriesStatistics;
+import org.apache.iotdb.db.metadata.schemaregion.SchemaRegion;
+import org.apache.iotdb.db.metadata.schemaregion.SchemaRegionManager;
+import org.apache.iotdb.db.metadata.storagegroup.IStorageGroupSchemaManager;
+import org.apache.iotdb.db.metadata.storagegroup.StorageGroupSchemaManager;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.db.metadata.template.TemplateManager;
+import org.apache.iotdb.db.metadata.utils.MetaUtils;
+import org.apache.iotdb.db.qp.physical.sys.ActivateTemplatePlan;
+import org.apache.iotdb.db.qp.physical.sys.AppendTemplatePlan;
+import org.apache.iotdb.db.qp.physical.sys.CreateTemplatePlan;
+import org.apache.iotdb.db.qp.physical.sys.DropTemplatePlan;
+import org.apache.iotdb.db.qp.physical.sys.PruneTemplatePlan;
+import org.apache.iotdb.db.qp.physical.sys.SetTemplatePlan;
+import org.apache.iotdb.db.qp.physical.sys.UnsetTemplatePlan;
+import org.apache.iotdb.db.rescon.MemTableManager;
+import org.apache.iotdb.tsfile.utils.Pair;
+import org.apache.iotdb.tsfile.write.schema.IMeasurementSchema;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * This class simulates the behaviour of configNode to manage the schema configs locally. The schema
+ * configs include storage group, schema region and template.
+ */
+public class LocalSchemaConfigManager {
Review comment:
```suggestion
public class LocalConfigManager {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org