You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2022/07/06 13:13:35 UTC
[GitHub] [iotdb] yunchan86 opened a new pull request, #6608: [IOTDB-3560]Support basic create and query template
yunchan86 opened a new pull request, #6608:
URL: https://github.com/apache/iotdb/pull/6608
template create and query
```
---------------------
Starting IoTDB Cli
---------------------
_____ _________ ______ ______
|_ _| | _ _ ||_ _ `.|_ _ \
| | .--.|_/ | | \_| | | `. \ | |_) |
| | / .'`\ \ | | | | | | | __'.
_| |_| \__. | _| |_ _| |_.' /_| |__) |
|_____|'.__.' |_____| |______.'|_______/ version UNKNOWN
IoTDB> login successfully
IoTDB> show schema templates;
+-------------+
|template name|
+-------------+
+-------------+
Empty set.
It costs 0.314s
IoTDB> create schema template t1 (temperature FLOAT encoding=RLE, status BOOLEAN encoding=PLAIN compression=SNAPPY)
Msg: The statement is executed successfully.
IoTDB> show schema templates;
+-------------+
|template name|
+-------------+
| t1|
+-------------+
Total line number = 1
It costs 0.021s
IoTDB> show nodes in schema template t1;
+-----------+--------+--------+-----------+
|child nodes|dataType|encoding|compression|
+-----------+--------+--------+-----------+
|temperature| FLOAT| RLE| SNAPPY|
| status| BOOLEAN| PLAIN| SNAPPY|
+-----------+--------+--------+-----------+
Total line number = 2
It costs 0.026s
IoTDB> create schema template t2 aligned (lat FLOAT encoding=Gorilla, lon FLOAT encoding=Gorilla)
Msg: The statement is executed successfully.
IoTDB> show schema templates;
+-------------+
|template name|
+-------------+
| t1|
| t2|
+-------------+
Total line number = 2
It costs 0.017s
IoTDB> show nodes in schema template t2;
+-----------+--------+--------+-----------+
|child nodes|dataType|encoding|compression|
+-----------+--------+--------+-----------+
| lon| FLOAT| GORILLA| SNAPPY|
| lat| FLOAT| GORILLA| SNAPPY|
+-----------+--------+--------+-----------+
Total line number = 2
It costs 0.007s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916429781
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/parser/ASTVisitor.java:
##########
@@ -2311,4 +2314,178 @@ public Statement visitShowRegion(IoTDBSqlParser.ShowRegionContext ctx) {
public Statement visitShowDataNodes(IoTDBSqlParser.ShowDataNodesContext ctx) {
return new ShowDataNodesStatement();
}
+
+ @Override
+ public Statement visitCreateSchemaTemplate(IoTDBSqlParser.CreateSchemaTemplateContext ctx) {
+ CreateSchemaTemplateStatement createTemplateStatement = new CreateSchemaTemplateStatement();
+ String name = parseIdentifier(ctx.templateName.getText());
+ List<List<String>> measurementsList = new ArrayList<List<String>>();
+ List<List<TSDataType>> dataTypesList = new ArrayList<List<TSDataType>>();
+ List<List<TSEncoding>> encodingsList = new ArrayList<List<TSEncoding>>();
+ List<List<CompressionType>> compressorsList = new ArrayList<List<CompressionType>>();
+
+ if (ctx.ALIGNED() != null) {
+ // aligned
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ }
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ } else {
+ // non-aligned
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ // parseTemplateMeasurementClause(templateClauseContext, createTemplateStatement);
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ }
+ }
+
+ createTemplateStatement =
+ new CreateSchemaTemplateStatement(
+ name, measurementsList, dataTypesList, encodingsList, compressorsList);
+ return createTemplateStatement;
+ }
+
+ private void parseTemplateMeasurementClause(
+ IoTDBSqlParser.TemplateMeasurementClauseContext ctx,
+ CreateSchemaTemplateStatement createTemplateStatement) {
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ // single template measurement
+ measurements.add(parseNodeNameWithoutWildCard(ctx.nodeNameWithoutWildcard()));
+ parseAttributeClause(ctx.attributeClauses(), dataTypes, encodings, compressors);
+ // createTemplateStatement.addMeasurements(measurements);
+ // createTemplateStatement.addDataTypes(dataTypes);
+ // createTemplateStatement.addEncodings(encodings);
+ // createTemplateStatement.addCompressor(compressors);
Review Comment:
already removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916433737
##########
confignode/src/main/java/org/apache/iotdb/confignode/manager/TemplateManager.java:
##########
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.manager;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.persistence.TemplateInfo;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class TemplateManager {
+ private static final Logger LOGGER = LoggerFactory.getLogger(TemplateManager.class);
+
+ private final IManager configManager;
+ private final TemplateInfo templateInfo;
+
+ public TemplateManager(IManager configManager, TemplateInfo templateInfo) {
+ this.configManager = configManager;
+ this.templateInfo = templateInfo;
+ }
+
+ public TSStatus createTemplate(CreateSchemaTemplatePlan createSchemaTemplatePlan) {
+ try {
+ TSStatus tsStatus = getConsensusManager().write(createSchemaTemplatePlan).getStatus();
+ return tsStatus;
+ } catch (Exception e) {
+ return new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode())
+ .setMessage("schema template failed to create because some error.");
+ }
+ }
+
+ public TGetAllTemplatesResp getAllTemplates() {
+ return templateInfo.getAllTemplate();
+ }
+
+ TGetTemplateResp getTemplate(String req) {
+ return templateInfo.getMatchedTemplateByName(req);
+ }
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916433043
##########
confignode/src/main/java/org/apache/iotdb/confignode/manager/ConfigManager.java:
##########
@@ -862,4 +878,36 @@ public void addMetrics() {
partitionManager.addMetrics();
nodeManager.addMetrics();
}
+
+ @Override
+ public TSStatus createSchemaTemplate(TCreateSchemaTemplateReq req) {
+ TSStatus status = confirmLeader();
+ if (status.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+ CreateSchemaTemplatePlan createSchemaTemplatePlan =
+ new CreateSchemaTemplatePlan(req.getSerializedTemplate());
+ return templateManager.createTemplate(createSchemaTemplatePlan);
+ } else {
+ return status;
+ }
+ }
+
+ @Override
+ public TGetAllTemplatesResp getAllTemplates() {
+ TSStatus status = confirmLeader();
+ if (status.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+ return templateManager.getAllTemplates();
+ } else {
+ return new TGetAllTemplatesResp();
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916428580
##########
confignode/src/test/java/org/apache/iotdb/confignode/persistence/TemplateInfoTest.java:
##########
@@ -0,0 +1,141 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.persistence;
+
+import org.apache.iotdb.commons.exception.IllegalPathException;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.db.mpp.plan.statement.metadata.template.CreateSchemaTemplateStatement;
+import org.apache.iotdb.tsfile.file.metadata.enums.CompressionType;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSEncoding;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.thrift.TException;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+
+import static org.apache.iotdb.db.constant.TestConstant.BASE_OUTPUT_PATH;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/7/6
+ */
+public class TemplateInfoTest {
+
+ private static TemplateInfo templateInfo;
+ private static final File snapshotDir = new File(BASE_OUTPUT_PATH, "snapshot");
+
+ @BeforeClass
+ public static void setup() throws IOException {
+ templateInfo = new TemplateInfo();
+ if (!snapshotDir.exists()) {
+ snapshotDir.mkdirs();
+ }
+ }
+
+ @AfterClass
+ public static void cleanup() throws IOException {
+ templateInfo.clear();
+ if (snapshotDir.exists()) {
+ FileUtils.deleteDirectory(snapshotDir);
+ }
+ }
+
+ @Test
+ public void testSnapshot()
+ throws IOException, TException, IllegalPathException, ClassNotFoundException {
+ int n = 2;
+ String templateName = "template_test";
+
+ List<Template> templates = new ArrayList<>();
+ // 创建模板
+ for (int i = 0; i < n; i++) {
Review Comment:
Replace the comment with English in TemplateTableTest
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916434119
##########
confignode/src/main/java/org/apache/iotdb/confignode/manager/IManager.java:
##########
@@ -264,4 +267,10 @@ TDataPartitionResp getOrCreateDataPartition(
/** Show datanodes */
DataSet showDataNodes();
+
+ TSStatus createSchemaTemplate(TCreateSchemaTemplateReq req);
+
+ TGetAllTemplatesResp getAllTemplates();
+
+ TGetTemplateResp getTemplate(String req);
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] CRZbulabula commented on pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
CRZbulabula commented on PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#issuecomment-1177100872
Please add interface definition in [this doc](https://apache-iotdb.feishu.cn/docs/doccnCp13O7QDU7O6GMvagUwEDc)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916430406
##########
confignode/src/main/java/org/apache/iotdb/confignode/consensus/request/ConfigPhysicalPlanType.java:
##########
@@ -65,6 +65,7 @@ public enum ConfigPhysicalPlanType {
CreateFunction,
DropFunction,
GetNodePathsPartition,
+ CreateSchemaTemplate,
Review Comment:
append it at the end
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916428969
##########
server/src/main/java/org/apache/iotdb/db/exception/metadata/template/SchemaTemplateException.java:
##########
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.exception.metadata.template;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.exception.IoTDBException;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+public class SchemaTemplateException extends IoTDBException {
+
+ public SchemaTemplateException(TSStatus status) {
+ super(status.message, status.code);
+ }
+
+ public SchemaTemplateException(Throwable cause) {
+ super(cause, TSStatusCode.STORAGE_ENGINE_ERROR.getStatusCode());
+ }
+
+ public SchemaTemplateException(String message) {
+ super(message, TSStatusCode.STORAGE_ENGINE_ERROR.getStatusCode());
+ }
+
+ public SchemaTemplateException(String message, Throwable cause) {
+ super(message, cause, TSStatusCode.STORAGE_ENGINE_ERROR.getStatusCode());
+ }
+
+ public SchemaTemplateException(String message, int errorCode) {
+ super(message, errorCode);
+ }
+
+ public SchemaTemplateException(IoTDBException e) {
+ super(e, e.getErrorCode(), e.isUserException());
+ }
+}
Review Comment:
this file deleted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] bigreybear commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
bigreybear commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r915370488
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/parser/ASTVisitor.java:
##########
@@ -2311,4 +2314,178 @@ public Statement visitShowRegion(IoTDBSqlParser.ShowRegionContext ctx) {
public Statement visitShowDataNodes(IoTDBSqlParser.ShowDataNodesContext ctx) {
return new ShowDataNodesStatement();
}
+
+ @Override
+ public Statement visitCreateSchemaTemplate(IoTDBSqlParser.CreateSchemaTemplateContext ctx) {
+ CreateSchemaTemplateStatement createTemplateStatement = new CreateSchemaTemplateStatement();
+ String name = parseIdentifier(ctx.templateName.getText());
+ List<List<String>> measurementsList = new ArrayList<List<String>>();
+ List<List<TSDataType>> dataTypesList = new ArrayList<List<TSDataType>>();
+ List<List<TSEncoding>> encodingsList = new ArrayList<List<TSEncoding>>();
+ List<List<CompressionType>> compressorsList = new ArrayList<List<CompressionType>>();
+
+ if (ctx.ALIGNED() != null) {
+ // aligned
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ }
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ } else {
+ // non-aligned
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ // parseTemplateMeasurementClause(templateClauseContext, createTemplateStatement);
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ }
+ }
+
+ createTemplateStatement =
+ new CreateSchemaTemplateStatement(
+ name, measurementsList, dataTypesList, encodingsList, compressorsList);
+ return createTemplateStatement;
+ }
+
+ private void parseTemplateMeasurementClause(
+ IoTDBSqlParser.TemplateMeasurementClauseContext ctx,
+ CreateSchemaTemplateStatement createTemplateStatement) {
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ // single template measurement
+ measurements.add(parseNodeNameWithoutWildCard(ctx.nodeNameWithoutWildcard()));
+ parseAttributeClause(ctx.attributeClauses(), dataTypes, encodings, compressors);
+ // createTemplateStatement.addMeasurements(measurements);
+ // createTemplateStatement.addDataTypes(dataTypes);
+ // createTemplateStatement.addEncodings(encodings);
+ // createTemplateStatement.addCompressor(compressors);
+ }
+
+ void parseAttributeClause(
+ IoTDBSqlParser.AttributeClausesContext ctx,
+ List<TSDataType> dataTypes,
+ List<TSEncoding> encodings,
+ List<CompressionType> compressors) {
+ if (ctx.aliasNodeName() != null) {
+ throw new SQLParserException("schema template: alias is not supported yet.");
+ }
+
+ TSDataType dataType = null;
+ if (ctx.dataType != null) {
+ if (ctx.attributeKey() != null) {
+ if (!parseAttributeKey(ctx.attributeKey())
+ .equalsIgnoreCase(IoTDBConstant.COLUMN_TIMESERIES_DATATYPE)) {
+ throw new SQLParserException("expecting datatype");
+ }
+ }
+ String dataTypeString = ctx.dataType.getText().toUpperCase();
+ try {
+ dataType = TSDataType.valueOf(dataTypeString);
+ dataTypes.add(dataType);
+ } catch (Exception e) {
+ throw new SemanticException(String.format("unsupported datatype: %s", dataTypeString));
+ }
+ }
+
+ Map<String, String> props = new HashMap<>();
+ if (ctx.attributePair() != null) {
+ for (int i = 0; i < ctx.attributePair().size(); i++) {
+ props.put(
+ parseAttributeKey(ctx.attributePair(i).attributeKey()).toLowerCase(),
+ parseAttributeValue(ctx.attributePair(i).attributeValue()));
+ }
+ }
+
+ TSEncoding encoding = IoTDBDescriptor.getInstance().getDefaultEncodingByType(dataType);
+ if (props.containsKey(IoTDBConstant.COLUMN_TIMESERIES_ENCODING.toLowerCase())) {
+ String encodingString =
+ props.get(IoTDBConstant.COLUMN_TIMESERIES_ENCODING.toLowerCase()).toUpperCase();
+ try {
+ encoding = TSEncoding.valueOf(encodingString);
+ encodings.add(encoding);
+ props.remove(IoTDBConstant.COLUMN_TIMESERIES_ENCODING.toLowerCase());
+ } catch (Exception e) {
+ throw new SemanticException(String.format("unsupported encoding: %s", encodingString));
+ }
+ } else {
+ encodings.add(encoding);
+ }
+
+ CompressionType compressor = TSFileDescriptor.getInstance().getConfig().getCompressor();
+ if (props.containsKey(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSOR.toLowerCase())) {
+ String compressorString =
+ props.get(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSOR.toLowerCase()).toUpperCase();
+ try {
+ compressor = CompressionType.valueOf(compressorString);
+ compressors.add(compressor);
+ props.remove(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSOR.toLowerCase());
+ } catch (Exception e) {
+ throw new SemanticException(String.format("unsupported compressor: %s", compressorString));
+ }
+ } else if (props.containsKey(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION.toLowerCase())) {
+ String compressionString =
+ props.get(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION.toLowerCase()).toUpperCase();
+ try {
+ compressor = CompressionType.valueOf(compressionString);
+ compressors.add(compressor);
+ props.remove(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION.toLowerCase());
+ } catch (Exception e) {
+ throw new SemanticException(
+ String.format("unsupported compression: %s", compressionString));
+ }
+ } else {
+ compressors.add(compressor);
+ }
+
+ if (props.size() > 0) {
+ throw new SQLParserException("schema template: property is not supported yet.");
+ }
+
+ if (ctx.tagClause() != null) {
+ throw new SQLParserException("schema template: tag is not supported yet.");
+ }
+
+ if (ctx.attributeClause() != null) {
+ throw new SQLParserException("schema template: attribute is not supported yet.");
+ }
+ }
+
+ @Override
+ public Statement visitShowSchemaTemplates(IoTDBSqlParser.ShowSchemaTemplatesContext ctx) {
+ ShowSchemaTemplateStatement showSchemaTemplateStatement = new ShowSchemaTemplateStatement();
+ // createTemplateStatement.setType();
+ return showSchemaTemplateStatement;
+ }
+
+ @Override
+ public Statement visitShowNodesInSchemaTemplate(
+ IoTDBSqlParser.ShowNodesInSchemaTemplateContext ctx) {
+ String templateName = ctx.templateName.children.get(0).getText();
+ ShowNodesInSchemaTemplateStatement showNodesInSchemaTemplateStatement =
+ new ShowNodesInSchemaTemplateStatement(templateName);
+ // showNodesInSchemaTemplateStatement.setTemplateName(ctx.templateName.);
Review Comment:
Excellent work ! While it might be better with commented codes removed.
##########
confignode/src/main/java/org/apache/iotdb/confignode/consensus/request/ConfigPhysicalPlanType.java:
##########
@@ -65,6 +65,7 @@ public enum ConfigPhysicalPlanType {
CreateFunction,
DropFunction,
GetNodePathsPartition,
+ CreateSchemaTemplate,
Review Comment:
It's all good to add new entry at middle, while it's better to append it at the end to keep enumerations compatible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] bigreybear commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
bigreybear commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r915370253
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/statement/metadata/template/CreateSchemaTemplateStatement.java:
##########
@@ -0,0 +1,196 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.mpp.plan.statement.metadata.template;
+
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.db.mpp.plan.analyze.QueryType;
+import org.apache.iotdb.db.mpp.plan.constant.StatementType;
+import org.apache.iotdb.db.mpp.plan.statement.IConfigStatement;
+import org.apache.iotdb.db.mpp.plan.statement.Statement;
+import org.apache.iotdb.db.mpp.plan.statement.StatementVisitor;
+import org.apache.iotdb.tsfile.file.metadata.enums.CompressionType;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSEncoding;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class CreateSchemaTemplateStatement extends Statement implements IConfigStatement {
+
+ String name;
+ Set<String> alignedDeviceId;
+ String[] schemaNames;
Review Comment:
As far as I know, `schemaNames` is a historical issue, could be removed if not relevant in 0.14
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] liuminghui233 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
liuminghui233 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r915402632
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/parser/ASTVisitor.java:
##########
@@ -2311,4 +2314,178 @@ public Statement visitShowRegion(IoTDBSqlParser.ShowRegionContext ctx) {
public Statement visitShowDataNodes(IoTDBSqlParser.ShowDataNodesContext ctx) {
return new ShowDataNodesStatement();
}
+
+ @Override
+ public Statement visitCreateSchemaTemplate(IoTDBSqlParser.CreateSchemaTemplateContext ctx) {
+ CreateSchemaTemplateStatement createTemplateStatement = new CreateSchemaTemplateStatement();
+ String name = parseIdentifier(ctx.templateName.getText());
+ List<List<String>> measurementsList = new ArrayList<List<String>>();
+ List<List<TSDataType>> dataTypesList = new ArrayList<List<TSDataType>>();
+ List<List<TSEncoding>> encodingsList = new ArrayList<List<TSEncoding>>();
+ List<List<CompressionType>> compressorsList = new ArrayList<List<CompressionType>>();
+
+ if (ctx.ALIGNED() != null) {
+ // aligned
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ }
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ } else {
+ // non-aligned
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ // parseTemplateMeasurementClause(templateClauseContext, createTemplateStatement);
Review Comment:
remove comment?
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/parser/ASTVisitor.java:
##########
@@ -2311,4 +2314,178 @@ public Statement visitShowRegion(IoTDBSqlParser.ShowRegionContext ctx) {
public Statement visitShowDataNodes(IoTDBSqlParser.ShowDataNodesContext ctx) {
return new ShowDataNodesStatement();
}
+
+ @Override
+ public Statement visitCreateSchemaTemplate(IoTDBSqlParser.CreateSchemaTemplateContext ctx) {
+ CreateSchemaTemplateStatement createTemplateStatement = new CreateSchemaTemplateStatement();
+ String name = parseIdentifier(ctx.templateName.getText());
+ List<List<String>> measurementsList = new ArrayList<List<String>>();
+ List<List<TSDataType>> dataTypesList = new ArrayList<List<TSDataType>>();
+ List<List<TSEncoding>> encodingsList = new ArrayList<List<TSEncoding>>();
+ List<List<CompressionType>> compressorsList = new ArrayList<List<CompressionType>>();
+
+ if (ctx.ALIGNED() != null) {
+ // aligned
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ }
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ } else {
+ // non-aligned
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ // parseTemplateMeasurementClause(templateClauseContext, createTemplateStatement);
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ }
+ }
+
+ createTemplateStatement =
+ new CreateSchemaTemplateStatement(
+ name, measurementsList, dataTypesList, encodingsList, compressorsList);
+ return createTemplateStatement;
+ }
+
+ private void parseTemplateMeasurementClause(
+ IoTDBSqlParser.TemplateMeasurementClauseContext ctx,
+ CreateSchemaTemplateStatement createTemplateStatement) {
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ // single template measurement
+ measurements.add(parseNodeNameWithoutWildCard(ctx.nodeNameWithoutWildcard()));
+ parseAttributeClause(ctx.attributeClauses(), dataTypes, encodings, compressors);
+ // createTemplateStatement.addMeasurements(measurements);
+ // createTemplateStatement.addDataTypes(dataTypes);
+ // createTemplateStatement.addEncodings(encodings);
+ // createTemplateStatement.addCompressor(compressors);
+ }
+
+ void parseAttributeClause(
+ IoTDBSqlParser.AttributeClausesContext ctx,
+ List<TSDataType> dataTypes,
+ List<TSEncoding> encodings,
+ List<CompressionType> compressors) {
+ if (ctx.aliasNodeName() != null) {
+ throw new SQLParserException("schema template: alias is not supported yet.");
+ }
+
+ TSDataType dataType = null;
+ if (ctx.dataType != null) {
+ if (ctx.attributeKey() != null) {
+ if (!parseAttributeKey(ctx.attributeKey())
+ .equalsIgnoreCase(IoTDBConstant.COLUMN_TIMESERIES_DATATYPE)) {
+ throw new SQLParserException("expecting datatype");
+ }
+ }
+ String dataTypeString = ctx.dataType.getText().toUpperCase();
+ try {
+ dataType = TSDataType.valueOf(dataTypeString);
+ dataTypes.add(dataType);
+ } catch (Exception e) {
+ throw new SemanticException(String.format("unsupported datatype: %s", dataTypeString));
+ }
+ }
+
+ Map<String, String> props = new HashMap<>();
+ if (ctx.attributePair() != null) {
+ for (int i = 0; i < ctx.attributePair().size(); i++) {
+ props.put(
+ parseAttributeKey(ctx.attributePair(i).attributeKey()).toLowerCase(),
+ parseAttributeValue(ctx.attributePair(i).attributeValue()));
+ }
+ }
+
+ TSEncoding encoding = IoTDBDescriptor.getInstance().getDefaultEncodingByType(dataType);
+ if (props.containsKey(IoTDBConstant.COLUMN_TIMESERIES_ENCODING.toLowerCase())) {
+ String encodingString =
+ props.get(IoTDBConstant.COLUMN_TIMESERIES_ENCODING.toLowerCase()).toUpperCase();
+ try {
+ encoding = TSEncoding.valueOf(encodingString);
+ encodings.add(encoding);
+ props.remove(IoTDBConstant.COLUMN_TIMESERIES_ENCODING.toLowerCase());
+ } catch (Exception e) {
+ throw new SemanticException(String.format("unsupported encoding: %s", encodingString));
+ }
+ } else {
+ encodings.add(encoding);
+ }
+
+ CompressionType compressor = TSFileDescriptor.getInstance().getConfig().getCompressor();
+ if (props.containsKey(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSOR.toLowerCase())) {
+ String compressorString =
+ props.get(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSOR.toLowerCase()).toUpperCase();
+ try {
+ compressor = CompressionType.valueOf(compressorString);
+ compressors.add(compressor);
+ props.remove(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSOR.toLowerCase());
+ } catch (Exception e) {
+ throw new SemanticException(String.format("unsupported compressor: %s", compressorString));
+ }
+ } else if (props.containsKey(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION.toLowerCase())) {
+ String compressionString =
+ props.get(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION.toLowerCase()).toUpperCase();
+ try {
+ compressor = CompressionType.valueOf(compressionString);
+ compressors.add(compressor);
+ props.remove(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION.toLowerCase());
+ } catch (Exception e) {
+ throw new SemanticException(
+ String.format("unsupported compression: %s", compressionString));
+ }
+ } else {
+ compressors.add(compressor);
+ }
+
+ if (props.size() > 0) {
+ throw new SQLParserException("schema template: property is not supported yet.");
+ }
+
+ if (ctx.tagClause() != null) {
+ throw new SQLParserException("schema template: tag is not supported yet.");
+ }
+
+ if (ctx.attributeClause() != null) {
+ throw new SQLParserException("schema template: attribute is not supported yet.");
+ }
+ }
+
+ @Override
+ public Statement visitShowSchemaTemplates(IoTDBSqlParser.ShowSchemaTemplatesContext ctx) {
+ ShowSchemaTemplateStatement showSchemaTemplateStatement = new ShowSchemaTemplateStatement();
+ // createTemplateStatement.setType();
Review Comment:
remove this comment?
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/parser/ASTVisitor.java:
##########
@@ -2311,4 +2314,178 @@ public Statement visitShowRegion(IoTDBSqlParser.ShowRegionContext ctx) {
public Statement visitShowDataNodes(IoTDBSqlParser.ShowDataNodesContext ctx) {
return new ShowDataNodesStatement();
}
+
+ @Override
+ public Statement visitCreateSchemaTemplate(IoTDBSqlParser.CreateSchemaTemplateContext ctx) {
+ CreateSchemaTemplateStatement createTemplateStatement = new CreateSchemaTemplateStatement();
+ String name = parseIdentifier(ctx.templateName.getText());
+ List<List<String>> measurementsList = new ArrayList<List<String>>();
+ List<List<TSDataType>> dataTypesList = new ArrayList<List<TSDataType>>();
+ List<List<TSEncoding>> encodingsList = new ArrayList<List<TSEncoding>>();
+ List<List<CompressionType>> compressorsList = new ArrayList<List<CompressionType>>();
+
+ if (ctx.ALIGNED() != null) {
+ // aligned
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ }
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ } else {
+ // non-aligned
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ // parseTemplateMeasurementClause(templateClauseContext, createTemplateStatement);
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ }
+ }
+
+ createTemplateStatement =
+ new CreateSchemaTemplateStatement(
+ name, measurementsList, dataTypesList, encodingsList, compressorsList);
+ return createTemplateStatement;
+ }
+
+ private void parseTemplateMeasurementClause(
Review Comment:
this method may be useless
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916434366
##########
confignode/src/main/java/org/apache/iotdb/confignode/persistence/TemplateInfo.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.persistence;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.snapshot.SnapshotProcessor;
+import org.apache.iotdb.commons.utils.TestOnly;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.rpc.TSStatusCode;
+import org.apache.iotdb.tsfile.utils.ReadWriteIOUtils;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class TemplateInfo implements SnapshotProcessor {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(TemplateInfo.class);
+
+ // StorageGroup read write lock
+ private final ReentrantReadWriteLock templateReadWriteLock;
+
+ private final Map<String, Template> templateMap = new ConcurrentHashMap<>();
+
+ private final String snapshotFileName = "template_info.bin";
+
+ public TemplateInfo() throws IOException {
+ templateReadWriteLock = new ReentrantReadWriteLock();
+ }
+
+ public TGetTemplateResp getMatchedTemplateByName(String name) {
+ TGetTemplateResp resp = new TGetTemplateResp();
+ try {
+ templateReadWriteLock.readLock().lock();
+ Template template = templateMap.get(name);
+ resp.setStatus(new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode()));
+ resp.setTemplate(Template.template2ByteBuffer(template));
+ } catch (Exception e) {
+ LOGGER.warn("Error TemplateInfo name", e);
+ resp.setStatus(new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()));
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
+ return resp;
+ }
+
+ public TGetAllTemplatesResp getAllTemplate() {
+ TGetAllTemplatesResp resp = new TGetAllTemplatesResp();
+ try {
+ templateReadWriteLock.readLock().lock();
+ List<ByteBuffer> templates = new ArrayList<>();
+ this.templateMap.values().stream()
+ .forEach(
+ item -> {
+ templates.add(Template.template2ByteBuffer(item));
+ });
+ resp.setTemplateList(templates);
+ resp.setStatus(new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode()));
+ } catch (Exception e) {
+ LOGGER.warn("Error TemplateInfo name", e);
+ resp.setStatus(new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()));
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
+ return resp;
+ }
+
+ public TSStatus createTemplate(CreateSchemaTemplatePlan createSchemaTemplatePlan) {
+ try {
+ templateReadWriteLock.readLock().lock();
+ Template template =
+ Template.byteBuffer2Template(ByteBuffer.wrap(createSchemaTemplatePlan.getTemplate()));
+ Template temp = this.templateMap.get(template.getName());
+ if (temp != null && template.getName().equalsIgnoreCase(temp.getName())) {
+ LOGGER.error(
+ "Failed to create template, because template name {} is exists", template.getName());
+ return new TSStatus(TSStatusCode.DUPLICATED_TEMPLATE.getStatusCode());
+ }
+ this.templateMap.put(template.getName(), template);
+ return new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode());
+ } catch (Exception e) {
+ LOGGER.warn("Error to create template", e);
+ return new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode());
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
+ }
+
+ private void serialize(OutputStream outputStream) throws IOException {
+ ReadWriteIOUtils.write(templateMap.size(), outputStream);
+ for (Map.Entry<String, Template> entry : templateMap.entrySet()) {
+ serializeTemplate(entry.getValue(), outputStream);
+ }
+ }
+
+ private void serializeTemplate(Template template, OutputStream outputStream) {
+ // SerializeUtils.serialize(template.getName(), outputStream);
+ try {
+ ByteBuffer dataBuffer = template.serialize();
+ // byte[] data = dataBuffer.array();
+ // int length = data.length;
+ // ReadWriteIOUtils.write(length,outputStream);
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916432863
##########
confignode/src/main/java/org/apache/iotdb/confignode/persistence/TemplateInfo.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.persistence;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.snapshot.SnapshotProcessor;
+import org.apache.iotdb.commons.utils.TestOnly;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.rpc.TSStatusCode;
+import org.apache.iotdb.tsfile.utils.ReadWriteIOUtils;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class TemplateInfo implements SnapshotProcessor {
Review Comment:
Appand method in ClusterSchemaInfo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] CRZbulabula commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
CRZbulabula commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r915465929
##########
confignode/src/main/java/org/apache/iotdb/confignode/manager/ConfigManager.java:
##########
@@ -862,4 +878,36 @@ public void addMetrics() {
partitionManager.addMetrics();
nodeManager.addMetrics();
}
+
+ @Override
+ public TSStatus createSchemaTemplate(TCreateSchemaTemplateReq req) {
+ TSStatus status = confirmLeader();
+ if (status.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+ CreateSchemaTemplatePlan createSchemaTemplatePlan =
+ new CreateSchemaTemplatePlan(req.getSerializedTemplate());
+ return templateManager.createTemplate(createSchemaTemplatePlan);
+ } else {
+ return status;
+ }
+ }
+
+ @Override
+ public TGetAllTemplatesResp getAllTemplates() {
+ TSStatus status = confirmLeader();
+ if (status.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+ return templateManager.getAllTemplates();
+ } else {
+ return new TGetAllTemplatesResp();
+ }
+ }
+
+ @Override
+ public TGetTemplateResp getTemplate(String req) {
+ TSStatus status = confirmLeader();
+ if (status.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+ return templateManager.getTemplate(req);
+ } else {
+ return new TGetTemplateResp();
Review Comment:
`return new TGetTemplateResp().setStatus(status)`
##########
confignode/src/main/java/org/apache/iotdb/confignode/persistence/TemplateInfo.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.persistence;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.snapshot.SnapshotProcessor;
+import org.apache.iotdb.commons.utils.TestOnly;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.rpc.TSStatusCode;
+import org.apache.iotdb.tsfile.utils.ReadWriteIOUtils;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class TemplateInfo implements SnapshotProcessor {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(TemplateInfo.class);
+
+ // StorageGroup read write lock
+ private final ReentrantReadWriteLock templateReadWriteLock;
+
+ private final Map<String, Template> templateMap = new ConcurrentHashMap<>();
+
+ private final String snapshotFileName = "template_info.bin";
+
+ public TemplateInfo() throws IOException {
+ templateReadWriteLock = new ReentrantReadWriteLock();
+ }
+
+ public TGetTemplateResp getMatchedTemplateByName(String name) {
+ TGetTemplateResp resp = new TGetTemplateResp();
+ try {
+ templateReadWriteLock.readLock().lock();
+ Template template = templateMap.get(name);
+ resp.setStatus(new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode()));
+ resp.setTemplate(Template.template2ByteBuffer(template));
+ } catch (Exception e) {
+ LOGGER.warn("Error TemplateInfo name", e);
+ resp.setStatus(new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()));
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
+ return resp;
+ }
+
+ public TGetAllTemplatesResp getAllTemplate() {
+ TGetAllTemplatesResp resp = new TGetAllTemplatesResp();
+ try {
+ templateReadWriteLock.readLock().lock();
+ List<ByteBuffer> templates = new ArrayList<>();
+ this.templateMap.values().stream()
+ .forEach(
+ item -> {
+ templates.add(Template.template2ByteBuffer(item));
+ });
+ resp.setTemplateList(templates);
+ resp.setStatus(new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode()));
+ } catch (Exception e) {
+ LOGGER.warn("Error TemplateInfo name", e);
+ resp.setStatus(new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()));
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
+ return resp;
+ }
+
+ public TSStatus createTemplate(CreateSchemaTemplatePlan createSchemaTemplatePlan) {
+ try {
+ templateReadWriteLock.readLock().lock();
+ Template template =
+ Template.byteBuffer2Template(ByteBuffer.wrap(createSchemaTemplatePlan.getTemplate()));
+ Template temp = this.templateMap.get(template.getName());
+ if (temp != null && template.getName().equalsIgnoreCase(temp.getName())) {
+ LOGGER.error(
+ "Failed to create template, because template name {} is exists", template.getName());
+ return new TSStatus(TSStatusCode.DUPLICATED_TEMPLATE.getStatusCode());
+ }
+ this.templateMap.put(template.getName(), template);
+ return new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode());
+ } catch (Exception e) {
+ LOGGER.warn("Error to create template", e);
+ return new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode());
+ } finally {
Review Comment:
The same as before
##########
confignode/src/main/java/org/apache/iotdb/confignode/persistence/TemplateInfo.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.persistence;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.snapshot.SnapshotProcessor;
+import org.apache.iotdb.commons.utils.TestOnly;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.rpc.TSStatusCode;
+import org.apache.iotdb.tsfile.utils.ReadWriteIOUtils;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class TemplateInfo implements SnapshotProcessor {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(TemplateInfo.class);
+
+ // StorageGroup read write lock
+ private final ReentrantReadWriteLock templateReadWriteLock;
+
+ private final Map<String, Template> templateMap = new ConcurrentHashMap<>();
+
+ private final String snapshotFileName = "template_info.bin";
+
+ public TemplateInfo() throws IOException {
+ templateReadWriteLock = new ReentrantReadWriteLock();
+ }
+
+ public TGetTemplateResp getMatchedTemplateByName(String name) {
+ TGetTemplateResp resp = new TGetTemplateResp();
+ try {
+ templateReadWriteLock.readLock().lock();
+ Template template = templateMap.get(name);
+ resp.setStatus(new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode()));
+ resp.setTemplate(Template.template2ByteBuffer(template));
+ } catch (Exception e) {
+ LOGGER.warn("Error TemplateInfo name", e);
+ resp.setStatus(new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()));
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
+ return resp;
+ }
+
+ public TGetAllTemplatesResp getAllTemplate() {
+ TGetAllTemplatesResp resp = new TGetAllTemplatesResp();
+ try {
+ templateReadWriteLock.readLock().lock();
+ List<ByteBuffer> templates = new ArrayList<>();
+ this.templateMap.values().stream()
+ .forEach(
+ item -> {
+ templates.add(Template.template2ByteBuffer(item));
+ });
+ resp.setTemplateList(templates);
+ resp.setStatus(new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode()));
+ } catch (Exception e) {
+ LOGGER.warn("Error TemplateInfo name", e);
+ resp.setStatus(new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()));
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
+ return resp;
+ }
+
+ public TSStatus createTemplate(CreateSchemaTemplatePlan createSchemaTemplatePlan) {
+ try {
+ templateReadWriteLock.readLock().lock();
+ Template template =
+ Template.byteBuffer2Template(ByteBuffer.wrap(createSchemaTemplatePlan.getTemplate()));
+ Template temp = this.templateMap.get(template.getName());
+ if (temp != null && template.getName().equalsIgnoreCase(temp.getName())) {
+ LOGGER.error(
+ "Failed to create template, because template name {} is exists", template.getName());
+ return new TSStatus(TSStatusCode.DUPLICATED_TEMPLATE.getStatusCode());
+ }
+ this.templateMap.put(template.getName(), template);
+ return new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode());
+ } catch (Exception e) {
+ LOGGER.warn("Error to create template", e);
+ return new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode());
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
+ }
+
+ private void serialize(OutputStream outputStream) throws IOException {
+ ReadWriteIOUtils.write(templateMap.size(), outputStream);
+ for (Map.Entry<String, Template> entry : templateMap.entrySet()) {
+ serializeTemplate(entry.getValue(), outputStream);
+ }
+ }
+
+ private void serializeTemplate(Template template, OutputStream outputStream) {
+ // SerializeUtils.serialize(template.getName(), outputStream);
+ try {
+ ByteBuffer dataBuffer = template.serialize();
+ // byte[] data = dataBuffer.array();
+ // int length = data.length;
+ // ReadWriteIOUtils.write(length,outputStream);
Review Comment:
Remove these lines
##########
confignode/src/main/java/org/apache/iotdb/confignode/service/thrift/ConfigNodeRPCServiceProcessor.java:
##########
@@ -494,4 +497,20 @@ public TShowDataNodesResp showDataNodes() throws TException {
public void handleClientExit() {}
// TODO: Interfaces for data operations
+
+ @Override
+ public TSStatus createSchemaTemplate(TCreateSchemaTemplateReq req) throws TException {
+ return configManager.createSchemaTemplate(req);
+ }
+
+ @Override
+ public TGetAllTemplatesResp getAllTemplates() throws TException {
+ return configManager.getAllTemplates();
+ }
+
+ @Override
+ public TGetTemplateResp getTemplate(String req) throws TException {
+
Review Comment:
Remove this empty line
##########
confignode/src/main/java/org/apache/iotdb/confignode/manager/ConfigManager.java:
##########
@@ -862,4 +878,36 @@ public void addMetrics() {
partitionManager.addMetrics();
nodeManager.addMetrics();
}
+
+ @Override
+ public TSStatus createSchemaTemplate(TCreateSchemaTemplateReq req) {
+ TSStatus status = confirmLeader();
+ if (status.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+ CreateSchemaTemplatePlan createSchemaTemplatePlan =
+ new CreateSchemaTemplatePlan(req.getSerializedTemplate());
+ return templateManager.createTemplate(createSchemaTemplatePlan);
+ } else {
+ return status;
+ }
+ }
+
+ @Override
+ public TGetAllTemplatesResp getAllTemplates() {
+ TSStatus status = confirmLeader();
+ if (status.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+ return templateManager.getAllTemplates();
+ } else {
+ return new TGetAllTemplatesResp();
Review Comment:
`return new TGetAllTemplatesResp().setStatus(status)`
##########
confignode/src/main/java/org/apache/iotdb/confignode/manager/TemplateManager.java:
##########
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.manager;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.persistence.TemplateInfo;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class TemplateManager {
Review Comment:
We've decided that the management of cluster Template should been done by ClusterSchemaManager. Thus, there is no need to create a new TemplateManager. You can see the [ConfigNode framework](https://apache-iotdb.feishu.cn/docs/doccnUzGLXivqOoNjAo8Ole3cQe) doc for more our design details.
##########
confignode/src/main/java/org/apache/iotdb/confignode/persistence/TemplateInfo.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.persistence;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.snapshot.SnapshotProcessor;
+import org.apache.iotdb.commons.utils.TestOnly;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.rpc.TSStatusCode;
+import org.apache.iotdb.tsfile.utils.ReadWriteIOUtils;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class TemplateInfo implements SnapshotProcessor {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(TemplateInfo.class);
+
+ // StorageGroup read write lock
+ private final ReentrantReadWriteLock templateReadWriteLock;
+
+ private final Map<String, Template> templateMap = new ConcurrentHashMap<>();
+
+ private final String snapshotFileName = "template_info.bin";
+
+ public TemplateInfo() throws IOException {
+ templateReadWriteLock = new ReentrantReadWriteLock();
+ }
+
+ public TGetTemplateResp getMatchedTemplateByName(String name) {
+ TGetTemplateResp resp = new TGetTemplateResp();
+ try {
+ templateReadWriteLock.readLock().lock();
+ Template template = templateMap.get(name);
+ resp.setStatus(new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode()));
+ resp.setTemplate(Template.template2ByteBuffer(template));
+ } catch (Exception e) {
+ LOGGER.warn("Error TemplateInfo name", e);
+ resp.setStatus(new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()));
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
Review Comment:
Don't use `catch Exception`. Catch different exceptions to distinguish different scenarios would be better. For example, define a new TSStatusCode TEMPLATE_NOT_EXIST to represent the scenario that the required Template doesn't exist.
##########
confignode/src/main/java/org/apache/iotdb/confignode/persistence/TemplateInfo.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.persistence;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.snapshot.SnapshotProcessor;
+import org.apache.iotdb.commons.utils.TestOnly;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.rpc.TSStatusCode;
+import org.apache.iotdb.tsfile.utils.ReadWriteIOUtils;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class TemplateInfo implements SnapshotProcessor {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(TemplateInfo.class);
+
+ // StorageGroup read write lock
+ private final ReentrantReadWriteLock templateReadWriteLock;
+
+ private final Map<String, Template> templateMap = new ConcurrentHashMap<>();
+
+ private final String snapshotFileName = "template_info.bin";
+
+ public TemplateInfo() throws IOException {
+ templateReadWriteLock = new ReentrantReadWriteLock();
+ }
+
+ public TGetTemplateResp getMatchedTemplateByName(String name) {
+ TGetTemplateResp resp = new TGetTemplateResp();
+ try {
+ templateReadWriteLock.readLock().lock();
+ Template template = templateMap.get(name);
+ resp.setStatus(new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode()));
+ resp.setTemplate(Template.template2ByteBuffer(template));
+ } catch (Exception e) {
+ LOGGER.warn("Error TemplateInfo name", e);
+ resp.setStatus(new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()));
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
+ return resp;
+ }
+
+ public TGetAllTemplatesResp getAllTemplate() {
+ TGetAllTemplatesResp resp = new TGetAllTemplatesResp();
+ try {
+ templateReadWriteLock.readLock().lock();
+ List<ByteBuffer> templates = new ArrayList<>();
+ this.templateMap.values().stream()
+ .forEach(
+ item -> {
+ templates.add(Template.template2ByteBuffer(item));
+ });
+ resp.setTemplateList(templates);
+ resp.setStatus(new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode()));
+ } catch (Exception e) {
+ LOGGER.warn("Error TemplateInfo name", e);
+ resp.setStatus(new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()));
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
Review Comment:
The same as before
##########
confignode/src/main/java/org/apache/iotdb/confignode/persistence/TemplateInfo.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.persistence;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.snapshot.SnapshotProcessor;
+import org.apache.iotdb.commons.utils.TestOnly;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.rpc.TSStatusCode;
+import org.apache.iotdb.tsfile.utils.ReadWriteIOUtils;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class TemplateInfo implements SnapshotProcessor {
Review Comment:
We've decided that the storage of cluster Template should been done by ClusterSchemaInfo. Thus, there is no need to create a new TemplateInfo. You can see the [ConfigNode framework](https://apache-iotdb.feishu.cn/docs/doccnUzGLXivqOoNjAo8Ole3cQe) doc for more our design details.
##########
confignode/src/main/java/org/apache/iotdb/confignode/manager/IManager.java:
##########
@@ -264,4 +267,10 @@ TDataPartitionResp getOrCreateDataPartition(
/** Show datanodes */
DataSet showDataNodes();
+
+ TSStatus createSchemaTemplate(TCreateSchemaTemplateReq req);
+
+ TGetAllTemplatesResp getAllTemplates();
+
+ TGetTemplateResp getTemplate(String req);
Review Comment:
Add interface annotation to show what these interfaces do
##########
confignode/src/main/java/org/apache/iotdb/confignode/consensus/request/write/CreateSchemaTemplatePlan.java:
##########
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.consensus.request.write;
+
+import org.apache.iotdb.confignode.consensus.request.ConfigPhysicalPlan;
+import org.apache.iotdb.confignode.consensus.request.ConfigPhysicalPlanType;
+import org.apache.iotdb.tsfile.utils.ReadWriteIOUtils;
+
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.Objects;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/7/5
+ */
+public class CreateSchemaTemplatePlan extends ConfigPhysicalPlan {
+
+ private byte[] template;
+
+ public CreateSchemaTemplatePlan() {
+ super(ConfigPhysicalPlanType.CreateSchemaTemplate);
+ }
+
+ public CreateSchemaTemplatePlan(byte[] template) {
+ this();
+ this.template = template;
+ }
+
+ public byte[] getTemplate() {
+ return template;
+ }
+
+ @Override
+ protected void serializeImpl(DataOutputStream stream) throws IOException {
+ stream.writeInt(ConfigPhysicalPlanType.CreateSchemaTemplate.ordinal());
+ stream.writeInt(template.length);
+ stream.write(template);
+ }
+
+ @Override
+ protected void deserializeImpl(ByteBuffer buffer) throws IOException {
+ int length = ReadWriteIOUtils.readInt(buffer);
+ this.template = ReadWriteIOUtils.readBytes(buffer, length);
+ }
Review Comment:
Add serialize and deserialize tests in ConfigPhysicalPlanSerDeTest
##########
confignode/src/main/java/org/apache/iotdb/confignode/persistence/TemplateInfo.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.persistence;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.snapshot.SnapshotProcessor;
+import org.apache.iotdb.commons.utils.TestOnly;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.rpc.TSStatusCode;
+import org.apache.iotdb.tsfile.utils.ReadWriteIOUtils;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class TemplateInfo implements SnapshotProcessor {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(TemplateInfo.class);
+
+ // StorageGroup read write lock
+ private final ReentrantReadWriteLock templateReadWriteLock;
+
+ private final Map<String, Template> templateMap = new ConcurrentHashMap<>();
+
+ private final String snapshotFileName = "template_info.bin";
+
+ public TemplateInfo() throws IOException {
+ templateReadWriteLock = new ReentrantReadWriteLock();
+ }
+
+ public TGetTemplateResp getMatchedTemplateByName(String name) {
+ TGetTemplateResp resp = new TGetTemplateResp();
+ try {
+ templateReadWriteLock.readLock().lock();
+ Template template = templateMap.get(name);
+ resp.setStatus(new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode()));
+ resp.setTemplate(Template.template2ByteBuffer(template));
+ } catch (Exception e) {
+ LOGGER.warn("Error TemplateInfo name", e);
+ resp.setStatus(new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()));
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
+ return resp;
+ }
+
+ public TGetAllTemplatesResp getAllTemplate() {
+ TGetAllTemplatesResp resp = new TGetAllTemplatesResp();
+ try {
+ templateReadWriteLock.readLock().lock();
+ List<ByteBuffer> templates = new ArrayList<>();
+ this.templateMap.values().stream()
+ .forEach(
+ item -> {
+ templates.add(Template.template2ByteBuffer(item));
+ });
+ resp.setTemplateList(templates);
+ resp.setStatus(new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode()));
+ } catch (Exception e) {
+ LOGGER.warn("Error TemplateInfo name", e);
+ resp.setStatus(new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()));
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
+ return resp;
+ }
+
+ public TSStatus createTemplate(CreateSchemaTemplatePlan createSchemaTemplatePlan) {
+ try {
+ templateReadWriteLock.readLock().lock();
+ Template template =
+ Template.byteBuffer2Template(ByteBuffer.wrap(createSchemaTemplatePlan.getTemplate()));
+ Template temp = this.templateMap.get(template.getName());
+ if (temp != null && template.getName().equalsIgnoreCase(temp.getName())) {
+ LOGGER.error(
+ "Failed to create template, because template name {} is exists", template.getName());
+ return new TSStatus(TSStatusCode.DUPLICATED_TEMPLATE.getStatusCode());
+ }
+ this.templateMap.put(template.getName(), template);
+ return new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode());
+ } catch (Exception e) {
+ LOGGER.warn("Error to create template", e);
+ return new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode());
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
+ }
+
+ private void serialize(OutputStream outputStream) throws IOException {
+ ReadWriteIOUtils.write(templateMap.size(), outputStream);
+ for (Map.Entry<String, Template> entry : templateMap.entrySet()) {
+ serializeTemplate(entry.getValue(), outputStream);
+ }
+ }
+
+ private void serializeTemplate(Template template, OutputStream outputStream) {
+ // SerializeUtils.serialize(template.getName(), outputStream);
Review Comment:
Remove this line
##########
confignode/src/main/java/org/apache/iotdb/confignode/manager/TemplateManager.java:
##########
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.manager;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.persistence.TemplateInfo;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class TemplateManager {
+ private static final Logger LOGGER = LoggerFactory.getLogger(TemplateManager.class);
+
+ private final IManager configManager;
+ private final TemplateInfo templateInfo;
+
+ public TemplateManager(IManager configManager, TemplateInfo templateInfo) {
+ this.configManager = configManager;
+ this.templateInfo = templateInfo;
+ }
+
+ public TSStatus createTemplate(CreateSchemaTemplatePlan createSchemaTemplatePlan) {
+ try {
+ TSStatus tsStatus = getConsensusManager().write(createSchemaTemplatePlan).getStatus();
+ return tsStatus;
+ } catch (Exception e) {
+ return new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode())
+ .setMessage("schema template failed to create because some error.");
+ }
+ }
+
+ public TGetAllTemplatesResp getAllTemplates() {
+ return templateInfo.getAllTemplate();
+ }
+
+ TGetTemplateResp getTemplate(String req) {
+ return templateInfo.getMatchedTemplateByName(req);
+ }
Review Comment:
It's better to use consensus read for these two interfaces
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916434257
##########
confignode/src/main/java/org/apache/iotdb/confignode/persistence/TemplateInfo.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.persistence;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.snapshot.SnapshotProcessor;
+import org.apache.iotdb.commons.utils.TestOnly;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.rpc.TSStatusCode;
+import org.apache.iotdb.tsfile.utils.ReadWriteIOUtils;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class TemplateInfo implements SnapshotProcessor {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(TemplateInfo.class);
+
+ // StorageGroup read write lock
+ private final ReentrantReadWriteLock templateReadWriteLock;
+
+ private final Map<String, Template> templateMap = new ConcurrentHashMap<>();
+
+ private final String snapshotFileName = "template_info.bin";
+
+ public TemplateInfo() throws IOException {
+ templateReadWriteLock = new ReentrantReadWriteLock();
+ }
+
+ public TGetTemplateResp getMatchedTemplateByName(String name) {
+ TGetTemplateResp resp = new TGetTemplateResp();
+ try {
+ templateReadWriteLock.readLock().lock();
+ Template template = templateMap.get(name);
+ resp.setStatus(new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode()));
+ resp.setTemplate(Template.template2ByteBuffer(template));
+ } catch (Exception e) {
+ LOGGER.warn("Error TemplateInfo name", e);
+ resp.setStatus(new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()));
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
+ return resp;
+ }
+
+ public TGetAllTemplatesResp getAllTemplate() {
+ TGetAllTemplatesResp resp = new TGetAllTemplatesResp();
+ try {
+ templateReadWriteLock.readLock().lock();
+ List<ByteBuffer> templates = new ArrayList<>();
+ this.templateMap.values().stream()
+ .forEach(
+ item -> {
+ templates.add(Template.template2ByteBuffer(item));
+ });
+ resp.setTemplateList(templates);
+ resp.setStatus(new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode()));
+ } catch (Exception e) {
+ LOGGER.warn("Error TemplateInfo name", e);
+ resp.setStatus(new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()));
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] MarcosZyk merged pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
MarcosZyk merged PR #6608:
URL: https://github.com/apache/iotdb/pull/6608
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916431159
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/parser/ASTVisitor.java:
##########
@@ -2311,4 +2314,178 @@ public Statement visitShowRegion(IoTDBSqlParser.ShowRegionContext ctx) {
public Statement visitShowDataNodes(IoTDBSqlParser.ShowDataNodesContext ctx) {
return new ShowDataNodesStatement();
}
+
+ @Override
+ public Statement visitCreateSchemaTemplate(IoTDBSqlParser.CreateSchemaTemplateContext ctx) {
+ CreateSchemaTemplateStatement createTemplateStatement = new CreateSchemaTemplateStatement();
+ String name = parseIdentifier(ctx.templateName.getText());
+ List<List<String>> measurementsList = new ArrayList<List<String>>();
+ List<List<TSDataType>> dataTypesList = new ArrayList<List<TSDataType>>();
+ List<List<TSEncoding>> encodingsList = new ArrayList<List<TSEncoding>>();
+ List<List<CompressionType>> compressorsList = new ArrayList<List<CompressionType>>();
+
+ if (ctx.ALIGNED() != null) {
+ // aligned
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ }
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ } else {
+ // non-aligned
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ // parseTemplateMeasurementClause(templateClauseContext, createTemplateStatement);
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ }
+ }
+
+ createTemplateStatement =
+ new CreateSchemaTemplateStatement(
+ name, measurementsList, dataTypesList, encodingsList, compressorsList);
+ return createTemplateStatement;
+ }
+
+ private void parseTemplateMeasurementClause(
Review Comment:
already removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916430910
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/parser/ASTVisitor.java:
##########
@@ -2311,4 +2314,178 @@ public Statement visitShowRegion(IoTDBSqlParser.ShowRegionContext ctx) {
public Statement visitShowDataNodes(IoTDBSqlParser.ShowDataNodesContext ctx) {
return new ShowDataNodesStatement();
}
+
+ @Override
+ public Statement visitCreateSchemaTemplate(IoTDBSqlParser.CreateSchemaTemplateContext ctx) {
+ CreateSchemaTemplateStatement createTemplateStatement = new CreateSchemaTemplateStatement();
+ String name = parseIdentifier(ctx.templateName.getText());
+ List<List<String>> measurementsList = new ArrayList<List<String>>();
+ List<List<TSDataType>> dataTypesList = new ArrayList<List<TSDataType>>();
+ List<List<TSEncoding>> encodingsList = new ArrayList<List<TSEncoding>>();
+ List<List<CompressionType>> compressorsList = new ArrayList<List<CompressionType>>();
+
+ if (ctx.ALIGNED() != null) {
+ // aligned
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ }
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ } else {
+ // non-aligned
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ // parseTemplateMeasurementClause(templateClauseContext, createTemplateStatement);
Review Comment:
already removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916433175
##########
confignode/src/main/java/org/apache/iotdb/confignode/manager/ConfigManager.java:
##########
@@ -862,4 +878,36 @@ public void addMetrics() {
partitionManager.addMetrics();
nodeManager.addMetrics();
}
+
+ @Override
+ public TSStatus createSchemaTemplate(TCreateSchemaTemplateReq req) {
+ TSStatus status = confirmLeader();
+ if (status.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+ CreateSchemaTemplatePlan createSchemaTemplatePlan =
+ new CreateSchemaTemplatePlan(req.getSerializedTemplate());
+ return templateManager.createTemplate(createSchemaTemplatePlan);
+ } else {
+ return status;
+ }
+ }
+
+ @Override
+ public TGetAllTemplatesResp getAllTemplates() {
+ TSStatus status = confirmLeader();
+ if (status.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+ return templateManager.getAllTemplates();
+ } else {
+ return new TGetAllTemplatesResp();
+ }
+ }
+
+ @Override
+ public TGetTemplateResp getTemplate(String req) {
+ TSStatus status = confirmLeader();
+ if (status.getCode() == TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
+ return templateManager.getTemplate(req);
+ } else {
+ return new TGetTemplateResp();
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916430166
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/parser/ASTVisitor.java:
##########
@@ -2311,4 +2314,178 @@ public Statement visitShowRegion(IoTDBSqlParser.ShowRegionContext ctx) {
public Statement visitShowDataNodes(IoTDBSqlParser.ShowDataNodesContext ctx) {
return new ShowDataNodesStatement();
}
+
+ @Override
+ public Statement visitCreateSchemaTemplate(IoTDBSqlParser.CreateSchemaTemplateContext ctx) {
+ CreateSchemaTemplateStatement createTemplateStatement = new CreateSchemaTemplateStatement();
+ String name = parseIdentifier(ctx.templateName.getText());
+ List<List<String>> measurementsList = new ArrayList<List<String>>();
+ List<List<TSDataType>> dataTypesList = new ArrayList<List<TSDataType>>();
+ List<List<TSEncoding>> encodingsList = new ArrayList<List<TSEncoding>>();
+ List<List<CompressionType>> compressorsList = new ArrayList<List<CompressionType>>();
+
+ if (ctx.ALIGNED() != null) {
+ // aligned
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ }
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ } else {
+ // non-aligned
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ // parseTemplateMeasurementClause(templateClauseContext, createTemplateStatement);
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ }
+ }
+
+ createTemplateStatement =
+ new CreateSchemaTemplateStatement(
+ name, measurementsList, dataTypesList, encodingsList, compressorsList);
+ return createTemplateStatement;
+ }
+
+ private void parseTemplateMeasurementClause(
+ IoTDBSqlParser.TemplateMeasurementClauseContext ctx,
+ CreateSchemaTemplateStatement createTemplateStatement) {
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ // single template measurement
+ measurements.add(parseNodeNameWithoutWildCard(ctx.nodeNameWithoutWildcard()));
+ parseAttributeClause(ctx.attributeClauses(), dataTypes, encodings, compressors);
+ // createTemplateStatement.addMeasurements(measurements);
+ // createTemplateStatement.addDataTypes(dataTypes);
+ // createTemplateStatement.addEncodings(encodings);
+ // createTemplateStatement.addCompressor(compressors);
+ }
+
+ void parseAttributeClause(
+ IoTDBSqlParser.AttributeClausesContext ctx,
+ List<TSDataType> dataTypes,
+ List<TSEncoding> encodings,
+ List<CompressionType> compressors) {
+ if (ctx.aliasNodeName() != null) {
+ throw new SQLParserException("schema template: alias is not supported yet.");
+ }
+
+ TSDataType dataType = null;
+ if (ctx.dataType != null) {
+ if (ctx.attributeKey() != null) {
+ if (!parseAttributeKey(ctx.attributeKey())
+ .equalsIgnoreCase(IoTDBConstant.COLUMN_TIMESERIES_DATATYPE)) {
+ throw new SQLParserException("expecting datatype");
+ }
+ }
+ String dataTypeString = ctx.dataType.getText().toUpperCase();
+ try {
+ dataType = TSDataType.valueOf(dataTypeString);
+ dataTypes.add(dataType);
+ } catch (Exception e) {
+ throw new SemanticException(String.format("unsupported datatype: %s", dataTypeString));
+ }
+ }
+
+ Map<String, String> props = new HashMap<>();
+ if (ctx.attributePair() != null) {
+ for (int i = 0; i < ctx.attributePair().size(); i++) {
+ props.put(
+ parseAttributeKey(ctx.attributePair(i).attributeKey()).toLowerCase(),
+ parseAttributeValue(ctx.attributePair(i).attributeValue()));
+ }
+ }
+
+ TSEncoding encoding = IoTDBDescriptor.getInstance().getDefaultEncodingByType(dataType);
+ if (props.containsKey(IoTDBConstant.COLUMN_TIMESERIES_ENCODING.toLowerCase())) {
+ String encodingString =
+ props.get(IoTDBConstant.COLUMN_TIMESERIES_ENCODING.toLowerCase()).toUpperCase();
+ try {
+ encoding = TSEncoding.valueOf(encodingString);
+ encodings.add(encoding);
+ props.remove(IoTDBConstant.COLUMN_TIMESERIES_ENCODING.toLowerCase());
+ } catch (Exception e) {
+ throw new SemanticException(String.format("unsupported encoding: %s", encodingString));
+ }
+ } else {
+ encodings.add(encoding);
+ }
+
+ CompressionType compressor = TSFileDescriptor.getInstance().getConfig().getCompressor();
+ if (props.containsKey(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSOR.toLowerCase())) {
+ String compressorString =
+ props.get(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSOR.toLowerCase()).toUpperCase();
+ try {
+ compressor = CompressionType.valueOf(compressorString);
+ compressors.add(compressor);
+ props.remove(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSOR.toLowerCase());
+ } catch (Exception e) {
+ throw new SemanticException(String.format("unsupported compressor: %s", compressorString));
+ }
+ } else if (props.containsKey(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION.toLowerCase())) {
+ String compressionString =
+ props.get(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION.toLowerCase()).toUpperCase();
+ try {
+ compressor = CompressionType.valueOf(compressionString);
+ compressors.add(compressor);
+ props.remove(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION.toLowerCase());
+ } catch (Exception e) {
+ throw new SemanticException(
+ String.format("unsupported compression: %s", compressionString));
+ }
+ } else {
+ compressors.add(compressor);
+ }
+
+ if (props.size() > 0) {
+ throw new SQLParserException("schema template: property is not supported yet.");
+ }
+
+ if (ctx.tagClause() != null) {
+ throw new SQLParserException("schema template: tag is not supported yet.");
+ }
+
+ if (ctx.attributeClause() != null) {
+ throw new SQLParserException("schema template: attribute is not supported yet.");
+ }
+ }
+
+ @Override
+ public Statement visitShowSchemaTemplates(IoTDBSqlParser.ShowSchemaTemplatesContext ctx) {
+ ShowSchemaTemplateStatement showSchemaTemplateStatement = new ShowSchemaTemplateStatement();
+ // createTemplateStatement.setType();
+ return showSchemaTemplateStatement;
+ }
+
+ @Override
+ public Statement visitShowNodesInSchemaTemplate(
+ IoTDBSqlParser.ShowNodesInSchemaTemplateContext ctx) {
+ String templateName = ctx.templateName.children.get(0).getText();
+ ShowNodesInSchemaTemplateStatement showNodesInSchemaTemplateStatement =
+ new ShowNodesInSchemaTemplateStatement(templateName);
+ // showNodesInSchemaTemplateStatement.setTemplateName(ctx.templateName.);
Review Comment:
already removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] MarcosZyk commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
MarcosZyk commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r915363998
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/statement/metadata/template/CreateSchemaTemplateStatement.java:
##########
@@ -0,0 +1,196 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.mpp.plan.statement.metadata.template;
+
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.db.mpp.plan.analyze.QueryType;
+import org.apache.iotdb.db.mpp.plan.constant.StatementType;
+import org.apache.iotdb.db.mpp.plan.statement.IConfigStatement;
+import org.apache.iotdb.db.mpp.plan.statement.Statement;
+import org.apache.iotdb.db.mpp.plan.statement.StatementVisitor;
+import org.apache.iotdb.tsfile.file.metadata.enums.CompressionType;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSEncoding;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class CreateSchemaTemplateStatement extends Statement implements IConfigStatement {
+
+ String name;
+ Set<String> alignedDeviceId;
+ String[] schemaNames;
Review Comment:
It seems this attribute is useless. Shall we remove it?
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/analyze/Analyzer.java:
##########
@@ -1433,6 +1436,54 @@ public Analysis visitDeleteData(
return analysis;
}
+
+ @Override
+ public Analysis visitCreateSchemaTemplate(
+ CreateSchemaTemplateStatement createTemplateStatement, MPPQueryContext context) {
+
+ context.setQueryType(QueryType.WRITE);
+ List<List<String>> measurementsList = createTemplateStatement.getMeasurements();
+ for (List measurements : measurementsList) {
+ Set<String> measurementsSet = new HashSet<>(measurements);
+ if (measurementsSet.size() < measurements.size()) {
+ throw new SemanticException(
+ "Measurement under an aligned device is not allowed to have the same measurement name");
+ }
+ }
+ Analysis analysis = new Analysis();
+ analysis.setStatement(createTemplateStatement);
+
+ /*PathPatternTree pathPatternTree = new PathPatternTree();
+ PartialPath partialPath = new PartialPath(createTemplateStatement.getName());
+ for (String measurement : createAlignedTimeSeriesStatement.getMeasurements()) {
+ pathPatternTree.appendFullPath(
+ createTemplateStatement.getPaths(), measurement);
+ }
+
+ SchemaPartition schemaPartitionInfo;
+ schemaPartitionInfo = partitionFetcher.getOrCreateSchemaPartition(pathPatternTree);
+ analysis.setSchemaPartitionInfo(schemaPartitionInfo);*/
Review Comment:
remove the comments
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/parser/ASTVisitor.java:
##########
@@ -2311,4 +2314,178 @@ public Statement visitShowRegion(IoTDBSqlParser.ShowRegionContext ctx) {
public Statement visitShowDataNodes(IoTDBSqlParser.ShowDataNodesContext ctx) {
return new ShowDataNodesStatement();
}
+
+ @Override
+ public Statement visitCreateSchemaTemplate(IoTDBSqlParser.CreateSchemaTemplateContext ctx) {
+ CreateSchemaTemplateStatement createTemplateStatement = new CreateSchemaTemplateStatement();
+ String name = parseIdentifier(ctx.templateName.getText());
+ List<List<String>> measurementsList = new ArrayList<List<String>>();
+ List<List<TSDataType>> dataTypesList = new ArrayList<List<TSDataType>>();
+ List<List<TSEncoding>> encodingsList = new ArrayList<List<TSEncoding>>();
+ List<List<CompressionType>> compressorsList = new ArrayList<List<CompressionType>>();
+
+ if (ctx.ALIGNED() != null) {
+ // aligned
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ }
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ } else {
+ // non-aligned
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ // parseTemplateMeasurementClause(templateClauseContext, createTemplateStatement);
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ }
+ }
+
+ createTemplateStatement =
+ new CreateSchemaTemplateStatement(
+ name, measurementsList, dataTypesList, encodingsList, compressorsList);
+ return createTemplateStatement;
+ }
+
+ private void parseTemplateMeasurementClause(
+ IoTDBSqlParser.TemplateMeasurementClauseContext ctx,
+ CreateSchemaTemplateStatement createTemplateStatement) {
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ // single template measurement
+ measurements.add(parseNodeNameWithoutWildCard(ctx.nodeNameWithoutWildcard()));
+ parseAttributeClause(ctx.attributeClauses(), dataTypes, encodings, compressors);
+ // createTemplateStatement.addMeasurements(measurements);
+ // createTemplateStatement.addDataTypes(dataTypes);
+ // createTemplateStatement.addEncodings(encodings);
+ // createTemplateStatement.addCompressor(compressors);
Review Comment:
remove comments
##########
server/src/main/java/org/apache/iotdb/db/exception/metadata/template/SchemaTemplateException.java:
##########
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.exception.metadata.template;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.exception.IoTDBException;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+public class SchemaTemplateException extends IoTDBException {
+
+ public SchemaTemplateException(TSStatus status) {
+ super(status.message, status.code);
+ }
+
+ public SchemaTemplateException(Throwable cause) {
+ super(cause, TSStatusCode.STORAGE_ENGINE_ERROR.getStatusCode());
+ }
+
+ public SchemaTemplateException(String message) {
+ super(message, TSStatusCode.STORAGE_ENGINE_ERROR.getStatusCode());
+ }
+
+ public SchemaTemplateException(String message, Throwable cause) {
+ super(message, cause, TSStatusCode.STORAGE_ENGINE_ERROR.getStatusCode());
+ }
+
+ public SchemaTemplateException(String message, int errorCode) {
+ super(message, errorCode);
+ }
+
+ public SchemaTemplateException(IoTDBException e) {
+ super(e, e.getErrorCode(), e.isUserException());
+ }
+}
Review Comment:
There's already some template related exception in package exception.metadata. Please check whether the existing exception class can be used.
##########
confignode/src/test/java/org/apache/iotdb/confignode/persistence/TemplateInfoTest.java:
##########
@@ -0,0 +1,141 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.persistence;
+
+import org.apache.iotdb.commons.exception.IllegalPathException;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.db.mpp.plan.statement.metadata.template.CreateSchemaTemplateStatement;
+import org.apache.iotdb.tsfile.file.metadata.enums.CompressionType;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSEncoding;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.thrift.TException;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+
+import static org.apache.iotdb.db.constant.TestConstant.BASE_OUTPUT_PATH;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/7/6
+ */
+public class TemplateInfoTest {
+
+ private static TemplateInfo templateInfo;
+ private static final File snapshotDir = new File(BASE_OUTPUT_PATH, "snapshot");
+
+ @BeforeClass
+ public static void setup() throws IOException {
+ templateInfo = new TemplateInfo();
+ if (!snapshotDir.exists()) {
+ snapshotDir.mkdirs();
+ }
+ }
+
+ @AfterClass
+ public static void cleanup() throws IOException {
+ templateInfo.clear();
+ if (snapshotDir.exists()) {
+ FileUtils.deleteDirectory(snapshotDir);
+ }
+ }
+
+ @Test
+ public void testSnapshot()
+ throws IOException, TException, IllegalPathException, ClassNotFoundException {
+ int n = 2;
+ String templateName = "template_test";
+
+ List<Template> templates = new ArrayList<>();
+ // 创建模板
+ for (int i = 0; i < n; i++) {
Review Comment:
Replace the comment with English.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916434193
##########
confignode/src/main/java/org/apache/iotdb/confignode/persistence/TemplateInfo.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.persistence;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.commons.snapshot.SnapshotProcessor;
+import org.apache.iotdb.commons.utils.TestOnly;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.rpc.TSStatusCode;
+import org.apache.iotdb.tsfile.utils.ReadWriteIOUtils;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.BufferedInputStream;
+import java.io.BufferedOutputStream;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class TemplateInfo implements SnapshotProcessor {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(TemplateInfo.class);
+
+ // StorageGroup read write lock
+ private final ReentrantReadWriteLock templateReadWriteLock;
+
+ private final Map<String, Template> templateMap = new ConcurrentHashMap<>();
+
+ private final String snapshotFileName = "template_info.bin";
+
+ public TemplateInfo() throws IOException {
+ templateReadWriteLock = new ReentrantReadWriteLock();
+ }
+
+ public TGetTemplateResp getMatchedTemplateByName(String name) {
+ TGetTemplateResp resp = new TGetTemplateResp();
+ try {
+ templateReadWriteLock.readLock().lock();
+ Template template = templateMap.get(name);
+ resp.setStatus(new TSStatus(TSStatusCode.SUCCESS_STATUS.getStatusCode()));
+ resp.setTemplate(Template.template2ByteBuffer(template));
+ } catch (Exception e) {
+ LOGGER.warn("Error TemplateInfo name", e);
+ resp.setStatus(new TSStatus(TSStatusCode.EXECUTE_STATEMENT_ERROR.getStatusCode()));
+ } finally {
+ templateReadWriteLock.readLock().unlock();
+ }
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916431234
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/parser/ASTVisitor.java:
##########
@@ -2311,4 +2314,178 @@ public Statement visitShowRegion(IoTDBSqlParser.ShowRegionContext ctx) {
public Statement visitShowDataNodes(IoTDBSqlParser.ShowDataNodesContext ctx) {
return new ShowDataNodesStatement();
}
+
+ @Override
+ public Statement visitCreateSchemaTemplate(IoTDBSqlParser.CreateSchemaTemplateContext ctx) {
+ CreateSchemaTemplateStatement createTemplateStatement = new CreateSchemaTemplateStatement();
+ String name = parseIdentifier(ctx.templateName.getText());
+ List<List<String>> measurementsList = new ArrayList<List<String>>();
+ List<List<TSDataType>> dataTypesList = new ArrayList<List<TSDataType>>();
+ List<List<TSEncoding>> encodingsList = new ArrayList<List<TSEncoding>>();
+ List<List<CompressionType>> compressorsList = new ArrayList<List<CompressionType>>();
+
+ if (ctx.ALIGNED() != null) {
+ // aligned
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ }
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ } else {
+ // non-aligned
+ for (IoTDBSqlParser.TemplateMeasurementClauseContext templateClauseContext :
+ ctx.templateMeasurementClause()) {
+ // parseTemplateMeasurementClause(templateClauseContext, createTemplateStatement);
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ measurements.add(
+ parseNodeNameWithoutWildCard(templateClauseContext.nodeNameWithoutWildcard()));
+ parseAttributeClause(
+ templateClauseContext.attributeClauses(), dataTypes, encodings, compressors);
+ measurementsList.add(measurements);
+ dataTypesList.add(dataTypes);
+ encodingsList.add(encodings);
+ compressorsList.add(compressors);
+ }
+ }
+
+ createTemplateStatement =
+ new CreateSchemaTemplateStatement(
+ name, measurementsList, dataTypesList, encodingsList, compressorsList);
+ return createTemplateStatement;
+ }
+
+ private void parseTemplateMeasurementClause(
+ IoTDBSqlParser.TemplateMeasurementClauseContext ctx,
+ CreateSchemaTemplateStatement createTemplateStatement) {
+ List<String> measurements = new ArrayList<>();
+ List<TSDataType> dataTypes = new ArrayList<>();
+ List<TSEncoding> encodings = new ArrayList<>();
+ List<CompressionType> compressors = new ArrayList<>();
+ // single template measurement
+ measurements.add(parseNodeNameWithoutWildCard(ctx.nodeNameWithoutWildcard()));
+ parseAttributeClause(ctx.attributeClauses(), dataTypes, encodings, compressors);
+ // createTemplateStatement.addMeasurements(measurements);
+ // createTemplateStatement.addDataTypes(dataTypes);
+ // createTemplateStatement.addEncodings(encodings);
+ // createTemplateStatement.addCompressor(compressors);
+ }
+
+ void parseAttributeClause(
+ IoTDBSqlParser.AttributeClausesContext ctx,
+ List<TSDataType> dataTypes,
+ List<TSEncoding> encodings,
+ List<CompressionType> compressors) {
+ if (ctx.aliasNodeName() != null) {
+ throw new SQLParserException("schema template: alias is not supported yet.");
+ }
+
+ TSDataType dataType = null;
+ if (ctx.dataType != null) {
+ if (ctx.attributeKey() != null) {
+ if (!parseAttributeKey(ctx.attributeKey())
+ .equalsIgnoreCase(IoTDBConstant.COLUMN_TIMESERIES_DATATYPE)) {
+ throw new SQLParserException("expecting datatype");
+ }
+ }
+ String dataTypeString = ctx.dataType.getText().toUpperCase();
+ try {
+ dataType = TSDataType.valueOf(dataTypeString);
+ dataTypes.add(dataType);
+ } catch (Exception e) {
+ throw new SemanticException(String.format("unsupported datatype: %s", dataTypeString));
+ }
+ }
+
+ Map<String, String> props = new HashMap<>();
+ if (ctx.attributePair() != null) {
+ for (int i = 0; i < ctx.attributePair().size(); i++) {
+ props.put(
+ parseAttributeKey(ctx.attributePair(i).attributeKey()).toLowerCase(),
+ parseAttributeValue(ctx.attributePair(i).attributeValue()));
+ }
+ }
+
+ TSEncoding encoding = IoTDBDescriptor.getInstance().getDefaultEncodingByType(dataType);
+ if (props.containsKey(IoTDBConstant.COLUMN_TIMESERIES_ENCODING.toLowerCase())) {
+ String encodingString =
+ props.get(IoTDBConstant.COLUMN_TIMESERIES_ENCODING.toLowerCase()).toUpperCase();
+ try {
+ encoding = TSEncoding.valueOf(encodingString);
+ encodings.add(encoding);
+ props.remove(IoTDBConstant.COLUMN_TIMESERIES_ENCODING.toLowerCase());
+ } catch (Exception e) {
+ throw new SemanticException(String.format("unsupported encoding: %s", encodingString));
+ }
+ } else {
+ encodings.add(encoding);
+ }
+
+ CompressionType compressor = TSFileDescriptor.getInstance().getConfig().getCompressor();
+ if (props.containsKey(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSOR.toLowerCase())) {
+ String compressorString =
+ props.get(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSOR.toLowerCase()).toUpperCase();
+ try {
+ compressor = CompressionType.valueOf(compressorString);
+ compressors.add(compressor);
+ props.remove(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSOR.toLowerCase());
+ } catch (Exception e) {
+ throw new SemanticException(String.format("unsupported compressor: %s", compressorString));
+ }
+ } else if (props.containsKey(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION.toLowerCase())) {
+ String compressionString =
+ props.get(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION.toLowerCase()).toUpperCase();
+ try {
+ compressor = CompressionType.valueOf(compressionString);
+ compressors.add(compressor);
+ props.remove(IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION.toLowerCase());
+ } catch (Exception e) {
+ throw new SemanticException(
+ String.format("unsupported compression: %s", compressionString));
+ }
+ } else {
+ compressors.add(compressor);
+ }
+
+ if (props.size() > 0) {
+ throw new SQLParserException("schema template: property is not supported yet.");
+ }
+
+ if (ctx.tagClause() != null) {
+ throw new SQLParserException("schema template: tag is not supported yet.");
+ }
+
+ if (ctx.attributeClause() != null) {
+ throw new SQLParserException("schema template: attribute is not supported yet.");
+ }
+ }
+
+ @Override
+ public Statement visitShowSchemaTemplates(IoTDBSqlParser.ShowSchemaTemplatesContext ctx) {
+ ShowSchemaTemplateStatement showSchemaTemplateStatement = new ShowSchemaTemplateStatement();
+ // createTemplateStatement.setType();
Review Comment:
already removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916431505
##########
confignode/src/main/java/org/apache/iotdb/confignode/consensus/request/write/CreateSchemaTemplatePlan.java:
##########
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.consensus.request.write;
+
+import org.apache.iotdb.confignode.consensus.request.ConfigPhysicalPlan;
+import org.apache.iotdb.confignode.consensus.request.ConfigPhysicalPlanType;
+import org.apache.iotdb.tsfile.utils.ReadWriteIOUtils;
+
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.Objects;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/7/5
+ */
+public class CreateSchemaTemplatePlan extends ConfigPhysicalPlan {
+
+ private byte[] template;
+
+ public CreateSchemaTemplatePlan() {
+ super(ConfigPhysicalPlanType.CreateSchemaTemplate);
+ }
+
+ public CreateSchemaTemplatePlan(byte[] template) {
+ this();
+ this.template = template;
+ }
+
+ public byte[] getTemplate() {
+ return template;
+ }
+
+ @Override
+ protected void serializeImpl(DataOutputStream stream) throws IOException {
+ stream.writeInt(ConfigPhysicalPlanType.CreateSchemaTemplate.ordinal());
+ stream.writeInt(template.length);
+ stream.write(template);
+ }
+
+ @Override
+ protected void deserializeImpl(ByteBuffer buffer) throws IOException {
+ int length = ReadWriteIOUtils.readInt(buffer);
+ this.template = ReadWriteIOUtils.readBytes(buffer, length);
+ }
Review Comment:
Add CreateSchemaTemplatePlanTest method tests in ConfigPhysicalPlanSerDeTest
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916429898
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/statement/metadata/template/CreateSchemaTemplateStatement.java:
##########
@@ -0,0 +1,196 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.mpp.plan.statement.metadata.template;
+
+import org.apache.iotdb.commons.path.PartialPath;
+import org.apache.iotdb.db.mpp.plan.analyze.QueryType;
+import org.apache.iotdb.db.mpp.plan.constant.StatementType;
+import org.apache.iotdb.db.mpp.plan.statement.IConfigStatement;
+import org.apache.iotdb.db.mpp.plan.statement.Statement;
+import org.apache.iotdb.db.mpp.plan.statement.StatementVisitor;
+import org.apache.iotdb.tsfile.file.metadata.enums.CompressionType;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSEncoding;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class CreateSchemaTemplateStatement extends Statement implements IConfigStatement {
+
+ String name;
+ Set<String> alignedDeviceId;
+ String[] schemaNames;
Review Comment:
schemaNames already removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916429721
##########
server/src/main/java/org/apache/iotdb/db/mpp/plan/analyze/Analyzer.java:
##########
@@ -1433,6 +1436,54 @@ public Analysis visitDeleteData(
return analysis;
}
+
+ @Override
+ public Analysis visitCreateSchemaTemplate(
+ CreateSchemaTemplateStatement createTemplateStatement, MPPQueryContext context) {
+
+ context.setQueryType(QueryType.WRITE);
+ List<List<String>> measurementsList = createTemplateStatement.getMeasurements();
+ for (List measurements : measurementsList) {
+ Set<String> measurementsSet = new HashSet<>(measurements);
+ if (measurementsSet.size() < measurements.size()) {
+ throw new SemanticException(
+ "Measurement under an aligned device is not allowed to have the same measurement name");
+ }
+ }
+ Analysis analysis = new Analysis();
+ analysis.setStatement(createTemplateStatement);
+
+ /*PathPatternTree pathPatternTree = new PathPatternTree();
+ PartialPath partialPath = new PartialPath(createTemplateStatement.getName());
+ for (String measurement : createAlignedTimeSeriesStatement.getMeasurements()) {
+ pathPatternTree.appendFullPath(
+ createTemplateStatement.getPaths(), measurement);
+ }
+
+ SchemaPartition schemaPartitionInfo;
+ schemaPartitionInfo = partitionFetcher.getOrCreateSchemaPartition(pathPatternTree);
+ analysis.setSchemaPartitionInfo(schemaPartitionInfo);*/
Review Comment:
already removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916431904
##########
confignode/src/main/java/org/apache/iotdb/confignode/manager/TemplateManager.java:
##########
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.confignode.manager;
+
+import org.apache.iotdb.common.rpc.thrift.TSStatus;
+import org.apache.iotdb.confignode.consensus.request.write.CreateSchemaTemplatePlan;
+import org.apache.iotdb.confignode.persistence.TemplateInfo;
+import org.apache.iotdb.confignode.rpc.thrift.TGetAllTemplatesResp;
+import org.apache.iotdb.confignode.rpc.thrift.TGetTemplateResp;
+import org.apache.iotdb.rpc.TSStatusCode;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * @author chenhuangyun
+ * @date 2022/6/28
+ */
+public class TemplateManager {
Review Comment:
TemplateManager deleted. add method in ClusterSchemaManager
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] yunchan86 commented on a diff in pull request #6608: [IOTDB-3560]Support basic create and query template
Posted by GitBox <gi...@apache.org>.
yunchan86 commented on code in PR #6608:
URL: https://github.com/apache/iotdb/pull/6608#discussion_r916434566
##########
confignode/src/main/java/org/apache/iotdb/confignode/service/thrift/ConfigNodeRPCServiceProcessor.java:
##########
@@ -494,4 +497,20 @@ public TShowDataNodesResp showDataNodes() throws TException {
public void handleClientExit() {}
// TODO: Interfaces for data operations
+
+ @Override
+ public TSStatus createSchemaTemplate(TCreateSchemaTemplateReq req) throws TException {
+ return configManager.createSchemaTemplate(req);
+ }
+
+ @Override
+ public TGetAllTemplatesResp getAllTemplates() throws TException {
+ return configManager.getAllTemplates();
+ }
+
+ @Override
+ public TGetTemplateResp getTemplate(String req) throws TException {
+
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org