You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2020/09/14 13:02:30 UTC
[GitHub] [incubator-iotdb] haimeiguo opened a new pull request #1731: [IOTDB-870] tags and attributes output
haimeiguo opened a new pull request #1731:
URL: https://github.com/apache/incubator-iotdb/pull/1731
tags and attributes in result dataset are in json format
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] liutaohua commented on a change in pull request #1731: [IOTDB-870] tags and attributes output
Posted by GitBox <gi...@apache.org>.
liutaohua commented on a change in pull request #1731:
URL: https://github.com/apache/incubator-iotdb/pull/1731#discussion_r490812091
##########
File path: server/src/main/java/org/apache/iotdb/db/utils/QueryUtils.java
##########
@@ -166,26 +157,28 @@ public static QueryDataSet getQueryDataSet(List<ShowTimeSeriesResult> timeseries
List<RowRecord> records = new ArrayList<>();
List<PartialPath> paths = new ArrayList<>();
List<TSDataType> dataTypes = new ArrayList<>();
- constructPathAndDataTypes(paths, dataTypes, timeseriesList);
+ constructPathAndDataTypes(paths, dataTypes);
for (ShowTimeSeriesResult result : timeseriesList) {
RowRecord record = new RowRecord(0);
updateRecord(record, result.getName());
updateRecord(record, result.getAlias());
updateRecord(record, result.getSgName());
- updateRecord(record, result.getDataType());
- updateRecord(record, result.getEncoding());
- updateRecord(record, result.getCompressor());
- updateRecord(record, result.getTagAndAttribute(), paths);
+ updateRecord(record, result.getDataType().toString());
+ updateRecord(record, result.getEncoding().toString());
+ updateRecord(record, result.getCompressor().toString());
Review comment:
same
##########
File path: server/src/main/java/org/apache/iotdb/db/utils/QueryUtils.java
##########
@@ -112,36 +122,16 @@ public static void filterQueryDataSource(QueryDataSource queryDataSource,
unseqResources.removeIf(fileFilter::fileNotSatisfy);
}
- public static void constructPathAndDataTypes(List<PartialPath> paths, List<TSDataType> dataTypes,
- List<ShowTimeSeriesResult> timeseriesList) {
- paths.add(new PartialPath(COLUMN_TIMESERIES, false));
- dataTypes.add(TSDataType.TEXT);
- paths.add(new PartialPath(COLUMN_TIMESERIES_ALIAS, false));
- dataTypes.add(TSDataType.TEXT);
- paths.add(new PartialPath(COLUMN_STORAGE_GROUP, false));
- dataTypes.add(TSDataType.TEXT);
- paths.add(new PartialPath(COLUMN_TIMESERIES_DATATYPE, false));
- dataTypes.add(TSDataType.TEXT);
- paths.add(new PartialPath(COLUMN_TIMESERIES_ENCODING, false));
- dataTypes.add(TSDataType.TEXT);
- paths.add(new PartialPath(COLUMN_TIMESERIES_COMPRESSION, false));
- dataTypes.add(TSDataType.TEXT);
-
- Set<String> tagAndAttributeName = new TreeSet<>();
- for (ShowTimeSeriesResult result : timeseriesList) {
- tagAndAttributeName.addAll(result.getTagAndAttribute().keySet());
- }
- for (String key : tagAndAttributeName) {
- paths.add(new PartialPath(key, false));
- dataTypes.add(TSDataType.TEXT);
- }
+ public static void constructPathAndDataTypes(List<PartialPath> paths, List<TSDataType> dataTypes) {
+ Collections.addAll(paths, resourcePaths);
+ Collections.addAll(dataTypes, resourceTypes);
}
public static QueryDataSet getQueryDataSet(List<ShowTimeSeriesResult> timeseriesList,
ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context) {
List<PartialPath> paths = new ArrayList<>();
List<TSDataType> dataTypes = new ArrayList<>();
- constructPathAndDataTypes(paths, dataTypes, timeseriesList);
+ constructPathAndDataTypes(paths, dataTypes);
Review comment:
I think that `constructPath` and `updateRecord` behavior belongs to `Dataset`, how about move it to `ShowTimeseriesDataSet ` class?
and next line , the `showTimeSeriesPlan` already exists in dataset...
@qiaojialin what do you think?
##########
File path: server/src/main/java/org/apache/iotdb/db/utils/QueryUtils.java
##########
@@ -152,10 +142,11 @@ public static QueryDataSet getQueryDataSet(List<ShowTimeSeriesResult> timeseries
updateRecord(record, result.getName());
updateRecord(record, result.getAlias());
updateRecord(record, result.getSgName());
- updateRecord(record, result.getDataType());
- updateRecord(record, result.getEncoding());
- updateRecord(record, result.getCompressor());
- updateRecord(record, result.getTagAndAttribute(), paths);
+ updateRecord(record, result.getDataType().toString());
+ updateRecord(record, result.getEncoding().toString());
+ updateRecord(record, result.getCompressor().toString());
Review comment:
i think that too many type cast , how about used only in `serialize/deserialize`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] qiaojialin merged pull request #1731: [IOTDB-870] tags and attributes output
Posted by GitBox <gi...@apache.org>.
qiaojialin merged pull request #1731:
URL: https://github.com/apache/incubator-iotdb/pull/1731
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] HTHou commented on a change in pull request #1731: [IOTDB-870] tags and attributes output
Posted by GitBox <gi...@apache.org>.
HTHou commented on a change in pull request #1731:
URL: https://github.com/apache/incubator-iotdb/pull/1731#discussion_r493323543
##########
File path: server/src/main/java/org/apache/iotdb/db/conf/IoTDBConstant.java
##########
@@ -70,6 +70,8 @@ private IoTDBConstant() {
public static final String COLUMN_DEVICES = "devices";
public static final String COLUMN_COLUMN = "column";
public static final String COLUMN_COUNT = "count";
+ public static final String COLUMN_TAG = "tags";
+ public static final String COLUMN_ATTRIBUTE = "attributes";
Review comment:
```suggestion
public static final String COLUMN_TAGS = "tags";
public static final String COLUMN_ATTRIBUTES = "attributes";
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #1731: [IOTDB-870] tags and attributes output
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #1731:
URL: https://github.com/apache/incubator-iotdb/pull/1731#discussion_r491860833
##########
File path: server/src/main/java/org/apache/iotdb/db/qp/executor/PlanExecutor.java
##########
@@ -566,7 +567,8 @@ private QueryDataSet processShowStorageGroup(ShowStorageGroupPlan showStorageGro
private QueryDataSet processShowTimeseries(ShowTimeSeriesPlan showTimeSeriesPlan,
QueryContext context) throws MetadataException {
List<ShowTimeSeriesResult> timeseriesList = showTimeseries(showTimeSeriesPlan, context);
- return QueryUtils.getQueryDataSet(timeseriesList, showTimeSeriesPlan, context);
+ ShowTimeseriesDataSet showTimeSeriesDataSet = new ShowTimeseriesDataSet(showTimeSeriesPlan, context);
+ return showTimeSeriesDataSet.getQueryDataSet(timeseriesList);
Review comment:
ShowTimeseriesData should play as a query processor.
It accepts a query plan and generates results.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #1731: [IOTDB-870] tags and attributes output
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #1731:
URL: https://github.com/apache/incubator-iotdb/pull/1731#discussion_r491293164
##########
File path: server/src/main/java/org/apache/iotdb/db/utils/QueryUtils.java
##########
@@ -112,36 +122,16 @@ public static void filterQueryDataSource(QueryDataSource queryDataSource,
unseqResources.removeIf(fileFilter::fileNotSatisfy);
}
- public static void constructPathAndDataTypes(List<PartialPath> paths, List<TSDataType> dataTypes,
- List<ShowTimeSeriesResult> timeseriesList) {
- paths.add(new PartialPath(COLUMN_TIMESERIES, false));
- dataTypes.add(TSDataType.TEXT);
- paths.add(new PartialPath(COLUMN_TIMESERIES_ALIAS, false));
- dataTypes.add(TSDataType.TEXT);
- paths.add(new PartialPath(COLUMN_STORAGE_GROUP, false));
- dataTypes.add(TSDataType.TEXT);
- paths.add(new PartialPath(COLUMN_TIMESERIES_DATATYPE, false));
- dataTypes.add(TSDataType.TEXT);
- paths.add(new PartialPath(COLUMN_TIMESERIES_ENCODING, false));
- dataTypes.add(TSDataType.TEXT);
- paths.add(new PartialPath(COLUMN_TIMESERIES_COMPRESSION, false));
- dataTypes.add(TSDataType.TEXT);
-
- Set<String> tagAndAttributeName = new TreeSet<>();
- for (ShowTimeSeriesResult result : timeseriesList) {
- tagAndAttributeName.addAll(result.getTagAndAttribute().keySet());
- }
- for (String key : tagAndAttributeName) {
- paths.add(new PartialPath(key, false));
- dataTypes.add(TSDataType.TEXT);
- }
+ public static void constructPathAndDataTypes(List<PartialPath> paths, List<TSDataType> dataTypes) {
+ Collections.addAll(paths, resourcePaths);
+ Collections.addAll(dataTypes, resourceTypes);
}
public static QueryDataSet getQueryDataSet(List<ShowTimeSeriesResult> timeseriesList,
ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context) {
List<PartialPath> paths = new ArrayList<>();
List<TSDataType> dataTypes = new ArrayList<>();
- constructPathAndDataTypes(paths, dataTypes, timeseriesList);
+ constructPathAndDataTypes(paths, dataTypes);
Review comment:
It seems that the constructPathAndDataTypes is only used for ShowTimeseries, so moving it to ShowTimeseriesDataSet is ok.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #1731: [IOTDB-870] tags and attributes output
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #1731:
URL: https://github.com/apache/incubator-iotdb/pull/1731#discussion_r488594464
##########
File path: server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeSeriesResult.java
##########
@@ -35,17 +35,19 @@
private String dataType;
private String encoding;
private String compressor;
- private Map<String, String> tagAndAttribute;
+ private Map<String, String> tag;
+ private Map<String, String> attribute;
Review comment:
tags and attributes are better
##########
File path: server/src/main/java/org/apache/iotdb/db/conf/IoTDBConstant.java
##########
@@ -70,6 +70,8 @@ private IoTDBConstant() {
public static final String COLUMN_DEVICES = "devices";
public static final String COLUMN_COLUMN = "column";
public static final String COLUMN_COUNT = "count";
+ public static final String COLUMN_TAG = "tag";
+ public static final String COLUMN_ATTRIBUTE = "attribute";
Review comment:
tags and attributes?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] haimeiguo commented on a change in pull request #1731: [IOTDB-870] tags and attributes output
Posted by GitBox <gi...@apache.org>.
haimeiguo commented on a change in pull request #1731:
URL: https://github.com/apache/incubator-iotdb/pull/1731#discussion_r490764270
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/MManager.java
##########
@@ -994,14 +993,13 @@ private boolean match(PartialPath fullPath, String[] prefixNodes) {
if (tagFileOffset < 0) {
// no tags/attributes
res.add(new ShowTimeSeriesResult(ansString.left.getFullPath(), ansString.right[0], ansString.right[1], ansString.right[2],
- ansString.right[3], ansString.right[4], Collections.emptyMap()));
+ ansString.right[3], ansString.right[4], Collections.emptyMap(), Collections.emptyMap()));
} else {
// has tags/attributes
Pair<Map<String, String>, Map<String, String>> pair =
tagLogFile.read(config.getTagAttributeTotalSize(), tagFileOffset);
- pair.left.putAll(pair.right);
res.add(new ShowTimeSeriesResult(ansString.left.getFullPath(), ansString.right[0], ansString.right[1], ansString.right[2],
- ansString.right[3], ansString.right[4], pair.left));
+ ansString.right[3], ansString.right[4], pair.left, pair.right));
Review comment:
edited. thank you :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #1731: [IOTDB-870] tags and attributes output
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #1731:
URL: https://github.com/apache/incubator-iotdb/pull/1731#discussion_r491860833
##########
File path: server/src/main/java/org/apache/iotdb/db/qp/executor/PlanExecutor.java
##########
@@ -566,7 +567,8 @@ private QueryDataSet processShowStorageGroup(ShowStorageGroupPlan showStorageGro
private QueryDataSet processShowTimeseries(ShowTimeSeriesPlan showTimeSeriesPlan,
QueryContext context) throws MetadataException {
List<ShowTimeSeriesResult> timeseriesList = showTimeseries(showTimeSeriesPlan, context);
- return QueryUtils.getQueryDataSet(timeseriesList, showTimeSeriesPlan, context);
+ ShowTimeseriesDataSet showTimeSeriesDataSet = new ShowTimeseriesDataSet(showTimeSeriesPlan, context);
+ return showTimeSeriesDataSet.getQueryDataSet(timeseriesList);
Review comment:
ShowTimeseriesData should play as a query processor.
It accepts a query plan and generates results.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] liutaohua commented on a change in pull request #1731: [IOTDB-870] tags and attributes output
Posted by GitBox <gi...@apache.org>.
liutaohua commented on a change in pull request #1731:
URL: https://github.com/apache/incubator-iotdb/pull/1731#discussion_r490696999
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/MManager.java
##########
@@ -994,14 +993,13 @@ private boolean match(PartialPath fullPath, String[] prefixNodes) {
if (tagFileOffset < 0) {
// no tags/attributes
res.add(new ShowTimeSeriesResult(ansString.left.getFullPath(), ansString.right[0], ansString.right[1], ansString.right[2],
- ansString.right[3], ansString.right[4], Collections.emptyMap()));
+ ansString.right[3], ansString.right[4], Collections.emptyMap(), Collections.emptyMap()));
} else {
// has tags/attributes
Pair<Map<String, String>, Map<String, String>> pair =
tagLogFile.read(config.getTagAttributeTotalSize(), tagFileOffset);
- pair.left.putAll(pair.right);
res.add(new ShowTimeSeriesResult(ansString.left.getFullPath(), ansString.right[0], ansString.right[1], ansString.right[2],
- ansString.right[3], ansString.right[4], pair.left));
+ ansString.right[3], ansString.right[4], pair.left, pair.right));
Review comment:
how about:
```
Pair<Map<String, String>, Map<String, String>> pair = new Pair(Collections.emptyMap(),Collections.emptyMap());
if(tagFileOffset >= 0){
pair = tagLogFile.read(config.getTagAttributeTotalSize(), tagFileOffset);
}
res.add(new ShowTimeSeriesResult(ansString.left.getFullPath(), ansString.right[0], ansString.right[1], ansString.right[2], pair.left, pair.right);
```
##########
File path: server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeSeriesResult.java
##########
@@ -113,10 +119,21 @@ public void serialize(OutputStream outputStream) throws IOException {
ReadWriteIOUtils.write(encoding, outputStream);
ReadWriteIOUtils.write(compressor, outputStream);
- ReadWriteIOUtils.write(tagAndAttribute != null, outputStream); //flag
- if (tagAndAttribute != null) {
- ReadWriteIOUtils.write(tagAndAttribute.size(), outputStream);
- for (Entry<String, String> stringStringEntry : tagAndAttribute.entrySet()) {
+ //flag for tag
+ ReadWriteIOUtils.write(tags != null, outputStream);
+ if (tags != null) {
+ ReadWriteIOUtils.write(tags.size(), outputStream);
+ for (Entry<String, String> stringStringEntry : tags.entrySet()) {
+ ReadWriteIOUtils.write(stringStringEntry.getKey(), outputStream);
+ ReadWriteIOUtils.write(stringStringEntry.getValue(), outputStream);
+ }
+ }
+
+ //flag for attribute
+ ReadWriteIOUtils.write(attributes != null, outputStream);
Review comment:
tags and attr are duplicate code, how about:
```
private void writeNullable(Map<String,String> param, OutputStream out){
ReadWriteIOUtils.write(param != null, outputStream);
if (param != null) {
ReadWriteIOUtils.write(tags.size(), outputStream);
for (Entry<String, String> entry : param.entrySet()) {
ReadWriteIOUtils.write(entry.getKey(), outputStream);
ReadWriteIOUtils.write(entry.getValue(), outputStream);
}
}
}
```
##########
File path: server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeSeriesResult.java
##########
@@ -134,13 +151,25 @@ public static ShowTimeSeriesResult deserialize(ByteBuffer buffer) {
result.encoding = ReadWriteIOUtils.readString(buffer);
result.compressor = ReadWriteIOUtils.readString(buffer);
Review comment:
we have the enums of these strings, I suggest that we use the enums to serialize, u can find it in :
```
package org.apache.iotdb.tsfile.file.metadata.enums;
```
##########
File path: server/src/main/java/org/apache/iotdb/db/utils/QueryUtils.java
##########
@@ -127,12 +128,27 @@ public static void constructPathAndDataTypes(List<PartialPath> paths, List<TSDat
paths.add(new PartialPath(COLUMN_TIMESERIES_COMPRESSION, false));
dataTypes.add(TSDataType.TEXT);
Review comment:
duplicate code, how about:
```
private static PartialPath[] resourcePaths = new PartialPath[]{new PartialPath(COLUMN_TIMESERIES, false), .....};
private static TSDataType[] resourceTypes = new TSDataType[]{TSDataType.TEXT, .....};
public static void constructPathAndDataTypes(List<PartialPath> paths, List<TSDataType> dataTypes, List<ShowTimeSeriesResult> timeseriesList) {
Collections.addAll(paths,resourcePaths);
Collections.addAll(dataTypes,resourceTypes);
......
}
```
##########
File path: server/src/main/java/org/apache/iotdb/db/utils/QueryUtils.java
##########
@@ -127,12 +128,27 @@ public static void constructPathAndDataTypes(List<PartialPath> paths, List<TSDat
paths.add(new PartialPath(COLUMN_TIMESERIES_COMPRESSION, false));
dataTypes.add(TSDataType.TEXT);
- Set<String> tagAndAttributeName = new TreeSet<>();
+ //check if timeseries result has tag or attribute
+ boolean hasTag = false;
+ boolean hasAttribute = false;
for (ShowTimeSeriesResult result : timeseriesList) {
Review comment:
I don't think there's any need to check, it's ok to show `null || spaces`, like :
```
+--------------+-----+-------------+--------+--------+-----------+----+----------+
| timeseries|alias|storage group|dataType|encoding|compression|tags|attributes|
+--------------+-----+-------------+--------+--------+-----------+----+----------+
|root.sg1.d1.s1| null| root.sg1| INT64| RLE| SNAPPY|null| null|
|root.sg1.d1.s2| null| root.sg1| INT64| RLE| SNAPPY|null| null|
|root.sg1.d1.s3| null| root.sg1| INT64| RLE| SNAPPY|null| null|
+--------------+-----+-------------+--------+--------+-----------+----+----------+
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] liutaohua commented on pull request #1731: [IOTDB-870] tags and attributes output
Posted by GitBox <gi...@apache.org>.
liutaohua commented on pull request #1731:
URL: https://github.com/apache/incubator-iotdb/pull/1731#issuecomment-697197710
LGTM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] liutaohua commented on pull request #1731: [IOTDB-870] tags and attributes output
Posted by GitBox <gi...@apache.org>.
liutaohua commented on pull request #1731:
URL: https://github.com/apache/incubator-iotdb/pull/1731#issuecomment-694771710
Don't forget merge _`main`_ branch
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] liutaohua commented on a change in pull request #1731: [IOTDB-870] tags and attributes output
Posted by GitBox <gi...@apache.org>.
liutaohua commented on a change in pull request #1731:
URL: https://github.com/apache/incubator-iotdb/pull/1731#discussion_r492644306
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/MManager.java
##########
@@ -937,12 +937,11 @@ public int getNodesCountInGivenLevel(PartialPath prefixPath, int level) throws M
try {
Pair<Map<String, String>, Map<String, String>> pair =
Review comment:
Change to a more meaningful name, like `tagAndAttributePair`
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/MManager.java
##########
@@ -1003,18 +1002,13 @@ private boolean match(PartialPath fullPath, String[] prefixNodes) {
for (Pair<PartialPath, String[]> ansString : ans) {
long tagFileOffset = Long.parseLong(ansString.right[5]);
try {
- if (tagFileOffset < 0) {
- // no tags/attributes
- res.add(new ShowTimeSeriesResult(ansString.left.getFullPath(), ansString.right[0], ansString.right[1], ansString.right[2],
- ansString.right[3], ansString.right[4], Collections.emptyMap()));
- } else {
- // has tags/attributes
- Pair<Map<String, String>, Map<String, String>> pair =
- tagLogFile.read(config.getTagAttributeTotalSize(), tagFileOffset);
- pair.left.putAll(pair.right);
- res.add(new ShowTimeSeriesResult(ansString.left.getFullPath(), ansString.right[0], ansString.right[1], ansString.right[2],
- ansString.right[3], ansString.right[4], pair.left));
+ Pair<Map<String, String>, Map<String, String>> pair = new Pair<>(Collections.emptyMap(),Collections.emptyMap());
Review comment:
same
##########
File path: server/src/main/java/org/apache/iotdb/db/qp/executor/PlanExecutor.java
##########
@@ -566,7 +567,7 @@ private QueryDataSet processShowStorageGroup(ShowStorageGroupPlan showStorageGro
private QueryDataSet processShowTimeseries(ShowTimeSeriesPlan showTimeSeriesPlan,
QueryContext context) throws MetadataException {
List<ShowTimeSeriesResult> timeseriesList = showTimeseries(showTimeSeriesPlan, context);
- return QueryUtils.getQueryDataSet(timeseriesList, showTimeSeriesPlan, context);
+ return new ShowTimeseriesDataSet(showTimeSeriesPlan, context, timeseriesList);
}
protected List<ShowTimeSeriesResult> showTimeseries(ShowTimeSeriesPlan plan, QueryContext context)
Review comment:
I think this method can be removed, or move it to `ShowTimeseriesDataSet`
##########
File path: server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeseriesDataSet.java
##########
@@ -19,44 +19,123 @@
package org.apache.iotdb.db.query.dataset;
-import static org.apache.iotdb.db.utils.QueryUtils.transferShowTimeSeriesResultToRecordList;
-
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_ATTRIBUTE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_STORAGE_GROUP;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TAG;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ALIAS;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_DATATYPE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ENCODING;
import java.io.IOException;
import java.util.ArrayList;
+import java.util.Arrays;
import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
import org.apache.iotdb.db.exception.metadata.MetadataException;
import org.apache.iotdb.db.metadata.MManager;
import org.apache.iotdb.db.metadata.PartialPath;
import org.apache.iotdb.db.qp.physical.sys.ShowTimeSeriesPlan;
import org.apache.iotdb.db.query.context.QueryContext;
import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.read.common.Field;
import org.apache.iotdb.tsfile.read.common.Path;
import org.apache.iotdb.tsfile.read.common.RowRecord;
import org.apache.iotdb.tsfile.read.query.dataset.QueryDataSet;
+import org.apache.iotdb.tsfile.utils.Binary;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class ShowTimeseriesDataSet extends QueryDataSet {
private static final Logger logger = LoggerFactory.getLogger(ShowTimeseriesDataSet.class);
-
private final ShowTimeSeriesPlan plan;
private List<RowRecord> result = new ArrayList<>();
private int index = 0;
private QueryContext context;
+ private List<ShowTimeSeriesResult> timeseriesList;
+ private boolean hasSetRecord;
public boolean hasLimit = true;
- public ShowTimeseriesDataSet(List<PartialPath> paths, List<TSDataType> dataTypes,
- ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context) {
- super(new ArrayList<>(paths), dataTypes);
+ private static Path[] resourcePaths = {new PartialPath(COLUMN_TIMESERIES, false),
+ new PartialPath(COLUMN_TIMESERIES_ALIAS, false), new PartialPath(COLUMN_STORAGE_GROUP, false),
+ new PartialPath(COLUMN_TIMESERIES_DATATYPE, false), new PartialPath(COLUMN_TIMESERIES_ENCODING, false),
+ new PartialPath(COLUMN_TIMESERIES_COMPRESSION, false), new PartialPath(COLUMN_TAG, false),
+ new PartialPath(COLUMN_ATTRIBUTE, false)};
+ private static TSDataType[] resourceTypes = {TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT,
+ TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT};
+
+ public ShowTimeseriesDataSet(ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context,
+ List<ShowTimeSeriesResult> timeseriesList) {
+ super(Arrays.asList(resourcePaths), Arrays.asList(resourceTypes));
this.plan = showTimeSeriesPlan;
this.context = context;
+ this.timeseriesList = timeseriesList;
+ this.hasSetRecord = false;
+ }
+
+ public QueryDataSet getQueryDataSet() {
+ hasLimit = plan.hasLimit();
Review comment:
i think that it can be initialized in the `constructor`
##########
File path: server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeseriesDataSet.java
##########
@@ -19,44 +19,123 @@
package org.apache.iotdb.db.query.dataset;
-import static org.apache.iotdb.db.utils.QueryUtils.transferShowTimeSeriesResultToRecordList;
-
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_ATTRIBUTE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_STORAGE_GROUP;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TAG;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ALIAS;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_DATATYPE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ENCODING;
import java.io.IOException;
import java.util.ArrayList;
+import java.util.Arrays;
import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
import org.apache.iotdb.db.exception.metadata.MetadataException;
import org.apache.iotdb.db.metadata.MManager;
import org.apache.iotdb.db.metadata.PartialPath;
import org.apache.iotdb.db.qp.physical.sys.ShowTimeSeriesPlan;
import org.apache.iotdb.db.query.context.QueryContext;
import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.read.common.Field;
import org.apache.iotdb.tsfile.read.common.Path;
import org.apache.iotdb.tsfile.read.common.RowRecord;
import org.apache.iotdb.tsfile.read.query.dataset.QueryDataSet;
+import org.apache.iotdb.tsfile.utils.Binary;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class ShowTimeseriesDataSet extends QueryDataSet {
private static final Logger logger = LoggerFactory.getLogger(ShowTimeseriesDataSet.class);
-
private final ShowTimeSeriesPlan plan;
private List<RowRecord> result = new ArrayList<>();
private int index = 0;
private QueryContext context;
+ private List<ShowTimeSeriesResult> timeseriesList;
+ private boolean hasSetRecord;
public boolean hasLimit = true;
- public ShowTimeseriesDataSet(List<PartialPath> paths, List<TSDataType> dataTypes,
- ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context) {
- super(new ArrayList<>(paths), dataTypes);
+ private static Path[] resourcePaths = {new PartialPath(COLUMN_TIMESERIES, false),
+ new PartialPath(COLUMN_TIMESERIES_ALIAS, false), new PartialPath(COLUMN_STORAGE_GROUP, false),
+ new PartialPath(COLUMN_TIMESERIES_DATATYPE, false), new PartialPath(COLUMN_TIMESERIES_ENCODING, false),
+ new PartialPath(COLUMN_TIMESERIES_COMPRESSION, false), new PartialPath(COLUMN_TAG, false),
+ new PartialPath(COLUMN_ATTRIBUTE, false)};
+ private static TSDataType[] resourceTypes = {TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT,
+ TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT};
+
+ public ShowTimeseriesDataSet(ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context,
+ List<ShowTimeSeriesResult> timeseriesList) {
+ super(Arrays.asList(resourcePaths), Arrays.asList(resourceTypes));
this.plan = showTimeSeriesPlan;
this.context = context;
+ this.timeseriesList = timeseriesList;
+ this.hasSetRecord = false;
+ }
+
+ public QueryDataSet getQueryDataSet() {
+ hasLimit = plan.hasLimit();
+ for (ShowTimeSeriesResult result : timeseriesList) {
+ RowRecord record = new RowRecord(0);
Review comment:
duplicate code, `transferShowTimeSeriesResultToRecordList` and `getQueryDataSet `
##########
File path: server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeseriesDataSet.java
##########
@@ -19,44 +19,123 @@
package org.apache.iotdb.db.query.dataset;
-import static org.apache.iotdb.db.utils.QueryUtils.transferShowTimeSeriesResultToRecordList;
-
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_ATTRIBUTE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_STORAGE_GROUP;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TAG;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ALIAS;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_DATATYPE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ENCODING;
import java.io.IOException;
import java.util.ArrayList;
+import java.util.Arrays;
import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
import org.apache.iotdb.db.exception.metadata.MetadataException;
import org.apache.iotdb.db.metadata.MManager;
import org.apache.iotdb.db.metadata.PartialPath;
import org.apache.iotdb.db.qp.physical.sys.ShowTimeSeriesPlan;
import org.apache.iotdb.db.query.context.QueryContext;
import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.read.common.Field;
import org.apache.iotdb.tsfile.read.common.Path;
import org.apache.iotdb.tsfile.read.common.RowRecord;
import org.apache.iotdb.tsfile.read.query.dataset.QueryDataSet;
+import org.apache.iotdb.tsfile.utils.Binary;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class ShowTimeseriesDataSet extends QueryDataSet {
private static final Logger logger = LoggerFactory.getLogger(ShowTimeseriesDataSet.class);
-
private final ShowTimeSeriesPlan plan;
private List<RowRecord> result = new ArrayList<>();
private int index = 0;
private QueryContext context;
+ private List<ShowTimeSeriesResult> timeseriesList;
+ private boolean hasSetRecord;
public boolean hasLimit = true;
- public ShowTimeseriesDataSet(List<PartialPath> paths, List<TSDataType> dataTypes,
- ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context) {
- super(new ArrayList<>(paths), dataTypes);
+ private static Path[] resourcePaths = {new PartialPath(COLUMN_TIMESERIES, false),
+ new PartialPath(COLUMN_TIMESERIES_ALIAS, false), new PartialPath(COLUMN_STORAGE_GROUP, false),
+ new PartialPath(COLUMN_TIMESERIES_DATATYPE, false), new PartialPath(COLUMN_TIMESERIES_ENCODING, false),
+ new PartialPath(COLUMN_TIMESERIES_COMPRESSION, false), new PartialPath(COLUMN_TAG, false),
+ new PartialPath(COLUMN_ATTRIBUTE, false)};
+ private static TSDataType[] resourceTypes = {TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT,
+ TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT};
+
+ public ShowTimeseriesDataSet(ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context,
+ List<ShowTimeSeriesResult> timeseriesList) {
+ super(Arrays.asList(resourcePaths), Arrays.asList(resourceTypes));
this.plan = showTimeSeriesPlan;
this.context = context;
+ this.timeseriesList = timeseriesList;
+ this.hasSetRecord = false;
+ }
+
+ public QueryDataSet getQueryDataSet() {
+ hasLimit = plan.hasLimit();
+ for (ShowTimeSeriesResult result : timeseriesList) {
+ RowRecord record = new RowRecord(0);
+ updateRecord(record, result.getName());
+ updateRecord(record, result.getAlias());
+ updateRecord(record, result.getSgName());
+ updateRecord(record, result.getDataType().toString());
+ updateRecord(record, result.getEncoding().toString());
+ updateRecord(record, result.getCompressor().toString());
+ updateRecord(record, result.getTag());
+ updateRecord(record, result.getAttribute());
+ putRecord(record);
+ }
+ return this;
+ }
+
+ public List<RowRecord> transferShowTimeSeriesResultToRecordList(
+ List<ShowTimeSeriesResult> timeseriesList) {
+ List<RowRecord> records = new ArrayList<>();
+ for (ShowTimeSeriesResult result : timeseriesList) {
+ RowRecord record = new RowRecord(0);
+ updateRecord(record, result.getName());
+ updateRecord(record, result.getAlias());
+ updateRecord(record, result.getSgName());
+ updateRecord(record, result.getDataType().toString());
+ updateRecord(record, result.getEncoding().toString());
+ updateRecord(record, result.getCompressor().toString());
+ updateRecord(record, result.getTag());
+ updateRecord(record, result.getAttribute());
+ records.add(record);
+ }
+ return records;
+ }
+
+ private void updateRecord(RowRecord record, Map<String, String> map) {
+ String text = map.entrySet().stream()
+ .map(e -> "\"" + e.getKey() + "\"" + ":" + "\"" + e.getValue() + "\"")
+ .collect(Collectors.joining(","));
+
+ updateRecord(record, text.length() == 0 ? null : "{" + text + "}");
+ }
+
+ private void updateRecord(RowRecord record, String s) {
+ if (s == null) {
+ record.addField(null);
+ return;
+ }
+ Field field = new Field(TSDataType.TEXT);
+ field.setBinaryV(new Binary(s));
+ record.addField(field);
}
@Override
protected boolean hasNextWithoutConstraint() throws IOException {
+ if (!hasSetRecord) {
Review comment:
i think that this code can be optimizing:
1. Take the data directly from the `MManager` and initialize it in the `constructor`
2. if the total amount of data is less than the `fetchSize`, which means there is no more data, reducing the access to the `MManager`.
> when hasLimit = false, limit was means fetchSize
3. if `index == result.size()== fetchSize`, get data from `MManager` again.
##########
File path: server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeseriesDataSet.java
##########
@@ -19,44 +19,123 @@
package org.apache.iotdb.db.query.dataset;
-import static org.apache.iotdb.db.utils.QueryUtils.transferShowTimeSeriesResultToRecordList;
-
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_ATTRIBUTE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_STORAGE_GROUP;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TAG;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ALIAS;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_DATATYPE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ENCODING;
import java.io.IOException;
import java.util.ArrayList;
+import java.util.Arrays;
import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
import org.apache.iotdb.db.exception.metadata.MetadataException;
import org.apache.iotdb.db.metadata.MManager;
import org.apache.iotdb.db.metadata.PartialPath;
import org.apache.iotdb.db.qp.physical.sys.ShowTimeSeriesPlan;
import org.apache.iotdb.db.query.context.QueryContext;
import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.read.common.Field;
import org.apache.iotdb.tsfile.read.common.Path;
import org.apache.iotdb.tsfile.read.common.RowRecord;
import org.apache.iotdb.tsfile.read.query.dataset.QueryDataSet;
+import org.apache.iotdb.tsfile.utils.Binary;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class ShowTimeseriesDataSet extends QueryDataSet {
private static final Logger logger = LoggerFactory.getLogger(ShowTimeseriesDataSet.class);
-
private final ShowTimeSeriesPlan plan;
private List<RowRecord> result = new ArrayList<>();
private int index = 0;
private QueryContext context;
+ private List<ShowTimeSeriesResult> timeseriesList;
+ private boolean hasSetRecord;
public boolean hasLimit = true;
- public ShowTimeseriesDataSet(List<PartialPath> paths, List<TSDataType> dataTypes,
- ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context) {
- super(new ArrayList<>(paths), dataTypes);
+ private static Path[] resourcePaths = {new PartialPath(COLUMN_TIMESERIES, false),
+ new PartialPath(COLUMN_TIMESERIES_ALIAS, false), new PartialPath(COLUMN_STORAGE_GROUP, false),
+ new PartialPath(COLUMN_TIMESERIES_DATATYPE, false), new PartialPath(COLUMN_TIMESERIES_ENCODING, false),
+ new PartialPath(COLUMN_TIMESERIES_COMPRESSION, false), new PartialPath(COLUMN_TAG, false),
+ new PartialPath(COLUMN_ATTRIBUTE, false)};
+ private static TSDataType[] resourceTypes = {TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT,
+ TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT};
+
+ public ShowTimeseriesDataSet(ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context,
+ List<ShowTimeSeriesResult> timeseriesList) {
+ super(Arrays.asList(resourcePaths), Arrays.asList(resourceTypes));
this.plan = showTimeSeriesPlan;
this.context = context;
+ this.timeseriesList = timeseriesList;
+ this.hasSetRecord = false;
+ }
+
+ public QueryDataSet getQueryDataSet() {
+ hasLimit = plan.hasLimit();
+ for (ShowTimeSeriesResult result : timeseriesList) {
+ RowRecord record = new RowRecord(0);
+ updateRecord(record, result.getName());
+ updateRecord(record, result.getAlias());
+ updateRecord(record, result.getSgName());
+ updateRecord(record, result.getDataType().toString());
+ updateRecord(record, result.getEncoding().toString());
+ updateRecord(record, result.getCompressor().toString());
+ updateRecord(record, result.getTag());
+ updateRecord(record, result.getAttribute());
+ putRecord(record);
+ }
+ return this;
+ }
+
+ public List<RowRecord> transferShowTimeSeriesResultToRecordList(
+ List<ShowTimeSeriesResult> timeseriesList) {
+ List<RowRecord> records = new ArrayList<>();
+ for (ShowTimeSeriesResult result : timeseriesList) {
+ RowRecord record = new RowRecord(0);
+ updateRecord(record, result.getName());
+ updateRecord(record, result.getAlias());
+ updateRecord(record, result.getSgName());
+ updateRecord(record, result.getDataType().toString());
+ updateRecord(record, result.getEncoding().toString());
+ updateRecord(record, result.getCompressor().toString());
+ updateRecord(record, result.getTag());
+ updateRecord(record, result.getAttribute());
+ records.add(record);
+ }
+ return records;
+ }
+
+ private void updateRecord(RowRecord record, Map<String, String> map) {
+ String text = map.entrySet().stream()
+ .map(e -> "\"" + e.getKey() + "\"" + ":" + "\"" + e.getValue() + "\"")
+ .collect(Collectors.joining(","));
+
+ updateRecord(record, text.length() == 0 ? null : "{" + text + "}");
+ }
+
+ private void updateRecord(RowRecord record, String s) {
Review comment:
`putRecord` method should also be modified to `private`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-iotdb] liutaohua commented on a change in pull request #1731: [IOTDB-870] tags and attributes output
Posted by GitBox <gi...@apache.org>.
liutaohua commented on a change in pull request #1731:
URL: https://github.com/apache/incubator-iotdb/pull/1731#discussion_r492644306
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/MManager.java
##########
@@ -937,12 +937,11 @@ public int getNodesCountInGivenLevel(PartialPath prefixPath, int level) throws M
try {
Pair<Map<String, String>, Map<String, String>> pair =
Review comment:
Change to a more meaningful name, like `tagAndAttributePair`
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/MManager.java
##########
@@ -1003,18 +1002,13 @@ private boolean match(PartialPath fullPath, String[] prefixNodes) {
for (Pair<PartialPath, String[]> ansString : ans) {
long tagFileOffset = Long.parseLong(ansString.right[5]);
try {
- if (tagFileOffset < 0) {
- // no tags/attributes
- res.add(new ShowTimeSeriesResult(ansString.left.getFullPath(), ansString.right[0], ansString.right[1], ansString.right[2],
- ansString.right[3], ansString.right[4], Collections.emptyMap()));
- } else {
- // has tags/attributes
- Pair<Map<String, String>, Map<String, String>> pair =
- tagLogFile.read(config.getTagAttributeTotalSize(), tagFileOffset);
- pair.left.putAll(pair.right);
- res.add(new ShowTimeSeriesResult(ansString.left.getFullPath(), ansString.right[0], ansString.right[1], ansString.right[2],
- ansString.right[3], ansString.right[4], pair.left));
+ Pair<Map<String, String>, Map<String, String>> pair = new Pair<>(Collections.emptyMap(),Collections.emptyMap());
Review comment:
same
##########
File path: server/src/main/java/org/apache/iotdb/db/qp/executor/PlanExecutor.java
##########
@@ -566,7 +567,7 @@ private QueryDataSet processShowStorageGroup(ShowStorageGroupPlan showStorageGro
private QueryDataSet processShowTimeseries(ShowTimeSeriesPlan showTimeSeriesPlan,
QueryContext context) throws MetadataException {
List<ShowTimeSeriesResult> timeseriesList = showTimeseries(showTimeSeriesPlan, context);
- return QueryUtils.getQueryDataSet(timeseriesList, showTimeSeriesPlan, context);
+ return new ShowTimeseriesDataSet(showTimeSeriesPlan, context, timeseriesList);
}
protected List<ShowTimeSeriesResult> showTimeseries(ShowTimeSeriesPlan plan, QueryContext context)
Review comment:
I think this method can be removed, or move it to `ShowTimeseriesDataSet`
##########
File path: server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeseriesDataSet.java
##########
@@ -19,44 +19,123 @@
package org.apache.iotdb.db.query.dataset;
-import static org.apache.iotdb.db.utils.QueryUtils.transferShowTimeSeriesResultToRecordList;
-
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_ATTRIBUTE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_STORAGE_GROUP;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TAG;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ALIAS;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_DATATYPE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ENCODING;
import java.io.IOException;
import java.util.ArrayList;
+import java.util.Arrays;
import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
import org.apache.iotdb.db.exception.metadata.MetadataException;
import org.apache.iotdb.db.metadata.MManager;
import org.apache.iotdb.db.metadata.PartialPath;
import org.apache.iotdb.db.qp.physical.sys.ShowTimeSeriesPlan;
import org.apache.iotdb.db.query.context.QueryContext;
import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.read.common.Field;
import org.apache.iotdb.tsfile.read.common.Path;
import org.apache.iotdb.tsfile.read.common.RowRecord;
import org.apache.iotdb.tsfile.read.query.dataset.QueryDataSet;
+import org.apache.iotdb.tsfile.utils.Binary;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class ShowTimeseriesDataSet extends QueryDataSet {
private static final Logger logger = LoggerFactory.getLogger(ShowTimeseriesDataSet.class);
-
private final ShowTimeSeriesPlan plan;
private List<RowRecord> result = new ArrayList<>();
private int index = 0;
private QueryContext context;
+ private List<ShowTimeSeriesResult> timeseriesList;
+ private boolean hasSetRecord;
public boolean hasLimit = true;
- public ShowTimeseriesDataSet(List<PartialPath> paths, List<TSDataType> dataTypes,
- ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context) {
- super(new ArrayList<>(paths), dataTypes);
+ private static Path[] resourcePaths = {new PartialPath(COLUMN_TIMESERIES, false),
+ new PartialPath(COLUMN_TIMESERIES_ALIAS, false), new PartialPath(COLUMN_STORAGE_GROUP, false),
+ new PartialPath(COLUMN_TIMESERIES_DATATYPE, false), new PartialPath(COLUMN_TIMESERIES_ENCODING, false),
+ new PartialPath(COLUMN_TIMESERIES_COMPRESSION, false), new PartialPath(COLUMN_TAG, false),
+ new PartialPath(COLUMN_ATTRIBUTE, false)};
+ private static TSDataType[] resourceTypes = {TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT,
+ TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT};
+
+ public ShowTimeseriesDataSet(ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context,
+ List<ShowTimeSeriesResult> timeseriesList) {
+ super(Arrays.asList(resourcePaths), Arrays.asList(resourceTypes));
this.plan = showTimeSeriesPlan;
this.context = context;
+ this.timeseriesList = timeseriesList;
+ this.hasSetRecord = false;
+ }
+
+ public QueryDataSet getQueryDataSet() {
+ hasLimit = plan.hasLimit();
Review comment:
i think that it can be initialized in the `constructor`
##########
File path: server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeseriesDataSet.java
##########
@@ -19,44 +19,123 @@
package org.apache.iotdb.db.query.dataset;
-import static org.apache.iotdb.db.utils.QueryUtils.transferShowTimeSeriesResultToRecordList;
-
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_ATTRIBUTE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_STORAGE_GROUP;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TAG;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ALIAS;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_DATATYPE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ENCODING;
import java.io.IOException;
import java.util.ArrayList;
+import java.util.Arrays;
import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
import org.apache.iotdb.db.exception.metadata.MetadataException;
import org.apache.iotdb.db.metadata.MManager;
import org.apache.iotdb.db.metadata.PartialPath;
import org.apache.iotdb.db.qp.physical.sys.ShowTimeSeriesPlan;
import org.apache.iotdb.db.query.context.QueryContext;
import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.read.common.Field;
import org.apache.iotdb.tsfile.read.common.Path;
import org.apache.iotdb.tsfile.read.common.RowRecord;
import org.apache.iotdb.tsfile.read.query.dataset.QueryDataSet;
+import org.apache.iotdb.tsfile.utils.Binary;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class ShowTimeseriesDataSet extends QueryDataSet {
private static final Logger logger = LoggerFactory.getLogger(ShowTimeseriesDataSet.class);
-
private final ShowTimeSeriesPlan plan;
private List<RowRecord> result = new ArrayList<>();
private int index = 0;
private QueryContext context;
+ private List<ShowTimeSeriesResult> timeseriesList;
+ private boolean hasSetRecord;
public boolean hasLimit = true;
- public ShowTimeseriesDataSet(List<PartialPath> paths, List<TSDataType> dataTypes,
- ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context) {
- super(new ArrayList<>(paths), dataTypes);
+ private static Path[] resourcePaths = {new PartialPath(COLUMN_TIMESERIES, false),
+ new PartialPath(COLUMN_TIMESERIES_ALIAS, false), new PartialPath(COLUMN_STORAGE_GROUP, false),
+ new PartialPath(COLUMN_TIMESERIES_DATATYPE, false), new PartialPath(COLUMN_TIMESERIES_ENCODING, false),
+ new PartialPath(COLUMN_TIMESERIES_COMPRESSION, false), new PartialPath(COLUMN_TAG, false),
+ new PartialPath(COLUMN_ATTRIBUTE, false)};
+ private static TSDataType[] resourceTypes = {TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT,
+ TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT};
+
+ public ShowTimeseriesDataSet(ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context,
+ List<ShowTimeSeriesResult> timeseriesList) {
+ super(Arrays.asList(resourcePaths), Arrays.asList(resourceTypes));
this.plan = showTimeSeriesPlan;
this.context = context;
+ this.timeseriesList = timeseriesList;
+ this.hasSetRecord = false;
+ }
+
+ public QueryDataSet getQueryDataSet() {
+ hasLimit = plan.hasLimit();
+ for (ShowTimeSeriesResult result : timeseriesList) {
+ RowRecord record = new RowRecord(0);
Review comment:
duplicate code, `transferShowTimeSeriesResultToRecordList` and `getQueryDataSet `
##########
File path: server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeseriesDataSet.java
##########
@@ -19,44 +19,123 @@
package org.apache.iotdb.db.query.dataset;
-import static org.apache.iotdb.db.utils.QueryUtils.transferShowTimeSeriesResultToRecordList;
-
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_ATTRIBUTE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_STORAGE_GROUP;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TAG;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ALIAS;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_DATATYPE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ENCODING;
import java.io.IOException;
import java.util.ArrayList;
+import java.util.Arrays;
import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
import org.apache.iotdb.db.exception.metadata.MetadataException;
import org.apache.iotdb.db.metadata.MManager;
import org.apache.iotdb.db.metadata.PartialPath;
import org.apache.iotdb.db.qp.physical.sys.ShowTimeSeriesPlan;
import org.apache.iotdb.db.query.context.QueryContext;
import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.read.common.Field;
import org.apache.iotdb.tsfile.read.common.Path;
import org.apache.iotdb.tsfile.read.common.RowRecord;
import org.apache.iotdb.tsfile.read.query.dataset.QueryDataSet;
+import org.apache.iotdb.tsfile.utils.Binary;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class ShowTimeseriesDataSet extends QueryDataSet {
private static final Logger logger = LoggerFactory.getLogger(ShowTimeseriesDataSet.class);
-
private final ShowTimeSeriesPlan plan;
private List<RowRecord> result = new ArrayList<>();
private int index = 0;
private QueryContext context;
+ private List<ShowTimeSeriesResult> timeseriesList;
+ private boolean hasSetRecord;
public boolean hasLimit = true;
- public ShowTimeseriesDataSet(List<PartialPath> paths, List<TSDataType> dataTypes,
- ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context) {
- super(new ArrayList<>(paths), dataTypes);
+ private static Path[] resourcePaths = {new PartialPath(COLUMN_TIMESERIES, false),
+ new PartialPath(COLUMN_TIMESERIES_ALIAS, false), new PartialPath(COLUMN_STORAGE_GROUP, false),
+ new PartialPath(COLUMN_TIMESERIES_DATATYPE, false), new PartialPath(COLUMN_TIMESERIES_ENCODING, false),
+ new PartialPath(COLUMN_TIMESERIES_COMPRESSION, false), new PartialPath(COLUMN_TAG, false),
+ new PartialPath(COLUMN_ATTRIBUTE, false)};
+ private static TSDataType[] resourceTypes = {TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT,
+ TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT};
+
+ public ShowTimeseriesDataSet(ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context,
+ List<ShowTimeSeriesResult> timeseriesList) {
+ super(Arrays.asList(resourcePaths), Arrays.asList(resourceTypes));
this.plan = showTimeSeriesPlan;
this.context = context;
+ this.timeseriesList = timeseriesList;
+ this.hasSetRecord = false;
+ }
+
+ public QueryDataSet getQueryDataSet() {
+ hasLimit = plan.hasLimit();
+ for (ShowTimeSeriesResult result : timeseriesList) {
+ RowRecord record = new RowRecord(0);
+ updateRecord(record, result.getName());
+ updateRecord(record, result.getAlias());
+ updateRecord(record, result.getSgName());
+ updateRecord(record, result.getDataType().toString());
+ updateRecord(record, result.getEncoding().toString());
+ updateRecord(record, result.getCompressor().toString());
+ updateRecord(record, result.getTag());
+ updateRecord(record, result.getAttribute());
+ putRecord(record);
+ }
+ return this;
+ }
+
+ public List<RowRecord> transferShowTimeSeriesResultToRecordList(
+ List<ShowTimeSeriesResult> timeseriesList) {
+ List<RowRecord> records = new ArrayList<>();
+ for (ShowTimeSeriesResult result : timeseriesList) {
+ RowRecord record = new RowRecord(0);
+ updateRecord(record, result.getName());
+ updateRecord(record, result.getAlias());
+ updateRecord(record, result.getSgName());
+ updateRecord(record, result.getDataType().toString());
+ updateRecord(record, result.getEncoding().toString());
+ updateRecord(record, result.getCompressor().toString());
+ updateRecord(record, result.getTag());
+ updateRecord(record, result.getAttribute());
+ records.add(record);
+ }
+ return records;
+ }
+
+ private void updateRecord(RowRecord record, Map<String, String> map) {
+ String text = map.entrySet().stream()
+ .map(e -> "\"" + e.getKey() + "\"" + ":" + "\"" + e.getValue() + "\"")
+ .collect(Collectors.joining(","));
+
+ updateRecord(record, text.length() == 0 ? null : "{" + text + "}");
+ }
+
+ private void updateRecord(RowRecord record, String s) {
+ if (s == null) {
+ record.addField(null);
+ return;
+ }
+ Field field = new Field(TSDataType.TEXT);
+ field.setBinaryV(new Binary(s));
+ record.addField(field);
}
@Override
protected boolean hasNextWithoutConstraint() throws IOException {
+ if (!hasSetRecord) {
Review comment:
i think that this code can be optimizing:
1. Take the data directly from the `MManager` and initialize it in the `constructor`
2. if the total amount of data is less than the `fetchSize`, which means there is no more data, reducing the access to the `MManager`.
> when hasLimit = false, limit was means fetchSize
3. if `index == result.size()== fetchSize`, get data from `MManager` again.
##########
File path: server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeseriesDataSet.java
##########
@@ -19,44 +19,123 @@
package org.apache.iotdb.db.query.dataset;
-import static org.apache.iotdb.db.utils.QueryUtils.transferShowTimeSeriesResultToRecordList;
-
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_ATTRIBUTE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_STORAGE_GROUP;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TAG;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ALIAS;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_COMPRESSION;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_DATATYPE;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES_ENCODING;
import java.io.IOException;
import java.util.ArrayList;
+import java.util.Arrays;
import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
import org.apache.iotdb.db.exception.metadata.MetadataException;
import org.apache.iotdb.db.metadata.MManager;
import org.apache.iotdb.db.metadata.PartialPath;
import org.apache.iotdb.db.qp.physical.sys.ShowTimeSeriesPlan;
import org.apache.iotdb.db.query.context.QueryContext;
import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.read.common.Field;
import org.apache.iotdb.tsfile.read.common.Path;
import org.apache.iotdb.tsfile.read.common.RowRecord;
import org.apache.iotdb.tsfile.read.query.dataset.QueryDataSet;
+import org.apache.iotdb.tsfile.utils.Binary;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class ShowTimeseriesDataSet extends QueryDataSet {
private static final Logger logger = LoggerFactory.getLogger(ShowTimeseriesDataSet.class);
-
private final ShowTimeSeriesPlan plan;
private List<RowRecord> result = new ArrayList<>();
private int index = 0;
private QueryContext context;
+ private List<ShowTimeSeriesResult> timeseriesList;
+ private boolean hasSetRecord;
public boolean hasLimit = true;
- public ShowTimeseriesDataSet(List<PartialPath> paths, List<TSDataType> dataTypes,
- ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context) {
- super(new ArrayList<>(paths), dataTypes);
+ private static Path[] resourcePaths = {new PartialPath(COLUMN_TIMESERIES, false),
+ new PartialPath(COLUMN_TIMESERIES_ALIAS, false), new PartialPath(COLUMN_STORAGE_GROUP, false),
+ new PartialPath(COLUMN_TIMESERIES_DATATYPE, false), new PartialPath(COLUMN_TIMESERIES_ENCODING, false),
+ new PartialPath(COLUMN_TIMESERIES_COMPRESSION, false), new PartialPath(COLUMN_TAG, false),
+ new PartialPath(COLUMN_ATTRIBUTE, false)};
+ private static TSDataType[] resourceTypes = {TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT,
+ TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT, TSDataType.TEXT};
+
+ public ShowTimeseriesDataSet(ShowTimeSeriesPlan showTimeSeriesPlan, QueryContext context,
+ List<ShowTimeSeriesResult> timeseriesList) {
+ super(Arrays.asList(resourcePaths), Arrays.asList(resourceTypes));
this.plan = showTimeSeriesPlan;
this.context = context;
+ this.timeseriesList = timeseriesList;
+ this.hasSetRecord = false;
+ }
+
+ public QueryDataSet getQueryDataSet() {
+ hasLimit = plan.hasLimit();
+ for (ShowTimeSeriesResult result : timeseriesList) {
+ RowRecord record = new RowRecord(0);
+ updateRecord(record, result.getName());
+ updateRecord(record, result.getAlias());
+ updateRecord(record, result.getSgName());
+ updateRecord(record, result.getDataType().toString());
+ updateRecord(record, result.getEncoding().toString());
+ updateRecord(record, result.getCompressor().toString());
+ updateRecord(record, result.getTag());
+ updateRecord(record, result.getAttribute());
+ putRecord(record);
+ }
+ return this;
+ }
+
+ public List<RowRecord> transferShowTimeSeriesResultToRecordList(
+ List<ShowTimeSeriesResult> timeseriesList) {
+ List<RowRecord> records = new ArrayList<>();
+ for (ShowTimeSeriesResult result : timeseriesList) {
+ RowRecord record = new RowRecord(0);
+ updateRecord(record, result.getName());
+ updateRecord(record, result.getAlias());
+ updateRecord(record, result.getSgName());
+ updateRecord(record, result.getDataType().toString());
+ updateRecord(record, result.getEncoding().toString());
+ updateRecord(record, result.getCompressor().toString());
+ updateRecord(record, result.getTag());
+ updateRecord(record, result.getAttribute());
+ records.add(record);
+ }
+ return records;
+ }
+
+ private void updateRecord(RowRecord record, Map<String, String> map) {
+ String text = map.entrySet().stream()
+ .map(e -> "\"" + e.getKey() + "\"" + ":" + "\"" + e.getValue() + "\"")
+ .collect(Collectors.joining(","));
+
+ updateRecord(record, text.length() == 0 ? null : "{" + text + "}");
+ }
+
+ private void updateRecord(RowRecord record, String s) {
Review comment:
`putRecord` method should also be modified to `private`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org